Underwater/SCUBA footage processing. Examples include ProRes/ACR vs DNxHD/Resolv

[zen_nudist]

Anyone have experience processing DNGs recorded underwater? It's a challenge perfecting the white balance, color, exposure while minimizing noise. Exposure changes constantly; color falls off exponentially at depth; visibility in the water inhibits clarity of subjects; etc. So, I'm experimenting and looking for advice.

I'm experimenting with 2 very different processing workflows, both of which are explained below. The end goal at the end of each process is to create high quality .movs for archival purposes and to be used in a variety of videos, many of which are encoded to MP4 h.264 for YouTube (yes, I'm aware of the YouTube compression, but I'm seeing what happens there). I have a MP4 for each process linked below, using the same batches of DNGs. (A few from each clip are linked here: https://drive.google.com/open?id=1iAyHVo27pQDHm6XAdG2dwoG8V2DPD3lP

I'm looking for comparison/contrast comments and comments as to how I might be messing up in Resolve and/ or ACR. Frankly, I think both look pretty bad even when played at 720p on YouTube.

I use a 5DIII and use ML build Nightly.2018Feb04.FD3113. In camera, I record 14bit raw at 29.97fps at 1600px900p. That's the highest resolution I can get at 30fps, and I'm fine with the 1.20 difference in resolution right now. I use a 35mm f/2 prime, shooting around f/8, ISO under 1000, shutter speed of 1/60th. 

I always use MLVApp to create the CinemaDNG Uncompressed DNGs.

Here are the two ways I'm experimenting with processing the DNGs.

BRIDGE, ACR, AFTER EFFECTS, MEDIA ENCODER PROCESS

1) I select a DNG in Bridge, edit it in ACR. Copy those image qualities and paste to the rest of the DNGs in the clip folder.
2) I assemble the individual DNGs into a single video clip in After Effects and export to Media Encoder.
3) I export from Media Encoder to 1600x900p, 29.97 fps ProRes 422HQ.

Now, I have the archival-ready quality clips I can use in other projects.

I pull those ProRes .movs into premiere, interpret them to 24fps (to smooth out camera shake), edit them together, and export to MP4 via h.264, vbr 30, max 50.

Upload to YouTube.

This is the "Version 1" video.

RESOLVE PROCESS

Project Settings Camera RAW settings
RAW profile: CinemaDNG
Decode Quality: Full Res.
Decode Using: Project
Apply Pre Tone Curve: Checked

Create timeline from dng folders

Camera Raw Panel
Decode Quality:Full Res.
Decode Using: Clip
White Balance: Custom
Color Space: Blackmagic Design
Gamma: Blackmagic Design

Node 1
1) I start by using the either the "Pick Black Point" or "Pick White Point" tool in color wheels. This gives me the quickest approximation to contrast, clarity and white balance that I need.
2) I adjust exposure and midtone detail to make image as bright as possible while trying my best to minimize noise. This is a balancing act, and there will always be noise I have to deal with, at least in deep water. Sometimes exposure is over 2, sometimes it's in the negative.
3) A final color temp adjustment may be made

No other nodes ... no LUT desired at this point, if at all.

Export to DNxHD at 1080p 4444, 29.97fps

Now, I have the archival-ready quality clips I can use in other projects.

I pull those DNxHD .movs into premiere, interpret them to 24fps (to smooth out camera shake), edit them together, and export to MP4 via h.264, vbr 30, max 50.

Upload to YouTube.

This is the "Version 2" video.

[yourboylloyd]

That's an extremely long workflow for the first one. MLVAPP can cut your time "BRIDGE, ACR, AFTER EFFECTS, MEDIA ENCODER PROCESS" by almost quadruple. MLVAPP is pretty powerful, especially if you have a solid GPU. Of course Resolve is faster too.

But I understand wanting to work with camera raw.

1) I select a DNG in Bridge, edit it in ACR. Copy those image qualities and paste to the rest of the DNGs in the clip folder.
2) I assemble the individual DNGs into a single video clip in After Effects and export to Media Encoder.

I pull those ProRes .movs into premiere, interpret them to 24fps (to smooth out camera shake), edit them together, and export to MP4 via h.264, vbr 30, max 50.

Bro... what? You should be using the "Smart Import 2" script for After Effects. That would remove the Bridge-ACR step and take you straight into Camera Raw in AE. Here's a link to it: https://www.magiclantern.fm/forum/index.php?topic=13354.0 . Will probably save you hours of time. You can also do the warp stabilizer effect in AE to skip your having to use premiere step. I think you exporting the footage so many times may have something to do with the quality loss (I could be SUPER wrong about that though).

Also, next time consider shooting in 12-bit. There is practically no difference in dynamic range in 12-bit, but you can shoot at higher resolutions. I think having more resolution would help you in cases like these.

Denoiser III by redgiant is an absolute god-send. You can crank the shadows up and slap denoiser on there and you'll get crystal clear footage. I also think you should add sharpening (unsharp mask) to some of the footage. Sometimes the fish gets slightly blurry and this would improve on that end.

[c_joerg]

Anyone have experience processing DNGs recorded underwater? It's a challenge perfecting the white balance, color, exposure while

I’m doing only still images…
But I don’t care about color and white balance underwater anymore. As long I use RAW, I use the Auto Color Corrections" in CS6.
https://www.dslr-forum.de/showpost.php?p=13050490&postcount=37

[Dmytro_ua]

Why don't you use a 14bit lossless? You can go much higher resolution with it.
The build is in an Experimental branch but it works without any issues.

[masc]

MLVAPP is pretty powerful, especially if you have a solid GPU. Of course Resolve is faster too.

MLVApp can do a lot. But it won't use any GPU. It is 100% CPU based.

[Dmytro_ua]

I use a 5DIII

What underwater case did you use?

[zen_nudist]

What underwater case did you use?

Ikelite housing and strobe arm. Kraken Hydra 3500 underwater light (I really need another one, but god it's a lot of money).

[zen_nudist]

Why don't you use a 14bit lossless? You can go much higher resolution with it.
The build is in an Experimental branch but it works without any issues.

Hi, I'm not sure I follow. The build I'm using is a nightly non-experimental build. I record everything at 14-bit RAW. My card supports 1600x900 at 29.97fps continuous. Is there something I'm missing, or are you mistaken?

[zen_nudist]

That's an extremely long workflow for the first one. MLVAPP can cut your time "BRIDGE, ACR, AFTER EFFECTS, MEDIA ENCODER PROCESS" by almost quadruple. MLVAPP is pretty powerful, especially if you have a solid GPU. Of course Resolve is faster too.

But I understand wanting to work with camera raw.
Bro... what? You should be using the "Smart Import 2" script for After Effects. That would remove the Bridge-ACR step and take you straight into Camera Raw in AE. Here's a link to it: https://www.magiclantern.fm/forum/index.php?topic=13354.0 . Will probably save you hours of time. You can also do the warp stabilizer effect in AE to skip your having to use premiere step. I think you exporting the footage so many times may have something to do with the quality loss (I could be SUPER wrong about that though).

Also, next time consider shooting in 12-bit. There is practically no difference in dynamic range in 12-bit, but you can shoot at higher resolutions. I think having more resolution would help you in cases like these.

Denoiser III by redgiant is an absolute god-send. You can crank the shadows up and slap denoiser on there and you'll get crystal clear footage. I also think you should add sharpening (unsharp mask) to some of the footage. Sometimes the fish gets slightly blurry and this would improve on that end.

Thank you for your informative response. It's much appreciated. I'm going to:

1) Get a faster card so I can shoot 14 bit at 1920x1080 at 29.97fps continuous (I got ripped off with my Lexar 1066x apparently, in terms of write speed). I belief any extra resolution will help. In meantime, I'll experiment with the 12-bit suggestion you have.

2) Continue with the Resolve process: In terms of image quality results on YouTube, I don't see much difference between the Bridge/ACR/AE process and the much streamlined and faster Resolve process. I'll follow your advice about sharpening and de-noising in Premiere, but I'll use Neat Video, as I have that already

3) I'll experiment with skipping encoding to DNxHD and going straight from .dng to for-YouTube .mp4. It'll be interesting to see if that improves quality any.

4) I'll experiment with the ML 4K experimental build.

5) I'll report back my results on all this in case anyone else is interested in filming underwater with ML. I'm honestly surprised at the lack of info out there on this subject.

[Kharak]

You have to look into the experimental builds, 14 bit Lossless means your footage is compressed to the card without any quality loss, so instead of recording 90-100MB/s your are down to mostly 50-60MB/s depending on your scene, that also gives you 40-50% longer record time on your cards. You being underwater, I dont think you will be changing cards. Its a no-brainer.

[Dmytro_ua]

Get a faster card so I can shoot 14 bit at 1920x1080 at 29.97fps continuous (I got ripped off with my Lexar 1066x apparently, in terms of write speed). I belief any extra resolution will help. In meantime, I'll experiment with the 12-bit suggestion you have.

You don't need a faster card as there's a limit in a camera CF bus speed.
You're missing a lot not using a compressed lossless 14-bit. You can record 1920x1080 at 29.97fps continuous with a fullcolored realtime preview + proxy video to SD card (to have a preview in a camera). It's a regular Canon's video + ML RAW.
If you don't need proxies, you can get ML with higher resolution or higher FPS (up to 60 fps - 1920x800 but preview is a little broken).

All this will work with your current Lexar 1066x card

[Walter Schulz]

You don't need a faster card as there's a limit in a camera CF bus speed.
[...]
All this will work with your current Lexar 1066x card

He got a card maxing out at around 80 MByte/s so he has a point.

[Dmytro_ua]

He got a card maxing out at around 80 MByte/s so he has a point.

It's the fastest Lexar CF. There is enough for 1920x1080 at 29.97fps continuous  14-bit lossless.

Lexar 1066x Speed Rating:
Max. Read Speed 160 MB/s
Max. Write Speed 155 MB/s

[Walter Schulz]

https://www.magiclantern.fm/forum/index.php?topic=11017.msg213202#msg213202
https://www.magiclantern.fm/forum/index.php?topic=23121.msg209110#msg209110
https://www.magiclantern.fm/forum/index.php?topic=12630.msg135713#msg135713

In short: There are at least 2 different versions labeled "Lexar 1066x" and I don't know any other way to distinguish them but benchmark after purchase.
Disclaimer: Haven't put that much effort into investigating, though.

But you're right: 14 bit lossless should do the job.

[whitelight]

I think color-wise the 1st version looks much better

[zen_nudist]

You don't need a faster card as there's a limit in a camera CF bus speed.
You're missing a lot not using a compressed lossless 14-bit. You can record 1920x1080 at 29.97fps continuous with a fullcolored realtime preview + proxy video to SD card (to have a preview in a camera). It's a regular Canon's video + ML RAW.
If you don't need proxies, you can get ML with higher resolution or higher FPS (up to 60 fps - 1920x800 but preview is a little broken).

All this will work with your current Lexar 1066x card

Which build do you recommend for compressed lossless 14-bit? I do not see any modules or options or anything in my build allowing "compressed lossless." Thank you!

[Dmytro_ua]

Which build do you recommend for compressed lossless 14-bit?

At first, start with https://builds.magiclantern.fm/experiments.html  Module name: 4K raw video recording; lossless compression  Latest Build (2018-07-22 13:09)
When you'll have got used to it, then you can try a modified version by Danne from here: https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899

[zen_nudist]

At first, start with https://builds.magiclantern.fm/experiments.html  Module name: 4K raw video recording; lossless compression  Latest Build (2018-07-22 13:09)
When you'll have got used to it, then you can try a modified version by Danne from here: https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899

Thank you Dmytro_ua!

Whoa, after hours of experimenting, I was able to get the 4K raw video recording; lossless compression build working. Was able to get continuous recording at: 12 bit lossless; resolution of 3072 x 1320; 24fps; aspect ratio 16:9. That was the highest frame rate and resolution was able to get and achieve continuous recording.

In case anyone's interested, here's a video of the test shots.

Pulled DNGs (from MLV App) into Resolve, did a very light grade, exported from Resolve to DNxHR 4444 at original 3072x1320 resolution (Holy file size, Batman!); brought into Premiere (along with a comparison VLOG L clip from my GH5) and exported to MP4 from Premiere.

I look forward to doing test shots underwater, which I'll soon share here in case anyone's interested. I'm not sure how well I'll be able to frame shots with the live view issues present in crop mode to achieve the UHD capabilities. Down at 120 feet and inside sunken ships ... this will probably be barrier ... but we'll see.

Also, I'll get a faster card so I can record at 30 fps and even higher resolution ... 80 MBytes/s write speed just isn't fast enough for long-term use.

[Dmytro_ua]

Thank you Dmytro_ua!

Whoa, after hours of experimenting, I was able to get the 4K raw video recording; lossless compression build working. Was able to get continuous recording at: 12 bit lossless; resolution of 3072 x 1320; 24fps; aspect ratio 16:9. That was the highest frame rate and resolution was able to get and achieve continuous recording.

And now, you're ready to try Danne's anamorphic mode:
- anamorphic presets so it now gives continuous:
1808x2300(2:35.1) = 5425 x 2300
1360x2300(16:9)    = 4080 x 2300

https://www.magiclantern.fm/forum/index.php?topic=23041.msg216899#msg216899

[zen_nudist]

For anyone who might be interested in filming underwater...

Below is an example of some clips. Recorded 3072x1320 resolution, 12-bit lossless, 24fps, 2.35:1 aspect ratio in the crop mode. Exported CinemaDNG lossless from MLV App. Imported to resolve and did basic color correcting, added contrast and midtone details (no LUT applied). Exported individual clips from Resolve to DNxHR HQX 12-bit intermediate codec. Imported to Premiere for editing clips together (I didn't really do any editing ... just going through this process, as this is what I'll do for real projects). Exported to 1920x"whatevertheheightis" and exported to .mp4 (h.264) with bit rate of 30 and max of 50. Uploaded to YouTube.

I look forward to seeing how the quality compares in deep water. These shots were just from a shallow water snorkel in mid-day, thus lots of sunlight to help with image quality and color correction.

[Luther]

Beautiful images @zen_nudist !
Have anyone here tried a red filter (wratten number 23A) such as this one from Tiffen? I think this would give better results than any post production color correction. I have this filter, but I don't really do underwater images. Or a warm polarizer. Something like ExpoDisc for correct WB would also be nice.
The new "Haze Removal" tool on Rawtherapee might really help for photography. Maybe that's a nice tool to be implemented in MLVApp in the future.

This research seems cool too:
http://csms.haifa.ac.il/profiles/tTreibitz/datasets/sea_thru/index.html

Some python implementations here:
https://github.com/wangyanckxx/Single-Underwater-Image-Enhancement-and-Color-Restoration

[zen_nudist]

As a follow-up, I've gotten the SanDisk Extreme PRO 160 MB/s 256GB CF card to allow for greater resolution and/or higher fps.

Using build: ML crop_rec_4k.2018Jul22.5D113

Tests in the ML benchmark module shows a write speed of 114 and 111 (did two tests), so yes it's faster than my older Lexar 1066X ... which gave me 12bit lossless 24fps 3072x1320.

However, at 12 bit lossless, 24fps, and crop mode at 3.5K 1:1 centered 5x5, I cannot get any resolution greater than 3584x1320, even when I set the LCD "Preview" mode to "Frozen," as A1ex (or some other developer) states can improve speeds.

I've tried several aspect ratios, ranging from 16:9 to 2.35:1, but I get nothing greater than 3584x1320. I've also tried several crop modes, switching from 3.5K 1:1 centered 5x5.
A question is why I'm getting greater resolution along the X-axis and no increase along the Y-axis, even when switching to aspect ratios that allow for more resolution along the Y-axis?  I don't think it's the card, because when I wheel through the different resolutions, going from 1920 up through 3072 and 3520 and up into 4096, the green text at the bottom of the LCD remains green and indicates no data rate greater than 98.4. (I might be misinterpreting this, however). This leads me to believe it's some combination of crop mode and aspect ratio I'm not understanding.

//TL;DR// What do I need to do to get to 3520x1520 at 12 bit lossless, 24fps, as this guy (with the same cards) does:

*As for 30fps, I'm able to get 3072x1320 at 12-bit lossless, which is pleasing (for slowing down motion induced by underwater environment). I don't expect I'll be able to get 30fps at higher resolution.

[yourboylloyd]

A question is why I'm getting greater resolution along the X-axis and no increase along the Y-axis, even when switching to aspect ratios that allow for more resolution along the Y-axis?

I don't know the exact reason, but all ML cameras have a MAX vertical resolution in standard mode and in crop mode. Maybe someone else can explain that part.

I used to use this calculator to calculate the ratios that I needed. I don't know if it's been updated or not (I'm sure it hasn't kept up with the nightly builds), but here it is. You can see that the max resolution for the 5d3 in crop is 1320. https://rawcalculator.netlify.com/calculator_desktop

[zen_nudist]

I don't know the exact reason, but all ML cameras have a MAX vertical resolution in standard mode and in crop mode. Maybe someone else can explain that part.

I used to use this calculator to calculate the ratios that I needed. I don't know if it's been updated or not (I'm sure it hasn't kept up with the nightly builds), but here it is. You can see that the max resolution for the 5d3 in crop is 1320. https://rawcalculator.netlify.com/calculator_desktop

Hmm, but other folks have demonstrated 3520x1520 resolution with the 5d3 using the crop rec module.