3K/UHD 5D2 Raw development and Other Digic IV Cams

Started by reddeercity, April 06, 2017, 12:22:27 AM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

Milk and Coffee

Reddeercity, thanks again for all your hard work on this! I tested the new mlv_rec module "1880_3x3-2152_5xZoom-5d2+50d-support__7-5-2020-5.59pm-mlv_rec.zip" with the latest "crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip" build. I'm loving the new 3x3 resolution!

I noted four things for the 5D2:
1) In 3x3 1880, and in all bit depths, I'm getting pink spots in the sun, see .dng here > https://www.mediafire.com/view/ssal2p12hvreadi/C-12-BIT-111.dng/file Is this something that needs to be repaired in MLV app?
2) In live view movie mode, ML reports 1/48s shutter speed, but the top LCD of my 5D2 reports 1/90s. And when switching out of live view, and capturing a still, it is captured at 1/90s.
3) Forgive me if you've already addressed this, but the fastest shutter speed I can achieve in mlv_rec (3x3 1880) is 1/100s despite the top LCD reporting all shutter speeds up to the max 1/4000s.
4) mlv_play isn't working and causes no modules to load.

Thanks again for your continued development!
Canon 5D Mark II, Mac/OSX

masc

@Milk and Coffee:
1) you overexposed / clipped the green channel. This brings pink color. Try to reconstruct highlights!
5D3.113 | EOSM.202

Milk and Coffee

Canon 5D Mark II, Mac/OSX

reddeercity

Can't help without the original  dng/mlv file shorten  , not jpeg compression (lossless) or png .
just use passthou when exporting dng's in mlv app. and it will export dng's at the original bit depth
Did you try Adobe ACR ? (Photoshop/After Effects)


reddeercity

Quote from: Milk and Coffee on July 11, 2020, 11:32:13 PM
Original DNG here https://www.mediafire.com/view/ssal2p12hvreadi/C-12-BIT-111.dng/file
Quote from: reddeercity on July 11, 2020, 11:29:44 PM
Can't help without the original  dng/mlv file shorten  , not jpeg compression (lossless) or png .
......



As i said this is jpeg Lossless , i need the uncompressed dng , see what it say at the compression line
5D Mark ii Dose not shoot Lossless Compression
The file size of the dng should be around 4-5Mb not 1.23Mb

Milk and Coffee

Oh! I thought you meant JPEG as a file type. Let me shoot another test as I don't have the original mlv. Or can I make another uncompressed DNG from this one?
Canon 5D Mark II, Mac/OSX

reddeercity

read this post to get up to speed  https://www.magiclantern.fm/forum/index.php?topic=19336.msg228941#msg228941
Ok , I haven't had time to make a video yet on the procedure to enable the new crop_rec.mo with multiplies 3x crop_mode preset .
So here is the way to enable crop_rec preset's

Note** Always start with No.1 Step**
1: Start off in 3x3 , enable the "3.5k centered  preset (2784x1160) , refresh liveview (half shutter)
then press the 5xZoom button "centered preset" will load from there .

2: To enable 3k 1:1 (3016x1080) , press trash can button , to bring up the ml menu , enable the 3K 1:1 preset
**Note** do not exit 3.5k centered preset before enable 3k preset , just go to the crop_rec menu & enable.
fresh liveview with half shutter , the lcd screen will be black , don't worry now press the play button twice (2x)
then cycle though the 5xZoom button (usually  press it 1 time , sometimes it needs to be press twice)  .
The most important part is Press the play button 2 times (but not too fast thou) .
Now once you are in 3k (3016x1080) you can go to 4k 3x1 (4096x590) from there.

3: Just enable the 4k1x3 preset while in the 3k 1:1 preset and refresh liveview (half shutter) the preset will load itself from there .

4: 3x3 48p FHD (Full HD) needs to load from the default 3xcrop_mode (2152x1078) , enable the 48p preset then refresh live
(you may have to cycle thou the 5xZoom button on the odd time) , but it should load as soon as you refresh liveview .

 

reddeercity

I found some time today to shoot a short video on how to enable the new crop_Rec.mo  with multiplies 3xcrop_mode preset's
Shot it on my 50D in 1x3 1376x1742 (4128x1742) @24fps , did a quick grade in mlv.app and exported and uploaded prores 444 to Youtube
Make sure you watch in 4k  ;)



Milk and Coffee

Quote from: reddeercity on July 11, 2020, 11:29:44 PM
Can't help without the original  dng/mlv file shorten  , not jpeg compression (lossless) or png .
just use passthou when exporting dng's in mlv app. and it will export dng's at the original bit depth
Did you try Adobe ACR ? (Photoshop/After Effects)

New sample DNG's here (included both uncompressed/passthrough, and lossless compressed) http://www.mediafire.com/folder/lelofkxvrvyhw/Pink-Sun

Different clip, but same problem; pink pixels in the center of the sun in Davinci Resolve. It's especially apparent with "highlight recovery" turned on.
Canon 5D Mark II, Mac/OSX

reddeercity

I don't use Resolve , it does a trouble job of raw from magic lantern specialty from 5D2 , I never seen anything good from it (there reason why it's free)
Your better off using MLV App to develop your dng's , I use Adobe After Effects CS6 .

I See many problems here , First you Badly Over Expose the image & there too much data loss in the high light to be reconstructed .
From the Metadata from the dng you use F2.0 @ 1/48th shooting the sun , plus your are out of focus .
Better set up would be , F8 plus a (Variable ND) and or ND 0.6-0.9 or F16 with a Variable ND 0.3-400 .
There no programs the can fix data loss , you need to trust the ML meters in liveview  and ETTR (expose to the right) .

I run the dng thought Adobe A.E. cs6 and i did get a not too bad image with out any pink sun , but it took some time .


Here the exported tif from cs6 1-Uncompressed-77_0-00-00-00.tif
Also here is the 1-Uncompressed-77.dng from After Effect with the ACR adjustment make

Milk and Coffee

Thanks for taking the time! It was just a quick clip from my apartment patio. I intentionally overexposed to make sure the pink highlights would show in Davinci.

Your graded image looks great! Good highlight roll off!

Any Davinci users that can shed some light on the pink pixels happening in Davinci?
Canon 5D Mark II, Mac/OSX

70MM13

it's just a matter of making sure the highlights actually reach the top of the signal range.

this took about 30 seconds in resolve.

boosted highlights with the log wheels, and then made a quick curve to extend the range to fill out the scopes as shown in the overlaid scopes.

https://ibb.co/zntbzhs

Milk and Coffee

I appreciate all the reply's, and grading suggestions! Maybe I didn't clarify enough: The DNG is a sample to demonstrate the pink pixels in the sun. I didn't shoot it with intentions of using it. 

Seems to be a bug with Davinci? Or with ML? Didn't know if anyone had any insight as to why they are appearing.

70MM13, did you notice the pink pixels?
Canon 5D Mark II, Mac/OSX

ZEEK

I haven't had that issue yet with the 5D2. It has to do I believe with the White Level of the Camera. The topic was discussed, you can check it out here [Reply #3289]:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg219884;topicseen#msg219884
EOS M

reddeercity

Has nothing to do with it , read the thread where you posted from , it has to do with the eosm with lossless compression .
That does not apply here at all !
There no information in the high light at all from the posted sample , it a user setup problem.
As I posted the corrected image from ACR/A.E. , resolve can't handle the high light reconstruction like ACR , actually there no program/app that can reconstruct high light like ACR .

70MM13

i'm not sure if you guys looked at the image i sent...

i pushed it just enough to get rid of them for my taste.  i didn't want to push any further than necessary or else there would be even more useful visual information lost.  if any remained in my image, you must be able to see something i cannot see here.

so just try it yourself and push a little further if you are still seeing them.  and i had highlight recontstruction ON.  i like what it does to the halo of the sun.  there are definitely more pink pixels with reconstruction on.

if you are hypersensitive to such almost invisible things, you can simply create another layer, select that colour with the eyedropper and drop the saturation to zero on that layer.  gone without a trace.

i don't care what program anyone uses.  i'm just offering help with resolve as the poster asked.

Danne

Pink dots in highlights is not related to lossless compression. On the eosm I modified one of the cmos settings which was causing the issue. Reverting the change fixed the issue. Maybe this could be fixed on 5d2 as well tweaking a cmos reg. Or maybe it's wrong in the preset you recorded in.
Resolve is superior to premiere in so many ways nowadays imo handling dng files which is the more fair tool comparison. Anyone can pull the highlight slider in acr on s single image. Try applying it to a sequence and the flicker party started. But, with some dcp tweaking acr actually works really good even for dng sequences but still. Cutting in after effects takes forever. Too bad adobe don't develop their own programs for their own dng format. Strange.

70MM13

do you remember what cmos setting exactly was involved?  it might be very useful to know this for other times this issue pops up!

Danne

I don't. Dig around from the post from Zeek. Shouldn't be far from the fix.

reddeercity

@70MM13 , yes i looked at your sample .
Quote from: Danne on July 17, 2020, 01:54:44 PM
.... Maybe this could be fixed on 5d2 as well tweaking a cmos reg. Or maybe it's wrong in the preset you recorded in.
I don't think you look at the sample "1-Uncompressed-77.dng" , it's a default 3x3 from mlv_rec  (1880x1058)
Don't asume  it a crop_rec issue or anything without totally check the sample .

easily fix in RawTherapee with one step , High Light Recovery - Luminance Recovery
That's because of the data loss from over exposing too much .




Ok i take it back , there is another program (RawTherapee) that handles high light as good if not better then ACR/After Effect CS6

70MM13

that's the clip i used...

i agree about how great rawtherapee is.  in fact, i used it to process my first music video with the 5d3 in raw video back in 2017.  talk about a slow process, but it looked beautiful.  there were a few issues however in using rawtherapee for video:

first, it crops the image by 8 pixels on each axis.  not a fatal flaw, but not nice.

second, third, fourth etc, several of the tools can't handle changing light conditions.  unfortunately, it was a problem with some of the best tools.

if someone forked rawtherapee for video usage, it would be a very valuable asset for us all...  it does better debayering than anything i have tried.

Milk and Coffee

Thanks for all the useful replies! So it sounds like it's an issue with Davinci resolve. Hope they are still updating their cDNG functionality now with all the other raw formats out there.
Canon 5D Mark II, Mac/OSX

70MM13

i wouldn't hold my breath.  their priority is the blackmagic cameras, and rightfully so.

but it would be nice!

reddeercity

Being checking on my source code in bitbucket and notice if you update my branch to the latest commit 8eae4f2
you will get a raw.c error
../../src/raw.c:1840:12: error: redefinition of 'lv_raw_type'
static int lv_raw_type = PREFERRED_RAW_TYPE;

Before bitbucket shut down updating I forgot to comment out raw.c #line 229 like this Sorry  :-[
//static int lv_raw_type = PREFERRED_RAW_TYPE;//
This will fix the compiling error , I did a quick test and compiles fine now .

I wish i had updated the source more , but I'm looking in to git now to host my code unless a1ex has setup a new site to host ml code ?

If you're new to coding and wondering what's a good code editor , being using Visual Studio Code  (on win7 , cross platform on mac ,etc. ..) to review the changes from the hg
makes debugging real easy , show the diff between code changes , even helps with code syntax and show any error right now .  :D