3K/UHD 5D2 Raw development and Other Digic IV Cams

[reddeercity]

New test build
crop_rec-4k3x1_2784x1160_FHD48p_12.18am-5D2-eXperimental.2019Jul01.5D2212.zip
Got 48p in 3x3 @ 1856x688 working (kind of) , should have been 1.67x vertical but adtg reg don't apply .
the whole adtg structure seem not to work with 5d2/d4 , I've look in to adtg_gui.mo and the adtg hooks are different then crop_rec
so I'll have to try and use the adtg_gui hooks the way they are code in there . 
Also 2784x1160 should be centered vertically & horizontal now even when zooming in & out with a (zoom len), should stay in the center now .

Short test 9 second with the crop_rec in FHD 48p 1856x688_48fps-M01-0032.mov

To recap;
48p @ 1856x688 (3x3)
2784x1160 centered vertically & horizontally
3x1 4K @ 23.976p

Same as before , it may lockup , freeze , etc. ... you guy know the routine
Edit: 48p can be hard to enable , what I did was first put the cam in 1:1 (3x crop_mode)
enable 4k Anamorphic for a few second then switch to "FHD 48p" press the play but twice .
It will say in the left corner 1:1 but it's really 3x3 FHD , need to change that to say 3x3 .

[Galterius]

Will , I see no aliasing in the clip I downloaded but it is at least 2-3 stops underexposed (try to use ETTR technique)
I always use Adobe After Effects CS6 to process my MLV files , so I hardly ever come across aliasing issues .
What I find that help with aliasing is to use "Edge Defringe" in A.E.
 



here is the XMP file from adobe after effects to adjust the dng's M30-2057_000000.xmp
and here's the exported h264 mp4 file 4096x1784_h264-A.E._M30-2057.mp4
Not sure why you are using 12bit , 10bit should be enough
FYI: I use Technicolor's CineStyle Profile for my Liveview  to judge my exposure with & I always get very clean shadows .
It's never fail me yet !

Well then my MLV app has a problem because the image is awful, completely unsharp and werd colors everywhere... I will try with the 1.6 version... and also in AE.
I will also send you another clip so you can tell me what you think.
btw, the clip is underexposed but i exposed for the highlights (maybe its not the right way to do things ^^)



Here is a png exported from MLV app without any correction (converted to jpeg because it was too big), its unusable...

[masc]

Well then my MLV app has a problem because the image is awful, completely unsharp and werd colors everywhere... I will try with the 1.6 version... and also in AE.
I will also send you another clip so you can tell me what you think.
btw, the clip is underexposed but i exposed for the highlights (maybe its not the right way to do things ^^)

Here is a png exported from MLV app without any correction (converted to jpeg because it was too big), its unusable...

MLVApp has no problem with that. The problem is still the 3x1 readout with line skipping. Current official version is v1.7. The way how you expose (+/-) doesn't matter, as long it doesn't clip. The used aperture is more important (open aperture brings less moiree). The picture you posted shows the same like all 3x1 test pictures. Have you tried another debayer algorithm? The jpg/png/dng is not important. A shortend MLV would be more important (1 or 2 frames is enough).

[Galterius]

Here is a shortened MLV (I chose Fast Pass in export settings)

https://drive.google.com/file/d/1xKjikSsYZnvyqrkQI4BWmStF-5AVwOto/view?usp=sharing

[masc]

Here is a shortened MLV (I chose Fast Pass in export settings)

https://drive.google.com/file/d/1xKjikSsYZnvyqrkQI4BWmStF-5AVwOto/view?usp=sharing

117MB? Shortend? Did you set the CutIn / CutOut?

[Galterius]

Here it is sorry (20mb)
https://drive.google.com/file/d/14tazB5SekYdMNuBOmREfc3DqNrZDIW8a/view?usp=sharing

[2blackbar]

Here is a png exported from MLV app without any correction (converted to jpeg because it was too big), its unusable...

You should ETTR (expose to the right) with raw , otherwise you will get what you got.You have histogram which could tell you how much you can bump exposure up.
Recording with RAW is different than recording with h264, what you see on the screen is not what you will get, in liveview you have picture style applied and much more contrast than actual RAW frame.You should almost overexpose with raw.
I filmed with 5d2 today and dont have what you got, use magic zoom or peaking to get sharp focus
Id say all problems with this picture is your mistakes and filming like regular h264.

[Galterius]

You should ETTR (expose to the right) with raw , otherwise you will get what you got.You have histogram which could tell you how much you can bump exposure up.
Recording with RAW is different than recording with h264, what you see on the screen is not what you will get, in liveview you have picture style applied and much more contrast than actual RAW frame.You should almost overexpose with raw.
I filmed with 5d2 today and dont have what you got, use magic zoom or peaking to get sharp focus
Id say all problems with this picture is your mistakes and filming like regular h264.

Ok I will try that, it should fix that aliasing ?

[masc]

Here it is sorry (20mb)
https://drive.google.com/file/d/14tazB5SekYdMNuBOmREfc3DqNrZDIW8a/view?usp=sharing

This is better, thanks. See some statement below...

You should almost overexpose with raw.

I would not recommend this, because if one channel clips just a little, it will look really bad. I always record at ~EV-2/3 for not getting crushed clips.

I filmed with 5d2 today and dont have what you got, use magic zoom or peaking to get sharp focus

Did you try 3x1 4K mode? If not I agree you won't get this. And you just will get those artifacts if the focus was correct. If you watch the MLV, it is very sharp - for a 3x1 file. Use a sharpener and you'll see the focus is okay. If you use the sharpener combined with "Chroma Separation" checkbox in MLVApp, you desaturate the color artifacts. But again... the moiree is still there. Moiree / aliasing will always be there when recording with line skipping. No matter what program used for processing and no matter what the exact settings were when recording.

[Galterius]

So I have to expose looking at the histogram, just before it says OVER ? (so it says E0.3, or E0.4...)

the weirdest thing is that I had no issue before, check this :

[masc]

So I have to expose looking at the histogram, just before it says OVER ? (so it says E0.3, or E0.4...)

the weirdest thing is that I had no issue before, check this :...

Here, no hard contrast edge is in focus... no chance for moiree.

[Galterius]

Here, no hard contrast edge is in focus... no chance for moiree.

ok, thanks. I definitely need to learn a lot but I am so excited about that. I will try again and again

[2blackbar]

E0.3, is too high already, try to keep it with at least 1.0 or something but you need to test this yourself and get a hang of it so you wont have to rely on histogram that much, in some scenarios(under tree for example) you have to choose if you want to keep bright areas not clipped or avoid noise in shadows, i go for second option because overexposing in raw does not look as bad for my taste but underexposing is a killer for some shots , you cant save them to look decent and have to do denoising which will steal the details.
 Second clip is underexposed too.
Maybe youre like me and you like when its dark and moody but in raw you should not try to get that in camera , expose to get most dynamic range from the image and pick whats most important part of the image, sometimes some guy in far background with bright sunglasses will overexpose your histogram( it will say overexposed even if your close subject is underexposed in dark shadow) , hes not important, let him clip and preserve dynamic range of your important subject.
 By writing "You should amost overexpose" i meant , i go for overexposed and if iso200 is overexposed then i go to iso 100 if its not overexposed and its about one stop till overexposed, if i expose for liveview to make it look nice on the screen then image is way too dark later in MLVApp, thats why i dont judge by looking at the liveview, mostly histogram and past experiences help me to judge what ISO should be used but its best to use ISO100 because this one has most dynamic range, higher ISO's will lose a bit from dynamic range.

[Igor_Braun]

Awesome, awesome...
Tested 48FPS

48 fps original:

48 fps 2 time slower (23,98):

Also, i was excited about less rolling shutter but noticed that it's pretty the same as in regular 1856 23.98 fps
Want to test 1x3 4k so much. And compare it with 3x1.

@Reddeercity Why do you use 1/130 instead of something like 1/100. I see a kind of strobe/rolling effect on the video. I think you may avoid it with 1/100

[Galterius]

E0.3, is too high already, try to keep it with at least 1.0 or something but you need to test this yourself and get a hang of it so you wont have to rely on histogram that much, in some scenarios(under tree for example) you have to choose if you want to keep bright areas not clipped or avoid noise in shadows, i go for second option because overexposing in raw does not look as bad for my taste but underexposing is a killer for some shots , you cant save them to look decent and have to do denoising which will steal the details.
 Second clip is underexposed too.
Maybe youre like me and you like when its dark and moody but in raw you should not try to get that in camera , expose to get most dynamic range from the image and pick whats most important part of the image, sometimes some guy in far background with bright sunglasses will overexpose your histogram( it will say overexposed even if your close subject is underexposed in dark shadow) , hes not important, let him clip and preserve dynamic range of your important subject.
 By writing "You should amost overexpose" i meant , i go for overexposed and if iso200 is overexposed then i go to iso 100 if its not overexposed and its about one stop till overexposed, if i expose for liveview to make it look nice on the screen then image is way too dark later in MLVApp, thats why i dont judge by looking at the liveview, mostly histogram and past experiences help me to judge what ISO should be used but its best to use ISO100 because this one has most dynamic range, higher ISO's will lose a bit from dynamic range.

Ok thanks for the information, I will try different exposures to see how it works.
By the way what do you turned on in your display settings ? Are vectorscope, or other diagrams necessary ? I don't know which ones I should use to help me expose correctly.

[2blackbar]

I used to use histogram and waveform with h264, with raw i use histogram and expose at one stop before oveexposure  because i didnt tested if waveform reads raw image(which would be good) or what is on liveview(which would be bad),taking into account most important subject in shot and trying to not over-under expose it , even if i want moody darker image, i prefere to darken in post because with raw i see more in shadows than h264 which means i see more noise in shadows, underexposed raw footage on my 5D2 and M looks quite bad.
Histogram informs you about brightest part in the frame so if you are underexposed by 2 stops and someone in background holds in hand tiny piece of mirror that reflects the sun then you will get overexposed info which is not correct, i mean technically it is but youre still underexposed and shouldnt look at the tiny overexposed parts of the image that dont really matter, sometimes i just pan/tilt camera until that small thing disappears from frame so it wont overexpose.
Ffor example if small cloud in the sky is overexposing, i tilt down so sky is not in the frame anymore and i adjust proper exposure, then i tilt up so sky is in the frame, if its overexposed then well.. too bad.But with sky its not a great example because i tend to care about it not being overexposed.
In some shots its kinda important subject.
Last time i remember being under tree , i had to adjust for under tree lighting/brightness, not brightness thats far away which was overexposed, if i go midway and underexpose under tree while having not overexposed far background then i still have crap image because im underexposed under tree where my main subject is.
I hope its not too confusing .Just expose for your subject, not for background, its great if you have both properly exposed but if you cant then go for properly exposing subject.There are exceptions from all this and its up to you to decide, i just found out that underexposing with raw is not a good idea if you dont like noise.I can brighten in post about half-one stop before noise is unacceptable while i can darken the image SOOOOO much its insane with RAW.

[reddeercity]

Ok guys this is really off topic here , can you please talk about this on the many other raw video processing threads this is for development of crop_rec for D4 cameras . Clearly the problems are user errors/cam setup . Please learn your camera setting and what there are for , search the forum all your questions have already been answered many times .

[Igor_Braun]

Today i was testing FHD in the streets but after several recorded clips all the modules disappeared.


So i got to my place, installed 48 fps build to my other card, went to the street again and got this:

[2blackbar]

Is 12fps the limit of anamorphic mode or is it just me? Cant get it to work with 23.976 override.If i dont use override it locks at 12fps

[Bender@arsch]

@Igor_Braun
Use default setting on ML. I see you use Magic zoom on both cards - > don't work with FHD48.
Format card complete from camera, re- install and see what happens. Or maybe battery out for 10 sec.

@2blackbar
12fps is not correct. Don't use fps override in crop mode.
Maybe same problem. Format re-install, try default setting

@Reddeercity
For me all new builds work great. Without failer.
Thanks for your work!

But:
- 3.5K is centered better, but not perfectly, -> little bit up more we need.
- I think Temperature is a little bit to fast to high.
- The vertical lines disappear with me after approx. 10 seconds
- can you implementat greyscale automatically in 4K its a little bit user-friendly

What build comes next?

[reddeercity]

- 3.5K is centered better, but not perfectly, -> little bit up more we need.
- I think Temperature is a little bit to fast to high.
- The vertical lines disappear with me after approx. 10 seconds
- can you implementat greyscale automatically in 4K its a little bit user-friendly

Thanks for the feed back ,
Ok , I'll look at 3.5k centered again , temperature wise as long as your under 68°c but 48p will heat up a bit I was seeing around 58°c but normally it's around 48-53 in 4k Anamoph.
About those lines , I had too disable a pause between cmos's 1 & 2 when being applied because the  statement (! is_5D2) wasn't always working but I'm working on a solution .
Until then just wait a few second before you recording . The greyscale preview should be automatic unless you are in auto mode preview , choose ml b/w/ greyscale for preview
but I'll look in to it at some point too .

What build comes next?

More likely this --> 4k (4096x1770) half framerate of 24p (12fps) in 1:1 (should be close at least 10fps I think)
Then this -- being back 3K (3008x1080 maybe with a little more res)
and this 1920x1080 @ 45p in 1:1(3x crop_mode) 
The last one will be 1x3 4k , as this need my full attention (adtg hooks need to fixed for d4 cams)
Then I can move on to Lossless .

[zcream]

4k (4096x1770) 12 bit lossless compression may be enough for the card speed limit. Fingers crossed

Sent from my Redmi 4A using Tapatalk

[Walter Schulz]

4096 x 1770 x 12 x 24 /8 = 249 MByte/s.
Pipe dream about required compression rate IMO.

[OlRivrRat]

      @Walter

   I'm not @all knowledgeable about calculations

so would like to know >

4096 x 1770 x 12 x 24 /8 = 260997120

How does 260997120 = 249MB/s

[Walter Schulz]

Because ML uses
1 MB = 1024 kB = 1024 x 1024 B
(virtually nobody uses the correct term KiB to tell binary based k (= 2¹⁰ = 1024) from decimal k (= 10³ = 1000) and its the same for MiB and GiB.
And that's why an empty 32 GB card is showing 29.8 GB (=GiB) available in ML's screen. Storage manufacturers are using decimal G = 1000 x 1000 x 1000 for some time (was different when I started in IT).