Author Topic: 3K/UHD 5D2 Raw development and Other Digic IV Cams (Read 531588 times)

Danne · « **Reply #800 on:** June 15, 2019, 08:47:24 AM »

Do an exception in raw.c where auto black level is done...

reddeercity · « **Reply #801 on:** June 15, 2019, 09:27:24 AM »

Got "pack12 stuff" figured out , and yes it's for centering vertical offset just need to do some math .
https://www.magiclantern.fm/forum/index.php?topic=19336.msg217565#msg217565

Galterius · « **Reply #802 on:** June 15, 2019, 07:45:55 PM »

Quote from: reddeercity on June 15, 2019, 08:33:15 AM

Ok playing around with 2784x1160 @ 23.976 Real Time liveview preview , I think I figured out the issue with black level in 2.6K
and applied then fix to 2784x1160 . Now I'm having issue with the cmos[1] vertical offset so you will have to use the advanced
crop_rec menu and adjust cmos[1]lo to 18 & cmos[1]hi to 62

issue seems to be with the "cmos_new[1] PACK12(9+2 ,42+1)"
I'm not sure what "pack12" comes from , if I could understand what the number are calling (some kind of centering) for vertical offset
if I disable the pack12 the image cleans up but it pushes up with a white bar at the bottom , so you have to fine adjust it with the advanced
crop_rec menu like the image above until I figure out how to code it centered vertically .
So remember , expect problems , lockup's or anything odd as it could happen at any time !
This build is for test proposes only !! I really looking for any black level changes while recording .
magiclantern-crop_rec-2784x1160_24p_11.39pm-5D2-eXperimental.2019Jun14.5D2212.zip

Edit: so if you add the cmos[1] lo18 & hi62 together you get 80 now that's the same as the white area from the image height
Original vertical is 1160 if you crop off the white area it's 1080 , so does this mean "pack12etc. ...." is just the vertical centering .
So tomorrow I'll change the "pack12 (9+2 ,42+1)" to "pack12 (18+62)" and see what happens

Thats awesome ! i will try it ASAP to see if the black level issue is fixed ! Thanks a lot !

Galterius · « **Reply #803 on:** June 15, 2019, 08:15:12 PM »

So I just tried it (2784x1160), here are some clips, colors are clipping every 3/5 seconds (maybe its because of the shutter speed, I could not set it to 1/48)
It still seems a little bit too green (but maybe I am wrong), I will try on skin tones

https://drive.google.com/drive/folders/1gfgZUH7ruZd4HlZC5amwb1lDWoKYg1P3?usp=sharing

reddeercity · « **Reply #804 on:** June 15, 2019, 09:11:29 PM »

In canon menu set to 1080 30p
NTSC .
In magic lantern set shutter to
1/60th of second or close to it (1/63th) while your in 3x3 (FHD)
and it will change to 1/48th to 1/52th of a second . When you enable crop_rec

Galterius · « **Reply #805 on:** June 15, 2019, 10:51:39 PM »

Quote from: reddeercity on June 15, 2019, 09:11:29 PM

In canon menu set to 1080 30p
NTSC .
In magic lantern set shutter to
1/60th of second or close to it (1/63th) while your in 3x3 (FHD)
and it will change to 1/48th to 1/52th of a second . When you enable crop_rec

ok thats what I did

by the way you just wrote your 2000th message here

reddeercity · « **Reply #806 on:** June 16, 2019, 07:05:26 AM »

2784x1160 @ 23.976 real time Liveview preview
I was right about the "PACK12 Stuff"(vertical fine offset adjustment) just needed to put in the correct offset number's
So now there no need to go in to the advanced crop_rec menu , should be very close to correct vertical centering now .

Caveat if you load Snd_mo (sound module) you may/will get corrupted frames !!
I highly recommend only loading crop_rec.mo & mlv_rec.mo that's it ! I'm running very close to the edge on this preset .
I've squeezed every bit of performance I know how to get to make this work .

The Raw dump size is 2792x1160 but with mod8 rules it get shaved down to 2784x1160
The total active area is 2952x1212 so if we do some simple math to find out Mega pixel per second (MP/s) the speed of the sensor
2.952x1.212x23.976= 85.78 MP/s , is this the limit ? maybe but I haven play around with the other headtimers (1&2) yet so
there might be some more room to reduce the overhead even more , time will tell but until then........

magiclantern-crop_rec-2784x1160_24p_10.24pm-5D2-eXperimental.2019Jun15.5D2212.zip

Edit: when you are set to 1/60th (ML 1/63th) of a second for shutter speed the crop_rec will be 1/52th second shutter
that's the closest I can get right now to 1/48th of second .

Edit#2: I updated the first post on the first page of this thread with the new test build .

Galterius · « **Reply #807 on:** June 16, 2019, 11:26:41 AM »

Awesome !! I will also try it ASAP !!

By the way, is it normal if I get lots of vertical lines when using 4096x590 ? Weird thing is that these lines appear mostly on the blurry background, and where its focussed, there are no lines...
I will try to send some clips so you can see what i am talking about

#Edit : here is a clip, no color correction, exported to Apple Prores 4444 (yea thats overkill but I wanted to try it)

RH42 · « **Reply #808 on:** June 16, 2019, 11:19:51 PM »

@Galterius Have you switched off the Auto Lighting Optimizer in the Canon menu? Just a thought - you probably have it switched off, but worth checking. I had problems caused by that under certain lighting conditions back in the day.

reddeercity · « **Reply #809 on:** June 17, 2019, 05:01:54 AM »

Yea I know the problem , it's because of 2 thing
1) The black bar on the right side is part of the problem
2) The override vertical fine adjustment (cmos[1]lo & hi in crop_rec advanced menu is
fighting the with the code

Code: [Select]

PACK12 (9+1 , 42+1)
which is doing the small thing , this need to be changed ,
I haven't done that yet (as I just discovered how to code it correctly last night )
Soon , I may look at it tonight yet .

Galterius · « **Reply #810 on:** June 18, 2019, 02:23:14 PM »

Quote from: RH42 on June 16, 2019, 11:19:51 PM

@Galterius Have you switched off the Auto Lighting Optimizer in the Canon menu? Just a thought - you probably have it switched off, but worth checking. I had problems caused by that under certain lighting conditions back in the day.

I will try that !! Hope it will fix it !

Edit 1 : it is switched off...

Galterius · « **Reply #811 on:** June 18, 2019, 02:24:07 PM »

Quote from: reddeercity on June 17, 2019, 05:01:54 AM

Yea I know the problem , it's because of 2 thing
1) The black bar on the right side is part of the problem
2) The override vertical fine adjustment (cmos[1]lo & hi in crop_rec advanced menu is
fighting the with the code
Code: [Select]
PACK12 (9+1 , 42+1)
which is doing the small thing , this need to be changed ,
I haven't done that yet (as I just discovered how to code it correctly last night )
Soon , I may look at it tonight yet .

Ok, thanks a lot ! (I can wait, I have other builds to wait

)

reddeercity · « **Reply #812 on:** June 19, 2019, 05:21:28 AM »

I'm very close to having 4k Anamorphic (10bit raw 4064x590->4064x1770) coded , still having a problem with "pack12" applying to cmos[1] , but there no problem with
the 3x crop preset 2784x1160 , I need to write some more yet , almost there

Here a quick test , from my back yard yesterday , MLV App -> to 10bit dng's ->import in to CS6 Adobe A.E. , grade with ACR ->export ProRes4444->upload to YouTube
To see the 4k stream watch on chrome ,
FYI: ISO 100 1/50th , Canon EF24-70mm f2.8L , crop factor 1.38 shot at 24(33.12)mm & 70(96.6)mm @ f8 & ProMaster Variable ND for video set to ND3

Sorry for the double post of the Video

thought it should be here also .

Galterius · « **Reply #813 on:** June 19, 2019, 09:00:47 AM »

Quote from: reddeercity on June 19, 2019, 05:21:28 AM

I'm very close to having 4k Anamorphic (10bit raw 4064x590->4064x1770) coded , still having a problem with "pack12" applying to cmos[1] , but there no problem with
the 3x crop preset 2784x1160 , I need to write some more yet , almost there
Here a quick test , from my back yard yesterday , MLV App -> to 10bit dng's ->import in to CS6 Adobe A.E. , grade with ACR ->export ProRes4444->upload to YouTube
To see the 4k stream watch on chrome ,
FYI: ISO 100 1/50th , Canon EF24-70mm f2.8L , crop factor 1.38 shot at 24(33.12)mm & 70(96.6)mm @ f8 & ProMaster Variable ND for video set to ND3
Sorry for the double post of the Video thought it should be here also .

amazing ! Did you notice any issue with black level ?

banertop · « **Reply #814 on:** June 19, 2019, 02:27:34 PM »

aliasing on that bricks?

masc · « **Reply #815 on:** June 19, 2019, 08:09:57 PM »

Quote from: waza57 on June 10, 2019, 05:57:46 PM

I do not have enough time to dig through the whole thread but it would be really nice to have a summary of the registry changes to get the progress that reddeercity found:

2624x1200 @ 23.976 fps with the full vertical heigh
4096x590 (Anamorphic 4096x1770) @ 23.976 fps
....and other things that don't remember.

These only in these conditions I could easily update the source code.

Igor_Braun · « **Reply #816 on:** June 20, 2019, 12:29:13 AM »

Quote from: reddeercity on June 19, 2019, 05:21:28 AM

I'm very close to having 4k Anamorphic (10bit raw 4064x590->4064x1770) coded , still having a problem with "pack12" applying to cmos[1] , but there no problem with
the 3x crop preset 2784x1160 , I need to write some more yet , almost there
Here a quick test , from my back yard yesterday , MLV App -> to 10bit dng's ->import in to CS6 Adobe A.E. , grade with ACR ->export ProRes4444->upload to YouTube
To see the 4k stream watch on chrome ,
FYI: ISO 100 1/50th , Canon EF24-70mm f2.8L , crop factor 1.38 shot at 24(33.12)mm & 70(96.6)mm @ f8 & ProMaster Variable ND for video set to ND3
Sorry for the double post of the Video thought it should be here also .

@Reddeercity, sorry for living one of your message without an answer. Had difficult days...
This is an absolutely cool resolution on your video, very noticeable on the bricks . Just tested out 4kanamo preset compared with 1856 one in real life and noticed that 4kanamo is really much more detailed than 1856. Actually, before i didn't believe that there is a real difference...

reddeercity · « **Reply #817 on:** June 20, 2019, 04:17:00 AM »

Even though there maybe a little aliasing , (I think the brick is really me bumping the camera) resolution always wins out !
1856 , is every third column (5616/3=1872) so you are missing data really . 3x1 is every column just with 4k it's at a crop factor of 1.38 .
If I had to choose between 3x3(FHD) 1856x1248 & 3x1 4064x590->4064x1770 , pick 3x1 4k Anamorphic all day long .

reddeercity · « **Reply #818 on:** June 20, 2019, 04:38:59 AM »

New test build crop_rec-4kAnamorph_12.36am-5D2-eXperimental.2019Jun17.5D2212.zip
I'm dropping 2600x1200 in favor of 2784x1160 @ 23.976fps realtime preview , just too many issue with 2.6k beside that
2784x1160 is at 2.4 A.R. and the sensor runs cooler temperature wises also .
I reduced the 4k Anamorphic resolution from 4096 to 4064 , to maintain 23.976fps at 4096 I can only get 24.001fps or 23.98fps
but I didn't change the vertical it's still 590->1770 , I still have a problem with vertical centering so you have to use the
advanced crop_rec menu and set

Code: [Select]

cmos[1]lo->11
cmos[1]hi->61

then do the normal thing to get to it , make sure you refresh liveview before recording raw video .
Best way I found is to use the play button twice then that get to 3x3 , then enable crop_rec
should good from there , once I get those centering regs working you won't have to go though
all that refreshing of live view . This is the build I used to make the video in the above test video
of my back yard.

Edit: I'm slowly removing all those test builds on the first page of the this thread , I'll leave 3008x1080
but all the 2.6k build are gone , if you really need it just request it and I'll look at it .
So 2784x1160 will be the new standard for 3x crop_rec & 4k Anamorphic 4064x590 for now .

reddeercity · « **Reply #819 on:** June 21, 2019, 04:39:26 AM »

New test build , magiclantern-crop_rec-4k-3x1_2784x1160_1920centered_7.54pm-5D2-eXperimental.2019Jun20.5D2212.zip

Ok in 4k 3x1 Anamorphic , you now longer have to go to the advanced crop_rec menu to set the centered vertical , got that figured out now.
I notice when you enter in to that preset (4k 3x1), you will see some vertical lines at first , you need to refresh liveview !
best way , is to turn off "Raw Video" tab press the half shutter button , that will take you a frozen liveview , press trash button to enter ML menu
turn on "Raw Video" tab (should say 4064x590) and press half shutter to refresh and now you are good to go should have clean liveview now .

2784x1160 is the new 5x zoom preset .

I also add 1920x1200 1:1 centered in 3x3 , this one is still having some problems , one of the reg's didn't apply

Code: [Select]

ADTG12 0x1000 0x6 ->0x5 need to do some more work on it .

Have to add several new lines of code to crop_rec to get 3x1 4k to work , basically 3x1 preset structure was missing
took me a little while to wrap my head around what was missing . Once I debug the problem with "ADTG Reg's" not applying
all the rest of the preset will come very quickly , since all the rest of the preset rely on adtg12 reg's working .

Galterius · « **Reply #820 on:** June 21, 2019, 05:21:02 PM »

Quote from: reddeercity on June 21, 2019, 04:39:26 AM

New test build , magiclantern-crop_rec-4k-3x1_2784x1160_1920centered_7.54pm-5D2-eXperimental.2019Jun20.5D2212.zip

Ok in 4k 3x1 Anamorphic , you now longer have to go to the advanced crop_rec menu to set the centered vertical , got that figured out now.
I notice when you enter in to that preset (4k 3x1), you will see some vertical lines at first , you need to refresh liveview !
best way , is to turn off "Raw Video" tab press the half shutter button , that will take you a frozen liveview , press trash button to enter ML menu
turn on "Raw Video" tab (should say 4064x590) and press half shutter to refresh and now you are good to go should have clean liveview now .

2784x1160 is the new 5x zoom preset .

I also add 1920x1200 1:1 centered in 3x3 , this one is still having some problems , one of the reg's didn't apply
Code: [Select]
ADTG12 0x1000 0x6 ->0x5 need to do some more work on it .

Have to add several new lines of code to crop_rec to get 3x1 4k to work , basically 3x1 preset structure was missing
took me a little while to wrap my head around what was missing . Once I debug the problem with "ADTG Reg's" not applying
all the rest of the preset will come very quickly , since all the rest of the preset rely on adtg12 reg's working .

Awesome !! I will try again 4064x590, to see if I still have some issues with vertical lines !

Edit : I just tried what you said (turn off raw, refresh, turn it back on), and I don't have vertical lines ! Black level is still off I think, even in 2784x1160, sadly. (I did not try the new build yet)

reddeercity · « **Reply #821 on:** June 22, 2019, 03:53:38 AM »

Quote from: Galterius on June 21, 2019, 05:21:02 PM

........ Black level is still off I think, even in 2784x1160, sadly. (I did not try the new build yet)

What ? you say black level is off ,but you didn't try the new build ?
Then how do you know the black level is off ?
black level is ok here no problems

Galterius · « **Reply #822 on:** June 22, 2019, 03:43:40 PM »

Quote from: reddeercity on June 22, 2019, 03:53:38 AM

What ? you say black level is off ,but you didn't try the new build ?
Then how do you know the black level is off ?
black level is ok here no problems

Yea sorry thats a bit stupid ^^
Did you have problems with black levels on previous builds ?

OlRivrRat · « **Reply #823 on:** June 23, 2019, 12:59:55 AM »

Since almost none of this HiRes RawVid Fancy Dancing ever seems to work for Me,

I get more & more hesitant to take the time to try it > Finally talked Myself into giving

the latest build a shot on a newly acquired SanDisk ExtremePro 160MB/s 64GB card &

all I can seem to get out of the 3.5k 1:1 centered x5 is about 106 to 125 Frames before

it Quits (MLV Files on card range 432-537MB) & the Cam warms up to 50ºC pretty quick ~

Haven't yet tried to process the MLV to see what I actually got ~

reddeercity · « **Reply #824 on:** June 23, 2019, 04:58:54 AM »

@ OlRivrRat , what is the raw bit rate ?
I have a older Lexar 1066x 32GB (bought it when the cards just when from 1000x to 1066x)
I get up to 15+ seconds 2784x1160 @ 10bit , just did I quick test again and I got 311 frames (12.9 seconds)
then I stop it . I usually set the "Buffer Fill Method" in the Raw Video Tab to "0" , this can make big difference
in write speed , I see an average write speed of 78.1MB/s which peaks to 80 MB/s . You may what to play around with those setting
and see if there's a different (0->4)

News:

Author Topic: 3K/UHD 5D2 Raw development and Other Digic IV Cams (Read 531588 times)