3K/UHD 5D2 Raw development and Other Digic IV Cams

[Galterius]

@reddeercity, its really weird because the black level is set correctly, but its still very green...

https://ibb.co/2347cMx

[Igor_Braun]

@reddeercity, I'm keeping break my brain with configuring my VB Ubuntu for ML, heh.. Now I have all the source files (thank you for helping me on that thread). But still, have a lot of errors.
Look, actually, i started to do it all only for "centering" you 4k anamorphic preset for comparing it with "regular" 1856-790.
The reason I can't do that now is that in 4096 preset the image is being captured from the left top corner so we have distortion and the image is not as sharp as if it would be captured from the center at apertures like 1.4, 2.0, 2.8.
Could you please center the image capturing in the future?

I really DO believe in this awesome 4096x590, and actually believe more than in 1632x1828 @ 23.976 fps (1x3), because in my personal opinion, in this case, the true horizontal resolution is more important (but would like to try 1632 also, of course).

You, David, do really great things...

[reddeercity]

@Igor_Braun , to center the 3x1 4096x590 on cmos[2] in the advanced crop_rec menu
instead of

Code Select Expand

0x1E
select

Code Select Expand

0x6E
that will center it from what I see here

[reddeercity]

@reddeercity, its really weird because the black level is set correctly, but its still very green...

Using MLV App. ? turn off the "Camera Matrix" and use "Don't use Camera Matrix"

Also use the latest built , that one was a test build with preview reg's turn off with 2.6k 
this is the newest build magiclantern-crop_rec2.6k-1x1-4k-3x1-5D2-eXperimental.2019May30.5D2212.zip
Read the first post of this thread for instruction on how to set it up correctly

[reddeercity]

Found a interesting thing today , I can record H264 (.MOV)  when I change the column pinning in 3x3

Code Select Expand

CMOS[2] 0x40E -> 0x10E 
so I end up with 3x1 1920x1080 -> 1920x3240 @ 29.97fps


1920x1080_MVI_8796_00000.png
Original H264 file I started with from my 5D2 1920x1080_MVI_8836.MOV


1920x3240_MVI_8796_00000.png
Adobe After Effect I stretched the vertical 3x , saved as a Black Magic 8bit(M)JPEG 1920x3240_MVI_8796.avi

[Igor_Braun]

@Igor_Braun , to center the 3x1 4096x590 on cmos[2] in the advanced crop_rec menu
instead of

Code Select Expand

0xE
select

Code Select Expand

0x6E
that will center it from what I see here

Oh, thanks, and what about CMOS [1]lo for vertical centering?

[Galterius]

Using MLV App. ? turn off the "Camera Matrix" and use "Don't use Camera Matrix"

Also use the latest built , that one was a test build with preview reg's turn off with 2.6k 
this is the newest build magiclantern-crop_rec2.6k-1x1-4k-3x1-5D2-eXperimental.2019May30.5D2212.zip
Read the first post of this thread for instruction on how to set it up correctly

yea using mlv app

thanks @reddercity ! I will try it and keep you updated ! thanks for your work (my 5d2 is having like a 2nd life  )

[Igor_Braun]

@reddeercity
I think I could find out the settings for center the image vertically too.
There are some black bars at the sides (especially at the right side), but not too much thou.
So, the settings to center 4k 4096x590 preset horizontally and vertically are:


And it need to apply them before switching the crop mode on as you said.
There is a  lot of moire, but personally me I don't really care about that.

Will try to find combination for avoid the black bars

[reddeercity]

@ Igor_Braun , Good job ! yes that better but I think CMOS[1]lo is better at "9" .
the left hand border black bar , never notice it before , that I need to fix with code .
Timer "A" needs to be reflexed a little more , then I need to re-adjust Timer "B" to get 23.976fps .
I fix that later on the weekend at the latest .

[reddeercity]

Ok it seem that I can put the 5D2 in to center crop for H264(AVC) (.MOV) and record it just as if I was in FHD(3x3) 
These are the reg's to make it work , I set canon menu frame rate to 24 ntsc



I dump the image buffer and this what it cam out

Center_Crop_HD-015.png


Center_Crop_LV-015.png

DNG converted to PNG with dcraw

Center_Crop_1880x1247_RAW-015.png

Center_Crop_1880x1247_RAW-015.DNG
Center_Crop_1920x1080_MVI_8842.MOV

Why is this important ? First I can put the camera in 1:1 pixel , same as 3x crop_mode but in 3x3 (FHD) for AVC (H264 .MOV) recording not Raw .
This is the first step in getting a bigger res with native canon compression (h264) and hopefully that will help get (M)JPEG (.AVI) working.
Next is how to increase the horizontal & vertical (well mainly horizontal) so we can equal the raw resolutions , for UDH/4k 8bit AVC or (M)JPEG .   
Two problem I can see right off the start ,
1st ; change the HD Buffer for increase resolutions (if this can be done , we should have full color realtime preview in extended resolutions as it limited to HD size)
2nd ; the H264/AVC encoder is preset to 1920x1080 in the rom , even thou it resizes the h264 will always be 1080p unless we can get control over it .
This is where (M)JPEG .AVI could save us , there no preset size from what I can see in the rom , so theoretically we should be able to record the extended raw resolutions
which could be great benefit for the low speed write camera's plus we can start to use picture style profiles again like Technicolor Cinestyle but in a 422 color space .
Not sure how all this could be implemented , maybe a custom H264 module or a (M)JPEG Module ?
Maybe a1ex has a idea for that     

[banertop]

man,
that would be nice

better h264!

good work!

Thanks again

[Galterius]

@reddeercity I did what you told me and the image is very nice now ! Thanks !

https://www.youtube.com/watch?v=huFwnUV46Po

what I did : import in MLV app, change to "don't use camera matrix" and choose the "film" preset, export in CinemaDNG Uncompressed, import in AE, color grading with Camera Raw, dynamic link with Pr and export in 4k so youtube doesn't destroy it ^^$

It still looks a liiiitle bit too green but thats okay

[masc]

what I did : import in MLV app, change to "don't use camera matrix" and choose the "film" preset, export in CinemaDNG Uncompressed, import in AE, color grading with Camera Raw, dynamic link with Pr and export in 4k so youtube doesn't destroy it ^^$

That makes no sense: if you export in MLVApp to dng, all settings (except RAW corrections) will get lost, because dng is a RAW format. Camera Matrix and Profile Film will only be visible in a non-RAW format like ProRes, H264, ...

[reddeercity]

Playing around with 3x3 H264 & different pinning mode , set the cam same as in post #784 but left the pinning at default  CMOS[2] 0x40E
Ended up with a full view (3x1 binning) H264 1920x1080 -> 5568x1080 , Yea Anamorphic .MOV ! thou it's not as good as in 5x Zoom (3x Crop_mode)
the edges are jagged , but it does work , so any Reg other then the raw buffer ones are applied to H264 , as long as there's a clean canon preview
the encoder will save it to h264 .

DNG from image dump , converted to png with dcraw

RAW-017_1880x1247.png


RAW-017_5640x1247.png

Nice thing about 3x1 I can use the 2:1 Anamorphic preview option , thou it's should be 3:1 ,
I'll have to look in to the code and see if I can add a 3:1 preset view mode



Here the original 5D2 H264 MVI_8849.MOV file .
H264 frame from Adobe A.E.

5568x1080_MVI_8849_1_00000.png

Here the export from A.E. , Blackmagic 8bit (M)JPEG 5568x1080_24fps_MVI_8849.avi
Like I said the edges are jagged , but the real test was if I could record 3x1 in AVC(H264) and I can so if we could record AVC in 5xZoom (3x Crop_mode)
in 3x1 we would have a very clean aliasing and moiré free H264 4k Anamorphic file 

[Galterius]

That makes no sense: if you export in MLVApp to dng, all settings (except RAW corrections) will get lost, because dng is a RAW format. Camera Matrix and Profile Film will only be visible in a non-RAW format like ProRes, H264, ...

yea thats true... Then I will try with Prores
The problem is that when using "don't use camera matric" colors are really flat, and its hard to get good skin tones

[Galterius]

By the way I will try the 4096x590 anamorphic build

[Galterius]

So, I just did a comparison test between the build from May 24 (preview reg's modified) and the one from May 30 that @reddeercity sent me to see if there is a difference in colors.

resolution : 2560x1200

So here is a google drive folder with 2 folders (one folder for each build) : each one contains an mlv file, the ml setting file and a png exported with mlv app, so you can tell if there is a difference.

https://drive.google.com/drive/folders/1j8EoJQy6sFCdp9x_SbhC7moSpwwfmANY?usp=sharing


I did that to see if there was a problem with the black levels, but the difference is very small, maybe the build from May 30 is a little bit less green, I don't know...

I don't really know if its useful, but maybe ^^

I will now test the 4096x590 build

[Galterius]

So I just tried 4096x590 : it works, but when its enabled, I can't change the shutter speed, iso...
i also get a lot of aliasing and moiré a big vertical line on the right, and in some clips vertical lines, weird pixels... and overall it looks really unsharp which is weird...


this is prores 422 exported from MLV app : https://drive.google.com/file/d/1p8rOlaJ0X28Heth8Bsyxcnq-YYAE9vZQ/view?usp=sharing



4096x590 test

[ilia3101]

yea thats true... Then I will try with Prores
The problem is that when using "don't use camera matric" colors are really flat, and its hard to get good skin tones

I don't recommend using the "don't use camera matrix" option unless colours are clipping, it's just less accurate. Also you showed a different shot, processed with other software, to show the issues were fixed, what happened with the first one though?

The only reason "don't use camera matrix" might seem to make the "very green" issue go away, is the fact that it desaturates the image, but it will become visible as soon as you add saturation.

I think it's better for the main developers here to fix the black level issues in their magic lantern branch® than deny it or blame MLV App.

[Galterius]

I don't recommend using the "don't use camera matrix" option unless colours are clipping, it's just less accurate. Also you showed a different shot, processed with other software, to show the issues were fixed, what happened with the first one though?

The only reason "don't use camera matrix" might seem to make the "very green" issue go away, is the fact that it desaturates the image, but it will become visible as soon as you add saturation.

I think it's better for the main developers here to fix the black level issues in their magic lantern branch® than deny it or blame MLV App.

yea true, i didn't know that "don't use camera matrix" just desaturated the colors, but there is definitely an issue with the black level. We can almost solve it with color grading but the problem is still here.

The first shot I showed here is still really green...

Sorry, I am still very new to this "world" so I don't know a lot of things   

[Bender@arsch]

@Galterius
That Problem is not new for me, i Posted this problem many days before, and so far there is no solution.

The main problem is that it behaves intermittently, so it is not possible to edit the RAW settings easily. And this probelm is only on all 2.6k builds.

But I found out that it is reducible, if one takes a static video (camera + tripod + no movement). If you move the Camera (especially between hard contrasts) the greenish blacklevel cames out.

maybe someone understands why?

[Bender@arsch]

Here is a short test with the Anamorphic build.

There is much aliasing but better crop-factor than 3K, 2.6K build. And with 14bit setting (and global draw alow) I have a full-low-frame-live-preview. So it is useful.

this is not the centered version.

After cutting the blackbar I have 4065 pixels. I also have a few stripes at the beginning, but after waiting 10 seconds the problem disappears.

[Galterius]

@Galterius
That Problem is not new for me, i Posted this problem many days before, and so far there is no solution.

The main problem is that it behaves intermittently, so it is not possible to edit the RAW settings easily. And this probelm is only on all 2.6k builds.

But I found out that it is reducible, if one takes a static video (camera + tripod + no movement). If you move the Camera (especially between hard contrasts) the greenish blacklevel cames out.

maybe someone understands why?

ah thats sad... We will wait

[Galterius]

By the way I did some tests in 4096x590 this morning but I have a lot of vertical lines in my shots, in the 4 I did... Maybe it has something to with the shutter speed, I had tp set it on 1/40 on the camera, and on the screen it was written 1/42 (this is the closest to 1/48 I can get)


Bender@arsch which shutter speed did you choose ? An you recorded in 14 bit ? What difference does it make ? IDoes it turns "on" some sort of live view while recording with this resolution ?

[reddeercity]

Ok playing around with 2784x1160 @ 23.976 Real Time liveview preview , I think I figured out the issue with black level in 2.6K
and applied then fix to 2784x1160 . Now I'm having issue with the cmos[1] vertical offset so you will have to use the advanced
crop_rec menu and adjust cmos[1]lo to 18 & cmos[1]hi to 62


issue seems to be with the "cmos_new[1] PACK12(9+2 ,42+1)"
I'm not sure what "pack12" comes from , if I could understand what the number are calling (some kind of centering) for vertical offset
if I disable the pack12 the image cleans up but it pushes up with a white bar at the bottom , so you have to fine adjust it with the advanced
crop_rec menu like the image above until I figure out how to code it centered vertically .
So remember , expect problems , lockup's or anything odd as it could happen at any time !
This build is for test proposes only !! I really looking for any black level changes while recording .
magiclantern-crop_rec-2784x1160_24p_11.39pm-5D2-eXperimental.2019Jun14.5D2212.zip

Edit: so if you add the cmos[1] lo18 & hi62  together you get 80 now that's the same as the white area from the image height
Original vertical is 1160 if you crop off the white area it's 1080 ,  so does this mean "pack12etc. ...." is just the vertical centering .
So tomorrow I'll change the "pack12 (9+2 ,42+1)" to "pack12 (18+62)" and see what happens