3K/UHD 5D2 Raw development and Other Digic IV Cams

[Macke1]

Thanks! everything is good except + can't zoom in, but will try more

[Igor_Braun]

@maybe your lens is in autofocus mode?

[Macke1]

Thanks! it worked, have some more problems to deal with.. hope I solve it myself, if I don't Im asking again

[reddeercity]

Update 50D , I have 48fps  preset working now just needs a few more tweaks in the preview regs .


1568x700_M17-1958_frame_1.png

Here a short 4 second 10bit .mlv sample from my Google drive 50D_48fps_M17-1958.MLV 275MB
Exported a 4 second  h264 (mp4) from mlv app 1568x700_48fps_M17-1958.mp4

Still working on the bug for 4k 3x1 , I may have a new crop_rec for 50d in a few days with 48p & center crop .

[Wlad81]

Great! Is it from the whole sensor?

Can 50D work on 50fps, even in slightly less resolution?

[masc]

Great! Is it from the whole sensor?

See wikipedia: 50D resolution: 4752 × 3168
Divided by 3x3: 1584 x 1056
Now look again to the resolutions posted by reddeercity.

Can 50D work on 50fps, even in slightly less resolution?

If you just read this thread...
https://www.magiclantern.fm/forum/index.php?topic=19336.msg225537#msg225537

@reddeercity: nice work!

[reddeercity]

50D update: I got a Full Res preset figured out in 5xZoom .
4768x2678 @ 6.5fps I did have it push to 8fps but it was too unstable .
Also I had the full height (4768x3166) @2.5fps (I think it should be able to get around 4-5fps but it was getting hot & lockup) 



Here a 16bit PNG from MLV Producer 50D-6.5fps-M25-2350_00001.png
I had  problems using MLV App , It had bad pink hi-lights that couldn't be fixed.

I also export 2 h264 clips , 1 at the native frame rate (6.5fps) and the 2nd at 24fps ( would gave me 2.0 second from 8 seconds)
I didn't have the right shutter speed so it's a little jerky.
50D-M25-2350-4768x2678-6.5fps.mov
50D-M25-2350-4768x2678_6.5fps-to-24fps.mov

[heder]

Regarding Oc'ing

Ok , loaded it up and it reads the CF Card info
Printed this on the screen


Some useful info at least , this is a Lexar 1066x 32GB
I see the card runs as a PCMCIA & the UDMA in Advance i/o --0x0003  , UDMA6 ? More then likely from the dm-spy logs

This image (upper) is really odd. Look at this image (lower - older cf_acc version) from the 40D.



There are more questions than answers ..

Why are features supported the excat same ? 40D (digic 3) only runs in slow PIO mode, while digic 4 run lighting fast UDMA mode. The digic 4 code says UDMA , and the digic 3 code only says PIO mode.

This is really fishy ..

1. Features supported are the exact same (no can't be !).
2. No mention to UDMA support or enabled in upper image, but digic 4 runs UDMA
3. No mention to PIO mode 5,6 support in lower image, but digic 3 runs 80 ns cycle (pio mode 6)
4. Both upper and lower image, says support PIO 3+4 only.
5. lots of others words are exactly the same.

The Features Enable words are different, but the CFA3.0 or CFA4.0 specifications does not mention anything about these bits, they are reserved. Im talking about word 86. Sorry can't show the specs, they are "copyrighted". so need to use google yourself.

Seems like the CF card controller is a part of the DIGIC-SOC chip itself, and the cf card controllers "words" are just dummy data. Or am I missing something ? it not then this basically means that OC'ing can only be done with CFDMA I/O (0xC0620000 for digic 4) 

[Wlad81]

Do you want to say that it may be possible to hack 40D CF controller to work in PIO 6 mode?

[reddeercity]

Why are features supported the excat same ? 40D (digic 3) only runs in slow PIO mode, while digic 4 run lighting fast UDMA mode.
The digic 4 code says UDMA , and the digic 3 code only says PIO mode.
 .........
 it not then this basically means that OC'ing can only be done with CFDMA I/O (0xC0620000 for digic 4)

Seem 40D uses  (40D = 0xC0700000) instead of 0xC0620000
from here wiki/Register_Map#CFDMA
When i did my investigation i used "digic poke" not sure if it's works on 40D , i could clock down the CF on the 5D2
to 20MB/s and rise it in steps (35,50 etc. ...) but i could not go to udma7  just 6 which is 80MB/s .
I followed this https://www.magiclantern.fm/forum/index.php?topic=12862.msg206010#msg206010
this may help also https://www.magiclantern.fm/forum/index.php?topic=12862.msg206591#msg206591

I guess a1ex did have his 5d2 cf card bus over clocked ,
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207053#msg207053
I guess  /magic-lantern/branch/hdparm can help do this , i have the source but haven't had the time to compile on 5d2/50d yet.
not sure if this helps you 

[Wlad81]

i have the source but haven't had the time to compile on 5d2/50d yet.

Waiting for this.

[reddeercity]

@heder see this https://magiclantern.fandom.com/wiki/Register_Map/40D

Code: [Select]

CF Interface block
 +0x2000 [8]   PIO DATA (used to transfer data in polled mode)
 +0x2001 [8]   Read: Error data, Write: Features
 +0x2002 [8]   Sector count
 +0x2003 [8]   Sector number
 +0x2004 [8]   Cylinder low byte
 +0x2005 [8]   Cylinder high byte
 +0x2006 [8]   Drive/Head
 +0x2007 [8]   Read: Status, Write: CF command
 +0x200E [8]   Read: Alt Status, Write: Device control

Extern DMA Controller Interface (DMA PIO mode - not polled mode) (digic3<->cf controller ?)
 +0x800C [32]  Data register to feed with data (usually 0x2000)
 +0x8010 [32]  unknown, set to 0x100. maybe sector size?
 +0x8014 [32]  unknown, set to PIO_W:0x1E, PIO_R:0x16, DMA:0x00,
 +0x8024 [32]  PIO: CF Command register to use, OR'ed with 0xA000
 +0x8028 [32]  PIO: set to ?
 +0x8030 [32]  PIO: set to ?
 +0x8034 [32]  PIO: set to ? or ? depending on command reg
 +0x8038 [32]  PIO: set to ? or ? depending on command reg
 +0x8040 [32]  Enable interrupt when flag changed? 0, 1, 0x2000000
 +0x8044 [32]  Interrupt reason, write with negated value upon interrupt
 +0x8048 [32]  set to 1, maybe transfer count?

Tranfer mode control registers ? (cf controller <-> cf card ?)
 +0x8100 [32]  0x0 (init) or 0x2 (software OR's with 0xFFFFFFF9 then AND with 0x2)
 +0x8104 [32]  0x4

PIO access timing registers (cf controller <-> cf card ?)
 +0x8108 [32]  access timing PIO mode - fastest mode = 0xC0 ~ 20mb/s
 +0x810C [32]  access timing PIO mode - fastest mode = 0xC0 ~ 20mb/s
 +0x8110 [32]  access timing PIO mode - fastest mode = 0xC0 ~ 20mb/s
 +0x8114 [32]  access timing PIO mode - fastest mode = 0xC0 ~ 20mb/s

Notes:
When setting timing registers 0x8108-0x8114 incorrectly, the FIO_Write function will recover from any
errors and return a error to the caller via return value. The FIO_Read will not, and a call will FIO_read
will stall the camera.
 
UDMA Mode Settings ? (CF identify drive command says UDMA is not supported)
 +0x8200 [32] 0x0     (?)
 +0x8204 [32] 0x0     (?)
 +0x8208 [32] 0x3FFFF (?)
 +0x820C [32] 0x3FFFF (?)
 +0x8210 [32] 0x3FFFF (register is changable)
 +0x8214 [32] 0x3FFFF (register is changable)


SDCON Interface used by CF
0xC0C.....

Some useful info , even for 5d2/50d

[heder]

@heder see this https://magiclantern.fandom.com/wiki/Register_Map/40D
...

I wrote that section on 40D 

I have digged into the CF-1.4/2.0/3.0/4.0 specifications (gosh ... boring) and the words comming out of the CF from both 5D's and 40D controller is crap, and partly dummy data, no mention to UDMA or newer PIO modes. I have'nt disassemble a real canon camera and searched for a CF chip, so I can't for sure say if it is a real chip of SoC part of the digic processor. One thing is however clear,most chips inside the camera are specific for canon cameras,

It feels like :

• CF chip/SoC module developed/bought by Canon - possibly specification CF 2.0
• Since they don't sell it to 3 party, they don't need to follow the entire specification (words = partly dummy)
• I would guess digic 2 ~ CF 2.0
• digic 3 = CF 3.0
• digic 4>= CF 4.0

I will keep digging into this for a while, the real timing values are in the controller area 0xc0628xxx (digic 4) 0xc0708xxx (digic 3).

I will create a new thread soon regarding CF investigation

[reddeercity]

I wrote that section on 40D 

Great Job  looks like it will help me with 5d2/50d

I will create a new thread soon regarding CF investigation

Great ! I'll definitely be involved in that 

[reddeercity]

Working on 50D Full Height in FLV preset , So the max full height is 3176 in the frame below I open up the raw height to 3185
if you crop off the bottom of the image to remove white/gray pixels you get 3176 vertical

1984x3185 in 5xzoom (FYI - Lens Canon EF24-70mm f2.8L @ 63mm =100mm cropfactor1.6 @ F4 ISO800 , shutter .5")


1984x3185_50D_Full_Height_M31-1853_frame_1.png

I also recorded a few second 1984x3185 @ 3.885fps
50D_3.885fps_M31-1853.mp4
and I exported a 24fps version , 3 second from 21
50D_24fps_M31-1853.mp4

I did have full width 4768x3185 @1.96fps but when I tried to save a raw image dump it lockup 
I was able to save the LV dump

[Wlad81]

Working on 50D Full Height in FLV preset

Maybe there's something that I don't understand, but what is the need in this mode? Vertically oriented low frame rate video, when most screens are horizontally oriented? There are only smartphones that are vertically oriented, but what is the need of shooting raw to watch the final result on the small screen?

The very thing is needed about 50D is the compressed 10, 12 and 14 bit raw mode.

[d!g!c]

This is to increase vertical resolution. Current limit is 1080 in 5x crop mode.

[reddeercity]

Ok , I'm able to record Full Liveview on 50D 4768x3176 @ 1.96 fps 10bit raw video (at time of writing , but should be able to get 3.5-4fps)


Here 16bit PNG from MLV Producer , used the "Fast Color" LUT that's provided in MLV Producer 
50D_FLV_Lut_M01-2254_00001.png

Export a 11Mb h264 clip @ 24fps , so 3second from 44 ( the original frame was 1.96fps)
50D_FLV_Lut_24fps_M01-2254.mov

I Still need to adjust the horizontal position on the right hand side to remove the black bar  , then code in to the crop_rec .

[deanziyangyu]

@deanziyangyu
Can you record a short smartphone movie from your ml settings and how you try to enable the Anamorphic mod?

Usually way is (fresh install) :
1. enable crop_re, mlv_rec and if you want mlv_snd module -> restart

2. Raw video (MLV) set to "on" and set shutterspeed to maybe 1/50 (not 1/8000 or so - > maybe don't work)

3. Open liveview, open ml menu, set resolutions to the highest value. Close ml - > simply press half shutter.

4. Press "+", open ml, crop rec is set to "off", now go to 4k Anamorphic. Half shutter.

5. Have fun

Finally got around to take some screenshots. Here is the video: https://imgur.com/SEoLdZS
I pressed "+" at 8 sec and it goes into x5 crop. At 17 sec, ML displays the aforementioned "To use this mode, exit ML menu and press the zoom button (set to x1)".
If I do nothing here and exit to live view to record a clip it looks like this: https://imgur.com/cge21XP To me it looked as if the leftmost and rightmost 1/3 is letterboxed off with the middle having a 1.33x vertical stretch, at the resolution of 2144px.
If I re-enter x1 mode as suggested by ML, a recording would looks like this: https://imgur.com/tKsmqXF This is like the the previous image but without the vertical stretch and at a lower resolution.
At this moment (30 sec) the Canon liveview looks as if the bottom is chopped off and the upper part having a 3x horizontal stretch; A ML screenshot here. https://imgur.com/hMCVdVa You can see a horizontally-stretched laptop.
If i re-enter video menu here I see the 1856px resolution with x3.02 crop: https://imgur.com/HbObcEp and no way of selecting higher resolution than this: https://imgur.com/CPgrb0t

My many attempts at simply enabling 4k anamorphic mode on 5D2 have had zero success. @reddeercity Would you hopefully shed some light here as well? I am using your December 31st build and I have tried many other builds as well.

[reddeercity]

Here a simple picture tutorial https://www.magiclantern.fm/forum/index.php?topic=19336.msg218753#msg218753
2 thing I saw from your screen shots ,
1st: NO "AF" , there can not be any Auto Focusing enabled in Crop_Rec this a limitation with exception
every thing must be manual and that include top mode dial .

2nd: No Auto White balance , & you must have the shutter set to 1/48th !!

The 5D2 must always be in manual mode for ML Crop_Rec .

[deanziyangyu]

Here a simple picture tutorial https://www.magiclantern.fm/forum/index.php?topic=19336.msg218753#msg218753
2 thing I saw from your screen shots ,
1st: NO "AF" , there can not be any Auto Focusing enabled in Crop_Rec this a limitation with exception
every thing must be manual and that include top mode dial .

2nd: No Auto White balance , & you must have the shutter set to 1/48th !!

The 5D2 must always be in manual mode for ML Crop_Rec .

Thank you for your tips. I set Mode dial, WB and focusing to manual, and followed your tutorial again, but wasn't able to get 4096x590 on the menu however I try. I can only get shutter speed at either 1/45 or 1/50 using Expo. override or SS fine tuning. And the weird 3.02 crop and live view white bar still persist.
Although I have no use for it, I can neither make the camera record 48fps.
May be there is something special with my particular 5D2? Are every variant built the same?

[reddeercity]

did you go to the first post on the thread ?
If you did you would see the very big mistake you made
There's a reason Why I make these tutorial as simple as possible

Next setup Liveview preview , This is the most important part without this set correctly it will not work !

Liveveiw needs to be set to ML Gray Scale

[Bender@arsch]

@deanziyangyu

I see your video multiple times to understand what going wrong and finally a can replicate your problem.

There is one simple change you need to set:
Camera on, open Menu (Canon), go to the video settings and change PAL to NTSC.

Finished!

Another little think is, if you are in 5x zoom, don't press "+" again

@Reddeercity
In your newest version you can use preview option auto. It works while recording;) grayscale is not necessary.

[deanziyangyu]

@deanziyangyu

I see your video multiple times to understand what going wrong and finally a can replicate your problem.

There is one simple change you need to set:
Camera on, open Menu (Canon), go to the video settings and change PAL to NTSC.

Finished!

Another little think is, if you are in 5x zoom, don't press "+" again

@Reddeercity
In your newest version you can use preview option auto. It works while recording;) grayscale is not necessary.

Great! Except I am in NTSC mode already... Using PAL mode with crop_rec leads to multiple lockups and battery pulls on my camera.
Either way thank you so much for the help! I might just have to stick with a combination of FF and 3K shots.

[reddeercity]

Ok I made a short video on setup crop_rec.module for the 5D2 and enabling 3x1 4k .
I didn't spend too much time here on the video , it was more to test 1x3 4k on the 50d 
So Yea 1x3 4k (1376x1746 = 4128x1746 @ 24fps) , just did a quick grade & export from MLV App .
Looks good so far now the challenge is to code it without it locking up like 5D2   

                                                                                                                                          Make sure you watch it in 4K 

Back to the setup problem of @ deanziyangyu , follow this it should would work without a problem .
One last time You Can't have Crop Mark Enabled !! this will cause crop_rec not to work