3K/UHD 5D2 Raw development and Other Digic IV Cams

Started by reddeercity, April 06, 2017, 12:22:27 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.


it's not a good workflow, but for curiosity's sake, you might try rawtherapee.

the latest version is quite powerful with dual demosaicing capability.

it would be an interesting test, even just on a single frame...


Is it correct to say you are working a 48fps release, what will that resolution be and is it Squeeze or Unsqueezed? Also is the most stable 10/12bit the Oct10 release in MagicLantern Experimental Nightly Build?

RawTherapee sound interesting, I use MLV App but I'm using Camera Raw in AfterEffects the most. I just installed RawTherapee, what is the workflow you use with Magic Lantern? I'm filming a Feature Film exclusively with Magic Lantern, I want the best option to process Magic Lantern footage,  so my Question is what's you favorite stable firmware, and if you were risky and brave to film a feature with Magic Lantern what would be your workflow? Please anyone feel free to answer above questions, I'm very competent but I love to have options and keep my options open so every time I ask a question on this forum its for real productivity. Sometimes I ask question on this forum then it gets loss in the clouds and ignore. Thanks for Any Advice or Guidance
Canon 5D Mark II


Can 5D2 be set to mv720p 50 or 60 fps? If so it should be possible to set 100c to 3x3 binning and give mv1080p 50 or 60 fps with a little reduced height.


There's no real 720p 50/60 mode on 5d2 , it's come about though reg's , actually a1ex figured it out
I'm just fine tuning it ,It's really FHD with squeezed reduced height 1888x704 @48fps I can push it to 750 so it's 3/5
I can do 3x3 with hi-frame rate 72+ @ 1280x720 (raw size) not vertical squeezed.
I did the squeezed version to get full height that's centered. Never really tried to do a un-squeezed FHD at reduced height
I don't think I'll get 48fps more like 40 if that , the sensor it too slow I think for that 5d2 is only 96MP/s where the 5d3 is 192MP/s
and that's the big difference between D4 & D5 . The 5d3 has 8 channels off the sensor where the 5d2 has only 4 channels .


So how about 10-12bit 4096x1770 @ 23.976 fps (A.R. 2.31) in 1:1 FHD sound ?
I got a clean 4096x590(squeezed 3x) version I'm thinking of add to the crop_rec
I can record 10bit continuously @ 69MB/s  :D and MLVApp can process then easily
There still a little aliasing , but not that bad you really have to push it to produce it.
Never notice any moiré pattern at all , In this mode the sensor is 1:1 pixel horizontally hence 4096
and the vertical is line skipping every third line from what I can see so 1x3.

4096x1770-M29-0050_frame_1.dng un-sqeezed from mlvapp 1.4v
4104x591-RAW-033.DNG squeezed , from the image dump
h264 clip 4096x1770-M29-0050.mov 70Mb

Pretty much works flawlessly , not issue at all I can see only thing I don't like is there more rolling shutter then I would like .


5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109


Quote from: reddeercity on December 28, 2018, 04:20:10 AM
So I would really be happy with either MLVApp 1.4v Or MLVProducer with a small lean towards mlvProducer .
As far as A.E. is concerned , It was the least desirable image with aliasing & moiré patterns .
I again played a bit around with some freestyle code and your 48fps MLV. The pink in on the chart looks like the green channel is clipped from the low side, not from the high side. So I implemented just for testing a stupid algorithm which reconstructs this part of the green channel. Then I switched to IGV debayer and this is what you'll get then (+ Sharpen=50 to see even more detail):

The problem is: this is not at all scientific, so we'll kill some areas color with that...

Code raw_processing.c (first the old highlight reconstruction, second the "lowlight reconstruction"):
                /* Check if its the highest green value possible */
                if (tmp1 == processing->highest_green)
                    pix[1] = (pix[0] + pix[2]) / 2;
                /* reconstruct low green */
                if (tmp1 < 15000)
                    if( pix[1] < 1.1*pix[0] && pix[1] < pix[2] )
                        pix[1] = (pix[0] + pix[2]) / 2;

Quote from: reddeercity on December 29, 2018, 09:48:29 AM
So how about 10-12bit 4096x1770 @ 23.976 fps (A.R. 2.31) in 1:1 FHD sound ?
Great!  8) Wanna have! ;)
5D3.113 | EOSM.202


i mean simply amazing - for 4096x1770 @ 23.976 fps....
just amazing


Thanks , did a short clip test in a real world situation (outdoors , shot a skyline @ sunset) .
Didn't like the artifact and saw some aliasing that look not good .
Timers need to be reworked to reduce artifacts before I could use it .
So what did I do to get this , I basically stop the vertical columns (every third one) CMOS[1] 0x40=>0xe
if you do the math 5632/3=1877.333 so round it off the same
as the silent picture or the liveview raw image dump 1880x1248 . By increasing the horizontal to 4096 I still have line skipping
vertically (every 3rd line) , but if this was not so I wouldn't be able to record 4k squeezed . So we can call this 4K with trickery
Or 4K line skipping vertically , need to think able this some more .

On the other hand seeing how far I can push Hi-Frame rate squeezed -1280x432 (1280x720 un-squeezed).
Seems I can get to 83.815fps pretty cleanly , thou I can push it to 85.881fps but the image break ups badly , (image moves sideways) 
More then likely if I reduce horizontal from 1280 to less then 1000 I'm sure to get more but this doesn't interest me below 1280x720
So I should be able to give the 5d2 82.5fps safely up to 83.88fps   

Edit: Update -- I push the 4k compressed vertical to 29.970 FPS ,
So that's 4096x590 @29.97 (Un-compressed 4096x1776 @ 29.97 fps)

Unfortunately it's not continuous at 10bit 29.97fps , I can get around 5000 frames (2.5min) where at 23.976 is continuous .
So that's a plus  :) , it's making this more attractive (even with it's flaws) a little slow-mo in 4k even if it's line skipping



Reddeercity, you do amazing things... look, after I wrote you about the zoom (magnification) jumping out automaticaly, it stopped to do so. Now it works properly. I actually don't know what happened (mystic). So, everything is fine now, but it have been jumping out for almost 7 years till that moment... hm.... So if it happen again I'll show you. Thanks


Quote from: reddeercity on December 29, 2018, 08:48:56 AM
I can do 3x3 with hi-frame rate 72+ @ 1280x720 (raw size) not vertical squeezed.

You can do 1280x720 unsqueezed, but can you not increase horizontal resolution? I think it's worth it even if vertical resolution or framerate has to be reduced. An 1880x700 48fps preset would be really good.

Also does anyone else with a 5D2 find that when shooting in 2880x1080 the rolling shutter is really extreme?

It is not possible to get usable footage handheld at all. I'm asking because my camera is together by only one screw after repair, so it may just be that.


I check in the Frame over ride menu in the advance tab and compared to the other mode ,
Normal crop_mode at 23.976 & 30 plus in FHD at 23.976 (frame override)

2880x1080 crop preset  33.5 -- rolling shutter

Normal 3x crop_mode 2144x1074 @ 23.976 frame override -- 25.6 rolling shutter

FHD 1:1 1856x1044 @ 23.976 frame override -- 23.8 rolling shutter (best)

So yes this more rolling shutter ,
I've being trying to reduce this in normal FHD (1856x1044) I have rolling shutter down to 22.8
Looking to be around 15 but sensor too slow I bet .

FHD 1:1 1856x1044 Low rolling shutter

Have you try a higher shutter speed  e.g. 1/60th , or  1/90 this should help a little (never tried)
the reason I say 1/60th because timer A is set to 31fps  and timer B just slows it down to 23.976
This is needed to extend the resolution , even in 4k line skipping  (4096x590) to get 23.976 timer A
was pushed pretty high (58 fps) then timer B reduce it to 23.976 .

Unless the timer over head can be reduced (remember a1ex did some test) he was able to reduce more rolling shutter and here too
I think on the 5d3 the rolling shutter is around 18 I think .
So I hope to have less rolling shutter , but that work in process .


Pushing to full width (5632) with the vertical compressed so 5632x416 @ 23.99fps = 5632x1248 @23.99fps
Works good , but the full width I can only get 1248 vertical which give an A.R. of 4.5:1  :o
If you are wondering why I can't get a higher vertical resolution , from what I can see the width is
the main factor , as I reduce the horizontal raw size e.g. 4096 I get increased vertical e.g. 1776 etc. ...
So the width & height is in direct proportional to it's self , if I was in 3x crop_mode I wouldn't have this limit .



It's just tooooo narrow , I would have to get at least 1877 (3:1 A.R.) or 2440 (2.30:1 A.R.) vertical .
So I'll just keep 4096x590 (4096x1776) and develop it further  That seems to be the Sweet Spot for 5d2  :))

Happy NewYear everyone !!
It should be only a few more weeks before my crop_rec module is ready for testing .


Quote from: reddeercity on January 01, 2019, 06:24:17 AM
Unless the timer over head can be reduced (remember a1ex did some test) he was able to reduce more rolling shutter and here too
I think on the 5d3 the rolling shutter is around 18 I think .
So I hope to have less rolling shutter , but that work in process .

In 1080p 3x3, the minimum achievable rolling shutter would be...

Quote from: a1ex on November 22, 2018, 08:16:08 AM
the minimum timer A value in 1080p is 554

... 554 / 24 MHz = 23.08 μs/line.

You can already get this with FPS override, by manually changing timer A manually to 554 (in the Advanced menu).

Values without FPS override (i.e. with stock Canon firmware):
- 1080p24/30: 576 / 24 MHz = 23.8 μs/line
- 1080p25: 600 / 24 MHz = 25 μs/line

Theoretical speed: 5D2 reads out 4 columns at a time (i.e. 4 analog channels), with a pixel clock of 24 MHz; that would give 96 MPix/second. You just can't get more than that without overclocking (no idea whether this frequency is adjustable by software or not).

That means, the absolute minimum, assuming one can find a way to somehow get rid of the horizontal overheads (which is likely impossible), would be 1880 / 4 / 24 MHz = 19.58 μs/line. Keeping the left OB area (C0F06084: 0x10036,  C0F06088: 0x4f40432) would require 0x432 * 2 / 4 / 24 MHz = 22.38 μs/line.

The remaining (unexplained) overhead is a whopping 23.08 - 22.38 = 0.7 μs/line. Even if that can be reduced somehow, I'm pretty sure it's not worth the effort. You simply won't get significant rolling shutter improvements without overclocking the sensor (or without reducing the horizontal resolution).

1100D, on the other hand, is an interesting beast:

Quote from: a1ex on November 27, 2018, 09:09:31 AM
720p25: A = 1000, B = 1280, readout size 1496 x 967 (this includes black borders)
720p30: A = 960, B = 1112, same readout size
Timer A can be pushed to 872 in both modes.

It runs at 32 MHz x 4 channels (i.e. pixel clock is the same 700D & co.), but for some reason, it has a huge overhead. In regular (24/25/30p) movie mode, C0F06084/88 is set to 0x200c3/0x3c903af, i.e. xmax = 1886 (likely rounded to 1888), out of which 390 pixels are cropped from the left side. ML still has to crop 68 pixels of left OB; that gives 1428 active horizontal pixels, 458 (!) for OB width and 2 pixels for rounding (guess); these require "only" 14.75 μs/line. The unexplained overhead would be 12.5 μs/line, or exactly 400 clock ticks. This one is worth fine-tuning, if you ask me.

- 720p24/30: 960 / 32 = 30 μs/line
- 720p25: 1000 / 32 = 31.25 μs/line
- FPS override: 872 / 32 = 27.25 μs/line
- theoretical value: (872 - 400) / 32 = 14.75 μs/line! (without altering OB areas)

For a 16:9 frame, which would be just 1428x804, it would require only 11.86 ms!

Assuming one can somehow drop 400 pixels from the left OB, that would free another 100 clock ticks, i.e. 11.63 μs/line, or 9.35 ms for a 16:9 LiveView frame.

5D3 also operates at 24 MHz, but has 8 channels. In 1080p 3x3, timer A values are (see here):

- 24/30p: 440
- 25p: 480
- minimum value: 398

That gives 18.3 and 20 μs/line with Canon firmware, and 16.58 μs/line with FPS override.

C0F06800/C0F06804 are set to 0x10017/0x528011b; that would require 0x11b * 8 / 8 / 24 MHz = 11.8 μs/line (unexplained overhead: 4.8 μs/line, i.e. 115 clock ticks). Optical black area: Canon crops away 0x17 * 8 = 184 pixels, ML skips 146 + 2, active area width is 1932, total 2264 = 0x11b * 8.

That way, should we find out how to remove that overhead, a 16:9 frame (1920x1080) would require only 12.75 ms.

If we could also get rid of 300 OB pixels (out of 332), that would cut about 1.56 microseconds per line.

Of course, these are just theoretical values; I don't know how to reduce these overheads, I just tried to estimate whether it's worth trying to figure them out, or not. On 5D2, the potential improvement would be very small (the sensor is already used at close to its full speed in LiveView). On 5D3 and 1100D... it's worth taking a closer look.


Thanks, much clearer now I'm starting understand this . Yes I see where the 5d3 would be good there.
Now I thinking about 5D4 since this happened  :D
I'm keeping my eye's open for a good deal on one , starting to see a few used ones out for sale now .
After I'm done with the 5d2 development , I looking forward to the 5d4.

Now about the 5d2 , I got a new preset UHD Vertical Compressed 3840x624 @ 23.976 (3840x1872 2.0:1 A.R.)
I'm likening this a little more then 4k for the better vertical height , it's a hard to get continuous @ 10bit (more height)
I will keep both presets (3840 & 4096) , the rest will be from the 3x crop mode .

UHD 3840x1872



Can't wait... =))

"After I'm done with the 5d2 development , I looking forward to the 5d4."
This also sounds inspiring.


Quote from: reddeercity on December 27, 2018, 07:43:23 AM
You should see the same list of reg's I posted not just cmos reg's  , that tells me either
didn't enabled the "ENGIO Reg's" in the advanced tab of adtg_gui.mo or you didn't refresh liveview
e.g. load a h264 of review a photo , this a must or nothing will work and you will not get access to the hidden reg's
Sorry, forgot to say that 50D doesn't have movie playback.
Already enabled "ENGIO Regs" and apparently I can find all relevant regs including adtg12[100c] that were mentioned here, just need to switch category to avoid headache after switching to LV end 3X mode :)

Anyway after some days of swpping batteries (damn LV drawn them quickly...) and trying to match full width, I'm still stuck on solving some problems:
1. There is a pink bar at the bottom of the frame when I increase reg 0xc0f06088 (more resolution, more pink area), no matther how TImer A is modified
2. Increasing width will make room for rigth side of framebut will result also in black bar... In other words image is not centered
    Expected to fix with CMOS[2] but no apparently usefull (Need to redo some testing about) and with 0xc0f06084 (small diff from default appears so leave room for left data but then increasing more will only result in moving image to rigth)
3. a couple of gray/green vertical line in the second half of the frame

First DNG is default ML preset, second DNG is attempt of full width:

0xc0f06008 : 0x27b04f5 (from 0x27b27b)
0xc0f06084 : 0x1004b (from 0x1004a)
0xc0f06088 : 0x45209bb (from 0x450452)

Not completely sure about photo mode register's values for FUll Resolution LiveView:

0xc0f06008 : 0x4f504f5
0xc0f06014 : 0xc9c
0xc0f06084 : 0x1004b (from default 1004a)
0xc0f06088 : 0xc9d09bb

They were retrived first time taking FRSP and watch previus values of regs in adtg_gui and then after taking a regular picture while not in LV.
Timing values and resolution appears more higher than needed, is this normal?

What is the relation between Timer A/B regarding FPS and rolling shutter?
I read somewhere on forum one time and can't find anymore (Also still need to finish read half of this thread)

Here is a spreadsheet where I'm trying to summarize what tested and trying not to forget their outcomes


Great Job !!
That expected , did you try and rec a .mlv file ? Looks like a image dump
more then likely the pink lower bar with not be in the raw video
It was the same for the 5D2 , I had the same thing on full width (5632) & 4k (4096)

CMOS[2] should center it , check photo mode and see what it is.
On the 5d2 CMOS[2]0xe centers for full width (5632) and in
default 3xcrop_mode CMOS[2]0x10e centers it (2144) but in 4096(4k)
CMOS[2]0x6e centers it , I check to see by refreshing liveview with the half shutter button 
Seems the image is always right justified , and it can only be moved to the left from what I seen on 5d2 .

That all for now , I'll answer the rest of your questions later tonight
in the middle of some 4k line skipping experiments in 5x zoom (3xcrop_mode) . :)


Hi Reddeercity,

The squeezed mode is interesting. I'm thinking about use it in fullhd+ like (1920 x 360 up to 2400 x 432) at a high frame rate like 72 to optimise aliasing.
With an offset of 1 line in height every 1/72 second, we can reconstruct with dual-iso a good fullhd+ (1920x1080 up to 2400x1296 fps24 ) ?
I know we need to adapt mlv_dump or mlv_app to take in account the height shift but do you think it's doable in crop.c ?
Here is a schema to explain my thoughts :


Height shifts and metadata area already under control through metadata and taken care of in Mlv App. You reach the metadata parts in crop_rec.c in here:
static unsigned int raw_info_update_cbr(unsigned int unused)


Whaou , thanks Danne for the quick and precise answer !


Maybe I'm missing the point, but I don't see any connection between the previous 3 replies. Metadata has nothing to do with benoit's diagram; maybe Danne's reply was meant for some other post?

One has to control the phase of the line skipping sequence (i.e. from a 3-line group, which line gets captured and which two lines get skipped). This is not under our control at the time of writing. If it is controllable by software, it would be from one of the ADTG registers.

To find it, one requires a way to change ADTG registers without moving the camera (such as, some sort of USB remote interface, or some register brute-forcing script running unattended). Why? You would have to identify an image shift of exactly 1/3 pixels vertically (after the line skipping).

In the mean time, the easiest way to use all physical pixels on the sensor (including 5D2) is to read every single line, and bin every 3 columns (that is, full height 1x3 readout).

Simulation to show the difference between 3x1 (with line skipping) and 1x3 (with column binning):
Quote from: a1ex on January 01, 2019, 05:31:20 PM
binning-modes.html (18.5 MB)

However, recovering the full resolution from a 1x3 readout is not exactly straightforward.


Yes, 1x3 will be the way so I assume 5D mark III and maybe 700D would be the better start off cams to work on as they can can expand height with less restrictions than eosm and 100D for instance.
My metadata reply was simply referring to info passed into mlv through crop_rec.c which Mlv app follows.