crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

[Danne]

Added support for eosm. Generic code confirmed working:
https://bitbucket.org/Dannephoto/magic-lantern/commits/a9eff349f8cd79669ef307d56c88d69987c454bf

in crop_rec menu:
    "OFF"
    "2.5K 2520x1304"
    "3K 3072x1304"
    "4K 4032x2558"
    "3x3 720p"


Build download:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/magiclantern-Nightly.2018Jul01.EOSM202_2K-3K-4K.zip

[theBilalFakhouri]

@Ballinger

This is a great accomplishment in terms of unlocking these higher resolutions of course.  Realistically, from my own personal experience with the 700D and a class 3 write speed Sandrisk Extreme Plus SD card, everything is severely limited unless one overclocks the SD bus via Danne's Crop 4K build.  I can only yield maybe about 3 seconds of footage before frame skipping, even at the 2.5K preset.  Does anyone have any different experience? 

No , it's just same thing for all small cameras --> The higher resolutions including 2520x1384 needs at least 80MB/s write speed (a bit more or less depending on the scene) with sd_uhs @160MHz (68MB/s) we can get ~55MB/s effective write speed in 23.976. You can see in 4K preset 4096x2560 @9.5fps with sd_uhs the write speed ~62MB/s (Drop down the resolution to 3520x1980 16:9 or a bit less from "4K 1:1" preset you will see continuous recording @ 9.5fps and write speed 62MB/s)

The solution is to overclock sd_uhs up to 208MHz (104MB/s) and as I can guess it will make effective write speed at least 80MB/s or more.

And of course lower bit-depths like 10-bit lossless will help.

BTW -- you can get continuous 2520x1080 @23.976 using FPS override without crop_rec at 11-bit lossless and using sd_uhs.

EDIT: You can use also FPS override with crop_rec to get continuous recording at lower FPS (e.g 2520x1384 @ 18FPS & 4096X2560 @ 6FPS for time-lapses).

[Levas]

There seems to be a limit for the amount of adtg registers you can override in crop_rec.c
When I want to override more the 10 adtg registers in one preset, everything after 'adtg_new[9]' doesn't get override.

Where can I change the code in crop_rec.c , so it accepts more then 10 adtg registers ?
I want to go further
adtg_new[10]
adtg_new[11]
etc.

[theBilalFakhouri]

@Levas

See the line 727 at this commit .

[Levas]

Thanks, exactly the line of code I was looking for

[Levas]

@Bilal, Levas, Danne, mk11174,

I've just reworked the PowerSaveTiming and ReadOutTiming registers in a generic way, according to this description. That means, you no longer have to hardcode these in your presets - they should work out of the box if you override the other registers like usual. Can you test on your models? Didn't do the integration yet (working on that); just adapt this commit to your local branches.

Can't get it to work here with my version of Crop_rec, tried to alter the crop_rec version of 'Crop_rec_4K_mlv_snd' branch, but got loads of compiling errors.
Maybe if there is a version of crop_rec.c from the 'Crop_rec_4K_mlv_snd' branch with more camera's in it, probably then I could succeed ?
Or a template version of it, with empty presets for the 6d, but code for where to put the presets available.

At the moment I'm using the vertical registers as follows ?
8172 = same value as first three characters of 6804
8178 = same value as first three characters of 6804
81F8 = same value as first three characters of 6804 - 1

8173, 8179 and 81f9 I have set to '0'.

713c = same value as first three characters of 6804
7150 = same value as first three characters of 6804 -1

Not sure if this is logical or correct, but works for me.

Not sure what values to use for registers 8173, 8179 and 81f9.

[Levas]

@Alex, I'm wondering how the digital gain is applied for 12 bit lossless preset ?

I see some references in MLV_lite to putting a digital gain value in bpp_digital_gain.
But I don't see where this translates to settings in register(s).
Which register(s) is/are used for digital gain ?

Like to try what happens if the digital gain is hardcoded in the crop_rec preset.
(I know liveview get's darker, but just want to try and test it)
And do I need to correct white level, and is there a way to do this in Crop_rec ?

Was thinking, maybe we could use crop_rec more as a total filming preset option.
By putting not only resolution and fps in the preset, but also recording type, like 12 bit lossless.

Already have tried hardcoding some lower analog gain values in the crop_rec preset (8882...to...8888 registers), for some small improvement in dynamic range.
But I ended up with pink highlights...does that mean I went to far, or does it mean I have also adjust the white level somewhere ?
I determined the values for analog gain by overexposing the camera and setting the registers back to where it just hit the edge of the magic lantern raw histogram.

[a1ex]

Look at Danne's version: https://bitbucket.org/Dannephoto/magic-lantern/src/crop_rec_4k_mlv_lite_snd_sd_uhs_HDR_croprec/modules/crop_rec/crop_rec.c

Drop your values for 0x8172, 0x8179 etc and paste this block somewhere in your adtg_hook. Make sure adtg_new's size is at least 13.

Code Select Expand


            {
                /* assuming FPS timer B was overridden before this */
                int fps_timer_b = (shamem_read(0xC0F06014) & 0xFFFF) + 1;
                int readout_end = shamem_read(0xC0F06804) >> 16;    /* fixme: D5 only */

                /* PowerSaveTiming registers */
                /* after readout is finished, we can turn off the sensor until the next frame */
                /* we could also set these to 0; it will work, but the sensor will run a bit hotter */
                /* to be tested to find out exactly how much */
                adtg_new[4]  = (struct adtg_new) {6, 0x8172, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (6D/700D) */
                adtg_new[5]  = (struct adtg_new) {6, 0x8178, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3/6D/700D) */
                adtg_new[6]  = (struct adtg_new) {6, 0x8196, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3) */

                adtg_new[7]  = (struct adtg_new) {6, 0x8173, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (6D/700D) */
                adtg_new[8]  = (struct adtg_new) {6, 0x8179, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3/6D/700D) */
                adtg_new[9]  = (struct adtg_new) {6, 0x8197, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3) */

                adtg_new[10] = (struct adtg_new) {6, 0x82B6, nrzi_encode(readout_end - 1) }; /* PowerSaveTiming ON? (700D); 2 units below the "ON" timing from above */

                /* ReadOutTiming registers */
                /* these shouldn't be 0, as they affect the image */
                adtg_new[11] = (struct adtg_new) {6, 0x82F8, nrzi_encode(readout_end + 1) }; /* ReadOutTiming */
                adtg_new[12] = (struct adtg_new) {6, 0x82F9, nrzi_encode(fps_timer_b - 1) }; /* ReadOutTiming end? */
                break;
            }


Guess for compiling errors: if you declare variables within a case label, you need { curly brackets }.

Digital gain is applied in raw.c; it's mlv_lite that's asking for different gains.

For the dynamic range stuff, once you get pink highlights, you have reached the limit for that gain you are adjusting (if you lower it further than that, you'll no longer get additional highlight). The idea is to lower these gains just as long as you keep getting additional detail. Once your whites drop before your declared white level, you probably went too far; in this case, you can try adjusting other gains and find a sweet spot. I should probably implement some sort of auto-tuning procedure.

[Levas]

@Alex, the above works!

Could even remove the case crop preset thing, since this works for all crop presets, no need to define these registers per crop_presets
So cleans up a lot of code 

[Levas]

Digital gain is applied in raw.c; it's mlv_lite that's asking for different gains.

Looked in raw.c, but only could find the register c0f0 8030, but that one only darkens or brightens the liveview, doesn't affect the raw stream ?
For lower lossless bit depths, the raw stream must be darkened too, right ?

[a1ex]

Good point, will do that.

Some older questions regarding video mode detection: identifying it from C0F06804 is not always possible. A comment from my not-yet-committed code:

Code Select Expand


/* properties are fired AFTER the new video mode is fully up and running
* to apply our presets, we need to know the video more DURING the switch
* we'll peek into the PathDriveMode structure for that
*
* Example: x10 -> x1 on 5D3
* This sequence cannot be identified just by looking at C0F06804;
* some ADTG registers that we need to overide are configured before that.
*
* GuiMainTask: GUICMD_PRESS_BUTTON_SOMETHING(0x0)
*     CtrlSrv: DlgLiveView.c PRESS_TELE_MAG_BUTTON KeyRepeat[0]
*         Gmt: gmtModeChange
*         Evf: evfModeChangeRequest(4)
*         Evf: PATH_SelectPathDriveMode S:0 Z:10000 R:0 DZ:0 SM:1
*         (lots of stuff going on)
*         Evf: evfModeChangeComplete
*         (some more stuff)
*         Gmt: VisibleParam 720, 480, 0, 38, 720, 404.
*         Gmt: gmtUpdateDispSize (10 -> 1)
*     PropMgr: *** mpu_send(06 05 09 11 01 00)     ; finally triggered PROP_LV_DISPSIZE...
*/


In particular, when switching from x10 to x1, first ADTG[8000] is changed from 5 to 6, then (later) C0F06804 is changed from 0x56601EB to 0x528011B (so if we rely on this register for video mode detection, we miss the ADTG register). Then, other ADTG/CMOS registers are set, but ADTG[8000] is no longer touched.

I think I've found a way around that by looking at this message:

Code Select Expand


PATH_SelectPathDriveMode S:%d Z:%lx R:%lx DZ:%d SM:%d


Need to check whether it's portable.

C0F08030 affects the raw stream if raw type is 0x12, for example (look for entries "scaled by digital ISO" - that's raw data after some preprocessing). The default raw type (CCD), not affected by digital ISO, is, to my understanding, the first in the pipeline (with minimal processing; there is something called "HIV shading" that happens during image capture; I've disabled it at some point by zeroing out the correction data; here's what I think it's a totally raw file).

That's why 14-bit lossless (using raw type CCD) works and 8..12-bit lossless (using raw type 0x12 to darken the image) doesn't work at high resolutions. The latter needs some more stuff reconfigured at the right resolution. No idea what. Changing HIV_V_SIZE didn't help. Locking modified registers from adtg_gui didn't help. Manually resizing the EDMAC transfer used for HIV shading didn't help either. Guess I need to reconfigure all the other image processing modules that affect raw type 0x12; Canon knows which ones

[Levas]

Ah I totally missed that there's another raw stream used for lower lossless options.
Raw-type 12, isn't that the one used a few years ago.
Remember some discussions about problems with raw recording and when all went to raw type '10' most of it was solved 

But, raw type 10 works with lossless compression.
But can't be affected by digital gain...
But it is affected by analog gain 

Why you're not using analog gain registers (ADTG2 8882,8884,8886 and 8888 plus the ADTG 8,9,a and b) ?
I'm only using full stop iso's and the registers don't change much up too iso 6400, and because of that exactly the usable iso range for video 

[Levas]

I've disabled it at some point by zeroing out the correction data; here's what I think it's a totally raw file).

Looks a lot of what I see when I change values of CMOS 4 on the 6d, not sure, but I think you can see a sort of darkframe, when fiddling with this cmos registers.

[a1ex]

Yeah, that could work. IIRC I left the analog gains alone to prevent future conflicts with the ISO stuff. Maybe I was overthinking it

[Levas]

Is there an easy way to test this with only crop_rec?
I can already overwrite analog registers in a crop preset, can set them to values so ml raw histograms ends two stops before the end.
But if I do that, I end up with pink highlights.

Is there an easy way to fix the white level?
And can it be done in a crop_preset ?
Or is the only way to fix white level in Mlv_lite or raw.c ?

[a1ex]

Try lowering the values until the highlights end up just below the declared white level. Then go back (increase them again) a little bit, so they end up just above that. That's pretty much it for that gain.

If you reduce that gain even further, you won't gain any more DR.

Then play with some other gains, evaluate the DR etc. It's a huge rabbit hole

[Levas]

That's the method to optimize DR.
But for now I'm after better lossless compression, and for that I want to lower DR, right?
That's how it works I guess, lower the analog gain 2 stops before the end of ML histogram should be about the same as 12 bit lossless right, or not 

[a1ex]

Yeah, in this case, mlv_lite definitely needs to know about the new white levels somehow. Will think about it, but currently working on something else.

[Levas]

Have some settings where clipping point is before the end of the histogram and want to determine the real white level, or let's say I want to know what the maximum value is for the pixels in a cr2 file.
I have cr2 files with blown out, pink highlights, can I determine the real white level with exiftool ?
Did exiftool plus cr2 file in terminal, without extra command and I get a lot of information, also this:

Code Select Expand


Normal White Level              : 14958
Specular White Level            : 15470
Linearity Upper Margin          : 10275


Does this tell anything about where the real white level should be ?
Is the maximum value for a pixel in this case, 10275 instead of 16384 ? 

[a1ex]

CR2 is another story; white levels there are different.

From the white level in a MLV: (old_white - 2048) * gain + 2048. In your case, for -2 EV, gain is 1/4.

[a1ex]

Getting close to implementing some much-needed refactors, to make the code more generic, but got a small stumbling block at shutter override: the shutter blanking registers on all D5 APS-C models (EOS M, 700D, 650D and 100D). Couldn't figure it out in QEMU.

What you have to do: load adtg_gui.mo and write down the address for registers 805F and 8061 (at the bottom of screen).

I've found these lines in logs from Bilal:

Code Select Expand


0002805f:     37c (was 21c)      ISO=400 Tv=50 Av=35 lv=0 zoom=10 mv=0 res=-1 crop=-1 task=Evf pc=ff56d980 addr=40452d18
00028061:     37c (was 21c)      ISO=400 Tv=50 Av=35 lv=0 zoom=10 mv=0 res=-1 crop=-1 task=Evf pc=ff56d980 addr=40452d1c


These registers are defined for EOS M:

Code Select Expand


#define FRAME_SHUTTER_BLANKING_ZOOM   (*(uint16_t*)0x40481B20) // ADTG register 805f
#define FRAME_SHUTTER_BLANKING_NOZOOM (*(uint16_t*)0x40481B24) // ADTG register 8061
#define FRAME_SHUTTER_BLANKING_READ   (lv_dispsize > 1 ? FRAME_SHUTTER_BLANKING_NOZOOM : FRAME_SHUTTER_BLANKING_ZOOM) /* when reading, use the other mode, as it contains the original value (not overriden) */

but FRAME_SHUTTER_BLANKING_WRITE is disabled. Don't remember the reason - it should be tested, as I need it enabled for shutter speed overrides in crop_rec.

QEMU shows different addresses, so I'm unable to cross-check them.

Also looking for some basic testing on cameras currently working with crop_rec, just to be sure I didn't break anything with the refactors. The new presets are not included yet. For advanced users only (who can compile from source and understand the code).

[dfort]

What you have to do: load adtg_gui.mo and write down the address for registers 805F and 8061 (at the bottom of screen).

Short answer:

Register	700D	EOSM
805F	404512F4	40481B20
8061	404512FC	40481B24

Ok--so this confirms the values that were already found for the EOSM but the journey I took to get there was rather interesting so here comes a longer answer.

Reply #1745 points out some "much-needed refactors" so I assume that is happening in the config_var_refactor branch. That branch was recently merged into the lua_fix branch but there isn't a new build posted on the experiments download page yet so another assumption is that this isn't ready for regular users yet. Tried compiling adtg_gui in the lua_fix branch and it didn't compile. Tried the one posted on the modules download page but that didn't work either. Ok, we're only looking for register addresses at this point so I compiled the iso-research branch and got the addresses.

700D



Something weird happened on the EOSM. I couldn't get to the 805F and 8061 registers at first because the camera was set to 1280x720/60. However, I could see the 805E and 8060 registers. (Note that on the 700D all these registers are visible at the same time.)

EOSM



Switched over to 1920x1080/24 and the registers we're looking for showed up.

EOSM



Note that I might have had to push the zoom button for the registers to show up but I'm not sure if that was necessary.

I did a search for FRAME_SHUTTER_BLANKING_ZOOM / FRAME_SHUTTER_BLANKING_NOZOOM and here is where things started getting interesting. As a1ex mentioned, FRAME_SHUTTER_BLANKING_WRITE is disabled on the EOSM. What I found was that all of the FRAME_SHUTTER_BLANKING constants are disabled on the 100D. In addition, it uses the 805E and 8060 registers instead of 805F and 8061. Also interesting is that the 5D3 uses 805E / 8060 while the 6D uses 805F / 8061 and both of these cameras have all the FRAME_SHUTTER_BLANKING constants enabled.

Looks to me like 805F / 8061 is for mv1080 mode (maybe why FRAME_SHUTTER_BLANKING_WRITE is disabled on the EOSM?) while 805E / 8060 is for mv720 mode. So why are we using one set and not the other? Maybe there should be a way to use both sets of registers depending on the movie settings?

Ok--still need to look into the refactors. Should we be using the lua_fix branch for this? Maybe merge lua_fix into crop_rec_4k? Or should we start with just the config_var_refactor branch?

[a1ex]

The refactors are in the crop_rec_4k branch (making things more generic and easier to port on other cameras). Some of them are not yet pushed, as I was waiting for these tests. Just added a few more.

The stuff from other branches is not related to crop_rec or video recording.

I'm still struggling to reliably detect video mode switches. The PathDriveMode stuff described above works pretty well in most cases, except first startup and switching to movie mode from photo mode. The latter was reported to cause blank screen with some demanding presets, but now it fails with even the simplest 3x preset. Besides that, at least on 60D, the PathDriveMode is updated way too late, after the video mode switch is done, so I'm afraid it might be the same on other APS-C models.

Looks to me like 805F / 8061 is for mv1080 mode (maybe why FRAME_SHUTTER_BLANKING_WRITE is disabled on the EOSM?) while 805E / 8060 is for mv720 mode.

Can you double-check this? My hypothesis was that 5D3 uses 805E / 8060, one in zoom modes (x5, x10) and the other in non-zoom modes (1080p, 720p), while the other cameras use 805F / 8061 for the same purpose; they also set 805E / 8060 to 0, but that's probably unused. I do not see nonzero values for 805E / 8060 in your screenshots.

With the last changeset, I'd expect shutter speeds to be correct (i.e. without this issue). That should apply to all presets once they are included. Also expecting correct shutter speeds when using FPS override (without crop_rec).

Still looking for addresses of ADTG 805F and 8061 for 650D and 100D. Currently, crop_rec doesn't load on these models.

[theBilalFakhouri]

Also expecting correct shutter speeds when using FPS override (without crop_rec).

The correct shutter speeds are working in 700D when using FPS override and this is very cool but with a problem showed in this video:

When this happening?
After stopping RAW video in mv1080 but during and before the recording there are no problems and in H.264 as the video showed.
No problems in x5. In x10 as the video.

And also a problem when enabling mlv_lite + mlv_snd in crop_rec_4k branch. It was working right?:

[a1ex]

@Bilal: before compiling, merge it with either crop_rec_4k_mlv_snd or crop_rec_4k_mlv_lite_snd (they both provide mlv_snd compatibility with mlv_lite, in a slightly different way).

Will check H.264 later.

Edit: watched again a few times; the issue happens with plain FPS override, as soon as you start recording H.264. If you comment out FRAME_SHUTTER_BLANKING_WRITE, it no longer shows these artifacts, right?