crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

Started by a1ex, April 01, 2017, 11:15:41 AM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

a1ex

@dfort:

Full-res LV is just an automation around crop_rec's preset with the same name, which is only available on 5D3. Check the commit to see how it works:

- enables Full-res LV in crop_rec [ this part fails if that option is not available ]
- refreshes the LiveView
- takes a regular silent picture
- disables crop_rec [ this part works, as all cameras running crop_rec have an "OFF" option ]

There's no easy way to check whether a particular choice is available in the crop_rec menu, other than manually walking the list of choices.




@kizza1234, @whysodifficult:

Lossless = without any loss (14-bit lossless = bitwise identical to 14-bit raw after decompression, e.g. same MD5 if you want). Removing data not visually useful = lossy (that's what H.264 does). Reducing the bit depth is also a lossy operation.

Quote from: a1ex on April 15, 2017, 01:12:36 PM
14-bit lossless will give raw data identical to uncompressed 14-bit (100% identical, not one single bit different, contrary to some earlier report), while "12-bit lossless" should be interpreted as "14-bit to 12-bit conversion - lossy by definition - followed by lossless compression", so it should have the same number of useful levels as uncompressed 12-bit. Please note the 12-bit lossless is not identical to 12-bit uncompressed - they differ by a constant value and possibly by some round-off error, and the same is true for lower bit depths.

Quote from: dmilligan on June 15, 2017, 02:40:04 AM
"Lossless" compression algorithms compress data in such a way that the original data can be reconstructed exactly, bit for bit. "Lossy" compression algorithms compress data in such a way that the original data cannot be reconstructed exactly, but you can come close and the differences are optimized to be things that aren't very noticeable based on human perception. They are two very different beasts.

14 - 12 = 2, not 4.

Read the help text at the bottom of screen in ML menu - it should cover these things (display the current data rate, explain how each bit depth preset is created and so on).

kizza1234

Hi all, sorry for the misinformation A1ex is correct and was wrong about the compression :/
Gets so confusing lossy and lossless haha

12bit lossless is beast!

OlRivrRat

      @Alex

   Thanks Much for the Full-res LV explanation. Might be good to remove the Option from Cam's

on which it will not work.

   In the 5D3 World is there an advantage to FRSP Full-res LV over FRSP Full-res?
ORR~DeanB  ~~  80D-ML  &  SL1+ML  &  5D2+ML  &  5DC+ML  &  70D+ML(AliveAgain)

a1ex

Full-res LV works at fast shutter speeds (up to to 1/15000, but with 128ms rolling shutter) and has trouble at slow shutter speeds (didn't try to push it too far), while the "classic" version works only at slow shutter speeds (with gradient - fastest shutter speed being between about 1/33" at the top of the frame and 1/6" at the bottom).

They use very different backends (one works in LiveView by modifying the resolution; the other exits LiveView temporarily and uses the still photo capture functions), so their UI behavior is also noticeably different (and it's not trivial to switch between them).

IDA_ML

Quote from: a1ex on February 09, 2018, 06:01:56 PM
Full-res LV works at fast shutter speeds (up to to 1/15000, but with 128ms rolling shutter) and has trouble at slow shutter speeds (didn't try to push it too far) ...

With regard to this I have a few questions to you, A1ex:
-----------------------------------------------------------

1) As explained on the Experimental page, this feature works very well on the 5DMkIII which allows continuous recording at 5796x3870 resolution and a speed of 5 fps.  I was wondering if this same Full-res LV feature could be implemented also in the 100D, 700D and other similar cameras, at a lower frame rate of course, due to the lower write speed of about 40 MB/s.  I remember you saying that 2 fps should be doable.  This feature would be great for FRSP time lapses since there is no dead time between the frames recorded and this would result in very smooth time lapses.  Combined with Dual ISO, this would be another dream come true for those using these cameras.

2) If the answer to the above question is yes, would it be possible to add a menu option applying also lossless compression, (selectable between 14, 12 and 8 ... 11 bits) to the frames to save card space and pack whole image sequences in MLV containers to greatly enhance sorting?  If also a change in the file name is made, say a letter "T" for time lapse, this will made file sorting even easier.  The same question applies also to shooting single FRSP DNGs using IS lenses that you recently added to the above cameras - I love this feature!  If also such picture sets, shot at different locations, could be packed in MLV containers with a different letter in the file name, say "S" for picture sets consisting of various single LL DNGs, this will save the photographer a lot of time sorting his pictures according to the location at which the picture set was shot.

3) Right now, the thumbnails of the single FRSP-DNGs show up as black rechtangles in Explorer.  Is it possible that they show up as small picture thumb nails to make finding a specific picture easier?  This works with Cannon's CR2 files.  Is it possible to make it work also with ML DNGs, keeping in mind that they use the same type of lossless compression?

Thank you in advance for your answers.

a1ex

1) Yes, but it has to be done by somebody who has the camera in his hands, as it's model-specific and mostly trial and error with the adtg_gui module. For 700D, mk11174 got very close. Some more hints around the forum (and first post from this thread, of course).

2) Yes, but these are very easy coding tasks (anyone can give it a try); I prefer to focus on the more difficult stuff.

BTW - now that lossless compression works well on the cameras that support it, I was thinking to remove the uncompressed options, to keep the code simpler.

3) The DNGs do have previews, but they are uncompressed (uncommon); Explorer may expect them to be plain JPEGs? It would require additional reverse engineering to understand how to create JPEGs. The same routines, if figured out, would also provide MJPEG recording.

(the previews do show up in Geeqie, which is what I use for sorting pictures)

Danne

Quote
BTW - now that lossless compression works well on the cameras that support it, I was thinking to remove the uncompressed options, to keep the code simpler.
I'm all for simplifying. Question though. Stabilitywise, are the similar now? Not filming a whole lot these days I remember early stops with lossless due to specific conditions, too bright scenery, hdr recording. Did not encounter this with uncompressed modes. However, maybe separate compressed and uncompressed in two branches meaning crop_rec_4k solely lossless and 10bit_12bit branch compressed? My two cents.

Lars Steenhoff

Please keep the uncompressed as its the one format that is suppored by all  the post software since the beginning of magic lantern, And I have had more problems with compressed than uncompressed.
Also uncompressed is probably faster to display as there is no need to decompress.
And for higher isos there is no worry to reach the compression limit.

please keep it  8)

IDA_ML

I fully agree - MLV_REC is now redundant.  I still would leave the 10, 12 and 14-bit UNcompressed options in the RAW video mode just in case we experience some instability as Danne pointed out.

Geeqie seems to be a working option on Mac and Linux but not for Windows, unfortunately?  Are you aware of a similar software for Win?

a1ex

We were talking about silent pictures, but probably it's not a bad idea for video either. Still, for video we've got a problem with older models - I was unable to get lossless compression in LiveView on 60D, for example, so that would break the compatibility.

For silent pictures (not video), Greg was able to get lossless compression working outside LiveView on 500D, to some extent, though we did not try too hard to decode the pictures. So, at least outside LiveView, I'm confident lossless compression can be ported on all models.

BTW, updated ISO research modules to load on the latest crop_rec_4k builds (including 100D) without having to compile from source. Will test later. The raw_diag module should also load on lua_fix, as it has no special dependencies, but adtg_gui requires the patch manager.

IDA_ML

Quote from: a1ex on February 10, 2018, 02:01:40 PM
BTW, updated ISO research modules to load on the latest crop_rec_4k builds (including 100D) without having to compile from source. Will test later. The raw_diag module should also load on lua_fix, as it has no special dependencies, but adtg_gui requires the patch manager.

Yes, I always wanted to test these miracle crooked ISO numbers that provide lower noise and improved dynamic range but never knew how to do that on the 100D.  I will start reading on that as soon as I can and hope to be able to test soon.  I will also be looking forward to your findings on this.

theBilalFakhouri

Quote from: a1ex on February 10, 2018, 12:59:23 PM
.... For 700D, mk11174 got very close. ...
@a1ex I can't see C0FO[****] Registers, the others shows very well.
What should I do to see these C0FO Registers?
I use latest build from 4k branch on 700d.

a1ex

For 700D, define ENGIO_WRITE_FUNC and ENG_DRV_OUT_FUNC in adtg_gui.c (copy from stubs.S; they used to require fiddling, but I hope it's no longer the case), then select Advanced -> DIGIC registers.

Currently trying to sketch a walkthrough; here's the first draft (still editing).

Here's another adtg_gui walkthrough for a different purpose; the basics are still valid.




Background:

- What are these registers? (TLDR: that's how the ARM CPU interacts with other peripherals - by writing to / reading from registers)
- What are ADTG and CMOS registers? (TLDR: CMOS = sensor controller, ADTG = ADC + timing generator, ENGIO = Canon's image processing engine)

For changing the LiveView resolution, we are interested in:

- ENGIO registers 0xC0F06800/4 on D5, 0xC0F06084/8 on D4 (raw.c: raw_lv_get_resolution); these adjust the resolution of the captured raw buffer (it's the first set of registers we have to tweak, but it's not enough)
- Video timer aka FPS override (these are directly related to resolution; we did not know that at first, but found out later)
- CMOS registers (to select what image area to scan, and also the column binning factors)
- ADTG registers (line skipping factor and some others)

The number of registers is huge (overwhelming). Hundreds? No. Thousands. Many thousands. Address space covers millions of possible registers. We only understand a tiny part of them. Don't be discouraged.

The logging code has noticeable overhead and may push LiveView beyond its limits. Be prepared to encounter crashes or corrupted images, and be prepared to search for workarounds. For example, on 5D3, I can switch from 50p to 25p with logging enabled, but not the other way around. Enabling logging in x5 mode gives corrupted frames, unless I reduce the FPS. There is an option named Force low FPS, just for that.




Let's start with some basics, for example FPS override.

There are two timers: A and B. FPS is MainClock / timerA / timerB. MainClock is TG_FREQ_BASE in fps-engio.c (24MHz on 5D3, 32MHz on 100D). Browse the FPS override submenu in a few video modes, compute the FPS from timer values and make sure you understand the math before proceeding.

Now try changing FPS from adtg_gui. Timer A is 0xC0F06008, B is 0xC0F06014. Start from a low-FPS video mode (24/25p), turn off FPS override, enable ADTG registers, enable DIGIC*) registers, select to show Image size regs only. Go to PLAY mode and back - the registers are identified only when Canon code changes them. Locate the two registers in the menu.

*) actually ENGIO - it wasn't clear to me when I wrote the module. I should rename it.

On 5D3, in 1080p25, you will see: C0F06008 = 0x1df01df, C0F06014 = 0x7cf. Do the math to get 25 FPS. Do not skip this step.

Now increase the value of timer B (e.g. 0x7df). Go to PLAY mode and back to apply the changes (they are only applied when Canon code changes these registers again). Notice the FPS value printed by ML on the top bar changed.

Do the same with timer A (for example, change it to 0x1ef01ef). Notice there are some more registers with the same value (0x1df01df or 0x1df). Update them too.

Level 1 complete - back to theory.

Timer A is directly related to horizontal resolution. Timer B is directly related to vertical resolution. They are not the same, but if you increase one, you may also have to increase the FPS timers.




Now that we know how to adjust stuff, let's find out what registers are relevant to image resolution. The first step is to look at the registers configured by Canon code, save them to a log file and compare them.

For example, let's compare 1080p25 with 720p50. On 5D3, the raw resolutions in these two modes (usable in mlv_lite) are 1920x1290 and 1920x672 (same resolution horizontally, different vertically). Full buffers (including black bars) are 2080x1318 and 2080x692 (edmac.mo -> Show EDMAC channels -> raw_write_chan in edmac-memcpy.c -> channel 4 on 5D3 and 18 on other models, or Debug -> Show console).

Unfortunately, on 5D3 I can only compare the ADTG/CMOS registers (without the DIGIC/ENGIO ones), as it locks up when trying to log the latter in 50p. It might work on other models, or it might not. We'll still get useful info out of this.

Step by step:
- select 720p50 in Canon menu
- enable ADTG Registers
- go to PLAY mode and back to get all the registers
- select Show -> Modified from now on (also displayed as "since HH:MM:SS")
- go to PLAY mode and back, make sure it shows nothing (if it does, repeat a few times)
- select 1080p25 in Canon menu
- go to ADTG registers menu and notice some registers appeared
- to double-check: go back to 720p50 and make sure it shows nothing
- back to 1080p25 and the registers should appear again
- select Advanced -> Log Registers Now (to save the modified registers to a log file)

That's the general procedure for "diff-ing" two video modes.

There are already a few dozens of registers, but you can ignore some of them:
- shutter blanking: that actually gives the shutter speed
- ADTG 8/9/a/b and 8882/4/6/8: ISO gains
- likely most other registers with "small" values

The interesting stuff:
- ADTG2[800c]: line skipping factor (2 in 1080p and 4 in 720p)
- ADTG2[8178, 8179, 82F8, 82F9, 8196, 8197]: likely related to vertical resolution (look them up in crop_rec source)

You may also diff the 1080p and x5 zoom modes - this should work with DIGIC registers as well, without locking up. There you'll notice registers 0xC0F06800/4 for adjusting the capture resolution, and many many others.

Finally, you can compare the x5 zoom with the regular photo mode. This may require some fiddling - I've got it working by using plain FPS override set to 10 fps to avoid image distortion, rather than adtg_gui's "Force low FPS" (which appears to interfere with photo mode).

Level 2 complete. You may now examine the differences between Canon video modes, look up the registers in adtg_gui.c and make some guesses about their possible meaning.

TODO: logs.




Let's now try to adjust the horizontal resolution. We've already got some 20MB reserved for the raw buffer from Canon code, so we don't need to worry about memory yet. Let's start from some 1:1 crop mode (x5 zoom or something else).

Horizontal resolution is adjusted from the low halves of 0xC0F06800/4. First do the math for the current video mode (raw_lv_get_resolution). Enable ADTG Registers and DIGIC Registers to see them, then select "Show: Image size regs only" to find them faster, then go to PLAY mode and back to refresh all the registers. Select "Force low FPS" if the image is broken.

Examples:

5D3 1080p: [0xC0F06800] = 0x10017, [0xC0F06804] = 0x528011b => 2080x1319 (column_factor = 8 for 5D3, 4 for 100D).

5D3 x5: [0xC0F06800] = 0x10017, [0xC0F06804] = 0x56601eb => 3744x1381.

Increase horizontal resolution in x5 by 1 step (8 pixels on 5D3) => change 0xC0F06804 to 0x56601ec. Go to PLAY mode and back to x5 to apply the change. Notice the image is "slightly" distorted (some sort of diagonal look). This is good!

Enable raw video (mlv_lite), set preview to Framing, set image resolution and aspect ratio to maximum.

Before the change: 3584x1320.
After: 3592x1320.

Level 3 complete. You have successfully increased the resolution in x5 mode by a whopping 8 pixels!

(the real-time preview is broken, don't know how to fix it yet, but the preview looks good with the Framing option, so the raw data is valid)

Still with me?

Quentin

Impressive findings, maybe scary for ordinary users.

April 1st 2018 is the Benchmark  ;)

theBilalFakhouri

Quote from: a1ex on February 10, 2018, 05:36:34 PM
Still with me?
Finally I have defined ENGIO_WRITE_FUNC and ENG_DRV_OUT_FUNC, I was having some issues with compiling and understanding somethings.
I will reply as soon as I understand other things  :D
Thanks for this amazing explaining!

kizza1234

Will it be possible for resolutions above 2k to have working full live view (other than 5x)? Or is it a hardware limitation?

Thanks heaps.

a1ex


theBilalFakhouri

Okay a1ex for Level 1

BaseClk = 32 MHz = 32000000 Hz (Canon 700D)
Timer A = 639 | Timer B = 1999
clk1 = 32000000 / (639+1) = 50000
fps = 50000 / (1999+1) = 25fps

I get the values from my camera  C0F06008 = 0x27f027f  &  C0F06014 = 0x7cf (Same as 5D3).
But how I can calculate these values without FPS override (I get timer A & B values from FPS override) How is 0x7cf  = 1999 what the math ?
EDIT: I understood that is 0x7cf  a Hexadecimal value.

After tweaking C0F06008 &  C0F06014 I got 34.04 from 25fps which is nice experiment, I tweaked even more, I end with around 42 to 45 fps with working liveview but the image was more darker comparing to 25fps (liveview freezing & corrupted frames at 45fps while recording, at 42fps was fine) .

whysodifficult

kizza1234
jimiz
Kharak
a1ex
Thanks for your answers!

It's pity we can't extend hi-res possibilities with Ninja-like device. (I'm watching 60 fps videos in youtube, such a smoothness, but i'm dreaming))

Was interesting to know A1ex's answer about normal LV on hi-res.

Well then i'll try uncompressed modes as well, maybe there will be difference for me as well.

pc_bel

The last improvements are fantastic!!!!  MLV_lite with audio recording!!!!
I follow this topic from message number 1 and the amount of work seems to be giant. Thanks to everybody!!!
Even although I read all the messages related to the new audio implementation, I have a doubt. My audio is still out of sync. I'm using MLVFS to get the DNGs and using the last ML version (5feb) in a 5Dmk3. Is this a bug to solve? or is a problem with my particular workflow? (MLVFS to get DNGs and direct import to PremierePro CC2018).
Thanks.

pc_bel

Sorry, one more question.
I can't make to work H264 proxy recording when I use mlv_lite+audio. It stops working in few seconds. Is it normal, or its me?
Thanks.

SanchYESS

Quote from: pc_bel on February 14, 2018, 10:13:49 AM
...Even although I read all the messages related to the new audio implementation, I have a doubt. My audio is still out of sync. I'm using MLVFS to get the DNGs and using the last ML version (5feb) in a 5Dmk3. Is this a bug to solve? or is a problem with my particular workflow? (MLVFS to get DNGs and direct import to PremierePro CC2018).
Thanks.

Audio records a bit extra before first and after last frame (which is may be helpful), I use mlv_dump, import video and audio into Resolve separately then sync manually. A bit longer but fine by me.

rob_6

Quote from: SanchYESS on February 14, 2018, 04:26:18 PM
Audio records a bit extra before first and after last frame (which is may be helpful), I use mlv_dump, import video and audio into Resolve separately then sync manually. A bit longer but fine by me.

SanchYESS,

When you sync manually the clips in resolve are they all synced at the same point for any clip length? For example do you just slide the audio back 10 frames for each clip? Or is it a different point for each clip?

Hopefully its the same offset so we can use one of the mov converters that can offset the audio...

Thanks!

anto

Need opinion please.
I know they are the same performance with ML, but wich is the best choose EOS M, 100D or 700D ?

pc_bel

Thanks SanchYESS!!! :)

Hope it has solution, because syncing every clip is time consuming if you have thousands of them  :-[

Maybe I can edit and then sync all clips together... If all have the same offset...