12-bit (and 10-bit) RAW video development discussion

[a1ex]

Just tried to backport raw backend's memory management from crop_rec_4k - updated the allocate-raw-lv-buffer branch (which was initially created to add raw video support for 1100D) and merged it into raw_video_10bit_12bit. I ran a quick test on 60D, with a DEFAULT_RAW_BUFFER_SIZE modified to 4MB to make sure it reallocates on larger resolutions (this size was larger than what's required in 1080p, but smaller than what's required in x5 zoom), but that was pretty much all I've tried.

Unfortunately it doesn't build on older models (those without CONFIG_EDMAC_RAW_SLURP), so we are one step back from integration into mainline. But at least 1100D joins the party (thanks Danne), hopefully as a first class citizen (but staying that way depends on user feedback).

[a1ex]

DEFAULT_RAW_BUFFER is used as a pointer to some valid raw buffer (allocated by Canon code). On recent models (60D and newer), it is allocated when entering LiveView (evfActive), but that pointer is no longer valid when leaving LiveView (it's still there, but stale)

Some more notes about this.

On models with EvfState, the raw buffer address (or a pointer to it) can be found in debug logs as "Mem1 Address".

On 60D, 600D, 1100D, 1200D and 1300D (other models seem to be different), a bunch of buffers are preallocated when entering LiveView and kept in some structure; they are retrieved by ADST_GetMemAddrArray (no idea what it stands for). With the memory browser (enabled by default in the dm-spy-experiments branch, or startup-log builds), you can look at 0x26400 (60D) or 0x8A90 (1100D), dereference the pointer and see all these buffers aligned.

I'm trying to find out the size of this buffer.

First puzzle: where it's allocated? QEMU doesn't help - the emulation doesn't go that far.

Since I have no idea where that happens, except that it's somewhere in LiveView initialization (which is a huge jungle), I'm going to print some buffer from this structure along with each debug message (my_DebugMsg in dm-spy.c, dm-spy-experiments branch), and see where this is set:

Code: [Select]

    len += snprintf( buf+len, buf_size-len, "%05X> %s:%08x:%08x: ", us_timer, task_name_padded, lr-4, MEM(MEM(0x26400)) );


Code: [Select]

EC8C8>        Gmt:ff0fcf64:e1a00000: [GMT] gmtInit
EC93E>        Gmt:ff0fcfcc:00000000: [GMT] gmtStart:0x2
...
D3CE7>        Gmt:ff101fec:00000000: [GMT][WAKU] Event 15 Result State 6->6
D3D6D>        Gmt:ff0f8bc8:41b07800: [EVF] evfComAct(0)

Still, between these two messages, there's a bunch of stuff happening, so I still have no idea where that buffer is allocated.

[reddeercity]

I hope this helps with some info -- Run startup-log.2018Jan29.5D2212 to get so Log files I uploaded to my bitbucket account
 dm-0000-nonraw-startup-log.2018Jan29-5D2.log , dm-0000-raw-startup-log.2018Jan29-5D2.log , dm-0001raw-recording-startup-log.2018Jan29-5D2.log .
First logs is after loading up dm-spymem built without any thing loaded , next is with raw enabled the final one is with raw recording in 1:1 (1856x928 23.976p)

[a1ex]

Thanks, would require a bit of context switch to analyze them, as LVState models are a completely different beast from EVFState.

I've got a hypothesis about the raw buffer on EvfState models - full chunks (size = srm_buffer_size) are allocated for LiveView use (SRM_AllocateMemoryResourceForLiveViewYuv) and then divided later; the addresses end up in the ADST_GetMemAddrArray (ADdress STore?) at some point, but where that exactly happens is a mystery. In any case, we've got an experiment named CONFIG_MARK_UNUSED_MEMORY_AT_STARTUP that fills the entire memory with a known pattern, so we can see what parts of memory are used and what parts are likely free (in the Free Memory dialog - the color bar). With this tool, I've noticed the raw buffer address appears to be repeatable (always allocated in the same spot), and even if it's not, it's likely just in a different chunk - but that chunk is going to be divided in the same way.

Here's a test I'd like to run on all EvfState models (those with 10/12-bit raw video builds on the Experiments page). Instructions in the commit message. In addition, I'd like a screenshot of the Free Memory dialog in movie mode. I've ran the test on 60D and 5D3 1.1.3; these are the only EvfState models I own.

The test can be done on either on the allocate-raw-lv-buffer branch (possibly better for older models), or on the current crop_rec_4k branch. I don't expect this to affect the results in any way, so just pick one. If it doesn't work, pick the other.

Will edit the following summary as the test results will arrive:

5D3 1.1.3: 0x4B152000 - 0x4CDFFFFC (repeatable; tested photo mode, 1080p25, 720p50, x5, with and without HDMI monitor)
5D3 1.2.3: ? (old notes: 0x4d31a000 - 0x4ee00000)
6D: 0x4B328000 - 0x4CFFFFFC
650D: same values as 700D/EOSM
700D: same values as EOSM (repeatable; tested mv720, mv1080, mv1080crop, x5)
100D: 46798100-46CC40C4
70D: 4B328000-4CFFFFFC on camera, 4B328000-4D7FFFF0 in QEMU
EOSM: 46798080 - 47F24060 (repeatable; tested mv720, mv1080, mv1080crop, x5)
EOSM2: 48798100 - 48CC40C4

60D: 48332200 - 49F6807C (repeatable; tested photo mode, 1080p25, 720p50, x5, with and without HDMI monitor)
600D: ? (internals like 60D)
1100D: ? (rough guess: 0x46b00000 - 0x473b0000)
1200D: ? (requires merging)
1300D: placeholder (not yet ready to run the test, but it's in the same group)

Hopefully these are all models with CONFIG_EDMAC_RAW_SLURP.

[dfort]

EOSM is a little quirky. Couldn't get screenshot to work while the Show Memory display was open so I did a quick cell phone shot--in movie mode:



The camera always starts up in mv720 mode:



but it does display changes--I switched to Movie Crop Mode and zoom mode:

[a1ex]

If the camera cannot start in the tested mode, you can start the camera with raw video turned off, then switch to that mode (x5, crop mode, whatever), then enable raw video. The raw buffer size detection is performed once, right before this buffer gets filled with data (it's harder to do it afterwards).

Could not reproduce the screenshot issue in QEMU (it captures the screenshot of the Free Memory dialog and saves it correctly)...

[dfort]

Ok--here's the EOSM started up in 1080p mode (Canon menu). Starting in 720p mode gives the same results. Tried again with the screenshot of the Show memory screen, still won't work. After a few tries the led doesn't even do the countdown blink.



Move Crop Mode


zoom mode

[dfort]

700D

Can't do a screenshot of the Show memory display on the 700D either.



mv1080


mv720


zoom


mv1080crop - Movie Crop Mode

[Arthur21Dayne]

Hey guys, Im pretty new to ML, especially to 10 and 12 bit hacks, which im using on my eos m rn, I also have a 5d2 and a 50d so I was wondering if there will ever be a 10 bit or crop mode (3x3) hack for any of those cameras in near future. Ive been around the forum a while now and couldnt find a satisfying answer to that question. Is anybody working on it? Would it even be possible, if not, why not?

[canneloni]

Did the test on 100D (hope I did it right) with latest crop_rec build (magiclantern-crop_rec_4k.2018Feb04.100D101). Folder with pictures can be found here:

https://drive.google.com/open?id=1nHumQo7ZMGHq7lz-Ml6FPgdyGBRSGv7_

While doing the free_memory test, the camera crashed and created a log file, wich also can be found in the linked folder.

[dfort]

...have a 5d2 and a 50d...Is anybody working on it?

I gave it a shot. It is partially working on the 5D2 and much better on the 50D. Haven't given up but ran into a wall. Start reading from here:

https://www.magiclantern.fm/forum/index.php?topic=5601.msg194222#msg194222

There are still test builds posted on my downloads page.

[EDIT] There have been lots of changes to the allocate-raw-lv-buffer and raw_video_10bit_12bit branches. It probably won't make any difference to the Digic 4 LVState cameras, 5D2/50D/500D/550D/7D, but I merged in the latest changes and posted new test builds to my downloads page. Please report only good news!

[reddeercity]

Just for kicks I tested the latest (5D2 10-12bit) knowing nothing was really changed.
As to be expected 1:1 & 3x crop_mode are broken in all bit depths with mlv_lite (I thought 14bit work be ok but not)
full mlv + audio , 1:1 broken in all bit depths thou 3x crop mode seem to be recording ok from the very short clips I shot (10-15 sec.)
So this makes me believe that there has to be something in mlv_lite that causes a total failure in all bit depth + in 3x crop ,
but full mlv+audio in 3x crop mode seems to be some what functional (liveview framing not correct , pink cast on lcd screen)
but this does not cause corrupt frames so far.  I did get a "Raw error detect" and crashed ml , only after I tried moving the 3x crop window while recording (this feature used to work in nighty's)

Code: [Select]

ML ASSERT:
raw_info.bits_per_pixel == bpp
at ../../src/raw.c:2020 (raw_lv_request_bpp), task raw_rec_task
lv:1 mode:3
raw_rec_task stack: 176f68 [177118-176118]
0xUNKNOWN  @ ff8773e0:177110
0x000904C8 @ 929328:176fc0
0x0004EA74 @ 90548:176f98
0x0004E378 @ 4ead0:176f68
Magic Lantern version : Nightly.2018Feb04.5D2212
Mercurial changeset   : 24d40e1a8c3c (raw_video_10bit_12bit_LVState-wip)
Built on 2018-02-04 17:03:59 UTC by rosiefort@RosieFoComputer.
Free Memory  : 133K + 3968K
This happen when a had the crash from 3x crop by trying to move the crop window while recording raw


Notice there a conflict in scr/raw pull-request 16 - 10bit-12bit-for-lvstate-cameras/diff at raw.cT1635 , did you manage to resolve it ?

the resolution change was from 3x crop & 1:1 , so that prove it ,2312x1127 is crop mode up to 2152 & 1:1 2040x1267 full HD/full res

[dfort]

Notice there a conflict in scr/raw pull-request 16 - 10bit-12bit-for-lvstate-cameras/diff at raw.cT1635 , did you manage to resolve it ?

Thanks for pointing that out. It is a problem. Then again I started this with the assumption that what was needed on the Digic 4 LVState cameras was to find the DEFAULT_RAW_BUFFER. Turned out I was wrong.

[reddeercity]

I see , yes I remember reading this forgot all about this .
did a google search for RAW_SLURP to understand the syntax and usage etc. ... to see if there something else would work , came up with this  "slurp_raw" not sure if that's the same thing
see link below .
http://search.cpan.org/~dagolden/Path-Tiny-0.104/lib/Path/Tiny.pm#slurp,_slurp_raw,_slurp_utf8
https://metacpan.org/pod/Path::Tiny#spew,-spew_raw,-spew_utf8

There is something that's been overlooked I think and that's the LCD Display it's self , I mean that digic iv from what I understand is 4:1:1 color space were the digic v & up are 4:2:2
as explained by a1ex here in Display access from bootloader / Portable binary test thread
I have this feeling that there need to be something to take in account for the differences , I guess that could be the Vsync issue but from my experience with broadcast equipment this seems
it need to be conformed to 4:2:2 color (up sample) , this my be why 3x crop mode works (normally) because we are using only part of the screen . But I could  be total wrong too 

[reddeercity]

Ok that confirm it from my dm-startup log , LCD screen is YUV 4:1:1 , but is that just for face detection ?

Code: [Select]

C7654> LiveViewMg:ffa0c904:18:01: [DETFEN] DetectFace_SetDataYUV411VTypeColor space address , possible to modify to 4:2:2 , or is it hardware only 4:1:1

Code: [Select]

D28A1> LiveViewMg:ff8e057c:9b:03: YuvAddress = 0x50000064 -> 0x50000080I see a error in the log too , not sure what this means , everything seem to work ok

Code: [Select]

EC92E> LiveViewMg:00096198:00:00: *** register_interrupt("HEADERROR", 0x6c, 0xff987004 "[CAP]:ISR >>>>>>>>> HEAD ERROR >>> AF SHIFTER ERROR", 0x0), from ff9871d0It's late can see any more , continue tomorrow .

[a1ex]

I was talking about bootloader (think about the Loading message from the firmware update). There, the image overlay (unused in Canon code, just in our demo) is YUV411 in D4 and YUV422 in D5. On DIGIC <= 5, the display hardware accepts 2 image overlays: BMP (at the top), palette-based, with transparency, and image (YUV-based, BT 601 for old models / 709 for newer ones, not 100% sure where is the age threshold). On the bootloader, D4/5 use a 4-bit palette for BMP; in main firmware, they both use a 8-bit palette.

In LiveView (main firmware), it's 422 on both (unless you connect a low-res monitor, when it drops to 411, thus breaking some of our overlays). On DIGIC 2 (unsure about DIGIC 3), the image buffers (such as what gets displayed on the LCD) are 411. The firmware may use additional buffers for other stuff (didn't really look into these).

"Slurp" is our own naming, you won't find it anywhere else. It refers to the transfer of the raw image data into main memory, after it was already captured by Canon code; the transfer itself is done with our own code (which allows us to fine-tune the parameters and the timing - when it happens).

That message is not an error. It means "register an interrupt handler for interrupt 0x6c, named HEADERROR, so that whenever this event happens - probably some error in the image capture process - the function 0xff987004 gets called (and that function uses that string, which is likely to provide a hint on what that function does, so it gets displayed in the log)".

If this low-level stuff sounds strange, be sure to check the Basic concepts section from the QEMU guide. Just be careful not to confuse "register an event handler" with "MMIO register"

[reddeercity]

Thank you , So much to understand  That prove it You can teach a OLD Dog new tricks 

[reddeercity]

Just Thinking out load , been wondering about raw_type for the digic 4 I think it's type 5 , would a different type work ? or is there not other setting .
Can't find any info yet , still looking . 

Edit:is there any info about lv_raw_size ? 5d2 uses "16" is there any other sizes ? or how can I find if there is ?  still looking

Code: [Select]

/* 5D2 uses lv_raw_size >> 16

[reddeercity]

I think I found raw_type or something close in lv_set_raw / lv_select_raw thread but it's listed for 60d
can I assume for all digic 4 ?

Code: [Select]

RSHD     => 0x0B
SHADE    => 0x01
HIVSHD   => 0x07
ORBIT    => 0x09
DEFCORRE => 0x04
CCD      => 0x05 (currently used)
DEFMARK  => 0x06
So this means CCD , make sense  is for imaging , but I also have read something about RSHD has something to do with it or it may be the Liveview LCD can't find it right now.

[Walter Schulz]

Without going into details:
https://builds.magiclantern.fm/features.html -> Internals -> CONFIG_EDMAC_RAW_SLURP

[reddeercity]

Not sure what you mean here ?
I know that " CONFIG_EDMAC_RAW_SLURP" is not working on 5D2 , in fact on all digic 4 cam that what I'm tying to get functional

[dfort]

Don't worry too much about CONFIG_EDMAC_RAW_SLURP. There is a difference between the 60D and 600D which are EVF_STATE cameras and 5D2/7D/50D/500D/550D which are LVState cameras. All EVF_STATE cameras supported by ML have been ported to use the reduced bit depth (8..12bit) code in the raw_video_10bit_12bit branch. There is a sync issue to resolve with LVState cameras.

[Walter Schulz]

Not sure what you mean here ?

I answered a person who prefered to delete his/her question about this one.

[dfort]

Ha ha--so we've been having a discussion over nothing?

@reddeercity and I have been on this quest to get these remaining Digic 4 LVState cameras (5D2/7D/50D/500D/550D) working on this 10bit/12bit branch and ultimately on the crop_rec_4k branch. The strange part is that if we don't do anything at all with the code and just compile this branch, some of the settings seem to work quite well. Modifying the code we've been able to get some other modes partially working at the expense of the modes that worked without tweaking the code. Sort of like robbing Peter to pay Paul.

[reddeercity]

Yea I know , I still convinced that it will work , I did some test the other day with the 10_12bit_broken_3xcrop_mode_2017Dec10.5D2212.zip
That I complied with the change I did (mainly the write channel ) ( the one that a1ex said not to use but works in 1:1)
I what to see what problem or crashes would happen in hopes of maybe seeing something that can help resolve this problem .
So with mlv_lite  without hdmi connected there was no issue crashes etc. ...  but with hdmi in standby ,
I would get the flicker with pink  or out of sequence image(see below) when I start recording raw video the flicker stops and liveview is good to record from hdmi .
I can't say the same as  full mlv + audio I got a memory overflow  didn't matter if hdmi was connected or not .
printed a message on screen "mlv_rec.c line 161" has to do with the SRM memory
maybe audio pushed it over the edge not sure .
 

Next I'm going to be looking at big LVState diagram , I did look them over a year ago but really didn't understand that much now I hope I can make head or tales out of it