12-bit (and 10-bit) RAW video development discussion

[Andy600]

- without raw video and without photo raw overlays enabled (e.g. global draw off, no other modules loaded)

all logged at 50u

Without LV

With LV

- with raw video enabled, old method (run it on top of regular nightly or raw_video_10bit_12bit)

still to do

- with raw video enabled, new method (on top of dfort's modified version)

using dfort mod

Also, for my own curiosity: log a still photo, short exposure, outside LiveView, with 500us

500u

(or 1 ms if that doesn't cover the entire process).

1ms

Look at the blue LED to see when it logs and when it stops.

Stays on then flickers briefly before showing preview and 'logged' message.

Take a look at the EDMAC connections screen as well (Show EDMAC channels, scroll to the right).

Ok. what am I looking for?

[dfort]

I'm at work today so I can't test but I can compile.

Merged  the EDMAC module: fixes for DIGIC 4, connection map pull request and updated the builds on my downloads page.

Lots of studying and testing still to do on this.

[a1ex]

Pictures: plain LiveView (1) vs LiveView with raw enabled using new method (2, 3):







Notice how channel #2 writes some data into RAM, but not for every single frame. It's transferring every now and then, either 32x15 or 32x225 or 32x240 bytes.

What exactly it's doing? To find out:
- capture a log using dm-spy-experiments branch, make sure to include EDMAC setup functions (SetEDmac, StartEDmac etc) and see what debug messages are nearby
- dump the data from that memory address and attempt to make some sense out of it

Already having such a log done for 500D (linked earlier), channel #2 is configured with same transfer size, so I'm looking there:

Code Select Expand


1E2F2> LiveViewMg:ff0da8e4:98:02: SetWbIntegRegisterForAe 0
1E31F> LiveViewMg:ff1fabf0:99:02: StartWbPass
1F3D5> LiveViewMg:000927a0:00:00: *** ConnectWriteEDmac(0x2, 0x2), from ff1fac44
1F46A> LiveViewMg:000927a0:00:00: *** RegisterEDmacCompleteCBR(0x2, 0xff1fa56c, 0x43ffa1e4), from ff1fac54
1F4D0> LiveViewMg:000927a0:00:00: *** SetEDmac(0x2, 0x43ff83e0, 0x15a200, 0x20000000), from ff1fac68
1F76E> LiveViewMg:000927a0:00:00: *** StartEDmac(0x2, 0x0), from ff1fac74


White balance.

That's why channel #2 is not safe to reuse on 50D, 5D2, 550D, 500D and 7D (links pointing to EDMAC screenshots).

[IDA_ML]

A brief feedback on the Dec. 11-th build for the 7D that I have tested:
==============================================

Good news:

1) If you add the RAW-tweak module from the old Dec. 1-st/2016 build, discussed above, to the Modules directory, you can now playback your 10/12-bit recorded clips in the camera!  Playback works fine on normal and 5x-crop recorded clips.  You can find this build here:

http://www.magiclantern.fm/forum/index.php?topic=5601.msg187903#msg187903

Bad news:

2) The overall camera behaviour remains pretty much the same as described in my post #1390 on page 56.  Camera screen still freezes when you press the magnification button once but 5x-crop recording with sound works just fine in this condition up to 2496x1198 resolution.  Getting out of the frozen screen state does no longer require camera restart with taking battery out, all you need to do is to press the play button and the camera returns to its normal LifeView screen where framing, focusing and metering is possible.

Conclusion:
------------
In the normal non-crop mode, the camera provides very stable and consistent 10/12-bit continuous recording with sound up to 1728x972 resolution and up to 30 fps.  If the frozen-screen issue in the 5x-magnification mode could be fixed this would make the EOS 7D a fully capable camera for 10/12-bit RAW video recording with synchronous sound and resolutions up to 2496x1198 at 24 fps.

Keep up your excellent work, guys!

[tonij]

Regarding the 7D frozen screen issue in the 5x magnification mode, is there anything that could be learned from this build: http://www.magiclantern.fm/forum/index.php?topic=5601.msg187903#msg187903
It has a working liveview (canon mode) with global draw off for 10/12 bit in 5x mode.

[dfort]

is there anything that could be learned from this build

Sure, another one of my old builds coming back to haunt me.

Code Select Expand

Magic Lantern Nightly.2016Dec01.7D203
Camera   : 7D
Firmware : 203
Changeset: b3dfbe7194f3 (x-perimental) tip
Built on : 2016-12-01 23:00:07 by rosiefort@RosieFoComputer
-e


@a1ex - that "-e" is showing up only on these old builds.

It has a working liveview (canon mode) with global draw off for 10/12 bit in 5x mode.

My understanding is that it works only in zoom (5x) mode but now that I've got access to a 7D I'm not having much luck getting that working.

-In Movie menu set FPS override to 24 FPS Exact.
...
-In Overlay menu set Global Draw to OFF, return to Liveview in 5x zoom
...
Edit: I've noticed there's a few pixles in the bottom right corner of the video that do strange things..

Is it necessary to set FPS override? That is something being reported on the 5D2 on these latest experiments too. Also, is turning off Global Draw necessary?

Pixels on the bottom right doing "strange things?"

That old build doesn't have CONFIG_EDMAC_RAW_SLURP or CONFIG_ALLOCATE_RAW_LV_BUFFER which is what the latest test test builds for Digic IV LVState are using. Maybe it was just luck that it worked. With the new builds you should have mv1080 and mv720 pretty much working and sometimes zoom mode, depending on the luck factor.

By the way I have been re-reading this topic, making corrections on some of my posts and studying up on things.

Late breaking news -- @nuvon did find the SRM_BUFFER_SIZE for the 1100D. It is: 0x14e8000

[IDA_ML]

Sure, another one of my old builds coming back to haunt me.

Ha ha, Dfort, this old build of yours keeps coming back to haunt you because this is the best ML build made for the 7D in the history of mankind!  Once it came out, I installed it and used it throughout the entire 2017 for my amateur videos.  If worked so well at 10 bits with sound in the 5x-crop mode that I never thought to replace it.  And if I needed the normal mode, I used it at 14 bits - still very stable.  The only way to stop this old buddy haunting you is to write a better one - possibly with 4K crop recording and lossless compression.  Only then, your old trusted Dec. 1/2016 build will find peace and will rest in peace too  ! 

[dfort]

...this old build of yours keeps coming back to haunt you because this is the best ML build made for the 7D in the history of mankind!

Yes, we can learn from the past. Fortunately all this is being done on Bitbucket and reported on the forum so it is rather easy to go back and see what was being done back then.

I got a PM from @DeafEyeJedi. Hopefully he'll share some of the video he shot on the 7D to illustrate this issue:

First MLV LITE 10-bit works like normal but then afterward it's all choppy with first frame repeating every other frame or so. We have seen this behavior before on other cameras. Don't remember which.

It is quite disappointing to shoot a bunch of footage in the field only to find out when you get home that it didn't turn out so I updated the test builds with raw_twk. This way you can preview 10bit/12bit MLV's in camera. Today's builds also include the edmac module.

Noticed that there have been downloads of all cameras in one day so there's interest in this. It would be good to get some more reports from testers.

Don't try random pointers hoping they will work.

Ok--so now that you pointed out that DEFAULT_RAW_BUFFER is returned as a pointer and CONFIG_ALLOCATE_RAW_LV_BUFFER is not, I can see there is a significant difference between the two.

Code Select Expand

#elif defined(DEFAULT_RAW_BUFFER)
    return (void*) DEFAULT_RAW_BUFFER;
#elif defined(CONFIG_ALLOCATE_RAW_LV_BUFFER)
    return raw_allocated_lv_buffer;


I'm still very much a beginner in this stuff and haven't quite wrapped my head around what a pointer is. Most of what I learned in coding comes from Googling stackoverflow. This is a rather simplistic answer but it helps clear things up a bit:

(void*) - "One can say void is nothingness void* is everything"

There's a lot more reading and testing to do. One thing to try is CONFIG_RSCMGR_UNUSED_SPACE_TEST. Looking at the code it says:

Code Select Expand

        /* take an unused block from RscMgr and manage it with AllocateMemory's low-level allocators */
        /* the block must be tested to make sure it's really free before using it */
        /* to do so, define CONFIG_RSCMGR_UNUSED_SPACE_TEST,
         * then use and abuse the camera for a while in this configuration */
...
        .preferred_max_alloc_size = 5 * 1024 * 1024,


5 * 1024 * 1024 = 5242880 or 0x500000 Right? We're shrinking SRM_BUFFER_SIZE somewhat ( 0x1F80000 - 0x1000 on the 7D) but that is a much larger value than the preferred_max_alloc_size noted there.

I take it that playing around with CONFIG_ALLOCATE_RAW_LV_BUFFER is safer than DEFAULT_RAW_BUFFER but don't we need both of these in order to make the jump to the crop_rec_4k branch? As I recall the whole issue why the Digic IV LVState cameras weren't working properly was because nobody was able to find the DEFAULT_RAW_BUFFER on these cameras. As @IDA_ML, @reddeercity and others have noted they can record 10bit in zoom mode but this might just be a "lucky" coincidence.

[a1ex]

Ok--so now that you pointed out that DEFAULT_RAW_BUFFER is returned as a pointer and CONFIG_ALLOCATE_RAW_LV_BUFFER is not, I can see there is a significant difference between the two.

They are both pointers; the only difference is who allocates them (Canon code or our code).

As I recall the whole issue why the Digic IV LVState cameras weren't working properly was because nobody was able to find the DEFAULT_RAW_BUFFER on these cameras.

No, the whole issue was (and still is) proper sync to avoid corrupted frames (in all video modes).

The sync issue has two components. An important one was frame size in 10/12-bit modes (which affected EDMAC behavior - it went out of sync), which was solved by RAW_SLURP. From current reports, this was not sufficient to solve all camera models x video modes. The remaining issue is probably that RAW_SLURP does not always start transferring the raw data at the right moment (like Canon's lv_save_raw hopefully does). We need to find when to call raw_lv_vsync: once for every LiveView frame - we are already doing that - but when exactly? This time we have to sync with Canon's image capture process. The newer models (60D and newer) already do that (evfReadOutDoneInterrupt).

The buffer does not affect sync at all (it's just a place in memory where the data is going to be stored) and the EDMAC channel must not conflict with anything else, but that's common sense. These were required by RAW_SLURP. BTW - changing that 0x1000 is not going to make things any better or worse, as long as the allocation succeeds.

It is also possible to solve the frame size issue without RAW_SLURP (and therefore without a custom buffer and without changing the EDMAC channel), with cache patches. That will keep the other shortcomings of lv_save_raw (side effects in x5/x10 on most old models) and the backend for cache patching is still quite fragile in my opinion, so I'd prefer to avoid this route.

The keys for solving the sync issue are in that big LVState diagram, the debug logs from LiveView (dm-spy-experiments branch; for models other than 500D you will have to find some stubs) or EDMAC activity logs/graphs (currently captured for 50D, but incomplete). I hope to be able (but cannot promise) to solve the sync issues after analyzing all these logs.

All the logs have to be captured in 3 LiveView modes (see #1450) in order to have something to compare. Repeat for all video modes (1080p, 720p, 5x, whatever else your camera has). Repeat for all camera models (not just 50D). The two logging tasks are also for camera models already present on the Experiments page (for double-checking and for having something to compare against).

[CITY-U1001]

50D
MLV_rec 1568х882 10bit all good, on Transcend 400x 32Gb. raw_twk work fine for mlv_play.
build: raw_video_10bit_12bit.2017Dec12.50D109

[IDA_ML]

Man, you are moving fast, Dfort!  Amazing work!  Thank you so much!

I have noticed that you have a 7D now.  Congratulations!  I am sure, once you get a feeling for this fantastic camera, you will never reach for your other ones.  I have been testing your latest builds for hours now and I still have some juice in the battery left.

I have tested your Dec. 12 build on my 7D but I have to run for work now.  I will report on what I found later.

Keep up!

[tonij]

Shot some 10bit today with the Dec 12 build on 7d.
All 24fps exact in normal mode and 5x mode with sound.
Found myself having to restart the camera after each recorded clip or the next clip would get that "earthquake shaking"

[Danne]

Not sure where to put this but here we go:
Tried compiling and enable a version from dfort branch crop_rec_4k_7D_wip and when mlv_lite enabled camera always starts off with error 70. Maybe stubs aren´t correct? Didn´t check into it too much.

[coco770108]

Thx @dfort
I have tried the last version (2017dec12) released from your bitbucket, on 500D it can be work on normal mode, in 5X mode the screen will be frozen.

[IDA_ML]

Here is a brief feedback on the raw_video_10bit_12bit.2017Dec12.7D203 version:

Good news:
=======

1) It is amazing how long the battery life of the 7D is with these new builds.  I played last night for several hours and also this morning for a while and I sill have some energy left in my battery although it is at least 5 years old.  Surprisingly, the maximum temperature I reached did not exceed 51 deg. C which is very good.

2) Raw-tweak works very well and is extremely useful.  The same module used in the Dec.1/2016 build works flawlessly in your current builds as well.  Amazing!

Bad news:
=======

3) In the normal uncropped modes I no longer get this clean, stable and non corrupt video that I got on the December 11 build.  As Tonij said, I get "earthquake shaking" all the time, regardless of the resolution and bitrate set and this happens in both: the MLV and RAW video modes.  With my card (Transcend 16GB/x1000), "earthquake shaking" happens all the time, even if I turn the camera off and then on again.

4) Also in the 5x-crop modes things have gotten worse.  Sometimes the screen freezes, sometimes it shows jumping corrupt frames, and all this makes composing and metering impossible.  Focus does not engage when I press the back button for focusing.  I have to get out of this mode by pressing the play button and only then composing, focusing and metering is possible. 

5) Recording in the 5x-crop mode is still possible at the maximum 2496x1198 resolution but I rarely get more than 300 frames recorded at 24 fps.  With the Dec. 11 build I was sometimes able to record 500-520 frames without a single corrupt frame. Now, I get lots of corrupt (jumping) frames with the Dec. 12 build. 

6) I have noticed that features from the crop_rec_4K branch have been added (e.g. prerecording). I did not have time to play with these features but given the fact that recording is not stable and consistent enough, this does not make much sense.

7) I would like to update this post with another observation of mine concerning RAW-video recording with the RAW_REC module.  Sometimes recording in the 5x-magnification mode is OK, sometimes the recorded clip is full of corrupt frames, regardless of the resolution.  When corrupt frames start to appear in the clips, the camera needs to be turned off and on again and then one has the chance to record a few more clips without corruption.  At some random and not reproducible moment the corrupt frames start appearing again.  Sometimes this happens also in the normal uncropped mode but less frequently.  The MLV_REC module seems to work in a more stable way and with synchronous sound. 

My suggestion to Dfort:
==============
Including too many new features at this point, especially crop_rec_4K, in my opinion, does not make much sense.  Fixing things one at a time and getting more stable and consistent operation of the existing ones would be better.  I would suggest that you go back to your December 11 build and start from there.  That build was quite stable in the uncropped modes and also recorded fine in the 5x-ones.  You could try to fix the freezing-screen issue first which, in my opinion, was the only serious problem with that Dec.11 build.  It may help if you again compare your code from the old "haunting" Dec. 1/2016 build in its 5x-crop recording part with your current code and see if that can help.  There might be some slight difference that causes the current problems.  I installed that old build again yesterday and confirm: in the 5x-crop mode you can easily get out of the frozen-screen state just by switching the preview from Canon to Auto and back to Canon or by turning Global Draw off and on again as Tonij described.  Once you unlock the frozen screen, LifeView and recording in that mode work normally for all 5x-crop clips to come.  You have to repeat this screen unlocking procedure only if you turn the camera off and on again or if you switch to the uncropped mode and then again to the 5x-crop mode.

I hope, this helps and keep my thumbs pressed for more achievements from you !   

[dfort]

Tried compiling and enable a version from dfort branch crop_rec_4k_7D_wip...

Attacking this all at once is overwhelming, believe me I tried.

@Danne -- You discovered one of my experimental branches. I was able to get the 7D to compile but there are a lot of dummy stubs in there. It was too much trying to get all of the features working at once so I decided to work on this one feature at a time. With a1ex's guidance a few of us were able to get some features working on various Digic V cameras but these older Digic IV models are more challenging.

@coco770108 -- Welcome! Good to know that someone is testing the 500D. a1ex is running that one in QEMU and much of what we know about these Digic IV LVState cameras seem to be coming from the research done on that camera.

Found myself having to restart the camera after each recorded clip or the next clip would get that "earthquake shaking"

I noticed that shaking too. Seems like the sync issue is somewhat random but restarting clears it up? We should consider that when capturing the logs.

@IDA_ML -- We've had some private conversations so I know you are a scientist and your reports are the most detailed. I'm not adding any crop_rec_4k features yet and the only things that have changed recently were the addition of a couple of modules that should not change anything unless you enable those modules, in theory at least.

Testers - what would really help is if you could provide the logs that a1ex is asking for:

The keys for solving the sync issue are in that big LVState diagram, the debug logs from LiveView (dm-spy-experiments branch; for models other than 500D you will have to find some stubs) or EDMAC activity logs/graphs (currently captured for 50D, but incomplete). I hope to be able (but cannot promise) to solve the sync issues after analyzing all these logs.

All the logs have to be captured in 3 LiveView modes (see #1450) in order to have something to compare. Repeat for all video modes (1080p, 720p, 5x, whatever else your camera has). Repeat for all camera models (not just 50D). The two logging tasks are also for camera models already present on the Experiments page (for double-checking and for having something to compare against).

This will be more complicated. Andy600 has already created some of these logs on the 50D. I know that Danne, DeafEyeJedi and others have submitted logs - IDA_ML you can do this! I'm currently working on getting those missing stubs and will post a special version of the dm-spy-experiments branch.

I haven't tried it yet but if anyone wants to get a jump on this here's my dm-spy-experiments work in progress branch.

[Danne]

I see. I was aiming for to inject ErwinH mlv_lite.c and mlv_snd.c into that specific version. Great work by the way.

[dfort]

I was aiming for to inject ErwinH mlv_lite.c and mlv_snd.c into that specific version.

I'd like to get the new mlv_lite merged into this but haven't succeeded with that yet. Maybe someone else could have a go at it?

Got it. 0x02 flashes green but 0x01 doesn't. I'm using 0x01 on the 50D so I assume it's safe!?

@dfort this needs checking on the 5D2 with the EDMAC module.

I don't have access to a 5D2. On the 7D both 0x01 and 0x02 flashes green. The "Find free EDMAC channels" of the EDMAC module confirmed that channel #16 (0x10) "seems to work!" Others that are listed as seems to work are: 3-6, 10-13, 16-17, 19-22, and 25-29.

Compare this to the 50D:

The scan says 'seems to work' for channels 2-6, 8, 10-13, 19-22 and 24-29.

So the code to find free channels seemed to work on the 7D but we're using channel 1 on the 50D so it validated my earlier post:

You might notice that there is some code to find free edmac channels but experimenting around I found that sometimes what isn't listed might work better.

Testers -- the EDMAC module is in the latest builds that I posted:





Here are some 7D EDMAC logs ([EDIT] mv1080 video mode):

EDMAC model test
50µs without raw video and without photo raw overlays enabled
50µs with raw video enabled, old method (run it on top of regular nightly)
50µs with raw video enabled, new method (on top of dfort's modified version)
500µs still photo, short exposure, outside LiveView

[reddeercity]

ON 5D2 Set to movie mode -- shot a short 10bit mlv + take a Full Res Silent photo while edmac.mo was logging
edmac000_5d2.log & EDMAC.LOG from dfort builds

Code Select Expand

# Magic Lantern Nightly.2017Dec12.5D2212 (94fa79203875+ (raw_video_10bit_12bit_LVState-wip) tip)
# Built on 2017-12-12 18:21:30 UTC by rosiefort@RosieFoComputer
# Configuration saved on 2017/12/13 20:06:55

Channel#16 has the buffer I use for UHD , but I see I bigger buffer at EDMAC#12 that I can try out , lot of info -- need to study more




Misc. Screen Shots while EDMAC Logging -- Lasts image after I took a full res silent photo



Edit: Also took a plain silent photo and notice it was 1880x1248 instead of 1880x1250 which I though was strange .

[Andy600]

Channel#16 has the buffer I use for UHD , but I see I bigger buffer at EDMAC#12 that I can try out , lot of info -- need to study more

Show EDMAC channels (EDMAC module settings) and push the joystick up. Does the channel flicker green? If it does you can't use it.





e.g. Ch2 here does flicker green but Ch1 does not.

[ilia3101]

The "Find free EDMAC channels" of the EDMAC module confirmed that channel #16 (0x10) "seems to work!" Others that are listed as seems to work are: 3-6, 10-13, 16-17, 19-22, and 25-29.

About which camera? Are you on about the yellow <w!> channels? Is it worth trying a few of those out?

Incase anyone missed it: 5D2 EDMAC screenshot (with some info to read).

[dfort]

I'm testing on a 7D.

You can try other channels but the main issue seems to be sync, not the channel we're using.

It would be great if you could run those EDMAC module tests and post logs for the 5D2.


Sent from my iPhone using Tapatalk

[IDA_ML]

Dfort,

I tried to generate some logs on the 7D but I am not sure if I am doing this porperly.  Here is what I did:

1) I activate "Start logging"
2) I go back to the RAW or MLV video mode and see the "Press shutter halfway to start" message
3) I press the record button to start video recording and then I press half shutter
4) I get the "Out of memory message".  No log file is recorded.

A log file gets recorded only if I skip step 3) above but is it really supposed to work that way?  Isn't it supposed to log during real-time video recording to understand what is going on?

I also looked at some of the log files that you provided but this is like reading Chinese to me.  Also, my old eyes cannot read these sequences of small numbers and text any more.  I am sorry to say that but I cannot help here either.

Please provide more detailed instructions on how this logging procedure should be run and also EXACTLY at what camera settings, (more details please, on how the camera should be set).  We testers cannot provide useful information if we do not understand how things work.

[dfort]

...
3) I press the record button to start video recording and then I press half shutter
...

A log file gets recorded only if I skip step 3) above but is it really supposed to work that way?

I believe so.

Isn't it supposed to log during real-time video recording to understand what is going on?

Not for this test, though I'm not 100% sure.

I also looked at some of the log files that you provided but this is like reading Chinese to me.

If you read through the previous posts you'll see that a1ex runs these logs through a script that creates graphics that clear up what is going on. This is what we found out with Andy600's 50D log files:

Pictures: plain LiveView (1) vs LiveView with raw enabled using new method (2, 3):







Notice how channel #2 writes some data into RAM, but not for every single frame. It's transferring every now and then, either 32x15 or 32x225 or 32x240 bytes.

As you can see this is much easier to read.

Next steps to take:

What exactly it's doing? To find out:
- capture a log using dm-spy-experiments branch, make sure to include EDMAC setup functions (SetEDmac, StartEDmac etc) and see what debug messages are nearby
- dump the data from that memory address and attempt to make some sense out of it

We're missing some stubs for the 7D, 50D and 1100D in the dm-spy-experiments branch. That's what I'm working on. Should have it in a couple of days or so.

By the way, I've been thinking about that old build and will do some more testing on it. It only works in zoom mode but maybe by changing the EDMAC write channel it might work in mv1080 and mv720 modes—and probably stop working in zoom mode? Also, the FPS override might be a key to solving the sync issue.

[IDA_ML]

Thanks for this clarification, Dfort.  Complex stuff!  It will be very difficult to handle without A1ex's contribution, I think.  Good that he is willing to help!

By the way, yesterday I talked to a friend of mine who uses his 50D extensively for RAW video work. I can't tell you how happy he is with your latest developments on his favorite camera.  So, what is going on in this thread makes a lot of sense to many people all over the globe, believe me!