crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

[SanchYESS]

Sorry, I don't edit video, and I have no plans to learn this overnight; will leave it to the community for now. By design, all video and audio frames are timestamped at microsecond level. However, whether your workflows use this metadata or not, I have no idea.

Adding pre-recording support will require implementing the same logic in mlv_snd; with the vanilla version from g3gg0, the easiest thing to do was to simply record all the sound (including the paused sections from the video clip) and let the postprocessing tools figure it out later (from timestamps). The only side effect is a bit of storage space wasted, and a bit of extra burden on the postprocessing tools.

What I can try would be to add an option to mlv_dump to discard the extra audio data (not matched by any video frames), but I'd be surprised if there aren't any post workflows that can already handle this.

Offset is ufortunately slightly different.
I currently have max 133 clips but barely the third of them needs to be synced so I can do it manually (and I have separately recorded audio so anyway I have to) but it would be great to add option to sync audio while converting with mlv_dump. Anyone knows if some other tool can use timestamps to sync?

[Danne]

Is audio off in the beginning? Could you upload a sample MLV with unsynched audio? Let's check time stamp.
By the way. What converter are you using to process dng files? Mac, Win? You mention mlv_dump. Tried Switch, Mlv app, mlvfs, mlvp, raw2cdng to mention a few...

[pc_bel]

Hi Danne,

I can send you a out of sync mlv file.

I send you a PM.

Thanks.

[theBilalFakhouri]

Need opinion please.
I know they are the same performance with ML, but wich is the best choose EOS M, 100D or 700D ?

If the size does not matter to you:

For photography EOS M doesn't have built-in flash, both EOS M & 100D can shoot continuous 4fps, the 700D can shoot up to 5fps + it's have fully articulated screen which is very useful feature to see what's happening on the screen without external monitor in different types of shooting like low/high angle and shooting yourself ..etc.

In 1080p (actual resolution will be 1736x976) RAW video mode for EOS M the camera shoots with 5x3 bining pattern which means you will have more aliasing, moire and losing in vertical resolution, the solution was to enable crop_rec module, and re-enabling 3x3 bining pattern but you will get very wide screen (1736x688) unlike Canon 100D & 700D you can shoot normally in 3x3 bining without this things.

This the only differences, I went for 700D only for the screen 

You can ask more questions in the right place at General Chat thread

[Danne]

Got a MLV from pc_bel so I thought we´d look at metadata. Are we looking at timestamp of AUDF and VIDF? If so.
First audio block:

Code Select Expand

Block: AUDF
  Offset: 0x12b5ab00
  Number: 187
    Size: 38656
    Time: 9.071000 ms
   Frame: #0000
   Space: 232
First video block:

Code Select Expand

Block: VIDF
  Offset: 0x00000600
  Number: 13
    Size: 1815040
    Time: 89.100000 ms
   Frame: #0000
    Crop: 152x132
     Pan: 152x133
   Space: 32

Audio starts:
Time: 9.071000 ms
Video starts:
Time: 89.100000 ms

89.100000-9.071000=80.029
Audio starts 80.029 ms too early, is this correct information? Is audio always early or could video block start before audio under any circumstance?

[OlRivrRat]

      @Danne

   I'm guessing that, since You Wizards of ML Are Way More Savvy than I on

the Inner Workings of Our EOSs, the following has already been considered >

   "Audio starts 80.029 ms too early, is this correct information?"

I would guess that the Audio Starts Being Recorded "On Time" & that the Video

Starts Being Recorded "Late", due maybe to the complexity of both & the time it

takes for the Processor to Deal With the Extra Complexity of the ML Vid'.

   Could it also be that the Audio Recording is being handled "By Canon" while

Video is being handled "By ML".

   Probably obvious that I possess No Savvy about this stuff > Just some ideas

I thought I'd throw out there.

                     ORR ~ DeanB

[pc_bel]

Is audio always early or could video block start before audio under any circumstance?

I'm searching in my mlv files and at this time, all the clips have de audio late. Keep on searching.

[pc_bel]

Well,
After viewing 30-40 mlv files, few have audio in sync and the rest are out of sync. Always audio late. All clips are recorded same day with same adjustments in camera, so I can't explain why the differences in audio sync.

By now only solution is sync them in post. Hope some dev find a solution.

Thanks, specially to Danne for his interest!!!

[a1ex]

Took a closer look and hopefully managed to get a slightly better audio sync. Caveat: I only know this from debug messages; I'm afraid I don't have the equivalent of pixel peeping skills on the audio side (to tell whether audio is in sync or off by some split-second), sorry.

If my understanding of the video and audio pipelines is correct, then:
- video frames are timestamped as soon as the image is read out (evfReadOutDoneInterrupt); the capture process probably happens a little earlier, unsure about the exact timing
- audio frames are timestamped right after StartASIFDMAADC (when the audio recording starts)
- first audio frame is now started shortly after first video frame was captured (~ 2ms in my tests, but I had the trace module active)

It would have been probably better to timestamp the beginning of the exposure, but finding that requires either polling the EDMAC channel that brings the raw image into RAM, or a better understanding on how LiveView works, or - the easiest way - estimating it from FPS and rolling shutter info. Are these ~20ms or so important when it comes to audio sync?

[dfort]

Are these ~20ms or so important when it comes to audio sync?

Finally, a question I might be able to answer with some degree of confidence. When syncing film dailies we did our best to get sync to the nearest perf. There are 4-perforations per frame of 35mm film so at 24fps that's the equivalent of about 10.5ms, right? Well nobody can actually tell if sync is off unless they examine the waveform (audio pixel peeping) up to about 1-frame so that would be 42ms. When we went digital we were willing to accept sync within 2-frames which the average person can't detect so that's 84ms. I've worked on TV movies where the dailies came in with the sync off 3-frames and that didn't seem to bother anyone except the assistant editor--that would be me.

So no, those ~20ms or so variations shouldn't matter. That's not even a frame out of sync.

[Audionut]

Flicks?

[dfort]

Anything within 29429400 flicks is golden.   

[Danne]

@A1ex:
2 ms is more than enough for synch. Thanks a lot for fixing.

I added millisecond cutting support in Switch thanks to ffmpeg so audio beginning should be spot on now when processing MLV files into dng files. Should work for all MLV files but needs to be tested.
Commit:
https://bitbucket.org/Dannephoto/switch/commits/a253b813e80b7c4048799847f9e58681952b343e

Download section:
https://bitbucket.org/Dannephoto/switch/downloads/

*Update
Not working yet! Will fix the bug soon...
FIXED!

[theBilalFakhouri]

The module adtg_gui.mo needs to be fixed with latest updates.

[pc_bel]

Thanks A1ex for fixing the audio delay!!!!

Still not tested but I'm sure is ok.

Thank you!!!

[IDA_ML]

Question to A1ex,
===========

This has probably been discussed already but I couldn't find where.  So, I apologize for asking.

The 3K/4K_crop_recording mode is fantastic since it provides high-resolution RAW video with superb quality and detail, however, it has one serious drawback.  The crop factor imposes some serious restrictions on the lenses used and the videographer needs high-quality fast wide angle lenses to be able to compensate for it. These are expensive.  On the other hand, if one films in-door at low light, excellent low-cost lenses like the 50/1,4 or 85/1,8 may become too long and unusable due to the crop factor.   

I have been going through the first post of this thread as well as the explanation concerning 4K RAW recording on the Experimental page and there are two statements about the 5DMkIII that I got very interested in:

1) Continuous: 3072x1308 @ 24p, 1920x1080 @ 45p, 1920x960 @ 50p, 1920x800 @ 60p, 4096x2560 @ 8p, 5796x3870 @ 5p

2) Full-width LiveView - decrease vertical resolution in the crop_rec submenu, all the way to 5796x400 @ 48 fps

My question is:
=========

If the 5DMkIII can record in 4096x2560 @ 8 fps and 5796x400 @ 48 fps wouldn't it be possible to make it record at a more usable and higher than FHD resolution and 24 fps WITHOUT the crop factor or with a much smaller crop factor?  This is possible already at 1920x1080 (FHD) resolution without aliasing.  What about 3072x1728@24 fps or 4096x1744@24 fps for a limited time of course?  Or maybe 2900x1812@24 fps for continuous recording?

Additional question:
---------------------
The maximum vertical resolution of the 7D in the 5x-magnification mode is 1200 pixels while it is only 1080 in the 100D.  What restricts this number to 1080 in the 100D if both sensors are APS-C and have about the same pixel count?

Thank you in advance for your reply.

[a1ex]

1) From first post:

* 3072x1920 @ 24p (1:1 crop)
* 4096x1440 @ 25p (1:1 crop)

You can further crop any of these from the raw video options.

Column binning factor is limited to either 1 or 3; line skipping/binning factor is a bit more flexible, but...

2) Without crop_rec, all video modes are configured by Canon code (whatever resolution they provide, is the maximum we can record).

[Levas]

Wanted to try out the slightly better audio sync changes on the 6d, but got some errors.
(Compiled the build myself, so not sure if I did everything right, but I guess that's not the problem here.)

On screen it says, 'Data corruption at slot 36' for every frame.
This error only happens with lossless recording options.
Everything is fine with normal 14 to 10 bit recording.
Unfortunately I can get the 'screenshot after 10seconds' option to work.

Here's the  'Data corruption at slot 36' for every frame error log

Error logs:

Code Select Expand


ML ASSERT:
0
at mlv_lite.c:2630 (compress_task), task compress_task
lv:1 mode:3

compress_task stack: 1e4340 [1e43d0-1e33d0]
0x0044C93C @ b7f0a4:1e4370
0x0044C478 @ 44c998:1e4340

Magic Lantern version : Nightly.2018Feb20.6D116
Mercurial changeset   : 02e5918a6ed5 (crop_rec_4k_mlv_lite_snd)
Built on 2018-02-20 19:00:42 UTC by magic_lantern@Vasses-iMac.
Free Memory  : 343K + 2335K


And the camera crashes when I look into the 'Free memory dialog', here's the crash log.

Code Select Expand


ASSERT: hSuite != 0
at RscMgr.c:2228, guess_mem:ff0f41ec
lv:1 mode:3

guess_mem stack: 1e32f8 [1e33c8-1df3c8]
0xUNKNOWN  @ ea80:1e33c0
0x0045C15C @ 44db3c:1e3378
0x0045C0E8 @ 45c1f4:1e3368
0x0045BFC8 @ 45c13c:1e3358
0x004A3150 @ 45bfe8:1e3350
0x00001900 @ ff0f41e8:1e3330
0x0044C478 @ 44c4f4:1e32f8

Magic Lantern version : Nightly.2018Feb20.6D116
Mercurial changeset   : 02e5918a6ed5 (crop_rec_4k_mlv_lite_snd)
Built on 2018-02-20 19:00:42 UTC by magic_lantern@Vasses-iMac.
Free Memory  : 336K + 2267K


[rob_6]

Thanks for the updates on sounds syncing A1ex and everyone that helped! I have done a few tests. It looks like it is much closer to sync now, but it seems to me that the audio track is 2 frames early. I need to manually adjust each audio track 2 frames forward to get it right on. I am using 113 firmware with crop record 3.5k enabled with sound. I am also using MLVFS for processing. Anyone else getting the same result?

I hope this helps. Let me know if there is something else I can test to help narrow down the offset.

Thanks!

Rob

[dfort]

(Compiled the build myself, so not sure if I did everything right, but I guess that's not the problem here.)

Actually it looks like it is. The 6D hasn't been merged into the crop_rec_4k branch yet. I've got an open pull request but haven't merged in the mlv_lite_snd changes to it yet.

[vstrglv]

Thanks for the updates on sounds syncing A1ex and everyone that helped! I have done a few tests. It looks like it is much closer to sync now, but it seems to me that the audio track is 2 frames early. I need to manually adjust each audio track 2 frames forward to get it right on. I am using 113 firmware with crop record 3.5k enabled with sound. I am also using MLVFS for processing.

I have tested 1920x1080@25fps, 1/60, 14bit,12bit,10bit, 14lossless,12lossless,10lossless,  FPS-off, 5DMkIII 113 firmware. Sinc is better  than 1 frame. magiclantern-crop_rec_4k.2018Feb17.5D3113, mlv_dump.exe

[pc_bel]

Tested!
Audio for me is now in sync!!!
5Dmk3 1.1.3 Crop_recFeb17th
Thanks.

[pc_bel]

Looking closer, audio is 2 frames before images as Rob_6 said.

Much much better than before. Thanks.

[IDA_ML]

Column binning factor is limited to either 1 or 3; line skipping/binning factor is a bit more flexible, but...

Thank you for your reply, А1еx.

What about a combination of the two binning/line skiping factors 1 and 3?  Wouldn't it be possible to implement an algorithm that randomly alternates between 1 and 3 during sensor sampling?  This technique is commonly used for weighting interdigital transducers and reflector gratings in surface acoustic wave (SAW) devices to achieve certain filter or reflection characteristics and/or to suppress unwanted interference.  It is also used for precise adjustment of the center frequency in narrowband SAW devices such as resonators which is hard to do due to discrete step size of the pattern generator for photo masc fabrication.  If for example, a resonant frequency of 433 MHz is needed, the pattern generator can provide only 430 or 440 MHz due to its discrete step size.  The solution is a random combination of these two consecutive steps to achieve precisely the desired 433 MHz.  Some of the transducer/reflector vertical strips are sampled at 440 MHz and the rest - slightly more often at 430 MHz.  In my opinion, if implemented in sensor sampling, this technique will not only solve the crop factor problem but also reduce moire and aliasing when line skipping.  Just a thought.

[a1ex]

Not sure if this parameter can be altered during image capture, and even if it does, it would have very tight timing requirements. Normally, the main CPU configures the exposure parameters, starts it, and waits until it's finished (actually it has a lot of other tasks to run during this wait).

We have no datasheet for the ADTG chip (the timing generator that handles line skipping/binning); all we know about it is what we have found by trial and error (adtg_gui), so I have no idea where to start in order to configure the chip for variable line skipping (or whether it can do that).