Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)

[ted ramasola]

Tested the Jan 25 Nightly on the 5DmkII.

With MOVIE RESTART=ON MLV raw video will not stop recording. If you press SET to stop, it will start by itself.
This issue is not present with older builds like the Jan 6th NB and by a.d.
This is suppose to be active in H264 recording and should not have an effect in Raw recording.

HDMI MONITORING TEST
Tested playback functionality.
Immediately after recording a raw clip, I went to File manager> view mlv file.
Playback of COLOR or FAST was possible and it showed on HDMI monitor.
However after EXIT of mlv playback screen went Black and no function possible.
Had to remove battery to reboot camera.
On Older build using Jan 6 nB or A.D. Playback was possible and EXIT from playback was OK,
liveview came back and additional recording was possible.

CAPACITY BEYOND 40GIG TEST
Jan 25 Nightly build will still skip and stop approx, 39GB (Canon menu), 35.9GB (PC properties, 38,582,760,448 bytes)
The Jan 6 NB and by A.D. does not have this issue and will record till your CF card is full.
NBs after Jan 6 up to the 25th will Stop at approx that size or around 10minutes. This also applies to the 7D.


...

[Shield]

Out today shooting some 2-4 min clips.  One clip was 3:16 and the associated .wav file was 3:07.  It just didn't record the last 10 seconds for some reason.  I was able to sync it up with my external source so I was able to determine that was the problem.  Clips just before and after this were fine, and there was no lockup or any other problems.
Would anyone like me to upload the mlv file somewhere?

[ted ramasola]

Out today shooting some 2-4 min clips.  One clip was 3:16 and the associated .wav file was 3:07.  It just didn't record the last 10 seconds for some reason.  I was able to sync it up with my external source so I was able to determine that was the problem.  Clips just before and after this were fine, and there was no lockup or any other problems.
Would anyone like me to upload the mlv file somewhere?

on my tests using 7D and 5DmkII using various resolutions the results are always as follows.

when recording MLV+SND, the wav file would always be
LONGER by 7-10 frames.
It would be ahead by 1-2 frames from the video so it needs to be nudged back to sync.

Knowing this and you won't have problems syncing in post. H264 was also off by 1-2 frames so this is not an ML issue.

However, if your recording stops ABRUPTLY, due to card full for instance, the VIDEO would be LONGER and the WAV file much shorter.

[g3gg0]

However, if your recording stops ABRUPTLY, due to card full for instance, the VIDEO would be LONGER and the WAV file much shorter.

yeah thanks for the feedback.when card gets full, I cannot clear buffers anymore.
then you lose some megabytes of audio.
not good, right. But I don't know a quick solution.

[Audionut]

Can you monitor the file size and end recording before card full, giving a chance to clear the buffers?

[a1ex]

With raw_rec, I create a backup.raw file for this. Whenever the card gets full, I delete that backup file and try to save the buffers again.

Last time I checked (months ago) it was working quite well.

@Audionut: this requires keeping track of all file writes since the last time Canon code reported free space. Possible, but requires some major refactoring (maybe a file i/o backend that wraps around Canon calls in a way similar to memory backend).

[Danne]

I filled my 64gb card three weeks ago and with Alex wizardry (I think backupsolution) I managed to extract my files. There was even a messaged asking if my card was full .

[Canon eos m]

Posted here as advised by Ted:

Ted, also to add if the raw video is taken when the sd card is the preferred card, the 5D3 hangs. This is technically a glitch.

While I understand you can set your preference and all but I am sure there is a programming way to prevent the sd card from leading when it clearly is not meant to.

My take is that it will discourage mass adoption of the ML platform small things like these are not addressed. Pretty much like texting changed for ever when predictive texting was introduced.

[ted ramasola]

To add my test on the 7D, testing Jan 25 TL.

MLV Raw+Snd

It still skips and stops at approx 10 min or 40,929 MB ( 39.9GB )
Even though it recording constantly at 77.3/80/76 MB/s 9%idle.

The last Tragic Lantern version that does not stop at 10 minutes and will fill up card is Jan 7.
(This probably coincides with the Jan 6 a.d build for the 5d2-last build with no skip/stop problems)

after that up to jan 25 will skip and stop at 10min or 40gig.

Also to add an observation which I hope will help, I reverted back to Jan 7 TL and did a capacity test again.
I observed that the LV info keeps on updating constantly and the CF card activity led light keeps flashing but upon reaching approx 10min
or 40gig which is the threshold for other builds before stopping, I notice every info stop updating and the CF activity led stops flashing, like the cam froze for a few seconds,
but after maybe 2 and 1/2 seconds the activity and led flashing continuous and so does recording.

[Shield]

on my tests using 7D and 5DmkII using various resolutions the results are always as follows.

when recording MLV+SND, the wav file would always be
LONGER by 7-10 frames.
It would be ahead by 1-2 frames from the video so it needs to be nudged back to sync.

Knowing this and you won't have problems syncing in post. H264 was also off by 1-2 frames so this is not an ML issue.

However, if your recording stops ABRUPTLY, due to card full for instance, the VIDEO would be LONGER and the WAV file much shorter.

The 5d3 is different; never once had the audio (at least lately) not sync up perfectly with the video.  This wasn't an issue of running out of card space, lockup, heat, anything else.  Let's say in the span of 15 minutes I recorded 4 two-three minute clips all without shutting off the camera.  Clips 1,2,4 were fine.  Clip 3 the audio shut off about 10 seconds early.  No idea why.

[shinokubo]

You could use this MLVbrowsesharp to convert your mlv to raw: http://www.magiclantern.fm/forum/index.php?topic=8447.0

Then use RAWanizer to export proxies and the DNGs. http://www.magiclantern.fm/forum/index.php?topic=5557.0

Thanks for the reply Ted,

I do make my proxies with Resolve right now, but that way I can't have them with audio. Rendering the individual source clips tries to render the sources (.dngs) audio, which is non existent.

As I am on OSX I couldn't try RAWnizer, but by having a look at it, the routine is not gonna give me proxies with audio either.

We were shooting with a slate, so we did synchronize by hand which was time consuming with several hundred takes.

I'll have another look at the mlv_snd module thread, maybe it's already clear how big the offset in the beginning of the clip is. That way we could produce proxies with no sound through Resolve and merge them with the appropriate audio with a ffmpeg script.

I'll keep you updated if I find a proper solution.

[QuickHitRecord]

I just updated to the Jan 27 ML nightly build for the 5Diii with the MLV_SND module. Everything seems to be working well so far except for one thing:

When recording MLVs (Global Draw ON, all modes; Global Draw Disable OFF), I have the crop overlays, audio meters, raw histogram, etc. available to me when I first turn the camera on. But as soon as I start recording a clip, they disappear, and they do not reappear when I stop recording. I have to turn the camera off and then on again to get them back. Anyone know what is going on?

I can confirm that this was not an issue with the Jan 7 nightly build that I was running before I updated.

UPDATE: I've been able to resolve this. Turns out that I had my card on write protect, so it was reverting to my old settings every time that I turned it off. Unlocking it, turning Global Draw ON, Liveview seemed to address the issue.

[crazypig]

Dear developers,

Is it possible to add TIMECODE in MLV file and the Audio WAVE file?

So we can auto sync the audio in Davinci Resolve...

5D3 can render timecode for h264 video.

Best regards,

Sean

[bnvm]

Dear developers,

Is it possible to add TIMECODE in MLV file and the Audio WAVE file?

So we can auto sync the audio in Davinci Resolve...

5D3 can render timecode for h264 video.

Best regards,

Sean

Yes this would be awesome and a huge time saver.

[tonybeccar]

Hello! I made a batch mlv to dng converter for windows!!! Take a look!

http://www.magiclantern.fm/forum/index.php?topic=10198.msg98284;topicseen#msg98284

[g3gg0]

Yes this would be awesome and a huge time saver.

maybe i dont understand - but the audio is already in sync to video. maybe a frame off as i've read.
also MLV has microsecond accurate timestamps per frame.
not sure what to add else...

[andy kh]

maybe i dont understand - but the audio is already in sync to video. maybe a frame off as i've read.
also MLV has microsecond accurate timestamps per frame.
not sure what to add else...

let me explain ..as of now we have to sync audio manually by draging the audio below the video clip for every clips but with time code all video and audio can sync automatically so it saves lot of time

[kgv5]

I think it is impossible because mkv gives separate frames  (NLE treats them as an image sequence) plus separate sound file. Having all synced on the timeline from the beginning is possible only when sound and image are "baked in" into one file.

[crazypig]

Black Magic Cinema Camera is using TIMECODE to sync the CinemaDNG file and Audio file in Davinci Resolve

In Black Magic Cinema Camera, you can also output the timecode to external audio recorder...and sync it in post easily.

Now ML RAW just like Black Magic Cinema Camera. Recording DNG files and a WAVE audio file....(only different is without timecode for the DNG files and WAVE file, so we can't sync it easily by software)

Because there have at least 100 footages in each video production project. If we sync each footages by manually, it take a long time.....

 g3gg0~~~ Please add the TIMECODE in each MLV file...

[g3gg0]

g3gg0~~~ Please add the TIMECODE in each MLV file...

as ive already described, MLV file format has MICROSECOND EXACT TIMESTAMPS FOR EVERY KIND OF INFORMATION. *
just in case i didnt emphasize it strong enough.
and as it already has this information, it doesnt make sense to add any more timing information.

when exporting into DNG, you separate that timestamps from each other.


i dont have any of the tools that you use, but cant you simply tell it to pick up the .dngs and the .wav in one directory as a single A/V pair?


*=depending on some circumstances it may happen that you are 1-3 frames off.

[1%]

I think his issue is that none of the tools support any of the timecode info. Stuff like H264 can have a timecode track the NLE can read but a sequence of DNGs and a wav file do not. I'm not aware of any separate TC file format which would solve this unless the video was converted by MLV dump into video/audio/TC in one file.

[g3gg0]

thats the point.
MLV has that timecode information implicitely.
there is real time clock plus the timer value on every block start.

i dont know how the tools would recognise the information from DNG and WAV only.
but the information is there and is happily printed when you pass -v as parameter.

[crazypig]

thats the point.
MLV has that timecode information implicitely.
there is real time clock plus the timer value on every block start.

i dont know how the tools would recognise the information from DNG and WAV only.
but the information is there and is happily printed when you pass -v as parameter.

Dear g3gg0,

I try to send you Blackmagic Pocket Cinema Camera CDNG and WAVE file to you tomorrow .

Maybe can find out how they paste the timecode information that can recognise in Davinci Resolve.

Really really thank for your great help.

Best regards,

Sean

[kgv5]

@ g3gg0

I dont know if this is a known bug but i have it constantly and i think it is easy to reproduce (my camera is doing this every time)

5D3, ML build 28-01-2014 (as far as i remember it was also present in the previous builds).  CF komputerbay 1000x 64BG, Sd sandisk extreme 45mb/s.

When I record with mlv_rec and have sound recording ON and when i am in the 3X crop mode camera hangs every time when frame skips and than requires battery out. Every time camera is not responding. FPS override doesnt matter. It is like that ONLY when sound recording is ON (tested with 48khz). When it is off than after frame skip camera normally comes back to idle. 
This can be extremely annoying especially when somebody forget to turn on FPS override before entering crop mode  - with default 30fps it skips quite fast. This can really ruin some important shot because bringing the camera to life lasts 20-30 seconds.

[g3gg0]

@kgv5:
thanks for the detailed bug report, will check tonight.

@crazypig:
hm as CinemaDNG is another story which chmee is a specialist for,
i am not sure if i can really implement it.
but i can have a look into it.