Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)

[Canon eos m]

Hi g3gg0,

I have the 10th April build of the bleeding edge on my 5DM3.

For some strange reason the frames skip massively.

Where could I send you a sample. I will record and reproduce the skips and also note the settings that I get. All I can say is that the mlv version is still a bit unstable for my use. Maybe it is me doing it all wrong.

So, please let me know where I could send in a sample with the settings at the time of the problem and I will do so. 

[Canon eos m]

Please ignore this post since I was trying to post the error from the mlv_dump command but failed to attach an image. Admin please delete since this is a unintended post.

[Canon eos m]

Please ignore this post since I was trying to post the error from the mlv_dump command but failed to attach an image. Admin please delete since this is a unintended post.

Edit:

Okay, the command I type is:

C:\>mlv_dump M13-0937.mlv -f [start]-[end] -o M13-0937.mlv

The output that I get is:

Mode of operation:
-Input MLV file: 'M13-0937.mlv'
-Rewrite MLV
-Output into: 'M13-0937.mlv'
File M13-0937.mlv opened
File M13-0937.m00 opened
File M13-0937.m01 not existing
File M13-0937.IDX opened <XREF>
Processing .....
Reached end of chunk 1/2 after 0 blocks
Reached end of chunk 1/2 after 0 blocks
Processed 0 video frames
Done

[g3gg0]

which values did you insert for [start] and [end] ?

[Canon eos m]

Tried all 0 to 40 for start and end respectively.

[g3gg0]

this please:
C:\>mlv_dump M13-0937.mlv -f 1-3 -o M13-0937_out.mlv

please never specify input and output the same file!

[Canon eos m]

this please:
C:\>mlv_dump M13-0937.mlv -f 1-3 -o M13-0937_out.mlv

please never specify input and output the same file!

same error as before.

[g3gg0]

then please post the output of mlv_dump -v M13-0937.mlv on pastebin.

a) please the full log
b) make sure this is your original file as i expect the one you have used previously is overwritten as you specified the same file as input and output

[Canon eos m]

then please post the output of mlv_dump -v M13-0937.mlv on pastebin.

a) please the full log
b) make sure this is your original file as i expect the one you have used previously is overwritten as you specified the same file as input and output

http://pastebin.com/vLCTrXTf
File name: M13-0948
Total Frames: 318
Black Frames: (58, 64,65,66,77,78,87,88,99,100,102,118,119,120,134,135,140,150,151,157,158,163,169,170,182-185,207,208,214,215,230,231,245,246,250,251,268,269,275,276,288,289,298,306,307,311-313

[g3gg0]

thats only part of the output. please dump everything.

you can e.g. dump the output of mlv_dump into a file using "> log.txt"

[Canon eos m]

thats only part of the output. please dump everything.

you can e.g. dump the output of mlv_dump into a file using "> log.txt"

I copied everything that I could see 

Is there a tut that I can first attend to learn how to capture the entire log and using log.txt to dump the output because it is really not proper for me to take up your time teaching basic stuff to folks like me.

All I can say now that after being following ML for about 5-6 months regularly, I am beginning to get a hang of this stuff. I know I am almost there. There are only two things that are stopping me from realizing the full potential of this magical firmware (footage wise): (i) the frame skipping and (ii) sound recording.

Once I get this done, I am ready to take on the world. 

[g3gg0]

just add a > log.txt behind the command.
then it will dump the text into a file named log.txt

what do you mean with frame skipping and sound recording?
is there some trouble?

edit:
or just upload the footage somewhere, so i can look myself.

[Canon eos m]

just add a > log.txt behind the command.
then it will dump the text into a file named log.txt

what do you mean with frame skipping and sound recording?
is there some trouble?

edit:
or just upload the footage somewhere, so i can look myself.

I meant the black frames (sorry for using the wrong terminology) and sound recording is a bit odd for me - get it sometimes and don't in other situations. But frankly, was not keeping a close watch. Will now onwards record the settings and failed attempts at recording sound if any.

Still at work. Will reach home in about an hour and will action the log.txt command and generate the dump to send to you.

Will also upload a part of the footage with black frames if I can tonight for you to see and guide.

[g3gg0]

are you sure you didnt activate card spanning and forgot to copy the video files on the other card?

[Canon eos m]

are you sure you didnt activate card spanning and forgot to copy the video files on the other card?

Card spanning is always ON when I use MLV. Not sure what you mean by forgot to copy the video files on the other card.

[Jockerl]

Card Spanning means that the files are written to the CF AND SD card to archive higher speeds. When you convert the mlv-files you need the files from the SD card, too.

[Canon eos m]

Card Spanning means that the files are written to the CF AND SD card to archive higher speeds. When you convert the mlv-files you need the files from the SD card, too.

OH NO!

I knew what spanning was ... but didn't realise that files from both cards had to be used. Let me try to record a few clips first and use files from both cards before I bother g3gg0. Maybe, that is why I have some files that are clean with sound while others have either no sound or black frames. Not sure but let me see this was the reason.

[Canon eos m]

just add a > log.txt behind the command.
then it will dump the text into a file named log.txt

what do you mean with frame skipping and sound recording?
is there some trouble?

edit:
or just upload the footage somewhere, so i can look myself.

smallest file I could find with the black frame problem: https://www.dropbox.com/s/a6yk1qot0u5kt18/M13-0935.MLV

[KMikhail]

A few comments on the already exceptional piece of programming mastery:

I have CF 128GB 1066x KB card and SD 128GB 600x Transcend (GD is on - peaking and histogram, only video MLV  mode considered)

a) 16384K buffer is generally slower than 15625K with my CF only (up to 88 MB/s, after several minutes of warm up).
b) In burnout mode some of the 25MB+ buffers achieved 90-91 MB/s in writing.
c) Going to 30fps eats a lot of bandwidth, easily 5-15MB/s for CF.
d) The fastest speed with SD I've seen was 18.5 MB/s, 20.1 MB/s reading.
e) Turning spanning on will degrade CF performance further and as a result SD card (a tad slowed too) doesn't help a whole lot.

Is there any way to select buffers for CF and SD for independent or spanning modes?
Sometimes I think having 1:2.40 would be nice, since 1920x800 is nicely divisible by 16 and actually used by some BDs.
I am still not entirely sure how to absolutely 100% guarantee video-to-sound sync. There is usually a difference in length between two tracks. And I strongly dislike this 24 vs. 23.976 thing, though I understand where it comes from.

At this point I'd say the only thing truly missing is ability to get 30fps with spanning, as I guess it is unreasonable to assume that any single card will sustain it along with a stable MLV with sound.

[g3gg0]

thanks for your feedback.

Is there any way to select buffers for CF and SD for independent or spanning modes?

right now -> edit source code and recompile.
its hard to make that UI-editable, but i can think of a config-file only option.

Sometimes I think having 1:2.40 would be nice, since 1920x800 is nicely divisible by 16 and actually used by some BDs.

same as above

I am still not entirely sure how to absolutely 100% guarantee video-to-sound sync. There is usually a difference in length between two tracks. And I strongly dislike this 24 vs. 23.976 thing, though I understand where it comes from.

yeah, the audio is just longer in the end. its totally in sync, but i can stop audio recording only in 250 ms steps or so.
so you just have to cut the end.
it however *could* change with the new audio api where it is possible to cancel while a transfer is in progress.
but consider that as highly experimental and not even ready for alpha testing.

At this point I'd say the only thing truly missing is ability to get 30fps with spanning, as I guess it is unreasonable to assume that any single card will sustain it along with a stable MLV with sound.

yeah SD transfers while capturing video and CF card transfers will slow down everything else.
this is relateed to EDMAC/CDFMA modules which cause load on the SDRAM interface.

[Canon eos m]

smallest file I could find with the black frame problem: https://www.dropbox.com/s/a6yk1qot0u5kt18/M13-0935.MLV

Hi g3gg0, did you get a chance to review the file I uploaded. Would really love to know what I may be doing wrong. Did search the forum for spanning related threads but did not find much to go by.

I am now thinking of experimenting with the raw_rec.mo. Have been avoiding it so far due the sound capabilities of ML. What do you think.

[Canon eos m]

are you sure you didnt activate card spanning and forgot to copy the video files on the other card?

Hi g3gg0, sorry for the trouble and BIG THANK YOU for the help. I could not figure out the mlv_dump. But my problem is resolved since I copied all the mlv related files from the SD card into the same folder where I had copied the mlv files from the CF card. The the black frames (skipped frames) problem has vanished. Obviously, there is some data on the SD card that was missing to complete the video.

I am so feeling good and the quality from my 5DM3 is breathtaking!

Awesome!  Awesome!  Awesome! 

[MGerard]

I just tried the latest nightly build and sometimes during recording (MLV_rec enabled), the console pops up (on a 5D3). Found no way to disable it unless I restart the camera...

this is the assert:

ML ASSERT:
hdr->blockSize > 0
at mlv_rec.c:1969 (mlv_prepend_block), task raw_rec_task
lv:1 mode:3


Magic Lantern version : Nightly.2014Apr16.5D3113
Mercurial changeset   : 9058cbc13fa4 (unified) tip
Built on 2014-04-15 23:11:59 UTC by jenkins@magiclantern.fm.
Free Memory  : 151K + 3815K
 

[KMikhail]

its hard to make that UI-editable, but i can think of a config-file only option.

Thanks for your answers. I actually looked up for config already, fruitlessly, so it would be very nice to have!

[g3gg0]

ML ASSERT:
hdr->blockSize > 0

fixed, thanks.