Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)

Started by g3gg0, July 15, 2013, 10:58:23 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

RenatoPhoto

Reporting 5D3 testing with sound module.

During cropped mode I cannot go above 2240x1260 pixels.  The LV has some sort of an error and quits working.  If I disable the sound module it works OK.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

g3gg0

Quote from: RenatoPhoto on December 29, 2013, 06:24:49 PM
During cropped mode I cannot go above 2240x1260 pixels.  The LV has some sort of an error and quits working.  If I disable the sound module it works OK.

thanks! will check where this comes from.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

AnotherDave

When I turn the camera on, MLV Sound & MLV Raw seem to be turned off by default.  Is this the case, or do I not have a setting flipped or something?


Jakobmen

Quote from: AnotherDave on December 29, 2013, 07:14:02 PM
When I turn the camera on, MLV Sound & MLV Raw seem to be turned off by default.  Is this the case, or do I not have a setting flipped or something?

yup for me too...its like i need to go to canon audio menu to make it come back on.
Canon 5D3

1%

7D doesn't seem to record any audio frames, dunno why I see the red buffers . Canon audio on or off. 6D I need to play with more but I did get some wavs out.

7D Log:

http://pastebin.com/8EjPPFbi


How do you stop the ADC? At first I was getting audio failed to stop 4->0. I added a stopasifdmaadc but not sure if it helps. At least on 6D it recorded more than once after that.

g3gg0

on 7D i guess it misses this part:

    /* experimental for 7D now, has to be made generic */
    /* Enable audio Device */
    void SoundDevActiveIn (uint32_t);
    SoundDevActiveIn(0);


this is something for the "next-gen" wave support in magic lantern.

sound recording is simply stopped by not further queueing any new audio buffers.
there are always two buffers (double buffered) and as soon one is filled, you get the callback
where you have to enqueue the second-to-next buffer.

if you dont queue a buffer during the callback, you will get another callback for the last buffer being filled.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Danne

Just baked in everything in the same command script to simplify batchconverting .mlv with audio on mac osx
Thanks to dubzeebass, g3gg0, jerrykil, daisermac

download following https://drive.google.com/file/d/0B4tCJMlOYfirU1FncGxENHhFeE0/edit?usp=sharing

1 – Extract recursive0.3.command and mlv_dump.osx  in the same folder
2 – doubleclick recursive0.3.command (type password. If permission needed go to your folder in terminal and write chmod +rxw recursive.command0.3 this will give all permissions needed)
3 - doubleclick recursive0.3.command again and type in password if needed. This should place your mlv_dump.osx  according to the script(mlv_dump will disappear)
4 – place your .mlv files in same folder as recursive.command.03 and doubleclick which should start converting the files to folders

I,m not a coder so feel free to critizise


1%

That stuff is old news. I have full audio support for 7D (pel should port this back). Meters are going while its recording. Its just AK4646 with a different audio task you can't override. You have to re configure it after canon like on 600D.

I can record old style wav just fine but then I record with MLV_snd and get nothing... on 6D I'm getting 4GB wavs for some reason (with data), mlv dump just keeps going and going until that limit is reached. I can't run trace because it crashes from out of memory.

If you leave the asif going it crashes out on a bunch of cams or won't let you restart it until it times out. When I run the FFT test I also get action on it on both cameras.

Also: 6D with correct .raw file and wav (5MB) in MLV browser (first recorded clip, canon audio off, audio override)
Next clip is 4gb, 930kb resaved. At least all data seems to be there on 6D.

File Header (MLVI)
    Size        : 0x00000034
    Ver         : v2.0
    GUID        : 1380004378867732245
    FPS         : 23.970000
    File        : 0 / 0
    Frames Video: 0
    Frames Audio: 0


Ok, got 1 wav out of the 7D, correctly sized and both channels out of 5 files. Canon audio off, separate wav override. Maybe its a timing issue. All entries in the log look identical:

mlv_snd_flush_entries: 105 entries to free in queue
/* 00:01:27.710749  */ mlv_snd_flush_entries: 98 entries to free in queue
/* 00:01:27.711923  */ mlv_snd_flush_entries: entry is MLV slot
/* 00:01:27.712237  */ mlv_snd_flush_entries: NULL slot 0 entry
etc....

All clips show 0 audio frames.

*update... I hear some audio in MLV viewer but wavs are not extracted by mlv browser. Audio playback is jerky... like silence silence sound, silence silence sound, etc.

zachnfine

I've run into three problems with the latest (er, yesterday's) build, I figured maybe documenting them here would help development or someone could tell me what I'm doing wrong.

1. I loaded the mlv-rec and mlv-snd modules and set it for 1920x1080p23.976 and tried recording some raw video to my Komputerbay 128Gb 1050X CF card. It gets about 48-70 frames in, reports a skipped frame or two, and stops recording. Every time with the same result. I think spanning is off, and there's only files recorded to CF, but I'm not sure (need to poke around and find that setting again). Recording at the same resolution and frame rate using the normal raw recording module works flawlessly to the same card.

2. I booted the camera with mlv-rec and mlv-snd enabled, and (without recording) hit the zoom button to see a close-up view for focusing. After focusing, I didn't hit the zoom button again to pop back to the normal view and instead just hit record thinking the view would pop back to normal and recording would commence. Instead the view froze, and remained on the screen after I shut down the camera. I had to remove the battery to get it to power off.

3. It's been noted earlier in the thread, but the mac version of mlv_dump claims the .mlv file lacks a header unless the .IDX file is moved, deleted, or renamed out of the application's view. If the IDX is present, no decoding happens. When I renamed the .IDX file to end in .IDX.bak, mlv_dump was able to see the file and I could view the metadata for its contents just fine.

g3gg0

Quote from: 1% on December 29, 2013, 08:51:33 PM
Ok, got 1 wav out of the 7D, correctly sized and both channels out of 5 files. Canon audio off, separate wav override. Maybe its a timing issue. All entries in the log look identical:

[...}

*update... I hear some audio in MLV viewer but wavs are not extracted by mlv browser. Audio playback is jerky... like silence silence sound, silence silence sound, etc.


when adding the lines i mentioned the 7D can record 5/5 times.
maybe you missed the necessary
> SoundDevShutDownIn();

about jerky playback... well maybe it is because the video plays back with 1 fps instead of 25...?
MLVViewer is no media player, it is for previewing ;)

just export and decide by the .wav
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

g3gg0

btw. added 7D support. see bitbucket, no binaries yet.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

zachnfine

Quote from: zachnfine on December 29, 2013, 10:18:04 PM
1. I loaded the mlv-rec and mlv-snd modules and set it for 1920x1080p23.976 and tried recording some raw video to my Komputerbay 128Gb 1050X CF card. It gets about 48-70 frames in, reports a skipped frame or two, and stops recording. Every time with the same result. I think spanning is off, and there's only files recorded to CF, but I'm not sure (need to poke around and find that setting again). Recording at the same resolution and frame rate using the normal raw recording module works flawlessly to the same card.

I just looked at the SD and CF card and found the answer to this problem -- it was trying to record everything to the SD card. I had the preferred card set to CF, so I figured I had this set properly. Maybe there's some other mechanism to use to tell ML to just never, ever try to record anything to the SD card?

zachnfine

OK, got the latest nightly build and the latest mlv_rec modules installed now and am ready to try again.

Is it wrong that in the ML menu, the descriptive text for mlv_snd.mo is "Adds SMPTE output functionality to mlv_rec"?

1%

I'll give the commands a go but have the feeling it will just reset the audio settings (to 23db internal mic)... look at what registers those touch.

It seems I'm getting wav 5/5 times too, just MLV browse isn't dumping it and I never thought to try MLV player.

QuoteIs it wrong that in the ML menu, the descriptive text for mlv_snd.mo is "Adds SMPTE output functionality to mlv_rec"?

Yep.

g3gg0

uhm just tried - it shows the correct description.
but i realized that i put an old version into it, not the most recent one.

reuploaded.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

g3gg0

Quote from: 1% on December 29, 2013, 11:20:47 PM
It seems I'm getting wav 5/5 times too, just MLV browse isn't dumping it and I never thought to try MLV player.

MLVBrowseSharp doesnt dump for you?
which OS? which error? did you get the latest version?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

zachnfine

Got continuous recording working now, with possible skipped frames.

This may be something unrelated to mlv and simply a new (to me) way that the nightly ML is noting skipped frames, but I'm not sure what to make of this "71 frames / 4115 expected" text at the bottom of the frame while recording.

I've got a screenshot I'd be happy to post, but clicking the "Insert image" button in this form just inserts a couple of img tags into my post -- doesn't seem that there's an upload option. Maybe this'll work:
Or just this link: https://www.evernote.com/shard/s1/sh/370ae243-27cf-48dd-9b16-853ebc97ae58/e5767bbf15dc638cf8dba588652309ee/res/ed0c02ff-8f4b-495c-9dc6-73a542de586c/skitch.jpg?resizeSmall&width=832

It appears to be recording fine and the buffer graph at top is not filling past the end, but that text at the bottom alternates between saying things like "00:39.941 frames. continuous OK !" and "58 frames / 5102 expected". I'm not sure how to interpret this.

When I installed the latest nightly I blew away my settings. So I'm not sure what in my previous settings caused it to be recording to SD.

1%

Same 7D files I thought were 1/5 were 5/5 when played in MLV viewer.

Quote
which OS? which error? did you get the latest version?

Win7 x64
Downloaded again today from first post when I saw commits. Both dated 12/29

Both 6D and 7D files are coming out 0 audio frames a bunch in the log but audio is there and looks complete. The 6D files are 4GB after the first clip. Most of the 6D extracts even with 0 audio frames reported.


g3gg0

Quote from: 1% on December 29, 2013, 11:35:40 PM
Both 6D and 7D files are coming out 0 audio frames a bunch in the log but audio is there and looks complete. The 6D files are 4GB after the first clip. Most of the 6D extracts even with 0 audio frames reported.

just tried on 7D:

audio works fine, here a log.

Exec: 'mlv_dump.exe --batch --dng -v -o "d:\Users\g3gg0\Desktop\Neuer Ordner\M30-0051.MLV.frame_" "F:\DCIM\100EOS7D\M30-0051.MLV"'
MLV Dumper v1.0
-----------------
Mode of operation:
   - Input MLV file: 'F:\DCIM\100EOS7D\M30-0051.MLV'
   - Verbose messages
   - Convert to DNG frames
   - Output into 'd:\Users\g3gg0\Desktop\Neuer Ordner\M30-0051.MLV.frame_'
File F:\DCIM\100EOS7D\M30-0051.MLV opened
File F:\DCIM\100EOS7D\M30-0051.IDX opened (XREF)
XREF table contains 99 entries
Processing...
File Header (MLVI)
    Size        : 0x00000034
    Ver         : v2.0
    GUID        : 3806773218881307577
    FPS         : 25.000000
    File        : 0 / 0
    Frames Video: 73
    Frames Audio: 16


sure you use *my* release?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Yoshiyuki Blade

I've been playing with the MLV dump compression options and I have problems when using the -b parameter. When outputting files to DNG, the images are distorted in one form or another (I can't even recognize what's in the image). Most of the time it's all white with some random bits of noise. Tried playing around with different values, but with no luck. This parameter gives just a little more compression than without it, but even that's a nice thing to have. The files were recorded on a 5D2. All the other parameters (-e -l -z) appear to work fine in the short tests I've done.

edit: spoke a little too soon. There are a few artifacts coming from somewhere else, though not as severe.
edit2: Recompressed it and it looks fine now o_O I think having the MLV Player running while its compressing isn't a good idea XD Outputting the frames crashes after a while though.

Levas

On 6d
working with the newest available build (11 december build for 6d). And the mlv-rec and mlv-snd module from yesterday (copied from the 5d build, from the download link in the 1st post.)
First clip is with audio and error free.
Every clip after the first one gives the error (audio failed to stop, state 4), in those files I can't extract a full wave file with MLV-dump.osx.

I'm I doing things right by using the 11 december build for the 6d and replacing the mlv-rec.mo and mlv-snd.mo file for the ones in available in the newest 5d build ?



Jakobmen

so with MLV_REC whats the best format the cards should be in ? exFAT or
FAT32.. all my cards are KB 64gb 1000 exFAT


still getting skip frame when it starts to roll over to the next 4GB file
Canon 5D3

zachnfine

Is anyone able to record long clips? I got a 1920x1080p23.976 clip with audio recorded to my Komputerbay 1050X 128Gb card, and it appeared to be running continuously ok (except for onscreen messages about "expected frames" that I don't really understand), but recording stopped soon after the 4Gb limit was hit. My card is formatted exFat, and I can record to it with the standard raw recording for 10 minutes and beyond without issue.

I'm processing the file now, it appears to have actual audio (I listened to the wav), but I haven't yet viewed the clip to see if there are any apparent skipped frames.

zachnfine

BTW, here's another way to script the processing of MLV files, using tcsh:

% foreach foo ( *.MLV )
foreach? mlv_dump --dng -o $foo:r_ "$foo"
foreach? mlv_dump -m -o $foo:r_meta.mlv "$foo"
foreach? mlv_dump -v $foo:r_meta.mlv > $foo:r.txt
foreach? rm $foo:r_meta.mlv
foreach? rm $foo:r_meta.mlv.wav
foreach? end

I dropped my .MLV and .M00 file for one clip into a folder, then cd'd into that folder and typed the lines above (the "%" and later "foreach?" are prompts, no need to type those).

zachnfine

FAT32 FTW!

I reformatted my CF card using OS X Disk Utility to FAT32, and now I can record long stretches with no skipped frames (where onscreen was previously displayed "X Frames Expected" now it just says "continuous frames OK"), and no skips at 4Gb split points.

So some issue must be getting in the way of recording MLV 2.0 to exFat formatted CF. But Fat32 seems to work a treat.