MLV Lite

Started by dmilligan, February 15, 2016, 03:42:22 AM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

Danne

We don't know the exact amount of frames in the mlv file before it's spit out and indexed. There is a frames tag in the header but it's inexact.
Also note that H.264 blackframes varies in the beginning part. It's first after the beginning black frames are cut off that we can calculate the end part.

dfort

About counting black frames from the end of the H.264 file.

Quote from: dfort on October 07, 2017, 05:42:00 PM
Out of four days of shooting that amounted to a few hundred clips and several hours (estimated about 300 shots, 4 hours) there were only two clips where the length of the raw footage didn't match up with the trimmed H.264 proxy. Being the perfectionist that he is Danne tried his best but the issue isn't with Switch--it turned out that sometimes the H.264 recording stops before the MLV. One short clip was off by 1 frame and the other, an 8 minute shot that spanned two H.264 files was off by 2 frames.

When the H.264 recording stopped before the MLV there were no black frames at the end of the H.264 file. Even though there is a mismatch in length, the current method of starting from the head of the clip keeps the H.264 and MLV files in sync and the editing software should have no problem re-linking between the proxy and raw files as long as you don't use the last frame of your raw fine in the edit. This shouldn't happen because you're making the creative editing decisions with the proxy files and the raw files are only for color grading, right?

The issue brought up by @ domasa is that the thumbnails in the camera are all black. This isn't a problem in the editing system because the black frames are trimmed of by the Switch application. Still, it would be nice to see proper thumbnails in camera if you want to check your shots in the field. This isn't a high priority item for me because when I want to playback a clip it is usually the last one I shot and that's what the Canon playback (a.k.a. review) will show. Note that with H.264 proxy there should be no need to playback the MLV files in camera so you don't need to turn on mlv_play.

Lars Steenhoff

Besides the in camera thumbnails, I also can see the need for thumbnails on the computer since thats when I'm using to quickly identify a clip to review.
It would also be nice if .mlv files had a thumbnail image on the finder.  any idea if this would be theoretically possible to be generated by the camera ?

IDA_ML

I have uploaded a MLV clip (about 700MB) shot with the 100D at 12-bit lossless compression + SOUND using the MLV lite module with FPS Override: OFF.  For those who wish to try, here is the download link which will be active for 7 days:

https://we.tl/shydYVJ0JV

I tested all options of the MLV Lite module with sound at the 1728x972 resolution and 23,97 fps and here is what I found:

1) At 8...11 bit LL video and sound are recorded synchronously and continuously;

2) At 12 bit LL video and sound are recorded synchronously and continuously;

3) At 14 bit LL video and sound are recorded synchronously but recording stops after about 450 frames;

4) At 10 bit uncompressed sound is recorded for about 3s and then stops while video continues recording until buffer is full;

5) At 12 and 14 bit uncompressed there is no sound at all; video recording continues until buffer is full.

My overall impression is that for short clips (up to 15 s.) synchronization between audio and video is perfect.  Audio and video tracks are of equal length. I did not test syncing at very long clips though.  Video quality is excellent, clips open well with the 32-bit version of MLVFS and grade well in DaVinci Resolve Lite on Win7x64!  There is no noticeable  quality degradation of the 8...11 bit LL mode vs. the 12 or 14-bit LL or uncompressed 10/12/14 bit modes.  Just the noise in the darkest areas at very high ISOs (3200 and 6400) looks slightly different but differences are very subtle.   The dark screen at 8...11 bit LL playback is also gone, (Thank you, A1ex!). Camera is very stable and I did not notice any crashes when switching modes.  The Movie crop mode works well too.  All in all, I am very pleased with the 100D at the 8 ... 11 bit LL mode with sound and think that now it is a very solid camera for RAW video recording with synchronous sound.  Excellent work from all developers involved!


IDA_ML

Hello ErwinH,

After testing the 100D and reporting that it works very well with synchronous sound in the losslessly compressed RAW video modes (see my previous post above), I decided to check how it will work with the 5DMkIII.  I visited a friend of mine who has this camera and we tested the October 23-th build for the 1.1.3 FW that we downloaded from here:

https://bitbucket.org/ehoutsma/magic-lantern/downloads/

Here is what we found with the MLV-sound function enabled on the 5D3 with FPS override OFF:

1) In the end of each recording we get a red line on the screen saying "audio failed to stop".

2) Upon inspecting the recorded MLV file on a computer, cDNG converted with MLV-dump, we notice that the WAV-file is there with its full size but, when trying to play it, no sound comes out of the speakers.  The progress bar is clearly indicating that the WAV-file is being played.

3) The same behavior is observed also when recording in the 14/12/10 bit uncompressed modes in the MLV-recording module.  With the regular October 31-st nightly build, sound is recorded fine in the 14-bit mode MLV-recording mode.  No "audio failed to stop" message appears in the end of the recording. 

I was wondering if you may find some time to have a look at your Oct. 23-rd build for the 5DMkIII.  If the sound issue could be fixed with the lossless modes on this camera, this will boost its usability enormously!  Please let me know if you may need a .MLV file from the 5D3 that has this behavior.

Thank you again for your amazing work.

P.S.: If someone else with the 5D3 could confirm the above behavior, this would be helpful.

JCut

First post!  :)  I've been using ML on my 6D for a couple years mainly for photos (checking raw histogram to ETTR), discovered how incredible ML raw video is, and bought a used 5D3 after hitting the SD slot limits of the 6D...

I'm currently using ErwinH's Oct23 build MLV Lite LL with sound on my 5D3, firmware 1.1.3, and it works for me.  I haven't experienced the issue reported by IDA_ML :

1.  I only tested with 12bit Loss Less, with MLV Sound working, continuous recording

2.  I can start/stop recording fine when recording @ 24p.  No errors/messages.

3.  I'm using MLVFS (32-bit - thanks to BouncyBall!) directly editing the MLV with Davinci Resolve 14.

4.  I also tried some crop mode recording in 1920x1080 @ 48p  and 1920x800 @60p , and it works with sound as well.  However, since I run it with realtime color LiveView, I get dropped frames after some time.  I noticed when editing those clips that stopped due to dropped frames, the sound cuts out a couple seconds before the the video cuts out.  I dunno if that could be related to the "audio failed to stop" message or not.

Thank you all for everyone's incredible work!  Magic Lantern keeps getting better and better!

IDA_ML

Quote from: JCut on November 03, 2017, 11:40:00 PM
1.  I only tested with 12bit Loss Less, with MLV Sound working, continuous recording

4.  I also tried some crop mode recording in 1920x1080 @ 48p  and 1920x800 @60p , and it works with sound as well.  However, since I run it with realtime color LiveView, I get dropped frames after some time.  I noticed when editing those clips that stopped due to dropped frames, the sound cuts out a couple seconds before the the video cuts out.  I dunno if that could be related to the "audio failed to stop" message or not.

At what resolution did you test the 12bit Loss Less, with MLV Sound?  Did you also try the Crop mode at 3K and 24fps?  Could you also check the sound recording in the 14/12/10 bit uncompressed modes in the MLV-recording module?

I will ask my friend to repeat the tests on his 5D3, maybe with a faster CF card and see if he can get rid of the "sound failed to stop" message.

Thanks for your important feedback, JCut !

JCut

Quote from: IDA_ML on November 04, 2017, 02:31:39 AM
At what resolution did you test the 12bit Loss Less, with MLV Sound?  Did you also try the Crop mode at 3K and 24fps?  Could you also check the sound recording in the 14/12/10 bit uncompressed modes in the MLV-recording module?

I will ask my friend to repeat the tests on his 5D3, maybe with a faster CF card and see if he can get rid of the "sound failed to stop" message.

Thanks for your important feedback, JCut !

Hi IDA_ML , I tested 12 bit LL with MLV Sound at 1920x1080.  (wider aspect ratios work also).  I didn't try out the 3K mode yet... However, I checked the sound recording in the 14/12/10 bit uncompressed modes in the MLV_REC module:

1.  14 bit uncompressed, 1920 x 1080 @ 24p - Sound works, no unusual messages
2.  12 bit uncompressed, 1920 x 1080 @ 24p - Sound works, no unusual messages
3.  10 bit uncompressed, 1920 x 1080 @ 24p - Sound works, no unusual messages

As for your friends "sound failed to stop" message, I'm using Sandisk Extreme Pro 160MB/s CF 128GB cards.  Does he have any other modules loaded ?  For Lossless MLV , I only have MLV_LITE, Crop_Rec, MLV_SOUND, and FILE_Man modules loaded, nothing else.



Lars Steenhoff

Can you try also h264 proxy recording on canon 5dmk3?

I did get some sound issue with that. ( static noise )

JCut

Quote from: Lars Steenhoff on November 04, 2017, 11:50:34 AM
Can you try also h264 proxy recording on canon 5dmk3?

I did get some sound issue with that. ( static noise )

Hi Lars,

I tried out the proxy recording lossless with sound, and I'm having similar problems as you.

1. 14 bit lossless, proxy record on = static on the sound file

2. 12 bit - same static

3.  I also experienced a different issue when proxy record was on. After a couple seconds the recording would stop and I would get a message "recording automatically stopped" or something like that. I also saw a red message flash briefly saying "sound failed to stop"

IDA_ML

Thanks JCut,

I will send him your settings.  The problem could be that he is using a slower card and loading to many modules on it.  Did you test synchronisity between sound and video already?  That would be helpful too.

Lars Steenhoff

Sandisk extreme 160MB,  128 GB.

The card is not the problem I think.

Proxy recording works well on the card with the normal mlv lite + h264 proxy
mlv lite + h264 proxy + mlvsound = static

JCut

Quote from: IDA_ML on November 05, 2017, 09:42:45 AM
Thanks JCut,

I will send him your settings.  The problem could be that he is using a slower card and loading to many modules on it.  Did you test synchronisity between sound and video already?  That would be helpful too.

I'm currently on vacation, with no computer to check MLV files... But I'm recording lots of stuff with this mlv lossless + mlv sound build, so I'm crossing my fingers everything will be fine  ;)

I should be able to report back by this weekend!

ErwinH

I've been using my 700D build to record with mlv lite + mlv snd and as far as I can tell the recordings are running smoothly, didn't ran into any issues.

I do get the Audio failed to stop "error", but that's not a real error. Recording is stopped, and audiometers continue to function. Might have something to do with that.

What's the reason why you want to combine h264 proxy with mlvsnd? h264 proxy has the same audio as mlvsnd (when recording at 48khz) so there's no reason to record it using mlvsnd.

Danne

What´s the reason for Audio failed to stop "error"?

In mlv_sound.c
static void mlv_snd_stop()
{
    trace_write(trace_ctx, "mlv_snd_stop: stopping worker and audio");
   
    mlv_snd_state = MLV_SND_STATE_SOUND_STOPPING;
   
    /* wait until audio and task stopped */
    uint32_t loops = 100;
    while((mlv_snd_state != MLV_SND_STATE_SOUND_STOPPED) && (--loops > 0))
    {
        msleep(20);
    }

    if(mlv_snd_state != MLV_SND_STATE_SOUND_STOPPED)
    {
        bmp_printf(FONT(FONT_MED, COLOR_RED, COLOR_BLACK), 10, 130, "audio failed to stop, state %d", mlv_snd_state);
        trace_write(trace_ctx, "mlv_snd_stop: failed to stop audio (state %d)", mlv_snd_state);
        beep();
    }


MLV_SND_STATE_SOUND_STOPPED didnt work? Yet the audio stopped, but
Checking after the loop:
    /* some models may need this */
    StopASIFDMAADC();
    // SoundDevShutDownIn();  /* no model seems to need this */
    audio_configure(1);


Is this the stub that is working?

Lars Steenhoff

@ErwinH

Reasons for proxy h264 other than sound:

- Realtime Playback of clips in camera after shot ( on set shot review )
- Using it in a proxy workflow - editing with the h264 and only sync them up in the color correction phase.
- easy to send someone the files after shoot to start edit because they are smaller.
- possible to transfer footage to ipad/phone for client review

edit i see you mean why sound on both!

well to have a quick sound preview with mlvplayer and to dont have to sync them up for preview


12georgiadis

Yes, with sound in both proxy and mlv, it bypasses a step in switch app and then save time

dfort

That StopASIFDMAADC / SoundDevShutDownIn issue was resolved in the crop_rec_4k branch but the unified branch still has a hack where SoundDevShutDownIn is using the address for StopASIFDMAADC on some cameras. In order for audio recording to work on H.264 and MLV Lite simultaneously will probably require re-working mlv_snd and possibly requiring the SoundDevShutDownIn stub.

Recording sound in both the proxy and raw file may seem like a good thing in theory, in practice it doesn't really have much advantages. The black frames need to be trimmed off of the H.264 proxies in order for them to match the raw files. That step is necessary before editing starts or you will have issues later on. Switch can do this very quickly in the field as you're copying the files off the cards. Most people are probably processing their MLV files after the shoot so it should be no problem having Switch sync the audio automatically.

Speaking of theory and practice, it shouldn't be necessary to have the audio with the DNG files because you only need to conform the picture for color grading.

12georgiadis

Quote from: dfort on November 09, 2017, 07:07:38 PM
That StopASIFDMAADC / SoundDevShutDownIn issue was resolved in the crop_rec_4k branch but the unified branch still has a hack where SoundDevShutDownIn is using the address for StopASIFDMAADC on some cameras. In order for audio recording to work on H.264 and MLV Lite simultaneously will probably require re-working mlv_snd and possibly requiring the SoundDevShutDownIn stub.

Recording sound in both the proxy and raw file may seem like a good thing in theory, in practice it doesn't really have much advantages. The black frames need to be trimmed off of the H.264 proxies in order for them to match the raw files. That step is necessary before editing starts or you will have issues later on. Switch can do this very quickly in the field as you're copying the files off the cards. Most people are probably processing their MLV files after the shoot so it should be no problem having Switch sync the audio automatically.

Speaking of theory and practice, it shouldn't be necessary to have the audio with the DNG files because you only need to conform the picture for color grading.
Oh, I forgot about the black frames so you're totally right. One thing might be to have this proxy mode+snd available for all cameras that could handle it with mlv without sound (5d2, 7D, 700d, 6D, and 100D). I mean, when crop_rec_4k would work on both of them.

Danne

Most people are probably processing their MLV files after the shoot so it should be no problem having Switch sync the audio automatically.
Switch is prepared for the day that mlv_lite has audio. It will simply not extract audio into the dng folder when a wav file already is in there.
Producing the audio or even the cutting part of the proxy file is not that time consuming. What is taking time is the matching with the mlv file. Processing all dng files is neccessary for exact calculations.

zachnfine

Hmm, I'm trying this with 5D113 firmware and "magiclantern-croprec4kmlvsnd.2017Oct23.5D3113", 3072x1308 12-bit lossless, and have made a few little recordings. I'm processing the resulting MLV files with the 11/7 version of 'switch'. Each time, I end up with a director full of nice-looking 3072x1308 dng files and a single wav file of 2k in size that's silent.

It's possible that the audio is just not getting properly recorded, but I don't know for sure that the audio isn't present just fine in the MLV file and that Switch isn't just having trouble extracting it. I'm not sure how to tell since I don't know the best, approved manner by which to extract audio ffrom MLV Lite recordings.

zachnfine

Hmm, I tried the same file with mlv_dump.osx (a very old mlv_dump, dating back to 2014, since I can't seem to find a binary for the latest mlv_dump (searches just lead me to old forum threads). The result was a bunch of nice looking dng files and a 44-byte wav file that seemed to contain no data. I looked at both its contents and those of the wav file that Switch had generated and they both appeared to be pretty much content free, but the Switch version included some Blackmagic metadata that must be there for compatibility with Davinci Resolve.

So I guess I'm recording files with blank data.

Danne

I don't think audio is working with crop modes.

jankrueck

I came here, because someone was posting on the 4k branch :D
So still no sound for resolutions higer than HD ?

50mm1200s

I'm getting glitches :(
Setup is 50D crop_rec (3x), ML build from @dford, res 1920x1080 at 24fps. If you need any other info I can provide it...