How to Sync RAW and Audio Issues

RenatoPhoto · May 30, 2013, 07:46:45 PM

I found some discussions burried in the threads about this aspect of video so I am creating a new thread to discuss it further if it is needed. The last discussion that I could find a solution read:

Quote from: Shield on May 26, 2013, 11:42:56 AM
Well, I've basically been up all night trying to get the audio sync working for the 5d3. After much trial and error, I've found that about a 103% "time stretch" does a really great job. It's between 103-104%; probably 103.2 would be damn near perfect. Make sure you whack the end of the audio file off first before you do the stretch.
I tested this on a 4 minute clip and it's off by maybe a nanosecond all the way at the end.
Keep in mind this is with FPS override ENABLED, and set to 24p. If I disable this, the video gets waaaaay ahead of the audio and then "catches up" and I lose even more sleep. Hope this helps someone who shoots without an external recorder. Workflow = Adobe after effects CS6.

You can trim the furthest point of the wav file in your composition by hitting the "alt" and ] keys (will remove everything to the right).
Shawn

Please contribute here if you have any other discussions about this topic.

RenatoPhoto · May 31, 2013, 05:49:14 PM

Reporting 5D3 w/1729277 + Lexar 1000X with FAT 32 format

I tested 23.97 fps (canon std), 29.97 fps (canon std), and 25 fps (fps override)

In all three cases a stretch factor of 1.027 seemed satisfactory for sync audio.

The audio had to be offset from the start of the video anywhere from 3 to 5 frames.

I noticed that the audio had some sort of defects every few seconds. I am no audio expert so I dont know how to describe them. I sort off muted tiny parts of the audio. I filmed a drum beat with screwdriver and every so often it seems that the bang was missing something. Maybe called a drop out.

1% · May 31, 2013, 05:54:27 PM

Sounds like same issues I had with 600D.

a1ex · May 31, 2013, 05:57:22 PM

I would rewrite the audio thingie using the silent pic burst algorithm (no copying needed and zero overhead in the CBRs).

1% · May 31, 2013, 06:10:17 PM

Meaning write audio with edmac? Edmac > dma_memcpy

First need a way to find more free edmacs.

a1ex · May 31, 2013, 06:12:33 PM

No copying at all. Silent pic burst writes the frames directly in the final buffers.

Danne · May 31, 2013, 10:34:14 PM

Cool to hear about trying fixing the audio. Really like the possibility to record a separate file with the same name as the moviefile. As I tried it out there is some strange drifting going on. I can,t say It,s consistent but seems you already know that. So stretching won,t help really.
Still, I,m no programmer and have no idea how to fix this. Very thankful for all your efforts.
//D

Shield · June 01, 2013, 01:49:53 AM

I am honored to be quoted in this thread (I mean what the heck do I know) but even more thrilled the heavyweights are starting to look at the audio issues.
Thank you so much ML staff and developers - you guys are amazing.

noisyboy · June 01, 2013, 04:40:06 AM

If it's dropouts is it because the sample buffer is too small? As in latency issues? I guess with recording raw AND uncompressed WAV this would kick the processor in the nuts?

Shield · June 01, 2013, 07:51:14 AM

Actually I've been doing more playing around with this tonight.

It's not a time stretch of 103 (although that's pretty close).

Here's what I'm working on right now (no guarantee this is perfect, but it might be as good as it gets for right now):

First of all, since technically the recorded wav file is just repeated over and over and is actually 12:25:39:243 (2,147,283,647 samples).
I cut the garbage off at the end in Soundforge.

Scene 1: 2876 DNG files. AE says the video length is 01:59:20
The audio file, after the garbage has been cut off at the end, is 1:53. So now I need to stretch 1:53 to 1:59:30. This is 105.9 and lines up perfectly, but I'm sure it will be slightly behind in the beginning of the clip and slightly ahead at the end.

So I'm only sharing this tidbit for people who shoot really short scenes - this might help with onboard. But almost 6 seconds of audio is just too much to lose - I need to do some more investigating.

Shield · June 01, 2013, 07:56:03 AM

Of course as soon as I say that and open the audio file in Sony Vegas, it's 1:57:17 in length.

My head hurts. Going to bed.

Shield · June 01, 2013, 08:04:17 AM

Hmmm, I might be onto something here. Dragging the audio file on the Adobe after effects 6 timeline says it's only 1:53.

But winamp, Sony Vegas, MediaInfo all say 1:57. Like Paul Harvey said - standby for news.

Shield · June 01, 2013, 08:14:07 AM

Nevermind after effects was caching the filename.

New project though:

It's exactly 2 seconds off for a 1:59:19 project.

The audio file is showing 1:57:19

Could it just be dropping a second per minute cumulatively?

It's late. I'm going to bed.

Danne · June 01, 2013, 11:25:36 PM

Quote from: Shield on June 01, 2013, 08:14:07 AM
Nevermind after effects was caching the filename.

New project though:

It's exactly 2 seconds off for a 1:59:19 project.

The audio file is showing 1:57:19

Could it just be dropping a second per minute cumulatively?

It's late. I'm going to bed.

Nice findings!

Jakobmen · June 02, 2013, 05:14:34 AM

same problem here

mageye · June 02, 2013, 03:07:02 PM

In my experience the syncronisation issues that are happening are because there are dropouts in the recording (well gaps actually). The files do not need to be re-pitched. That does not work.

romeus · June 03, 2013, 03:41:10 AM

So if you record the sound on an external recorder like h4n and you sync with a claperboard there are no issues anymore

Oedipax · June 03, 2013, 06:41:10 AM

Quote from: romeus on June 03, 2013, 03:41:10 AM
So if you record the sound on an external recorder like h4n and you sync with a claperboard there are no issues anymore

Assuming you haven't dropped any video frames.

marten · June 03, 2013, 09:23:08 AM

Quote from: mageye on June 02, 2013, 03:07:02 PM
In my experience the syncronisation issues that are happening are because there are dropouts in the recording (well gaps actually). The files do not need to be re-pitched. That does not work.

This is my experience too

Shield · June 03, 2013, 12:54:54 PM

Quote from: romeus on June 03, 2013, 03:41:10 AM
So if you record the sound on an external recorder like h4n and you sync with a claperboard there are no issues anymore

The post above you shows that dropouts are happening with the onboard audio recording. What would make you draw the conclusion that the sync would ever work? I'm confused.

squig · June 03, 2013, 01:02:14 PM

I think he's talking about syncing the video. I've tested my PCM D50 and lip synced a 12min recording. That was with a week old build with FPS override on, I'll test the latest build tomorrow.

romeus · June 03, 2013, 04:43:28 PM

Like Squig said I'm talking about syncing the video with an external recorded audio

Shield · June 05, 2013, 02:47:12 AM

Has anyone made any progress with the onboard audio? To me it's just about perfect really except for this. I find myself spending so much time trying to sync the audio in post - so used to having a good "reference" audio track.

swinxx · June 07, 2013, 10:53:51 AM

i would be interested in this too..
is there any progress with the recent builds?

thx.

ewinemiller · June 07, 2013, 01:53:48 PM

Quote from: swinxx on June 07, 2013, 10:53:51 AM
i would be interested in this too..
is there any progress with the recent builds?

thx.

As of the build available 6/6 evening east coast time, no.

I recorded a couple of test videos. One was fairly close over nearly a minute of time (~2 frames out). The other was ~3 seconds out over 15 seconds. The longer clip I could get to line up pretty well with some sizing and start twiddling, but the short one was just a mess.

I also noticed that even with an external mic, I'm only getting mono WAV files. Is this normal or am I missing something?

Thanks.

News:

How to Sync RAW and Audio Issues

noisyboy