Author Topic: How to Sync RAW and Audio Issues  (Read 34906 times)

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
How to Sync RAW and Audio Issues
« on: May 30, 2013, 07:46:45 PM »
I found some discussions burried in the threads about this aspect of video so I am creating a new thread to discuss it further if it is needed.  The last discussion that I could find a solution read:

Well, I've basically been up all night trying to get the audio sync working for the 5d3.  After much trial and error, I've found that about a 103% "time stretch" does a really great job.  It's between 103-104%; probably 103.2 would be damn near perfect.  Make sure you whack the end of the audio file off first before you do the stretch.
I tested this on a 4 minute clip and it's off by maybe a nanosecond all the way at the end.
Keep in mind this is with FPS override ENABLED, and set to 24p.  If I disable this, the video gets waaaaay ahead of the audio and then "catches up" and I lose even more sleep.  :)  Hope this helps someone who shoots without an external recorder.  Workflow = Adobe after effects CS6.

You can trim the furthest point of the wav file in your composition by hitting the "alt" and ] keys (will remove everything to the right).
Shawn

Please contribute here if you have any other discussions about this topic.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: How to Sync RAW and Audio Issues
« Reply #1 on: May 31, 2013, 05:49:14 PM »
Reporting 5D3 w/1729277 + Lexar 1000X with FAT 32 format

I tested 23.97 fps (canon std), 29.97 fps (canon std), and 25 fps (fps override)

In all three cases a stretch factor of 1.027 seemed satisfactory for sync audio.

The audio had to be offset from the start of the video anywhere from 3 to 5 frames.

I noticed that the audio had some sort of defects every few seconds.  I am no audio expert so I dont know how to describe them.  I sort off muted tiny parts of the audio.  I filmed a drum beat with screwdriver and every so often it seems that the bang was missing something.  Maybe called a drop out.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: How to Sync RAW and Audio Issues
« Reply #2 on: May 31, 2013, 05:54:27 PM »
Sounds like same issues I had with 600D.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: How to Sync RAW and Audio Issues
« Reply #3 on: May 31, 2013, 05:57:22 PM »
I would rewrite the audio thingie using the silent pic burst algorithm (no copying needed and zero overhead in the CBRs).

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: How to Sync RAW and Audio Issues
« Reply #4 on: May 31, 2013, 06:10:17 PM »
Meaning write audio with edmac? Edmac > dma_memcpy

First need a way to find more free edmacs.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: How to Sync RAW and Audio Issues
« Reply #5 on: May 31, 2013, 06:12:33 PM »
No copying at all. Silent pic burst writes the frames directly in the final buffers.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7137
Re: How to Sync RAW and Audio Issues
« Reply #6 on: May 31, 2013, 10:34:14 PM »
Cool to hear about trying fixing the audio. Really like the possibility to record a separate file with the same name as the moviefile. As I tried it out there is some strange drifting going on. I can,t say It,s consistent but seems you already know that. So stretching won,t help really.
Still, I,m no programmer and have no idea how to fix this. Very thankful for all your efforts.
//D

Shield

  • Senior
  • ****
  • Posts: 254
Re: How to Sync RAW and Audio Issues
« Reply #7 on: June 01, 2013, 01:49:53 AM »
I am honored to be quoted in this thread (I mean what the heck do I know) but even more thrilled the heavyweights are starting to look at the audio issues.
Thank you so much ML staff and developers - you guys are amazing.

noisyboy

  • Guest
Re: How to Sync RAW and Audio Issues
« Reply #8 on: June 01, 2013, 04:40:06 AM »
If it's dropouts is it because the sample buffer is too small? As in latency issues? I guess with recording raw AND uncompressed WAV this would kick the processor in the nuts?

Shield

  • Senior
  • ****
  • Posts: 254
Re: How to Sync RAW and Audio Issues
« Reply #9 on: June 01, 2013, 07:51:14 AM »
Actually I've been doing more playing around with this tonight.

It's not a time stretch of 103 (although that's pretty close).

Here's what I'm working on right now (no guarantee this is perfect, but it might be as good as it gets for right now):

First of all, since technically the recorded wav file is just repeated over and over and is actually 12:25:39:243 (2,147,283,647 samples).
I cut the garbage off at the end in Soundforge.

Scene 1:  2876 DNG files.  AE says the video length is 01:59:20
The audio file, after the garbage has been cut off at the end, is 1:53.  So now I need to stretch 1:53 to 1:59:30.  This is 105.9 and lines up perfectly, but I'm sure it will be slightly behind in the beginning of the clip and slightly ahead at the end.

So I'm only sharing this tidbit for people who shoot really short scenes - this might help with onboard.  But almost 6 seconds of audio is just too much to lose - I need to do some more investigating.

Shield

  • Senior
  • ****
  • Posts: 254
Re: How to Sync RAW and Audio Issues
« Reply #10 on: June 01, 2013, 07:56:03 AM »
Of course as soon as I say that and open the audio file in Sony Vegas, it's 1:57:17 in length.


My head hurts.  Going to bed.

Shield

  • Senior
  • ****
  • Posts: 254
Re: How to Sync RAW and Audio Issues
« Reply #11 on: June 01, 2013, 08:04:17 AM »
Hmmm, I might be onto something here.  Dragging the audio file on the Adobe after effects 6 timeline says it's only 1:53.

But winamp, Sony Vegas, MediaInfo all say 1:57.  Like Paul Harvey said - standby for news.

Shield

  • Senior
  • ****
  • Posts: 254
Re: How to Sync RAW and Audio Issues
« Reply #12 on: June 01, 2013, 08:14:07 AM »
Nevermind after effects was caching the filename.

New project though:

It's exactly 2 seconds off for a 1:59:19 project.

The audio file is showing 1:57:19

Could it just be dropping a second per minute cumulatively?

It's late.  I'm going to bed.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7137
Re: How to Sync RAW and Audio Issues
« Reply #13 on: June 01, 2013, 11:25:36 PM »
Nevermind after effects was caching the filename.

New project though:

It's exactly 2 seconds off for a 1:59:19 project.

The audio file is showing 1:57:19

Could it just be dropping a second per minute cumulatively?

It's late.  I'm going to bed.

Nice findings!

Jakobmen

  • Member
  • ***
  • Posts: 122
  • Canon 5D3
Re: How to Sync RAW and Audio Issues
« Reply #14 on: June 02, 2013, 05:14:34 AM »
same problem here
Canon 5D3

mageye

  • Senior
  • ****
  • Posts: 296
  • Atheist
Re: How to Sync RAW and Audio Issues
« Reply #15 on: June 02, 2013, 03:07:02 PM »
In my experience the syncronisation issues that are happening are because there are dropouts in the recording (well gaps actually). The files do not need to be re-pitched. That does not work.

5DMKII | 500D | KOMPUTERBAY 32GB Professional 1000x |Canon EF 50mm f/1.8 II | Samyang 35mm f/1.4 ED AS UMC | Canon EF 75-300mm f/4-5.6 III | Zoom H2 (4CH. audio recorder) | Mac OS X 10.9.2 | Photoshop CC | After Effects CC | Final Cut Pro 7

romeus

  • New to the forum
  • *
  • Posts: 18
Re: How to Sync RAW and Audio Issues
« Reply #16 on: June 03, 2013, 03:41:10 AM »
So if you record the sound on an external recorder like h4n and you sync with a claperboard there are no issues anymore

Oedipax

  • Freshman
  • **
  • Posts: 65
Re: How to Sync RAW and Audio Issues
« Reply #17 on: June 03, 2013, 06:41:10 AM »
So if you record the sound on an external recorder like h4n and you sync with a claperboard there are no issues anymore

Assuming you haven't dropped any video frames.

marten

  • Freshman
  • **
  • Posts: 98
Re: How to Sync RAW and Audio Issues
« Reply #18 on: June 03, 2013, 09:23:08 AM »
In my experience the syncronisation issues that are happening are because there are dropouts in the recording (well gaps actually). The files do not need to be re-pitched. That does not work.

This is my experience too :(

Shield

  • Senior
  • ****
  • Posts: 254
Re: How to Sync RAW and Audio Issues
« Reply #19 on: June 03, 2013, 12:54:54 PM »
So if you record the sound on an external recorder like h4n and you sync with a claperboard there are no issues anymore

The post above you shows that dropouts are happening with the onboard audio recording.  What would make you draw the conclusion that the sync would ever work?  I'm confused.

squig

  • Hero Member
  • *****
  • Posts: 518
  • Crash test dummy MK3
Re: How to Sync RAW and Audio Issues
« Reply #20 on: June 03, 2013, 01:02:14 PM »
I think he's talking about syncing the video. I've tested my PCM D50 and lip synced a 12min recording. That was with a week old build with FPS override on, I'll test the latest build tomorrow.

romeus

  • New to the forum
  • *
  • Posts: 18
Re: How to Sync RAW and Audio Issues
« Reply #21 on: June 03, 2013, 04:43:28 PM »
Like Squig said I'm talking about syncing the video with an external recorded audio

Shield

  • Senior
  • ****
  • Posts: 254
Re: How to Sync RAW and Audio Issues
« Reply #22 on: June 05, 2013, 02:47:12 AM »
Has anyone made any progress with the onboard audio?  To me it's just about perfect really except for this.  I find myself spending so much time trying to sync the audio in post - so used to having a good "reference" audio track.   :)

swinxx

  • Hero Member
  • *****
  • Posts: 602
Re: How to Sync RAW and Audio Issues
« Reply #23 on: June 07, 2013, 10:53:51 AM »
i would be interested in this too..
is there any progress with the recent builds?

thx.

ewinemiller

  • New to the forum
  • *
  • Posts: 45
Re: How to Sync RAW and Audio Issues
« Reply #24 on: June 07, 2013, 01:53:48 PM »
i would be interested in this too..
is there any progress with the recent builds?

thx.

As of the build available 6/6 evening east coast time, no.

I recorded a couple of test videos. One was fairly close over nearly a minute of time (~2 frames out). The other was ~3 seconds out over 15 seconds. The longer clip I could get to line up pretty well with some sizing and start twiddling, but the short one was just a mess.

I also noticed that even with an external mic, I'm only getting mono WAV files. Is this normal or am I missing something?

Thanks.