RAW Video Post processing (raw2dng) [Mac/Win/Linux]

Started by g3gg0, May 14, 2013, 11:42:07 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

g3gg0

Please ask questions about how to post process RAW-Video here.

Keywords: raw2dng, 14-bit raw, dng, video

All questions regarding RAW VIDEO POST PROCESSING on any OS
HERE

All questions regarding RAW_REC MODULE COMPILATION/INSTALLATION/USAGE
HERE: http://www.magiclantern.fm/forum/index.php?topic=5405.0




raw2dng (reference implementation, command-line tool)

Source code: raw2dng.c and chdk-dng.c
Windows executable: raw2dng.exe (drag and drop the .raw file over the executable)
Mac executable: raw2dngOSX.zip (run in terminal, don't forget chmod +x)
Windows executable for cameras with pink dots (650D, 700D, EOS-M): raw2dng_cs2x2.exe (it does some chroma smoothing, which happens to remove the pink dots too; use it when PinkDotRemover doesn't work)






GUI application for MAC (scrax):
Thread
Just drag a raw file on it and it will convert to dng and save the file in a subfolder inside the .raw folder





Linux scripts
Bash script for conversion to mjpeg (needs ffmpeg and ufraw-batch): raw2avi.sh
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

scrax

raw2dgn.app
More info on the GUI version discussion topic

raw2dng
Osx command line

Know issues
- Can't convert file bigger than 2Gb.
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

g3gg0

Quote from: scrax on May 14, 2013, 11:46:12 AM
by the way:where are the raw files saved on the card? on root or in DCIM?

should be DCIM folder. my lv_rec modules placed them in the root which was not ideal, so alex placed it in DCIM\xxx\
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

a1ex

raw2dng (reference implementation, command-line tool)

Source code: raw2dng.c
Windows executable: raw2dng.exe (also runs in Wine, for Mac and Linux)

Usage: drag and drop the .raw file over the executable, or use the command line.

Linux shell script for conversion to mjpeg: raw2avi.sh

scrax

Quote from: g3gg0 on May 14, 2013, 11:48:25 AM
should be DCIM folder. my lv_rec modules placed them in the root which was not ideal, so alex placed it in DCIM\xxx\
good that's perfect since MLTools checks only the DCIM folder, since there should there the captured stuff.
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

scrax

Quote from: a1ex on May 14, 2013, 11:48:36 AM
raw2dng (reference implementation, command-line tool)

Source code: raw2dng.c
Windows executable: raw2dng.exe (also runs in Wine, for Mac and Linux)

Usage: drag and drop the .raw file over the executable, or use the command line.

Linux shell script for conversion to mjpeg: raw2avi.sh

here a link to the raw2dng compiled for mac (no wine needed)
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

scrax

Quote from: g3gg0 on May 14, 2013, 11:42:07 AM
Mac executable: raw2dngOSX.zip
Usage: drag and drop the .raw file over the executable, or use the command line.

just a small note, for osx drag and drop a .raw on the raw2dng executable don't works and double clicking it will run it without any raw file selected
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus | EOS 5D MLbeta5- EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX, PS, LR, RawTherapee, LightZone -no video experience-

HD Cam Team

Hi,

Great idea about this thread.

Is it possible to somehow use FFmpeg to get video directly from the current .raw file?

http://www.ffmpeg.org/index.html

Thanks
HD Cam Team
Web:
www.HDCamTeam.com | Twitter: @HDCamTeam | Facebook: HDCamTeam

a1ex


electronicwomble

Thanks so much for all your work on this! Your turnaround times for all this dev work are amazing :)

Any ideas about where to look for converting DNGs to CinemaDNGs ready for Resolve?

HD Cam Team

HD Cam Team
Web:
www.HDCamTeam.com | Twitter: @HDCamTeam | Facebook: HDCamTeam

g3gg0

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Yoshiyuki Blade

Regarding audio recording with the raw footage, is there some kind of nonstandard sample rate that the audio recorder goes by? I could not properly sync the audio with the video no matter what at 48KHz or 44.1KHz, but if I set it to something weird like 46.6KHz, it was kinda close to sync. The footage in general is kinda jumpy and has weird stutters (not frameskipping, the module reported no dropped frames and the buffer was never exceeded) so idk if the weird jumps could cause a problem with the audio sync or if the audio is messed up too.

g3gg0

yet audio recording is not scope of this module, i (and i bet alex too) didnt try too much.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Danne

Quote from: scrax on May 14, 2013, 12:04:34 PM
just a small note, for osx drag and drop a .raw on the raw2dng executable don't works and double clicking it will run it without any raw file selected

Nice to have the files in a subfolder. Next step would be to be able to throw the on the mac-app script all at once. Is it hard to do Scrax? Now I,m extracting them one by one.
Thanks!
/D

Yoshiyuki Blade

Quote from: g3gg0 on May 14, 2013, 01:22:59 PM
yet audio recording is not scope of this module, i (and i bet alex too) didnt try too much.

Ah indeed. I understand that audio is not the focus your efforts atm so it's not a big issue for me either. I was curious to see how much "overhead" the audio recorder would add while recording RAW and it turns out to be virtually negligible. The syncing issues might've existed before RAW recording too, but I never explored it. After fooling around a little more, I needed to adjust it to about 46.8KHz to look reasonably synced in the ~1.2 mins my card does in RAW.

olik

Hey folks,

I do not have after fx and I failed to open the DNGs with Davinci resolve software, FCX and premiere pro, any other software that could work?

thanks for help

o.

a1ex


cinema5D

As posted in the other article we put together a small GUIDE covering the basic steps to get the 5D recording RAW: http://c5d.at/1o4
Please give us feedback if anything in there is wrong or easier to achieve or outdated and we will update the guide.

Also we would like to let you know that the DNG format is extremely unwelcome to non-photographers. If there is any way to get cinemaDNG files instead, that would be very helpful to many.
But hey, it's amazing to be at this point here, so we will all be patient for this to evolve.

Thanks and good light!

ch_d

Quote from: g3gg0 on May 14, 2013, 11:42:07 AM
All questions regarding RAW VIDEO POST PROCESSING on any OS
HERE

What would be really great: Something like nice workflow for proxy editing:
a) convert the raw to dng (allready here, called raw2dng)
b) ! in the same step as raw2dng make a proxy for editing (like mjpeg or even prores proxy)

so you can edit the files and reconnect them with the raw dng afterwards. like online/offline editing.
5D MII

olik


Colemar

scrax,
   raw2dng for Mac is not able to resolve filepaths with name spaces.  Once I copy the .raw dataset to a contiguous filepath name, converter works great.

EDIT:  Unrelatedly, has anyone noticed .raw files that record to 4GB or larger won't convert in raw2dng?  On Mac OSX I get an "this ain't a lv_rec file" error.

thanks.

a1ex

Are these files larger than 4GB or they are cropped at 4GB?

The metadata is at the end of the file, so if it's missing, all you can do is to fill in the metadata yourself or hack the source code of the converter.

Colemar

Quote from: a1ex on May 14, 2013, 03:50:11 PM
Are these files larger than 4GB or they are cropped at 4GB?

The metadata is at the end of the file, so if it's missing, all you can do is to fill in the metadata yourself or hack the source code of the converter.

they are larger, usually ~4.28GB according to OSX Finder properties.  It seems like a corrupt header because I also have a couple ~2.07GB .raw files that are not recognised in the same manner.  Is this a quick fix or do you want a smaller file Alex?


mucher

Thanks for the white paper. Good exercise for the brain.

Here I've found a link to convert DNG to tiff first, saidly, Resolve can open the tiff sequence.

http://www.completedigitalphotography.com/?p=521