Magic Lantern Forum

Using Magic Lantern => Raw Video => Raw Video Postprocessing => Topic started by: g3gg0 on May 14, 2013, 11:42:07 AM

Title: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: g3gg0 on May 14, 2013, 11:42:07 AM
Please ask questions about how to post process RAW-Video here.

Keywords: raw2dng, 14-bit raw, dng, video

All questions regarding RAW VIDEO POST PROCESSING on any OS
HERE

All questions regarding RAW_REC MODULE COMPILATION/INSTALLATION/USAGE
HERE: http://www.magiclantern.fm/forum/index.php?topic=5405.0



raw2dng (reference implementation, command-line tool)

Source code: raw2dng.c (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/lv_rec/raw2dng.c) and chdk-dng.c (https://bitbucket.org/hudson/magic-lantern/src/tip/src/chdk-dng.c)
Windows executable: raw2dng.exe (http://acoutts.com/a1ex/raw2dng.exe) (drag and drop the .raw file over the executable)
Mac executable: raw2dngOSX.zip (https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dngOSX.zip) (run in terminal, don't forget chmod +x)
Windows executable for cameras with pink dots (650D, 700D, EOS-M): raw2dng_cs2x2.exe (http://acoutts.com/a1ex/raw2dng_cs2x2.exe) (it does some chroma smoothing, which happens to remove the pink dots too; use it when PinkDotRemover (http://www.magiclantern.fm/forum/index.php?topic=6658) doesn't work)

(http://www.eoshd.com/wp-content/uploads/2013/05/ml-raw.jpg)



GUI application for MAC (scrax):
Thread (http://www.magiclantern.fm/forum/index.php?topic=5508.0)
Just drag a raw file on it and it will convert to dng and save the file in a subfolder inside the .raw folder

(https://dl.dropboxusercontent.com/u/123918/MagicLantern/MLTools/img/raw2dng.png)


Linux scripts
Bash script for conversion to mjpeg (needs ffmpeg and ufraw-batch): raw2avi.sh (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/raw_rec/raw2avi.sh)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 14, 2013, 11:46:12 AM
raw2dgn.app
More info on the GUI version discussion topic (http://www.magiclantern.fm/forum/index.php?topic=5508.0)

raw2dng
Osx command line

Know issues
- Can't convert file bigger than 2Gb.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: g3gg0 on May 14, 2013, 11:48:25 AM
by the way:where are the raw files saved on the card? on root or in DCIM?

should be DCIM folder. my lv_rec modules placed them in the root which was not ideal, so alex placed it in DCIM\xxx\
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 14, 2013, 11:48:36 AM
raw2dng (reference implementation, command-line tool)

Source code: raw2dng.c (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/lv_rec/raw2dng.c)
Windows executable: raw2dng.exe (https://bitbucket.org/hudson/magic-lantern/downloads/raw2dng.exe) (also runs in Wine, for Mac and Linux)

Usage: drag and drop the .raw file over the executable, or use the command line.

Linux shell script for conversion to mjpeg: raw2avi.sh (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/raw_rec/raw2avi.sh)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 14, 2013, 11:52:18 AM
should be DCIM folder. my lv_rec modules placed them in the root which was not ideal, so alex placed it in DCIM\xxx\
good that's perfect since MLTools checks only the DCIM folder, since there should there the captured stuff.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 14, 2013, 11:53:57 AM
raw2dng (reference implementation, command-line tool)

Source code: raw2dng.c (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/lv_rec/raw2dng.c)
Windows executable: raw2dng.exe (https://bitbucket.org/hudson/magic-lantern/downloads/raw2dng.exe) (also runs in Wine, for Mac and Linux)

Usage: drag and drop the .raw file over the executable, or use the command line.

Linux shell script for conversion to mjpeg: raw2avi.sh (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/raw_rec/raw2avi.sh)

here (https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dngOSX.zip) a link to the raw2dng compiled for mac (no wine needed)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 14, 2013, 12:04:34 PM
Mac executable: raw2dngOSX.zip (https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dngOSX.zip)
Usage: drag and drop the .raw file over the executable, or use the command line.

just a small note, for osx drag and drop a .raw on the raw2dng executable don't works and double clicking it will run it without any raw file selected
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: HD Cam Team on May 14, 2013, 12:14:45 PM
Hi,

Great idea about this thread.

Is it possible to somehow use FFmpeg to get video directly from the current .raw file?

http://www.ffmpeg.org/index.html

Thanks
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 14, 2013, 12:16:34 PM
Yes, the Linux script does just that.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: electronicwomble on May 14, 2013, 12:34:42 PM
Thanks so much for all your work on this! Your turnaround times for all this dev work are amazing :)

Any ideas about where to look for converting DNGs to CinemaDNGs ready for Resolve?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: HD Cam Team on May 14, 2013, 12:53:41 PM
Yes, the Linux script does just that.

Something for Windows?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: g3gg0 on May 14, 2013, 12:55:21 PM
Something for Windows?
no all-in-one tool yet.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Yoshiyuki Blade on May 14, 2013, 01:12:41 PM
Regarding audio recording with the raw footage, is there some kind of nonstandard sample rate that the audio recorder goes by? I could not properly sync the audio with the video no matter what at 48KHz or 44.1KHz, but if I set it to something weird like 46.6KHz, it was kinda close to sync. The footage in general is kinda jumpy and has weird stutters (not frameskipping, the module reported no dropped frames and the buffer was never exceeded) so idk if the weird jumps could cause a problem with the audio sync or if the audio is messed up too.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: g3gg0 on May 14, 2013, 01:22:59 PM
yet audio recording is not scope of this module, i (and i bet alex too) didnt try too much.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 14, 2013, 01:39:01 PM
just a small note, for osx drag and drop a .raw on the raw2dng executable don't works and double clicking it will run it without any raw file selected

Nice to have the files in a subfolder. Next step would be to be able to throw the on the mac-app script all at once. Is it hard to do Scrax? Now I,m extracting them one by one.
Thanks!
/D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Yoshiyuki Blade on May 14, 2013, 01:46:55 PM
yet audio recording is not scope of this module, i (and i bet alex too) didnt try too much.

Ah indeed. I understand that audio is not the focus your efforts atm so it's not a big issue for me either. I was curious to see how much "overhead" the audio recorder would add while recording RAW and it turns out to be virtually negligible. The syncing issues might've existed before RAW recording too, but I never explored it. After fooling around a little more, I needed to adjust it to about 46.8KHz to look reasonably synced in the ~1.2 mins my card does in RAW.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: olik on May 14, 2013, 02:07:32 PM
Hey folks,

I do not have after fx and I failed to open the DNGs with Davinci resolve software, FCX and premiere pro, any other software that could work?

thanks for help

o.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 14, 2013, 02:14:11 PM
I use ufraw.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: cinema5D on May 14, 2013, 02:24:25 PM
As posted in the other article we put together a small GUIDE covering the basic steps to get the 5D recording RAW: http://c5d.at/1o4
Please give us feedback if anything in there is wrong or easier to achieve or outdated and we will update the guide.

Also we would like to let you know that the DNG format is extremely unwelcome to non-photographers. If there is any way to get cinemaDNG files instead, that would be very helpful to many.
But hey, it's amazing to be at this point here, so we will all be patient for this to evolve.

Thanks and good light!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ch_d on May 14, 2013, 02:31:18 PM
All questions regarding RAW VIDEO POST PROCESSING on any OS
HERE

What would be really great: Something like nice workflow for proxy editing:
a) convert the raw to dng (allready here, called raw2dng)
b) ! in the same step as raw2dng make a proxy for editing (like mjpeg or even prores proxy)

so you can edit the files and reconnect them with the raw dng afterwards. like online/offline editing.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: olik on May 14, 2013, 02:33:41 PM
I agree with cinemaDNG, if it helps, I've found some CinemaDNG white-papers http://download.macromedia.com/pub/labs/cinemadng/cinemadng_p1_spec_091009.pdf
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Colemar on May 14, 2013, 03:36:25 PM
scrax,
   raw2dng for Mac is not able to resolve filepaths with name spaces.  Once I copy the .raw dataset to a contiguous filepath name, converter works great.

EDIT:  Unrelatedly, has anyone noticed .raw files that record to 4GB or larger won't convert in raw2dng?  On Mac OSX I get an "this ain't a lv_rec file" error.

 thanks.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 14, 2013, 03:50:11 PM
Are these files larger than 4GB or they are cropped at 4GB?

The metadata is at the end of the file, so if it's missing, all you can do is to fill in the metadata yourself or hack the source code of the converter.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Colemar on May 14, 2013, 03:58:16 PM
Are these files larger than 4GB or they are cropped at 4GB?

The metadata is at the end of the file, so if it's missing, all you can do is to fill in the metadata yourself or hack the source code of the converter.

they are larger, usually ~4.28GB according to OSX Finder properties.  It seems like a corrupt header because I also have a couple ~2.07GB .raw files that are not recognised in the same manner.  Is this a quick fix or do you want a smaller file Alex?

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: mucher on May 14, 2013, 03:58:39 PM
Thanks for the white paper. Good exercise for the brain.

Here I've found a link to convert DNG to tiff first, saidly, Resolve can open the tiff sequence.

http://www.completedigitalphotography.com/?p=521
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 14, 2013, 04:05:48 PM
Ah, I know. Change the converter source so it uses 64-bit integers, and should work.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Colemar on May 14, 2013, 04:15:36 PM
Change the converter source so it uses 64-bit integers

I bet you're right, scrax do you have time to recompile for 64bit OSX?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: 1% on May 14, 2013, 04:18:30 PM
So for 4GB converter has to be modified for 64 bit integers too? I'm having problems displaying uint64s with bmprintf... on a real os the converter printing functions should be ok?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Colemar on May 14, 2013, 04:34:25 PM
So for 4GB converter has to be modified for 64 bit integers too?

yes but more specifically to run in the 64bit kernel I think, because I have unrecognised .raw files that are ~2.1GB also.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 14, 2013, 05:56:02 PM
Nice to have the files in a subfolder. Next step would be to be able to throw the on the mac-app script all at once. Is it hard to do Scrax? Now I,m extracting them one by one.
Thanks!
/D
MLTools  do that,it scan the card for file to import/convert/merge this is not yet added but will be
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 14, 2013, 05:56:51 PM
I bet you're right, scrax do you have time to recompile for 64bit OSX?
I have to try, not sure how to do, is there some special option to set?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 14, 2013, 07:32:54 PM
MLTools  do that,it scan the card for file to import/convert/merge this is not yet added but will be

What a nice tool. Didn,t know about It until now. I,ll keep en eye. How about donation link? Is it up and running?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: bensyverson on May 14, 2013, 07:48:35 PM
Right now sizeof(lv_rec_file_footer_t) is reading 200 bytes, whereas the actual footer (from a Canon 5D II) seems to be 192 bytes... What am I missing?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 14, 2013, 08:09:01 PM
The code is not ready for 64-bit, you need to tweak it. I have no experience with 64-bit code.

I've compiled it with -D_FILE_OFFSET_BITS=64 in 32-bit mode and works with files greater than 4GB too (at least for me).
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: bensyverson on May 14, 2013, 08:24:57 PM
It would be helpful to know how many bytes each of the fields in the footer should be... Is that information available somewhere? If so then I should be able to get it working for 64 bit...
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 14, 2013, 10:09:32 PM
The code is not ready for 64-bit, you need to tweak it. I have no experience with 64-bit code.

I've compiled it with -D_FILE_OFFSET_BITS=64 in 32-bit mode and works with files greater than 4GB too (at least for me).
I've updated the file in the link so osx user try and report back, if now it works.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: aaphotog on May 14, 2013, 10:19:19 PM
I've updated the file in the link so osx user try and report back, if now it works.
which link?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Ultiva on May 14, 2013, 11:26:24 PM
Hi!

First of all
Big THX to the ML Team for making this possible!
If this was a religion you guys could all live in a nice palace in Rome!

Anyways - i was wandering - How much noise reduction do you people apply in your postprocessing workflow? When I create prores4444 clips with after effects my clips have quiet a bit of digital noise- Removal is easy with adobe cam RAW but  is this normal ? WHat are your experiences?-I mean my "RAW-pictures" usually dont need that kind of "attention".
And btw we are talking daylight shots at ISO 160. ;)

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Rush on May 14, 2013, 11:34:00 PM
Ultiva, just a little advice - with H.264 it is best to expose to the middle of histogram:
(http://upload.wikimedia.org/wikipedia/commons/3/3c/Histogram-Normal.png)

with RAW it is optimal to expose to the right:
(http://upload.wikimedia.org/wikipedia/commons/a/aa/Histogram-ETTR.png)

To not deal with noise in underexposed areas.

btw, looking forward for RAW histogram to work in movie mode.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: stevegalle on May 15, 2013, 12:44:01 AM
I've updated the file in the link so osx user try and report back, if now it works.

Hi Scrax, if you're talking about the files here: https://bitbucket.org/600dplus/magic-lantern-things/downloads
unfortunately they're still not working for >2GB

Steve
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Colemar on May 15, 2013, 12:48:34 AM
I've updated the file in the link so osx user try and report back, if now it works.

stevegalle is correct, it is the same error message:
Quote
Error: This ain't a lv_rec RAW file

ls: *.dng: No such file or directory
Nothing converted

it happens on any .raw file I have tried larger than 2GB
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: nkeppol on May 15, 2013, 02:01:28 AM
Thanks for all the hard work ML team! This is all super exciting. I'm going to donate again, you guys deserve it!

I just wanted to chime in with my experiences.

I've been using a 90MB/s San Disc Extreme Pro 32GB CF card and recording 1920x1080 without a problem. File sizes seem to be around 4.27 GB before it stops recording.

stevegalle is correct, it is the same error message:
it happens on any .raw file I have tried larger than 2GB

I've successfully converted and opened files under 2GB but I'm experiencing the same >2GB error messages in raw2dng on OSX Mountain Lion.

Happy to test any possible solutions. 
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: mjneubrander on May 15, 2013, 03:50:46 AM
I found a way to get prores4444/ap4h using free tools ... without adobe software... It is quite quick. I do not have any editing software to test the video out on. Opening it in kmplayer confirms ap4h but it looks a bit over exposed. Give me a heads up if this is not right or feel free to alter it and write a working version.
 
Setup
1. Download AnotherGUI 1.6 and prores4444 preset from http://www.authorityfx.com/encoding-videos-in-prores-4444-on-windows/
2. Optional enjoy the youtube tutorial or read the page on using AnotherGUI
3. Download AnotherGUI 1.7 beta from http://doom10.org/index.php?topic=1943.0. This one actually works with win 7
4. Extract both versions and copy the "AnotherGUI Presets.xml" from the directory with AnotherGUI 1.6 to the directory of AnotherGUI 1.7 beta and overwrite it. This has the prores4444 preset in it unless you wish to write you own ffmbc command line code.
5. Grab ffmpeg http://ffmpeg.zeranoe.com/builds/ I used ffmpeg-20130514-git-56ba331-win64-static. AnotherGUI 1.7 beta will be looking for ffmpeg when it starts.
6. Extract ffmpeg with your choice of compression software.
7. Copy all "ffmpeg.exe", "ffplay.exe", and "ffprobe.exe" to the directory with AnotherGUI 1.7 beta
8. Grab ffmbc from http://code.google.com/p/ffmbc/ I used FFmbc-0.7rc8-win64
9. Extract ffmbc with your choice of compression software.
10. Copy the "ffmbc.exe" into the folder with AnotherGUI 1.7 beta

Process
1. Extract dng from raw using raw2dng.exe. ~ 30s / 1000 frames
2. Use ImageMagick. Open a command prompt and change directory to the where the dng files are execute "mogrify -format tiff *.dng" to convert the dng to a sequence of high bit depth tiff files. ~5min 40s / 1000 frames
3. Open AnotherGUI 1.7 beta. Select the first file in the tiff sequence. Select the prores4444 preset. Press Go to convert the sequence of tiff to a .mov. ~1min 45s / 1000 frames

Testing done on a SandyBridge i7 laptop with 1003 frames of 1280x540 raws

Visit the AnotherGUI developers preset gallery for other video formats
http://www.stuudio.ee/anothergui/presets.html
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: silvertonesx24 on May 15, 2013, 04:37:40 AM
When I run raw2dng off my system drive (SSD) where both the application and raw file are in the same folder, it works great.

When I try to run raw2dng off a different internal drive, I get

Error: could not open /Volumes/etccccc
and
mkdir: /1302172738: Permission denied

What simple solution am I missing here?

OSX, 10.7
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: paramon on May 15, 2013, 05:53:26 AM
even after setting AE default preferences to 24fps instead of 30, i still get weird  motion
what's missing?

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: zmelms on May 15, 2013, 06:01:07 AM
Thanks for all the hard work ML team! This is all super exciting. I'm going to donate again, you guys deserve it!

I just wanted to chime in with my experiences.

I've been using a 90MB/s San Disc Extreme Pro 32GB CF card and recording 1920x1080 without a problem. File sizes seem to be around 4.27 GB before it stops recording.

I've successfully converted and opened files under 2GB but I'm experiencing the same >2GB error messages in raw2dng on OSX Mountain Lion.

Happy to test any possible solutions.

Same issue here,  Also it appears I have to do the files one at a time and wait for my clock to advance to a new minute as that's how it's naming the files, based on the clock.  Ideas,  updated download?

Thanks, Z
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: kgv5 on May 15, 2013, 12:06:57 PM
Guys

i tried to use twixtor and it seems to have a problem with frame blending in RAW DNG sequences.

I thought I could use FPS override sometimes (17-20 fps) to have better resolution with not so much movement (6D and 37-39 mb/s write speed).

If sameone had any successes with things above plese reveal some good workflow.

Thanks
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: tin2tin on May 15, 2013, 01:12:01 PM
If the image sequence is tif, png, jpg, bmp or tga and the sequence starts with 0,1,2,3 or 4 (ex. Img_0000.png) then EyeFrame Converter can be used to convert into various formats like DNxHD, Prores, MJPEG and MPEG2 I Frame HD. If there is a similar named wav file located in the same folder as the image sequence it will automatically be added to the video.

http://eyeframeconverter.wordpress.com
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: sicetime on May 15, 2013, 05:06:12 PM
This is interesting to me, that this thread hasn't gotten more attention.

I would say one of the hardest part of using this new amazing feature is the LENGTHY post process.

Am I right in saying that raw2dng and then after effects is the best way right now (mac)? This process takes AGES for me, anyone come up with something better/faster?

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: paramon on May 15, 2013, 05:16:15 PM
it's still faster than film :)

BIG thank you to the ML team, the raw image out of 5D is just stunning, and it is real RAW
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on May 15, 2013, 05:19:11 PM
This is interesting to me, that this thread hasn't gotten more attention.

Everyones out shooting or typing.

AE is the only show in town until raw2dng does Cinema DNG, but I doubt it will be much faster. RAW is big, big is slow.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 15, 2013, 05:59:21 PM
I,d say lightroom works pretty good aswell though some complications when recovering highlights might occur. Might not be even with every frame. Is there not a conversion tool that creates regular dng-prores?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 15, 2013, 11:21:13 PM
I tried converting with photoshop camera raw which is doing a clearly better synchronised job than lightroom. Lightroom creates all kinds of inconsistencys in diferent frames. Too bad, really like the idea to batch process the workflow in lightroom.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 15, 2013, 11:37:04 PM
Talking to myself here ;). I work with acr 6,7 rather old version still it works good. I then tried lightroom 3 on my mac and that version also worked rather good. Lightroom 4 is not doing a ggod job when applying different recovery actions.
//D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: duncanidaho25 on May 16, 2013, 12:20:48 AM
After using RAW2DNG I imported the files into Adobe Camera RAW
I noticed that down below the images it says  8 bit DNG.  It gave me an option of changing this to 16 bit (but not 14). 

I also noticed that it gives you an option to change the size:

1880 x 840 (1.6 MP) (the size I shot the RAW files in.  I was able to get 4 or 5 seconds before dropped frames on my 5d mark ii)
2048 x 915 (1.9 MP)
3072 x 1373 (4.2 MP)

Would this be a good (or the best) way (with minimal loss of quality) to upsize for 1080p or even larger?
It also gives you a chance to change the resolution.  What would be best 72?  300?  Sharpen for Screen I imagine is best.  Thanks for your help and expertise!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 16, 2013, 12:39:22 AM
It,s 240dpi/inch, equivalent to 94dpi/cm
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: jwcarr on May 16, 2013, 01:36:29 AM
Does anyone have a suggestion as to why I am getting the error message: "This ain't a lv_rec RAW file" when trying to convert my .RAW file with raw2dng on a Mac? How can I get this working?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: jwcarr on May 16, 2013, 02:05:37 AM
Looks like it is an issue when I shoot at 1920x1080. I just got raw2dng to work on a 1920x720 file.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Colemar on May 16, 2013, 02:44:19 AM
Does anyone have a suggestion as to why I am getting the error message: "This ain't a lv_rec RAW file" when trying to convert my .RAW file with raw2dng on a Mac?

the devs are working on it.  It looks like a problem working with 64bit integers as opposed to the 32bit current build, but a developer can possibly give more info
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: 1% on May 16, 2013, 03:00:46 AM
It errors for me on >4GB files in windows.. so something with 64bit fix doesn't work. I haven't tried linux to do a >4gb file yet tho.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Colemar on May 16, 2013, 03:20:03 AM
It errors for me on >4GB files in windows..

is that a build in-development?  The latest Windows build from a1ex https://bitbucket.org/hudson/magic-lantern/downloads/raw2dng.exe (https://bitbucket.org/hudson/magic-lantern/downloads/raw2dng.exe) worked fine for me all day on any filesize I threw at it..
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: 1% on May 16, 2013, 04:03:02 AM
I'll try it, could be my compiler.

Nope.. over 4gb file didn't process.. the 10gb didn't either. Its also much larger for some reason.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Colemar on May 16, 2013, 04:25:45 AM
Nope.. over 4gb file didn't process.. the 10gb didn't either.

just exits immediately or throws a message?  I've noticed on Mac OSX it will not recognise filepaths that are not contiguous.  any spaces and it throws an error message with the requested filepath truncated at the space..that could also be an issue in the Windows build..Linux ok?

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: 1% on May 16, 2013, 04:44:47 AM
Dunno, what is the biggest file you've processed?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Colemar on May 16, 2013, 04:48:38 AM
Dunno, what is the biggest file you've processed?

4.27GB on Windows build of raw2dng..just installed the new build but haven't tested yet.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: 1% on May 16, 2013, 04:55:37 AM
4.27 is at the 4gb limit not over.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Colemar on May 16, 2013, 05:05:18 AM
that's interesting...essentially, what is raw2dng doing to the .raw file?  does it just parse it into individual frames, or does it also de-bayer, or something totally different?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: bryanfieldhouse on May 16, 2013, 12:05:28 PM
I'm wondering if this device would recognise the ***.raw file on the compact flash card and back it up to the inbuilt sata hard drive?

That way you wouldn't have to spend so much on compact flash cards and could be backing up a card while shooting with another...  8)

http://www.creativevideo.co.uk/index.php?t=product/nextodi_nd-2730

Bryan
Title: RAW2DNG BATCHelor
Post by: fatpig on May 16, 2013, 01:12:08 PM
Here is a little Batch tool i wrote for Windows, maybe it is of use to some. :)

RAW2DNG BATCHelor 1.0 Beta

With permission I can include the raw2dng.exe in it.
If ImageMagick is available inside its Folder it can generate a thumbnail aswell.
Let me know what you think.

It seems ImageMagick has to be installed, and (for version 1.0) copied to the folder.

http://fatpigtures.com/software/
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 16, 2013, 02:16:55 PM
Workflow right now accordingly:

1 - import dng to after effects (followed Neumanns tutorial) http://www.youtube.com/watch?v=0fGIIAQXXxQ
2 - instead of editing in after effects I export the file to prores 422HQ (following tutorial how to get proresHQ in adobe after effects 5) http://www.youtube.com/watch?v=QRXCdI7yXLA
3 - import the proresHQ to preffered software, I use adobe premiere pro cs 5
Title: Any way to fix files with "lv_rec" error?
Post by: CrewOfOne on May 16, 2013, 07:13:41 PM
A previous comment mentioned that the  "lv_rec" error results from a missing file footer ... Is that correct? More to the point, is there anyway to fix this manually? I spent the whole morning carefully shooting lovely RAW vs. H264 comparison footage, and when raw2dng rejected the files, it was rather disappointing.

I understand all the usual caveats about Magic Lantern and testing before shooting, but if there were a way to manually fix these files, I would greatly appreciate hearing about it!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Ultiva on May 17, 2013, 12:31:25 AM
Hey ML-Forum- and especially the mac users.

Yes the new GUI raw2dng -App for MAC is quiet cool and working like a charm unless you try to convert files >2GB-

I know - we have read the 64 bit theory about it- but did anyone get it to work? Because unfortunately i havent(tried the GUI -App and the raw2dng terminal command)

Thanks guys!
Ultiva
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: andyshon on May 17, 2013, 01:23:42 AM
The Mac GUI seems a little fussy about file path. It likes obvious places, like desktop, movies, etc. Pop 'em deep in the directory structure and it says no such path.

Also, over half my clips wont convert, either on Mac or on Windows. I get either:

Error: This ain't a lv_rec RAW file
or
Error: API version mismatch: -2147483629

Seems random. All these clips are 1880x720 shot on the 5DmkII. Sometimes it's the big ones that work so doesn't seem to be about size.

Any ideas?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 17, 2013, 01:35:48 AM
Hey ML-Forum- and especially the mac users.

Yes the new GUI raw2dng -App for MAC is quiet cool and working like a charm unless you try to convert files >2GB-

I know - we have read the 64 bit theory about it- but did anyone get it to work? Because unfortunately i havent(tried the GUI -App and the raw2dng terminal command)

Thanks guys!
Ultiva

So far seems not working even with the last patches.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 17, 2013, 01:40:25 AM
Added a know issue list to this topic for the Osx GUI version
raw2dgn.app
GUI version

Know issues
- Don't works with spaces in folder names for .raw source
- Can't convert file bigger than 2Gb.

raw2dng
Osx command line

Know issues
- Can't convert file bigger than 2Gb.

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 17, 2013, 01:45:31 AM
just exits immediately or throws a message?  I've noticed on Mac OSX it will not recognise filepaths that are not contiguous.  any spaces and it throws an error message with the requested filepath truncated at the space..that could also be an issue in the Windows build..Linux ok?
this is a problem with the GUI wrapper converting space wrong, win and linux works more like the command line for osx.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: cinema5D on May 17, 2013, 02:16:35 AM
Here's the simplest post workflow in OSX we found around here, and we think it's most easy after going through hell two days ago: https://vimeo.com/66355682 (https://vimeo.com/66355682)

Thanks for your work here. Will try Lightroom tomorrow.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: KMA_WWC on May 17, 2013, 03:23:22 AM
Also found a small bug for a Mac OS app. If there are files with the same naming convention in the same folder, app won't convert it. For example: I have my M0000001.RAW file and when I export a new video/raw file with the same name (it ask me if I want to keep both and I say yes), I get M0000001 2.RAW which won't convert for some reason. Not sure if anyone reported it, but though it could help in the future.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on May 17, 2013, 05:35:49 AM
FOR MAC

- Take .DNG Files and open with ADOBE CAMERA RAW - ALL THE IMAGES (You can change exposure, color, curves, etc.)
- SAVE AS TIFF (Or JPG, less quality)
- OPEN IMAGE SEQUENCE With COMPRESSOR
- Put the preset that you want (Pro Res HQ in my case).
- Use the video editor of your preference (Final Cut X in my case)

Enjoy! :D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 17, 2013, 11:00:20 AM
Also found a small bug for a Mac OS app. If there are files with the same naming convention in the same folder, app won't convert it. For example: I have my M0000001.RAW file and when I export a new video/raw file with the same name (it ask me if I want to keep both and I say yes), I get M0000001 2.RAW which won't convert for some reason. Not sure if anyone reported it, but though it could help in the future.

have you read the know issue? it's the space in the path the problem.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: CrewOfOne on May 17, 2013, 12:35:05 PM
Great tip about Compressor, budafilms!

For those who don't have Compressor, here's an alternate method:

- Use Adobe Camera Raw (within Photoshop) to open DNG and save as TIF sequence
- Within FCP or Premiere: Settings -> General -> Still Frame Duration: 1 Frame
- Drag and drop TIF sequence folders onto sequence
- Export as preferred flavor of ProRes, then re-import
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 17, 2013, 01:29:01 PM
An easy to follow mac-tutorial focusing on fast conversion to AppleProRes422HQ and a bunch of other stuff like creating xmp-templates, setting upp after effects, premiere pro.

http://www.youtube.com/watch?v=MSHVi7Jx9r0&feature=youtu.be

https://vimeo.com/66390822

Thanks!
//D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: cinema5D on May 17, 2013, 02:07:23 PM
Hey guys,

We're happy we've got a working 25p mode. LINK (http://www.cinema5d.com/news/?p=18082)
Only we would really like to convert larger files. The raw2dng app for Mac seems to be unable to work with the larger files. Is there any fix for that in sight? Thanks a lot for your efforts!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 17, 2013, 02:10:45 PM
Hey guys,

We're happy we've got a working 25p mode. LINK (http://www.cinema5d.com/news/?p=18082)
Only we would really like to convert larger files. The raw2dng app for Mac seems to be unable to work with the larger files. Is there any fix for that in sight? Thanks a lot for your efforts!

I second that. Really need to be able to convert larger files on a mac.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 17, 2013, 02:15:21 PM
Quote
We're happy we've got a working 25p mode.

LOL, it worked from the beginning... and you don't need FPS override for it, just use Canon menu.

The Windows converter should handle files larger than 4GB (I've tried under Wine).
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 17, 2013, 02:48:29 PM
LOL, it worked from the beginning... and you don't need FPS override for it, just use Canon menu.

The Windows converter should handle files larger than 4GB (I've tried under Wine).

When it comes to wine I feel like a retard. I never seem to get it to work and I have never seen anyone give any steps on howto either. It,s on my computer though so feel free to explain if you have the time Alex or anyone else :).
Thanks a lot anyway.
cheer
//D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 17, 2013, 03:06:13 PM
I run it like this in terminal: wine raw2dng.exe video.raw

http://www.davidbaumgold.com/tutorials/wine-mac/
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on May 17, 2013, 07:25:17 PM
When it comes to wine I feel like a retard. I never seem to get it to work and I have never seen anyone give any steps on howto either.

it's wine that's retarded, whoever developed it was obviously pissed at the time. After testing tomorrow I'll down a good bottle of red and give it another crack, it should start to make sense then. If all else fails I'll get my 7 year old to figure it out for me.

We could approach the 5DtoRGB devs, maybe they could help. I'm happy to contact them myself but I don't speak bocce.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Habitat on May 17, 2013, 09:04:43 PM
Anyone know the terminal commands to activate raw2dng through there? It was working fine but now doesnt on my mac...kinda heartbreaking
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 17, 2013, 10:08:20 PM
it's wine that's retarded, whoever developed it was obviously pissed at the time. After testing tomorrow I'll down a good bottle of red and give it another crack, it should start to make sense then. If all else fails I'll get my 7 year old to figure it out for me.

We could approach the 5DtoRGB devs, maybe they could help. I'm happy to contact them myself but I don't speak bocce.

 ;D ;D ;D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: sicetime on May 18, 2013, 12:51:44 AM
Is there any update on the mac gui app that can handle over 2gb files or is it just not possible to do? anyway a simple tester can help?

Thanks!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: zachnfine on May 18, 2013, 12:55:07 AM
I took my AdobeDNG files and moved them into a replication of the CinemaDNG directory structure (following the 'MyMovieNickPatonIkonoskopDNG' example supplied on Adobe's CinemaDNG page). I then imported that folder into Davinci Resolve's media pool. The result was the same as when I imported the DNG sequence into resolve without the directory structure -- the sequence came in and had the right number of frames and duration, but Resolve thinks each 1920x1080 frame is 128x96 and renders them as a super-low-resolution low bit depth image (looks to be posterized down to about 4 colors).

Maybe there's something needing a tweak in the dng headers created by Raw2dng.exe? The dng files load just fine in Adobe Camera Raw, but DaVinci Resolve sees them as 128x96.

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: sicetime on May 18, 2013, 12:58:11 AM
Also, I am running vmware fusion and when I try to open raw2dng.exe it just crashes automatically, any idea why that's happening?


The box pops up for a split second, then disappears, it's weird.

running windows xp through vmware fusion on a older snow leopard macbook pro (please don't judge).

TIA
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on May 18, 2013, 06:28:07 AM
Is there any update on the mac gui app that can handle over 2gb files or is it just not possible to do? anyway a simple tester can help?

Thanks!

There's a known 64bit issue. Anything is possible.
Title: Re: RAW2DNG BATCHelor
Post by: Ultiva on May 18, 2013, 01:59:33 PM
Here is a little Batch tool i wrote for Windows, maybe it is of use to some. :)

RAW2DNG BATCHelor 1.0 Beta

With permission I can include the raw2dng.exe in it.
If ImageMagick is available inside its Folder it can generate a thumbnail aswell.
Let me know what you think.

It seems ImageMagick has to be installed, and (for version 1.0) copied to the folder.

http://fatpigtures.com/software/

Big thanks to fatpig and a1ex for helping me out with my raw files 2GB+ in MACOSX
It is not that hard. Because i am no genius and figured it out- Here is a step by step.

First do as A1ex suggested and use wine. To install it follow the steps he already linked for you:

http://www.davidbaumgold.com/tutorials/wine-mac/#part-0

There is one addition i would like to make though. In the part where he tells you to install Xcode David tells you to download & install it via the appstore. What he doesnt tell you is that after downloading you need to

1. manually open it (it installs in programs) in order to install it
2. go to preferences --> downloads --> and click on install "command line tools"

After you have done so you can continue and follow his instructions in terminal.

Now that you have got wine, download the RAW2DNG Batchelor by fatpig(see above)

Create a folder named RAW on your desktop

Put RAW2DNG.exe and the extracted Raw2DNG_batchelor.exe in this folder.
If you like copy your RAW files into this folder

Open Terminal and type these commands (without the $)

$ cd desktop (press enter)

$ cd RAW (press enter)

$ wine raw2dng_batchelor.exe (press enter)

Now a little window should open . Hit browse - point to the folder where your raws are and finally press "make dngs"

The beautiful thing is- if you have multiple Raws in that folder - it converts all of them and puts it in seperate subfolders.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: sicetime on May 18, 2013, 03:29:36 PM
OK so I'm an idiot, I now realize what drag raw to the executable file means. And it is now working, Thanks to Ultiva who has the wine app working above, I'll follow that later.

Is there a way, on the windows side of things in raw2dng.exe, to specify where the files (dngs) go? All my files end up in documents and settings.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: 1% on May 18, 2013, 04:10:27 PM
On windows I set open with raw2dng.exe.. files end up in the folder the raw file is in.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: VinceProd on May 18, 2013, 07:20:30 PM
I have this message when i try to converte my raw files on my imac :
" Error: This ain't a lv_rec RAW file ls: *.dng: No such file or directory Nothing converted "

???
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 18, 2013, 10:10:15 PM
I have this message when i try to converte my raw files on my imac :
" Error: This ain't a lv_rec RAW file ls: *.dng: No such file or directory Nothing converted "

???

probably too big files. Bigger than 2gb is still a problem converting
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: eoshq on May 19, 2013, 01:26:23 AM
For those using Adobe Photoshop or Lightroom for raw processing.

Photoshop (ACR) and Lightroom have the exact same raw processing engine and results, as long as they are of the same generation.

That is:
Lightroom 5 = ACR 8 (CC and CS6 with June 2013 update)
Lightroom 4 = ACR 7 (CS6)
Lightroom 3 = ACR 6 (CS5)

Different generations, with different process versions, like Lightroom 3 vs Lightroom 4, will give different results.

Starting with Lightroom 4 and ACR 7, Adobe introduced process version 2012, which has some automatic highlight reduction without user control. This is may cause inconsistent rendering from frame to frame. You can switch back to process 2010 in the camera calibration area to turn this feature off, but also loose the advances made in highlight recovery and shadow control made in process 2012.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: CaptainHook on May 19, 2013, 07:51:18 AM
Hi,

I noticed a path with spaces in the raw2dng mac app fails.. i solved this by modifying your script in the package to be like this:

----
#!/bin/sh
time=$(date +"%y%m%d%H%M")
if [[ $1 ]]; then
   fldr="$(dirname "$1")"/
   ./raw2dng "$1"
   if [[ $(ls *.dng) ]]; then
      echo "Moving .dng to $fldr$time"
      mkdir "$fldr$time"
      mv *.dng "$fldr$time"
   else
      echo "Nothing converted"
   fi
else
   echo "DRAG SOMETHING IN THE APP"
fi
exit 0

----
Works okay now for me.

Thanks again everyone at ML for all the hard work. :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 19, 2013, 09:45:47 AM
Hi,

I noticed a path with spaces in the raw2dng mac app fails.. i solved this by modifying your script in the package to be like this:

----
#!/bin/sh
time=$(date +"%y%m%d%H%M")
if [[ $1 ]]; then
   fldr="$(dirname "$1")"/
   ./raw2dng "$1"
   if [[ $(ls *.dng) ]]; then
      echo "Moving .dng to $fldr$time"
      mkdir "$fldr$time"
      mv *.dng "$fldr$time"
   else
      echo "Nothing converted"
   fi
else
   echo "DRAG SOMETHING IN THE APP"
fi
exit 0

----
Works okay now for me.

Thanks again everyone at ML for all the hard work. :)

Where did you put that new information? I,m a noob on this one. Maybe you could put the new one up for download?
Thanks.
//D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on May 19, 2013, 10:02:40 AM
I did some timed post workflow tests on my 2.5Ghz quad core Macbook Pro 16Gb 480Gb SSD 1Gb Radeon 6770M. A 1Gb raw file (17seconds @ 1920x720) takes 10 sec to convert to DNG and 2:22 to transcode to ProRes 4444 in After Effects. A 17 sec H.264 file takes 30 sec to transcode to Prores 4444 in compressor. That's 5x longer for raw but I imagine it would be significantly faster with a CUDA GPU and slower with a spinning disk drive. Adding a film convert grade to the ProRes transcode took an extra 10 sec. Transcoding to ProRes proxy was 10 sec faster than 4444.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: CaptainHook on May 19, 2013, 10:57:00 AM
Where did you put that new information? I,m a noob on this one. Maybe you could put the new one up for download?
Thanks.
//D

-> Right click on the app and choose "show package contents"
-> Go into Contents, then Resources
-> Open the file named "Script" in a text editor and replace with what i posted above - save and reopen the app.

I would rather not post a new download and confuse matters - i'm sure scrax or someone else from ML will update the app if they choose to. :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: CaptainHook on May 19, 2013, 12:37:13 PM
Any ideas about where to look for converting DNGs to CinemaDNGs ready for Resolve?

The DNG's will open in Resolve and play as a single clip but they're not interpreted correctly. Running them through Adobe's DNG convertor helps (the aspect ratio is at least displayed correctly after doing this) but resolve still doesn't know how to adjust wb/tint/etc for the files.

Checking the DNG's from raw2dng with dng_validate shows the following problems:

*** Warning: IFD 0 Model is not NULL terminated ***
*** Warning: IFD 0 UniqueCameraModel is not NULL terminated ***
*** Warning: The ExposureTime is <= 0 ***
*** Warning: Too little padding on left edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on top edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on right edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on bottom edge of CFA image (possible interpolation artifacts) ***

My hunch is once those are corrected in chdk-dng.c, resolve will correctly read the DNG's since they should be to be spec. Running dng_validate on BMCC dng's produces no warnings and checking them with the verbose flag shows lots of XMP data that gives WB/tint/etc info that i'm sure will eventually come from ML for raw2dng. :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 19, 2013, 01:05:14 PM
Can you send me a DNG from BMC?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: CaptainHook on May 19, 2013, 01:28:57 PM
Sent you a PM with links to a couple of frames, hope it helps. :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 19, 2013, 01:33:19 PM
Hi,

I noticed a path with spaces in the raw2dng mac app fails.. i solved this by modifying your script in the package to be like this:

----
#!/bin/sh
time=$(date +"%y%m%d%H%M")
if [[ $1 ]]; then
   fldr="$(dirname "$1")"/
   ./raw2dng "$1"
   if [[ $(ls *.dng) ]]; then
      echo "Moving .dng to $fldr$time"
      mkdir "$fldr$time"
      mv *.dng "$fldr$time"
   else
      echo "Nothing converted"
   fi
else
   echo "DRAG SOMETHING IN THE APP"
fi
exit 0

----
Works okay now for me.

Thanks again everyone at ML for all the hard work. :)

thank's captain Hook, will try it and update the linked zip.

EDIT: here the fixed new version: raw2dng.app beta 0.2 (https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dng.app.0.2.zip)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: CaptainHook on May 19, 2013, 01:50:48 PM
Glad to have helped. :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ZUPPORT on May 19, 2013, 01:52:46 PM
Canon 5d3
ML 14Bit uncompressed RAW video File (1.2GB)
raw2dng 2.app

and it does not work !

--------------------------------------------------

raw2dng converter GUI for OsX
Beta ver.0.2
Error: API version mismatch: 15052800

ls: *.dng: No such file or directory
Nothing converted


--------------------------------------------------

any ideas ??

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 19, 2013, 02:00:13 PM
Canon 5d3
ML 14Bit uncompressed RAW video File (1.2GB)
raw2dng 2.app

and it does not work !

--------------------------------------------------

raw2dng converter GUI for OsX
Beta ver.0.2
Error: API version mismatch: 15052800

ls: *.dng: No such file or directory
Nothing converted


--------------------------------------------------

any ideas ??

delete any older version and redownload it, seems you have unzipped it in the same place of older version?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ZUPPORT on May 19, 2013, 02:03:02 PM
i deleted all older versions & i reboot the mac...

same:


--------------------------------------------------

raw2dng converter GUI for OsX
Beta ver.0.2
Error: API version mismatch: 15052800

ls: *.dng: No such file or directory
Nothing converted


--------------------------------------------------
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 19, 2013, 02:03:19 PM
Canon 5d3
ML 14Bit uncompressed RAW video File (1.2GB)
raw2dng 2.app

and it does not work !

--------------------------------------------------

raw2dng converter GUI for OsX
Beta ver.0.2
Error: API version mismatch: 15052800

ls: *.dng: No such file or directory
Nothing converted


--------------------------------------------------

any ideas ??

Whenever I move the app it can,t find the directory. Also try Captainhooks changes in the script.

-> Right click on the app and choose "show package contents"
-> Go into Contents, then Resources
-> Open the file named "Script" in a text editor and replace with what i posted above - save and reopen the app.

Script:

#!/bin/sh
time=$(date +"%y%m%d%H%M")
if [[ $1 ]]; then
   fldr="$(dirname "$1")"/
   ./raw2dng "$1"
   if [[ $(ls *.dng) ]]; then
      echo "Moving .dng to $fldr$time"
      mkdir "$fldr$time"
      mv *.dng "$fldr$time"
   else
      echo "Nothing converted"
   fi
else
   echo "DRAG SOMETHING IN THE APP"
fi
exit 0

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on May 19, 2013, 02:08:34 PM
Version 0.2 is working fine for me.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 19, 2013, 02:10:38 PM
For me too, I've tried to mode it in different folder, HD and it always worked even if the raw was in a folder with spaces.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 19, 2013, 02:12:20 PM
delete any older version and redownload it, seems you have unzipped it in the same place of older version?

Hi Scrax! Thanks for your app. I have a couple of questions. When I convert a raw stream it works good but when trying to convert the next one often it ends up with nothing since it doesn,t create a new folder? Do I have to rename the first converted folder before proceeding to the next raw-stream? Do you think it will be possible i a near future to be able to convert large then 2gb files with the app :)?
Thanks again
//D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 19, 2013, 02:13:01 PM
Version 0.2 is working fine for me.

Where is the 0.2 version??
//D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on May 19, 2013, 02:16:25 PM
scroll up

I'm getting the same error message on the last file that filled the card, must be missing some data.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on May 19, 2013, 02:21:02 PM
Do you think it will be possible i a near future to be able to convert large then 2gb files with the app :)?

I imagine eventually someone will develop a lossless compressed CinemaDNG batch converter, but for now we work with what we've got.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 19, 2013, 02:26:37 PM
Hi Scrax! Thanks for your app. I have a couple of questions. When I convert a raw stream it works good but when trying to convert the next one often it ends up with nothing since it doesn,t create a new folder? Do I have to rename the first converted folder before proceeding to the next raw-stream? Do you think it will be possible i a near future to be able to convert large then 2gb files with the app :)?
Thanks again
//D

Here a video of some different way to use it, but I had that problem you mentioned too, since the folder name is based on the date you can convert only one file for minute or it will give an error because the folder is already there, will add seconds too in the 0.3 first topic has links updated.


youtube is processing it so will be ready in a few minutes i suppose
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 19, 2013, 02:34:30 PM
for those who want to manually update the app this is the script I have now:

Code: [Select]
#!/bin/sh
time=$(date +"%y%m%d%H%M%S")
echo "raw2dng converter GUI for OsX"
echo "Beta ver.0.3"
if [[ $1 ]]; then
   fldr="$(dirname "$1")"/
   ./raw2dng "$1"
   if [[ $(ls *.dng) ]]; then
      echo "Moving .dng to $fldr$time"
      mkdir "$fldr$time"
      mv *.dng "$fldr$time"
   else
      echo "Nothing converted"
   fi
else
echo "Drop a ML .RAW video file here"
fi
exit 0

it adds seconds in the folder name so we can convert more than one raw for minute without problems
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: CaptainHook on May 19, 2013, 02:38:24 PM
Error: API version mismatch: 15052800

Have you tried another RAW file? That error is giving an odd api version - it should be "1" according to this check:

if (raw_info.api_version != 1)
FAIL("API version mismatch: %d\n", raw_info.api_version);
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 19, 2013, 02:41:28 PM
beta 0.3 out here: https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dng.app.0.3.zip with the second in folder name fix
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ZUPPORT on May 19, 2013, 02:44:49 PM
it works !!!

so much ! tnx ! tnx !!!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 19, 2013, 02:52:42 PM
beta 0.3 out here: https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dng.app.0.3.zip with the second in folder name fix

That worked beautifull Scrax :)!
How about drag an drop the files all at once, would it be possible or is that asking too much right now?
Thanks a lot, both for the video and the updated script.
//D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 19, 2013, 03:16:09 PM
That worked beautifull Scrax :)!
How about drag an drop the files all at once, would it be possible or is that asking too much right now?
Thanks a lot, both for the video and the updated script.
//D

I've made a topic for the app gui development so I can update it without asking each time to update the first topic to the admin here, let's all discuss all those thing here: http://www.magiclantern.fm/forum/index.php?topic=5508.0
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ZUPPORT on May 19, 2013, 04:01:58 PM
My camera raw version 8.1 (CS6 plugin) don´t open the .dng files !
8.1 is the newest ?! isn't it ?? i dont find a newer update....
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: paramon on May 19, 2013, 05:23:56 PM
i am on 7.0 CS6, it works OK
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: sicetime on May 19, 2013, 06:29:29 PM
On windows I set open with raw2dng.exe.. files end up in the folder the raw file is in.

WORKED PERFECTLY.. THANK YOU!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on May 19, 2013, 06:32:21 PM
An easy to follow mac-tutorial focusing on fast conversion to AppleProRes422HQ and a bunch of other stuff like creating xmp-templates, setting upp after effects, premiere pro.

http://www.youtube.com/watch?v=MSHVi7Jx9r0&feature=youtu.be

https://vimeo.com/66390822

Thanks!
//D
Final Cut X, when import tiff files and put in the timeline, you can do a compound clips, and save the half of space!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Stedda on May 19, 2013, 06:54:45 PM
My camera raw version 8.1 (CS6 plugin) don´t open the .dng files !
8.1 is the newest ?! isn't it ?? i dont find a newer update....

8.1 Beta works just fine for me...
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 19, 2013, 08:44:47 PM
Final Cut X, when import tiff files and put in the timeline, you can do a compound clips, and save the half of space!

Sounds nice, a little like working with nesting i premiere pro? I prefer getting straight to PrRes422 though...
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: eoshq on May 19, 2013, 09:31:16 PM
Would it be possible for raw2dng.exe to somehow add the camera model to the dng file as it is created, so that we can use Adobe and custom camera profiles for the various camera models?

As it stands, the raw profile is shown as embedded and cannot be changed. It would be helpful to be able to use Adobe profiles as well as VSCO and my own custom profiles with raw video just like we can with raw stills for the 5D3.

I am aware the colors may be different, and so maybe the 5D3 raw still profiles will not be optimized for raw video, but even just having the ability to create your own profile with a color checker passport and use them with raw video would be a big plus.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 19, 2013, 09:33:59 PM
Scrax just compiled a new script that allows for dropping a whole folder on the app. Batch-converting that is. Sweet!
Latest builds in here http://www.magiclantern.fm/forum/index.php?topic=5508.0
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: 1% on May 19, 2013, 09:34:10 PM
Yea, I like the 8.1 beta.. its working just fine.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 19, 2013, 10:08:27 PM
The DNG's will open in Resolve and play as a single clip but they're not interpreted correctly.

Does this one (http://a1ex.magiclantern.fm/bleeding-edge/raw/21380018.DNG) open in DaVinci Resolve?

(I couldn't install it)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Andy600 on May 19, 2013, 10:12:54 PM
Nope, not here. It doesn't appear in the library
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: CaptainHook on May 20, 2013, 12:03:09 AM
Does this one (http://a1ex.magiclantern.fm/bleeding-edge/raw/21380018.DNG) open in DaVinci Resolve?

(I couldn't install it)

No, it still has some issues. Putting it through Adobe DNG convertor does allow Resolve to see it though, but with same problems before of no RAW controls like wb/tint/exposure having an effect on the image.

Here's the info from dng_validate for your file above:

http://f.cl.ly/items/2p231l3u1S1r1q3r1M25/ml.txt

This is the file after running it through Adobe DNG convertor (its added preview image, changed bits per sample, etc etc):

http://f.cl.ly/items/0w1l2s451R1H0c1V130E/adobe.txt

Here's a DNG frame from the BMCC for comparison:

http://f.cl.ly/items/362k3x1o471V2p3t2o3T/bmcc.txt

We still have these warnings on your DNG also:

*** Warning: The ExposureTime is <= 0 ***
*** Warning: Too little padding on left edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on top edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on right edge of CFA image (possible interpolation artifacts) ***
*** Warning: Too little padding on bottom edge of CFA image (possible interpolation artifacts) ***

Hope that helps. :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: tin2tin on May 20, 2013, 12:25:22 AM
Would it be possible to add specification of the full path for the import file in "" and a full destination directory in "" to raw2dng?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: eoshq on May 20, 2013, 01:29:34 AM
Does this one (http://a1ex.magiclantern.fm/bleeding-edge/raw/21380018.DNG) open in DaVinci Resolve?

(I couldn't install it)

Awesome... I can use a camera profile with this DNG, because it is recognized by Adobe as being from a 5D3. Thank you.  :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on May 20, 2013, 02:03:46 AM
Would it be possible for raw2dng.exe to somehow add the camera model to the dng file as it is created, so that we can use Adobe and custom camera profiles for the various camera models?

As it stands, the raw profile is shown as embedded and cannot be changed. It would be helpful to be able to use Adobe profiles as well as VSCO and my own custom profiles with raw video just like we can with raw stills for the 5D3.

I am aware the colors may be different, and so maybe the 5D3 raw still profiles will not be optimized for raw video, but even just having the ability to create your own profile with a color checker passport and use them with raw video would be a big plus.

You can create your own profiles in adobe camera raw. I'm working on a log sLUT now, when I'm happy with it I'll share it. I hear visioncolor are creating one and I expect others will share theirs too.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: peoplemerge on May 20, 2013, 03:45:26 AM
I also had the problem with >2G files.  These came from a 5D2 from source built yesterday.

I started by hacking the file with tail -c <various sizes>, same errors.

Next I tried calculating the # frames assuming frame sizes would be fixed from file to file and hardcoding raw2dng.c.  Photoshop couldn't open the resulting files saying the raw format wasn't supported.  No big surprise.

Finally (I'm on a mac) I installed wine and tried fatpig's BATCHelor.  Success!

I'm convinced there's an number size or endianness issue hidden there.  I'll try hacking it later tonight if I have time.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: peoplemerge on May 20, 2013, 04:00:02 AM
I may have spoken to soon...
2 of the 3 files that couldn't be consumed by the mac version worked with fatpig.
The last one (also the largest at 3,099,096,000 bytes) still couldn't be read.

I wonder if on some of these it had to do with the way I was stopping the recording.  On my camera, when the raw video stops, it shows video is continuing to record.  I might have shut off the camera this time.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: tin2tin on May 20, 2013, 10:23:11 AM
Would it be possible to add specification of the full path for the import file in "" and a full destination directory in "" to raw2dng?
I might add that if this was added then EyeFrame could be used as a Windows(& Linux under WINE) GUI for batch processing RAW files to dng by adding a two lines preset file.

BTW. ffmbc only appears to be able to open the first frame of a RAW video file. Hopefully this could be fixed at some point.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: olik on May 20, 2013, 12:18:23 PM
Has anyone tired with the new build from today to open the DNGs in Davinci resolve?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: fatpig on May 20, 2013, 07:08:04 PM
Batch RAW to DNG for Windows is here!

I wrote this little tool for you.
It's called RAW2DNG BATCHelor 1.0
I would be happy if you like it.
It automates the conversion and generates a nice thumbnail. :)
dowload here:

http://fatpigtures.com/software/
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ZUPPORT on May 21, 2013, 08:12:23 AM
still can`t open .dng files !
mac osx 10.7 + cameraraw 8.1 (newest update)
will not open the DNG´S !

any ideas ??!

• in Photoshop CS6
https://dl.dropboxusercontent.com/u/33878668/camerarawupdate.jpg

• the Adobe DNG converter
https://dl.dropboxusercontent.com/u/33878668/dng.jpg

• & After effects !
https://dl.dropboxusercontent.com/u/33878668/ae.jpg
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 21, 2013, 09:12:25 AM
raw2dng update for 4GB files, from rodobot: http://www.magiclantern.fm/forum/index.php?topic=5247.msg38080#msg38080

Can you check if it's working on Windows? (see first post for download)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: platu on May 21, 2013, 09:58:55 AM
raw2dng update for 4GB files, from rodobot: http://www.magiclantern.fm/forum/index.php?topic=5247.msg38080#msg38080

Can you check if it's working on Windows? (see first post for download)

Working on Windows...  just tried it on a 6gb file.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 21, 2013, 10:00:44 AM
still can`t open .dng files !
mac osx 10.7 + cameraraw 8.1 (newest update)
will not open the DNG´S !

any ideas ??!

• in Photoshop CS6
https://dl.dropboxusercontent.com/u/33878668/camerarawupdate.jpg

• the Adobe DNG converter
https://dl.dropboxusercontent.com/u/33878668/dng.jpg

• & After effects !
https://dl.dropboxusercontent.com/u/33878668/ae.jpg

Is not camera raw, is something that got changed in last code, having that problem after updating sources.

Before dng files were without preview and Photoshop opened them right, now they seem to have a wrong preview and can't be opened in Photoshop

Error is:
Quote
Could not complete your request because an unexpected end-of-file was encountered.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: LB on May 21, 2013, 10:25:57 AM
Hey folks,

I do not have after fx and I failed to open the DNGs with Davinci resolve software, FCX and premiere pro, any other software that could work?

thanks for help

o.

Photoshop opens them too, you can process them in batch.

GoPro CineForm Studio also handles them (quite speedily) but it doesn't seem to understand the embedded profile and the colors are beyond horrendous so it's not really usable at this point.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on May 21, 2013, 10:32:40 AM
Photoshop opens them too, you can process them in batch.
Can you confirm that also dng made with last nightly can be opened?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: marten on May 21, 2013, 03:40:37 PM
Just finished making a simple batch conversion app for windows to make mp4 proxy files.

http://www.magiclantern.fm/forum/index.php?topic=5557.0 (http://www.magiclantern.fm/forum/index.php?topic=5557.0)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: hjfilmspeed on May 21, 2013, 05:22:45 PM
YES!!! sweet! Nice work marten! This is very useful! I wish i could help you dev but i know nothing about programming but i will test once i get my Komputerbay card which is looking delayed.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: sicetime on May 22, 2013, 09:45:23 PM
I tried opening the dng's in Assimilate's scratch and had no luck as well, just wanted to give a heads up about that, I guess they only support cinemadng's womp womp.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: peoplemerge on May 24, 2013, 12:16:29 AM
I figured out how to parse my remaining 3Gb file showing "This ain't a lv_rec RAW file" using a1ex's hack suggestion early this thread.  With the code changes allowing >2Gb but this file still failing, I realized that the footer must not have been written.  There are a number of reasons that could have caused this, like a battery dying, and I had footage I wanted to keep.  Here's what I did to recover the file, in case anyone finds themselves in this boat.

When I recorded the .RAW, I used the same settings as the previous file in the series.
The previous file (which was only 34M) outputted the following when I processed it with raw2dng
$ /path/to/raw2dng M0000008.RAW
Resolution  : 1720 x 720
Frames      : 16
Frame size  : 2167200 bytes
Frame skip  : 1
FPS         : 23.976
Processing frame 16 of 16...
Done.

So first, you need to calculate the # frames of the file you want to process.
$ ls -al M0000009.RAW
-rwxrwxrwx  1 david.thomas  staff  3101263200 May 22 19:02 M0000009.RAW

Mine is 3101263200 bytes.  Then you need a calculator. Divide that file size by the frame size.  In my case 3101263200/2167200 = 1430 frames.

Next you need to read the last frame from the working file onto the tail of the broken one.
$ tail -c 2167200 M0000008.RAW >> M0000009.RAW

Next, hack the raw2dng.c file and build:
$ diff -Naur raw2dng.c.orig raw2dng.c
--- raw2dng.c.orig   2013-05-22 18:50:44.000000000 -0700
+++ raw2dng.c   2013-05-22 19:05:44.000000000 -0700
@@ -38,12 +38,8 @@
     raw_info = lv_rec_footer.raw_info;
     fseek(fi, 0, SEEK_SET);
 
-    if (strncmp((char*)lv_rec_footer.magic, "RAWM", 4))
-        FAIL("This ain't a lv_rec RAW file\n");
-   
-    if (raw_info.api_version != 1)
-        FAIL("API version mismatch: %d\n", raw_info.api_version);
-   
+lv_rec_footer.frameCount=1430;
+
     printf("Resolution  : %d x %d\n", lv_rec_footer.xRes, lv_rec_footer.yRes);
     printf("Frames      : %d\n", lv_rec_footer.frameCount);
     printf("Frame size  : %d bytes\n", lv_rec_footer.frameSize);

As you can see, I've taken out the checks and hardcoded the frame count.

Yep, it's a hack, but I got 59 seconds of video from my little brother's med school graduation out of it!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 24, 2013, 02:30:12 PM
I figured out how to parse my remaining 3Gb file showing "This ain't a lv_rec RAW file" using a1ex's hack suggestion early this thread.  With the code changes allowing >2Gb but this file still failing, I realized that the footer must not have been written.  There are a number of reasons that could have caused this, like a battery dying, and I had footage I wanted to keep.  Here's what I did to recover the file, in case anyone finds themselves in this boat.

When I recorded the .RAW, I used the same settings as the previous file in the series.
The previous file (which was only 34M) outputted the following when I processed it with raw2dng
$ /path/to/raw2dng M0000008.RAW
Resolution  : 1720 x 720
Frames      : 16
Frame size  : 2167200 bytes
Frame skip  : 1
FPS         : 23.976
Processing frame 16 of 16...
Done.

So first, you need to calculate the # frames of the file you want to process.
$ ls -al M0000009.RAW
-rwxrwxrwx  1 david.thomas  staff  3101263200 May 22 19:02 M0000009.RAW

Mine is 3101263200 bytes.  Then you need a calculator. Divide that file size by the frame size.  In my case 3101263200/2167200 = 1430 frames.

Next you need to read the last frame from the working file onto the tail of the broken one.
$ tail -c 2167200 M0000008.RAW >> M0000009.RAW

Next, hack the raw2dng.c file and build:
$ diff -Naur raw2dng.c.orig raw2dng.c
--- raw2dng.c.orig   2013-05-22 18:50:44.000000000 -0700
+++ raw2dng.c   2013-05-22 19:05:44.000000000 -0700
@@ -38,12 +38,8 @@
     raw_info = lv_rec_footer.raw_info;
     fseek(fi, 0, SEEK_SET);
 
-    if (strncmp((char*)lv_rec_footer.magic, "RAWM", 4))
-        FAIL("This ain't a lv_rec RAW file\n");
-   
-    if (raw_info.api_version != 1)
-        FAIL("API version mismatch: %d\n", raw_info.api_version);
-   
+lv_rec_footer.frameCount=1430;
+
     printf("Resolution  : %d x %d\n", lv_rec_footer.xRes, lv_rec_footer.yRes);
     printf("Frames      : %d\n", lv_rec_footer.frameCount);
     printf("Frame size  : %d bytes\n", lv_rec_footer.frameSize);

As you can see, I've taken out the checks and hardcoded the frame count.

Yep, it's a hack, but I got 59 seconds of video from my little brother's med school graduation out of it!

This is a sweet solution but I have no idea how to hack my software, is it hard?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: peoplemerge on May 24, 2013, 06:44:12 PM
This is a sweet solution but I have no idea how to hack my software, is it hard?

You need to have a software development environment set up, look for the wiki pages on that.  Then you need to change the source code (written in C) and build the tool.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 24, 2013, 08:32:16 PM
You need to have a software development environment set up, look for the wiki pages on that.  Then you need to change the source code (written in C) and build the tool.
I see, thanks :). When and if the occasion arrives I know what I have to do for a week ;)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: peoplemerge on May 24, 2013, 11:46:11 PM
Yeah, I think when enough people have last-frame corruption a permanent solution will materialize.  I would need to study the format (https://bitbucket.org/hudson/magic-lantern/src/tip/src/raw.h) and brush up on my C, but there's probably a way to extract the image metadata so you don't need another file as reference.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: cube on May 25, 2013, 03:11:51 PM
Hi,

Does anyone know if there is any proper workflow for Magic Lantern RAW recording and post production in ACES Color Space? Does it require any specific IDT?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Andy600 on May 25, 2013, 03:16:04 PM
Hi,

Does anyone know if there is any proper workflow for Magic Lantern RAW recording and post production in ACES Color Space? Does it require any specific IDT?

1) No not yet,  and 2) I'm wondering the same thing.

Good choice of name BTW lol. I was thinking of using Lutty or something  ;D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: cube on May 26, 2013, 12:36:36 AM
1) No not yet,  and 2) I'm wondering the same thing.

Good choice of name BTW lol. I was thinking of using Lutty or something  ;D

DaVinci Resolve 9.1 has CinemaDNG IDT.
The question is: Is RAW2DNG transcoding correct taking into consideration the ACES concept?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: mageye on May 26, 2013, 03:37:14 PM
Just a general tip here for people that may be trying to get a more 'natural' skin tone when using ACR. I am sure there are plenty more definitive guides but this one helped me. I do intend on going through more tutorials on the subject of skin tones and colour cast.

http://www.iheartfaces.com/2012/09/how-to-correct-green-color-cast-in-photoshop/ (http://www.iheartfaces.com/2012/09/how-to-correct-green-color-cast-in-photoshop/)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: mageye on May 26, 2013, 03:53:57 PM
Question:

Does anybody know if, once you have used ACR to import your DNG (into After Effects), you can change ACR settings for the imported video?

It seems to me that once imported these are the settings that you have to stick with. It's really not ideal.

What are other peoples thoughts on this? or perhaps you have a better workflow that allows more control?. :-\
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on May 26, 2013, 04:45:14 PM
There's an "edit original" command in the edit menu.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: CarVac on May 26, 2013, 06:19:58 PM
I may be the only Linux user asking for help but...

I couldn't get the bash script to run; it complained that there was no "raw2dng".

So I realized I needed that program, but when I tried to compile it it needed two headers.

So I grabbed both headers and the makefile and modified the .c file to not look two directories up for the other header, and it still didn't compile.

Then I decided to clone the whole ML Unified branch and build that, but then I realized I needed the gcc-arm-none thing and got that.

Then, it started to compile, but it stopped:

boot-hack.o: In function `my_init_task':
boot-hack.c:(.text+0x544): undefined reference to `_bss_end'
boot-hack.c:(.text+0x548): undefined reference to `_text_start'
boot-hack.o: In function `my_task_dispatch_hook':
boot-hack.c:(.text+0x5f8): undefined reference to `_task_overrides_start'
boot-hack.c:(.text+0x5fc): undefined reference to `_task_overrides_end'
boot-hack.o: In function `copy_and_restart':
boot-hack.c:(.text+0x7e4): undefined reference to `_bss_start'
boot-hack.c:(.text+0x7e8): undefined reference to `_bss_end'
boot-hack.o: In function `my_big_init_task':
boot-hack.c:(.text+0x8d0): undefined reference to `_init_funcs_start'
boot-hack.c:(.text+0x8d4): undefined reference to `_init_funcs_end'
boot-hack.c:(.text+0x8e8): undefined reference to `_tasks_start'
boot-hack.c:(.text+0x8ec): undefined reference to `_tasks_end'
config.o: In function `config_parse':
config.c:(.text+0x2b8): undefined reference to `_config_vars_start'
config.c:(.text+0x2cc): undefined reference to `_config_vars_end'
config.o: In function `config_save_file':
config.c:(.text+0x440): undefined reference to `_config_vars_start'
config.c:(.text+0x444): undefined reference to `_config_vars_end'
config.o: In function `get_config_vars_start':
config.c:(.text+0x538): undefined reference to `_config_vars_start'
config.o: In function `get_config_vars_end':
config.c:(.text+0x544): undefined reference to `_config_vars_end'
config.o: In function `config_var_was_changed':
config.c:(.text+0x590): undefined reference to `_config_vars_start'
config.c:(.text+0x594): undefined reference to `_config_vars_end'
config.o: In function `config_var_restore_default':
config.c:(.text+0x5dc): undefined reference to `_config_vars_start'
config.c:(.text+0x5e0): undefined reference to `_config_vars_end'
property.o: In function `prop_add_internal_handlers':
property.c:(.text+0x1d0): undefined reference to `_prop_handlers_start'
property.c:(.text+0x1d4): undefined reference to `_prop_handlers_end'
collect2: error: ld returned 1 exit status
make[1]: *** [magiclantern] Error 1


Did I try the right things?

How can I get raw2avi to work on linux?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: mageye on May 26, 2013, 06:25:51 PM
Thank you squig very much. That really helps. Lots ;D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on May 26, 2013, 10:52:16 PM
@CarVac

Right question wrong area.  Please post you question here:

RAW Video Module raw_rec [Compiling/Installation]
http://www.magiclantern.fm/forum/index.php?topic=5405.0
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Shield on May 27, 2013, 08:28:19 AM
I figured out how to parse my remaining 3Gb file showing "This ain't a lv_rec RAW file" using a1ex's hack suggestion early this thread.  With the code changes allowing >2Gb but this file still failing, I realized that the footer must not have been written.  There are a number of reasons that could have caused this, like a battery dying, and I had footage I wanted to keep.  Here's what I did to recover the file, in case anyone finds themselves in this boat.

When I recorded the .RAW, I used the same settings as the previous file in the series.
The previous file (which was only 34M) outputted the following when I processed it with raw2dng
$ /path/to/raw2dng M0000008.RAW
Resolution  : 1720 x 720
Frames      : 16
Frame size  : 2167200 bytes
Frame skip  : 1
FPS         : 23.976
Processing frame 16 of 16...
Done.

So first, you need to calculate the # frames of the file you want to process.
$ ls -al M0000009.RAW
-rwxrwxrwx  1 david.thomas  staff  3101263200 May 22 19:02 M0000009.RAW

Mine is 3101263200 bytes.  Then you need a calculator. Divide that file size by the frame size.  In my case 3101263200/2167200 = 1430 frames.

Next you need to read the last frame from the working file onto the tail of the broken one.
$ tail -c 2167200 M0000008.RAW >> M0000009.RAW

Next, hack the raw2dng.c file and build:
$ diff -Naur raw2dng.c.orig raw2dng.c
--- raw2dng.c.orig   2013-05-22 18:50:44.000000000 -0700
+++ raw2dng.c   2013-05-22 19:05:44.000000000 -0700
@@ -38,12 +38,8 @@
     raw_info = lv_rec_footer.raw_info;
     fseek(fi, 0, SEEK_SET);
 
-    if (strncmp((char*)lv_rec_footer.magic, "RAWM", 4))
-        FAIL("This ain't a lv_rec RAW file\n");
-   
-    if (raw_info.api_version != 1)
-        FAIL("API version mismatch: %d\n", raw_info.api_version);
-   
+lv_rec_footer.frameCount=1430;
+
     printf("Resolution  : %d x %d\n", lv_rec_footer.xRes, lv_rec_footer.yRes);
     printf("Frames      : %d\n", lv_rec_footer.frameCount);
     printf("Frame size  : %d bytes\n", lv_rec_footer.frameSize);

As you can see, I've taken out the checks and hardcoded the frame count.

Yep, it's a hack, but I got 59 seconds of video from my little brother's med school graduation out of it!


I'm clearly not as smart as you, but I fumbled my way through creating this, which should allow you to get back all of your "corrput" mov files:

http://www.magiclantern.fm/forum/index.php?topic=5732.msg41171#msg41171

Enjoy,
Shawn
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on May 27, 2013, 08:26:12 PM
Importan Question:
If I use JPG instead TIFF to save image from Adobe Raw, I lost to much quality in the video created from that files? It´s a huge difference in size for me, and I´m not the person who knows about quality, compression, etc. Thanks!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 27, 2013, 10:27:29 PM
Importan Question:
If I use JPG instead TIFF to save image from Adobe Raw, I lost to much quality in the video created from that files? It´s a huge difference in size for me, and I´m not the person who knows about quality, compression, etc. Thanks!
I,d say if you are satisfied with your dng-files and you don,t need to much work after exporting from dng you should be fine. It,s still a heck lot better than the h264 codec from the camera moviemode
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: peoplemerge on May 28, 2013, 07:50:17 AM

I'm clearly not as smart as you, but I fumbled my way through creating this, which should allow you to get back all of your "corrput" mov files:

http://www.magiclantern.fm/forum/index.php?topic=5732.msg41171#msg41171

Enjoy,
Shawn

Much better!  Thanks for this.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Mirazimov on May 28, 2013, 01:49:30 PM
Hi there! i have a one problem, when i'm open dmg file in PS CS 6 a see broken image. Any ideas?
(http://i46.fastpic.ru/thumb/2013/0528/fb/c701808486812c6e8193fa4157251cfb.jpeg) (http://fastpic.ru/view/46/2013/0528/c701808486812c6e8193fa4157251cfb.jpg.html)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: 3pointedit on May 29, 2013, 03:21:24 AM
raw2dng.exe for windows, from 1st post seems to work, however it doesn't generate any DNG files in the folder. It processes the frame count (only 112 frames).
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: mnteddy on May 29, 2013, 03:40:01 AM
If anyone is interested, I just had a bash script put together for raw2dngOSX to automate the DNG extraction process. Read more and download here. (http://www.magiclantern.fm/forum/index.php?topic=5835.msg42194#msg42194)

Personally, I've been using this, then Adobe Bridge's ACR and "Export to Hard Drive" function to create a JPG image sequence which I bring into compressor to create ProRes files. Not the most elegant workflow, but it's the best looking finished product I've been able to muster. As much as I love the speed of the Cineform workflow, I just can't bring myself to use it's terribly soft debayering. It's just so bad.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: agora1 on May 29, 2013, 04:54:59 AM
Hi, why for example in Photoshop they indicates only 8 bits
I know is 8,16,32 bits per channel so which software can tell us a
project is a 14 bits raw
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: pacman829 on May 29, 2013, 06:22:28 AM
Hi, why for example in Photoshop they indicates only 8 bits
I know is 8,16,32 bits per channel so which software can tell us a
project is a 14 bits raw



you would have to process the files at 16 bit (or 32bit) in order to make use of all the raw data ..

There is no 12 or 14 bit setting, and if you set it to 8bit you loose a good amount of detail
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: 3pointedit on May 29, 2013, 11:11:14 AM
raw2dng fails to write frames for me. Anyone else have a broken windows .exe or is it just me? :(
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: peoplemerge on May 30, 2013, 04:22:37 AM
Hi, why for example in Photoshop they indicates only 8 bits
I know is 8,16,32 bits per channel so which software can tell us a
project is a 14 bits raw

At the bottom of my photoshop screen, it shows "8-bit" right?  It's a blue clickable link.  Change it to 16.  Then you output 16 bit tiffs 2x larger.  I did a visual inspection and I didn't see the difference between these which is a surprise.  I didn't take a file with subtle gradations like a sunset sky however and I know final grading that I would do thereafter would be more noticeable.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on May 30, 2013, 10:06:49 AM
Update: latest raw2dng should fix vertical stripes / banding issues in 5D Mark III.

Download from first post.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: aaphotog on May 30, 2013, 10:17:47 AM
Having an issue, anyone else had this happen?
I go from .RAW file to raw2dng(even 0.9)
DNG's into photoshop ACR
Export out of ACR to 16 bit .TIF files
Open image sequence in 'quicktime 7 pro'
now heres where the problem occurs. When I export out of QT7 as ProRes 422 or lower quality, it's perfectly fine.
But when I try and export as 4444, the video seems to be blank. All of the other exports have a thumbnail, my 4444 exports have a black thumbnail and when opening in VLC, it won't open.
I've converted to 4444 many nightly builds and raw2dng's ago, a couple of times. Now, Im not sure whats going on.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 30, 2013, 03:18:12 PM
probably codec issue. Vlc doesn,t play prores but premiere import the files just fine. Also viewable in quicktime. ProRes is mainly for editing I believe.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: agora1 on May 30, 2013, 04:59:16 PM
where I can found the latest raw2dng for mac
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on May 30, 2013, 06:00:39 PM
where I can found the latest raw2dng for mac

You need to familiarize yourself with the Forum.

Please go to the Post-processing Workflow main board, then look at the already existing topics.  You will find:

raw2dng.app 0.10 Osx GUI development, bugs and updates

and also

MLTools (OsX utility to manage ML install, files import and scripts conversions)

Also make sure you read and follow the forum rules here: http://www.magiclantern.fm/forum/index.php?topic=934.msg1141#msg1141
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on May 30, 2013, 06:14:23 PM
Hi! Has anybody come up with a workflow synching recorded audio together with the separated .raw moviefiles? Trying in premiere but I still can,t get the audio to synch. I,m at this build Magic Lantern May 29th(5419e04).
I tried setting fps to 24 manually and also through the canon menu but no luck. It drifts.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on May 30, 2013, 07:48:22 PM
Hi! Has anybody come up with a workflow synching recorded audio together with the separated .raw moviefiles? Trying in premiere but I still can,t get the audio to synch. I,m at this build Magic Lantern May 29th(5419e04).
I tried setting fps to 24 manually and also through the canon menu but no luck. It drifts.

Please got this new thread I have created to further discuss this issue:

How to Sync RAW and Audio Issues
http://www.magiclantern.fm/forum/index.php?topic=5904.new#new
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Maximiliaan on May 30, 2013, 08:06:04 PM
I keep on getting this "Error: This ain't a lv_rec RAW file" error.
The footage was shot in a normal way. Any way to recover what's inside the .raw files? Mac.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on May 30, 2013, 08:26:09 PM
I keep on getting this "Error: This ain't a lv_rec RAW file" error.
The footage was shot in a normal way. Any way to recover what's inside the .raw files? Mac.

Please do a search before you post.   Read the Forum Rules here:
http://www.magiclantern.fm/forum/index.php?topic=934.msg1141#msg1141

Now press the Search button and type:   lv_rec RAW
and you will find an article: 


How to replace file footer, aka fix the Error: This ain't a lv_rec RAW file
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on May 30, 2013, 08:59:27 PM
A tip for OSX: I wrote to Apple about Apple Compressor because doesn´t recognize image sequences and original sizes. The solution when import image sequence was very easy: delete de first frame, because Compressor doesn´t recognize 00000.

For MAC USERS, you can import all the footage, from different shots, and export in the format that you want: pro res 4444, 442, uncompressed, resizes, etc. And Compressor use optical flow to get the file, that means better quality.

Las comment for Quicktime users: I don´t know why Quick Time 7 change de color that you work in ACR. That´s why you must color correct again the footage. In this case Compressor doesn´t have this problem.

Thanks!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: carlostrator on May 31, 2013, 03:12:16 AM
A tip for OSX: I wrote to Apple about Apple Compressor because doesn´t recognize image sequences and original sizes. The solution when import image sequence was very easy: delete de first frame, because Compressor doesn´t recognize 00000.

Having the same issues... Searched a lot but with no success. JPEG's are ok, but TIFF won't work (QuickTime Error -50, on compressor).

Tell us what they replied, it should be able to work.

I'm using Compressor 4.0.7
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: blackroom on May 31, 2013, 09:16:20 AM
hi, sorry if i'm missing this somewhere but i can't find a dng extractor that works with 5D2 spanned files on windows. tried the 5D2 raw2dng , the first post here and newest rawazier 042. does the mac version work for spanning? thanks everyone!

update:
aha, u have to drag the last raw file in the spanned set to extract, i used the 5D2_WIN_raw2dng

ollright!

update2: oh no its all pink noise. ok i'll stop talking to myself and go to bed.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: kasar on May 31, 2013, 02:41:19 PM
Until now I had no success in joining the splitted raw files (r00). I tried it with the terminal an mac but I don't know very much about this and had no success. Could anybody please tell me, how to do this on a mac, please step by step? Thanks a lot!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: carlostrator on May 31, 2013, 08:11:57 PM
Until now I had no success in joining the splitted raw files (r00). I tried it with the terminal an mac but I don't know very much about this and had no success. Could anybody please tell me, how to do this on a mac, please step by step? Thanks a lot!

Don't be lazy, read the entire topic, it's just 8 pages long.

Split the raw files by raw2dng, a program they made to split those files in GUI, so casual users could split them without touching terminal. just search, there's a topic just for the program and i guess it's linked here.

Then use a raw manager to process the files and create tiff's or jpeg's and then put them together with your codec of preference.

The step by step guides are already all here, you just have to do your homework and search it.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on May 31, 2013, 08:41:07 PM
Having the same issues... Searched a lot but with no success. JPEG's are ok, but TIFF won't work (QuickTime Error -50, on compressor).

Tell us what they replied, it should be able to work.

I'm using Compressor 4.0.7

a) try with this http://mac.softpedia.com/get/System-Utilities/Compressor-Repair.shtml   If not...
b) reinstall compressor... if not
c) you have Quicktime problems, different version and you have to choose 7 or X.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: shawnwytch on May 31, 2013, 09:10:22 PM
for some reason when i click the windows exe it just blinks for me it doesnt stay open what could be causing this? Ive tried runing as administrator
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: carlostrator on May 31, 2013, 10:17:34 PM
a) try with this http://mac.softpedia.com/get/System-Utilities/Compressor-Repair.shtml   If not...
b) reinstall compressor... if not
c) you have Quicktime problems, different version and you have to choose 7 or X.

Tried it all, i'm only using quicktime X now. I'll try quicktime 7 without correction, see if i get a clean copy, but would be nice to make this by compressor 4, get my workflow easier.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: agora1 on June 01, 2013, 12:20:26 AM
Hi Renato,
I follow you instructions but the answer was incorrect so somebody can help me.
ML is great but unable to work raw video post processing for file 4 Gig or more
I can donate a lot of money but i am sure somebody have the answer to open 4GIG RAW file
Thanks
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on June 01, 2013, 12:28:52 AM
I don't know the details of mac, but you should first read the standard thread to generate DNG in mac environment.

here:
raw2dng.app 0.10 Osx GUI development, bugs and updates
www.magiclantern.fm/forum/index.php?topic=5508.0
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: driftwood on June 01, 2013, 01:16:27 AM
Quick and dirty, Import each RAW set of DNGs into Adobe After Effects (and change exposure, colour, highlights, shadows, etc...), then 'interpret' the footage as whatever frame rate it was recorded as and save setting using 'remember interpret' and apply it to all your clips, make a composition file and change the file duration roughly calculated to the total length of all your clips (you can always extend/reduce by editing the composition later), then select all clips and drop them into the composition - space them out - and render the composition out to lossless. Bring the rendered output file into an NLE for further editing.
* Dont forget to set Project workspace to 16bit in AE.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: agora1 on June 01, 2013, 01:42:26 AM
To Renato Photo AND ALEX
I search for about 8 hours and I am very disappointing I respect all the procedures before I post.
I do the something for windows not only mac .I know ML work hard but before announce 14 bit raw video  make sure we can work
with a minimum of functionality.
I noT VABLE TO WORK WITH RAW FILE 4GIG SO I CAN SHOOT ONLY FOR FEW MINUTES.
SO I NEED HELP .WHO CAN HELP ME
THANKS
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: agora1 on June 01, 2013, 01:46:09 AM
TO DRIFTWOOD,
I KNOW YOU HAVE MAKE FANTASTIC PATCHES FOR THE GH2 CAN YOU HELP ME?
MANY THANKS
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on June 01, 2013, 01:46:59 AM
FYI, this is what we annonuced:

Quote
Again. This is EXPERIMENTAL, so:
 - bloody beginners and non-geeks should not touch the whole thing. wait until it is "beginner-proof". we will tell you on the website.
 - you know that you are a bloody beginner, when you read the whole thread and you still cannot get it to work.
 - DON'T be disappointed if it doesn't work or we figure out the whole thing is unstable and/or unusable
 - NO, there is no manual yet
 - NO, there is no all-in-one tool that fits every use case
 - NO, we don't have tutorials how to use it
 - NO, not all models are supported yet ;)
 - we are just at the moment testing how good it works and what we have missed and what to improve
 - you are welcome to post comparisons, experiences (both good and bad), or even deep analysis or just cool videos
 - if you are a programmer and you see potential for improvements, grab the source and support :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on June 01, 2013, 03:37:06 AM
I noT VABLE TO WORK WITH RAW FILE 4GIG SO I CAN SHOOT ONLY FOR FEW MINUTES.
SO I NEED HELP .WHO CAN HELP ME
THANKS

Don't shout.

Maybe this will help-
http://www.cinema5d.com/news/?p=17898
http://www.cinema5d.com/?p=18065

The easiest way around the 4gig limit is to ex-fat format your card.

Please don't double/cross post.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: lmf on June 01, 2013, 11:58:06 AM
5.5G of RAW video can not be transferred with RAW2dng.exe DNG sequence,Is there any way around this?
This is the picture link
https://docs.google.com/file/d/0B2jTfLSVyAKkeHlsbXBWdGZrVzQ/edit

My machine is 60D, using the UPDATE # 6:
Compiled 31-05-2013
  with LV_AF_RAW (not recomended but removes dead / hot pixels)
https://docs.google.com/file/d/0B-HdscXfsKpgaGJXOEJGaGRXaUU/edit?usp=sharing
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: christaylor on June 01, 2013, 07:06:40 PM
Hello,

this may seem like a silly question- I've looked around and can't seem to see much in this direction, I see there's a linux raw2avi bash script, but I can't see how to get DNG files out.. DNG is what interests me, not avi... Most of the conversation is Mac and Windows based, if anyone could explain how to basically do the raw2dng part of the workflow in linux, that would be really really helpful..

Thanks for all the work everyone is doing!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: togg on June 01, 2013, 08:59:58 PM
I have a little question. With raw video is it worth it to use pro res 4444 or 422 HQ will be enough?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on June 01, 2013, 11:18:44 PM
I have a little question. With raw video is it worth it to use pro res 4444 or 422 HQ will be enough?

I transfer to prores 422HQ. For me it,s enough. I even started working with converting to either jpeg or tiff, importing to premiere pro. Means I can do basic coloring in lightroom which I really like.
//D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: togg on June 02, 2013, 02:26:10 AM
I transfer to prores 422HQ. For me it,s enough. I even started working with converting to either jpeg or tiff, importing to premiere pro. Means I can do basic coloring in lightroom which I really like.
//D

Thanks a lot for the answer :) I'll try all of that in a few days.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on June 02, 2013, 02:38:06 AM
I have a little question. With raw video is it worth it to use pro res 4444 or 422 HQ will be enough?

You have to test and judge for yourself.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: squig on June 02, 2013, 02:41:13 AM
TO; squig
THANKS BUT I KNOWS THESE LINKS
IF I FORMAT MY COMPACT FLASS TO EX FORMAT (I KNOW THE EX-FORMAT SUPPORT MORE THAN 4GIG)
SO THE UNIQUE SOLUTION IS TO WAIT AN UPDATE FROM ML ?
DO YOU AGREE?
THANKS

Reformat, enable the bootflag, and re-install ML.magiclantern.wikia.com/wiki/Unified/Install See "Install ML on other cards"

And stop shouting!  ::)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: togg on June 02, 2013, 03:49:09 PM
You have to test and judge for yourself.

You're right. I will have the chance to start testing in a few days.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on June 02, 2013, 04:17:17 PM
Quote
Hey guys,

When I'm transcoding DNxHD into MPEG-4 H264, it takes 1 1/2 hour and gives me a file of 1,30MB.

My settings are:
MPEG-4 with h.264 compression, quality 100%, multipass
frame size unscaled, 25fps, no frame blending/downscaling/interlaced scaling/deinterlace video

Thanks for your help!

EDIT: well, just put the multipass off and everything works fine and fuckin fast.  ::)  8)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on June 03, 2013, 08:15:47 AM
I transfer to prores 422HQ. For me it,s enough. I even started working with converting to either jpeg or tiff, importing to premiere pro. Means I can do basic coloring in lightroom which I really like.
//D

Using 4444 FROM TIFF you have a huge information if you gonna Color post. If you create from JPG the video have compensate the information and its the same Pro Res, HQ or 4444.

Maximum quality I Get was TIFF 16 bits  and exported as 4444.

(You have the option of uncompressed video, buy I think the life is very short).
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: eyeland on June 03, 2013, 11:24:13 AM
I have a little question. With raw video is it worth it to use pro res 4444 or 422 HQ will be enough?
Apart from the very valid advice above (mostly concerning workflow and the extend to which you wish to retain flexibility all the way to the end) it also depends on your desired final output. I have always been rather frustrated with the compression used by services like youtube and vimeo but this could change in the future, so even if you only intend to use your video on the web, it could still make sense to save a HQ master for later re-uploads. (I remember reading about H265 being in development which could give us "much" better looking web videos?)
Maybe someone could find us a guide that deals with raw workflow in general so we can keep this thread focused on the specific workflow of ML raw?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: pavelpp on June 03, 2013, 07:11:34 PM
Can someone please check if dng frame sizes are correct? I am getting smaller resolution dng's, than what I have configured in the camera (550d). So for 1408x480 video I am getting 1392x464 dng's ...
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on June 03, 2013, 11:03:36 PM
Using 4444 FROM TIFF you have a huge information if you gonna Color post. If you create from JPG the video have compensate the information and its the same Pro Res, HQ or 4444.

Maximum quality I Get was TIFF 16 bits  and exported as 4444.

(You have the option of uncompressed video, buy I think the life is very short).

When exactly are you exporting to TIFF 16bits? From AFX after loading the DNG's?
And how do you make a video then?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Danne on June 03, 2013, 11:39:49 PM
You can create 16-bit tiff from adobe camera raw or from after effects then import them to premiere pro and nest the whole bunch on timeline. It,s a pretty messy workflow since importing longer tiff-sequences will take a while.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Peter on June 04, 2013, 10:30:08 AM
You can create 16-bit tiff from adobe camera raw or from after effects then import them to premiere pro and nest the whole bunch on timeline. It,s a pretty messy workflow since importing longer tiff-sequences will take a while.

Premiere Pro (at least CS6) doesn't support 16bit TIFF - "unsupported bit depth"
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Kim.dh on June 04, 2013, 11:57:35 PM
What would be the most suitable file format for Premiere Pro be? 8-bit tiff?

Would be awesome if someone can provide a tutorial for dng -> mp4; including the optimal way to grade raw video, twixtoring lower fps and/or upscaling resolution.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on June 05, 2013, 05:16:02 AM
When exactly are you exporting to TIFF 16bits? From AFX after loading the DNG's?
And how do you make a video then?

HI RANDOM,
- FROM ACR, Under the windows, you have to chosse the bits: 8 or 16. Make a click over the number.
- When you have all the folder save it (as Tiffs),  open in Compressor and export as you want.

If you work with 8 bits, When you export the footage from Compressor, the real information is in 8 bits transformed to 10 or 12 depends the codec you have choose. So, I recommend work in 16 and the Compressor downscale the bits to 10. I hope this help you  ;)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on June 07, 2013, 12:12:09 AM
HI RANDOM,
- FROM ACR, Under the windows, you have to chosse the bits: 8 or 16. Make a click over the number.
- When you have all the folder save it (as Tiffs),  open in Compressor and export as you want.

If you work with 8 bits, When you export the footage from Compressor, the real information is in 8 bits transformed to 10 or 12 depends the codec you have choose. So, I recommend work in 16 and the Compressor downscale the bits to 10. I hope this help you  ;)

Great, thank you! Okay so I export a 16bit-TIFF sequence from AFX and then? How can I load and edit it in Premiere?
It gives me single TIFF-images after exporting.

Which compressor is recommandable for use? Transcoding to h264 (or anything) before loading into Premiere or afterwards?

If somebody just could give a short advice about his workflow, I'm getting crazy of all this :D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on June 07, 2013, 12:16:55 AM
Great, thank you! Okay so I export a 16bit-TIFF sequence from AFX and then? How can I load and edit it in Premiere?
It gives me single TIFF-images after exporting.

Which compressor is recommandable for use? Transcoding to h264 (or anything) before loading into Premiere or afterwards?

If somebody just could give a short advice about his workflow, I'm getting crazy of all this :D

Random, H264 is not a good option to export your footage for the codec limitation. Forget it excepto upload a final video to internet. Before, for editing and color post, use a codec with more information as pro res, avi uncompressed to edit IN YOUR software like premire or better, final cut...

Regards
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on June 07, 2013, 12:18:37 AM
RAW2DNG file 0,11 crete a filder called jpg empty. This is for what? prepare a folder to leave the files worked in ACR? Thanks.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on June 07, 2013, 05:24:58 AM
RAW2DNG file 0,11 crete a filder called jpg empty. This is for what? prepare a folder to leave the files worked in ACR? Thanks.

If you are talking about raw2dng.app  it's a problem with the command used to generate the jpg, please post those bugs in the raw2dng.app topic or I'll see them only for luck.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: buildbyflying on June 07, 2013, 06:39:47 AM
Pouring through the forum on post=processing 2gb+ files.
Are users getting raw2dng to work on windows?
It seems I'm reading about users doing this with Batchelor on mac (but I've found nothing to confirm about windows)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on June 07, 2013, 01:01:56 PM
Pouring through the forum on post=processing 2gb+ files.
Are users getting raw2dng to work on windows?
It seems I'm reading about users doing this with Batchelor on mac (but I've found nothing to confirm about windows)
I don't think batchelor works on mac
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: tin2tin on June 07, 2013, 02:31:23 PM
Would it be possible to add -o "output dir" to the command line options of raw2dng?

Then it wouldn't be necessary(takes time) to copy the source RAW files to the same folder as raw2dng before processing them.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on June 07, 2013, 02:33:56 PM
Just type it without the -o ;)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on June 07, 2013, 02:39:28 PM
Would it be possible to add -o "output dir" to the command line options of raw2dng?

Then it wouldn't be necessary(takes time) to copy the source RAW files to the same folder as raw2dng before processing them.
to avoid that on osx I start raw2dng from the folder where I want the dng with correct addresses to raw2dng and RAW file

like raw2dng is in ~/User/App/raw2dng
RAW is in ~/User/Video/M000001.RAW
dng need to go in ~/User/Video/M000001/000000.dng...

"go to ~/User/Video/M000001/"
cd ~/User/Video/M000001/
"start raw2dng from there"
~/User/App/raw2dng ../M000001.RAW
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on June 07, 2013, 02:40:02 PM
Just type it without the -o ;)
is that new?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: tin2tin on June 07, 2013, 02:44:01 PM
I'm not sure I understand. Without -o I'm only specifying the import folder/file(which also is the output folder/file).

What I'm asking is the option to specify a different output folder than the source folder.

raw2dng "sourcefolder" "outoutfolder" - isn't that read as two source folders?

EDIT:
Just tested that with the latest raw2dng:
Code: [Select]
raw2dng myraw.raw myfolderMyfolder was already created, but the resulting dng files ended up in the same folder as the myraw.raw file.

EDIT2:
raw2dng "C:\Users\user\Desktop\myraw.raw"
Places the files in the location of raw2dng.  :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: scrax on June 07, 2013, 02:57:23 PM
I'm not sure I understand. Without -o I'm only specifying the import folder/file(which also is the output folder/file).

What I'm asking is the option to specify a different output folder than the source folder.

raw2dng "sourcefolder" "outoutfolder" - isn't that read as two source folders?

Just tested that with the latest raw2dng:
Code: [Select]
raw2dng myraw.raw myfolderMyfolder was already created, but the resulting dng files ended up in the same folder as the myraw.raw file.

when yu do raw2dng "source" you are in the .raw folder, that is causally the same of the destination folder. Destination folder is the one you are when starting raw2dng.exe so just start it from myfolder:
Code: [Select]
dir myfolder
..\raw2dng.exe ..\myraw.raw
not really sure that the code is right for win but sort of
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on June 07, 2013, 03:00:25 PM
Try this: raw2dng myraw.raw myfolder/
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: tin2tin on June 07, 2013, 03:05:46 PM
He he, this works!  8) 

Thanks!!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Steadycamer on June 07, 2013, 06:50:43 PM
I made a script where you can drag & drop multiple raw file for batch processing

it will create as many directory as raw files (directory will be named with raw filename)

http://www.steadycamer.com/batch2dng.rar (http://www.steadycamer.com/batch2dng.rar)


* raw2dng.exe,   batch2dng.bat  and raw files have to be in the same directory!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on June 07, 2013, 08:52:15 PM
Random, H264 is not a good option to export your footage for the codec limitation. Forget it excepto upload a final video to internet. Before, for editing and color post, use a codec with more information as pro res, avi uncompressed to edit IN YOUR software like premire or better, final cut...

Alright thanks, so I can do it like DNG->losless AVI->Premiere->DNxHD->h264 by Stremclip.
And how about workflow with TIFFs or DPX? DNG->TIFF16 bit>losless AVI (?)>Premiere>DNxHD 10bit (?)>h264?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on June 08, 2013, 02:52:02 AM
Alright thanks, so I can do it like DNG->losless AVI->Premiere->DNxHD->h264 by Stremclip.
And how about workflow with TIFFs or DPX? DNG->TIFF16 bit>losless AVI (?)>Premiere>DNxHD 10bit (?)>h264?

Appears to be fine for me.
I prefer Pro Res, because I used to edit in Final Cut X.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: johanhenrikssn on June 08, 2013, 09:34:15 AM
I want to start by giving a huge thanks to everyone involved with Magic Lantern and this forum. It's impressing to see the tool progress.

I shot some footage that looks like it's sped up after post-processing it. It gives the impression that it was recorded in another, lower frame rate than 25, when looking at it after it's been post-processed. I did not have FPS override to ON in magic lantern, could this have affected it?

I used the june 3rd-version with 5D MKIII. Camera settings i Canons menu set to PAL and video set to 1080p 25fps ALL-I.

In post when converting to dng's with raw2dng.app, it says that it was shot with 25 fps. That's problably just camera metadata though? I tried the workflow with both After Effects and using Quicktime 7 via Photoshop. When importing the image sequence inside After Effects it came in as 30 fps. I interpreted the image sequence from 30 to 25 and created a comp from that. That's when I get that sped up look. Anyone else experienced this problem?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on June 08, 2013, 06:24:12 PM
Just uploaded a new raw2dng.exe. If anybody is still having banding issues with it, please upload the first DNG from the video.

Regarding file format, starting from today, frame size might be a bit higher than w * h * 14/9 (it's rounded to 4096 bytes). If your third party converter uses the same file reading loop as raw2dng, it should be fine. If not... you'll have trouble at certain resolutions.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: pavelpp on June 08, 2013, 07:48:30 PM
Guys, just wanted to share my workflow on Mac here, if anyone is interested:

1) use raw2dng.app to export frames
2) import dng's to Adobe Lightroom
3) white balance / tint correction, if needed
4) export to tif (frames become 4 times the size of dng!)
5) Next you need to make sure you have ffmpeg installed. In Terminal cd to the tiff directory and run command:

ffmpeg -f image2 -r 24 -i %6d.tif -vcodec libx264 -preset veryslow -qp 0 -pix_fmt yuv420p -vf unsharp=5:5:0.5 movie.mkv

This is supposed to produce almost lossless output, however I am not sure about that. Maybe it's better to use yuv422p or yuv444p ? Unsharp filter is used here, because by default the output is quite blurry.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: fatpig on June 09, 2013, 01:29:48 PM
hi guys,
I am looking to implement spanning support in BATCHelor.

Can someone tell me if the latest raw2dng supports spanned files without having to join them first?
this is another step in the chain i would like to avoid.

is it possible to use raw2dng on the .RAW , the .R00 and the .R01 separately, or does it only work for .RAW that is joined beforehand?
I would like to avoid joining files on harddisk first..

PS: if someone could send me R00 and RAW i would appreciate it. :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on June 09, 2013, 01:34:31 PM
No, but you are welcome to implement it :D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: John Hable on June 09, 2013, 09:51:49 PM
hi guys,
I am looking to implement spanning support in BATCHelor.

Can someone tell me if the latest raw2dng supports spanned files without having to join them first?
this is another step in the chain i would like to avoid.

is it possible to use raw2dng on the .RAW , the .R00 and the .R01 separately, or does it only work for .RAW that is joined beforehand?
I would like to avoid joining files on harddisk first..

PS: if someone could send me R00 and RAW i would appreciate it. :)

IIRC this isn't possible.  In some cases the frames get split between between the files, so there is no way to process just one at a time.  Also, the footer is only at the end of the last file.  If you want to extract frames without merging you have to:

1.  Take the .RAW file as input.
2.  Look in the directory structure, counting upwards (R00, R01, R02, etc) to find the last file in the chain.
3.  Extract the footer from the last file in the chain.

Then to extract the frame data you have to figure out how many bytes it is from the start, and read that section from the data knowing that it might be split across two files.  So yes, you can extract the files without joining them first, but no, it's not possible to handle each of the sub-files independently.


Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on June 10, 2013, 08:58:41 PM
Just uploaded a new raw2dng.exe. If anybody is still having banding issues with it, please upload the first DNG from the video.

Regarding file format, starting from today, frame size might be a bit higher than w * h * 14/9 (it's rounded to 4096 bytes). If your third party converter uses the same file reading loop as raw2dng, it should be fine. If not... you'll have trouble at certain resolutions.

Is there a reason why my bitdefender detects a Gen:Variant Kazy.171435 virus in raw2dng.exe?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on June 10, 2013, 09:34:30 PM
Uploaded new raw2dng (https://bitbucket.org/hudson/magic-lantern/downloads), with bad pixel fix from marekk (http://www.magiclantern.fm/forum/index.php?topic=5614.msg49285#msg49285). Very cool trick, didn't know about it.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: dlrpgmsvc on June 10, 2013, 11:19:38 PM
Do there is a postprocessing solution that takes a ml raw video file, extract the dng files and fills the dropped frames automatically with twixtor ?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on June 12, 2013, 01:10:11 AM
Where can I get the old raw2dng.exe without detecting this virus?
Doesn't work for me anymore, neither with rawanizer.
Title: Re: RAW Video Post processing [Mac/Win/Linux] can't convert to DNG
Post by: mkrjf on June 12, 2013, 12:56:27 PM
I installed ML on my MkIII and generated just under a second or so of video for several clips (don't have 1000x cards yet just 400x).
Using RawMagic I get deep magenta for a clip of a white wall illuminated with 5600K LED. Photos with ML active for same scene are normal - looks like white wall with range from bright to dark shadow. (and yes I posted that to the RawMagic thread ;)

Also tried raw2dngapp for OSX - segmentation fault with some reference to a library
Tried raw2dng in terminal - also segmentation fault with no error messages

So not super productive although I greatly appreciate all the development work that has been done.
Do other mac users (latest SnowLeopard updates on macbookpro17) not have issues?
I saw someone is working on FFMBC tool - which is great - not sure what status of that work is.
I have cinelerra - but the tool for that is .exe (sorry I am not getting pc just to run your program no matter how great :(

Would be great to have ffmbc command line that would take raw and output prores444 or some other mastering quality standard codec.
I guess the author of ffmbc does not have a canon 5dmk3 or else the raw codec would exist ;)

BTW what is the actual dynamic range captured in the raw file output by ML - and is the debayering done by raw2dng? and is it using a standard algorithm? If technical spec.s are documented please share the link. I have searched but maybe am not using the right tags.

THX!
Mike
Title: Re: RAW Video Post processing [Mac/Win/Linux] can't convert to DNG
Post by: chmee on June 12, 2013, 01:58:05 PM
..BTW what is the actual dynamic range captured in the raw file output by ML - and is the debayering done by raw2dng? and is it using a standard algorithm? If technical spec.s are documented please share the link. I have searched but maybe am not using the right tags.
raw2dng has a simple dabayering-calculator for the preview-picture - the main picture (in a subIFD) isn't changed. its still bayered raw data.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Redrocks on June 12, 2013, 02:53:47 PM
Touch wood, it looks like ML RAW video is here to stay and most of us will be using the ML variety to some degree for the forseeable future. The many workflow guides are a godsend, but I searched the thread and found no real mention of it: is anyone interested in conducting some experiments into RAW workflow?

ML RAW video is still bleeding edge and there could be changes to the actual files we are transfering to our desktops, but for now we have 14 bit rough diamonds to polish up and it seems there are many ways to do this. It would be handy if we had a standard set of files that we could use to a) nail down the process for getting the best out of our footage b) establish benchmarks for the constantly updating software we use.

We agree on one set of shots covering the ranges we encounter (low light, sunny, charts...) we host them and each of us has access to the same files. We then run them through our own personal workflow and deliver in the agreed format with info about what we did.

Obviously open to all thoughts on this.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on June 12, 2013, 03:20:07 PM
Sounds good, and having some reference footage is great for fine-tuning the convertors.

For example, it was quite hard to find a banding correction algorithm that covers pretty much all practical cases, and I could use this to verify any future changes in this algorithm.

I'd say you can open a separate thread for this; it may become quite large.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Redrocks on June 12, 2013, 04:04:24 PM
I think a set of constants will help all of us in different ways. Do I start the thread on the workflow or RAW board?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: tob on June 13, 2013, 12:57:02 AM
I want to start by giving a huge thanks to everyone involved with Magic Lantern and this forum. It's impressing to see the tool progress.

I shot some footage that looks like it's sped up after post-processing it. It gives the impression that it was recorded in another, lower frame rate than 25, when looking at it after it's been post-processed. I did not have FPS override to ON in magic lantern, could this have affected it?

I used the june 3rd-version with 5D MKIII. Camera settings i Canons menu set to PAL and video set to 1080p 25fps ALL-I.

In post when converting to dng's with raw2dng.app, it says that it was shot with 25 fps. That's problably just camera metadata though? I tried the workflow with both After Effects and using Quicktime 7 via Photoshop. When importing the image sequence inside After Effects it came in as 30 fps. I interpreted the image sequence from 30 to 25 and created a comp from that. That's when I get that sped up look. Anyone else experienced this problem?


I have exactly the same setup/camera as you have and the same problem, but new version of ML 10 June. I have exactly the same issue. I used the raw2dng.exe - 10 june. 
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on June 13, 2013, 06:47:10 AM
If I want to get the highest result using a workflow with exporting a TIFF 16bit sequence from AFX, how can I make Premiere CS6 able to handle it? It's just working until 8bit, but isn't there a workaround?

If not, there's no chance for getting more than 8bit if I want to edit my footage in Premiere, right?
And what happens if I export to DNxHD 10bit? Shouldn't be any advantage compared to 8bit then.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: deleted.account on June 13, 2013, 10:16:23 PM
Anyone know if raw2dng can use standard output, or if not can the option be added a1ex?
Title: Re: RAW Video Post processing [Mac/Win/Linux] - what is in the output?
Post by: mkrjf on June 14, 2013, 05:02:01 AM
raw workflow thread would be nice

I also figured out why my video had magenta cast - it was just preview issue - not using correct LUT

But looking at magicraw created dng in Adobe - image looks fine but is listed by Adobe as 8bit

Now for what is captured anyway (should be in another thread?):
Since 2Mpixel frame is taking up 4Mbytes - that would imply 16bit per pixel (14/16 of which is data?) - can someone outline the detailed mapping of sensor bayer elements to data in the DNG file?
There is a claim that Canon 5Dmk3 'photosites' for 1920x1080 recording use 9 bayer elements (as below) to get one 'pixel'
GBG
RGR
GBG
http://www.eoshd.com/comments/topic/2696-big-news-hands-on-with-continuous-raw-recording-on-canon-5d-mark-iii/page-6
But their selection of 9 elements does not seem right in their diagram (some would be almost only red info).

what exactly does magic lantern code do to read the sensor and generate 14bits of what per pixel?

Or is magic lantern just reading alternating bayer elements - so that we actually have 14 bits of information in a 16 bit datum at 1024K of green, 512K of red and 512K of blue - 960x540x2 true green resolution, 960x540 true blue resolution, 960x540 true red resolution.
So spatial resolution should be 1K by 1K?

Where Adobe CS4 (for example) is debayering (what algorithm) and interpolating to 1920x1080 at 24bpp RGB (all derived not actual)
and preview just dumped to screen - not sure how that added up to magenta ;)

So using 9 'bayer elements' to get 1 'pixel' seems like it would be higher spatial and color resolution than simple alternating R/B rows and */G columns.

Any Canon / ML experts that can explain what goes into a ML raw video pixel for 5DMK3? Is it using Canon algorithm to derive pixels or bypassing it? I am guessing bypassing it.
And is it different algorithm for crop mode versus subsample of full sensor? Some people are saying crop is sharper - is that accurate?

Guess it is impossible to read whole sensor and debayer on the fly - that would be a weighted convolution operation and too much cpu on camera?
Higher end Canon for cinema does something like that, right?





Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on June 14, 2013, 09:54:40 AM
Any news or link about work directly in Final Cut X? Thanks!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: frspp on June 16, 2013, 08:13:53 PM
Could the opening post include also commandline howto (taken from http://www.magiclantern.fm/forum/index.php?topic=5533.msg37819#msg37819 ):

Joining files
WIN
    Start dialog: cmd
    1. cd C:\Users\my_destination_folder
    2. copy /B xyz.RAW+xyz.R00+xyz.R01 xyz.RAW

OSX/LINUX
    Start Terminal
    1. cd /Volumes/my_destination_folder
    2. cat xyz.RAW xyz.R00 xyz.R01 > xyz.RAW

    Tips: cmd and Terminal support drag & drop => Don't need to type!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: mkrjf on June 17, 2013, 05:33:03 PM
Okay so no response on internal read algorithm for ML :(
So I will hunt through the source when I get a chance and try to reverse engineer and document
If sensor could be read efficiently from alternating pixels - perfect interpolation would be possible in post processing resulting in high quite 4K video!!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Sthirasukha on June 20, 2013, 11:12:26 PM
I don't want to be boring but can someone tell me where I can find a Raw2dng able to convert a 10 GB raw file (I mean bigger than 4GB)?
I tried with the Mac and PC version but none of them can do it.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: camilitox on June 22, 2013, 06:25:15 PM
here (https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dngOSX.zip) a link to the raw2dng compiled for mac (no wine needed)
Has anyone the compiled version for linux? Thank you. (Sorry for my bad English) :D
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: pavelpp on June 24, 2013, 06:07:33 PM
Am I stupid or OSX version just does not work? I run in terminal and get
Code: [Select]
-bash: raw2dng: command not found

I did chmod a+x for the executable, does not seem to help...
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: meditant on June 24, 2013, 06:51:37 PM
aw2dng stop working !

On my mac 10.8.4, its generate dng and .mov but the .mov files can be read by quicktime X. and the .dng files, can't be opened by resolve, a strange lowres picture with flat blue filter. i can open the dng with photoshop ????

I use resolve 9.1.4 lite, and a 5D Mark III

3 day ago thats work fine (but maybe with resolve 9.1.3 i dont remember) ! (I reinstall 9.1.3 but the same bug, i reinstall 9.1.4)

The same raw file converted by magicraw is ok and can be open by resolve.

I uninstall raw2dng, and reinstall, reboot and do standard administration tash but no change.

The same problem with the GUI os x app raw2dng.

a capture of resolve :

(http://www.franck-rondot.com/images/divers/raw2dngbug.jpg)
Do you have idea ??

Regards
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: camilitox on June 26, 2013, 04:20:57 AM
Hi there, I want to share, the script, which I use to convert RAW to PNG in Linux. with aspect ratio 2.35:1, with the generation of the .toc file for editing with cinelerra =)

Code: [Select]
#!/bin/bash
# Converts RAW videos to PNG sequence for use with cinelerra
# usage: raw2png foo.raw
fbname=`basename "$1" .RAW`
wine raw2dng $1
ufraw-batch --out-type=tif *.dng
rm *.dng
mkdir "$fbname"
avconv -f image2 -r 24.0 -i %06d.tif -c:v png -s 1920x840 -r 24.0 ./"$fbname"/%06d.png
echo PNGLIST >> "$fbname".toc
echo 24 >> "$fbname".toc
echo 1920 >> "$fbname".toc
echo 840 >> "$fbname".toc
ls $PWD/"$fbname"/*.png >> "$fbname".toc
rm *.tif

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Sthirasukha on July 02, 2013, 12:47:17 AM
Just for your information, it's possible to convert the DNG files (from Raw2dng) to Tif with Capture One (6.4 and 7). ACR is not the only one.
It might help some of us even if the workflow is better with Rawmagic + Resolve.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: MarkvC on July 03, 2013, 01:20:39 PM
Not to keep bringing this up but is anyone having success converting RAW's to DNG's on files larger than 4GB. I have tried both RAW2DNG and RawMagic on my mac and had no success. Smaller clips than 4GB's work great.

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: tin2tin on July 04, 2013, 08:16:45 AM
Have you tried this:
http://www.magiclantern.fm/forum/index.php?topic=5404.msg51806#msg51806
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: kshoban on July 07, 2013, 02:28:13 AM
Hello everyone, I think I'm posting in the correct location (at least I hope so). I am incredibly frustrated. I have been searching these forums for the past 2 days and all I see are people saying "I uprezzed the video." Well, that's great, but What the heck? Can someone please explain to me how to uprez raw video for the the 600d in adobe premiere cs5? Right now I am only able to record at 640x369 and 704x416. I'm trying to uprez to at least 720p. An explanation on how to do this in either premeiere cs5 or after effects would be amazingly helpful. thank you so much. If this is posted in the wrong forum can someone please help direct me to the proper place to post this? thansk again.

just FYI:

I have already changed the footage to tiff and made it into a .mov file to edit in premiere.

-Kevin
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: rue on July 08, 2013, 10:54:56 PM
Dear all, think I'm doing sth. stupid regarding the framerates in my first little 50D raw workflow, so hints and help appreciated:

- Recorded with the May 18th build, FPS Override 23,968
- raw2dng > LightRoom 3 (color & exposure correction) > Export to 1920 x 1080 JPEG
- Import as Image Sequence into PPro CS6 always gives me clips interpreted with 25 fps

How can I convice PPro to interpret the sequence in 23,968 to avoid the ugly artifacts when exporting it?

Sorry, I'm sure I'm missing sth. obvious here  ;)

THX
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: rue on July 10, 2013, 08:04:51 AM
Ok - found the reason for my framerate issue: when you import still-image sequences PPro interprets them according to the settings under Preferences > Media > Intermediate Timebase and not according to the project/sequence settings. This setting defaults according to your locale (so in my case 25 fps). Unfortunately you can't change it for the image sequence (interpret footage) once it is imported.

Hence - make sure that the intermediate timebase is set according to the framerate you wish your image sequence to be interpreted BEFORE you import it.

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: yanone on July 13, 2013, 03:13:11 PM
Hi everyone, I’m new here. You people rock!

I have two feature requests for the converter tools.

Problem: Combining several >4GB files into one via "cat * > file.raw" takes forever because it happens in place on the same disk (and I already have a speedy SSD. Don't even want to think about what happens on an HDD).

Related: What I started to do with <4GB files is to extract DNGs directly from the CF card to the hard disk with RawMagic, effectively saving one step of processing, because it's the bus speed in my system (USB2 and cheap card reader) that is the speed bottleneck. So instead of copying the raw file first and then extracting the DNGs in a second step, I load the raw files directly from the CF card into RawMagic.
In other words: Extracting the files directly from CF takes exactly as long as only copying them.
This way I also save half of the required disk space because the raw files never even make it to the hard disk.
By the way, the same bus speed issue from above applies here, because extracting DNGs from hard disk to the same hard disk requires simultaneous reading/writing.

Request 1: Please implement that the raw converters read all individual "filename.*" files (.RAW, .R00, .R01, ...) consecutively. The minimalistic "cat * > file.raw" command makes me believe that this should be possible. And the files’ relation is unambiguous. So one would only insert the first (.RAW) file into the process and the converters would look up existing .R** files in the same directory.

Request 2: Please implement an output path option for the command line tool. As far as I understand it it works in place directory-structure-wise.
This way one could use the direct CF->HDD conversion outlined above with the command line tool. I'm a big fan of command line tools as I expect to implement some sort of automatized copying/sorting with scene numbers and comments and stuff.

I will use ML/RAW for shooting a dance art film in about a month, whether you guys approve of that (stability-wise) or not  :)

Thank you.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: slingerland on July 16, 2013, 10:58:05 PM
My work-flow is at this moment 1) Have photoshop raw convert  the dng filies in to either tiff, and then I go the quiktime 7 route (not so happy about that route because of the way it comes out) have photoshop make a video file of it, mp4 or a quiiktime movie made of "animation". Last one gives big files. I want to get to finalcutpro so i also used compressor to convert the mp4 to prores. There is got to be a better route?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: SoulState on July 17, 2013, 12:26:16 AM
My 2 workflows:
First (Slow)
1. Convert from raw to 16-bit dng with raw2cdng (16 bits give me best results in terms of color reproduction and and "filmic look")
2. Import dng's into After Effects with ACR plugin, some tweaking
3. Apply log lut ( PHOTOGS LOG LUT http://www.megafileupload.com/en/file/431990/index.php ) as starting point for filmic look and further color correction (Not really needed, but i like flat log "filmic" look, you can make this in ACR, but using lut for that is faster).
4. Export video with HuffYUV codec (losless) (i use huffyuv from ffdshow) or any codec like x264 if you want final compressed file.

Second (Fastest)
1. Convert from raw to 16-bit dng with raw2cdng
2. Import dng's into Photoshop with ACR plugin, some tweaking, synchronize and export as 100% jpegs (i use jpeg to save some space, but you can save stills in some losless format)
3. Import jpegs to VirtualDub, set the framerate, and resize (if needed) from 1728x972 (60d footage) to 1920x1080 (it's 111.111% upsizing), for upsize i use Lancoz3 mode, it's little sharper than precise biqubic (a=-1.00). I usually don't resize, if i want just to look at resulting video clip.
4. Export video with HuffYUV codec (losless) (i use huffyuv from ffdshow) or any codec like x264 if you want final compressed file.

Second method about 2-3 times faster if you don't need heavy color correction or effects in AE.
In first time i used Lightroom for second method, but with using just ACR plugin in Photoshop i have much faster workflow.

I have only one question - how to convert Silent Picture DNG to normal DNG to open it in ACR? My silent pics dng won't opens nor AE with ACR plugin, nor Photoshop's ACR nor with Lightroom... ( Camera is 60d). I shoot some timelapses, but just can't open dng raws...
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: fatpig on July 19, 2013, 10:19:49 PM
Is somebody working on automatic processing of .R00, .R01, etc. Files in RAW2DNG?

Would be huge for my Software BATCHelor.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Northshoots on July 23, 2013, 11:19:14 AM
I got back today from a forest trip where I filmed more than 85 raw sequences. So converting them manually become not an option. I wrote a script for mass conversion of the files. Feel free to give suggestions for improvement or hack your self http://shieldroute.blogspot.se/2013/07/quick-and-dirty-multiply-raw-file.html

My workflow was:
move full card to -> date\1\
                           -> date\2\
                           -> date\3\
 etc
so the scripts checks only subfolders for a given folder. Then it enters there creates new folder with a name same as file name and converts it using raw2dng.
Resulting structure
date\1\ML_001.RAW
date\1\ML_001\000000.dng
date\1\ML_001\000001.dng
.................
date\1\ML_001\00xxxxx.dng
date\1\ML_002.RAW
date\1\ML_002\000000.dng
date\1\ML_002\000001.dng
.................
date\1\ML_002\00xxxxx.dng

hope someone will find it useful and will be less frustrated coming home with may sequences. There is no fall back safely implemented, but I will added if some one needs it.

Cheers
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: egc on July 23, 2013, 01:33:54 PM
Linux shell script for conversion to mjpeg: raw2avi.sh (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/raw_rec/raw2avi.sh)

Hallo!

First of all: THANKS a lot for this script!!

As raw2dng does not exist for Linux (at least i could not find it) i think it is better to tell people they can install wine and change the script in line 8 from
Code: [Select]
${DIR}/raw2dng $1to
Code: [Select]
wine ${DIR}/raw2dng $1
That works fine and people don't have to waist days trying to compile raw2dng or changing to Windows just to do the raw2dng step.

Thanks!

Best regards
egc
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: yanone on July 29, 2013, 06:42:01 PM
One question about DNGs that I haven't seen discussed anywhere in the forum so far:

Processing them with ACR in AE produces inconsistent results when the white and black points (=lighting) change between frames. The brightness/darkness recovery sliders (the most important in my opinion) seem to respond to a relative black and white point per image.

Do you see a way to synchronize the black and white points (whatever that even means, I'm no expert) across images? Maybe by measuring the lightest and darkest pixel across all images of a scene and then changing certain settings in the files accordingly? Maybe even by drawing a black and a white pixel into the files as a punky hack?

I'm not asking the software authors here to implement this, only about implementability. I might like to try myself.
Or is there a much easier way I'm not aware of? I'm asking about ACR because the grading in Resolve doesn't work for me. Plus ACR has many abilities that I'm familiar with from photography like abberation reduction and stuff.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: coopybook on August 04, 2013, 03:13:13 AM
I want to start by giving a huge thanks to everyone involved with Magic Lantern and this forum. It's impressing to see the tool progress.

I shot some footage that looks like it's sped up after post-processing it. It gives the impression that it was recorded in another, lower frame rate than 25, when looking at it after it's been post-processed. I did not have FPS override to ON in magic lantern, could this have affected it?

I used the june 3rd-version with 5D MKIII. Camera settings i Canons menu set to PAL and video set to 1080p 25fps ALL-I.

In post when converting to dng's with raw2dng.app, it says that it was shot with 25 fps. That's problably just camera metadata though? I tried the workflow with both After Effects and using Quicktime 7 via Photoshop. When importing the image sequence inside After Effects it came in as 30 fps. I interpreted the image sequence from 30 to 25 and created a comp from that. That's when I get that sped up look. Anyone else experienced this problem?


I have exactly the same setup/camera as you have and the same problem, but new version of ML 10 June. I have exactly the same issue. I used the raw2dng.exe - 10 june.

I have the same problem on Canon 60D. Does anyone found solution?
I have tried all possible varieties, changes frame rates, NTSC,PAL, resolution, - results always the same.
Becouse of this problem i can't get together video and external recorded sound. That is realy bad!
If anyone have any ideas how to fix it, please share!!!
Thank You!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on August 04, 2013, 03:39:44 AM
Exist a way to get, from RAW2DNG or MAGIC Raw,  a way to join raw with .WAV?

Anybody knows a way to do this faster?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: coopybook on August 10, 2013, 01:05:04 PM
I have the same problem on Canon 60D. Does anyone found solution?
I have tried all possible varieties, changes frame rates, NTSC,PAL, resolution, - results always the same.
Becouse of this problem i can't get together video and external recorded sound. That is realy bad!
If anyone have any ideas how to fix it, please share!!!
Thank You!

Regarding the 24 fps problem:

Hi guys I had the same problem with my 60D raw videos but I found the fix:

First: make sure you do have framerate override on and set to 23.98 or 24fps just to eliminate any in-camera issues.

Second: It took me a while to realize that the problem was with a default setting in after effects. Unless you tell it not to, AE will interpret all imported files as 30fps (which causes the speed up to 24fps material). To fix, right click on your footage in the project bin and choose: Interpret footage > main. Then input the correct framerate depending on what you sent your override at. VOILA

Here is a visual example that I found:
the setting:
http://i.stack.imgur.com/JehOz.png

the dialog window:
http://i.stack.imgur.com/z7Yy0.png
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: tool on August 12, 2013, 12:38:12 AM
Moved__
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Fringuello on August 13, 2013, 02:04:09 AM
su windows 8 non funziona? :-[
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: manniac on August 17, 2013, 10:42:54 PM
extracting dngs from the raw (5DIII) is quite slow on my machine (win7 64bit). for some reason the first 100 pictures or so are quite fast and everything after is slow. at the same time my 6 core cpu doesnt seem to be overly occupied..
Does anyone have tipps how to improve performance on raw2dng.exe?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Fringuello on August 22, 2013, 04:35:36 PM
scusate perchè su imac non funziona  :-[
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ilguercio on August 22, 2013, 04:43:55 PM
Again, this is an English board.
Everybody needs to understand you.
Capisci?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Fringuello on August 23, 2013, 12:54:25 AM
ok, I have a mac dragging the raw file on raw2dng nothing happens.
thanks
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: David J. Fulde on August 25, 2013, 09:28:34 PM
Will the GUI work with DualISO?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: pavelpp on August 29, 2013, 10:57:14 AM
I keep getting a segmentation fault, when trying to convert a raw video with raw2dng executable on OS X 10.7.5.

Code: [Select]
Resolution  : 1728 x 736
Frames      : 54
Frame size  : 2228224 bytes
Frame skip  : 1
FPS         : 19.998
Processing frame 1 of 54...Segmentation fault: 11
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on August 29, 2013, 11:01:26 AM
Read this http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

and upload the video file.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: alexp on August 30, 2013, 02:57:55 AM
Hey guys, I have a workflow question. After editing reference files in FCP7, how do you replace them with image sequences? Is that even possible? Until now I've been using QT7 to export image sequences into Pro Res, which takes a lot of time doing one by one. Trying to simplify this process. Please share...

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Midphase on August 31, 2013, 08:39:14 PM
Hey guys, I have a workflow question. After editing reference files in FCP7, how do you replace them with image sequences? Is that even possible? Until now I've been using QT7 to export image sequences into Pro Res, which takes a lot of time doing one by one. Trying to simplify this process. Please share...

On the Mac, I think the workflow which still makes the most sense is to use something like Resolve to batch output to ProRes for editing. I think ProRes is fine for finishing too actually, if you export at ProRes 422HQ using a LOG type of look, you'll retain pretty much most of the visual information that you need. Most people who shoot on the ARRI Alexa shoot in ProRes and not raw and they seem to be just fine with the quality output, so once you process the image sequences into ProRes, I don't know if you need to go back to the DNG format for finishing.

But yeah, use Resolve for batch creating the ProRes files...it's a whole lot faster.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: alexp on August 31, 2013, 11:15:13 PM
On the Mac, I think the workflow which still makes the most sense is to use something like Resolve to batch output to ProRes for editing. I think ProRes is fine for finishing too actually, if you export at ProRes 422HQ using a LOG type of look, you'll retain pretty much most of the visual information that you need. Most people who shoot on the ARRI Alexa shoot in ProRes and not raw and they seem to be just fine with the quality output, so once you process the image sequences into ProRes, I don't know if you need to go back to the DNG format for finishing.

But yeah, use Resolve for batch creating the ProRes files...it's a whole lot faster.

Thanks Midphase. Will the free version of DaVinci Resolve Lite batch output files to Pro Res?

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: dickson on September 02, 2013, 03:36:51 AM
Could the opening post include also commandline howto (taken from http://www.magiclantern.fm/forum/index.php?topic=5533.msg37819#msg37819 ):

Joining files
WIN
    Start dialog: cmd
    1. cd C:\Users\my_destination_folder
    2. copy /B xyz.RAW+xyz.R00+xyz.R01 xyz.RAW

OSX/LINUX
    Start Terminal
    1. cd /Volumes/my_destination_folder
    2. cat xyz.RAW xyz.R00 xyz.R01 > xyz.RAW

    Tips: cmd and Terminal support drag & drop => Don't need to type!

I do this (in Windows) on one raw and one r00 file, hence copy /B xyz.RAW+xyz.R00 xyz.RAW - that's right, isn't it? And I get one RAW file, a lot smaller than the originals, and it won't open in raw2dng. Any ideas?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on September 02, 2013, 02:50:14 PM
I do this (in Windows) on one raw and one r00 file, hence copy /B xyz.RAW+xyz.R00 xyz.RAW - that's right, isn't it? And I get one RAW file, a lot smaller than the originals, and it won't open in raw2dng. Any ideas?

Read:
RAW Video: PostProcessing -- Beginners Guide --
http://www.magiclantern.fm/forum/index.php?topic=5654.0

Your format is wrong
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: dickson on September 02, 2013, 10:30:36 PM
Read:
RAW Video: PostProcessing -- Beginners Guide --
http://www.magiclantern.fm/forum/index.php?topic=5654.0

Your format is wrong

Thanks! I changed the formatting to exfat, but the 5d mkii doesn't seem to recognize it.

I'll change back and try the file-renaming.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on September 02, 2013, 10:50:52 PM
Thanks! I changed the formatting to exfat, but the 5d mkii doesn't seem to recognize it.

I'll change back and try the file-renaming.

No, I was not clear.  Your file naming format is wrong!

Try this:

copy/b file1.raw+file2.raw+.... filenew.raw

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: dickson on September 02, 2013, 11:36:05 PM
No, I was not clear.  Your file naming format is wrong!

Try this:

copy/b file1.raw+file2.raw+.... filenew.raw

I changed the file names as you suggested, but I'm getting the same thing - a new file much smaller than the originals, and that won't open. (The original r00 will open, but the dngs are completely garbled.)

Update: figured it out with FSJ, thanks for your help!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ifleeter on September 10, 2013, 09:02:54 PM
Not sure where to post this. I have converted my raw files with the raw2dng converter and the first frame is great but then all the rest have this Pink banding.

(http://farm4.staticflickr.com/3708/9716520995_be45f17128_z.jpg)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on September 10, 2013, 09:55:38 PM
Not sure where to post this. I have converted my raw files with the raw2dng converter and the first frame is great but then all the rest have this Pink banding.

http://www.catb.org/~esr/faqs/smart-questions.html
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ifleeter on September 10, 2013, 10:17:47 PM
http://www.catb.org/~esr/faqs/smart-questions.html

So, I'm gathering no one has a clue what this is. That's all I wanted to know. Alright, thanks! 
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: feureau on September 11, 2013, 12:49:33 PM
So, I'm gathering no one has a clue what this is. That's all I wanted to know. Alright, thanks!

Oh, we know what that is. But you need to read this first: http://www.catb.org/~esr/faqs/smart-questions.html
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ifleeter on September 13, 2013, 05:39:14 PM
Oh, we know what that is. But you need to read this first: http://www.catb.org/~esr/faqs/smart-questions.html

So, I'm guessing you guys thought I was asking for help and for you to solve everything for me. So I didn't have to read. But If you read what I posted "Not sure where to post this. I have converted my raw files with the raw2dng converter and the first frame is great but then all the rest have this Pink banding." I didn't ask for any help at all. I was just posting to get discussion about what I had came across. I didn't know that I had to be super to the point in what I was saying. Next time I will be straight forward and add "Any Ideas?" or "What do you guys think?" I just thought that was common knowledge (especially to a group of people who are smart and breaking new ground with these DSLR's) that I just wanted to talk about it. I mean that's what this forum is right? Discussion about magic Lantern? And then even more to the point to this thread, discussion about "RAW Video Post processing [Mac/Win/Linux]"

So, I sincerely apologize. Which means I'm sorry for posting in your thread and not being able to further your discussion. I shouldn't of posted because my rhetorical statement was unnecessary and highly illogical.

My second attempt at forming a logical statement that has a question that can be further discussed in a logical informing manner.

Question: 1 (Attempt 2)

Hey guys!

I'm new here! :D I have very little posts. I really enjoy learning new things and I've been looking into this and reading about it here, in these forums! I did a search of the forum and couldn't find what I was looking for.
(http://s12.postimg.org/ee791a07x/Pink_Frames.png)
So, according to the forum rules. I may ask the question in the thread specified. This just so happens to be that thread!  :D

When working with Magic Lantern I have came across something rather odd, to me at least. Using the file footer replacement thread i replaced my file footer. I then have converted my raw files with the raw2dng converter and the first frame is great but then all the rest have this Pink banding.
(http://s22.postimg.org/iuv71z3sh/pink_banding.jpg)
I would very much like to discuss this in an informing way. I don't want you to solve it for me, I just want to talk about it and learn if other people have came across this same issue.

:End Question:
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on September 13, 2013, 06:29:01 PM
If I knew the strightforward and simple answer I would have given already but, this is a bit of a mess because it could be many things.  I am not a developer so I dont have enough knowledge but here is my suggestion:

From the link above you will find the following advise:

Before You Ask

Before asking a technical question by e-mail, or in a newsgroup, or on a website chat board, do the following:

    Try to find an answer by searching the archives of the forum you plan to post to.

    Try to find an answer by searching the Web.

    Try to find an answer by reading the manual.

    Try to find an answer by reading a FAQ.

    Try to find an answer by inspection or experimentation.

    Try to find an answer by asking a skilled friend.

    If you're a programmer, try to find an answer by reading the source code.


Ok, so you have done step one:  Try to find an answer by searching the archives of the forum you plan to post to.

Now do step two:

in Google search copy and paste this:

pink frames site: http://www.magiclantern.fm/forum/index.php

This will get you lots of info, unfortunately our search function is worthless since the forum was updated, I dont know why.  :'(  So you have to use Google.

Step three above... etc.

--------------

Here is what I know about pink frames:

Usually a problem at the beginning of the implementation stage of RAW to a new camera.  Never had a situation as you have posted, where the first frame is good and the rest are trash.

Ok to try to trouble shoot this issue by a more advanced person you will probably need to provide.

Again from the previous link here is some helpful advise:

Be precise and informative about your problem

    Describe the symptoms of your problem or bug carefully and clearly.

    Describe the environment in which it occurs (machine, OS, application, whatever). Provide your vendor's distribution and release level (e.g.: “Fedora Core 7”, “Slackware 9.1”, etc.).

    Describe the research you did to try and understand the problem before you asked the question.

    Describe the diagnostic steps you took to try and pin down the problem yourself before you asked the question.

    Describe any possibly relevant recent changes in your computer or software configuration.

    If at all possible, provide a way to reproduce the problem in a controlled environment.


So far you have done step 1.

What about step 2.. 
In step 2 specific information about the camera model, the build that you tested, the card that you used (may be a faulty car!), the settings in the RAW module, anything elese you can add!

If you provide little information you will get little help.

Continuing with step 3, 4, 5, etc.


Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: feureau on September 14, 2013, 06:31:07 AM
TL;DR - we solved the pink thingy in various front.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on September 14, 2013, 02:03:51 PM
@ifleeter: I found a similar post but no solution yet here:
Salvaging Pink Crazy DNGs
http://www.magiclantern.fm/forum/index.php?topic=8237.msg75930;topicseen#msg75930

@ feureau;  Can you please give us a link to the post?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on September 14, 2013, 02:09:46 PM
Quote
Using the file footer replacement thread i replaced my file footer.

The frame size in the file footer must match exactly the frame size from your raw file (usually w x h x 14/8 rounded to next 4096 multiple).
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ifleeter on September 14, 2013, 02:26:59 PM
@RenatoPhoto Thank you for the suggestions, I will begin to do so. Also, thanks for link! Sounds exactly like my issue.

@a1ex Thanks for the tip, I'll look into that as well.

This is just want I wanted, some discussion on the issue. I understand that people come into forums and expect everything handed to them. So I understand why I was replied to like I was. I was just looking for some good ol informal productive discussion, because by all means I am not a programmer and don't even want to pretend I am. That stuff is over my head.

Thanks again for the help I'm going to look into it all.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: coralio on September 16, 2013, 06:29:05 PM
Hi there,

It seems that the Camera Model (5DM2) is not included in the DNG when convert using Raw2DNG for Mac. So I cannot use my VSCO Camera Profiles. Any hint on how to solve this? RawMagic instead allows me to choose the Camera Model,  but I feel more comfortable using the last build of Raw2DNG.

Thanks a lot,
Pablo
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Malex on September 26, 2013, 11:37:42 AM
Hi guys, I guess many of you heard about the new features for Adobe CC coming out next month, I am pretty excited about few of the new features!
Basically if you are an Adobe user, here is the new workflow we will be using :

- create DNG files like usual (rawmagic, rawanizer or raw2dng or mlv2dng)
- import those DNG into Premiere
- create your proxy from the dng and start editing
- clic the send to Speedgrade button to start grading your footage,
- send back to Premiere
- switch your sequence to DNG files and render out!

This workflow is in my opinion one of the fastest, easiest there will be from next month.
I guess that Davinci resolve has more features than Speedgrade, but with that new Direct link feature things just got perfect! and now with the Mercury Playback Engine stuff it should be has powerfull as Davinci.

here are 2 videos showing quickly the new features:
http://tv.adobe.com/watch/adobe-at-ibc-2013/adobe-premiere-pro-cc-speedgrade-cc-direct-link/
http://tv.adobe.com/watch/adobe-at-ibc-2013/adobe-premiere-pro-cc-ultra-hd4kray-support/

here are is the page with all the new features:
http://blogs.adobe.com/premierepro/2013/09/adobe-premiere-pro-cc-october-2013-release.html

I just need to learn Speedgrade now...

What do you guys think about that workflow ? is it ideal ?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: dude on September 26, 2013, 12:23:30 PM
Thats really nice, because we don t need the way over after effects anymore.
Does the master Collection benefit from that? Will there be a update für Premiere pro cs6 to use tzhe dngs?
Will they then be debayered by premiere or like in ae, with acr?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Malex on September 26, 2013, 12:30:17 PM
The update is for Premiere CC, not CS6, and yes debayer is done is Premiere.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: mityazabuben on October 01, 2013, 07:55:32 AM
raw2dng.exe  is missing...
everyone talkin about raw2cdng program. Is it same as raw2dng?
Thanks.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: prabhath.mp on October 05, 2013, 10:50:50 PM
raw2dng (reference implementation, command-line tool)

Source code: raw2dng.c (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/lv_rec/raw2dng.c)
Windows executable: raw2dng.exe (https://bitbucket.org/hudson/magic-lantern/downloads/raw2dng.exe) (also runs in Wine, for Mac and Linux)

Usage: drag and drop the .raw file over the executable, or use the command line.

Linux shell script for conversion to mjpeg: raw2avi.sh (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/raw_rec/raw2avi.sh)

Hi Alex :)

Hope you are well.

I just tried to run RAW2DNG.exe on Ubuntu with Wine, however, it just executes in background.

My computer just has Ubuntu 13.04 (no dual boot option).

I tried to follow the procedure for linux (.sh format) and that was a little complicated for me to understand (apologies).

Please advise if there is any alternate or if I am missing any step.

Thank you and hope to hear from you soon :)

Cheers,
Prabhath MP
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: dude on October 15, 2013, 02:36:45 PM
Hi, changed from other thread:
I do not find raw2dng.exe for mk III.
The problem is the missing metadata, acr always opens the dngs with embedded profile.
raw2cdng has the same results.
i downloaded raw2dng from a mkII build, with that acr says that the footage is mkII. i thought raw2dng is camera-independent?
Some ideas how to get the metadata in acr? seems like i always downloaded older versions of raw2dng...
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on October 27, 2013, 05:38:45 AM
Here are a lot of software to transform raw to DNG.
All DNGs files works in ACR.

Try Davinci Resolve directly, version 10. Legal and free!

Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: dude on October 29, 2013, 09:25:54 PM
Did you check this kickstarter Project?
Quite nice...
http://www.kickstarter.com/projects/primehd/raw-2-cf-raw-convert-your-frames-into-video-fast
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: ivannoel on October 30, 2013, 02:13:30 AM
Hi guys.

This MUST have been dealt with before, sorry...
I am shooting a feature, and shot a 12min shot today, in two (multiple) files. But my 5D3 froze in the middle of the second take, and I am left with a RAW file (in 6 multiples) that Rawmagic won't reckognise, I suppose this is because the cam didnt shut down properly.
Does anyone know how to salvage all those 4.29gb files that I do have?
Thanks
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on October 30, 2013, 11:59:38 AM
Under post processing you can find:

http://www.magiclantern.fm/forum/index.php?topic=8237.0
http://www.magiclantern.fm/forum/index.php?topic=5732.0
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: chmee on November 03, 2013, 12:40:35 PM
You'll find (Cinema)DNG-Converter here -> http://magiclantern.wikia.com/wiki/RAW_Video

regards chmee
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: mrnv45 on November 08, 2013, 01:43:32 AM
i was having similar issues

but what i did was

when selecting footage..

File type : Any File
Format : Camera Raw
Check: raw image sequence

that got rid the pink, instead of just selecting the .dng files.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: dude on November 12, 2013, 09:42:49 PM
One question about flickering-
i am processing with acr and the 2012 profile.
because the highlights sucked, it decided to pull down the "lights" bar for a test, and was expecting heavy luma flickering. but there is nothing.
is it always flickering when pulling the sliders which shouldn t be pulled, or just only in special cases?
it s a normal evening shot with burned highlights
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: larry 777 on November 13, 2013, 01:21:51 PM
Hi ,
Thank you so much for the magic lantern software, it is really fantastic tool.
I can't go higher than 1728x972 in raw video continuous, with 7D, transcend 64gig x1000 (67.1 Mbs at 23,976p)
aspect ratio is 16:9. ( using 29/09 britom's build )
Is it possible to upscale to 1920x1020 in after effects for full hd resolution?
I tried to find the answer in the forum but I couldn't.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on November 14, 2013, 01:21:21 PM
Only via cropped video see here:
http://www.magiclantern.fm/forum/index.php?board=49.0
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: larry 777 on November 15, 2013, 12:32:44 PM
Thanks, so 1920x1080 won't be possible with 7d because of sensor ?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: chris bouk on November 15, 2013, 07:57:42 PM
hey guys....
good evening from Greece..
I tried to film my first video RAW with my 7D...
the file has a name : M15-2125.MLV and it is about 400MB....only for 15 secs.....I think so...
I did drag and drop the file to raw2dng.exe but I have no results....can someone explain the procedure...for example where the file will go...etc...etc...because I am new to this procedure and I don;t know what to do....
after droping the file to raw2dng.exe what should I do for the next steps...
something clear?

thank's in advance for your time!
Chris
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: chris bouk on November 15, 2013, 08:41:29 PM
....fixed...
captured RAW video and now I'm converting the files from dng to psd to open with AE like a multiple files....will see...
any suggestion for the next step to finish the video is very acceptable! Do we have to do all this long job for a short video in RAW?
Working 10000 frames in Ps and then in AE and then APremiere for a 30 seconds video? Is there a shorter way?
best regards
Chris
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on November 15, 2013, 10:20:43 PM
http://www.magiclantern.fm/forum/index.php?topic=7494.0
http://www.magiclantern.fm/forum/index.php?topic=6021.0

May help, and there are others here:
http://www.magiclantern.fm/forum/index.php?board=54.0
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: chris bouk on November 16, 2013, 01:06:05 AM
...I have more than 4 hours reading posts about post processing a RAW video file, but I am more confused than before...I tried everything,,but I think I will stay in my Adobe solutions fron Ps, Premiere Pro and After Effects....but my problem is that I haven;t use After Effects in the past and I don't know the way to create a sequence from the frames (psd files from the movie) and send them to Premiere...Could you please give me some details, some instructions?

thnk you for ypur time


Best regards

Chris
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on November 16, 2013, 03:08:09 AM
Sorry I dont use Aftereffects.  I only use ACR to generate tiffs and import them to Vegas.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on November 16, 2013, 03:22:14 AM
Hey Chris,
My workflow is:

1) Extracting .DNG files from RAW file with raw2dng -> CLICK (http://magiclantern.wikia.com/wiki/RAW_Video)
2) Import the first .DNG into Adobe After Effects (16bit/32bit project) and mark "Image Sequence", make certain color corrections in ACR
3) Export to 16bit .DPX files (very large) that you can find as "C1neon Sequence"
4) Import into Adobe Premiere Pro CC again marking "Image Sequence", upscaling and editing, color grading in SpeedGrade via directlink or directly in Premiere
5) Export either to h264 or into Avid DNxHD 10bit/175 and transcoding into h264 via MPEG STREAMCLIP -> CLICK (http://www.squared5.com/)

In step 3 you can also export into losless AVI or any other format you want to work with.

Hi guys, I guess many of you heard about the new features for Adobe CC coming out next month, I am pretty excited about few of the new features!
Basically if you are an Adobe user, here is the new workflow we will be using :

- create DNG files like usual (rawmagic, rawanizer or raw2dng or mlv2dng)
- import those DNG into Premiere
- create your proxy from the dng and start editing
- clic the send to Speedgrade button to start grading your footage,
- send back to Premiere
- switch your sequence to DNG files and render out!


Are you successfull importing the DNGs into Premiere? Used raw2dng it says "this file doesn't include media data".
Do you work with raw2cdng (that's extracting real cinema DNGs)?

Does anybody know where to get raw2cdng apart from the link given in the ML wiki? The page is blocked by my anti-virus.
 
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: reddeercity on November 16, 2013, 05:55:14 AM
I Use Version 1.30
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: chmee on November 16, 2013, 12:15:39 PM
@random what kind of problem does your antivirus show up?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Malex on November 16, 2013, 07:26:06 PM
Are you successfull importing the DNGs into Premiere? Used raw2dng it says "this file doesn't include media data".
Do you work with raw2cdng (that's extracting real cinema DNGs)?

Does anybody know where to get raw2cdng apart from the link given in the ML wiki? The page is blocked by my anti-virus.

No not succesful, pink cast, so I converted also DNG to DPX, and it plays nice in Premiere (but not so good in Speedgrade). I encountered one issue with DPX in speedgrade, I don't know how to repeat the bug, but sometimes I do stuff in Speedgrade and then the pink cast comes back and stays there, I have to quit without saving and reopen the project so it opens fine again, weird heh!

I tried raw2cdng 1.4  (which I downloaded here : http://www.magiclantern.fm/forum/index.php?topic=5618.325  from chmee post) but without success, it make the footage dark, but no more pink cast. But other people got better result it seams.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: chris bouk on November 17, 2013, 01:18:49 AM
Hey Chris,
My workflow is:

1) Extracting .DNG files from RAW file with raw2dng -> CLICK (http://magiclantern.wikia.com/wiki/RAW_Video)
2) Import the first .DNG into Adobe After Effects (16bit/32bit project) and mark "Image Sequence", make certain color corrections in ACR
3) Export to 16bit .DPX files (very large) that you can find as "C1neon Sequence"
4) Import into Adobe Premiere Pro CC again marking "Image Sequence", upscaling and editing, color grading in SpeedGrade via directlink or directly in Premiere
5) Export either to h264 or into Avid DNxHD 10bit/175 and transcoding into h264 via MPEG STREAMCLIP -> CLICK (http://www.squared5.com/)

In step 3 you can also export into losless AVI or any other format you want to work with.

Are you successfull importing the DNGs into Premiere? Used raw2dng it says "this file doesn't include media data".
Do you work with raw2cdng (that's extracting real cinema DNGs)?

Does anybody know where to get raw2cdng apart from the link given in the ML wiki? The page is blocked by my anti-virus.

thank you for your help!
I didnt saw the sequence option and I had so many files in my timeline....!!!!
I did export in Avi mode, because it was my first video sample in RAW, during night, high ISOs and not good quality, but I wanted to figured out what the **** is this RAW capability of our cameras....and.....yes man.....is amazing!!!
soon, I will film a new short video and upload to share the results!
thank you man!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on November 17, 2013, 03:09:08 AM
@random what kind of problem does your antivirus show up?

Screenshot (https://www.dropbox.com/s/gyv56li27t4ysgy/BitDefener%20ML.JPG)
Translation: "BitDefener has blocked this page. This page is blocked by BitDefender cloud security")

Usually it shows up when it finds automatic installing malware, or if the page is on a black list. Not saying this is the case, anti-virus also can be wrong, but I have all my movie footage on this computer and the backup is 10.000km away (yeah, I know...), so I'm really careful. It also found a trojana in current raw2dng's (forgot the virus number) so I always used RAWanizer 0.5.0, due to the lack of answers here on my question why it shows up virus (months ago).

Maybe somebody could upload the raw2cdng 1.3 & 1.4 in dropbox or something? Would be great!

Quote
No not succesful, pink cast, so I converted also DNG to DPX, and it plays nice in Premiere (but not so good in Speedgrade). I encountered one issue with DPX in speedgrade, I don't know how to repeat the bug, but sometimes I do stuff in Speedgrade and then the pink cast comes back and stays there, I have to quit without saving and reopen the project so it opens fine again, weird heh!

I tried raw2cdng 1.4  (which I downloaded here : http://www.magiclantern.fm/forum/index.php?topic=5618.325  from chmee post) but without success, it make the footage dark, but no more pink cast. But other people got better result it seams.

Once I got raw2cdng I'm gonna try out if I get the same results. Damn, would have been awesome to directly import into Premiere.


@Chris
You're welcome! Show us your results when it's done! (For highest quality, try to export into 16bit DPX or TIFF instead of losless AVI but don't forget to put the AFX project to 16bit/32bit, too)
 
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: reddeercity on November 17, 2013, 04:35:42 AM
Ramdom, Here a link to a video tutorial i posted about 2 week ago for a Cdng workflow
with Adobe , AE & premiere pro.
http://www.magiclantern.fm/forum/index.php?topic=8177.msg86395#msg86395
I was using raw2cdng version 1.30
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: RenatoPhoto on November 17, 2013, 01:18:50 PM
Screenshot (https://www.dropbox.com/s/gyv56li27t4ysgy/BitDefener%20ML.JPG)
Translation: "BitDefener has blocked this page. This page is blocked by BitDefender cloud security")

Usually it shows up when it finds automatic installing malware, or if the page is on a black list. Not saying this is the case, anti-virus also can be wrong, but I have all my movie footage on this computer and the backup is 10.000km away (yeah, I know...), so I'm really careful. It also found a trojana in current raw2dng's (forgot the virus number) so I always used RAWanizer 0.5.0, due to the lack of answers here on my question why it shows up virus (months ago).

Never heard of such a thing.  You should create a new thread about this so it is not buried deep in this area for other to comment.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on November 17, 2013, 07:43:21 PM
Ramdom, Here a link to a video tutorial i posted about 2 week ago for a Cdng workflow
with Adobe , AE & premiere pro.
http://www.magiclantern.fm/forum/index.php?topic=8177.msg86395#msg86395
I was using raw2cdng version 1.30

Thanks for the video, I'm gonna check this out. So there's no one here having success with directly importing cdngs into premiere?
what a shame. anyways, do you think you could upload raw2cdng for me in dropbox or anything?

Never heard of such a thing.  You should create a new thread about this so it is not buried deep in this area for other to comment.

Yes I'm gonna do that. I researched a little and found a forum that was on BitDefenders blacklist because they had some infected scripts in their code, obviously without knowing about them. Since they deleted those, BitDefender took them from the list and everything worked out fine again.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: chmee on November 17, 2013, 11:04:05 PM
honestly saying i'm tracking visits on my site via piwik - just to see downloads and click-actions. did a sitecheck fi here ->
http://sitecheck.sucuri.net/results/www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

so, the directlink to the app is

http://dl.phreekz.de/raw2cdng/raw2cdng.1.3.0.zip
or experimental beta 1.4.0
http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.0.beta.zip
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on November 17, 2013, 11:24:36 PM
The tracking shouldn't be an issue, anyways I have a scriptblocker and cookies disabled, furthermore so many pages try tracking but aren't blocked.
The site check looks good. Don't know why BitDefender is confused. Anyway, I'd prefer to "trust" it due to my backups being on another continent right now.

The problem is, when I try to download it from the direct link, no matters if I open it in browser or right click on the URL, Bit Defender blocks the download so I only get a broken archive. That's why I asked for a dropbox upload to get it without going via your homepage. 
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: chris bouk on November 17, 2013, 11:54:02 PM
Hey Chris,
My workflow is:

1) Extracting .DNG files from RAW file with raw2dng -> CLICK (http://magiclantern.wikia.com/wiki/RAW_Video)
2) Import the first .DNG into Adobe After Effects (16bit/32bit project) and mark "Image Sequence", make certain color corrections in ACR
3) Export to 16bit .DPX files (very large) that you can find as "C1neon Sequence"
4) Import into Adobe Premiere Pro CC again marking "Image Sequence", upscaling and editing, color grading in SpeedGrade via directlink or directly in Premiere
5) Export either to h264 or into Avid DNxHD 10bit/175 and transcoding into h264 via MPEG STREAMCLIP -> CLICK (http://www.squared5.com/)

In step 3 you can also export into losless AVI or any other format you want to work with.

Are you successfull importing the DNGs into Premiere? Used raw2dng it says "this file doesn't include media data".
Do you work with raw2cdng (that's extracting real cinema DNGs)?

Does anybody know where to get raw2cdng apart from the link given in the ML wiki? The page is blocked by my anti-virus.

....so...
I did, 2 different methods...
first I have to say, working with AE CS6 is easier, because the software recognises the .dng files and no processing in PS before...
I did 2 different videos, the one exporting to AVI Lossless via AE CS6 with some 700MB of output video and the second saving in DPX format and then opening the files as a sequence in AP CS6 and exporting the media as H264 16bit always with a final file of 7MB and a video quite at the same quality! I watched the results only in my laptop's screen and not to my desktop big screen to see the differences of the quality, in the small screen seems quite the same...but the video with the double procedure (AE and AP) has twice the time to be done but you have smaller, (huge difference) file sizes and videos that can be played very easy from any media player! 7MB vs 700MB....huge difference...
tomorrow I will upload some examples...
thank's for your advices!
Chris
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: random on November 18, 2013, 01:40:32 AM
Losless AVI is only supposed to connect AFX and Premiere, just like the DPX, that's why it's uncompressed. It should never be a final format.
Personally I prefer the DPX, or saving as a Premiere file like in reddeercity's video, I didn't try yet though.
Because h264 is a 8bit codec, 16bit isn't possible as far as I know.

cDNG/16 -> Afx (32) -> DPX/16 -> Pr -> h264/8 (or DNxHD/10 -> h264/8) works best for me.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: chris bouk on November 19, 2013, 07:51:22 AM
one of my first tries...

https://www.youtube.com/watch?v=GJ30MZ7GU0Q (https://www.youtube.com/watch?v=GJ30MZ7GU0Q)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: sick stu on December 02, 2013, 05:00:03 AM
Hi all,
I havent read the whole discussion have you tried the free Cineform codec?
Thats the only one i use as the compression is very very low hardly any i think.
Its a slightly compressed Uncompressed codec...somthin like that.
My work flow is (WIN7 32bit) Raw > Cdng> After Effect CS4> Minor Colour correction> batch export to Cineform.MOV
Then into Premiere, final edit > Cineform.MOV archive.
I was exporting to Cineform.AVI but was getting bad moire so i tried Cineform.MOV (Quicktime Quality slider set to MAX)
and that results in alot less moire.
The files are quite big but not that bad.
Hope that helps.
Stuart.
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: budafilms on December 08, 2013, 08:03:23 AM
....fixed...
captured RAW video and now I'm converting the files from dng to psd to open with AE like a multiple files....will see...
any suggestion for the next step to finish the video is very acceptable! Do we have to do all this long job for a short video in RAW?
Working 10000 frames in Ps and then in AE and then APremiere for a 30 seconds video? Is there a shorter way?
best regards
Chris

OPen Raw to DNG and use Resolve Lite. It's free.
Forget Avi and Premiere. Except if you work with birthdays and weddings and all that  ugly stuff....
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: zcream on December 16, 2013, 04:32:16 PM
Any chance of supporting lossless compressed dng ?
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on December 16, 2013, 04:46:02 PM
Sure, just grab the code from cr2hdr (adobedng-bridge.c).
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: 5Djp on January 05, 2014, 07:52:01 AM
raw2dng.exe is giving me skipped frames
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: Fukurou on January 07, 2014, 08:46:24 AM
Edited.
Wrong thread sorry. Please deleted this post.
Title: convert RAW to dng and a video file at the same time
Post by: nibbla on January 21, 2014, 01:51:11 AM
Hi out there.
is it possible do convert several RAW files at once to dng-files and get in that step also a second version of converted video files? ...something like a mirror to start work on and see whitch takes are needed. that files could be small - just to edit and watch them and get a cut of your movie before bringing the necessary dng files for gradingn etc in.
The question is: do i need to convert every RAW file to a videofile manually before start editing? a tool wich i could run over night and spit out dng files and a video file would be awesome. Or a tip for the workflow in Adobe CS6.

Im very amazed of the RAW quality out of the 5d3 - thank you guys for your work!!!

If there is a thread about that question already - i did not see it.

thank you for any informations! Regards from Vienna  :)
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: B-rad on January 24, 2014, 02:38:27 AM
Hi

Just downloaded and installed the nightly build for Jan23 on my 60D to test out the RAW workflow. Everythings been good until I try and open the DNG files from RAW2DNG (which I just downloaded from the link at the beginning of this post,  http://www.magiclantern.fm/forum/index.php?topic=5404.0). My Photoshop CS6 with Camera Raw 8.3 says "Photoshop cannot open this file...if you are trying to open up a raw file from a digital camera make sure you have the latest camera raw update...etc". Camera Raw 8.3 is meant to be the latest for CS6. In camera I am able to play back the RAW files using the playback option, so I feel like those files should be OK. And the converter seems to do its thing normally as well. Any ideas?

If anyone wants I can send you one of the DNG files and see if you can open it.

Thanks
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: a1ex on January 25, 2014, 08:23:34 PM
Updated raw2dng.exe (http://acoutts.com/a1ex/raw2dng.exe) (small metadata fix from Andy600).

Also added raw2dng_cs2x2.exe (http://acoutts.com/a1ex/raw2dng_cs2x2.exe) for cameras with pink dots (650D, 700D, EOS M) - you may know it from the PinkDotRemover thread (http://www.magiclantern.fm/forum/index.php?topic=6658). It's a quick one-size-fits-all solution to pink dots, but it uses no knowledge about the position of these dots (which means it might remove some useful detail too). Use it when PDR doesn't work.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: thomasridout on January 27, 2014, 03:49:13 AM
Hi,

I'm getting pink DNG files I have searched the forums but cant find too much on it. How can I avoid getting these pink images?

The raw2dng spits out a prores that looks fine but the dng's are all pink? :S

I think I can fix them in lightroom but just wondering why

I tried raw2dng and RAWMagic
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: B-rad on January 27, 2014, 10:19:43 PM
Used the linked file for raw2dng.exe from a1ex two posts above and everything's working now...Thanks!
Title: Drop RAW and keep DNGs?
Post by: schlemiel29 on February 05, 2014, 01:25:02 PM
When I use raw2dng I get a set of DNGs which are nearly the same size than the raw file. Because these files are size of GB, I would like to save hard disk space. Are the DNG the same quality than RAW or is a lossy algorithm used? Should I drop RAW and keep DNG, so I can drop additional single DNGs similar to cut the movie to relevant parts?
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: a1ex on February 05, 2014, 01:32:02 PM
raw2dng does some small corrections like fixing vertical stripes (banding) or bad pixels, but other than that it's lossless.

There are other tools that can save files with a lower bit depth (lossy). You can also use Adobe DNG Converter (can do both lossy and lossless compression). If you use MLV, you have some lossless compression options too.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: g3gg0 on February 05, 2014, 01:43:24 PM
and baldand is developing a lossless/lossy compression with up to 65% saving for MLV (for offline archiving, not during record)
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: morsafr on February 05, 2014, 04:14:30 PM
Simple question but answer seems hard to find (at least for me): where can I get the last version of raw2dng for OS X?

I saw A1ex post a few lines above with a link to Windows version but I'm currently running OS X.

Thanks!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Danne on February 05, 2014, 04:51:35 PM
Update by RenatoPhoto:
For Mac download:
Mac GUI for cr2hdr: https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
dcraw and exif tool: https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr.zip

I think the links are from A.D who build a gui for mac. More links here:
https://bitbucket.org/a_d_/magic-lantern/downloads/
http://www.magiclantern.fm/forum/index.php?topic=7139.msg86573#msg86573
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: feureau on February 05, 2014, 06:34:53 PM
and baldand is developing a lossless/lossy compression with up to 65% saving for MLV (for offline archiving, not during record)

Yay! \o/ I've been wondering how I'd store these ungodly file sizes.

What's the future for MLV btw? (any "roadmaps"?)
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: morsafr on February 06, 2014, 11:19:24 PM
Thanks Danne for the links but I cannot find raw2dng inside the Zip  :(
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Donnyriz on February 09, 2014, 09:33:53 PM
Hi,

I'm getting pink DNG files I have searched the forums but cant find too much on it. How can I avoid getting these pink images?

The raw2dng spits out a prores that looks fine but the dng's are all pink? :S

I think I can fix them in lightroom but just wondering why

I tried raw2dng and RAWMagic

I too have been searching for an answer to this problem.  All of the my DNG's that come from Raw2DNG are completely overlayed with pink. When I open them in ACR, the pink is not there.  But when opened in Resolve, it looks terrible.  Ive tried using RAW Magic also, and the files are even worse in Resolve.

RAW2DNG sample http://s226.photobucket.com/user/donnyriz/media/Screenshot2014-02-09at121629PM.png.html
RAW2DNG in resolve http://s226.photobucket.com/user/donnyriz/media/Screenshot2014-02-09at123103PM.png.html

RawMagic in Resolve  http://s226.photobucket.com/user/donnyriz/media/Screenshot2014-02-09at122633PM.png.html

I would rather be able to open the DNGs right in resolve rather than going through ACR first.

Anyone else have this issue?
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Malex on February 11, 2014, 09:43:40 AM
I too have been searching for an answer to this problem.  All of the my DNG's that come from Raw2DNG are completely overlayed with pink.
use the converter of Chmee, http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/  version 1.4.9
it has a partymode checkbox which remove the pink cast that appear in Premiere. (maybe works for Resolve too...)

here is the thread : http://www.magiclantern.fm/forum/index.php?topic=5618.475
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: davegravy on February 14, 2014, 05:31:35 AM
Hi Alex,

I'm was having an issue with green dots in overexposed areas from my 650d... i used your raw2dng whitefix version and it got rid of the green dots except left some black/grey dots in the overexposed regions during playback in Resolve 10 lite. I went to inspect the dngs with windows picture viewer and these dots couldn't be found! What could your program be doing to cause Resolve to interpret these pixels as dark grey?

Thanks!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: davegravy on February 14, 2014, 05:10:43 PM
Hi Alex,

I'm was having an issue with green dots in overexposed areas from my 650d... i used your raw2dng whitefix version and it got rid of the green dots except left some black/grey dots in the overexposed regions during playback in Resolve 10 lite. I went to inspect the dngs with windows picture viewer and these dots couldn't be found! What could your program be doing to cause Resolve to interpret these pixels as dark grey?

Thanks!

Nevermind, looks like a bug related to Resolve and possibly my graphics card...

(http://forum.blackmagicdesign.com/viewtopic.php?f=3&t=12941)

EDIT: Or not... the solution in the above link doesn't work for me, and I also noticed that the "black" pixels are actually a very dark green. If you have any ideas Alex I'd love to hear them!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Donnyriz on February 16, 2014, 09:01:56 AM
use the converter of Chmee, http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/  version 1.4.9
it has a partymode checkbox which remove the pink cast that appear in Premiere. (maybe works for Resolve too...)

here is the thread : http://www.magiclantern.fm/forum/index.php?topic=5618.475

Thanks, but Im on MAC. This appears to only be a WIN converter.
Its strange that all the DNG's have a pink overlay when previewed in the finder (even the thumbnails are pink) but it goes away in ACR. But I do not want to go through ACR. I just want to go str8 to resolve to make proxies.

Could this be something in my ML settings on camera? Though when I play back the files in the camera, 5Dm3, they look perfect.  Cant figure out why the pink overlay appears after conversion.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Donnyriz on February 26, 2014, 08:30:02 PM
Hi,

I'm getting pink DNG files I have searched the forums but cant find too much on it. How can I avoid getting these pink images?

The raw2dng spits out a prores that looks fine but the dng's are all pink? :S

I think I can fix them in lightroom but just wondering why

I tried raw2dng and RAWMagic

Wow. I guess I finally figured this out, sort of. So apparently the problem with the Pink DNGs is due to Snow Leopard (10.6.8). I looked at the same files on a Mountain Lion computer (10.8.5), and they looked perfect. They also loaded into Resolve without any issues. I have no idea what the hang-up with SL is, but doesn't matter. Ive upgraded now. Hope this helps someone else out!
Title: Re: RAW Video Post processing [Mac/Win/Linux]
Post by: pascal on March 08, 2014, 07:49:37 PM
Updated raw2dng.exe (http://acoutts.com/a1ex/raw2dng.exe) (small metadata fix from Andy600).
Many thanks for keeping your tech updated!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: guentergunter on March 25, 2014, 09:46:46 PM
Hello g3gg0,
since a few days now there's a discussion about the future resolutions available for the 5D2 (and maybe other cameras).

It started with a1ex:

The horizontal resolution is restricted to multiples of 8 bytes and 16 pixels according to latest findings. This restriction is valid at least for 5D3 and 6D (didn't do much testing on other cameras), but I'd like to keep the code portable without camera-specific exceptions.

So, before including this change in nightly builds, I'd like to ask you which is better: 1888 with 8 pixels of black border that you will have to crop, or 1856 without any border pixels? Between these 2 values, there are no valid resolutions that respect the alignment restrictions.

This change was discussed here:

http://www.magiclantern.fm/forum/index.php?topic=3904.msg106087#msg106087
https://bitbucket.org/hudson/magic-lantern/pull-request/438/raw-recording-force-line-size-to-be/diff


In my opinion, it would be no problem to simply discard black borders from an image while converting to dng:


...and it's also no coding effort to discard pixels while converting to dng. The user wouldn't even realize it!
This way, no one has to crop anything actively in post! It's simply done in background in the ever necessary converting (to dng) step.

I'm no programmer (just some very basic skills).
Thus it would be really great, if you could just leave a little comment about it as a programmer to help us sort things out for the 5D2 (and other cameras with the same problem):

Specific 5D2 threat (http://www.magiclantern.fm/forum/index.php?topic=5533.new#new)

Thank you very much!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: dfort on March 26, 2014, 07:26:45 PM
Mac executable: raw2dngOSX.zip (https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dngOSX.zip) (run in terminal, don't forget chmod +x)
Windows executable for cameras with pink dots (650D, 700D, EOS-M): raw2dng_cs2x2.exe (http://acoutts.com/a1ex/raw2dng_cs2x2.exe) (it does some chroma smoothing, which happens to remove the pink dots too; use it when PinkDotRemover (http://www.magiclantern.fm/forum/index.php?topic=6658) doesn't work)

Hi g3gg0, a1ex and scrax,

I don't want to double post but yesterday I posted this message: http://www.magiclantern.fm/forum/index.php?topic=5508.msg108656#msg108656 and realized that maybe I should have posted here instead.

Basically, I'm in need of removing pink dots on EOS-M raw files and PinkDotRemover isn't working for me. To make things a bit more complicated I'm on a Mac. I do have access to a Windows machine at work and raw2dng_cs2x2.exe does the trick but I'd rather keep my files on my personal computer.

If there isn't a Mac executable of  raw2dng_cs2x2 I'd be happy to try my limited skills at compiling it but I have no idea where to start.

Edit: Well I'm not completely clueless. I did find this to get started: http://www.magiclantern.fm/forum/index.php?topic=7645.0 but I am still at a loss where to find the source for raw2dng_cs2x2, I did find the vanilla raw2dng code.

Thanks!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: a1ex on March 28, 2014, 09:56:03 PM
Minor update to raw2dng (fixes some weird kind of cold pixels like in this image). See first post for the link.

(http://a1ex.magiclantern.fm/bleeding-edge/raw/101.jpg) (https://www.dropbox.com/s/9v0uut63qawuqhb/101.jpg)
https://bitbucket.org/hudson/magic-lantern/commits/ea59b5b6517a
http://www.magiclantern.fm/forum/index.php?topic=7122.msg109315#msg109315

Also, it looks like any image quality updates I implement in raw2dng will also take effect in mlv_dump (http://www.magiclantern.fm/forum/index.php?topic=7122) (so the fix is there too). Cool!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: davegravy on April 01, 2014, 02:56:24 AM
Thanks alex! Are all the versions which are posted in the first post updated with this? (specifically, is the cs2x2 version updated?)
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: a1ex on April 01, 2014, 07:34:23 AM
No; do you have a sample that requires both workarounds?
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: applewide on April 01, 2014, 08:33:55 AM
hi

may i ask what is the fastest work flow for 1 april 2014

mine is

1 raw2dng mac
2 ps adjust color and covert to tif
3 quicktime convert to mov 4444

is this the best?
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: budafilms on April 02, 2014, 03:13:09 AM
hi

may i ask what is the fastest work flow for 1 april 2014

mine is

1 raw2dng mac
2 ps adjust color and covert to tif
3 quicktime convert to mov 4444

is this the best?

Fastest for mac?

DNG to Da Vinci Resolve to Whatever you want.

Best? Depende your skills. I prefer Da Vinci, most appear to be ACR.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: applewide on April 02, 2014, 05:24:06 AM
hi

thanks buddafilms

i have come up with the new way

ML RAW 2, i think it is better, because it can work with colorchecker profile that i have it, raw 1 doesn't have this

1 using mystic GUI to convert to DNG
2 ps cs3 on imac adjust the color sync it and then export to tif, this is long not sure why, i have the fastest 32gb, i7 mac still slow
3 quicktime open in sequence and then export to 422 hq, then i can play it without the stutter

image is like 4k

i can only record with fastest card 128gb komputerbay 1050x still only 1920x818, if i try 1920x1080 it will drop after 10 seconds

i think if i can expoert from ps to mov, it would be so great step 3 is a pain, i understand that colour  adjustment is necessary
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: budafilms on April 02, 2014, 05:37:10 AM
hi

thanks buddafilms

i have come up with the new way

ML RAW 2, i think it is better, because it can work with colorchecker profile that i have it, raw 1 doesn't have this

1 using mystic GUI to convert to DNG
2 ps cs3 on imac adjust the color sync it and then export to tif, this is long not sure why, i have the fastest 32gb, i7 mac still slow
3 quicktime open in sequence and then export to 422 hq, then i can play it without the stutter

image is like 4k

i can only record with fastest card 128gb komputerbay 1050x still only 1920x818, if i try 1920x1080 it will drop after 10 seconds

i think if i can expoert from ps to mov, it would be so great step 3 is a pain, i understand that colour  adjustment is necessary

Check your CUDA Drives for Mac. But, I think your mac could have AMD not NVIDIA. Check it.
CS, Da Vinci, needs Cuda software up to date.

Depends you final work - internet, DVD - do you need export to Tiff? It's huge file size.

You don't mention your camera... 5D Mark III? In that case should work with 1920x1080 24 or 25p Continous!

Some user are reporting problems with last MLV, try the release from march 17. It works for me.

Nice image, really? ;)
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: applewide on April 03, 2014, 05:51:24 AM
  Chipset Model:   NVIDIA GeForce GTX 780M
  Type:   GPU
  Bus:   PCIe
  PCIe Lane Width:   x16
  VRAM (Total):   4096 MB
  Vendor:   NVIDIA (0x10de)
  Device ID:   0x119e
  Revision ID:   0x00a2
  ROM Revision:   3755

ok i shall try buddafilms


really nice, even it has exif embedded  in it


tif and jpg when convert to mov, which one is better quality for 422 HQ?

(https://farm6.staticflickr.com/5587/13594835995_a9ba341857_o.jpg)

this is ML1 i think no exif
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: budafilms on April 03, 2014, 11:20:29 AM
  Chipset Model:   NVIDIA GeForce GTX 780M
  Type:   GPU
  Bus:   PCIe
  PCIe Lane Width:   x16
  VRAM (Total):   4096 MB
  Vendor:   NVIDIA (0x10de)
  Device ID:   0x119e
  Revision ID:   0x00a2
  ROM Revision:   3755

ok i shall try buddafilms


really nice, even it has exif embedded  in it


tif and jpg when convert to mov, which one is better quality for 422 HQ?

(https://farm6.staticflickr.com/5587/13594835995_a9ba341857_o.jpg)

this is ML1 i think no exif

About your NVIDIA, it's on board, it's not too faster like an external GPU 780, and you can download from apple or nvidia the new drivers. Then you can select use OSX drivers or Nvidia drives. Maybe go faster for your export form ACR.

TIFF it's better than JPG. And 4444 much better than HQ. But think the final use of your work: Vimeo or youtube don't need this.

If you use Resolve, it's directly form DNG to Pro Res, HQ or better, 4444 to " touch" a little again in the soft that you used for edit.

As you know Uncompressed it's the most, but it's impossible for the space that need it. You can play with few seconds to see the difference.

If you continue using ACR, remember to set 16 bits, not less.

If you edit with final cut X, remember the soft convert all the files to Pro Res - not HQ - and then start to export. So, put the files more than Pro Res, it's the double job for your system, and you lost a generation.

Be smart with the final use of your work to take advantage of the quality and your time!!! ;)

Good work!!!!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: applewide on April 04, 2014, 10:44:09 AM
yeah thanks buddafilms, i could not find the drive, please give links

yes i use final cut pro x to edit, i want to speed up the process, it is taking too long

actually just adjust and convert to pro res is enough workflow, too long
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: budafilms on April 05, 2014, 07:55:38 AM
yeah thanks buddafilms, i could not find the drive, please give links

yes i use final cut pro x to edit, i want to speed up the process, it is taking too long

actually just adjust and convert to pro res is enough workflow, too long

Sure!

http://www.nvidia.com/object/macosx-cuda-5.5.47-driver.html

Give a night to Davinci Resolve ;)
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: applewide on April 05, 2014, 07:42:03 PM
i didn't realize, i need to install driver for mac os

i am changing vesa mount on imac 27 now, will try later in the week

so pro res 422 only right for fcpx, got it

i still can not figure how to record audio, any luck on that?
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: crabman0 on May 01, 2014, 10:46:38 PM
I did alot of reading on the RAW proccessing workflow, and have 1 thing unclear for me;
After the image sequence is turned into a clip (ima using Vegas) whats the rendering settings should be?
If my video is 960x544 14bit 23.96 fps. And also how do I uprezz it?
Thanks in advance! :)
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: bwatkins on May 15, 2014, 12:12:27 AM
I've been profoundly frustrated with a RAW post-processing problem that regards importing the DNG files into After Effects.  Here's a rundown of my woes:

1) Record ML RAW 1920x1080 24fps on 5d Mk III (Confirmed that the cam setting is 24fps and FPS override is set to 24fps exact).
2) Convert MLV to DNG using MLV Converter.
3) Confirm the DNG files match # frames needed for recorded duration.  (I.E.  10 seconds = 240 frames)
4) Import DNG sequence into After Effects (24 fps comp)
5) **BAM** the duration shown on the comp timeline is EXACTLY 25% fewer frames than should be: Meaning it's looking for an extra 6 frames/sec. for 30fps. But where?  All my settings are at 24fps? 

**FIGURED IT OUT**  Wrong settings in After Effects: Selected DNG sequence -> Interpret Footage -> Select 24fps (Appears to default to 30fps)

Stupidity on my part, but I'll leave this up for others who may have the same problem.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: jarandov on May 20, 2014, 11:27:11 AM
Hi bwatkins

what is your mlv converter? How can I find and download that. I am using MAC and I want to convert MLV file to DNG.  RAW2DNG doesn't support MLV files.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: jarandov on May 20, 2014, 01:07:15 PM
Hi guys

it could be great if someone create a program for mac like Rawmagic lite ( no problem to pay for it) if the program could handle mlv files correctly. I thinking about Mac version no windows. I hope that i can see some this program in close future.

Thanks in advance.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: HD Cam Team on May 21, 2014, 09:53:50 PM
Hi,

Sorry for my question, I've been out of the loop for long time.

Is there any app for Windows that converts the .raw file to some kind of video file? (there wasn't any for windows in 2013, only for Linux, but maybe there is one now)

Thanks a lot
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Frank7D on May 21, 2014, 10:59:29 PM
Hi,

Sorry for my question, I've been out of the loop for long time.

Is there any app for Windows that converts the .raw file to some kind of video file? (there wasn't any for windows in 2013, only for Linux, but maybe there is one now)

Thanks a lot

There's more than one: http://www.magiclantern.fm/forum/index.php?board=54.0 (http://www.magiclantern.fm/forum/index.php?board=54.0)
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: HD Cam Team on May 22, 2014, 07:16:02 AM
There's more than one: http://www.magiclantern.fm/forum/index.php?board=54.0 (http://www.magiclantern.fm/forum/index.php?board=54.0)

Thanks!

I see the are many topics there.

What application would you suggest to convert ML .raw files directly to a video file format (not sequence of images) in Windows, to then process in Premiere/AE ?

Thanks once again
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Frank7D on May 22, 2014, 06:36:46 PM
Thanks!

I see the are many topics there.

What application would you suggest to convert ML .raw files directly to a video file format (not sequence of images) in Windows, to then process in Premiere/AE ?

Thanks once again

Haven't used the video output option, but I've heard good things about this one: http://www.magiclantern.fm/forum/index.php?topic=9560.0 (http://www.magiclantern.fm/forum/index.php?topic=9560.0)

Personally I like using ACR so I convert to dngs and then work in AE.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: martin_a on May 23, 2014, 09:55:28 AM
Hi guys,

I've the last nightly build installed on my 5D Mark III running 1.2.3. I did a few tests yesterday with my 24-70mm f/2.8L II USM with AF. I've recorded a mlv clips with sound from a rode video mic pro on a LEXAR 32 Gb 1066x, I'm converting my clips from mlv to DNG with MlRawViewer and i have a few questions...


I've been searching answers on this forum for a few days but I found nothing... Thanks for your help !
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: martin_a on May 27, 2014, 09:37:48 AM
Nobody here ?  :-[
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: grantcallegari on June 01, 2014, 04:34:07 AM
Hi martin_a

I've had the same problem with the vertical banding. The only solution I've found is bumping up the luminance in ACR. It helps but not a great solution.

With AE are you going Import > File > then select the first image in the sequence and check the Camera Raw Sequence box if not already selected.

I haven't noticed an issue with white balance. I do have an issue with the colour/luma shift from AE to finished ProRes 4444 render.

Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: reddeercity on June 01, 2014, 05:32:13 AM
Quote
I haven't noticed an issue with white balance. I do have an issue with the colour/luma shift from AE to finished ProRes 4444 render
What's your project setting ?
I use 32bit for my project & use ProRes4444 RGB+A with Trillions+ colors and there is not Color/Luma Shift.
The only time I have seen color/luma shift is when the image is out of Video Range colors as you are going from
Full Range Color (0-255) and compressing it to Rec 709 (16-235) with ProRes. Watch your wave monitor in A.E.(ACR) very important  :D
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: grantcallegari on June 01, 2014, 05:52:43 PM
What's your project setting ?

I had my project set to 8bits per channel with millions of colors. Does having it set to 32bit make a big difference over 16bit? I have a warning telling me that some effect only support 8 and 16 bit per channel colour and using 32 bit may reduce color precision and may remove any high dynamic range values. Do I still use 32 bit in this case?

Thanks for the help!

Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: reddeercity on June 01, 2014, 09:56:16 PM
I had my project set to 8bits per channel with millions of colors. Does having it set to 32bit make a big difference over 16bit? I have a warning telling me that some effect only support 8 and 16 bit per channel colour and using 32 bit may reduce color precision and may remove any high dynamic range values. Do I still use 32 bit in this case?

Thanks for the help!
Are you grading in A.E. ?
What's your workflow ? Premiere Pro ?
I would use no less then 32bit project as this will yield a 12bit ProRes 4444 which is the closes to your source.
If you work in PP then you can convert to 12bit Cdng's and edit them native and dynamic link to A.E., Speed Grade or export EDL/XML to Resolve.
When I work only with .MOV ProRes I pergrade with ACR to get the file to legal Video Colors then go on from there.
Hope that helps. :)
 
Title: RAW Video Post processing (raw2dngApp and raw2dng) [Mac/Win/Linux]
Post by: mkrjf on June 22, 2014, 12:33:29 AM
Shot indie short on 5DMk3 with ML hack last weekend !yay!
But I can only convert some of the footage to prores444 .mov using raw2dngapp and raw2dng on Mac OSX (mavericks)
Sometimes the captured footage is one large raw file > 4GB other times it is a set of files (001, 002).
Cat method of just recombining files seems to work.

For some of the original one file RAW files raw2dngapp creates DNG files but no prores .mov. (see below)
Whole reason for me to use is to get the .mov foe editing. Would rather not have to edit with .dng.
I suspect the 'error' messages are invalid as all files have identical framerate and some process completely and others don't.

BTW can anyone confirm output is true prores 4444? Some applications report the output is prores4444 but others show as prores422 (who knows which program is correct) (using quicktime viewer, VLC, and mpeg stream clip)

Can someone confirm what FFMPEG command and what version of FFMPEG so I can repeat manually?
Only the ffmpeg output is shown not the cmd executing (maybe that could be echoed in next version?)
Sometimes I get a cryptic error: maxvalue=0.
Also - raw files smaller than a certain size always fail (not the issue here, just fyi).

raw2dng converter GUI for OsX
Beta ver.0.13

Wish raw2dng showed version! just shows usage

So, for example a 12.15 GB single raw file was captured for one take and raw2dngapp successfully converted and created 3.24 GB prores444 .mov - fabulous.

But for a 5.58 GB single raw file it created 1536 dng (not sure if it created all successfully) but then failed to create .mov - see error below (but sometimes there is error/warning of maxvalue=0)

raw2dng converter GUI for OsX
Beta ver.0.13

M13-1450 File Supported
Generating ProResHQ 4444 with FPS: 0.000
ffmpeg version 1.2.1-tessus Copyright (c) 2000-2013 the FFmpeg developers
  built on May  9 2013 21:58:14 with llvm-gcc 4.2.1 (LLVM build 2336.1.00)
  configuration: --prefix=/Users/tessus/data/ext/ffmpeg/sw --as=yasm --extra-version=tessus --disable-shared --enable-static --disable-ffplay --enable-gpl --enable-pthreads --enable-postproc --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-libspeex --enable-bzlib --enable-zlib --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libxavs --enable-version3 --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvpx --enable-libgsm --enable-libopus --enable-fontconfig --enable-libfreetype --enable-libass --enable-filters --enable-runtime-cpudetect
  libavutil      52. 18.100 / 52. 18.100
  libavcodec     54. 92.100 / 54. 92.100
  libavformat    54. 63.104 / 54. 63.104
  libavdevice    54.  3.103 / 54.  3.103
  libavfilter     3. 42.103 /  3. 42.103
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[image2pipe demuxer @ 0x10201f400] Could not parse framerate: 0.000.
pipe:0: Invalid argument
Done

If there are newer versions pleas point me to. I know I spark the 'didn't you google for t^%&^% to get the already posted answer?' but sometimes what to google for is the issue ;)
Thanks in advance
Eager to brag about the value provided by ML for indie filmmakers!!
Mike
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Planet-X on August 26, 2014, 09:29:36 PM
I've been trying (on and off) for about 6 months to get this to work.  Clearly I've done something wrong.  I believe that  I'm shooting RAW (I get what I think is RAW playback in the camera) but when I go into the card there is no folder with individual files.   There is only one file with a .MLV extension that the raw2DNG software says it doesn't recognize.  I'm sure I must have missed something in this forum.  Can anyone help me or point me in the right direction.  I should mention that I'm on a Mac G5 Intel 8 core.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Walter Schulz on August 26, 2014, 09:40:56 PM
http://magiclantern.fm/forum/index.php?board=54.0
MLV files are not identical to RAW video files.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Planet-X on August 27, 2014, 12:18:21 AM
Walter, thanks for the info... So do I need to convert these first?  What do I need to do here? From what I read, I thought there was suppose to be a folder with individual frames in it. I'm fairly astute on just about everything else about the camera.   Do I need third party software? Thanks  Will
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Walter Schulz on August 27, 2014, 05:16:17 AM
Open link, start reading there.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Planet-X on August 27, 2014, 08:46:59 PM
Walter... Many thanks for your kind reply.  The link seems to just return me here to this page.  I apologize for asking stupid questions which are probably covered in great detail someplace on this site.  This thread alone has more than 400 entires.  Is there an up-to-date guide to this conversion that I missed?   Sadly I'm not a programmer only a filmmaker.

Since I'm new here I'll give a little background on myself.   I'm new to posting but not new to the site. I'm a minor financial contributor to ML and great proponent of the work done here.  I'm a director and writer working mostly in Horror and Sci-Fi.  I've directed episodes of HBOs Tales from the Crypt, Showtime's Master's of Horror and several feature films for Warner Bros.  I've also worked as a Director Photography and in Visual FX.  I've been working as a writer and director since 1980 and a long-time member of the Director's GUild.  I just want to say "Hi" and thank all of you for doing such amazing work here.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Walter Schulz on August 27, 2014, 09:09:11 PM
Quote
The link seems to just return me here to this page.

Link will lead you to the forum "RAW Video Postprocessing" and this thread is only one of them. MLV is covered in other threads.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Planet-X on August 29, 2014, 02:11:10 AM
So... I think I get it, after reading a bazillion posts. Seemingly, the RAW video thing is really not any sort of practical tool.  A nice thing to play with if your a programmer....  raw2dng apparently doesn't support MLV files... (which seems odd as they were both created in this environment) and the latest Nightly Builds (unless I missed something in the camera menus) only output MLV files and not the advertised folder with discrete raw images in it.  I've done an extensive search of the internet and this site and apparently this is no converter for this.... Or at least non that I could find. There seems to be a kind of Cheshire Cat smile behind those clouds.   Frustrated Filmmaker and Mac User.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Dmaff06 on August 29, 2014, 03:36:14 AM
What is the best workflow for getting the mlv raw into a format I can edit in fcx? I have tried a few converters but I end up missing audio or there are bright pink dots or green lines across the screen.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: reddeercity on August 29, 2014, 05:39:18 AM
@ Dmaff06, I'm a FCPX user , for me I  found it was easer & faster to go cross-platform to Windows 7 .( I has a Laptop for this)
I extract my MLV/Raw+audio with MLVBrowseSharp (link-->  http://www.magiclantern.fm/forum/index.php?topic=8447.msg77759#msg77759)
on windows to raw+audio. There is also a Mac App version.
then convert them with raw2Cdng 16bit , I find v1.4.9 work the best as there display right in finder in Mac
and can be edited native in fcpx 10.1.3 as a compound clip.

Or Take the Cdng's import in to Apple Compressor or Motion & export a 16bit ProRes 4444 XQ .

Personally I use Adobe A.E. CC with ACR plugin for high light recover and basic adjustment etc....
then export 16bit prores 4444xq .

Or you could use RawMagic which will convert the MLV+audio to Cdng's , make a file from there in Compressor etc..
here are some links to look over
MlRawViewer 1.2.3 (CDNG/MLV/RAW Viewer & Encoder, Linux/Mac/Win) --> http://www.magiclantern.fm/forum/index.php?topic=9560.msg91165#msg91165
RAWMagic 1.0, a GUI CinemaDNG converter for Mac OS X!  -->http://www.magiclantern.fm/forum/index.php?topic=6218.msg46900#msg46900

Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: reddeercity on August 29, 2014, 05:52:53 AM
@ Planet-X , You should be able to record in Raw v1.0 as well as MLV v2.0
I shoot on a 5D2 and I use both formats ,  Raw v1.0 in 3x Crop mode  & MLV+audio in 1:1
Links in my last post for the FCPX user , those  programs support MLV native.
As will as Raw2Cdng link -->http://www.magiclantern.fm/forum/index.php?topic=5618.msg39204#msg39204 
so I'm not sure what you platform is (Mac or/& PC) but there is converter for all platforms .

P.S. I just notice you are on a Mac , but my post still applies
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Planet-X on August 29, 2014, 04:54:08 PM
Redeercity,  My sincere thanks for your kind reply. I'm using a 5 Mk2 and Mac G5 Intel 8 core.  I'm on the last version of FCP just before they went to FCP-X.  I have retained that version for numerous reasons too lengthy to go into here.  Are you saying Final Cut Pro X supports MLV files?  I didn't see that on the Apple website. 
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: reddeercity on August 29, 2014, 11:31:24 PM
No , not MLV but "Cdng's" with maverick osx and Nvidia  video card,
FCPX supports Cdng's as well as Open Exr, Your System need to be upgraded to take advantage of
the latest software. Even a MP1.1 is outdated(I have one) . That being said the only way of getting
MLV/Raw workflow with FCP7 & G5 Mac is to convert to 16bit Tiff's or Open Exr  import as a  image sequence.
Or make a reference movie from image sequence with qt7 pro.
Also make ProRes files , and if you have Adobe A.E. or Photoshop, that would be the best as then you have access to the raw setting with ACR plugin.
What is your OSX ? I think Lion is minimum for most of the App for MLV and I think G5's are limited to Snow leopard but not sure.

Note: FFMEG supports MLV but no app are use it yet,
Link to Post -->http://www.magiclantern.fm/forum/index.php?topic=11566.msg112415#msg112415
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: ONEfromTWO on November 02, 2014, 11:05:07 PM
Hello. Boys, help. raw2dng_cs2x2.exe works only one time, after it is necessary to move the program and clean a register, that it began to work. And so each time! Is there what that decision?
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: rainless on November 03, 2014, 10:36:35 AM
You probably need to have the latest version of the .NET frame work installed.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: chmee on November 03, 2014, 10:39:38 AM
@reddeercity
..regarding ffmpeg.. its too buggy yet. i tested some weeks ago, and the convertings only did work to mpg4 with no properties. no, its not nearly done (on windows - zeranoe build tested). sorry.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Oannes on November 27, 2014, 09:35:51 AM
I am seeing a lot of flickering in my final export and I did not mess with curves at all. Here is my work flow.

1. Turn on 5DMark2 camera
2. Flip on liveview
3. Access ML menu and load modules - raw_rec
4. Scroll to ML menu tab Movie and select RAW video option
5. Start RAW video
6. Hit record on my camera
7. Upload video footage to computer (pc)
8. Convert raw files to DNG format
9. Use Lightroom to convert DNG files to TIFF or JPG
10. Import TIFF or JPG into Premiere CC
11. Export the sequence in H.264 format 1280x720
12. Play video
13. Video has micro second gaps in it where there are dead spaces as a result of the bracketing exposure shots. ( youtu.be/37Em8DqObD0 )
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: pauledd on December 01, 2014, 02:06:06 PM
Hi there and sorry, but I dont get it. I am on Linux (64bit). I have a *.raw file. I want to convert it. The script "raw2avi.sh" complains about "raw2dng not found". Ok downloaded raw2dng.c &  chdk-dng.c but how to compile this?
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: gohai on December 07, 2014, 04:14:03 PM
Edit: Problem solved by manually adding padding and the footer.

Hello,

I am trying to use raw2dng on a bunch of RAW file from a 5D3 1.2.3 (w/ latest snapshot). All but one file work flawlessly, but with one of them I get a This ain't a lv_rec RAW file error. The file has a size of 4,294,967,295 bytes, if this makes a difference.

Is there any chance of somehow recovering the content of this clip? (at least partially)

Thanks
Gottfried

The first 100 bytes are like this
Code: [Select]
DC 2A 22 E1 CD AD 2B F5 E1 04 B0 B2 42 CE 08 2B 62 E2 0E B0 2B 06 E2 08 B0 A2 2B 0E 08 2B 12 E2 4D B1 2B F6 E2 18 B2 32 3E 0E E4 2A 52 E2 0E B0 2B 02 E2 2C AC A2 49 0E 4C 2B 02 E3 CE B1 2A 06 E3 C8 AF A2 40 8E 08 2B 72 E1 CE B5 2B 05 E1 54 B5 72 43 4E 78 2B 82 E2 4E AF 2B 15 E0 00 B1 E2 4F 0E F8 2A
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: xerxes4497 on December 15, 2014, 10:43:55 PM
Hey guys, I'm constantly getting this error with a long RAW recording in 4 files. Raw2CDNG constantly crashes when I drag the RAW file in. Shot on a 5D3.

Using 1.6.1 on the latest NET framework.

This file also doesn't extract in RAWanizer. Any help would be greatly appreciated. Thanks!

Problem signature:
  Problem Event Name:   CLR20r3
  Problem Signature 01:   raw2cdng_v2.exe
  Problem Signature 02:   1.5.0.8
  Problem Signature 03:   541ec237
  Problem Signature 04:   mscorlib
  Problem Signature 05:   4.0.30319.18444
  Problem Signature 06:   52717edc
  Problem Signature 07:   24d
  Problem Signature 08:   18
  Problem Signature 09:   System.ArgumentOutOfRange
  OS Version:   6.1.7601.2.1.0.256.48
  Locale ID:   1033
  Additional Information 1:   0a9e
  Additional Information 2:   0a9e372d3b4ad19135b953a78882e789
  Additional Information 3:   0a9e
  Additional Information 4:   0a9e372d3b4ad19135b953a78882e789
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Lasting Image on January 13, 2015, 10:46:43 PM
I'm totally new at this and hoping for some feedback before I waste time on a non-functional workflow. This is what I have in mind:
From ML RAW files on the CF card -
to RAWMagic lite conversion -
to SpeedGrade -
to Premiere Pro  -
to After Effects.

Does this make ANY sense?
Trying to get a A,B,C answer here on RAW workflows seems to require an engineering degree  :-\

Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Kharak on January 14, 2015, 12:20:49 AM
@lasting image

Why do you want to end with After Effects?

Nothing makes sense when you don't state your goal.. Is AE for VFX? Final edit?

I think its hard to get an A B C answer here, because there are 10 different workflows for MLV and all have there pros and cons and some people just enjoy one workflow better than another..

Don't think there is "the one" workflow yet.

How about using some of the supported Converters, MLRawViewer or MLVmystic? or no conversion and go with MLVFS is you are a macman. You wont get far with RawMagic here

edit: forgot my favorite. RAW2CDNG from Chmee
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Frank7D on January 14, 2015, 03:41:30 AM
MLVFS also works if you are a "pc man":
http://www.magiclantern.fm/forum/index.php?topic=13354.0
for example.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Lasting Image on January 14, 2015, 07:17:26 AM
@ Kharak

Thanks for the info.  Yes, using a Mac. I was thinking AE for a final edit but if that's the wrong choice -- I'm still on the ground floor looking up at a big fat learning curve and can't see the top of it yet. PLEASE let me know if you have a better place for me to start.  After my last post I found out about applying Lumetri correction effects in Premiere. That may be as good a place to final edit as any for now, I guess.  The MLVFS idea sounds good and I'll be looking into that tomorrow. Thanks again and do please let me know if you have other thoughts on workflow that may get me started. Thanks again!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: dmilligan on January 14, 2015, 01:38:44 PM
There's no reason to use AE, unless you need to some complicated compositing or VFX (motion tracking, 3D, image stabilization, 'fake' slo-mo, animation, etc.). And even if you did need to do stuff like this in AE, you still probably want to come back to Premiere for final edits (Premiere is made for doing final edits, AE not so much).

I would recommend:
- Mount your MLV files with MLVFS
- Load CDNG sequences directly in Premiere
- Either grade in Premiere (the tools are actually decent if you learn how to use them), or "Direct Link" to SpeedGrade
- Edit in Premiere (jump back into Sg at any time to do more grading)
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Lasting Image on January 14, 2015, 04:35:29 PM
@dmilligan

Excellent!  Thanks!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: WhiteMagic on January 20, 2015, 01:42:41 AM
Today I encountered "Error: This ain't a lv_rec RAW file" when trying to use raw2dng on a set of RAW video files that came from a single recording on my Canon 7D that exceeded the 4GB limit. I actually had a series of files named like this:

M19-1734.RAW
M19-1734.R00
M19-1734.R01
M19-1734.R02

raw2dng.exe showed me the same error cited above when I tried to execute it on all the files except last (.R02) because (as per the logic here: http://www.magiclantern.fm/forum/index.php?topic=5732.0 ) the other files did not have the appropriate hex values at the foot of the files.

Instead of messing around with HxD and editing HEX (not something I think I should have to do on my files), I used HxD's "Extras > File Tools > Concatenate..." feature to concatenate all these files together in order. (I'm sure there is a way to do this on the command line in Windows but it's not something I know how to do off the top of my head.)

After the concatenation, I was able to generate a large series of DNGs as I would expect if my clip had not gone over the 4GB file size limit.

I have searched through this thread and I have not been able to pinpoint any reason why raw2dng doesn't fix this issue by itself. If raw2dng reaches the end of a RAW file and sees it doesn't have the proper footer, why not check to see if maybe there is a .R00 file in the same directory and then .R01 and so on as necessary until the end is reached?
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: condra on February 07, 2015, 06:23:57 PM

Do I need to convert the DNG files with Adobe DNG Converter?
I can already open the DNG files that Raw2DNG spits out...
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: toasty on February 26, 2015, 03:04:18 AM
What is the quickest most convenient way to obtain a sequence of still images (*.png or other lossless format) files from the .mlv file?

Using Ubuntu Linux.

Thanks.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: JanKocbek on February 26, 2015, 09:47:35 AM
'Convert' button grayed out, what could be wrong?

edit: don't drag mov. files with your raw files into it
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: RAW on March 12, 2015, 10:11:30 PM
downloaded mlrawviewer 1.4.3 the pink dot remover having problem with black/dark scene.The pink/blue dots showing up.  need to wait for 2 sec of frames b4 it clean it up.   using 650d. help.

does the pink dot remover work on different video resolution? cuz i have downgraded (960x540)  to have a longer/smoother recording time.  hope this help.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: RAW on March 12, 2015, 10:21:42 PM
downloaded mlrawviewer 1.4.3 the pink dot remover having problem with black/dark scene.The pink/blue dots showing up.  need to wait for 2 sec of frames b4 it clean it up.   using 650d. help.

Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Renovatio on March 13, 2015, 07:21:42 PM
Hi guys, I've a little question to make:how is the workflow on Linux?

Recently a friend of mine show me something about lubuntu and told me a few things on linux kernel operation systems based. The time was not so long but I've gotcha this way could improve my poor pc performance. A few doubts about software, so asking workflow.

On windows right now my raw files become dng with raw2cdng little program, then with after effects and camera raw i open them and make color correction. Render in dnxhd and edit with sony vegas for final render video.

With Linux is just so.. simple ?

I'm afraid above all about color correction and tutorial I can find on YouTube for windows and Mac software.. Such as after effects..
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: revast on March 18, 2015, 11:53:37 AM
see ML RAW-tools for Linux (http://www.magiclantern.fm/forum/index.php?topic=9335.0), I tried to collect all the options available
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: theartofweb on March 31, 2015, 08:52:48 AM
Hi,

    I have RAW files named like this: M29-0249. When I drag and drop it onto raw2dng I get files like 00001.dng and so on. Is there a way to have the dng files named something like M23-0249-00001? It would make my life a lot easier if I did not have to go thru creating folders named like the raw files containing those dng sequences.

Thanks
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Danne on March 31, 2015, 09:30:31 AM
If youre on mac you can use cr2hdr-r or mlrawviewer.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: theartofweb on March 31, 2015, 06:09:18 PM
So basically mlrawviewer and raw2dng do the same jobs. Yet, mlrawviewer gives me M29-0307_DNG_000001 folder name style and M29-0307_000001 file name style which I absolutely prefer!

Thanks
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: vicnaum on April 16, 2015, 03:53:31 PM
That is working with my newly bought 650D flawlessly! Fantastic! Thanks!

Can someone tell me - how can I use raw2dng chroma smoothing somehow together with dual iso?
I've been so long out of forums, and so many info & tools out there... But they all seem not exactly what's needed...
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: keepersdungeon on May 31, 2015, 12:32:03 AM
So basically mlrawviewer and raw2dng do the same jobs. Yet, mlrawviewer gives me M29-0307_DNG_000001 folder name style and M29-0307_000001 file name style which I absolutely prefer!

Thanks
Cool didn't know that. Thanks
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: butchdon on July 05, 2015, 06:13:07 PM
Hi,

I have a Canon 6d with ver. 116 firmware and ML installed, with 32GB 80 Mb/s cards. I shot some raw video with ver. 113 firmware just to try it out. Now I not be able to process them. I downloaded and installed Resolve 11 Lite, but it can't import them. I tried going through this  thread and read through the first two pages to see that a lot of the discussion seemed to be at the developer level. I did not think I could handle 16 more pages.

What I need is a post listing the hardware/software (Windows)needed at minimum to import, edit, and publish the raw video into something usable. I'm a pro photographer on the side so I already have CC Photoshop, Lightroom, tons of plugins and other 3rd party software, so I'm not afraid of working through issues to get what I need. But  I'm reluctant to add two more items to my Adobe CC subscription, unless my videos become so wildly successful, Hollywood starts calling and the $$ start rolling in. I would not mind buying stand alone software if the price is reasonable. Free would be better, yet realizing it may not be as robust as a commercial product.

Thanks

Thanks
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: butchdon on July 06, 2015, 04:29:40 PM
evidently this software is for MAC only. Am I wrong?

Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: mothaibaphoto on July 06, 2015, 05:38:50 PM
@butchdon: Try MlRawViewer (Next thread in Raw Video Postprocessing) or raw2cdng (6-th thread)
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: butchdon on July 06, 2015, 07:12:25 PM
I did try it and that worked, once I hit the W key.

My problem is what do I do with the DNG files? I tried importing them into Resolve 11 Lite, but it crashes. I was getting an OpenCL GPU not installed error, but got that to go away. Now I get a No CUDA Acceleration error. I have Intel HD Graphics 4600 on board video with "dynamic" memory. Could that be the issue?

I have 12 Gb of memory, Intel i5-4440 3.1 GHz CPU, running 64 bit win 8.1.

If I can't run Resolve, what are my options that will not break the bank? Or will I have to upgrade my hardware to really make it all work?

Thanks
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: DeafEyeJedi on July 06, 2015, 09:05:59 PM
You can use AE CC with Smart Import 2... Works really well!
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: mothaibaphoto on July 07, 2015, 09:19:19 AM
@butchdon: Your hardware is perfect for AE, for example, I don't use Resolve, sorry...
AE and Smart Import 2 (http://www.magiclantern.fm/forum/index.php?topic=12422.0) is top quality solution. But you should subscribe for AE. Alternatively, you can:
1. Export MOV from MlRawViewer. Drawback: demosaicing algorithms worse than from Adobe, particularly in handling highlights.
2. Process sequences with Lightroom or Photoshop. Drawback: you possibly encounter flickering.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Scotty6700 on November 20, 2016, 11:27:31 AM
hey guys,
i have some 1920x1080 .mxf footage for wedding,
i've edited them in premiere pro cc 2015.3,then when i exported them in : h264,mp4 and checked both of render at maximum depth and maximum render quality ,some noises are seen in final rendered .mp4 file while the raw footage doesn't have noise like that,
how can i export this project with a quality near the raw footage,i don't have any problem for space and my client doesn't have too,but not so huge like i rendered 1 minute=about 1 Gigabytes for i think i chose nxd 1080 for sequence setting and i cheked match sequence settings for render.

i am searching for an output with quality near raw footage and maximum file size but not bigger the whole raw footage,
i hope you guys understand what i mean,
sorry for my poor english.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Baladeva on February 19, 2017, 07:42:57 AM
how can i do it in windows?I have mlv files and the app tells me that it is not a compatible file.


thank you.
Title: Re: RAW Video Post processing (raw2dng) [Mac/Win/Linux]
Post by: Walter Schulz on February 20, 2017, 06:47:15 AM
Read first post of this thread and use Windows executables on Windows.
Pay special attention to the phrase "drag and drop".
You might want to google for tutorials about "handling command line tools" or switch to tools with GUI instead.