RAW Video: PostProcessing -- Beginners Guide --

[RenatoPhoto]

Updated: October 3rd, 2013

By now you should have a few video raw files to process. 

If not then make sure you read here:

RAW video & ML -- Beginners Guide and FAQ -- READ FIRST
http://www.magiclantern.fm/forum/index.php?topic=5520.0

--------------------------------
THE ORIGINAL: RAW Video Post processing [Mac/Win/Linux]
http://www.magiclantern.fm/forum/index.php?topic=5404.0

--------------------------------

Remember: RAW means that your Picture Profiles have no effect on the final video colors, contrast, sharpness, etc.  All of this should be adjusted in Post-processing.

-------------------------------

Nice Tut for those interested in Davinci Resolve and AE workflow:

--------------------------------


Step 1  EXTRACTING THE DNG FRAMES

The RAW video files cannot currently be processed directly by any video editor.  Each RAW file encapsulates the frames in DNG format so we need to extract these frames first:

1.1 Please download the RAw2dng.exe  application for windows.
HERE: http://acoutts.com/a1ex/raw2dng.exe

Instructions: In Windows just drag and drop the RAW video files to this exe and it will extract the dng frames in the directory where the RAw2dng.exe file is located.

If your card is formatted inside the camera, it will have FAT32 format in which case the RAW files will be split in 4GB chunks.  You will need to join them in the following manner before you process them with Raw2dng.exe

1. Rename the files as  file1.raw  file2.raw .. etc
Use command line:
copy/b file1.raw+file2.raw+.... filenew.raw

or ..

1a. Rename files as file.001, file.002
Download file joiner splitter such as FFSJ v 3.3
http://www.softpedia.com/get/File-managers/File-Splitter-and-Joiner.shtml

2. Extract DNG with latest RAw2dng.exe
here: HERE: http://acoutts.com/a1ex/raw2dng.exe

If you format your card outside the camera to Exfat then the camera will record files larger than 4GB.  These files can be processed with the latest Raw2dng.exe  See link above


Nice new apps for Windows being developed for automatic batch processes here:

RAWanizer - windows batch generation of proxy files from RAW
http://www.magiclantern.fm/forum/index.php?topic=5557.0

RAW2DNG BATCHelor 1.1 Beta
http://www.magiclantern.fm/forum/index.php?topic=5645.msg39772#msg39772



1.1a For MAC users follow instructions here:

raw2dng.app 0.7 Osx GUI development, bugs and updates
HERE: http://www.magiclantern.fm/forum/index.php?topic=5508.0

Other technical details about DNG:

Install Adobe DNG Codec 2.0 (Needed to see thumb images in windows)
http://www.adobe.com/support/downloads/detail.jsp?ftpID=5495

DNG = Digital Negative is a lossless raw image format which was developed by Adobe.
http://en.wikipedia.org/wiki/Digital_Negative

In case you are wondering this is NOT CinemaDNG format
HERE: http://download.macromedia.com/pub/labs/cinemadng/cinemadng_p1_spec_091009.pdf

How to view DNG thumbnails in Windows:

Good review and methods here:
http://photographylife.com/how-to-view-dng-thumbnails-in-windows

I use FastPictureViewer Codec Pack, which is great!
http://www.fastpictureviewer.com/codecs/

A complete library of DNG support here:
http://www.barrypearson.co.uk/articles/dng/products_y7.htm#converters


Step 2 MAKING NECESSARY ADJUSTMENTS TO DNG FRAMES

Now you should have you RAW file broken into its individual frames. 

The next step is make some corrections of White Balance and Tint.  To do this you will need some software that can open the DNG files and allows to make the required adjustments.

The most powerful tool to do this is ACR (Adobe Camera Raw) which is a plug-in for Photoshop.  If you have it then:

2.1. Open all the files at once, if you computer does not have enough memory or speed you will have to note all of the settings and apply to smaller sets of images.   

2.2. On the left side of ACR you will see a thumbnail of all of the opened images.   Click on the first image on the left side and then click on Select All

2.3. Set the White Balance according to the scene, this is a subjective step but you can have ACR set it for you if you in Auto mode or simply choose from the options in the drop down menu which mostly represents the scene in which the video was recorded.  For example you can set it to Daylight, and this will adjust the WB=5500 and the Tint=10

2.4. Make sure that your settings in the Basic configuration are not moved from zero adjustment.  Read carefully Technical Note 1 below step 2.6

2.5. On the bottom left side of ACR click on the Save Images button where you need to make some selections as to where to save the files, the type of image format you want to save, and the quality of such format.  The  selection will depend on what type of software you have available for converting these images into a video file, and  the desired quality of the final process.  If you are not sure, select jpeg format and  the highest quality to make sure we do not lose any details i.e. Quality=12

2.6. Finally click on Save and then on Done.  The ACR will work converting these DNG frames to your selected directory.

TECHINICAL NOTE 1 :
In ACR 7.1 do not abuse the highlight, shadow recovery sliders because this might bring flickering from frame to frame. 
In ACR 6.7 do not move the fill light and recovery sliders.

If you experience flickering then start backing off on ACR (BASIC) adjustment sliders.  I have successfully used the sliders in ACR 7.1 Tone Curve for highlight and shadow control without experiencing any flickering.

Follow this issue here:
Preventing Color/Luma Shifting When Processing DNGs in Adobe Camera RAW
http://www.magiclantern.fm/forum/index.php?topic=5710.0

Other software that can open and adjust DNGs:

1. UFRAW
2. Lightroom 4.2
3. Dcraw
4...


Step 3  MAKING A VIDEO OUT OF THE NEW FRAMES (JPG, TIFF, DNG)

Most video production software will allow to import the frames and turn them into a video.   Usually it is referred as opening an image sequence and it allows to import all of the frames at once into an editable video.

One of the most powerful and free video editors is called VirtualDub which is able to import the frames, do some fancy editing, and save it.

Originally it could only save in avi format but now there are plug-ins can be added to save in other formats.

Download here: www.virtualdub.org/index

Here is quick how to:

Open VirtualDub
File Open
Select Image Sequence from selection type
Click on Video and Frame Rate
Change Frame Rate to (fps): 29.97 Click OK (Must match your RAW video settings)
Again Click on Video and then Compression
Select new Codec Like CineForm (Must install this one)
Click OK
Click File Save AS AVI

To install CineForm codec you must install GoPro Cineform Studio.  This codec will produce great results but if you want the full version which allows for greater color depth and 2K to 4K frame sizes you will have to purchase their Cineform Studio Premium program.

More details about this here:
Cineform Studio (free) 1080p transcoding issues:
http://www.magiclantern.fm/forum/index.php?topic=2695.0

That is it! 

Now you have RAW video converted to and AVI file.  Note that you may not be able to play this in a slow PC.  This AVI file will be very large and require very high bitrates which may not be compatible with older PCs.  For this reason I have mention above that you should install a compression codec to reduce the size of the file and make it a bit more readable in slow hard drives.

The beauty of this process is that it is conserving much of the original data contained in the RAW files.  If you compress this video to other formats like mpeg1 you will lose all of the great details provided by RAW video recording.

This is where all of the more advanced video post processing methods flourish in an effort to pass all of the great detail, and color depth to the viewers.  At this point one has to consider who are the viewers and the platforms available for viewing so the optimal choice of software and workflow will vary greatly.

Here you can find more discussions regarding more advanced workflows:

RAW Video Post processing [Mac/Win/Linux]
http://www.magiclantern.fm/forum/index.php?topic=5404.0


-------------------

Please help us keep our Forum Organized.  We have very specific areas for certain topics so try to conform so that everybody benefits.

If you have a question, don't post it! First use the search function on the forum (near the top left) and search for the key word.  If you have made a reasonable effort and cannot find an answer then please post your question in the appropriate forum.

[Peter]

Adobe just released version 8.1 of Adobe Camera Raw
http://blogs.adobe.com/lightroomjournal/2013/06/camera-raw-8-1-and-dng-converter-8-1-now-available.html

[guentergunter]

You can also open the DNG-files as a sequence in After Effects. Doing this, also opens up the Adobe Camera Raw, so that you can make all the adjustments.

When editing you video in Adobe Premiere, I suggest exporting the DNG-sequence to PSD-files. For these support 16 bit and Premiere can natively open them as psd-Sequence (in 16bit!).

[eatstoomuchjam]

If you want to edit on a more modest workstation and/or don't want to worry about the space/disk performance needs of raw video, it also works nicely to import your raw sequence into AE, adjust levels/color balance/etc as needed, and then export to 12-bit DNxHD/ProRes.  Once you've graded and done shadow/highlight recovery, the raw files become less necessary.

[guentergunter]

I think that's up to your opinion.
In my case, I do want to retain as much RAW as possible throughout the hole post process.

By the way: With ProRes, the picture becomes less sharp and it is only compatible to MAC.

[rawmania]

You can also open the DNG-files as a sequence in After Effects. Doing this, also opens up the Adobe Camera Raw, so that you can make all the adjustments.

When editing you video in Adobe Premiere, I suggest exporting the DNG-sequence to PSD-files. For these support 16 bit and Premiere can natively open them as psd-Sequence (in 16bit!).

You've  noticed the difference between editing in Premiere from psd 16 bit once and jpeg 8 bit on the other hand if you tried?
The is no Prores export option from AE in Windows?
Regards

[smartmart]

Hi, I'm just started with this all new and impressive feature - RAW recording. I own 5d Mark II. Tried yesterday some shooting and now I have 14 .RAW files. Joined them through a copy/b command, got one file (11.67 GB), but RAw2dng extract just the frames from the file 001, but not all the dngs from all joined .RAW files. Is this possible? Because I obviously want to import All the frames in ONE sequence. Now I thought I could extract those .RAW files to different folders, then batch rename so that I have continuous dng file numbering (i.e, from 00001 till 015000). But having to extract one joined file would be very nice, much less work to do.

Also I must admit, all my .RAW files are smaller than the 4 GB limit.

--------------------------------


Step 1  EXTRACTING THE DNG FRAMES

The RAW video files cannot currently be processed directly by any video editor.  Each RAW file encapsulates the frames in DNG format so we need to extract these frames first:

1.1 Please download the RAw2dng.exe  application for windows.
HERE: https://bitbucket.org/hudson/magic-lantern/downloads

Instructions: In Windows just drag and drop the RAW video files to this exe and it will extract the dng frames in the directory where the RAw2dng.exe file is located.

If your card is formatted inside the camera, it will have FAT32 format in which case the RAW files will be split in 4GB chunks.  You will need to join them in the following manner before you process them with Raw2dng.exe

1. Rename the files as  file1.raw  file2.raw .. etc
Use command line:
copy/b file1.raw+file2.raw+.... filenew.raw

or ..

1a. Rename files as file.001, file.002
Download file joiner splitter such as FFSJ v 3.3
http://www.softpedia.com/get/File-managers/File-Splitter-and-Joiner.shtml

[paulforte]

Is it possible to use After Effects as a online/offline workflow. For example.

DNG> ACR> AE> Export to low res proxy> Edit in FCP> Return to AE for final CC> Export high res master

[menoc]

For those of you who are new to Magic Lantern and are not very adept at the command line, here's a tutorial on how to merge ML RAW files without having to use the command line. This tutorial was created for the Mac, but it will also work in Windows. You'll need a hex editor and of course, a DNG extractor Utility like raw2dng, RAWMagic or raw2cdng.

HEX EDITORS:
Hex Fiend Hex editor for Macintosh
http://ridiculousfish.com/hexfiend/
HxD Hex editor for Windows
http://mh-nexus.de/en/hxd/

RAW UTILITY:
RAWMagic
http://www.mediafire.com/download/ht3ydb4s254ka5z/RAWMagic-beta7a.dmg
raw2dng
http://www.magiclantern.fm/forum/index.php?topic=5404.0

[Danne]

Thanks for the vid. Last time I used rawmagic this process worked by simply adding the .r00 files together with the .raw file. Don,t work in raw2dng yet however.

[tin2tin]

For Windows there is also http://eyeframe.tk

How to:
- Drag and drop RAW files into the gui.
- Spanned files will be merged automatically.
- The conversion to dng first and then to tiff will run automatically.
- The imported clips will show up in the list.
- Preview video by double left click thumbnail.
- Hit convert.
- Select the video format you want(+ proxies if needed).
- Click 'okay' and the batch conversion will start.

Btw.:
- Time code and reel name can be edited by right click in colum.
- The dcraw settings(dng>tiff) can be found and edited in "Documents/EyeFrame Data/dcraw.ini"

[shuntian]

I'm still having problem of join the file larger than 4g. I'm using the mk2, I get a raw file a r00 and a r01. I'm using the mac version raw2dng, how can I join them together.

[shuntian]

I just saw the reply #8, it just solve the problem perfectly, thank for sharing this.

[Videop]

Hi!

I found a cheap DNG converter. Free 30 day trial, then something like 20-30USD with 30 day money back if not happy.

Convert DNG to TIFF, JPG and more.

http://www.contenta-converter.com/download.php?language=en&keywords=converter%20file%20dng&gclid=COWArbWAiLgCFfR6cAodbSoALQ

[rawmania]

@ videop

How was the color after converting with Contenta converter?
What workflow used?

[Videop]

Hard to say, only did a quick test.

[Stevenbaril]

Having an issue with Raw2dng.exe. It won't open properly. Once I click on it the window briefly opens then immediately closes. I've tried running as admin, doesn't make a difference. Any suggestions?

[tin2tin]

Drag and drop a raw file onto it and it will create the dng files in the same folder.

[Stevenbaril]

Thanks Tin2Tin!

[kshoban]

Ok, so I have a t3i. I'm recording at 832x480 resolution with a 16:9 aspect ratio. This is not HD as expected. How do I uprez it to 1280x720? I am using adobe premiere cs5. Is this where I use cineform studio? Will cineform uprez the video for me? Thanks!

[raywillus]

Can anyone explain why the raw2dng app generates a mov. file  after i drag my Raw file to the app? Is that mov. file a raw video file or it is the normal video file?

[ToniX]

......One of the most powerful and free video editors is called VirtualDub which is able to import the frames, do some fancy editing, and save it....................Open VirtualDub/File Open/Select Image Sequence from selection type/Click on Video and Frame Rate

I have installed  Vdub  but I can't see any option in any menu that allow me to import an imagines sequence as it should.

Can you tell me where is it or how to enable it?

thank you.

[dogmydog]

Does anybody know how a workflow that uses SpeedGrade instead of DaVinci Resolve?

Thanks!

[Videop]

Can anyone explain why the raw2dng app generates a mov. file  after i drag my Raw file to the app? Is that mov. file a raw video file or it is the normal video file?

You can chose to output DNG and if you wish even a Prores file. The .mov is Prores.

Strange thing for me is my first try resulted in playable Prores with a thumbnail and after that I get a .mov file without thumbnail and not playable.

 

[BartvR]

......One of the most powerful and free video editors is called VirtualDub which is able to import the frames, do some fancy editing, and save it....................Open VirtualDub/File Open/Select Image Sequence from selection type/Click on Video and Frame Rate

I have installed  Vdub  but I can't see any option in any menu that allow me to import an imagines sequence as it should.

Can you tell me where is it or how to enable it?

thank you.

File -> Open video file -> Files of type -> Set to Image Sequence