5D2 RAW video Builds 14-Bit

[Hazer]

Just Run the RAW2DNG on my Pro 
yes it look like it, the only thing i don't see is Len info, and iso rating
i just run a quick test on a frame
i was checking the metadata in Adobe Photoshop Elements verion 9
I'm still looking for a alternative to Adobe workflow, without stepping on the
quality.

Thanks for the test.  I've used RawTherapee on Mac with some success -- it's free / open source so you can give that a try.  The nice thing is that is has numerous command line options, so if you're familiar with scripting (or willing to learn) you can setup a workflow that uses RawTherapee automatically without even opening the program.  Of course these will be automated settings so you will not have as much control as if you opened the app and made manual changes to each clip.

What I'm interested in with the Adobe workflow is having access to Picture Styles, both by Canon and also third party.  Some of these are nice starting points for color grade.  When you did the conversion on your Mac Pro, did you see Canon Picture styles (Landscape, Neutral, etc.) in ACR?  They would be under the "Camera Calibration" tab, and there should be a pulldown there which lists them.

If you don't see them, this is where I think the current build may be omitting the metadata that allows ACR to identify the camera properly.  Unless something is wrong with my setup -- which is entirely possible.

[Yoshiyuki Blade]

Thanks for the test.  I've used RawTherapee on Mac with some success -- it's free / open source so you can give that a try.  The nice thing is that is has numerous command line options, so if you're familiar with scripting (or willing to learn) you can setup a workflow that uses RawTherapee automatically without even opening the program.  Of course these will be automated settings so you will not have as much control as if you opened the app and made manual changes to each clip.

I've also been using Raw Therapee for Windows and the CLI is virtually a must for large scale conversions lol. The GUI is so unstable when dealing with thousands of images, I ended up learning how to avoid certain ridiculous crashes by using hotkeys and not touching the batch tab. It seems to crop 4 pixels on each side of the image for some reason too and I haven't been able to find any information on why that is. The great thing about it though is that it can output the raw with no color profile at 1.0 gamma, which is good for it8 calibration.

[reddeercity]

Thanks for the test.  I've used RawTherapee on Mac with some success -- it's free / open source so you can give that a try.  The nice thing is that is has numerous command line options, so if you're familiar with scripting (or willing to learn) you can setup a workflow that uses RawTherapee automatically without even opening the program.  Of course these will be automated settings so you will not have as much control as if you opened the app and made manual changes to each clip.

What I'm interested in with the Adobe workflow is having access to Picture Styles, both by Canon and also third party.  Some of these are nice starting points for color grade.  When you did the conversion on your Mac Pro, did you see Canon Picture styles (Landscape, Neutral, etc.) in ACR?  They would be under the "Camera Calibration" tab, and there should be a pulldown there which lists them.

If you don't see them, this is where I think the current build may be omitting the metadata that allows ACR to identify the camera properly.  Unless something is wrong with my setup -- which is entirely possible.

i have check in to that,there is  Camera Profiles under the Camera Calibration Tab
whether they are Canon profiles i'm not sure, but it dose list the Profiles you would see
on the 5D2.
As listed: Embedded ? this could be third Party
               Adobe Standard
               Camera Faithful
               Camera Landscape
               Camera Neutral
               Camera Portrait
               Camera Standard
               (But no third party profile, Technicolor CineStyle)

i did switch between the profiles and it dose change the image
i will have to Work with this and see if the embedded is the third party
Picture profiles, i'm very use to using  Technicolor CineStyle Profile for H264
but in Raw its total different image , and that what i having problems with
it very robust, Not flat and desaturated like CineStyle.

               

[Africashot]

i have check in to that,there is  Camera Profiles under the Camera Calibration Tab
whether they are Canon profiles i'm not sure, but it dose list the Profiles you would see
on the 5D2.
As listed: Embedded ? this could be third Party
               Adobe Standard
               Camera Faithful
               Camera Landscape
               Camera Neutral
               Camera Portrait
               Camera Standard
               (But no third party profile, Technicolor CineStyle)

i did switch between the profiles and it dose change the image
i will have to Work with this and see if the embedded is the third party
Picture profiles, i'm very use to using  Technicolor CineStyle Profile for H264
but in Raw its total different image , and that what i having problems with
it very robust, Not flat and desaturated like CineStyle.

             

If you like playing with picture styles, you may like this: http://cinescopophilia.com/download-the-vw-158-canon-dslr-picture-styles-picture-profiles/
Some of the film emulations are pretty cool, that said, I have hardly ever used any of them for serious work, the fun is in creating your own look matching each image, especially now with the RAW capabilities!

[Hazer]

i did switch between the profiles and it dose change the image
i will have to Work with this and see if the embedded is the third party
Picture profiles, i'm very use to using  Technicolor CineStyle Profile for H264
but in Raw its total different image , and that what i having problems with
it very robust, Not flat and desaturated like CineStyle.

Yep, those are the Canon picture styles!  That's great.  And you're seeing these using Magic Lantern .dng images that you converted right on the Mac?  If so I wonder what I'm doing wrong.  What build are you using and what version of Raw2dng are you using?

The picture profiles in ACR should look almost exactly the same as they do in the camera.  You just have to make sure there are no other settings applied, such as saturation and sharpness and so on.  You can test this with a CR2 still image from the camera.  When you load it into ACR with a particular picture style, it should look very similar to an in-camera JPEG taken with the same picture style.

If you load in a CR2 that is tagged with Cinestyle, that might load it into ACR and make it available in the drop down menu.  If this works, it should say Cinestyle, not embedded.  You'd probably have to test it.

[andyshon]

Thing move quickly on here, no flies on you chaps.

I had a shoot over the weekend, tides and moons for the bbc natural history unit. Mainly timelapse but they wanted some video too, and I foolishly decided to shoot ML RAW...

It was a little hairy at times, we lost one card full of video, totally corrupt, but that probably had nothing to do with ML. The results are pretty impressive. I've posted a few examples if anyones interested. It has a gritty quality that I'm not sure the NHU will appreciate but I really like, and the performance in some pretty extreme low light conditions was superb. Not sure I'd have got these images on anything else, certainly not on one of their varicams.



http://www.lightandtime.co.uk/media/redrocks/000287.dng



http://www.lightandtime.co.uk/media/redrocks/000000.dng



http://www.lightandtime.co.uk/media/redrocks/000105.dng

So thanks again to y'all, if they use any of the footage I'll donate my fee.

Particular thanks to a.d. who's builds I've been using all along. The new one is blisteringly fast for me but like others on here I am having problems converting longer clips. Getting the old 'aint lv_rec' error, seemingly for anything over 2GB. Been testing at 1880x800, will try more later if needs be, but I'm guessing I'm just being stupid and missing something. Updated raw2dng.app to 0.10. This runs it's own code right, it doesn't call the command line tool? Am I being a twat?

[a.d.]

@andyshon
Last week I watched your vimeo video and I'm impressed!

Does this fixed OSX user problem?
5D2_OSX_raw2dng.zip

In the zip folder is the command line:

1 cd destination_of_the_folder (drag&drop)
Terminal support drag and drop => don't need to type the path of the folder

2 /path_to_app/raw2dng /path_to_raw/JXX.RAW DNG_Name

[andyshon]

@a.d.

Thanks fella. Likewise, loving your work. Your footer fix saved that little film, which in turn helped sell the location to the bbc, so ta!

I'll try this later cheers. We just got two kittens, so this shit just got less interesting... Ha ha

[mvejerslev]

Any idea what has caused me and Hazer to lose EXIF data? Can I upload some samples somewhere?

[a.d.]

Sorry, I need to catch a train...
I suppose you use the wrong raw2dng.

[Hazer]

a.d. -- thanks for the links.  This new command line version solves the exif issue.  The camera model now displays correctly, and picture styles are available in ACR.

Thank you!

[mvejerslev]

Well I have a days worth of shooting, and no matter what raw2dng version I use (I'm on a PC), they come out with no exif data.. Perhaps something went wrong in the recording, although I have no idea what I did different.

[Hazer]

You tried the files a.d. just linked to?  Hm, strange.

One thing you can try, if you really need to get those files processed and nothing else is working:  There is a command-line tool I was experimenting with called exiftool that can copy exif data from one file to another.  I'm pretty sure you can run this on Windows.  But I'm on Mac, where the syntax is:

exiftool  -tagsfromfile sourcefile.dng destinationfile.dng

...where you replace "sourcefile.dng" and "destinationfile.dng" with the names of the files you're copying and to.  What I did here is take a still image with the camera in CR2 format, and used Adobe DNG Converter (a free download from Adobe) to convert this to .dng.  Then, use you that .dng as the source file and copy all those tags to the destination file.

When I did this to a Magic Lantern .dng, it opened up in Adobe Camera Raw and all the picture styles were then available.  The important tag seems to be called "CameraUniqueID", which needs to be set to "Canon 5D Mark II".  you can actually do that specifically with exiftool like this

exiftool -camerauniqueid="Canon 5D Mark II" destinationfile.dng

Anyway a bit convoluted and it would be better to sort out the Raw2Dng process so it works.  But if you need to attempt a rescue of sorts this may work for you.


Edit:  If you just want to do a quick check to see what exif data is even present in your Magic Lantern .dng files, just pick one .dng file and do this:

exiftool -s filename.dng

This will list everything that's there and you should see very quickly if it's an Adobe problem or a Raw2Dng problem.

[vikado]

The most recent build starts a new clip once the 4gb limit has been reached.  You can use terminal or command prompt to stitch them together.  On mac open a terminal window in the folder you have the clips, enter the following

cat [file1name].RAW [file2name].R00> [newfilename].RAW

hit enter.  The file will take a bit of time to compile, so don't try to convert it or move it until it shows the right file size.

Don't have a PC but I've seen comments on that process around as well.

absolutely wonderful news.

[andyshon]

Does this fixed OSX user problem?
5D2_OSX_raw2dng.zip

In the zip folder is the command line:

1 cd destination_of_the_folder (drag&drop)
Terminal support drag and drop => don't need to type the path of the folder

2 /path_to_app/raw2dng /path_to_raw/JXX.RAW DNG_Name

Yep, works for me. Converted a 3GB and a couple of 8GB files no worries. So I'm now looking at a load of shaky, out of focus footage of kittens... Hmm. Nice tones though.

And the .app gui seems to work now too. So it calls the raw2dng command from /usr/bin?

[mvejerslev]

You tried the files a.d. just linked to?

You mean the mac version? I'm on a P.C. but I tried every version of Raw2DNG I have, also known functioning ones and I still get no exif (verified with the excellent exiftool). Thanks for the codes for exiftool - I was looking for that. So if I wanted to batch process a library of dng's, copying exif from one good dng to a bunch of bad ones, what would the syntax be? Thanks again.

[mvejerslev]

Ok, used exiftool GUI to copy the exif data over from some sound files, so the files are saved. Now for some more testing as to the whole reason for the missing exif.

Incidentally, should anyone do the same as me, remember to set the ISO speed correctly for your files - Camera Raw needs info like this for its denoise profiling.

[thearc]

Here is another test shot.
Technicolor Profile Vs 5D RAW
RAW footage sent through raw2dng then photoshop then to resolve.
Regular Canon h264 with technicolor profile sent to resolve for correction as well.

Love it! Thanks guys! Way better dynamic range and even though the RAW is lower res it's way sharper too.

[Hazer]

Ok, used exiftool GUI to copy the exif data over from some sound files, so the files are saved.

Cool glad that worked.  To be clear, this is just a hack, because a lot of the exif in the "good" dng might not match the "bad" one:  resolution, ISO, etc.  But ACR will probably just ignore the data that doesn't fit, or maybe set some sliders incorrectly -- like white balance -- which you will then have to fix.  Hopefully you can figure out what is causing the problem in the first place.

[reddeercity]

Hello all 
i just finish a full card record (29.7 GB 31,934,595,752 bytes) on the May 31st built, for 5d2.
i have a Lexar 1000x 32 Gb card, thats aligned to 32K, i read a lot about people have Read Write
Problems, my average write speed is about 55MB/s read @ about 60 MB/s
But i can Record raw @ 1880x854 24p 64.3 MB/s with no frame drop & 1 buffer Star.
the only thing i have set different from whats been recommended is i set my Photo to JPG Small not  Large no raw.
wheather that makes a different or not im not sure , and i alway record  a few seconds then  stop and start the full card record.
i think that make a big different . I end up with 11,374 frames(7.633 Mintue) @ 2,782 KB in size.
i also been working with the Metadata, and there again i hear pepole having  problems.

in Photoshop i have not see this problem with my tests, but what i have Notice is under the Camera Calibration Tab
there is a Embedded  in Camera Profiles, so i did a test & set the picture profile to Technicolor CineStyle Profile
in camera, did my full card record test and when i enabled the embedded tab, it change to a flat, desaturated image close to
what i would see with CineStyle with H264 this will make Grading Faster.
i did no remenber seeing the Embedded Tab when i used Landscape Profile .

[mino.rimada]

But i can Record raw @ 1880x854 24p 64.3 MB/s with no frame drop & 1 buffer Star.

Great!, that is so great. I shot a video at 1880x756 with a previous build and got an average of 600 frames using a lexar 1000x 16g.
I should try your way..if it works for me too, I will be so happy...I hope that by the end of June there will be a more stable and practical way to shoot raw in the 5Dm2, I'm shooting a short film this month and I'm considering to go RAW, the workflow is so complicated but every minute invested is totally worthy.

[joshmonie]

Hi Guys,

I just want to get it out there that I'm a total newb before I go any further.

Ok, I'm needing some help on installing this all on my 5D2. As far as I'm aware, I download the current stable ML which is v2.3, unzip it and take just the 5D2-212.fir onto the root of my CF. Where do I find the autoexec.bin and ML folder? Can I just use the autoexec.bin and ML folder from a previous build and copy/paste the raw_rec.mo file into where it should be?

I was going to try this but was worried I'd brick the camera so thought I'd get some expert advice first!

Thanks in advance,
Josh

[reddeercity]

Hi Guys,

I just want to get it out there that I'm a total newb before I go any further.

Ok, I'm needing some help on installing this all on my 5D2. As far as I'm aware, I download the current stable ML which is v2.3, unzip it and take just the 5D2-212.fir onto the root of my CF. Where do I find the autoexec.bin and ML folder? Can I just use the autoexec.bin and ML folder from a previous build and copy/paste the raw_rec.mo file into where it should be?

I was going to try this but was worried I'd brick the camera so thought I'd get some expert advice first!

Thanks in advance,
Josh

I don't know if i'm a expert But the best & easist way to in stall ML with the raw record module,
this is what i do to install ML on my 5D2

If any thing go wrong pull the Battery out ASAP!

1-make sure you have Canon Firmware update which is 2.1.2 First
 
2- prepare your Card, make sure  you have a Fast Card i use
    Lexar 1000x 32GB Card , many other ones work but your limited to smaller Frame size recording
    600x cards work  like the Scan disk but will no record continuous @ larger Frame size.
    Downland the EOS Card 1.3  (http://pel.hu/eoscard)  for Windows, this will prepare
    the card for ML, it will make it Bootable & install the latest ML

3-download the May 31st RAW record module on Page 1,
    unzip install the new Autoexec file on the root of card
    and the raw module go in the ML Folder,

3-Start the camera up, press the  delete key takes you to the
   ML window, scroll over to the Module and load by the set key.
   back to the movie tab enable the Raw record tab
   to change setting hit the picture Profile button.

that pretty much it there is a few pit fall but you will work that out

One last thing Go have Fun recording RAW

[Clemens]

Great!, that is so great. I shot a video at 1880x756 with a previous build and got an average of 600 frames using a lexar 1000x 16g.
I should try your way..if it works for me too, I will be so happy...I hope that by the end of June there will be a more stable and practical way to shoot raw in the 5Dm2, I'm shooting a short film this month and I'm considering to go RAW, the workflow is so complicated but every minute invested is totally worthy.

I think if you are planning your shots in a good way it's worth a try. The workflow is not that much more complicated when you directly go with After Effects and then render out to ProRes422 HQ oder ProRes444.
Only thing that I really wish happens is, that they figure out a way to record with 12 Bit.

[joshmonie]

Hi again.

Having followed the instructions, I'm now having another issue. Upon turning the camera on after the ML "success" screen, I'm getting a "scripts dir missing" message. I can then go into ML as normal, load the raw module, etc. but when I turn it on and go into live view, I cannot start raw recording.

Any ideas?