I've been playing around with Luke Neumann's sample DNG files and as it turns out GoPro Cineform Studio Premium and Professional already support the conversion of these into CineForm RAW.
Use the same CinemaDNG workflow that we posted for the Blackmagic Cinema Camera here:
http://support.cineform.com/entries/22175187-Blackmagic-Cinema-Camera-CinemaDNG-to-CineForm-RAW-Workflow-Windows-
In the conversion settings, use 'Protune' or the 'optimize for 12 or 13 stops' option for the encode curve and then use 'Protune' for the decode curve.
After conversion use the following settings (provided by GoPro Senior Director of Software Engineering, David Newman) for the WORKSPACE and WHITE BALANCE panels:
(http://software.gopro.com/other/5DRAWVIDEO.png)
Finally, apply the 'ProtuneLook' in the LOOK (CUBE/3D LUT) panel.
This will give you a decent starting point with a file ready to take into an NLE or other application (without exporting i should add... just take this same file into your other apps and all the settings applied here will go along with it).
This is currently only available in the Windows version of GoPro CineForm Studio Premium or Professional, which you can download from here:
www.cineform.com/downloads/
Cheers!
Jake Segraves
Technical Support Manager
GoPro...Be a HERO.
Great seeing you here, Jake ! :)
Great to be here! ;)
This is a great video to describe what are the benefits (with BMCC, that - as Jake said - has the same workflow as raw files in 5D mkIII)
I should be asleep, instead I'm experiment with Canon 5D ML RAW in GoPro CineForm Studio. Here is more setup, conversion and correction using GoPro CineForm Studio Premium.
Import files (look RAW):
(https://pbs.twimg.com/media/BKiGoN8CQAA4_g_.jpg:large)
My advanced conversion settings:
(https://pbs.twimg.com/media/BKiG0GLCUAARWpV.png:large)
Add Color Matrix and White balance to the CineForm RAW clip:
(https://pbs.twimg.com/media/BKiHA5XCEAAMzZf.jpg:large)
Add a Preview LUT (ProtuneLook):
(https://pbs.twimg.com/media/BKiHTzbCMAAnYUe.jpg:large)
Sounds good, what about a tool mlraw2cfraw or even import the raw file into Cineform Studio?
I didn't find any of the demosaic options to be as good as the ACR one.
Advanced Detail 3 seem to work best but it's not on the same level.
Also going through cineform raw I didn't find any options to recover highlights on the same level as ACR/LR.
Any feedback appreciated.
Ok.. here are my thoughts about this workflow...
I have been playing around with CineForm Studio Premium for the last few days. This has potential to be a much faster workflow than any other methods I have seen discussed. One of the advantages that jumps out at me immediately is that importing a large number is DNGs into your product is very quick (almost immediate) and this allows you to play the raw sequence in your video player right away. Other workflows take much longer to get to this point so this is a good way to be able to play through all your footage to see what you have shot, discard any bad sequences and not waste any time converting sequences you won't be using. It's also pretty amazing how the raw workflow extends right into Premiere... when you import the converted Cineform Raw files into Premiere and still can update the files live from Cineform Studio and have those changes reflected immediately in your Premiere timeline... very nice.
However, there are a couple of things that give me pause about this workflow. I think a large number of people will ending up liking the Adobe Camera Raw interface for getting their image into a very usable state quickly. Besides very intuitive exposure controls, ACR includes time saving features such as decent noise reduction. This could eliminate the need for more render intensive noise reduction plugins later. In my test footage of a fairly bright window in a dim room, ACR let me bring the highlights down and adjust overall exposure much more easily than in Cineform Studio. Other than a white balance correction, minor tint shift, and a bit of noise reduction, it only took minutes to get my footage looking beautiful. In contrast, using Cineform Studio, I found that I needed to play around for quite a bit longer to get the the point where the footage was acceptable, and it still didn't look as good as what I had achieved using ACR. Highlight preservation was especially problematic. I'm not a professional colorist by any means but that is the point... I feel this experience I described will be repeated by many like me. Another observation...I had initially opened my DNG sequence in ACR, made my adjustments, synchronized these changes to all my files, and then imported those files into Cineform Studio. I was expecting (or hoping rather) that my changes might have been preserved in Cineform Studio but alas they weren't. I'm sure there is a reason for this but it would be useful to be able to do this. Cineform Studio still to me has the advantage of non-destructive raw editing which extends to the NLE, small edit-friendly files, not to mention the ability to preview your footage very quickly in your video player before any conversion or rendering takes place.
Just some food for thought...
What I've found so far:
- highlight restore implementation is much weaker than ACR. Cineform Studio & FirstLight does not provide controls to do it. Also if you load the cineform file in AE you cannot "dig" into the highlights.
In ACR it's amazing how much can be restored.
- Cineform doesn't implement any color noise reduction (or did not found it ?). I've found instances where there are artifacts after debayering through cineform.
- Debayering through ACR gives better details than Cineform, but here I may be wrong. Have to check more.
- there is a small bug when using regional settings in Windows (comma instead of point). 23.976 fps files will transcode insted to a monstrous 23976 fps file. I have to revert to English(US) region or reinterpret in AE.
- did not find a way to save colour matrix in Studio as a preset.
Any feedback appreciated,
Cristian
hi! Do you think that it is possible to convert ML dng raw to resolve format (cinema dng)?
Later update.
Color noise reduction is killing this workflow. Also it appears that debayer filter enhances also the color noise.
Here is a link to a dng file from canon raw movie. Also included screenshots for acr and cineform studio interfaces. Check the metalic bars, but noise is present in the entire image.
https://docs.google.com/file/d/0B3ncQl4hSqlnWklaQTliU09Jdzg/edit?usp=sharing (https://docs.google.com/file/d/0B3ncQl4hSqlnWklaQTliU09Jdzg/edit?usp=sharing)
Any help appreciated
Cristian
Quote from: tihon on May 18, 2013, 05:16:25 PM
hi! Do you think that it is possible to convert ML dng raw to resolve format (cinema dng)?
If you transcode to cineform raw quicktime format it opens in Davinci.
Cineform studio... I dunno. The codec is good but always broken with sony vegas. In v12, preview at full best size causes black screen on some clips.
Codec is 10 bit so I guess I'm losing color now converting? I've yet to find an intermediate that is better in terms of file size and quality so I live with the bugs. :(
Quote from: Kuky on May 18, 2013, 05:20:28 PM
If you transcode to cineform raw quicktime format it opens in Davinci.
But i think camera settings in resolve (exposition, wb,tint) will be dissabled as in dng files, Or not?
Quote from: 1% on May 18, 2013, 05:23:38 PM
Codec is 10 bit so I guess I'm losing color now converting? I've yet to find an intermediate that is better in terms of file size and quality so I live with the bugs. :(
Nope, cineform is able to work both in RAW and RGB/YUV space. CineformRAW it's ... well... RAW. They just use a proprietary compression to reduce size, but keep it raw.
The ideea behind this is great but I find the implementation not be as good as ACR. If some of this problems are solved could be the fastest way to preview/edit/colour correct keeping the media in raw format.
Quote from: Peter on May 18, 2013, 10:23:36 AM
Sounds good, what about a tool mlraw2cfraw or even import the raw file into Cineform Studio?
I haven't looked into details for want we could customize, create custom tool for transcoding would be straight forward. I was just finding the current tool is working well with the DNG sample file someone shared (thank you).
Quote from: Kuky on May 18, 2013, 11:05:34 AM
I didn't find any of the demosaic options to be as good as the ACR one.
Advanced Detail 3 seem to work best but it's not on the same level.
Also going through cineform raw I didn't find any options to recover highlights on the same level as ACR/LR.
Any feedback appreciated.
We do have a highlight recovery control, the slider in "Workspace" is called "SAT. CLIP PT".
I totally agree the demosaic in ACR is awesome, we are on the look out for an open source equivalent to add to our demosaic options. We like having a good range of filters as playback speed is also an import factor. You could edit with Advanced Detail 1 or 2 (I don't like 3) then switch to some (future) compute intensive demosaic for export.
Quote from: Kuky on May 18, 2013, 04:12:00 PM
- there is a small bug when using regional settings in Windows (comma instead of point). 23.976 fps files will transcode insted to a monstrous 23976 fps file. I have to revert to English(US) region or reinterpret in AE.
- did not find a way to save colour matrix in Studio as a preset.
I'm not sure how to reproduce that speed bug.
We never thought there would be a need to save the color matrix in a preset, as these are the first CinemaDNGs I've seen that didn't have a color matrix included.
Quote from: Kuky on May 18, 2013, 05:19:14 PM
Color noise reduction is killing this workflow. Also it appears that debayer filter enhances also the color noise.
That is why I wouldn't use Advance Detail 3 filters, but some like it, and it was used on parts of Slumdog Millionaire. We didn't not yet add a denoise stage to the demosaic.
Note: you are showing the old FirstLight tool which hasn't been updated in two years. The development has moved to the GoPro CineForm Studio application. They are many that still prefer the FirstLight interface, but slowly you can do more in the new tool.
Quote from: 1% on May 18, 2013, 05:23:38 PM
Cineform studio... I dunno. The codec is good but always broken with sony vegas. In v12, preview at full best size causes black screen on some clips.
Codec is 10 bit so I guess I'm losing color now converting? I've yet to find an intermediate that is better in terms of file size and quality so I live with the bugs. :(
There is not been any CineForm issue with Vegas, but yes, they has been a Vegas issue will some third party codecs (not just ours.) I surprised to hear the Vegas black frame bug still plagues anyone.
CineForm RAW hasn't been 10-bit since the SI-2K in 2007. Yet even in 2007, 10-bit log is good for 14-bit linear, it is still a dominate finishing format with 10-bit DPX files today. CineForm RAW is currently 12-bit log, and with VC-5 (our SMTPE efforts to standard CineForm) we extend this for crazy superHDR cameras of the future.
David, Neoscene is 10bit tough right? I could use CF Pro with 4:4:4 so debayering with adoberaw and then using cineform normal not raw would be an option?
Quote from: AndreasK on May 18, 2013, 06:27:40 PM
David, Neoscene is 10bit tough right? I could use CF Pro with 4:4:4 so debayering with adoberaw and then using cineform normal not raw would be an option?
NeoScene is the old 4:2:2 10-bit tool. GoPro Studio Premium is 4:4:4 and RAW and 12-bit.
Newest studio gives the option of 4:4:4/709/RGB in AE where is raw?
Black frame bug is from vegas using the CFHD decoder DLL and not VFW codecs. So some version of vegas work, some don't... I know its probably buggy for other things too, buggy in general.
Quote from: 1% on May 18, 2013, 06:39:02 PM
Newest studio gives the option of 4:4:4/709/RGB in AE where is raw?
Black frame bug is from vegas using the CFHD decoder DLL and not VFW codecs. So some version of vegas work, some don't... I know its probably buggy for other things too, buggy in general.
AE is only 444, so we have to develop the RAW to 444 via the parameters you set in Studio.
Yes, Vegas seem to be forever a little buggy, it is a shame as we have such a very gooding work relationship with their engineering team. While I like Adobe Premiere also, Sony Vegas natively implement our codec SDK, and this is cool.
That native use of the codec is what got me...
I like vegas more than premiere.. seems straight forward for edits/audio/etc. But always there is some cancer.
i've been using cineform studio premium for several weeks now for a short film we've been working on.
converting 5d mark ii h264 to 10-bit cineform. its great that it gives me much better flexibility in doing aggressive color grading.
and loving the white balance picker. saving me tons of time.
but downsampling from raw to 10-bit cineform isn't as good as raw to dng sequence using after effects.
ACR has a much better control than cineform. and i dont think cineform even has highlight recovery slider.
cineform is great for upconverting from 8-bit to 10-bit color space though.
just my two cents.
Can you tell me please: if i render my DNG file in After Effects to cineform 4-4-4 : it will be raw? or just compressed 12 bit 4-4-4?
Quote from: tihon on May 19, 2013, 12:10:15 AM
Can you tell me please: if i render my DNG file in After Effects to cineform 4-4-4 : it will be raw? or just compressed 12 bit 4-4-4?
i dont know what you mean by raw, but cineform is a losssless (visually lossless) codec.
the only difference is that you'll be working on a 12-bit color space vs 14-bit.
the converted file will be cineform raw.
any changes you make with your footage will be done on cineform studio premium and the beautiful thing is that the meta data is imbedded within the file itself so you can reset or change the meta data within CFSP and the changes will be immediate in video editing program.
Quote from: vikado on May 18, 2013, 11:36:20 PM
i've been using cineform studio premium for several weeks now for a short film we've been working on.
converting 5d mark ii h264 to 10-bit cineform. its great that it gives me much better flexibility in doing aggressive color grading.
and loving the white balance picker. saving me tons of time.
but downsampling from raw to 10-bit cineform isn't as good as raw to dng sequence using after effects.
ACR has a much better control than cineform. and i dont think cineform even has highlight recovery slider.
cineform is great for upconverting from 8-bit to 10-bit color space though.
just my two cents.
Cineform Raw is 12-bit log, which means it can store the same amount of data as 16-bit linear, more than the 14-bit linear that comes from the Canon DSLR sensors. Cineform doesn't need a highlight recovery slider because it automatically restores all the highlights contained in the raw file. For proof look at the default ACR settings vs Protune with no lut. Or set the output curve to Cineon 95-685 or Custom Log 400 to gain back all dynamic range from the file.
This workflow looks very promising.
I'm finding it impossible to replicate those settings in the WHITE BALANCE panel. I can type the value for TEMP and TINT, and then as soon as I type the value for RED the TEMP value changes, same goes for BLUE. Is there some way to actually enter the values from that screenshot without other values changing along the way?
AHA! It seems one can enter all of those specified values for WHITE BALANCE and not have them cancel/modify each other if one specifies the Protune LUT in the LOOK panel (the LUT is actually named "CStyleLUT" in that panel's drop-down menu and Protune in its thumbnail preview, or at least clicking the "Protune" thumbnail applies the "CStyleLUT", which is confusing enough that I'd call it a bug).
So set that LUT first, then change the WHITE BALANCE values, I guess.
So after entering all of those values and finally getting that clip set up properly, is there a way to apply those settings across a set of imported clips? I've got approximately 10 30-second clips that were shot in one session, they're all going to get the same settings. Selecting all of them and then clicking on those panels and setting the settings only seems to have stuck for the first clip in the list. The inability to apply to a batch of clips these input and output curves, demosaic type, white balance, and LUT settings, well that's going to be a bit of a pain.
And beware, double-clicking the "Protune" thumbnail to set the LUT to CStyleLUT also pops the COLOR MATRIX back to 'SOURCE'. So after setting the LUT one has to go back up to the COLOR MATRIX panel and set it back to CUSTOM.
I've given up on setting the RED, GREEN, and BLUE settings in WHITE BALANCE. Setting TEMP and TINT to the values in the screenshot seems to leave the clip looking all right, with RED at 1.7330, GREEN at 1, and BLUE at 1.3179. Changing the individual colors to match the screenshot has had some weird effects on the value of TEMP and TINT and the overall look of the image.
To be honest, when I let the color matrix setting go back to 'SOURCE', but keep all the other settings specified in the thread, the resulting image looks a lot more like the sort of flat log curve that I'm used to working with. How were the set of color matrix settings for that screenshot derived, what was the intent?
This workflow has promise, but the software seems to want to fight me. Maybe I'll get used to it and figure out the workflow.
Quote from: iaremrsir on May 20, 2013, 02:05:48 AM
Cineform Raw is 12-bit log, which means it can store the same amount of data as 16-bit linear, more than the 14-bit linear that comes from the Canon DSLR sensors. Cineform doesn't need a highlight recovery slider because it automatically restores all the highlights contained in the raw file. For proof look at the default ACR settings vs Protune with no lut. Or set the output curve to Cineon 95-685 or Custom Log 400 to gain back all dynamic range from the file.
i've read to leave it to protunes.
what are the pros cons on protunes vs cineon?
coming from 8-bit to 10bit?
Quote from: zachnfine on May 20, 2013, 08:48:42 AM
So after entering all of those values and finally getting that clip set up properly, is there a way to apply those settings across a set of imported clips?
in the "step 2" or "EDIT" panel. load the clip you want to copy the settings from as well as all the clips you want to paste the settings to.
first you select the source clip and selecting "Copy" or "ctrl + C" from the Edit menu and then selecting your destination clips and selecting "Paste" or "ctrl + v"
Quote from: vikado on May 20, 2013, 06:31:33 PM
i've read to leave it to protunes.
what are the pros cons on protunes vs cineon?
coming from 8-bit to 10bit?
You don't have to leave it on Protune. Protune set as the output curve will give you a "finished" look. When working in 10-bit and 12-bit, especially for color grading, cineon (logarithmic gamma curve), compresses all of the data into a range where each stop of light in the file gets around the same number of code words (bits of data for detail). This is why film is scanned to cineon log files. The entire dynamic range of the film frame can be captured and stored in the log file. Same principle applies to raw. That's why ARRI, Red, Sony, Blackmagic Design, etc. have there own versions of cineon log designed specifically for the sensors of their cameras. If you compare Alexa Log-C, RedLogFilm, and BMD Film (I actually did a post on BMCUser about using Cineon 95-685 to get extremely close to this curve with CF) , you'll see that they are all fairly similar looking with differences mainly in color science. With Cineform we can take a raw file and basically transform it to a mathematically correct cineon file, Log-C file, S-Log, or whichever curve you choose file. However, when compositing or editing, I'd work with the Protune output curve so you don't have to create proxies and you can stay with the CF raw file until you actually have to render an intermediate composite or something like that.
For up-converting the 8-bit to 10-bit, it depends largely on which profile you shot with. If you used Cinestyle, there is a preset in CF Studio to set the i/o curve to CStyle. For any other profile I'd say leave it at Video Gamma because in the H.264 file there's only about 6-8 stops of usable dynamic range depending on the ISO setting you shot with.
Quote from: iaremrsir on May 20, 2013, 07:55:18 PM
You don't have to leave it on Protune. Protune set as the output curve will give you a "finished" look. When working in 10-bit and 12-bit, especially for color grading, cineon (logarithmic gamma curve), compresses the all of the data into a range where each stop of light in the file gets around the same number of code words (bits of data for detail). This is why film is scanned to cineon log files. The entire dynamic range of the film frame can be captured and stored in the log file. Same principle applies to raw. That's why ARRI, Red, Sony, Blackmagic Design, etc. have there own versions of cineon log designed specifically for the sensors of their cameras. If you compare Alexa Log-C, RedLogFilm, and BMD Film (I actually did a post on BMCUser about using Cineon 95-685 to get extremely close to this curve with CF) , you'll see that they are all fairly similar looking with differences mainly in color science. With Cineform we can take a raw file and basically transform it to a mathematically correct cineon file, Log-C file, S-Log, or whichever curve you choose file. However, when compositing or editing, I'd work with the Protune output curve so you don't have to create proxies and you can stay with the CF raw file until you actually have to render an intermediate composite or something like that.
For up-converting the 8-bit to 10-bit, it depends largely on which profile you shot with. If you used Cinestyle, there is a preset in CF Studio to set the i/o curve to CStyle. For any other profile I'd say leave it at Video Gamma because in the H.264 file there's only about 6-8 stops of usable dynamic range depending on the ISO setting you shot with.
very useful information. thanks!
so basically if you're shooting raw, use cineon.
if you're upconverting, leave it at video gamma.
as for picture styles, i've been using visioncolor and lightform
Quote from: tihon on May 19, 2013, 12:10:15 AM
Can you tell me please: if i render my DNG file in After Effects to cineform 4-4-4 : it will be raw? or just compressed 12 bit 4-4-4?
It'll be 444 if it's coming out of After Effects, not raw. The only way to get CF Raw at the moment is through Studio Premium/Pro or the command line tool dpx2cf that's included with CF.
Quote from: vikado on May 20, 2013, 09:46:06 PM
very useful information. thanks!
so basically if you're shooting raw, use cineon.
if you're upconverting, leave it at video gamma.
as for picture styles, i've been using visioncolor and lightform
Glad to help! Not sure if I mentioned this earlier, but for raw you could use the CF databases to have a preset look to help for editing and a cineon database for grading after the edit is finished. And yeah since you're using regular video gamma profiles, stick to video gamma.
I'm working on a Windows shell tool to convert .RAW directly to CineForm RAW .AVI or .MOV files. My current test tool is working and converting to CineForm RAW at over 100 frames per second (old 2.6GHz i7 920 -- probably disk limited -- converting directly from CF Card will be as fast as you can read.) The problem is I haven't yet set up my 5D3, and I only have one .RAW clip to test with. Please provide links to hosted .RAW files for my testing before I post the tool.
Any links for .RAW? I don't want to post the tool without testing a range of clips.
I've got 3 short shots uploaded here. (https://drive.google.com/folderview?id=0B5vzb1fFuV73UW91LUY4OHBrMlE&usp=sharing)
These are Tungsten, 1920x1080, ISO 500. Unfortunately, I've nuked all of my daylight RAWs I shot earlier today. Let me know if you'd like to see something specifically.
Any chance we'll see a shell tool for Mac to take RAWs to Cineform RAW MOVs? Thanks!
Quote from: platu on May 18, 2013, 11:39:55 AM
Ok.. here are my thoughts about this workflow...
I have been playing around with CineForm Studio Premium for the last few days. This has potential to be a much faster workflow than any other methods I have seen discussed. One of the advantages that jumps out at me immediately is that importing a large number is DNGs into your product is very quick (almost immediate) and this allows you to play the raw sequence in your video player right away. Other workflows take much longer to get to this point so this is a good way to be able to play through all your footage to see what you have shot, discard any bad sequences and not waste any time converting sequences you won't be using. It's also pretty amazing how the raw workflow extends right into Premiere... when you import the converted Cineform Raw files into Premiere and still can update the files live from Cineform Studio and have those changes reflected immediately in your Premiere timeline... very nice.
However, there are a couple of things that give me pause about this workflow. I think a large number of people will ending up liking the Adobe Camera Raw interface for getting their image into a very usable state quickly. Besides very intuitive exposure controls, ACR includes time saving features such as decent noise reduction. This could eliminate the need for more render intensive noise reduction plugins later. In my test footage of a fairly bright window in a dim room, ACR let me bring the highlights down and adjust overall exposure much more easily than in Cineform Studio. Other than a white balance correction, minor tint shift, and a bit of noise reduction, it only took minutes to get my footage looking beautiful. In contrast, using Cineform Studio, I found that I needed to play around for quite a bit longer to get the the point where the footage was acceptable, and it still didn't look as good as what I had achieved using ACR. Highlight preservation was especially problematic. I'm not a professional colorist by any means but that is the point... I feel this experience I described will be repeated by many like me. Another observation...I had initially opened my DNG sequence in ACR, made my adjustments, synchronized these changes to all my files, and then imported those files into Cineform Studio. I was expecting (or hoping rather) that my changes might have been preserved in Cineform Studio but alas they weren't. I'm sure there is a reason for this but it would be useful to be able to do this. Cineform Studio still to me has the advantage of non-destructive raw editing which extends to the NLE, small edit-friendly files, not to mention the ability to preview your footage very quickly in your video player before any conversion or rendering takes place.
Just some food for thought...
Agree 100% with everything you write!
Playing around with this workflow. Although the controls are not very intuituve from my starting point as primarily a timelapse/still photographer and Adobe user, I realize diving into the world of RAW video editing is, well, a brand new world, that may require a lot training and reading/learning to be done.
However, I am having a really annoying issue with Cineform Studio (1.3.2.170). And I know this is something should be adressed with Cineform, but I just wanted to know if anyone experience the same, and or has a suggestions for solution/workaround. The program (CFSP) crashes the second I try to set framerate to 23.976 (GoProImport has stopped working). I can choose 24fps, or 15fps, but when I select 23.976 it crashes! It even crashes before I add the files to the import, so it has nothing to do with the DNG files, but the program it self it seem. In adition the greenish and flat look of the 5D3 files are far from the way my eyes perceived the scene and the way the files should look (as in ACR after a slight adjustment of hue of -40)
And why is my footage detected as default to 29.970fps in Cinefrom Studio ?? I can not find anywhere to define default values.
I am on a very powerful computer, so should not be the reason: win7x64 / Adobe CS6 / dual Xenon 2387 3.1Ghz 32cores / 128GBRAM / 30TB SAS RAID 30 (1.8GByte/sec sustained). All Windows updates installed.
Edit: I have found that a much better approach for me is starting with ACR, doing the initial adjustments, and a very good noise reduction if needed, then importing to AE as 23.976 without the annoying problems with CFSP crashing, and most importantly I get the colors CORRECT as they should be, as my eyes perceived the scene, much quicker, with better and more intuitive control of lifting shadows and lowering highlights. And from here I can export the file in Cineform Filmscan1 if I choose. And then I can again open the exported AVI file in Cineform Firstlight again and have the same Cineform controls as when importing with CFSP, should I want to, with the AVI file updating in my NLE / PP / AE.
So by doing it this way; DNG->ACR->AE->Cineform RAW AVI (Filmscan etc.), the colors get a MUCH better and more correct starting point than if I start the import in CFSP, as well as the controls in ACR (in my opinion) are better and more intuitive.
Quote from: mnteddy on May 26, 2013, 08:44:45 AM
I've got 3 short shots uploaded here. (https://drive.google.com/folderview?id=0B5vzb1fFuV73UW91LUY4OHBrMlE&usp=sharing)
These are Tungsten, 1920x1080, ISO 500. Unfortunately, I've nuked all of my daylight RAWs I shot earlier today. Let me know if you'd like to see something specifically.
Any chance we'll see a shell tool for Mac to take RAWs to Cineform RAW MOVs? Thanks!
Thank you, these samples are prefect and working without any code changes.
Awesome! Can't wait to try out your tool! :D
Here is version 1.0.
To run this tool you will need the Windows verison for GoPro CineForm Studio Premium or Professional installed -- either activated or working with a full 15-day trial (only need Premium, Pro is for 3D.) The trials are downloaded from here: http://cineform.com/downloads
Direct download for RAW2GPCF: http://miscdata.com/ML/RAW2GPCF.zip
Unzip are install here: C:\Program Files (x86)\CineForm\Tools
To run:
open a shell, and RAW2GPCF at the prompt
M:\cameras\CanonMagicLantern>RAW2GPCF
Usage: raw2gpcf inputfile.raw output.mov|avi [switches]
-c - use Cineon curve (black 95, white 685)
-cbX- use Cineon curve with 'X' black point (default 95.)
-cwX- use Cineon curve with 'X' white point (default 685.)
-dX - use debayer type 1-softest thru 4-sharpest (default auto)
-fX - the framerate override for AVI|MOV media.
-gX - use gamma curve
-iX - in point, start processing on frame X (default 0, disables audio.)
-lX - use log curve of power X (GoPro Protune.)
-oX - out point, e.g. -o10 would precess 0 thru 9 (default infinite.)
-qX - the encoding quality for AVI|MOV (default 4, range 1-5.)
-sX - skip every 'x' frames (default 0).
-(x1,y1,x2,y2) - window source Top Left (x,y) to Bottom Right (x2,y2)
usage: raw2gpcf M00000004.raw New5.avi (using current paths)
raw2gpcf V:\Conversions\newRAW.AVI E:\M0000052.RAW -l400 -d2 -q5
RAW2GPCF version: 1.00, (c)2013 GoPro.
I was going to add batch conversion, but I thought I should get some feedback first.
The current tool extracts the RAW XYZ colormatrix data, and converts that into CineForm Active Metadata as the "Source" color matrix, and the white balance gains will auto correct for the sensor's sensitivity differences. The highlight recovery (which many users can't find) is defaulted to 0.5 (seems to correct many highlight issues.) The tool is very fast as no demosaic is required to produce CineForm RAW files. If you convert from your Compact Flash source, the conversion will be at read speed with very little CPU usage (a on set ultrabook could do it.)
For a fast decode speed, I defaulted to be using the automode demosaicing which is a fast debayer (crappy bilinear) when decoding to 8-bit (QuickTime, Window Media Player) but reverts to a nicer image when there 10-bit or greater decodes requested (AE and Premiere float modes, etc.) You can set the demosaic in CineForm Studio Premium after conversion, just as you can with all the RAW parameters (the whole point of RAW.) You can also default the demosaic to any setting you like (-dX switch) again this is not baked.
Here are mnteddy sample clips (thank you) encoded with the defaults (Filmscan 1, RAW, auto debayer.) These are 10:1 compressed (images have lots of shallow depth of field which helps CineForm compression.) http://miscdata.com/ML/M0SamplesQ4.zip I also repeated these highly compressable images at Filmscan 2, yielding 6:1. http://miscdata.com/ML/M0SamplesQ5.zip I did these both in MOV files so users from both platforms can try them out.
Quote from: Northernlight on May 26, 2013, 01:43:42 PM
So by doing it this way; DNG->ACR->AE->Cineform RAW AVI (Filmscan etc.), the colors get a MUCH better and more correct starting point than if I start the import in CFSP, as well as the controls in ACR (in my opinion) are better and more intuitive.
Not CF Raw. The best you can get out of AE is a debayered 4:4:4 file at the moment.
Quote from: iaremrsir on May 27, 2013, 02:33:44 AM
Not CF Raw. The best you can get out of AE is a debayered 4:4:4 file at the moment.
Yes, the resulting images will have baked color. Not a bad thing if you have all of the information you need to do your finish, but RAW in my opinion requires you to be able to restore back to everything the sensor read, giving you the greatest flexibility.
I don't consider applying the demosaic as a necesary part of RAW. Consider the Protune CAM_RAW mode on GoPro HERO3 Black Edition (something I worked on,) that is about as raw as you can get in an H.264 file. All the demosaic is applied in camera, but color matrix and white balance are applied in post, just like classic uncompressed RAW.
If there is enough interest, I will enable YUV or 444 development, without baking the color. The only reason you might like 4:2:2 RAW, is that could run on the freely distributed GoPro CineForm Studio, just like Protune CAM_RAW works for GoPro users today.
Hey Dan, is there a mac solution in the works?
ACR/AE is rendering exposure shifts http://www.magiclantern.fm/forum/index.php?topic=5710.msg41052;topicseen#msg41052
ATM there is no real workable mac post workflow, we're drowning here. :'(
Unbaked is good.
Quote from: DANewman on May 27, 2013, 04:08:54 AM
If there is enough interest, I will enable YUV or 444 development, without baking the color. The only reason you might like 4:2:2 RAW, is that could run on the freely distributed GoPro CineForm Studio, just like Protune CAM_RAW works for GoPro users today.
That would be beyond awesome! If you need a crash test dummy, you got one ;D
In the meantime, I need feedback on the current tool.
For the cineon curve option, is it optimized for compression like Protune is? I tried conversion with the same file twice: once on default, and once on cineon. The cineon file looked terrible. The detail was gone and the compression artifacts were pretty bad. Changing the debayer method didn't help either. It just exaggerated the artifacts. The default settings worked fine though. However it still seems like I was able to get more detail by using raw2dng, then using CF Studio to convert the DNG files. Other than that, conversion was quite snappy as always and the auto white balance worked like a charm. Also the .5 for sat clip point seems to be the sweet spot of sorts. For every ML Raw clip I've brought in, I had to have it at or near .5, otherwise I got hot pink or lime green highlights.
"... seems like I was able to get more detail by using raw2dng, then using CF Studio to convert the DNG files. "
This doesn't make sense, same process internally. Likely subtle metadata differences -- you can copy settings from one clip to another.
Cineon was left over from the tool I ripped the code from, I never really tested that path. The Cineon curve was not designed for compression, it is film density emulation.
Happy to give it a try David, do you still need some .raw files?
Did you take a look at the BMDfilm output curve? Do you think it would be possible to get that as output so that I can grade in Resolve with a flat image (like the bmdfilm)?
Already found a bug..,kinda :)
I always record in 1928 width because we have 2 lines garbage on the left. recording 1928 and cutting 7 pixel on the left and 1 pixel on the right gives exactly the same FOV as a CR2 still. But if I convert this raw with the converter and want to play it back the program crashes. Be it the quicktime player or gopro studio or windows media player when trying an avi.
I tried to crop it via parameter but seem to do it wrong? How are the parameters? is -(7,0,1926,959) correct?
This code has too few legal range checks, so you can tell it so do things the codec can't support. First you can't use odd numbers for the windowing as the image is CFA RAW, color pattern is an even number repeat -- technically I could switch the bayer phase to handle odd numbers -- currently values these are rounded. Next the codec likes horizontal encodes of a 16 multiple and vertical of 2. So this should work -(8,0,1928,960) -- except I introduced a bug it doesn't. Fix coming.
I did have a look at the BMDFilm output curve, and it is a little odd. It is likely has BMD sensor calibration stuff in it, the curve we use for Protune and others, mathematically models the mapping of linear to log and back again, the BMD curve don't fit our models. PanaLog, S-Log, C-Log, Protune etc. are all nice mathematical equations, but not BMD. If we were to use that curve natively, either a lot of code changes to the codec, or an approximation of the curve would produce (mild) color errors in the matrix, white balance and saturation (all the linear light operations.)
I now have my 5D3 running RAW, although with tiny images until I get new media.
New build of RAW2GPCF (RAW to GoPro CineForm)
Direct download for RAW2GPCF: http://miscdata.com/ML/RAW2GPCF.zip (I replaced the odd link)
The old v1.00 build is now here http://miscdata.com/ML/RAW2GPCFv100.zip
Unzip are install here: C:\Program Files (x86)\CineForm\Tools
To run:
open a shell, and RAW2GPCF at the prompt
M:\cameras\CanonMagicLantern>raw2gpcf
Usage: raw2gpcf inputfile.raw output.mov|avi [switches]
-422- Encode to YUV 4:2:2 (default RAW.)
-444- Encode to RGB 4:4:4 (default RAW.)
-c - use Cineon curve (black 95, white 685)
-cbX- use Cineon curve with 'X' black point (default 95.)
-cwX- use Cineon curve with 'X' white point (default 685.)
-dX - use debayer type 1-softest thru 4-sharpest (default auto)
-fX - the framerate override for AVI|MOV media.
-gX - use gamma curve
-iX - in point, start processing on frame X (default 0, disables audio.)
-lX - use log curve of power X (GoPro Protune.)
-oX - out point, e.g. -o10 would precess 0 thru 9 (default infinite.)
-qX - the encoding quality for AVI|MOV (default 4, range 1-5.)
-sX - skip every 'x' frames (default 0).
-(x1,y1,x2,y2) - window source Top Left (x,y) to Bottom Right (x2,y2)
examples: raw2gpcf M00000004.raw NewRAW5.avi
raw2gpcf new422.AVI M0000005.RAW -d1 -q3 -422
raw2gpcf new444.AVI M0000005.RAW -d3 -q5 -444
raw2gpcf V:\Conversions\newRAW.AVI E:\M0000052.RAW -l400 -d2 -q5
RAW2GPCF version: 1.01 May 27, '13, (c)2013 GoPro.
Feature additions:
-444 : performs the demosaic and encodes to 4:4:4 (requires a GoPro Studio Premium license.)
-422 : performs the demosaic and encodes to 4:2:2 (requires a free install of GoPro CineForm Studio -- Premium not needed.) and is limited to 1920x1080 under the free encoder license.
Both these new modes are still RAW for color processing, only the demosaic is baked. While the 4:2:2 option will be popular for cost-conscious, the image development controls for RAW are more flexible in the Premium level GUI.
Also 4:2:2 and 4:4:4 modes are slower than the default RAW transcodes. RAW has the greatest flexibililty as I do want to add more demoasicing options in the future.
Bugs:
This fixes a crash in the windowing switch.
Gotta love the support companies are already showing for ML raw, much appreciated.
Canon might want to take note...
This is hardly an official company supported tool, just something I did on the side. Yet GoPro does support transcoding other cameras, so I used the code base for the Vision Research Phantom transcoder as the .cine files are very like .RAW. Don't expect this tool to become part of a regular software distribution. Yet notice how this tool requires you to have GoPro software, so it is a win-win I feel. ;)
Ahh, gotcha. Good point though, our raw workflow options (especially mac) are quite limited.
Hi David,
I tested the RAW2GPCF tool it work great thanks and its mighty fast, are their plans for a GUI and batch converting or the ultimate would be drag your .RAW files/folder into GPCF studio tweak then output CF Raw. I know you said that we should not expect this as part of the regular studio package but if it were included in the PRO $299 version I would buy that in a flash and other people might feel the same as it would save immensely on time spent on getting to the edit phase. I see this RAW development as something which is just going to explode in the next few months the workflow will become integral in its expansion and popularity and whoever get the workflow down and simple will have a fairly large consumer following i believe. My two cents.
Quote from: DANewman on May 27, 2013, 06:12:59 AM
In the meantime, I need feedback on the current tool.
The first thing that I would say is that it is very FAST. I spent practically a full day working through the typical Raw2dng -> ACR -> After Effects workflow on 100+ GB of raw footage I shot this weekend. It was enough to really turn me off to the adobe workflow. I decided to spend a bit more time with Cineform Studio once I noticed you posted a new conversion tool. Initially, I did find ACR to be easier to work with and still do, but the time savings of using your new tool makes it worth the extra effort in my opinion. I mean, let's face it, using RAW2GPCF is magnitudes faster if your goal is to get your footage into your NLE quickly, not to mention the benefits and flexibility you gain by keeping everything as raw files. The ACR/After Effects workflow may be fine for those doing personal projects with limited amounts of footage to process, but completely inadequate for most paid work if time is at all a factor. Batch processing is desperately needed still but I realize that you are looking for feedback first.
Quote from: DANewman on May 27, 2013, 06:20:35 PM
So this should work -(8,0,1928,960) -- except I introduced a bug it doesn't. Fix coming.
Works perfectly, thanks alot! Imho this is the way to film with the 5D3 (selecting 2048 it jumps to 1928) because otherwise you'll always have the 2 black rows on the left.
Quote from: DANewman on May 27, 2013, 06:20:35 PMI did have a look at the BMDFilm output curve, and it is a little odd. It is likely has BMD sensor calibration stuff in it, the curve we use for Protune and others, mathematically models the mapping of linear to log and back again, the BMD curve don't fit our models.
Oh ok didn't know that. I found a thread on bmcuser where somebody found a simulation using the Cineon 95 curve. I did some further testing and pushing with some frames of mine and found a pretty matching preset. BMDfilm compresses/lifts the shadows a little bit more but otherwise it fit's good imho. (Here's the link with the BMCC comparsion: http://www.bmcuser.com/showthread.php?2306-Cineform-Raw-BMD-Film-Curve&p=65225&viewfull=1#post65225)
Attached a screenshot of GoProCineformStudio with Protune curve, one with my simulated bmdfilm and one with the bmd to rec.709 lut applied. Yes I'm digging more into the shadows on the bmdfilm like grade and yes whitebalance probably has to get adjusted it was just a quick apply with no correction and my settings were originally for the bmcc.
(http://www.typo3-blog.com/m/5da.jpg)
(http://www.typo3-blog.com/m/5db.jpg)
(http://www.typo3-blog.com/m/5dc.jpg)
BTW I just opened the image for comparsion in Photoshop. As nice as Cineform is, the Adobe Denoiser does a great job on the bookshelf when lifting the shadows ...
This is a great option for fast-turnaround projects: if it runs at over 100 frames per second, it's basically just like converting the .raw file to a stream of .dng files! But smaller and with proper playback!
So, with something like Advanced Detail 1 debayering, how does detail look compared to the original H.264 video? CineForm RAW is clearly softer than an uncompressed DNG opened in ACR/AE (not because of the compression, but because of the awesome debayering that ACR uses, which CFR seemingly can't match). It should be somewhere between that soft ugly thing we used to love but don't want anymore, and the sparp, clean beauty of ACR
http://www.similaar.com/foto/blackmagic-workflow/index.html
Take a look at my comparsion pics Samuel. It may not be as sharp as ACR but no way it's as mushy as the H264 :)
https://plus.google.com/u/0/b/111134722005606605004/111134722005606605004/posts/TE4uJBWZU5m
As the de-mosaic is not baked, I happy to hear suggestions on for alternative algorithms to add, making the CineForm RAW results even better. Currently you can use CineForm RAW (as .MOV) directly in Resolve, so you aren't limit to our de-mosaics today.
Is Resolve doing the demosaic not cineform? So no matter what I select in GoPro Studio Resolve handels it? Or does this behaviour change since I use your patched dll?
Quote from: DANewman on May 28, 2013, 06:51:54 PM
As the de-mosaic is not baked, I happy to hear suggestions on for alternative algorithms to add, making the CineForm RAW results even better. Currently you can use CineForm RAW (as .MOV) directly in Resolve, so you aren't limit to our de-mosaics today.
I'm afraid I don't have any particular method to suggest. The ones I've coded myself were relatively basic ones that would not improve over what Resolve already has. Now, if somebody finds out what ACR is doing...
Quote from: AndreasK on May 28, 2013, 11:00:10 AM
Take a look at my comparsion pics Samuel. It may not be as sharp as ACR but no way it's as mushy as the H264 :)
https://plus.google.com/u/0/b/111134722005606605004/111134722005606605004/posts/TE4uJBWZU5m
edit: I'm dumb and what I wrote here was wrong
Quote from: AndreasK on May 28, 2013, 06:58:47 PM
Is Resolve doing the demosaic not cineform? So no matter what I select in GoPro Studio Resolve handels it? Or does this behaviour change since I use your patched dll?
The patch you are running also switches the de-mosaic. As the demosaic happens before color corrections, it is not possible for Resolve to do the de-mosaic and CineForm to the first color pass. For the out of the box experience, Resolve does both the demosaic and color from CineForm RAW files. The best solution for all would be to add a current generation de-mosaic filter to the CineForm decoder. CF Advance Detail 1, was award winning only 4 years ago, so we have some work to do.
ehm Samuel wait, I did not run cineform on that test. I did CR2 (still image), RAW (via ACR) and H264. To be honest I don't remember the sharpening settings but I think it was just the default 25 with radius 1.0 - For the CR2 I used resize sharper in photoshop to resize to HD resolution.
Isn't CF advance detail 2 the best option for good sharpnes with no excessive artifacts David?
Ahh I forgot. Ok let's assume I run the version with no patch, how can I set whitebalance and basis settings in Resolve to get rid of that green cast?
Quote from: AndreasK on May 28, 2013, 10:21:54 PM
Ahh I forgot. Ok let's assume I run the version with no patch, how can I set whitebalance and basis settings in Resolve to get rid of that green cast?
I don't know Resolve at all. The white balance values in Studio are linear gains that should be applied as linear channel gains in Resolve (same values should work.) The color matrix then calibrates the sensor to Rec709/sRGB), but I'm not sure that Resolve supports a manually entered color matrix.
@DANewman Really great utility!!! Thanks a lot! I've tried the "raw2gpcf -444- -c -q5" with and without the cineform curve (MOV files for Resolve). The most information I get is with the -c command (without it the highlights are completely blown out). It's fast and seems quite reliable but color noise is worse than in ACR (debayer process I guess?) and the "highlights recovery" of the curve creates some posterization.
Nevertheless, thanks again and I (we) would love to have a GUI and, specially, a batch tool.
EDIT: Could you explain the log curve options? I mean the -lX command. I've no idea what value to use or what the effects are but I would like to try the Protune curve with this tool.
I have updated my batch tool to run raw2gpcf as an option. Have a look at http://www.magiclantern.fm/forum/index.php?topic=5557.0 (http://www.magiclantern.fm/forum/index.php?topic=5557.0)
I have a question for @DANewman, how are you writing the console output? I have tried several ways to capture the console output but failed. I want to continually read while the processing is going on but somehow the output is not released until the process is finished. I have no problems with all other tools I'm using: dcraw, raw2dng, ffmpeg etc. Is there something you could change with it so the output will be more friendly to catch?
Otherwise this is really good for fast generation of video files. Thanks.
Quote from: AndreasK on May 28, 2013, 10:20:40 PM
ehm Samuel wait, I did not run cineform on that test. I did CR2 (still image), RAW (via ACR) and H264. To be honest I don't remember the sharpening settings but I think it was just the default 25 with radius 1.0 - For the CR2 I used resize sharper in photoshop to resize to HD resolution.
Isn't CF advance detail 2 the best option for good sharpnes with no excessive artifacts David?
ooops, I don't know how I read that, I thought I was seeing H.264 vs CFR vs RAW
I guess I was surprised to see such a big difference between the downsampled CR2 and the video RAW, and I just assumed it would be something intermediate, like CFR.
I made a similar comparison back when this was not public yet, and in my memory results were really close. Looking at them again, well, closer than H.264, but not quite the same:
http://www.similaar.com/aarchive/800_st.jpg
http://www.similaar.com/aarchive/800_lv.jpg
http://www.similaar.com/aarchive/3200_st.jpg
http://www.similaar.com/aarchive/3200_lv.jpg
Quote from: sergiocamara93 on May 28, 2013, 11:03:43 PM
@DANewman Really great utility!!! Thanks a lot! I've tried the "raw2gpcf -444- -c -q5" with and without the cineform curve (MOV files for Resolve). The most information I get is with the -c command (without it the highlights are completely blown out). It's fast and seems quite reliable but color noise is worse than in ACR (debayer process I guess?) and the "highlights recovery" of the curve creates some posterization.
...
I mean the -lX command. I've no idea what value to use or what the effects are but I would like to try the Protune curve with this tool.
I expect you are have issues with Cineon as an encoding curve, it is not the optimal choice as it is not very efficient for using the available codewords. Hightlight are not blown out, some are just above 1.0, in RAW they can be restored. Open Studio and try reducting the gain or exposure, all the highlights are still there. You can encode Protune, and decode to Cineon if you like, that is better than encoding Cineon.
-l90 -- SI-2K curve
-l400 -- Protune 12 stop optimized
-l900 -- Protune 13 stop optimized
While the default Protune log curve and -l400 curve both store 13+ stops, it is just a matter of emphasis, and how much do you need to dig into the shadows at the mild expense of midtones and highlights.
Quote from: DANewman on May 29, 2013, 06:08:32 AM
-l90 -- SI-2K curve
-l400 -- Protune 12 stop optimized
-l900 -- Protune 13 stop optimized
While the default and -l400 both store 13+ stops, it is just a matter of emphasis, and how much you need to dig into the shadows, are the mild expense of midtones and highlights.
Thanks, David, I'll try those. I was using cineon since it's the only one which is not giving me the pink highlights after recovery (with 28th Build in 5D Mark III). I input the files directly into Resolve so I guess I cannot take advantage of the protune-encoding/cineon(or other options)-decoding since Resolve ignores them.
I think Resolve reads cineform metadata in a different way, I've tried to input the R G B color matrix you posted but colors were off so I'll try the log options and see how it goes. Let me know if you need any specific feedback/tests for the utility, happy to help.
Quote from: marten on May 29, 2013, 12:18:21 AM
I have updated my batch tool to run raw2gpcf as an option. Have a look at http://www.magiclantern.fm/forum/index.php?topic=5557.0 (http://www.magiclantern.fm/forum/index.php?topic=5557.0)
I have a question for @DANewman, how are you writing the console output? I have tried several ways to capture the console output but failed. I want to continually read while the processing is going on but somehow the output is not released until the process is finished. I have no problems with all other tools I'm using: dcraw, raw2dng, ffmpeg etc. Is there something you could change with it so the output will be more friendly to catch?
Otherwise this is really good for fast generation of video files. Thanks.
Weird the console output will not flush without a '\n', and as the progress indicator is just a '.' per frame, you aren;t getting any flush output. There is a workaround add the '-v' switch.
You will get output like this:
M:\cameras\CanonMagicLantern>raw2gpcf M0000000.RAW aa.mov -v
found 8 cpus
1920x1080 53frames 23.976fps
export frame 0
export frame 1
export frame 2
export frame 3
export frame 4
export frame 5
export frame 6
export frame 7
...
Quote from: sergiocamara93 on May 29, 2013, 06:33:14 AM
Thanks, David, I'll try those. I was using cineon since it's the only one which is not giving me the pink highlights after recovery (with 28th Build in 5D Mark III). I input the files directly into Resolve so I guess I cannot take advantage of the protune-encoding/cineon(or other options)-decoding since Resolve ignores them.
I think Resolve reads cineform metadata in a different way, I've tried to input the R G B color matrix you posted but colors were off so I'll try the log options and see how it goes. Let me know if you need any specific feedback/tests for the utility, happy to help.
As you using Resolve, you will not be using the CineForm highlight recovery, use the equivalent in Resolve. The pink highlights are part of the RAW image, as green channel clips first. This is not an artifact the log curve you select, is might be harder to see with Cineon (on your 2.2 gamma monitor) but it is there just as strong. The log curve we use is designed to distribute the codewords more evenly throughout the useable stops.
You mean that ACR and Cineon "neutralize" the magenta cast somehow by balancing the channels? That's interesting to know, I though it was a metadata handling error. I'll have to try and balance the channels then for this workflow.
Yes that pink highlight restauration is ACR magic.
@Samuel: I was surprised by the difference too, the BMC is definately sharper in that test. But keep in mind that those are very very fine lines and small fonts so in real world the .RAW is just fine
Ok, did some more digging with the patched and unpatched version for Resolve.
Remember the green cast on the unpatched version David? It's apparently a BMCC only problem. The 5D-CF-RAW files open just fine in Resolve in the unpatched version. They open with very little contrast I expect this to be the decoding curve I set when converting. Unfortunately there is no way to set the whitebalance as Studio changes are completely ignored BUT the whitebalance seems right, are you reading it from the .RAW file? It looks ok whereas the DNG in ACR always have that magenta shift?
Ok now for the quality comparsion. Resolve works lovely :)
As it opened very flat I added some contrast, thats all I did in Resolve, if you compare it to the ACR I opened with sharpening 0 and noisereduction 0 they almost look absolutely identical! I then installed the patch, chose ADV Detail 2 and reexported (with no contrast adjustements) from Resolve. Too bad there is no way to set the whitebalance/decoding curve for Resolve. Is there maybe a way to modify the .mov file instead of doing just metadata via Studio?
(http://www.typo3-blog.com/m/sharp1_acr.jpg)
(http://www.typo3-blog.com/m/sharp1_resolve.jpg)
(http://www.typo3-blog.com/m/sharp1_cin.jpg)
And well the BMCC images are completely of, but wrong topic for this forum :)
OK, so Resolve does a great job too, great to know! Thanks for the test.
The cineform sample has some color artifacts; was it debayered by the cineform software, or with the Resolve algorithm? If by CineForm, which debayering method did you use? The one I like best is Advanced Detail 1 (coincidentally, the one DANewman said was winning awards only 4 years ago), but if there's a way to use CineForm RAW compression and get an image that is as good as that Resolve sample, that's what I want!
Quote from: DANewman on May 29, 2013, 06:39:55 AM
Weird the console output will not flush without a '\n', and as the progress indicator is just a '.' per frame, you aren;t getting any flush output. There is a workaround add the '-v' switch.
I did try it now with -v and the output on the console is exactly as you describe.
BUT the behavior is still the same when I try to catch the output. I'm not an expert on this, but could it be that on windows you need to end each line with an '\r\n", it seems like the \n only gives you a new line in the console but it will not flush until a \r. Would it be possible to output an \r\n when the -v flag is used?
It's really fast, love it!
Quote from: iaremrsir on May 27, 2013, 02:33:44 AM
Not CF Raw. The best you can get out of AE is a debayered 4:4:4 file at the moment.
Ok, but at some point the footage need to be demosaiced, and from the look of it, ACR gives you results which are in a completely different class than what GPCF currently is capable of.
Quote from: Samuel H on May 29, 2013, 10:17:58 AM
The cineform sample has some color artifacts; was it debayered by the cineform software, or with the Resolve algorithm? If by CineForm, which debayering method did you use?
It's the same .mov! Resolve normally totally ignores the Cineform Studio processing and reads pure raw data. With that you get this flat but high quality image. But there is no way to change output curve or white balance.
The 3rd picture is with David's Resolve patch. What that patch does is force Resolve to act like any other windows application meaning get the debayered processed image from CineformStudio. All metadata settings in Cineform are shown this way and I used advanced detail 2 as debayer algorithm.
So the best quality would be to go via Resolve unfortunately I don't know how to change whitebalance/curve with that. That's my question to David, is there a way to edit the .mov because there IS some wb-information stored in the file why else would it be fairly accurate and not totally of ;)
^ THAT. IS. AWESOME.
So, the image you labelled Resolve comes from a MOV file that is at least 5 times smaller than the stream of DNG files, and it plays back exactly how fast?
Finally a reason to get my paws on Resolve... At least to get an idea of what it can do, and what those limitations are (WB and curves).
I understand you can recode the DNGs as a CF-RAW mov file, edit them in Premiere (with real-time playback, choosing a low quality debayering method if needed), then export the whole project to be color-corrected and rendered in Resolve.
Seeing those results, this would be my preferred workflow for anything that doesn't have VFX.
Yep, just download it and see for yourself :)
As long as you export only 1920x1080 and don't need the Resolve-NR it's free.
Too bad the raw control is not implemented with cineform. With DNG you can set exposure/wb/shift, with R3D you can set a lot in Resolve directly :(
Really hoping to find a way to change wb/curve, I wonder how David's converter set's the WB from the .RAW
the problem may not be how David's converter sets the WB, but whether Resolve reads that metadata or just sets a default value
I just played with my BMC files once more. I have to transfer the RED/GREEN/BLUE values from the whitebalance panel in Cineform Studio to resolve to get a clean BMCC file.
The 5D-CF file loads just fine in Resolve, I have no idea why, David may know. And about the quality of that BMCC-debayer I'll continue in the other forum ;)
Quote from: marten on May 29, 2013, 11:11:51 AM
I did try it now with -v and the output on the console is exactly as you describe. BUT the behavior is still the same when I try to catch the output. I'm not an expert on this, but could it be that on windows you need to end each line with an '\r\n", it seems like the \n only gives you a new line in the console but it will not flush until a \r. Would it be possible to output an \r\n when the -v flag is used?
It's really fast, love it!
'/r' didn't work. Calling fflush() didn't work. No idea why the output can't captured, all standard printf()s. It seem I need 4K output before anything to a redirected tool.
David
Quote from: Samuel H on May 29, 2013, 01:30:15 PM
the problem may not be how David's converter sets the WB, but whether Resolve reads that metadata or just sets a default value
Currently I don't believe Resolve reads any CineForm metadata, which is why there are these color differences.
Yep but you do write something I guess? Why else is the 5D-CF file correct and BMCC totally green?
Quote from: AndreasK on May 30, 2013, 02:21:51 PM
Yep but you do write something I guess? Why else is the 5D-CF file correct and BMCC totally green?
No. Same data for both cameras, so this is a puzzle.
Yikes! :) But why is your CF whitebalance right and the raw2dng opened in ACR totally wrong?
Is there some camera wb-data transfered in your process? Maybe I should try set the wrong WB in camera and see if that wrong get's copied to CF.
In the end .RAW file there is a single color matrix field, but not an as shot white balance. The colormatrix is the weird XYZ to RAW mapping (reversed for our needs.) I reverse this matrix, exact the camera calibration RGB sensor range differences (green tints) which I use as white balance, then generate a pure camera (after calibration RGB Gains) RAW to D50 sRGB matrix, which drives the saturation. All looks pretty after that.
Does this have a1ex's banding fix?
Hello guys, I like RAW2GPCF, unfortunately it shows a lot of dots, then says it is done, but no output file is generated.
What are the requirements for it to work?
I have the Codec installed.
Also cineform changed my 24FPS to 23.976, have to see if there is a parameter to set it. Ok found that. Also cineon curve seems to take away some moire.
Quote from: fatpig on May 31, 2013, 05:17:11 PM
Hello guys, I like RAW2GPCF, unfortunately it shows a lot of dots, then says it is done, but no output file is generated.
What are the requirements for it to work?
I have the Codec installed.
If you request RAW or 444 and don't have a license to CineForm Studio Premium it might do that. Try -422 to confirm you get output, as that doesn't require more than the free license.
So default is raw? My 4:4:4 vids were a bit washed out. I'll try to rebatch for that.. compression is great as always.
444 appears to work, RAW makes 70MB/s files that have a lot of color problems.
ffmbc lets people on Windows encode in ProRes 422 and 4444. You should slip that as an option into your code, so people can choose between CineForm or ProRes.
Can the pink highlights be fixed?
Here's an example from my most recent transcode...
(http://www.eoshd.com/uploads/M0000069.proxy.jpg)
Update - Resolve doesn't have the pink highlights problem. Only Premiere and Quicktime Player.
Something is def, wrong with the color. I get multi colored outlines on my footage (kind like from af_raw) that aren't in the tiffs or DNG. Also less noticeable doing studio conversions.
The pink in the highlights is controlled by the "Sat. Clip Point" slider, in the Workspace area of CineForm Studio Premium. The camera's green channel clip first, so you need to do some highlight reconstruction or dynamaic range extension, this is what Sat. Clip Point" does, it is just poorly named.
This is an example of my prob:
(http://s14.postimg.org/t6vuw7std/M30_1221_proxy_avi_snapshot_00_02_2013_06_02_12.jpg) (http://postimage.org/)
Frame (http://postimage.org/app.php)
1%,
That looks like a bilinear demosaicing filter artifact. Choose a different filter, try a -d2 switch or set you desired in Studio in the Workspace control field.
I'll try D2, I'm usually batching.. why is raw such low rate/size compared to 444? I thought raw > 444?
d2 identical, d3, d4 as well.
Try it without the batching, it seems not all the switches are taking effect.
RAW is one chroma value per pixel, and through demosaic can be 4:4:4.
4:2:2 is an average of two chroma values per pixel (YU or YV) requires a baked demosaic.
4:4:4 is three chroma values (R, G & B) requires a baked demosaic.
RAW is a natural (lossless) 3:1 compression over 4:4:4. So compressing RAW CFA Bayer is the most efficient way to store any single sensor image.
Command line only:
(http://s22.postimg.org/mtwhm6zml/DNG.jpg) (http://postimg.org/image/mtwhm6zml/)
(http://s16.postimg.org/te2b9mvn5/blah_avi_snapshot_00_00_2013_06_02_15_29_01.jpg) (http://postimg.org/image/te2b9mvn5/)
Could you provide the source or a DNG frame. Thanks
Hello!
Any advice would be greatly appreciated..
The software has created the files fantastically quickly - both .avi and .mov versions
But both Resolve and Adobe AE and Premiere die right after they open the file.. I can see the first frame and software dies.
Both Resolve and Adobe die in the same way.. so looks like codec issue. I have updated Quicktime tothe latest, reinstalled Go Pro cineform studio.. nothing helps..
Any ideas what could it be?
Win 7 x 64
Thanks
http://www.qfpost.com/file/d?g=Ve9DzOGDb (http://www.qfpost.com/file/d?g=Ve9DzOGDb)
I think that's the first dng in the series.
Hello,
Do you planned a mac version ?
Regards
Quote from: 1% on June 03, 2013, 02:44:52 PM
http://www.qfpost.com/file/d?g=Ve9DzOGDb (http://www.qfpost.com/file/d?g=Ve9DzOGDb)
I think that's the first dng in the series.
You are correct, the artifact while reduced, it is still present with other in demosaic settings. This is due to the very high frequency nature of this image, and that the 5D doesn't have an optical low pass filter designed for video. The current demosaic filters in CineForm are designed for native resolution bayer cameras (SI-2K, KineRAW S35,etc), not bayer scaled images, like that in the 5D. Is this yet another reason I'm looking for new demosaic algorithms we can add, something closer to ACR would be nice.
Quote from: meditant on June 03, 2013, 06:30:04 PM
Hello,
Do you planned a mac version ?
Regards
Nothing official, just like it is not official for PC. I mentioned to one of the Mac engineers as GoPro, he says "cool, sounds fun". Hope he will consider it (on his weekend like I did for the PC.) ;)
QuoteHello!
Any advice would be greatly appreciated..
The software has created the files fantastically quickly - both .avi and .mov versions
But both Resolve and Adobe AE and Premiere die right after they open the file.. I can see the first frame and software dies.
Both Resolve and Adobe die in the same way.. so looks like codec issue. I have updated Quicktime tothe latest, reinstalled Go Pro cineform studio.. nothing helps..
Any ideas what could it be?
Win 7 x 64
Thanks
Same issue here.
This is 6D dng, don't have 5d... 50D soon tho.
A new version of the CineForm decoder that addresses crashing problems within Adobe products (and possibly Resolve) can be downloaded from here:
http://software.gopro.com/PC/new_CFHDEncoder.zip
Quote from: Jake Segraves on June 03, 2013, 09:10:14 PM
A new version of the CineForm decoder that addresses crashing problems within Adobe products (and possibly Resolve) can be downloaded from here:
http://software.gopro.com/PC/new_CFHDEncoder.zip
Thanks Jake.
The contents of this folder should be copied over the components in C:\Program Files (x86)\CineForm\Tools
David
this is my commandline for RAW2GPCF:
Quoteraw2gpcf M0000000.raw "C:\output.avi" -422 -c -f25
it runs for a while, displaying some dots and 0.000fps and then says DONE!
- but no output.
what am I doing wrong?
This is want it should look like
M:\Cameras\Canon5DMkIII>raw2gpcf M0000043.RAW new.avi "E:\temp\temp.avi" -422 -c
-f25
1920x720 594frames 25.000fps
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
..................................
done
The frame rate should be automatically extracted and the image size. What does yours report a size? If you are using the free GoPro CineForm Studio it is limited to 1920x1080 for direct codec access. The fact that is reports the size wrong is a worry.
Frame rate is reported as 0 in exif if the user stops recording by closing LiveView (instead of pressing stop).
Quote from: Jake Segraves on June 03, 2013, 09:10:14 PM
A new version of the CineForm decoder that addresses crashing problems within Adobe products (and possibly Resolve) can be downloaded from here:
http://software.gopro.com/PC/new_CFHDEncoder.zip
Thanks
But did not help :( The soft crashes the same way as before.. Any other advice would be greatly appreciated
Make you you did the following,
"The contents of this zip file should be copied over the components in C:\Program Files (x86)\CineForm\Tools"
If it only have with RAW and not 4:2:2, then this is the fix for the issue you describe. Make your you are not running any application that may be using the decoder, as that would prevent the component replacement from suceeding.
Quote from: DANewman on June 03, 2013, 07:24:23 PM
You are correct, the artifact while reduced, it is still present with other in demosaic settings. This is due to the very high frequency nature of this image, and that the 5D doesn't have an optical low pass filter designed for video. The current demosaic filters in CineForm are designed for native resolution bayer cameras (SI-2K, KineRAW S35,etc), not bayer scaled images, like that in the 5D. Is this yet another reason I'm looking for new demosaic algorithms we can add, something closer to ACR would be nice.
Is it possible to use debayering algorithms from open source tools like dcraw?
Found this list:
https://sites.google.com/site/demosaicalgorithms/Home
Not sure how old they are, or even if you'd be able to use them. Hopefully they help in some way ;D
Or possibly the AMaZE (Aliasing Minimization and Zipper Elimination) algorithm by Emil Martinec. https://code.google.com/p/rawtherapee/source/browse/rtengine/amaze_demosaic_RT.cc?spec=svn8d0a91f0fc6f616b9f3ffdb2c8a87ccc3903bb28&r=8d0a91f0fc6f616b9f3ffdb2c8a87ccc3903bb28
Also, are any of your debayer algorithms using AHD? I believe that's what Adobe uses.
A lot of red gives even Raw a hard time. I wonder if anybody of you can get out a proper image of this DNG file: https://dl.dropboxusercontent.com/u/10273766/000400.dng (https://dl.dropboxusercontent.com/u/10273766/000400.dng) (converted by Raw2Dng version 2013-05-30)
EDIT: This is ETTR exposed and exposure has to be pulled down at least by 2 stops before start playing with it.
I tried Adobe Camera Raw (ACR 7.4) and failed. In my final video I converted the specific clips to CineForm Raw by the tool David Newman provided. It's better but they still aren't the quality you would expect. Any ideas?
This is the final video:
https://vimeo.com/67621937 (https://vimeo.com/67621937)
Hello David,
First of all, thank you for the Cineform workflow, it makes things much more faster and easier.
However, I have several troubles on that way (WinXP SP3 / Canon 5d Mark II / GoPro Premium):
1. I can't load dng's into GoPro Studio Premium. While importing it immidiately shows me "Entry point not found" box that says:
"The procedure entry point gzdirect could not be located in the dynamic link library zlib1.dll." Then just grey preview box and nothing more.
It's not a prob though, since I successfully use DPX2CF.exe to process dng's.
2. RAW2GPCF.exe (v1.01 + latest "fixed" encoder) works fine, if dng's width is multiple of 16.
Since MarkII is recording at 1720 (or 1880), its multiples of 8. If you encode it like that, any player will crash. (Mplayer / VirtualDub / Adobe etc).
Therefore, I have to cut it with -(0,0,1711,968) command. Just a bit of annoyance.
3. >>> ... I'm looking for new demosaic algorithms we can add...
As I understand, current demosaicing options were developed with performance reasons in the first place, so it should be playable at real-time. And they do their job perfectly.
However, none of listed algorithms can be used for production output, because of artifacts and resolution loss.
On the other hand, the main problem of "quality" demosaicing is that it is processor-hungry and, well, slow.
May I ask you to think of adding production-quality options, that can be activated at the final stage of output rendering?
Suggested earlier demosaicing algorithms are pretty good imo, especially VCD ( variance of color difference) and mix of AHD and VCD interpolations.
I'm using Paul Lee's dcraw implementation for a long time, and these approaches give perfect image in 99% cases in regards of pixel accuracy, color artifacts and moire handling.
https://sites.google.com/site/demosaicalgorithms/modified-dcraw (https://sites.google.com/site/demosaicalgorithms/modified-dcraw)
Sincerely, Yaros Law.
Yea, still having the de-bayer issues. Other formats like mp4/pro-res no problem.
Thanks for everyone exploring the demosaicing options. While it is assumed our current filters where selected for performance, that is not entirely true. While there are some performance options, bi-linear and matrix 5x5, the others where the render grade filters of their day, finishing films from My Bloody Valentine to Slumdog Millionaire. As computer power has increased even more advanced algorithms are being used.
The problem we have is GoPro CineForm codec is not open source, while we are working on a SMPTE standard version of CineForm as VC-5, we can't use any GPL code -- although LGPL (lesser GPL v2.1 is fine as a dynamically linked library.) Several of the offered links point to GPL code, which sadly prevents its use. Stuck again.
One of the reasons the CineForm RAW workflow is so much easier than everything else, is the controllable demosaic is built into the decoder, effective making every tool RAW capable. So I still look for a friendly license (MIT & LGPL) or an inexpensive commercial license that could be included in a free decoder distribution.
now it crashes on me too... I copied the raw2gpcf.exe into my cineform tools folder, and when running it says "problem occurred..."
[spoiler]Problemereignisname: APPCRASH
Anwendungsname: RAW2GPCF.exe
Anwendungsversion: 0.0.0.0
Anwendungszeitstempel: 51a3899d
Fehlermodulname: RAW2GPCF.exe
Fehlermodulversion: 0.0.0.0
Fehlermodulzeitstempel: 51a3899d
Ausnahmecode: c0000094
Ausnahmeoffset: 0000b1bb
Betriebsystemversion: 6.1.7601.2.1.0.256.48
Gebietsschema-ID: 1031
Zusatzinformation 1: 0a9e
Zusatzinformation 2: 0a9e372d3b4ad19135b953a78882e789
Zusatzinformation 3: 0a9e
Zusatzinformation 4: 0a9e372d3b4ad19135b953a78882e789
[/spoiler]
I tried replacing the DLLs which only makes the crash occur a second or so earlier.
Thoughts?
Im on Win7 64 - I bet its another "rights" problem...
EDIT:
my .bat file looks like this:
[spoiler]"C:\Program Files (x86)\CineForm\Tools\raw2gpcf.exe" "X:\test\M0000000.RAW" "X:\test\M0000000.avi" -422 -c[/spoiler]
fatpig,
drop raw2gpcf.exe into dependency walker http://www.dependencywalker.com/ (32-bit version) it might tell you what you are missing.
OR
they is something difference about your source files.
OR
I messed up something regarding non-english default OSes.
Will these work? They're under gpl2+.
"1) LibRaw demosaic pack GPL2+
GitHub URL: git://github.com/LibRaw/LibRaw-demosaic-pack-GPL2.git
This pack includes these demosaic implementations:
* AFD and LMMSE implementations from PerfectRaw by Manuel Llorens
* VCD, Modified AHD, post-demosaic refinemend and median
filters by Paul Lee"
From https://github.com/LibRaw/LibRaw/blob/master/README.demosaic-packs
EDIT: never mind, I don't think the licenses are compatible.
(http://www0.xup.in/tn/2013_06/82731160.gif) (http://www.xup.in/dl,82731160/Zwischenablage01.gif/)
what do you think? putting that dll in the tools folder didnt help.
(http://www0.xup.in/tn/2013_06/82731160.gif) (http://www.xup.in/dl,82731160/Zwischenablage01.gif/)
what do you think? putting that dll in the tools folder didnt help.
EDIT: sorry for double post. The dll is in my c:/program files/internet explorer and in c:/program files (x86)/internet explorer
Need the 32-bit version Depends.exe to analyze a 32-bit application
Okay,
(http://www0.xup.in/tn/2013_06/18454463.gif) (http://www.xup.in/dl,18454463/Zwischenablage01.gif/)
GPSVC.dll - sounds like goproservice?
Quote from: DANewman on June 04, 2013, 06:31:39 PM
Thanks for everyone exploring the demosaicing options. While it is assumed our current filters where selected for performance, that is not entirely true. While there are some performance options, bi-linear and matrix 5x5, the others where the render grade filters of their day, finishing films from My Bloody Valentine to Slumdog Millionaire. As computer power has increased even more advanced algorithms are being used.
The problem we have is GoPro CineForm codec is not open source, while we are working on a SMPTE standard version of CineForm as VC-5, we can't use any GPL code -- although LGPL (lesser GPL v2.1 is fine as a dynamically linked library.) Several of the offered links point to GPL code, which sadly prevents its use. Stuck again.
One of the reasons the CineForm RAW workflow is so much easier than everything else, is the controllable demosaic is built into the decoder, effective making every tool RAW capable. So I still look for a friendly license (MIT & LGPL) or an inexpensive commercial license that could be included in a free decoder distribution.
I'm glad to hear you are actively looking into enhancing this solution with other demosaicing options. I am quite happy with this workflow... I'm able to get my final footage looking very close to what ACR can produce, it's just fine detail that is somewhat lacking. Highlight recovery is not as good and noise reduction would be nice. Keep at it because if these things are solved, I can't envision a quicker or more space efficient workflow.
Update to v1.02 of RAW2GPCF.exe
Direct download for RAW2GPCF: http://miscdata.com/ML/RAW2GPCFv102.zip
Unzip are install here: C:\Program Files (x86)\CineForm\Tools
This replaces the crappy OpenMP threading with some of our own threading library, this should remove a dependency (maybe cause some users issue) and it is up to 2X faster for YUV and RGB exports.
(http://www0.xup.in/tn/2013_06/54297870.gif) (http://www.xup.in/dl,54297870/Zwischenablage01.gif/)
Now I put all the files it showed as yellow inside the Tools folder, which are gpsvc.dll and sysntfy.dll and ieshims.dll.
It still crashed, I updated Cineform studio, no luck. Version 1.02 of RAW2GPCF- no luck, too.
I don't know what else to read from the dependencywalker - does someone understand what it says?
maybe someone can check where these dlls lie on their system? thanks! :)
@RAW2GPCF
I'm using this command line:
"C:\Program Files (x86)\GoPro\Tools\RAW2GPCF" C:\Users\user\Desktop\RawFiles\Raw\M01-1120.RAW C:\Users\Public\Documents\Video\aa.mov
In cmd it runs and ends fine with done, but there is no resulting file? I wonder why, could it be that the ratio isn't an HD one(60D footage)? I'm on Win 8 with CineForm Studio Premium.
EDIT: Ah, adding '-422' makes it produce a file.
Quote from: fatpig on June 05, 2013, 09:45:21 AM
(http://www0.xup.in/tn/2013_06/54297870.gif) (http://www.xup.in/dl,54297870/Zwischenablage01.gif/)
Now I put all the files it showed as yellow inside the Tools folder, which are gpsvc.dll and sysntfy.dll and ieshims.dll.
It still crashed, I updated Cineform studio, no luck. Version 1.02 of RAW2GPCF- no luck, too.
I don't know what else to read from the dependencywalker - does someone understand what it says?
maybe someone can check where these dlls lie on their system? thanks! :)
I think you should undo the changes you've made, as it seems you have now mixing 32 and 64-bit components that may break other things. Even before your changes, I see that IESHIMS.DLL is a 64-bit component where a 32-bit component should be. Sorry I'm not much of a Window system doctor. This tool requires very few system resources, just KERNEL32.DLL, USER32.DLL, OLE32.DLL (your system wouldn't boot without these) and MSVCR90.DLL. All the other files are dependences of dependences, we have no control over that. So the likely issue in MSVCR90.DLL, even though depends didn't flag it. GoPro CineForm Studio should have installed the correct version of the Visual Studio 2008 run time libraries (MSVCR90.DLL is part of), but you can install them again from here: http://www.microsoft.com/en-us/download/details.aspx?id=29
Quote from: tin2tin on June 05, 2013, 01:19:27 PM
@RAW2GPCF
I'm using this command line:
"C:\Program Files (x86)\GoPro\Tools\RAW2GPCF" C:\Users\user\Desktop\RawFiles\Raw\M01-1120.RAW C:\Users\Public\Documents\Video\aa.mov
In cmd it runs and ends fine with done, but there is no resulting file? I wonder why, could it be that the ratio isn't an HD one(60D footage)? I'm on Win 8 with CineForm Studio Premium.
EDIT: Ah, adding '-422' makes it produce a file.
If you are using the free GoPro CineForm Studio, you only have access to the YUV 4:2:2 encoder. For CineForm RAW and all its coolness, you will need the Premium version of the software.
hello DANewman.
I removed the files again and installed from the link you gave.
dependencywalker does show a little bit different.
but not much.
http://www.xup.in/dl,14209115/next.gif/ (http://www.xup.in/dl,14209115/next.gif/)
another screenshot, I really don't know how to proceed.
I am developing the BATCHelor app, and would love to include it.
But my main machine and my laptop, both german Win764 can not run the RAW2GPCF.exe
It either suggests the tool doesn't like the OS version -- no idea why. Or you have installed another incompatible application on both your systems. Do you have same issue with VR2CF.exe, DXP2CF.exe, CF2DPX.exe or ParseCFHD.exe? These are very similar shell tools that ship with Premium, and they never have this type of error report.
VR2CF.exe is the closest to RAW2GPCF.exe as it the code base it was built upon.
DPX2CF.exe is the next closest using DShow Encoding.
CF2DPX.exe doesn't use DShow, but similar to DPX2CF for other resources.
ParseCFHD.exe is the simpliest.
Test these tools, add maybe I can understand what is wrong/missing on your system.
It seems like my issue is the same or close to fatpig.. as my win7 x64 in with Russian interface..
Quote from: advent2 on June 05, 2013, 10:26:11 PM
It seems like my issue is the same or close to fatpig.. as my win7 x64 in with Russian interface..
I don't think so. You can at least run the tool, your problem within other tools - "But both Resolve and Adobe AE and Premiere die right after they open the file."
Seems we have a few isolated compatibility issues, but generally it works. For anyone having problems, try to determine if it is just Canon RAW related. Convert a GoPro or regular H.264 clip, and play with that to see whether your GoPro CineForm Studio has installed correctly. If that doesn't work, you have an issue for regular GoPro support, and not something for this forum.
as these tools you mention, like VR2CF.exe only ship with premium, are you sure that I don't need premium for your tool to work?
I am trying to obtain a Phantom Raw file to test with vr2cf.exe now..
also, on this site http://cineform.tistory.com/128
they say that vr2cf.exe needs another exe to run- and that these belong in the windows/system32 folder...
confusing.
which command line tool would I use for converting regular H.264?
EDIT: I am using it with 5DII Raw files. this is not the problem is it?
You don't need Premium for the tool to work with the -422 option for sources that are 1920x1080 or less and horizontal resolution divisible by 16. And it doesn't crash if you forget to -422 switch with the free tools, just no output. I thought you said it would crash just by running it, it is why I was suggestion libraries and depends -- I can't keep look up everyone's history (I don't work in support.) But looking back over your posts, you said it ran, but with no output, but then you have the wrong switches. When we got that sorted out, then is crashed, but only after some time. When know there was something wrong with your source files as it said 0.000 fps. Sorry, too many unknowns here, I'm confused as you are. I guess you can convert to DNG then use CineForm Studio to convert. Revisit this when the tool when it is more mature or there is more data to know what the issue is.
But one last thing to try. Let's test with known good footage.
Download M0000000.RAW from here https://drive.google.com/a/gopro.com/folderview?id=0B5vzb1fFuV73UW91LUY4OHBrMlE&usp=sharing#
Copy that to C:\
Copy RAW2GPCF.exe to C:\
Have GoPro CineForm Studio installed
Open a CMD shell
* cd to the root of C:\
> regsvr32 "C:\Program Files (x86)\Common Files\CineForm\CFEncoder2.ax"
// The above will ensure the correct encoder is registered, the installer should have does this. It will have a Pop-up to say success.
> .\RAW2GPCF M0000000.RAW test.avi -422
If that doesn't work, we know it is a configuration failure, not source or pathing issues.
(https://dl.dropboxusercontent.com/u/5056120/RAW2GPCFtest.png)
Thank you for this wonderful tool.
Am I correct that there is still no way to access metadata of Cineform raw in Resolve on a Mac?
That would be up to Blackmagic, please ask them as they do have the ability to read CineForm metadata.
Kevperry, thank you for the positive comment.
I finally got my 5D3 in RAW working reliably -- I needed to disable the auto off feature as the camera seemed to randomly shutdown. Now that I can record 10+GB files, I want to test this workflow.
It seems for the cost conscious not wanting to purchase too many 1000X 64GB cards, the main issue is the off-load speed. My old home computer lacks USB 3.0, limiting read speeds to something so terrible I don't care to messure. However my new Surface Pro does have one USB 3.0 port, but it is very limited in internal space.
So my setup for this test:
Surface Pro 128GB 1.7Ghz i5-3317U
CineForm Studio Premium 2.0 (beta, yes you don't have this yet.)
RAW2GPCF v1.02
USB 3.0 4 Port HUB (plugged into the only USB port)
USB 3.0 CF reader (connected to the hub)
USB 3.0 750GB 2.5" external drive (connected to the hub)
64GB KOMPUBAY 1000X card (connected to the reader)
All running off the Surface Pro battery (I left the power supply at the office.)
Source clip 11.7GB 1920x816 23.976 3m02s.
From a shell I converted to a Filmscan 1 quality CineForm RAW AVI file, directly to the external driver in 2m47s, to a 9:1 compressed to 1.3GB. Surface CPU was only at 30%, so it was bottlenecked by the CF read speed (est 70MB/s -- reader more than the card.) Even with this less than optimal setup, I can transfer, transcode and store RAW faster than I can record it. Only need one more 64GB card for mear continuous RAW acquisition.
I plan to shoot 5D RAW for my next 48 Hour Film Project, along we a bunch of HERO3 Black edition 2.7k. ;)
@DANewman:
Thanks for your tireless work! I really appreciate it.
If I get it working I will implement it in BATCHelor so everyone can benefit. :)
"divisible by 16" - that one got me thinking.
As I am on 5DII, not III - my res. is 1880 - which is not divisible by 16.
There may be the problem.
I will test out your proposed method!
EDIT: You did it! :) It works! Indeed, it is all in the resolution. Man this is fast! No way for 1880 to be included?
New question: with the full cineform studio app, how do I make it accept the DNGs? I dropped a folder with DNGs on the import bin. no import. Also tried your 5DIII file (after using raw2dng), which it doesn't accept, also. what did I miss?
I should have the tool auto crop to divisible by 16, so 1880 will be encoded 1872. In the 10 years we've had CineForm all the video resolutions have been divisible by 16 in the horizontal : 720 (SD), 960v(P2), 1280, 1440(HDV), 1920, 2048, 2400 (BMCC), 2704 (HERO3 Black), 2880 (Alexa), 4096... So we have assumed that multiple for speed reasons. It relates to the wavelet structure of the codec. Vertically we can support anything, although even numbers for RAW.
so, is there an option to force it to crop with a commandline action?
I tried -422 -c -(4,0,1876,0) - still crashes.
I can not open any DNG sequences in cineform studio GUI version. nothing happens when dropping my DNG folder in the import bin.
Is that normal?
@fatpig: Don't drop the folder, use the Import browse to navigate TO THE FIRST.DNG IN THE FOLDER. This works for me. Cheers!
I would love to.
Sadly, DNG is not listed as a File Type, so I cant see it.
See for yourself: http://www.xup.in/dl,69694161/hmm.gif/ :o
...you have to have the pro version for this, don't you?
dng support is only in Premium/Pro
The windowing switch is
-(4,0,1876,960)
You gave your image a zero height, which apparently crashes.
Okay, still crashes.
Maybe there is something else about the 5DII File. Do you have access to one?
Would it help if I uploaded a 5DII .RAW for testing purposes?
Please upload a 5D II RAW file as I don't have those. Have you tried the 5D3 samples I linked?
Hi DANewman, just wanted to say thanks for RAW2GPCF, it made me discover CineForm RAW, which seems the way forward to encode video. It just makes so much sense to compress before demosaicing and sharpening! Also it's really nice to have a single file for all of editing, grading and archival. I wish a mac version of RAW2GPCF were available though!
Anyway, it all works very well, the CineForm demosaicings maybe are not the best, but if you do a final grade in Resolve it does not matter.
Thanks again,
Bruno
Quote from: Brunos on June 06, 2013, 11:51:52 PM
Hi DANewman, just wanted to say thanks for RAW2GPCF, it made me discover CineForm RAW, which seems the way forward to encode video. It just makes so much sense to compress before demosaicing and sharpening! Also it's really nice to have a single file for all of editing, grading and archival. I wish a mac version of RAW2GPCF were available though!
Anyway, it all works very well, the CineForm demosaicings maybe are not the best, but if you do a final grade in Resolve it does not matter.
Thanks again,
Bruno
Thank you. Working on offering Mac tools (soonish) and better demosaicing options (eventually). Thank you for your feedback.
Quote from: DANewman on June 04, 2013, 06:31:39 PM
Thanks for everyone exploring the demosaicing options. While it is assumed our current filters where selected for performance, that is not entirely true. While there are some performance options, bi-linear and matrix 5x5, the others where the render grade filters of their day, finishing films from My Bloody Valentine to Slumdog Millionaire. As computer power has increased even more advanced algorithms are being used.
Thanks for all the work on the tools.
These advanced stills debayer options to one degree or another were already out long before Slumdog. I've noticed that stills software often seems to have used much more advanced algorithms. At the point that stills sharpening tools had all but done away with nasty white halos and such you'd still see commercial DVDs/blu-rays with nasty white halos all over the place, etc.
Maybe, in part, because stills software was aimed at only doing 1 frame at a time or maybe a few hundred while video stuff was aimed at tens of thousands or millions of frames. Some of it might also be that the two sides didn't cross pollinate much either perhaps.
And more recently as of a couple years ago the RAW control sliders have become much more powerful in stills software. The sliders in ACR, for instance, seem to work way better than the sliders in most video correcting software. Although in this case, some of it is that I'm not so family with video editing software and haven't likely used the best stuff either.
Of course the stills stuff doesn't let you do motion tracking corrections so easily like some video plug-ins and it's probably tricky to make sure all sorts of footage matches perfectly, if you need to do that sort of thing. And certain stills tools can cause frame to frame flicker if things are changing too much frame to frame.
The thing that is holding back Cineform RAW is the debayering/early stage sharpening/NR are soooooo far behind stills software that I just can't see leaving behind using the Photoshop/ACR/AE workflow for now. GoProStudio is nice to quickly scan footage to see if flipping through images in a stills viewer doesn't give you enough sense as to whether to process the clip or not. And using Cineform 444 film scan 1/2 is a nice codec to use as output to use in Premiere Pro or whatever to cut stuff and edit it all together and so on. It holds lots of bits just in case you still need to make some corrections still and seems to have barely any compression artifacts.
Using many options also crashes Premiere Pro out when using the Cinform RAW files made by GoproStudio. I didn't try that patch a few posts up yet though.
Quote from: Brunos on June 06, 2013, 11:51:52 PM
Anyway, it all works very well, the CineForm demosaicings maybe are not the best, but if you do a final grade in Resolve it does not matter.
Thanks again,
Bruno
So it demosaics as will in Resolve as in ACR?
Of course there are so many different settings that it's easy to get a false impression. It's also probably easy to start getting into a more digital and less film-like sharpness with the ACR workflow, balancing things out the detail level isn't so different compared to CF Advanced 2.
The CineformRAW demoasic does seem to be prone to creating even more false colors bits popping up (trying advanced level 2), some edges get odd chroma bits on them and there seems to be no general chroma NR either so you get that plus created chroma noise in addition perhaps.
The Resolve rendering of CIneformRAW avoids that but appears to be a bit prone to zipper/checkboard artifacts. Does anyone know what it means when Resolve lists Cineform 16bit output. What is that exactly? Also Resolve doesn't seem to understand the color data yet so it's almost impossible to figure out how to bring it back to normal looking colors.
Going back to Cineform RAW in general, perhaps it is the compression used or some luma NR, but relatively uniform areas of OOF color tend to become smooth (with occasional odd things showing up) or large grained on some dark transitions using the Cineform RAW files while the ACR method (using Cineform 444 FS2 once all the RAW stuff is complete) leaves them very finely grained and not smoothed over at all.
ACR processing stage takes forever, but things like Magic Bullet Colorista take forever during final render so I'm not sure if all that much total time is saved, although it is certainly much handier to leave long stages until the end and the ACR workflow has the long stages at the beginning.
Quote from: LB on June 07, 2013, 04:35:37 AM
... it is certainly much handier to leave long stages until the end and the ACR workflow has the long stages at the beginning.
This is why a little nicer demosaic will complete this optimal raw workflow. With a 10-20X shooting ratio for native filmmaking, that is a lot of processing for an ACR pass. Whereas in CineForm RAW the preview demosaic is real-time, allowing the edit without any commitment to extensive computation, until only the frames needed are selected.
New version v1.03
* Fixed support for FD Mk II files with corrupted or missing frame rate information. Defaults to 23.976 is not specified.
* Auto masking the image to divisible by 16 widths as required by the encoder.
http://miscdata.com/ML/RAW2GPCFv103.zip
You are the man !! Thanks so much!
what can i do about the black level?
Debayering is still unusable. I don't have these issues with pro-res or dnxhd.
maybe unusable for final render, but I think this is the way to go for fast proxies! :)
Quote from: 1% on June 07, 2013, 08:18:34 PM
Debayering is still unusable. I don't have these issues with pro-res or dnxhd.
Those codecs don't have RAW abilities, so you can't comparing the same thing.
Add I disagree that the currently demosaicing option aren't usable as is, even for finishing, many do and have done for years. Of course 5D3 is a lower on the resolution compared with SI-2K, BMCC and Alexa. HD finishing from >HD RAW source, makes demosaic selection far less of a factor. All that said, I do want newer demosaicing options, and have been asking for help on that.
One workflow that hasn't been considered. Using CineForm RAW as your fast ingest and real-time on-line edit format (discarding source RAW,) then doing a lossless project trim (thinking Adobe PPro here), then converting only these sources to DNG for an ACR demosaic. This is very like developing CineForm/ProRES/Whatever to DPX for an old school finish. Really Adobe should be supporting native CineForm RAW, as then we wouldn't be having this discussion.
Quote from: 1% on June 07, 2013, 08:18:34 PM
Debayering is still unusable. I don't have these issues with pro-res or dnxhd.
This might be the temporary solution to this problem until new debayer options are added to CF Studio.
Quote from: DANewman on May 27, 2013, 04:08:54 AM
I don't consider applying the demosaic as a necesary part of RAW. Consider the Protune CAM_RAW mode on GoPro HERO3 Black Edition (something I worked on,) that is about as raw as you can get in an H.264 file. All the demosaic is applied in camera, but color matrix and white balance are applied in post, just like classic uncompressed RAW.
If there is enough interest, I will enable YUV or 444 development, without baking the color. The only reason you might like 4:2:2 RAW, is that could run on the freely distributed GoPro CineForm Studio, just like Protune CAM_RAW works for GoPro users today.
But until then, Resolve debayer works well enough for most.
Quote from: DANewman on June 07, 2013, 08:56:50 PM
One workflow that hasn't been considered. Using CineForm RAW as your fast ingest and real-time on-line edit format (discarding source RAW,) then doing a lossless project trim (thinking Adobe PPro here), then converting only these sources to DNG for an ACR demosaic. This is very like developing CineForm/ProRES/Whatever to DPX for an old school finish. Really Adobe should be supporting native CineForm RAW, as then we wouldn't be having this discussion.
And this sounds awesome as well. I never thought of round-tripping back to DNGs. Is this already possible with CF2DPX? Or will a new tool have to be developed?
Quote from: DANewman on June 07, 2013, 08:56:50 PMReally Adobe should be supporting native CineForm RAW, as then we wouldn't be having this discussion.
Adobe has apparently ZERO interest in doing that (http://forums.adobe.com/message/4543438#4543438). Maybe you (as someone whose word might have some weight) could jump in to help those guys understand (good luck!).
Even if they do, it will most likely take like three iterations from now on (starting at the upcoming CC) to get it implemented... :-(
TL, DR: Same problem as fatpig, no output with just cineform raw.
Installed Cineform Decoder Update: http://cineform.com/gopro-cineform-decoder
Cineform found update immediately after (1.3.2.170), installed update, finally worked.
Hey guys, extremely new to all of this, but had the same problem as fatpig and others: 422 worked, but just Cineform RAW didn't create any output file.
As Dan suggested, I tried looking for the bug within Cineform Studio itself, took a normal MP4-video and tried to convert it in Cineform Studio Free. Didn't work, progress bar immediately stopped at 0%, wihtout crashing though. Just didn't do anything. Apparently a lot of ppl have this problem without an obvious solution.
So some ppl suggestes updating Cineform Studio free, but mine was already at the newest version 1.3.2.169. One post suggested, updating the Cineform Decoders aka the former Neo Player: http://cineform.com/gopro-cineform-decoder
Strange thing then: I updated the decoders, immediately after updating, the Cineform Studio update service found an update (1.3.2.170, but the Premium version). Downloaded and installed the Premium, transcoding within Cineform Studio still didn't work, but now it worked within RAWanizer. Win7 64bit here, hope this is of any help.
@DAN: thank you for all your efforts. it is beyond appreciated.
Im currently ripping the RAW to dng using ML's raw2dng, open adobe after effects - then using 16bit colourspace and project under AE, import 'multiple files' select first DNG from whichever folder you want to import clip from (which then subsequently opens Adobe beta 8.1 acr), here apply correction, exposure, etc... settings which are then applied automatically to all your following alphabetical DNGs during the import, Interpret footage to 23.976 (24 ie whatever fps was recorded), then 'remember Interpretation', then apply interp across all your selected multiple DNG imported sequential files, then make a 'New Comp from Selection', under the Comp menu checkbox 'sequence layers' and 'overlap' and choose '0:00:00:00' so they DON'T overlap (all the files are instantly comp'd sequentially - check fades OFF, then render the comp using Best Settings, Output module make a Cineform RAW setting and save it as 'Cineform lossless' or whatever (as pictured below), make an 'output to' filename of the comp and render away. Its not too bad :-)
(https://dl.dropboxusercontent.com/u/14239185/Go%20Pro%20CineformRAW%20codec%20under%20AE%20is%20444.png)
I might try some way to round trip it... but that's exactly what I want to do, throw away the raw for one solid grade-able video.
I tried batch converting with rawanizer or command line. MPC is playing it back like shown.
Quote from: fatpig on June 07, 2013, 07:29:11 PM
You are the man !! Thanks so much!
what can i do about the black level?
The black is set for the 5D3, maybe like fps, that is missing for the 5D2. I could add a switch for overriding or setting the black and white levels.
Quote from: iaremrsir on June 08, 2013, 12:06:38 AM
... I never thought of round-tripping back to DNGs. Is this already possible with CF2DPX? Or will a new tool have to be developed?
A new tool. Going back to DNG would be easy, doing so with the correct metadata including user image corrections would be way cooler and harder to do.
Quote from: moswanted on June 08, 2013, 02:17:31 AM
TL, DR: Same problem as fatpig, ...
@DAN: thank you for all your efforts. it is beyond appreciated.
Thank you. From your post it wasn't clear if you solved your issue. Fatpig's issue is solved with v1.03; is there an action that will help you?
Quote from: fatpig on June 07, 2013, 07:29:11 PM
You are the man !! Thanks so much!
what can i do about the black level?
The 5D2 is providing a default black level, and looking at your sample clip, the black level seems okay. What adjustments on the transcode do you want, particularly as you can change anything in post if the data is not clipped
the missing fps on 5DII are supposed to be my fault, because of some wrong handling with the hack. I heard if done right, the 5D does write the correct fps in the file-
black level switch would be great, also I dont know where to look up the black level for each camera
In the footer of a raw file, or in the exif info of a DNG from raw2dng. The value is not constant (it's somewhere around 1024, 2048, sometimes in-between).
I am on a Mac ....and This is THE easiest working workflow for me at the moment for ML raw. Using VMware fusion, I transcode from ml raw right to cineform raw using RAWanizer. Then on the Mac side I use Cineform Studio Premium to correct the raw files. You don't even have to save or render!!! The changes show up real time in Premiere, FCPx, Filmconvert Pro....the cineform Raw files play in just about anything.
I wish Resolve could edit the cineform raw metadata.....the changes made in Cineform Studio don't show up there. Looks like it just ignores the raw metadata altogether.
Thanks for your work Dan!
Hello David,
RAW2GPCF v1.03 gives me an error on start: The application has failed to start because the side by side configuration is incorrect please see the application event log or use the command line sxstrace.exe tool for more detail.
While previous versions still working perfectly fine. I'm at Win7 Pro x64 this time )
What I tried is to uninstall and reinstall all VC++ packages in chronological order, but for no effect.
Thank you, Cineform RAW in Resolve (mov container) is pure gold.
Quote from: Yaros525 on June 08, 2013, 09:04:19 PM
RAW2GPCF v1.03 gives me an error on start: The application has failed to start because the side by side configuration is incorrect please see the application event log or use the command line sxstrace.exe tool for more detail.
...
Thank you, Cineform RAW in Resolve (mov container) is pure gold.
Thank you.
Yesterday's build was from a different system, maybe using the wrong redist components.
Today's build is from my home system used to build the earlier revisions.
http://miscdata.com/ML/RAW2GPCFv104.zip
This added black and white levels controls, -blX and -wlX.
e.g. -bl1000 would ignore the metadata and use 1000 as the black level.
Quote from: kevperry on June 08, 2013, 05:19:43 PM
I wish Resolve could edit the cineform raw metadata.....the changes made in Cineform Studio don't show up there. Looks like it just ignores the raw metadata altogether.
Thanks for your work Dan!
We do have a hack for that, but it prevents you using the Resolve demosaic.
Interesting! Where can I get the hack for osx?
Quote from: kevperry on June 08, 2013, 11:08:27 PM
Interesting! Where can I get the hack for osx?
Sorry just like this tool the hack is still PC only, but closer to addressing that.
Hi guys, does anyone have a ProTune LUT to add to Resolve? I found a CineStyle one here (http://desktopvideoguy.blogspot.co.uk/search/label/LUT (http://desktopvideoguy.blogspot.co.uk/search/label/LUT)) which is supposedly similar to ProTune, but skintones tend to start off very gray. I tried to make my own but have found that it's very hard (impossible?) to make one which works well in different lighting conditions.
Wow! thanks , after installing 1.04 it finally worked for me :)
Pritty amazing ..thanks
Question .when I put the file in premiere.. it plays. now I want to change raw parameters.. I right click on the file, select Source Settings.. and Go Pro studio opens. I can do differnt changes there.. but the only way out is to convert to mp4 . Is this so or I can get those settings to Premiere without recoding to mp4? thanks!!!!
OOPS got it solved :) it just updates the video within few seconds on the timeline!once again, thanks!
Does this also work with a dng sequence?
Quote from: fatpig on June 09, 2013, 01:25:20 PM
Does this also work with a dng sequence?
I think you are talking about the black level switches. Yes it should, as that feature already existing with DPX2CF that ships with Premium. RAW2GPCF doesn't need to work with DNG files.
Quote from: advent2 on June 09, 2013, 07:53:09 AM
Wow! thanks , after installing 1.04 it finally worked for me :)
Pritty amazing ..thanks
Question .when I put the file in premiere.. it plays. now I want to change raw parameters.. I right click on the file, select Source Settings.. and Go Pro studio opens. I can do differnt changes there.. but the only way out is to convert to mp4 . Is this so or I can get those settings to Premiere without recoding to mp4? thanks!!!!
OOPS got it solved :) it just updates the video within few seconds on the timeline!once again, thanks!
Yes Premiere will cache frames for a while, can be annoying, but it is minor. Sometimes switching playback resolution will implemently flush the old cache frames. We requested cache control via the Adobe API for years, they haven't been all that interested. Occassionally a cached frame will stick, a bug new to CS6, so for exports with recent metadata changes in Studio, restart Premiere.
Quote from: Brunos on June 09, 2013, 12:50:18 AM
Hi guys, does anyone have a ProTune LUT to add to Resolve? I found a CineStyle one here (http://desktopvideoguy.blogspot.co.uk/search/label/LUT (http://desktopvideoguy.blogspot.co.uk/search/label/LUT)) which is supposedly similar to ProTune, but skintones tend to start off very gray. I tried to make my own but have found that it's very hard (impossible?) to make one which works well in different lighting conditions.
Tricky. LUTs are best for converting calibrated images to a creative look, but the issue with Resolve is it is ignoring all the calibration data like white balance and color matrix. While you can bake the color matrix into a LUT, you can't do that for white balance, which needs to be performed on the linear light image underneath/before the LUT/color matrix.
Hi,
version 1.04 works perfectly fine, thanks!
However, I can't seem to understand how black and white levels controls work. What's default setting for 5DMkII raw?
Is it normalized to [0...1023] instead of [1024...15000] ?
In Cineform Studio (EDIT):
When I set -wl1024 it looks fine.
When I set -wl1280 it looks contrasty with green cast.
When I set -wl900 it looks uncontrasty with magenta cast. (why shadows raised while i've altered white level?)
As for -blX , it just doesn't do anything for me. -bl0 looks the same as -bl1024.
Can you clarify, thanks!
New version v1.05
* Fixed the -wlX switch (white level), which was being incorrectly used as black level .
http://miscdata.com/ML/RAW2GPCFv105.zip
White and black levels range whatever the camera's bitdepth is. For Canon RAW (so far) it is 14-bit, so 0 to 16383.
Quote from: DANewman on June 13, 2013, 04:53:50 AM
New version v1.05
* Fixed the -wlX switch (white level), which was being incorrectly used as black level .
http://miscdata.com/ML/RAW2GPCFv105.zip
White and black levels range whatever the camera's bitdepth is. For Canon RAW (so far) it is 14-bit, so 0 to 16383.
I am trying to get this to work but every time I run the script command prompt flashes for a second and nothing happens... what am I doing wrong?
Many users do have trouble running this tool as there are install prerequisites for supported encoders and sometimes for particular switches. This new build addresses some of those issues. It will test what license level you are running and switch to 422 automatically if you can't encode RAW or 4:4:4, and it will now report if no suitable encoders can be found. If you are all up and running, there is nothing new for you in the build, but it should help first time users.
http://miscdata.com/ML/RAW2GPCFv106.zip
Hello again,
Really fine work! Especially for my weak laptop PC, which handles all RAW data nicely with CineForm! :)
Is it possible for you to make it read the .RAW, .R00, .R01, etc. split files from the 5DII sequentially without us having to join those beforehand?
This is really time consuming.
Thanks!
I guess I can simulate that with a non-ex FAT32 CF card. If you convert the .RAW alone, does it work or do you have to concatenate the files first?
Quote from: DANewman on June 16, 2013, 07:56:33 PM
Many users do have trouble running this tool as there are install prerequisites for supported encoders and sometimes for particular switches. This new build addresses some of those issues. It will test what license level you are running and switch to 422 automatically if you can't encode RAW or 4:4:4, and it will now report if no suitable encoders can be found. If you are all up and running, there is nothing new for you in the build, but it should help first time users.
http://miscdata.com/ML/RAW2GPCFv106.zip
I guess I am missing something essential here as I stll do not get it to work with this build, is there some sort of (dummy) instruction for the steps to be taken?
Hi-
the split .RAW files can not be processed without joining them first. RAW2GPCF immediately closes.
GingerHDR managed to skip the joining step so I guess it is technically possible.
Do you want me to send you a test file?
Quote from: Africashot on June 16, 2013, 02:00:49 PM
I am trying to get this to work but every time I run the script command prompt flashes for a second and nothing happens... what am I doing wrong?
Launch a shell (cmd) first. Then type raw2gpcf within the opened shell, does that work?
Quote from: fatpig on June 17, 2013, 11:39:52 AM
Hi-
the split .RAW files can not be processed without joining them first. RAW2GPCF immediately closes.
GingerHDR managed to skip the joining step so I guess it is technically possible.
Do you want me to send you a test file?
Yes, please, a sample would be helpful.
I am currently uploading a sample for your testing. It will take some time. :)
New version v1.07
* Add support for segments .RAW .R00 .R01 ...
http://miscdata.com/ML/RAW2GPCFv107.zip
These new feature is barely tested, but it worked on the sample file fatpig provided (thanks.) The -sX skip frame switch may not work perfectly, and the -iX (input point) will not work beyond the first segment (yet.)
Please Clarify:
A Cineform raw file opened in Resolve (without the patch) would contain all the 'raw' data of that original DNG right? The potentially cumbersome part is that you need to use Resolve's controls to get the image looking 'normal' rather than having a nice Cineform UI built in to Resolve. But you still have the ability to retain the highlights like you do in the raw file and affect color before the demosaicing which Resolve does in this case, cierto? So the .RAW to CDNG to CineformRAW workflow colored in Resolve gives us a result comparable enough to ACR with a much smaller file size and the ability to do an 'online only' workflow. (Minus some features like ACR's highlight regeneration based on unclipped channels.) Is this a correct understanding?
Using the Resolve patch for Cineform means that Cineform does the debayer and raw processing and 'hands' that image to Resolve. This means that the image is subject to older (but still respectable) debayering technology and that the raw data is no longer quite as 'raw'--more like medium rare.
Please shed light if this understanding is incorrect.
That is all correct. Although with the RAW2GPCF.exe tool you can bypass the CinemaDNG step.
Does anyone know the likelihood of Blackmagic adding Cineform Raw controls into Resolve? Well I guess before I ask that I should ask if there is a CF Raw SDK to use that will allow Studio Premium type control in other applications.
Quote from: DANewman on June 19, 2013, 10:15:27 PM
That is all correct. Although with the RAW2GPCF.exe tool you can bypass the CinemaDNG step.
Doing this method leaves dead pixels in the image for me. In CF Studio, I have the option to remove dead pixels on import of DNG files. Is it possible to implement this in raw2gpcf?
Quote from: iaremrsir on June 19, 2013, 11:05:02 PM
Doing this method leaves dead pixels in the image for me. In CF Studio, I have the option to remove dead pixels on import of DNG files. Is it possible to implement this in raw2gpcf?
That would be possible. I just need samples with dead pixels. Please share your dead pixels.
Quote from: iaremrsir on June 19, 2013, 11:01:51 PM
Does anyone know the likelihood of Blackmagic adding Cineform Raw controls into Resolve? Well I guess before I ask that I should ask if there is a CF Raw SDK to use that will allow Studio Premium type control in other applications.
Well Canon RAW might be cutting into BMCC sales, so maybe you could convince them to add the image development support for the non-free version of Resolve. Blackmagic has full CineForm SDK accesss, they can read all the image development metadata and default Resolves controls to a nice image if there was enough demand. All doable.
New version v1.08
* Added support for input point seeking with segments .RAW .R00 .R01 ...
http://miscdata.com/ML/RAW2GPCFv108.zip
Segments should now be fully supported.
Does anyone see the need for black balancing? This would be a few second recording with your lens cap on at your current resolution and shooting settings. This could be then used to determine, pixel by pixel, the black offset and removing any remaining fixed pattern noise, and useful for masking any dead pixels (hot pixels in particular.) Let me know if this is a feature needed. I have know idea if there are only a few users of this tool or hundreds+.
Thanks for all the work on this! Amazing, how fast you implemented the split file feature! I am looking forward to testing it!
I know at least that on german film sets blackbalancing is done on a daily basis, so there might be a reason for that, I myself never used it before..
For SI-2K we would black balance it regularly, particularly if the temperature was significantly different (as the noise level changes.) Black balance can increase the usable dynamic range, so I might give it a go.
Okay, I am no expert in this- but I will look into the topic to get a better understanding.
Do I see it correctly assuming paid users can have the full RAW workflow, but smaller file size with RAW2GPCF?
the -422 option does not give the same flexibility right now, of course.
New version v1.09
* Added support for black level calibration
http://miscdata.com/ML/RAW2GPCFv109.zip
The black balancing will remove any remaining fixed pattern noise (FPN) the sensor can produce. Canon 5D3 is very good a suppressing FPN. In my own 5D3 experiments, the black balancing did remove the hot pixels that appear at higher ISOs.
How to black balance:
1) With ISO & resolution parameters set, shoot 100+ frames of RAW with the lens cap on.
2) Then shot your scene(s) that uses these camera settings.
3) when converting using RAW2GPCF, add the -bc#### switch like this
>RAW2GPCF CF0803.avi M21-0803.RAW -bcM21-0802.RAW -422
The -bc switch points to the file that was recorded with the lens cap on.
Sample output:
>RAW2GPCF CF0803.avi M20-2209.RAW -bcM20-2208.RAW -422
M20-2208.RAW 1920x1080 107frames 23.976fps
min = 2028, max = 2844, avg = 2059
M20-2209.RAW 1920x1080 229frames 23.976fps
................................................................................
................................................................................
.....................................................................
done
Press any key to continue . . .
How it works:
All the frames in the black clip are averaged together, the more frames used the less noise in the pixel map for the black level of the sensor. So instead of subtracting a fixed black offset value for all pixels (all sensors have an offset), each pixel has a customized black level. Even with the nice pixels from the 5D3, there is variation in the black level for each pixel location (see output min = 2028, max = 2844, avg = 2059.)
Does this software support Mac Users
Quote from: dariSSight#1 on June 21, 2013, 05:41:17 PM
Does this software support Mac Users
Not yet, but I have someone interested in doing a Mac port.
very impressive!!
Thank you. It helps to be motivated by your own projects. I have a film project in 3 weeks time, so this tool needs to be production ready by then. It is cool that the 5D3 is low in FPN, but other cameras models may see greater benefit. I will likely black balance for 3200 ISO or greater, otherwise the images are pretty clean.
I tried latest on 50D footage.. its perfect... will have to see how 6D does. I feel comfortable deleting the 50D raw file.
I finally have wildcards for batch conversion. Just a bit more testing, but I hope post the batching version later today.
New version v1.10
* Added batch conversion support through '*' wildcards.
http://miscdata.com/ML/RAW2GPCFv110.zip
The wildcards allow you to injest and transcode a whole CF card with a single command.
If converting in place (on disk in the same folder, 'cd' to that folder)
raw2gpcf *.RAW *.AVI
basic auto conversion directly from card to local disk:
raw2gpcf L:\DCIM\100EOS5D\*.RAW M:\CineForm\*.AVI
If you black balance you will need the name of the black clip, but this will be used for the entire batch
raw2gpcf L:\DCIM\100EOS5D\*.RAW M:\CineForm\*.AVI -bcL:\DCIM\100EOS5D\M20-2205.RAW
Quote from: DANewman on June 21, 2013, 05:47:33 PM
Not yet, but I have someone interested in doing a Mac port.
;D
Cool, I'd like to try it out. I shoot a lot of low key scenes so the black balancing sounds very interesting.
I've read this thread front to back, but I still have little to no clue how to navigate this RAW to Cineform process. Can someone either explain this to me step-by-step like I'm a 4 year-old, or provide a link to the appropriate thread/post?
Thanks.
What platform are you trying to use? Have ever used a DOS shell or command prompt? Helps to know how far back to go?
Thanks. I'm on a Windows 7 PC.
I've opened up cmd and typed 'raw2gpcf' and hit enter, at which point several lines of text appears describing different prompts. I typed the name of my file, followed by the name of the new file I wished to create, and ended with -444 (I have Studio Premium). Once I clicked enter, it just said it couldn't read my raw file (I attempted with several from the 5D Mark III and 50D). Here's an example of what I typed: 'raw2gpcf X.raw Y.mov -444'
My ultimate goal is to perform a batch process on an entire card of raw files. Do I need any other software besides Studio Premium and your raw2gpcf? Do I need Rawanizer?
Thanks.
The only step you are missing is to include the path to your media and destination. So if your copy RAW to M:\media you command would be
Raw2gpcf M:\media\X.RAW M:\media\Y.mov -444
Thanks! I'll give 'er a go...
So how do I go about performing a batch process on several raw files at once? Thanks.
Quote from: pinger007 on June 24, 2013, 02:48:27 AM
So how do I go about performing a batch process on several raw files at once? Thanks.
Raw2gpcf M:\media\*.RAW M:\media\*.mov -444
Quote from: DANewman on June 21, 2013, 05:36:00 PM
The black balancing will remove any remaining fixed pattern noise (FPN) the sensor can produce. Canon 5D3 is very good a suppressing FPN. In my own 5D3 experiments, the black balancing did remove the hot pixels that appear at higher ISOs.
Worked perfectly ;D And it fixed the dead pixels for me as well. Thanks!
Now I hope more people can use it, as I think is handles all the current requests and issues. I know, I know, Mac version is still needed.
Quote from: DANewman on June 25, 2013, 11:15:25 PM
Now I hope more people can use it, as I think is handles all the current requests and issues. I know, I know, Mac version is still needed.
I'll be uploading a whole workflow thingy to YouTube pretty soon. I think I've got a few followers who'd dive at this tool. Again, many thanks! :D If you don't mind my asking, has there been any progress with the search for some different debayer algorithms?
Not yet on the demoasic filter, only found GPL, not LGPL/MIT licensed examples, or any nice algorithms not patent encumbered.
Quote from: DANewman on June 26, 2013, 05:46:16 AM
Not yet on the demoasic filter, only found GPL, not LGPL/MIT licensed examples, or any nice algorithms not patent encumbered.
Okay, this time pretty sure the demosaic is LGPL. Both were added to the repo recently. Im hoping to God these ones work haha
https://github.com/LibRaw/LibRaw/blob/master/internal/aahd_demosaic.cpp
https://github.com/LibRaw/LibRaw/blob/master/internal/dht_demosaic.cpp
Poked around the repo some more, thought this might be something worth looking at. I have no idea what WF is, but I'm guessing it's a type of banding.
https://github.com/LibRaw/LibRaw/blob/master/internal/wf_filtering.cpp
Good finds, these have possibilities. Which sources are having banding issues, as I'm not see that?
Quote from: DANewman on June 26, 2013, 05:33:47 PM
Good finds, these have possibilities. Which sources are having banding issues, as I'm not see that?
I'm not getting any from my cam. I just thought it might be good in situations where the dark frame subtraction doesn't get all of the vertical banding. I'm probably mistaken though. Question about dark frame subtraction: should we match every single exposure component (shutter, iso, aperture) to the normal clip when making dark frames, or can we just do ISO? Also, do we record the dark frames at infinity focus or does it matter? And glad to hear those debayer algorithms have some potential!
Black balance is effected by ISO and shutter, not aperture or focus (with the lens cap on, both don't matter.) I'm thinking to automate the black file selection so that a CF load could be automatic, using the most recent black balance file.
Quote from: DANewman on June 26, 2013, 10:27:22 PM
Black balance is effected by ISO and shutter, not aperture or focus (with the lens cap on, both don't matter.) I'm thinking to automate the black file selection so that a CF load could be automatic, using the most recent black balance file.
Okay, thanks for the explanation. That automation would require us not to move our dark frames right? So would it be smart to make dark frame recordings and make them their own folder?
I was think reading the first frames on every *.RAW file on an CF Card that is under 30 seconds long, if one is all black, that is you calibration file. You could still specify with -bcFilename.RAW, but you could to wildcard with auto-black-balancing like this
RAW2GPCF L:\DCIM\100EOS5D\*.RAW M:\CineForm\*.AVI -bc
By dropping the file info from the -bc, switch it could (not done yet) just auto scan the L:\DCIM\100EOS5D\*.RAW source. You would need to get into the practice doing a black black per CF load (or not deleting your calibration file.)
Quote from: DANewman on June 27, 2013, 01:57:38 AM
I was think reading the first frames on every *.RAW file on an CF Card that is under 30 seconds long, if one is all black, that is you calibration file. You could still specify with -bcFilename.RAW, but you could to wildcard with auto-black-balancing like this
RAW2GPCF L:\DCIM\100EOS5D\*.RAW M:\CineForm\*.AVI -bc
By dropping the file info from the -bc, switch it could (not done yet) just auto scan the L:\DCIM\100EOS5D\*.RAW source. You would need to get into the practice doing a black black per CF load (or not deleting your calibration file.)
That sounds good. Is there a way to make a check sequence? So if the program doesn't find the black frame file on the card, it reverts back to the latest one used. Also, does it matter if the dark frame file is of a higher resolution than the normal recorded files? If it doesn't, that saves some time for the people that have to lower resolution to shoot higher fps.
After thought: would searching through the card for the black frame file slow down the conversion process?
Resolution must match also. So black balance for high speed separately.
Scanning will be fast, faster than typing the name of the clip manually.
Quote from: DANewman on June 27, 2013, 03:12:00 AM
Resolution must match also. So black balance for high speed separately.
Scanning will be fast, faster than typing the name of the clip manually.
That's plenty fast for me then!
Going off the topic of black calibration for a second, is the program automatically applying the 5D3 color matrix? Because on my T3i footage, I get off colors when leaving the matrix to Source.
It is applying the color matrix that in stored in the .RAW file. If the color matrix is wrong for the T3i, the ML build for T3i needs to be updated for for correct matrix.
Okay, one more question. Is the white balance in Studio based on Kelvin or it is some other scale?
Kelvin value of the incident light, although the math was a little off in Studio 1.3, and it improved in the upcoming 2.0.
Okay was making sure I wasn't doing wrong. But 2.0 ;D Any need for a test monkey?
Quote from: DANewman on June 27, 2013, 03:12:00 AM
Resolution must match also. So black balance for high speed separately.
Scanning will be fast, faster than typing the name of the clip manually.
Hi Dan,
i was wondering if it is possible to prepare different clips with the cap on at the highest iso.
Like RAW800iso.RAW, RAW1600iso.RAW, RAW3200ISO.RAW, RAW6400ISO.RAW
and when we have a high iso footage we can apply the
"-bc RAWISOXXX.RAW" to remove the pattern noise.
Would it be good to speed up the workflow?
Another thing, i don't know why but when i open the normal -422 cineform clip in my MAC i see the pink cast in highlight.. whether it is not present in windows.. so the only thing i can do to remove it for my FCPX usage is to convert with l90 or l400 parameter.
is it normal?
thanks a lot!
Yes you could calibrate your camera for different ISOs. We need the ML RAW header to change to make this more useful. If the 192-byte RAW header included the ISO, I could automatically load to correct black balance file based on the ISO. Who do we ask?
The pink highlight requires you to tweak the Sat. Clip point., which is a control within the Premium user interface. I could add a control to set this default level from the command line -- it is non-destructive metadata.
It is very hard to kill the pink highlight..
i don't have a premium version right now.
this is what it looks like the horrible burn magenta channel in the candle light
https://www.evernote.com/shard/s52/sh/451006c7-7b9e-40ce-85bf-68a9be9ddd94/0a447db37aad41370655697e65b18819
(https://www.evernote.com/shard/s52/sh/451006c7-7b9e-40ce-85bf-68a9be9ddd94/0a447db37aad41370655697e65b18819)
i have tried the -l400 parameter... the l90 parameter which is slightly better but not so much.. and the -c parameter which is very bad..
any advices?
Today my advice is for you to use the Premium version, as the free version has only minimum corrections for RAW. Or you could turn off the color matrix, which removes the highlights issue, but you have to start with uncalibrated color.
Turn off the color matrix in the free version by clicking on the None preset in the bottom right of the GoPro CineForm Studio. You can batch this. After None is applied (any custom metadata), Ctrl+C (to copy), Ctrl+A (to select all) and then Ctrl+V to paste this default to all for CineForm clips.
New version v1.11
* Added automatic black calibration file detection with '*' wildcards.
* Added metadata injection.
http://miscdata.com/ML/RAW2GPCFv111.zip
Now to black balance, just record a less than 1GB clip with the lens cap on and the wildcard filename will find it
raw2gpcf L:\DCIM\100EOS5D\*.RAW M:\CineForm\*.AVI -bcL:\DCIM\100EOS5D\*.RAW
The metadata switches are really for advanced or future hacks. However for those have trouble with pink highlights, and who are not using Studio Premium (your loss), you can override the default highlight saturation clipping point with +HPNT=## (something between 0.3 and 0.9 should work, with 0.0 and 1.0 the maximum, default is 0.5.) This are linear light numbers so if green clips one stop before red and blue, 0.5 would be the clipping point. If green clips 2 stops before (unlikely) then 0.25 is the approximate clipping point.
I've tried bl balancing on 5d2 footage, and it looks abit off (?).
Seems like it adds balancing noise to it, not subtracts somehow.
Here is the link to before/after image. The crop has overbright/greytone/shadows data:
http://i.fotto.ru/ysjv.jpeg (http://i.fotto.ru/ysjv.jpeg)
So, it raises blacks + adds that 'nice' magenta cast to the shadows.
What do you think, Dave? Dunno maybe it is ok behavior.
Hi guys, first post on this forum, so please go easy. I'm on PC,Windows 7 Home Premium, GoPro Studio Premium,Creative Cloud.
The batch v1.11 converts ony first file and then crashes, so I only get one file at a time,can someone tell me what I am doing wrong?
Quoteconverting in place (on disk in the same folder, 'cd' to that folder)
raw2gpcf *.RAW *.AVI
When I import to GoPro Premium the converted files with RAW2GPCF v1.11 i do not see a preview in the software but changes do get applied to this file when I've got it on Premiere CC timeline and move a frame forward ( but with with issues around keyframes see other issue below ). Should I be seeing a preview in CF Premium upon importing it just like after converting a RAW DNG sequence from step 1 to 2 ?
Regarding the keyframe issue above, when I change any preset LUT in GoPro Premium ( trial for the moment) it kind of creates kayframes on the file in Premiere CC. I have to create a keyrame in GoPro Premium and delete it, for the clip to have the same preset on the whole clip now in Premiere CC. Is it weird?
Anyway, great work, lloks like as soon as those last bits and pieces get fixes I'll be buying the Premium as I can just edit these files with full resno frame dropping preview in Premiere CC on my i7 920 with GTX 285 2GB.
Great job David and the rest of Cineform guys!
Quote from: Yaros525 on June 29, 2013, 06:27:11 PM
I've tried bl balancing on 5d2 footage, and it looks abit off (?).
Seems like it adds balancing noise to it, not subtracts somehow.
Here is the link to before/after image. The crop has overbright/greytone/shadows data:
http://i.fotto.ru/ysjv.jpeg (http://i.fotto.ru/ysjv.jpeg)
So, it raises blacks + adds that 'nice' magenta cast to the shadows.
What do you think, Dave? Dunno maybe it is ok behavior.
It is doing the correct thing, although I might have been too generous with black offset when I quickly tested with my 5D3. It only looks more noisy as the black is lifted (hoping to dig more out of the shadows), but the black balance is helping, it you color correct to the same, the balance balanced image is slightly better. Now for the pink tint in the shadows, the black has no inherent tint, yet the white balance and color matrix can cause this when the black is offset too much. So I've halved the default black offset from 16 to 8, and added that as a user control.
http://miscdata.com/ML/RAW2GPCFv112.zip
you could use -bo0 to remove the black offset, or even -bo-10 to crush the blacks a little, all shadow tints will go.
Quote from: MichalT on June 29, 2013, 08:36:01 PM
Hi guys, first post on this forum, so please go easy. I'm on PC,Windows 7 Home Premium, GoPro Studio Premium,Creative Cloud.
The batch v1.11 converts ony first file and then crashes, so I only get one file at a time,can someone tell me what I am doing wrong?
When I import to GoPro Premium the converted files with RAW2GPCF v1.11 i do not see a preview in the software but changes do get applied to this file when I've got it on Premiere CC timeline and move a frame forward ( but with with issues around keyframes see other issue below ). Should I be seeing a preview in CF Premium upon importing it just like after converting a RAW DNG sequence from step 1 to 2 ?
Regarding the keyframe issue above, when I change any preset LUT in GoPro Premium ( trial for the moment) it kind of creates kayframes on the file in Premiere CC. I have to create a keyrame in GoPro Premium and delete it, for the clip to have the same preset on the whole clip now in Premiere CC. Is it weird?
Anyway, great work, lloks like as soon as those last bits and pieces get fixes I'll be buying the Premium as I can just edit these files with full resno frame dropping preview in Premiere CC on my i7 920 with GTX 285 2GB.
Great job David and the rest of Cineform guys!
Fix that new crash.
http://miscdata.com/ML/RAW2GPCFv113.zip
Don't use keyframes for that as you are trying to compensate for Premiere's annoying frame cache. When using Studio and Premire Pro together, it takes a while for the metadata changes to be correctly refreshed in Adobe tools.
hi everyone, i drag a raw file into the import window and all i get is a white image. any help?
RAW image (as in .RAW) are not supported in the import room. If you .DNG that will work. If you RAW2GPCF (that 95% of this thread is about) you don't need to use the import room, go directly to the edit room. If you doing everything correctly, and you have solid white in the edit room, and you're on the PC, you need to update you graphics drivers.
DANewman
Can you incorporate the upscaling mechanism that was once in NeoHD - RAW2GPCF ? (for us guys with lesser cameras e.g. T3i)
Thanks David for your response,
V 1.13 worked like a treat converting all, but still no preview in Studio Premium upon import in Step 2.
I just added Protune preset blindy ( well i can see the little icons in the preset box actually previewing the footage,bo not the big preview),
and only about a half of my footage import in premiere CC, the other half says "Unsupported format or damaged file" and I just can't import it. Help please.
The above is just so much faster,please don't make me go back to the currently working workflow of :
1.converting RAW files via raw2cdng into it's own folders of dng files
2.importing the sequences into Step 1 in Studio Premium
3.Converting and previewing in Step 2.
This way does work but I end up with 2x the data plus CF RAW files, but it works and has a preview in Studio Premium and all imports just fine to Premiere CC.
I will need sample data that doesn't show in Studio or doesn't import into Premiere. Not seeing either scenario.
David,
Here's a link to my Google Drive shared folder, with one of them files. Converted with v1.13,no big preview in Premium and unable to import into Premiere CC.
https://drive.google.com/folderview?id=0By_KH2cOvbf_Y3RwUXJqZmNUNFk&usp=sharing
Thank for you help
Quote from: AriLG on June 30, 2013, 09:29:34 AM
DANewman
Can you incorporate the upscaling mechanism that was once in NeoHD - RAW2GPCF ? (for us guys with lesser cameras e.g. T3i)
There could be scaling added for non-RAW encodes, but scaling can been done in the NLE without consequences.
Okay, I got my first sample with hot pixels since black calibration has been added. I was able to work around this by going RAW -> DNG -> CF Studio and using dead pixel removal. But the DNG doesn't retain the color matrix because Studio doesn't show correct color like raw2gpcf.
https://drive.google.com/folderview?id=0Bytymz6r53EjczB4a3cwNGoxTFE&usp=sharing
I'm in an airport with very poor bandwidth, so I'll try and download later. Studio via DNG with apply color matrix, except the RAW to DNG tools aren't transfering the color matrix metadata (it is in the .RAW) As for the black balance, that shouldn't be adding bad pixels, but fixing many of them.
Quote from: DANewman on July 01, 2013, 09:02:31 PM
I'm in an airport with very poor bandwidth, so I'll try and download later. Studio via DNG with apply color matrix, except the RAW to DNG tools aren't transfering the color matrix metadata (it is in the .RAW) As for the black balance, that shouldn't be adding bad pixels, but fixing many of them.
Okay, so the raw -> dng apps haven't fixed matrix transfer yet. And the black balance isn't adding hot pixels, it's just not removing them. Sorry if I didn't word that correctly in the last post.
Are the hot pixels on the black balance clip? Did the hot pixels occur after you recording the black balance? Wondering if it is a software failure or a calibration failure.
Quote from: DANewman on July 01, 2013, 09:14:56 PM
Are the hot pixels on the black balance clip? Did the hot pixels occur after you recording the black balance? Wondering if it is a software failure or a calibration failure.
Yes, there are hot pixels in the black balance clip. I replicated the results with another clip just to make sure. raw2gpcf tells me the average of the black balance frames and does its thing. It removed the FPN, but left the hot pixels.
Quote from: MichalT on June 30, 2013, 11:59:54 PM
David,
Here's a link to my Google Drive shared folder, with one of them files. Converted with v1.13,no big preview in Premium and unable to import into Premiere CC.
https://drive.google.com/folderview?id=0By_KH2cOvbf_Y3RwUXJqZmNUNFk&usp=sharing
Thank for you help
The file is working here in Studio Premium, I expected it would as I white screen indicates an OpenGL driver issue (updates can help.)
I don't have CC, but you may need to install CineForm importers manually as our installer as only good for CS6 and earlier.
http://support.cineform.com/entries/24341786-A-Do-It-Yourself-Guide-to-adding-CineForm-support-in-Adobe-CC-Mac-
Quote from: iaremrsir on July 01, 2013, 10:49:41 PM
Yes, there are hot pixels in the black balance clip. I replicated the results with another clip just to make sure. raw2gpcf tells me the average of the black balance frames and does its thing. It removed the FPN, but left the hot pixels.
I see dead pixels. Hot pixel charge up, and are visible on black, dead pixel never charge up and a visible on white. Maybe we need to add a white flat fielding pass, or at least a dead pixel mask.
Quote from: DANewman on July 01, 2013, 11:17:50 PM
I see dead pixels. Hot pixel charge up, and are visible on black, dead pixel never charge up and a visible on white. Maybe we need to add a white flat fielding pass, or at least a dead pixel mask.
What does flat field correction entail? Would we need to shoot something similar to how we shoot the dark frame, just with the sensor fully saturated? What does Studio use?
Hi Guys,
inside RAWANIZER i have set this parameter with the new Bc setting
"{OutputVideoName}.mov" -422 -l400 -bcX:\RAW\ISO 800.RAW
could it works for you?
actually it seems to convert.. even if with the actual file i am testing i cannot appreciate a shadow noise remove.
besides at the end of conversion i obtain the cineform file but the prompt shows this error
"Failed to Encode, do you have gopro cineform studio installed?"
actually i have ! so i wonder why this error appear at the end of the enconding process.
Quote from: iaremrsir on July 02, 2013, 12:43:23 AM
What does flat field correction entail? Would we need to shoot something similar to how we shoot the dark frame, just with the sensor fully saturated? What does Studio use?
I don't know, try capturing a bright light source with the lens off the camera. The resulting image should be a mask of all the dead pixels. Care to try and send me, only need a dozen frames.
Quote from: mclein on July 04, 2013, 12:29:02 AM
Hi Guys,
inside RAWANIZER i have set this parameter with the new Bc setting
"{OutputVideoName}.mov" -422 -l400 -bcX:\RAW\ISO 800.RAW
could it works for you?
actually it seems to convert.. even if with the actual file i am testing i cannot appreciate a shadow noise remove.
besides at the end of conversion i obtain the cineform file but the prompt shows this error
"Failed to Encode, do you have gopro cineform studio installed?"
actually i have ! so i wonder why this error appear at the end of the enconding process.
Anyone else experience this error?
Quote from: DANewman on July 04, 2013, 05:29:13 AM
I don't know, try capturing a bright light source with the lens off the camera. The resulting image should be a mask of all the dead pixels. Care to try and send me, only need a dozen frames.
At first glance it looks fully saturating the sensor revels the deal pixels. We just have to make sure that the entire capture area is covered and even. I'll upload when I get to my computer.
Also, for FFC do we need multiple exposures? I'd assume not, unless the dead pixels vary across exposure times.
EDIT: On second thought, shutter may not need to be changed, but ISO might. Will the change in sensor sensitivity cause other pixels to "die" as well? Or will all of the dead pixels revealed at the lowest ISO be the same across the board?
Found a bug that repeatedly causes the "do you have studio installed" message to show up. The program has a problem when you try to have it convert to a non-existent folder. It won't automatically create the folder. If you make the folder beforehand, the error goes away. Also, there's another bug. Not sure if it's a Windows problem, but it only occurs with CF programs when converting files. It seems to search for drives that don't exist on my computer causing the "No Disk" error every time it converts a new file.
Upon further inspection of the clips, the dead pixels don't change across resolution. So if we were to record the highest resolution possible at the different ISOs, we could use those clips for any resolution below it.
Here's the white frame. It's the highest res my cam will go. 1728x992
https://docs.google.com/file/d/0Bytymz6r53EjdUtZcVBtczFQUzQ/edit?usp=sharing
...
Sorry I've been away. I look into the last few posts when I return in a few days.
Im using the latest version of raw2gpcf (1.13) and I bought a cineform premium lic.
I am getting pink highlights, any raw2gpcf options to avoid this? thank you in advance
In workspace use the sat. Clip point control to remove the pink highlights.
Any updates on flat field correction and the debayer algorithms?
Quote from: DANewman on May 29, 2013, 06:08:32 AM
I expect you are have issues with Cineon as an encoding curve, it is not the optimal choice as it is not very efficient for using the available codewords. Hightlight are not blown out, some are just above 1.0, in RAW they can be restored. Open Studio and try reducting the gain or exposure, all the highlights are still there. You can encode Protune, and decode to Cineon if you like, that is better than encoding Cineon.
-l90 -- SI-2K curve
-l400 -- Protune 12 stop optimized
-l900 -- Protune 13 stop optimized
While the default Protune log curve and -l400 curve both store 13+ stops, it is just a matter of emphasis, and how much do you need to dig into the shadows at the mild expense of midtones and highlights.
I've noticed a bug, when I try to convert files as -l400 for 12 stops in raw2gpcf, they show up as Custom Log 255 for encode and decode. I'm guessing Custom Log is an extension of the Protune curve. Also, I noticed some functionality things when working with CF Raw Protune in AE. When in 32 bpc I can see that there are values above 1.0.
I can recover these slightly with the tools in AE. But, it doesn't recover all of the highlights like the Cineon 95-685 curve does. I like how Protune looks and is easy to grade and work with.
I'd just like to figure out how to keep the files completely Protune and fully recover highlights. Changing the Sat Clip Point slider from .5 makes the highlights pink or green.
Edit: When working in 32bpc, AE can read all of the raw data as if it has only been debayered but still does not have a curve applied. So we have access to the entire dynamic range with Protune even if there seems to be some clipping in Studio Premium. If the highlights are recoverable in ACR, they're recoverable with CF Raw in AE at 32bpc. I had it wrong before as I was trying to use the HDR Highlight Compression plugin; I should've used the Exposure plugin. This will keep detail in the Highlights without making the mids all plasticky. Still haven't figured out how the Sat Clip Point slider works I guess, because I can't get it to recover highlights.
The -l400 will be support with GoPro Studio 2.0 software out soon -- I guess that wasn't supported within v1.3.2.
Could you send me a short CineForm clip that you can't correct with the saturation clip point. I've never seen that.
@DANewman:
BUG: when the specified output directory does not exist, it says "check if cineform studio is correctly installed".
Maybe you could change it to something more conclusive like "output directory does not exist" :)
Quote from: DANewman on July 17, 2013, 04:42:54 AM
The -l400 will be support with GoPro Studio 2.0 software out soon -- I guess that wasn't supported within v1.3.2.
-l400 works when converting DNGs directly inside of Studio, but setting -l400 in raw2gpcf doesn't work for me. It always comes out as Custom Log 255.
Quote from: DANewman on July 17, 2013, 04:42:54 AM
Could you send me a short CineForm clip that you can't correct with the saturation clip point. I've never seen that.
Guess I should specify that it's only with clips that come from raw2gpcf. When I go raw -> DNG -> CFR, I get no problems with magenta or green highlights when using Sat Clip Point. But it also seems like Sat Clip Point has no effect on the CFR files that came from DNGs because it seems like all of the highlight data is there even when Sat Clip Point is at 0. I can recover highlights in AE just fine on both. Also I've noticed that recovered highlights from raw2gpcf clips go toward yellow instead of white.
Here are some clips that show the problem. All are at default import settings. Also note the ones that come from DNGs are clean. These also show the Custom Log 255 problem.
https://drive.google.com/folderview?id=0Bytymz6r53EjN2lnTkdIQXNxVzQ&usp=sharing
I mentioned this before, the raw - > DNG -> CFR is not using the camera's color matrix, so the comparison is with or without color matrix -- pink highlights relate to the color matrix and white balance. If you prefer to color correct without the color matrix you can do that on RAW2GPCF conversions. In the workspace area of Studio Premium you can select color matrix as "Neutral". If you are using free version of GoPro Studio, click on the "None" preset.
Quote from: DANewman on July 23, 2013, 07:07:57 PM
I mentioned this before, the raw - > DNG -> CFR is not using the camera's color matrix, so the comparison is with or without color matrix -- pink highlights relate to the color matrix and white balance. If you prefer to color correct without the color matrix you can do that on RAW2GPCF conversions. In the workspace area of Studio Premium you can select color matrix as "Neutral". If you are using free version of GoPro Studio, click on the "None" preset.
I get that; I was trying to figure out how to recover highlights in protune with sat clip point. But I guess it will always go to pink? I like grading from the raw2gpcf clips because they do include the source matrix. But I guess I'll just have to recover highlights in AE or whichever program I'm working in. Also, are the yellow highlights a matrix/color temp problem? Because everything else looks correct.
It is not really a matrix problem nor is it a raw2gpcf problem, it to be expected when the green channel clips in camera. It is easy to address in post, plus it teaches us to underexposure a little more.
Quote from: DANewman on July 24, 2013, 04:00:31 AM
It is not really a matrix problem nor is it a raw2gpcf problem, it to be expected when the green channel clips in camera. It is easy to address in post, plus it teaches us to underexposure a little more.
Okay, in post would I just use something like levels or a highlights adjustment to bring it to where all oh the channels clip? And what would we do for the unavoidable clips like speculars, the sun, etc?
I've not seen an image that can't be corrected with primaries and saturation clip point.
RAW as the camera shot it.
https://www.dropbox.com/s/jg5rkyxmwa6gzyd/RAWcolor.jpg (https://www.dropbox.com/s/jg5rkyxmwa6gzyd/RAWcolor.jpg)
Turning on White balance and the color matrix from camera metadata, with Sat. Clip point off (1)
https://www.dropbox.com/s/7t7aacdqpoxcqqg/CameraMatrix%2BCameraWB.jpg (https://www.dropbox.com/s/7t7aacdqpoxcqqg/CameraMatrix%2BCameraWB.jpg)
Primaries to remove clipping and fix shadows and mid-tones
https://www.dropbox.com/s/yvxf6hg2ipzz6z4/CameraMatrix%2BCameraWB%2Bprimaries.jpg (https://www.dropbox.com/s/yvxf6hg2ipzz6z4/CameraMatrix%2BCameraWB%2Bprimaries.jpg)
Setting Saturation Clip point
https://www.dropbox.com/s/uzwxaduent435qz/CameraMatrix%2BCameraWB%2Bprimaries%2BSatClipPoint.jpg (https://www.dropbox.com/s/uzwxaduent435qz/CameraMatrix%2BCameraWB%2Bprimaries%2BSatClipPoint.jpg)
If the sun or clipped object was in the scene, it would be correctly clipped to white.
Quote from: DANewman on July 24, 2013, 09:35:16 PM
I've not seen an image that can't be corrected with primaries and saturation clip point.
RAW as the camera shot it.
https://www.dropbox.com/s/jg5rkyxmwa6gzyd/RAWcolor.jpg (https://www.dropbox.com/s/jg5rkyxmwa6gzyd/RAWcolor.jpg)
Turning on White balance and the color matrix from camera metadata, with Sat. Clip point off (1)
https://www.dropbox.com/s/7t7aacdqpoxcqqg/CameraMatrix%2BCameraWB.jpg (https://www.dropbox.com/s/7t7aacdqpoxcqqg/CameraMatrix%2BCameraWB.jpg)
Primaries to remove clipping and fix shadows and mid-tones
https://www.dropbox.com/s/yvxf6hg2ipzz6z4/CameraMatrix%2BCameraWB%2Bprimaries.jpg (https://www.dropbox.com/s/yvxf6hg2ipzz6z4/CameraMatrix%2BCameraWB%2Bprimaries.jpg)
Setting Saturation Clip point
https://www.dropbox.com/s/uzwxaduent435qz/CameraMatrix%2BCameraWB%2Bprimaries%2BSatClipPoint.jpg (https://www.dropbox.com/s/uzwxaduent435qz/CameraMatrix%2BCameraWB%2Bprimaries%2BSatClipPoint.jpg)
If the sun or clipped object was in the scene, it would be correctly clipped to white.
That's why I couldn't get it to work! Thanks, makes so much more sense to me now.
I also get "Failed to encode. Do you have GoPro Cineform Studio Installed". This is in rawanizer after I updated it to the latest so perhaps it's their fault.
Quote from: tobyloc on July 28, 2013, 05:14:36 PM
I also get "Failed to encode. Do you have GoPro Cineform Studio Installed". This is in rawanizer after I updated it to the latest so perhaps it's their fault.
There is bug that shows that error if you get any if the paths wrong.
Any news or updates? Dead pixel fix? Demosaic algorithms?
Found a bug in GoPro Studio Premium 1.3.2.170:
Green / yellow fringing jumping around bright edges when the "Despeckle / Dead Pixel Fix" is activated and the slider is set to anything above 0. It's a bummer because it really removes Dead/Hot Pixels but adds these artifacts to the clips. When it's set to 0 the image is just right.
PS. I use DNGs from the Canon 5D Mark III converted with RAWanizer 0.6.1
Btw. this green fringing is also visible in DaVinci Resolve 9.1.5 but not in Adobe Camera RAW > http://www.magiclantern.fm/forum/index.php?topic=7399.msg63877#msg63877
(http://www.magiclantern.fm/forum/index.php?topic=7399.msg63877#msg63877)
(http://www.schweinert.com/upload/dead_pixel_fix_comparison.jpg)
That is a demosaic problem like I was having. Supposedly people are having luck using raw therapee or anything with the AMAZE algo... I have to try this on my speckle footage.
I have confirmed there was a bug in the despeckle code within the 1.3.2 version of CineForm Studio. 2.0 will be here in a few weeks, and it should addresses this failure.
hey there dont know if im right here but its a try worth :D
i build my own MiniRAW1 Camera which shoots 720p 120fps 12bit raw
my workflow is a little bit awkward:
1. i record from the camera controll software and stream .raw files
2. a friend of mine and i made a .raw converter on mac which convertes the 12bit .raw to .dng which works pretty good resolve and adc can open and edit the files.
but with 120fps and each frame is 1,4MB my HDD is nearly full and so i found the cineform RAW thing which reminds me a little bit of Redcode :-)
no here is the problem:
when i import a folder which contains the .dng sequenz into the cineform studio pro trial version 1.3 on my windows 7 64bit machine the first thing is that i cant see the preview and no matter what i put into the settings (avi, mov) the second i click on convert the program crashes :-(
any idea what i can try? cheers and thx janosch
attached is a sample dng which crashes cineform studio pro
https://copy.com/t1lnnyhpnPjp
cheers janosch
You have build your own MiniRAW1 Camera which shoots 720p 120fps and record at 12bit raw?
What camera is this? Do you have photos or any link for that?
here is my project a little bit described :-)
http://www.eoshd.com/comments/topic/2874-my-miniraw-1-camera-needs-help/?hl=miniraw
the camera is from point grey a flea 3 usb3
the lens on that dng no idea :D this was a very early testshot but i guess it was an fujinon tv 16 f1.4 c-mount lens or an canon j6x12 12.5-75mm
but now that i build a "case" around it i made my own mft mount so i can use lots of fine glass :-)
cheers janosch
AMAZING !!!
Quote from: arrinkiiii on August 25, 2013, 04:52:32 PM
AMAZING !!!
thx dude now it would be awesome if somebody can help me with cineform raw :-)
or even more better to make a .raw converter directly to cineform raw :D
cheers janosch
I keep getting a message when trying to convert my files using RAWanizer asking if I have cineform studio downloaded, and I do. Is there something im missing?
Quote from: Caleb_camera on August 27, 2013, 08:11:48 AM
I keep getting a message when trying to convert my files using RAWanizer asking if I have cineform studio downloaded, and I do. Is there something im missing?
I had this problem too. If my memory serves me right, it was fixed by reinstalling either Rawanizer and/ or Cineform.
I tried reinstalling everything with no luck. Looks like I will re-download Windows 7 on vmware.
Quote from: janoschsimon on August 25, 2013, 03:45:39 PM
hey there dont know if im right here but its a try worth :D
i build my own MiniRAW1 Camera which shoots 720p 120fps 12bit raw
my workflow is a little bit awkward:
1. i record from the camera controll software and stream .raw files
2. a friend of mine and i made a .raw converter on mac which convertes the 12bit .raw to .dng which works pretty good resolve and adc can open and edit the files.
but with 120fps and each frame is 1,4MB my HDD is nearly full and so i found the cineform RAW thing which reminds me a little bit of Redcode :-)
no here is the problem:
when i import a folder which contains the .dng sequenz into the cineform studio pro trial version 1.3 on my windows 7 64bit machine the first thing is that i cant see the preview and no matter what i put into the settings (avi, mov) the second i click on convert the program crashes :-(
any idea what i can try? cheers and thx janosch
attached is a sample dng which crashes cineform studio pro
https://copy.com/t1lnnyhpnPjp
cheers janosch
Sorry, I was away (went NY)
Unfortunately I don't have Cineform Studio (never used it). My guess is it must crash because of incorrect/missing tags in the DNG header?
Quote from: TheUnkn0wn on September 01, 2013, 06:01:51 PM
Sorry, I was away (went NY)
Unfortunately I don't have Cineform Studio (never used it). My guess is it must crash because of incorrect/missing tags in the DNG header?
hm dont know with dng_validate.exe from adobe i compared my dng with one from bmcc which works with cineform and i dont know which tag the problem or might be the problem cause the second i click on convert the cineform software crashes :-/
cheers janosch
Is the last raw2gpcf.exe version compatible with studio 2? Are there any plans to add support for the new raw file format?
I had an issue that I couldn't find here!!
I downloaded GoPro cineform studio pro Trial.. and I was converting .RAW to .AVI succesfully for a few hours.. but suddenly.. I got the error "Fail to encode, is GoPro CineForm Studio installed?". No matter what I do.. reinstall, download latest versions.. change computers.. same error always!!! damn!! why?
Using: Rawanizer, Gopro Studio v1.3 (now tested with 2.0), and RAW2GPCF v113
Any advice please?
Thanks!
Nevermind got it working!! It seems there's a bug.. if you specify a folder in "Video/proxy files target folder" then you get that error.. too bad!!
GoPro Studio 2.0 is out. Time I get back into ML RAW.
What is this "new raw file format" you speak of?
MLV.. its just raw with a wrapper and non sequential frames.
Add any new debayers?
Is it commonly being used (MLV)? Where is the specification?
Revisiting debayer soon.
Quote from: DANewman on September 13, 2013, 12:04:21 AM
Is it commonly being used (MLV)? Where is the specification?
Revisiting debayer soon.
It's not part of the main repo yet but it's being tested on various cameras.
G3gg0 has a thread about it here: http://www.magiclantern.fm/forum/index.php?topic=7122.0 (http://www.magiclantern.fm/forum/index.php?topic=7122.0)
and deeper info here: https://docs.google.com/spreadsheet/ccc?key=0AgQ2MOkAZTFHdHJraTVTOEpmNEIwTVlKd0dHVi1ULUE#gid=0 (https://docs.google.com/spreadsheet/ccc?key=0AgQ2MOkAZTFHdHJraTVTOEpmNEIwTVlKd0dHVi1ULUE#gid=0)
Hello there Dan,
Your command line utility is amazing. That alone pushed me over the edge to buy a studio premium license. I was an old neoscene user.
I have a couple of questions...
First, I noticed that you mentioned earlier you used cineform with a surface pro. How's that working out for you? Is the tablet fast enough to play cineform raw clips at full resolution or should I wait for Haswell?
And second, is there an inexpensive way to add a 3rd computer to my studio premium license? The old 2 machine paradigm that the industry has embraced for so long should be reformulated because now individuals need to install software on a desktop, laptop and tablet ;-)
Cheers.
contact
[email protected] about license extensions.
I like the Surface Pro, but do all my editing at half res as it is no desktop replacement. Yet I do a lot of edit with it.
@DANewman - Do we still need to use the hacked CFHDdecoder64.dll with Resolve or is it incorporated in CF2?
RAW2GPCF converted clips appear uncorrectly within newly-released Resolve 10. They are "all shades of pink" , and unusable for now. (
Sounds like a bug to be filed with blackmagic.
I just bought a surface pro and love it, but cineform studio is not working for me, it just hangs. :(
EDIT: Solved. Why would Microsoft not want to use intel's latest display drivers??????
Yes the Surface Pro is awesome little ultrabook with the correct drivers
Yay! Recent Resolve 10 beta 3, Cineform support is back! Works as intended.
Quote from: DANewman on May 27, 2013, 04:08:54 AM
Yes, the resulting images will have baked color. Not a bad thing if you have all of the information you need to do your finish, but RAW in my opinion requires you to be able to restore back to everything the sensor read, giving you the greatest flexibility.
I don't consider applying the demosaic as a necesary part of RAW. Consider the Protune CAM_RAW mode on GoPro HERO3 Black Edition (something I worked on,) that is about as raw as you can get in an H.264 file. All the demosaic is applied in camera, but color matrix and white balance are applied in post, just like classic uncompressed RAW.
If there is enough interest, I will enable YUV or 444 development, without baking the color. The only reason you might like 4:2:2 RAW, is that could run on the freely distributed GoPro CineForm Studio, just like Protune CAM_RAW works for GoPro users today.
Any updates or chances of this happening? This plus the Adobe CC update would be huge for workflow and quality. Bring DNG files into Media Encoder CC, use Adobe's debayer on the GPU for a speedy output to a debayered, semi-raw CF file. 8)
I really wish more devices used CineformRAW natively, both at acquisition and playback stages. It affords a terrific balance between quality (very little compromise vs uncompressed frames like DNG), and perfectly low bandwidth/storage requirements. Bravo! IMHO David Newman is a genius, and his Cineform team is awesome.
I personally am using Cineform for years, and even bought SI-2K camera in large part because it recorded into CineformRAW natively.
It's perplexing just why don't we see more cameras that would do the same.
Now, I no longer have SI-2K but do love BMCC, and now 5D records RAW as well, plus Digital Bolex is coming out - hopefully soon. How do we manage huge folders full of uncompressed DNG frames?
Of course the latest and greatest is GoPro Studio Premium - but it currently does not automatically add audio, and I already have production Cineform installed.
So for my purposes, the solution was RAW 2 CF RAW utility from yours truly. It makes CineformRAW out of 5D raw, BMCC DNGs, even from timelapse CR2s...
You can download private beta (conditionally) here if you like: http://kck.st/16j9MRP
RAW 2 CF RAW (http://primehd.com/raw-to-cineform) version 1.0 in private beta now! Get on the list (http://www.primehd.net/news/download-raw-2-cf-raw/) to be notified when public version is released.
Thanks to Chris Barcellos for great feedback on private beta of RAW 2 CF RAW (http://primehd.com/raw-to-cineform) converter, and for spreading the word.
I'm stuck using the RAW2GPCF. It just won't work. I have been using Rawanizer all the time but the blacks in my shots are coming out pink and noisey even though it looks perfect when I convert it to Cinema DNG. So I thought I needed to calibrate the black by using a >1GB raw file with the lens cap on, which I recorded. I realized after checking the topic that the black calabration can't be done in rawanizer (or can it...) but can actually be done in RAW2GPCF.exe alone and you don't need Rawanizer. I'm not very familiar with this type of command promp set up. I've typed in the code that I think should be typed into the program and press the return key and nothing happens, the program vanishes and no files have been created. I've typed the basic code without specifying the black calibration file and nothing seems to happen.
I bought Cineform and I've converted the files successfully to RAW already. I would use Cinema DNG but I don't have the workflow. Any advice?
Is the lack of sound a magic lantern issue or a conversion issue and whats the deal with this kickstarter raw2gpcf private converter mo7ies keeps blabbing on about?
(http://www.stevenstallone.com/a-pic/Cineform-DNG-2.png)
(http://www.stevenstallone.com/a-pic/Cineform-DNG.png)
Neither CineForm RAW or Cinema DNG add or subtract noise, so the differences here are selected black level and what post processing the Cinema DNG viewer is doing. Black calibration can help, but it mainly helps for fixed pattern noise and/or dead pixels.
To try RAW2GPCF, you need to launch in from a cmd prompt, for directly like I think you are trying'.
'Windows+R' type 'cmd' <press enter>
This will get you a shell where you can try out RAW2GPCF with it disappearing after it executes.
Is there a plan to implement ML RAW or MLV native import into this software? It would be amazing to be able to drag my days shoot into GP CF editor from my CF card and convert from the card onto my computer. This format seems to be the best option.
I also get an error every time i try to import the DNG files "corrupt or un supported" but then after converting it plays work seems to work properly.
@DANewman - SOrry if you have already answered this somewhere else, but what would be the best switch settings to convert the RAW files to something like "SLog2" from Sony, or something that is just a very flat at the start format.. but still maintaining all data(or as much as possible depending on RAW, 444 or 422)
Thanks!
I haven't looked at the details of Slog2, but you can control how flat the image is using the -lX switch that controls the Log curve, using -l900 will give at 13-stop log curve which is pretty flat in appearance. You can also pre-shape the developed image using the metadata switches. If you want to under exposure the output by a stop add +EXPS=0.5 as non-destructive metadata.
I have updated RAW2GPCF to support larger Log curves for a flatter look. Previous only -l255 was the largest, now up to -l65535 works, although -l900 is the highest recommended.
http://miscdata.com/ML/RAW2GPCFv114.zip
Quote from: DANewman on December 12, 2013, 11:04:22 PM
I have updated RAW2GPCF to support larger Log curves for a flatter look. Previous only -l255 was the largest, now up to -l65535 works, although -l900 is the highest recommended.
http://miscdata.com/ML/RAW2GPCFv114.zip
Thanks! I'll test it tomorrow. :-)
I've been away for a while. What is the latest in ML that I need to add support within RAW2GPCF?
Hi all and thnx for great work!!
Can someone be so kind and make a video (youtube or vimeo) tutorial for us noobs with not that much experience in computers
I use canon 5D3+ Latest nightly build 3.1.2014 and shoot video in raw and not in raw.mlv (i think sound is not working in those latest builds)
Option 1
- i have used RAW 2 cinemadng converter (made by chmee) and in 3 days i ended up with 450Gigz of data on one of my data HDDs
-after i get cdng i import in Davinci Resolve 10 lite and add bmd film (color space) and bmd film (Gamma) then select all clips and adding 3D LUTs
-exporting from davinci resolve to Premiere and and render to final export
Option 2
with Ginger directly import raw files from 5d mk3 into PremierePro CC or Lightroom for ACR fast color adjustments
Option 3
with Adobe Dynamic link use dng files in Adobe Speed Grade and export to Premiere pro for final export
Option 4
I would like to try GoPro Studio with Raw2cineform but it looks like hard time geting those dng with this command prompt for me
so i am asking is there some one who can make GUI for this?
Thank you all
Quote from: pind on January 03, 2014, 11:00:31 PM
Hi all and thnx for great work!!
Can someone be so kind and make a video (youtube or vimeo) tutorial for us noobs with not that much experience in computers
I use canon 5D3+ Latest nightly build 3.1.2014 and shoot video in raw and not in raw.mlv (i think sound is not working in those latest builds)
Option 1
- i have used RAW 2 cinemadng converter (made by chmee) and in 3 days i ended up with 450Gigz of data on one of my data HDDs
-after i get cdng i import in Davinci Resolve 10 lite and add bmd film (color space) and bmd film (Gamma) then select all clips and adding 3D LUTs
-exporting from davinci resolve to Premiere and and render to final export
Option 2
with Ginger directly import raw files from 5d mk3 into PremierePro CC or Lightroom for ACR fast color adjustments
Option 3
with Adobe Dynamic link use dng files in Adobe Speed Grade and export to Premiere pro for final export
Option 4
I would like to try GoPro Studio with Raw2cineform but it looks like hard time geting those dng with this command prompt for me
so i am asking is there some one who can make GUI for this?
Thank you all
It is very simple to use
@pind. Just, open prompt, and call to RAW2GPCF (paste on C:\ , for example). Example to do:
[on windows :-\ ] >
Win+R >
cmd >
cd /d C:\ >
RAW2GPCF"cd /d" is command to go on the directory.After this, will "printf" help list. For use, put the files names first and parameters after. Other example:
//in this example input file is called "test.raw" and output is in .avi, so "test.avi". Parameters in "teal" call
RAW2GPCF > test.raw test.avi -444 -d4 -q5
Enter this and software will work...
Quote from: DANewman on December 12, 2013, 11:04:22 PM
I have updated RAW2GPCF to support larger Log curves for a flatter look. Previous only -l255 was the largest, now up to -l65535 works, although -l900 is the highest recommended.
http://miscdata.com/ML/RAW2GPCFv114.zip
Hi David,
When I use the new version of the raw2gpcf (v114) with the -c -f25 -422 command set, the pictures turn out darker than they do in the previous version of raw2gpcf. Is there anything I can do about this. Another thing, how does one get a hang of all the features and command instructions that come up on the command prompt page. Some of these will require some more explanation for folks like me - absolute beginners.
Quote from: 1% on June 22, 2013, 07:45:22 PM
I tried latest on 50D footage.. its perfect... will have to see how 6D does. I feel comfortable deleting the 50D raw file.
Funny, I feel the same - now that I am beginning to get a hang of this stuff!
Quote from: Canon eos m on January 05, 2014, 08:16:28 AM
Hi David,
When I use the new version of the raw2gpcf (v114) with the -c -f25 -422 command set, the pictures turn out darker than they do in the previous version of raw2gpcf. Is there anything I can do about this. Another thing, how does one get a hang of all the features and command instructions that come up on the command prompt page. Some of these will require some more explanation for folks like me - absolute beginners.
Is it a lot darker? What options are you using? I few versions go I started using the black level offsets from the .RAW bitstream, that would have made a change, but hopeful for the good.
As a beginner I'm surprised you are using the -c (Cineon) switch, which even I don't use. I prefer the default (GoPro Protune), or a log curve like -l400 (which requires GoPro Studio version 2.0.)
I am just aping the command set. Will use it no further. Will help if there was a directory of commands available somewhere.
Quote from: DANewman on January 05, 2014, 07:05:27 PM
Is it a lot darker? What options are you using? I few versions go I started using the black level offsets from the .RAW bitstream, that would have made a change, but hopeful for the good.
As a beginner I'm surprised you are using the -c (Cineon) switch, which even I don't use. I prefer the default (GoPro Protune), or a log curve like -l400 (which requires GoPro Studio version 2.0.)
-l400 is too aggressive. I liked using -g2.2 with -l150 - the result was a bit dark but the colour representation was superb (close to the original skin tones, etc.). Any way to lighten the picture other than using -lx since -l300/-l400 was too aggressive on the hair (with a lot of colour noise).
Why not just use regular ProTune? The point is to retain data through compression, which ProTune does very well. Then inside studio if you want it flat, just set it to Cineon 95-685 or Log C. Oh and I wouldn't use -c option on conversion, it caused my footage to go to crap too.
iaremrsir is correct. You are not trying for the best look straight out of the conversion, but a good curve that preserves all your source image, that is easy-ish to color correct on top of.
For some reason on this last revision of raw2gpcf black calibration crashes the execution.
EDIT: False alarm, sorry. I put a space in between -bc and the name of clip when I shouldn't have.
Quote from: iaremrsir on January 07, 2014, 07:40:43 AM
Why not just use regular ProTune? The point is to retain data through compression, which ProTune does very well. Then inside studio if you want it flat, just set it to Cineon 95-685 or Log C. Oh and I wouldn't use -c option on conversion, it caused my footage to go to crap too.
I should also mention to make it flat, in addition to setting the output curve to Cineon or Log-C, you can set the matrix neutral. That way it'll work well with LUTs that expect a log input.
Hi David,
Know that this is the canon-ml forum but since we are talking about common stuff -cineform/ prores4:4:4, etc. I though that my question may be relevant purely from a post processing perspective.
First the reason leading to the question / objective:
Reason - I just discovered that my Gopro Black Hero 3+ can do Cinema RAW. This gives another push to the capability and my use of the Gopro as an additional camera for multi-camera shooting. Previously, I was using the Canon EOS M with my 5DM3. But since I discovered the Gopro's capabilities, I decided to switch. This helps me keep the 5DM3 fixed on a tripod, while I move / glide along with my Gopro handheld and then combine the footage in Premier Pro CC. The problem is that that the formats are not compatible and post processing (grading) is almost impossible.
Objective: To grade the footage from the Gopro and 5DM3 to make for a seamless combined clip with brilliant output.
The only option that I could think of was to somehow convert both footage in prores 4:4:4 format and then combine for best quality. The problem is that I don't know how to do that. Began with learning how to use Speedgrade (I have the license for the CC version) but don't like the interface, then tried Gopro trial version of Studio Premium is not very exciting to use either. Then I discovered the Davinci Lite 10 and things are looking bright. However, I don't know what workflow to adopt.
I now use raw2gpcf to convert my 5D3 footage to cineform format. Want to do the same for the Gopro footage somehow (unless this footage is already in the same format. Then being the grading process. Once the footage is combined should be simple to edit the footage using the multi-camera feature in Premier Pro CC.
What say you?
Quote from: DANewman on May 27, 2013, 04:08:54 AM
Yes, the resulting images will have baked color. Not a bad thing if you have all of the information you need to do your finish, but RAW in my opinion requires you to be able to restore back to everything the sensor read, giving you the greatest flexibility.
I don't consider applying the demosaic as a necesary part of RAW. Consider the Protune CAM_RAW mode on GoPro HERO3 Black Edition (something I worked on,) that is about as raw as you can get in an H.264 file. All the demosaic is applied in camera, but color matrix and white balance are applied in post, just like classic uncompressed RAW.
If there is enough interest, I will enable YUV or 444 development, without baking the color. The only reason you might like 4:2:2 RAW, is that could run on the freely distributed GoPro CineForm Studio, just like Protune CAM_RAW works for GoPro users today.
This is what I need to understand better. Would be good if you could explain what compression could I apply to the 5D3 and Gopro footage to bring these on a common platform.
Any update on the demosaic algorithms?
Quote from: Canon eos m on January 14, 2014, 01:35:07 PM
This is what I need to understand better. Would be good if you could explain what compression could I apply to the 5D3 and Gopro footage to bring these on a common platform.
I'd go to the guys of VisionColor and ask them to add GoPro Hero cams with ProTune Raw as an option to their VisionLog UCS. It enables the conversion of colors to match that of another.
Guys, Cineform RAW is still not so sharp unlike Adobe CR in AE/LR and even DaVinci? Or with new GoPro 2.0 software it's do a better job? As I can see, ACR is the best way to go for now?
Quote from: Pashi on January 15, 2014, 09:21:32 PM
Guys, Cineform RAW is still not so sharp unlike Adobe CR in AE/LR and even DaVinci? Or with new GoPro 2.0 software it's do a better job? As I can see, ACR is the best way to go for now?
CineForm Raw can be debayered by Davinci. So if you like Davinci's debayer, make Resolve your final stage and link with XML to the CF Raw files. ACR is too slow for me; Davinci is a happy medium between speed and debayer quality.
If only it ran on my computer :(
This is hands down the fastest way to go from .RAW to a complete and playable video. If only there was a way to make AE de-bayer the cineform files.
Quote from: 1% on January 16, 2014, 12:54:39 AM
If only it ran on my computer :(
This is hands down the fastest way to go from .RAW to a complete and playable video. If only there was a way to make AE de-bayer the cineform files.
Hopefully they implement the demosaic algorithms I showed them a while back. The licensing was compatible, so fingers crossed...
Quote from: iaremrsir on January 16, 2014, 01:38:16 AM
Hopefully they implement the demosaic algorithms I showed them a while back. The licensing was compatible, so fingers crossed...
I still have tabs open to those links (thank you again), so it will hopeful happen, just need to find the engineering cycles.
Quote from: iaremrsir on January 14, 2014, 08:10:28 PM
I'd go to the guys of VisionColor and ask them to add GoPro Hero cams with ProTune Raw as an option to their VisionLog UCS. It enables the conversion of colors to match that of another.
Will they accept request for a tailor-made request?
are cf raw files supposed to allow the raw settings within resolve? for me they've always been disabled when importing cfraw files as either mov or avi into resolve.
my workflow for now is :
record as mlv (5dmkII) - convert to raw (mlvbrowsesharp) - convert to cfraw (raw2gpcf) - import to resolve
but the resolve raw panel is disabled, so i thought perhaps i could use gopro2pro perhaps to do the raw settings i cant use in resolve (temp, debayer, sharp), but any changes in gopro2pro arent reflected in resolve either. so if theres something i am missing to allow me to use cfraw files in resolve, and then use resolves raw panel, please let me know, thanks.
Quote from: Canon eos m on January 19, 2014, 09:44:26 AM
Will they accept request for a tailor-made request?
They've been asking for them. http://vision-color.com/2013/11/18/announcing-visionlog-ucs/
Quote from: terranaut on January 19, 2014, 04:24:07 PM
are cf raw files supposed to allow the raw settings within resolve? for me they've always been disabled when importing cfraw files as either mov or avi into resolve.
my workflow for now is :
record as mlv (5dmkII) - convert to raw (mlvbrowsesharp) - convert to cfraw (raw2gpcf) - import to resolve
but the resolve raw panel is disabled, so i thought perhaps i could use gopro2pro perhaps to do the raw settings i cant use in resolve (temp, debayer, sharp), but any changes in gopro2pro arent reflected in resolve either. so if theres something i am missing to allow me to use cfraw files in resolve, and then use resolves raw panel, please let me know, thanks.
No, Resolve reads the raw data, but doesn't let you manipulate it. You can, however, use the Resolve fix and just make raw changes in Studio. If you decide to go the non-fix route, you'll get Resolve's debayer, but you'll have to fix white balance manually.
Just got the new update:
Quote2.0.1.317 (23 January 2014)
UPDATES:
...
Enhanced RAW format support
Updated third party camera support
Bug fixes and stability improvements
Can you give quick explanation of what these are please?
Also, will it ever be possible to use Adobe DCPs (Digital Camera Profile)?
I'm not familiar with Adobe Digital Camera Profile, is that a standard?
Come to think of it, I don't think using it with CineForm would benefit us much, because we already have so much control over everything already in terms of the matrix and curves. But if you want to look more, here's a good explanation.
http://dcptool.sourceforge.net/DCP%20FIles.html
Quote from: terranaut on January 19, 2014, 04:24:07 PM
are cf raw files supposed to allow the raw settings within resolve? for me they've always been disabled when importing cfraw files as either mov or avi into resolve.
my workflow for now is :
record as mlv (5dmkII) - convert to raw (mlvbrowsesharp) - convert to cfraw (raw2gpcf) - import to resolve
but the resolve raw panel is disabled, so i thought perhaps i could use gopro2pro perhaps to do the raw settings i cant use in resolve (temp, debayer, sharp), but any changes in gopro2pro arent reflected in resolve either. so if theres something i am missing to allow me to use cfraw files in resolve, and then use resolves raw panel, please let me know, thanks.
Sorry I completely missed this. Resolve doesn't give control over the raw parameters, it just debayers it and gives you an image. If you shoot a color checker or DSC one shot, you can balance all your footage much faster. You get CF compression w/ Resolve's debayer.
Any thoughts on MLV support?? :)
mlv + mlrawviewer out to cineform with amaze debayering in the middle which mlrawviewer already does. right now the pro res out there is kind of winning.
Quote from: tonybeccar on February 07, 2014, 06:27:04 AM
Any thoughts on MLV support?? :)
First I've heard of it, I've been busy. It is what should be supported next? If so, please point be to the specifications.
see:
http://www.magiclantern.fm/forum/index.php?topic=7122.0
and mlv_rec.mo
thanks iaremrsir for your answering me, i appreciate it!
thanks also to david whos efforts to help aren't just seen in the magic lantern community but others as well, much on his own time and efforts, sort of like the john nack of video codecs.
thanks also to everyone else in here whos gone through remarkable efforts to code, test, and share. it seems like ML has increased canons dslrs literally tenfold and makes me wonder when canon will address magic lantern directly, acknowledging how impressive it is as an idea, reality, and community.
it seems a given that MLV is replacing the RAW format, since its pluses like sound, metadata, etc are just as cdng was a step beyond dng. hopefully that new program to convert MLVs directly to DNGs kindly put out by tonybeccar will be able to create cineform files as well with some tweaks to the raw2gpcf code, this way even if magic lantern raw is never supported directly within the gopro program we can use another alternative with a gui, plus the possbility of syncing audio in that same cineform raw mov file.
though i've done photo raws for over a decade, i am fairly new to video raw so i have some mental workflow hiccups. likewise, i have used cineform for 8 years but only on the compressed side. magic lantern has moved me over to video raw and grasping davinci resolve now, but i am hung up on some things i cannot seem to search an answer to that i 'get'. i swear i've tried! if anyone can answer one of these 3 graphs id appreciate it.
1 ----------
CINEFORM CODEC COMPRESSED vs CINEFORM CODEC RAW
i have a initial raw file from canon/ml, and i convert it to a cineform mov file with raw2gpcf, is the output file now -
1- still a true raw file with the raw numbers losslessly compressed via cineform, and then put in a mov container (much like a photo raw file goes to a dng file)
2- or is it now a true mov file but with no compression, and its just of such high bits and quality of lossless compression that it comes veeeery close to being akin to raw (sort of like turning a photo raw into a 16b tif using lzw comp)
2 ----------
DEBAYERING
i am assuming that a RAW file stops becoming raw at the very moment in a program when those numbers are put through the debayer formula. i know when using lr/acr and photoshop, that debayer moment is when i hit OK in lr/acr and the raw file is debayered into a tif for further editing in photoshop.
if this is true, then the best adjustment ranges/quality happens within lightroom before debayering.
but for video raw files, i understand now that the moment of debayering doesnt happen AFTER resolves workflow, but instead within resolves just BEFORE resolves corrections workflow, so that resolve operates more or less like photoshop using tifs?
if this is true, then even though i cannot do raw adjustments in resolve, i can do raw adjustments in gopro?
if true, then it would be that resolves non-raw correction algorithms are better then cineforms raw correction algorithms, so someone just uses cineform for the codec and skips on the gopro program adjustments?
likewise, since resolve can natively read both the cineform raw and cineform nonraw codec(s), it skips cineforms own debayering and uses its own debayering filter, which is preferred anyways because davincis debayering is better quality than cineforms older debayering algorithms?
cineform has raw debayering quality options, which basically appears to us as a form of sharpening, and resolve either doesnt have debayering options or is its raw-panel 'sharpness' their version of debayering?
3 -
LOGS
so raw video itself has no log curve, but we can assign one. whereas compressed video does not have a log curve but you can assign a simulated one just to try a squeeze some extra DR? for instance with h264 recording i was using technicolors profile in-camera and when importing into a program i would assign that profile to the video to give me a 'log-like' DR.
when you convert a bare raw into a cineform raw, you assign a log, but that log curve can always be changed just as metadata until you debayer the video and then its baked?
is a video log simply implied numerics much like a 2d photoshop curve, or is it more that?
i see assigned logs as plain numerics, ie 90 or 400, and then i also see sonys slog, cineons clog, resolves bmdfilm, and gopro protune. are companies logs just a certain single numeric on a 2d curve (like at cineons site it says Clog is 400), or are companies logs a series of numerics on a 2d curve and thats what makes them a bit more special than a single numeric point, much like a 1d vs 3d lut where the 3d ones have more specifics?
4 -
COLOR CHECKER
i have an xrite colorchecker, and with photos i can use their lr plugin to get a custom colorprofile. i know the colorchecker can be used to manually set white balance, but is there a video plugin to use the whole colorchart automatically to color correct, not just white balance?
thanks for any input, as i have read up but a few things aren't completing circuits in my head. if i can nail down the raw workflow for video as i have for photos, why, my life would be complete. for the week.
gary g / wpb,fl
Q1: It will be CineForm compressed into an MOV or AVI. You can choose to compress native RAW (CineForm RAW) or developed (debayered) to RGB or YUV and compress that as CineForm.
Q2: There is no significant advantage to color correction upon RAW vs debayered to RGB, both have the same dynamic range. However, texture and detail are controlled by the demoasic, so if you can choose that later, I would think that is a bonus. Also RAW compressed is about 40% size or RGB compressed, with the same quality.
Q3: All compressed sources must have a log or gamma encoding, all compressor assume there is at least gamma correction (e.g. H.264, ProRES, etc.) There a quality/efficient gains using log encoding. Uncompressed RAW is linear, so you have to select a curve to compress it. This is not metadata, as is does manipulate the data before it is stored. The CineForm codec has some clever tricks to use metadata to describe decoding behavior, independently from the encoding curve applied. Most log curves are 1D LUTs. GoPro Protune is log 113, SI-Log is Log 90, I like log curves that are mathematically described, rather than loading a table
output = log10(input * (base - 1.0) + 1.0)/log10(base) // input and output ranges are normallized 0 to 1, base is the log base (e.g. 113)
Q4: I don't have that.
Quick question Dave, how does the base of the logarithm connect to the dynamic range it's optimized for? Also why not log base 2 since every stop is a doubling of light?
I thought something like that at first, wouldn't you use a curve that divided the stops equally amongst the available codewords: 10bit - 1024 codes, 12-bit - 4096 codes etc.? So for a 12-stop camera, using 12-bit log storage you would think you need 341-ish codes per stop (4096 / 12 = 341.33.) The source is in linear space is digitized, so say 14-bit (with noise in the lower bits), which has a highlight stop of 8192 values, which will be stored as 341 values in log space (efficient) -- this is fine -- whereas the bottom usable stop (for a 12-stop camera) is the lowest 3 bits, 8 values of mostly noise expanded out to 341 codewords. That is not going to work. It also plays havoc on compression efficiency as the last stop is mostly noise, and noise is uncompress-able.
Why do want to reduce the number codewords used for the highlight, but not give equal weight to the shadows, as the shadows are not all signal. We want to store the signal without too much overhead of the noise. For wider dynamic range systems there is a lower noise floor, so we need to use fewer bits in the highlights saving more codes for the lower stops. That is why there are different log curves.
Quote from: DANewman on February 12, 2014, 10:35:32 PM
I thought something like that at first, wouldn't you use a curve that divided the stops equally amongst the available codewords: 10bit - 1024 codes, 12-bit - 4096 codes etc.? So for a 12-stop camera, using 12-bit log storage you would think you need 341-ish codes per stop (4096 / 12 = 341.33.) The source is in linear space is digitized, so say 14-bit (with noise in the lower bits), which has a highlight stop of 8192 values, which will be stored as 341 values in log space (efficient) -- this is fine -- whereas the bottom usable stop (for a 12-stop camera) is the lowest 3 bits, 8 values of mostly noise expanded out to 341 codewords. That is not going to work. It also plays havoc on compression efficiency as the last stop is mostly noise, and noise is uncompress-able.
Why do want to reduce the number codewords used for the highlight, but not give equal weight to the shadows, as the shadows are not all signal. We want to store the signal without too much overhead of the noise. For wider dynamic range systems there is a lower noise floor, so we need to use fewer bits in the highlights saving more codes for the lower stops. That is why there are different log curves.
Thanks for the explanation! One thing I noticed, is that compared to something like Log-C or Cineon, the formula you gave clips the black at 0 if the blacks are in fact clipped in the raw data. And something like Cineon offsets it. What's the reasoning behind this the offset/raised blackpoint? How exactly would you read this formula if you were to graph it? I'm thinking that input would be x-axis and y-axis, but I don't know how to visualize that in 10-bit or 12-bit values. And also, how'd you come up with the base of the log, like the 113 and 90? I apologize for asking so many questions, I just want to understand this more. I want to do engineering that involves something with cameras and/or sound... Is there a name for that? Anyway, thanks again.
The Cineon curve is a byproduct of film scanning and it relates to the nature of negative film. The large lifted black is mostly a waste codewords (in my opinion) but it can help in film style workflows for those that still think that way. The data that represents black is always lifted slightly as you do want to encode the noise (just not too gained up) not clip it. The sensor data is often significantly lifted, so that DC offset is removed before applying the log curve.
Excel spread sheet with the formulas https://dl.dropboxusercontent.com/u/5056120/protuneVgamma.xls
Thanks for explaining all of this. And thanks for the spreadsheet as well, gives much better understanding of how it works compared to video gamma.
@DANewman
Would you consider passing me the source of RAW2GPCF ?
This is exactly what I am trying to do with the API of Flea3.
http://personal-view.com/talks/discussion/8944/3d-3k-12-bit-raw-camera-project
I am stuck due to lack of documentation related to various fields in the CF header. As well as
the lack of knowledge of packed vs unpacked RAW inputs.
This would demonstrate the problem.
The Flea3 cameras save this raw file sequence as I chose a sequence of images.
rawcgpcf cannot open the raw files.
>>>
https://dl.dropboxusercontent.com/u/9906333/fc2_save_2014-02-19-184008-0000.raw
>>>>
C:\Users\Owner\Documents\RAW2GPCFv114>RAW2GPCF.exe e:\*.raw e:\*.avi
e:\*.raw file could not be opened
done
>>>>
C:\Users\Owner\Documents\RAW2GPCFv114>dir e:
Volume in drive E is Record
Volume Serial Number is 6EC8-D020
Directory of E:\
19/02/2014 06:40 PM 6,220,800 fc2_save_2014-02-19-184008-0000.raw
19/02/2014 06:40 PM 6,220,800 fc2_save_2014-02-19-184008-0001.raw
19/02/2014 06:40 PM 6,220,800 fc2_save_2014-02-19-184008-0002.raw
>>>
zcream,
Use the CineForm codec SDK. http://twitter.com/CineFormSDK
What are you referring to here? "lack of documentation related to various fields in the CF header."
Use BYR4 (16-bit linear format -- easy) re: "the lack of knowledge of packed vs unpacked RAW inputs."
David
Hi David. The sample code is DPX2CF.
For a fast realtime compression, I would need to access the isochronous threaded API.
In the example given, threading can be done by just passing different files to a different thread.
For a single file, I would assume threading would be done by breaking it into segments and then passing it into the encoder using the threaded encoder API.
As there is no example of threaded encoding within a single file, I have not been able to figure out these details.
I had stopped coding to this API last month, and did not keep notes. There were a couple of fields that were not very clear. I will post them once I start to look at the code again.
My raw data is packed 12-bit data.
So we are looking at 2 pixels in 3 bytes. Would I need to unpack it to a 16-bit RAW file with unused bits at zero ?
If I am planning to do this in realtime, this will involve a penalty hit.
Quote from: DANewman on February 26, 2014, 07:18:10 PM
zcream,
Use the CineForm codec SDK. http://twitter.com/CineFormSDK
What are you referring to here? "lack of documentation related to various fields in the CF header."
Use BYR4 (16-bit linear format -- easy) re: "the lack of knowledge of packed vs unpacked RAW inputs."
David
The codec uses 16-bit SSE2, so the unpacking has to occur somewhere. This might be 5-10‰ of the compute load.
Quote from: DANewman on March 03, 2014, 03:50:36 PM
The codec uses 16-bit SSE2, so the unpacking has to occur somewhere. This might be 5-10‰ of the compute load.
Does CF only accept 16-bit or 8-bit ?
Quote from: zcream on March 04, 2014, 04:23:50 PM
Does CF only accept 16-bit or 8-bit ?
For RAW only 16-bit should be used. For YUV v210 is a 10-bit format, and there are several 10-bit RGB pixel formats.
I've done the conversion to 16-bit and am writing out a workable dng header.
Is there a preference for the Asynchronous vs Encoder route ?
I think only the Asynch offers multithreading, but don't see how to split up a file into chunks for single file mt (the example uses multiple files).
I would be encoding in realtime.
Also, can you deal with dark frame and bad pixel issues ?
Do you need the XYZ-RGB matrix specific to the camera passed to CF ?
Its quite exciting, if I can make this work - the flea3 camera offers 4K in 8-bit RAW for 700 bucks.
For sources that have north of 13 stops of dynamic range, should we just use the 13-stop curve?
Is this a curve used to map 12-bits linear RAW to 10-bits log ?
Can I get a reference or a citation ?
EDIT:- The Internet is a wonderful resource.
I read up on Log mapping and LUTs. These are called Linearization tables in the DNG spec.
David, I am just trying to figure out why my LoadLibrary called to the CFHDEncoder dll are failing.
I have a license - do I need to supply it as a string to the LoadLibrary call ?
And would this be the Activation code ?
ATM, I am not passing a license.
Quote from: iaremrsir on March 06, 2014, 07:43:09 PM
For sources that have north of 13 stops of dynamic range, should we just use the 13-stop curve?
That works, as do the other Protune curves, the 13-stop curve place a little more emphasis on the shadows.
I am REALLY SORRY for the question I am about to ask, but I am an ML user since... well... yesterday... Please forgive me :)
I have tried the RAW2GPCF, using the trial version of Gopro Studio Premium. The Rawanizer seem to work, but the output files are a mess - video looks like it is full of bad compression artifacts.
Is it just because of the trial version, or do I have a problem at some other level? I want to make sure everything works ok before I spend 300$, as it is quite a lot of money where I come from :)
Okay, so I've managed to use SpeedGrade to debayer DNGs, but with the ProTune curve. However, I'm having problems with exporting to the CF codec in either AVI or Quicktime. This could be an alternate way to get ProTune compressed files using SpeedGrade's daily workflow.
Quote from: iaremrsir on January 25, 2014, 07:31:24 PM
Come to think of it, I don't think using it with CineForm would benefit us much, because we already have so much control over everything already in terms of the matrix and curves. But if you want to look more, here's a good explanation.
http://dcptool.sourceforge.net/DCP%20FIles.html
The link to that site is weird. The contents as well. The guys providing it can't even get the download straight... Also - he states that a camera sensor has an RGB sensor ;-)))))
Weird weird weird
On Mac I can neither import .RAW .MLV PR4444 --- what good is this app for ??????????????
Quote from: CF on March 29, 2014, 11:44:27 PM
On Mac I can neither import .RAW .MLV PR4444 --- what good is this app for ??????????????
It's a Windows tool.
Quote from: iaremrsir on March 30, 2014, 07:40:12 AM
It's a Windows tool.
According to their site, it is for Mac.. It runs here, without the features, but acts more erratic than any code I have ever seen on OS X.
$999 for something Speedgrade does in blink....
Overpriced - if you ask me !
Quote from: CF on March 31, 2014, 02:30:57 AM
According to their site, it is for Mac.. It runs here, without the features, but acts more erratic than any code I have ever seen on OS X.
$999 for something Speedgrade does in blink....
Overpriced - if you ask me !
Wait I think I've confused which tool you're talking about. I'm talking about raw capabilities in GoPro Studio Premium
Long time reader, first time poster. I've read trough the entire thread and I think it's just incredible that CineForm co-founder David Newman is on the Magic Lantern forums and puts his free time in this project! I hope this project still motivates you.
I recently bought a used 50D (and two very expensive CF cards) just to experiment with Magic Lantern RAW Video. Up until now I've been having a tedious workflow: denoising h264 files with NeatVideo and converting that to CineForm. Batch converting Magic Lantern RAW Video to CineForm RAW (without the need to 'reconstruct' from a h264 file with NeatVideo) is a workflow dream come true. I can't wait for my CF cards to arrive.
I just wanted to show my appreciation and let you know I'm looking forward to improved debayering in CineFrom and MLV and Mac support from the conversion tool. (I was hoping to completely abandon Windows in favor of OS X, but if there's no Mac support for this tool I may have to reconsider my decision).
Thanks for giving to the ML community!
I haven't read through every single post yet because I am already so frustrated with this workflow.
This didn't work for me whatsoever.
When attempting to convert, all I got was "Error" on every single clip.
Quote from: AntonioPantoja on May 28, 2014, 06:56:42 AM
I haven't read through every single post yet because I am already so frustrated with this workflow.
This didn't work for me whatsoever.
When attempting to convert, all I got was "Error" on every single clip.
Can you give a little more info than this? Maybe a screenshot of the conversion screen.
Old thread. Is the Raw2GPCF still available? Is it still viable for use with the new MLV files (including audio)?
I am just now installing ML on a 5D2 and would like to take MLV Raw footage directly into GoPro Studio from time to time. Not sure what is currently the best way to do so.