GoPro CineForm Studio Premium/Pro Settings for 5D3 RAW Video

[marten]

I have updated my batch tool to run raw2gpcf as an option. Have a look at http://www.magiclantern.fm/forum/index.php?topic=5557.0

I have a question for @DANewman, how are you writing the console output? I have tried several ways to capture the console output but failed. I want to continually read while the processing is going on but somehow the output is not released until the process is finished. I have no problems with all other tools I'm using: dcraw, raw2dng, ffmpeg etc. Is there something you could change with it so the output will be more friendly to catch?

Otherwise this is really good for fast generation of video files. Thanks.

[Samuel H]

ehm Samuel wait, I did not run cineform on that test. I did CR2 (still image), RAW (via ACR) and H264. To be honest I don't remember the sharpening settings but I think it was just the default 25 with radius 1.0 - For the CR2 I used resize sharper in photoshop to resize to HD resolution.

Isn't CF advance detail 2 the best option for good sharpnes with no excessive artifacts David?

ooops, I don't know how I read that, I thought I was seeing H.264 vs CFR vs RAW

I guess I was surprised to see such a big difference between the downsampled CR2 and the video RAW, and I just assumed it would be something intermediate, like CFR.
I made a similar comparison back when this was not public yet, and in my memory results were really close. Looking at them again, well, closer than H.264, but not quite the same:
http://www.similaar.com/aarchive/800_st.jpg
http://www.similaar.com/aarchive/800_lv.jpg
http://www.similaar.com/aarchive/3200_st.jpg
http://www.similaar.com/aarchive/3200_lv.jpg

[DANewman]

@DANewman Really great utility!!! Thanks a lot! I've tried the "raw2gpcf -444- -c -q5" with and without the cineform curve (MOV files for Resolve). The most information I get is with the -c command (without it the highlights are completely blown out). It's fast and seems quite reliable but color noise is worse than in ACR (debayer process I guess?) and the "highlights recovery" of the curve creates some posterization.

...
 I mean the -lX command. I've no idea what value to use or what the effects are but I would like to try the Protune curve with this tool.

I expect you are have issues with Cineon as an encoding curve, it is not the optimal choice as it is not very efficient for using the available codewords.  Hightlight are not blown out, some are just above 1.0, in RAW they can be restored.  Open Studio and try reducting the gain or exposure, all the highlights are still there.  You can encode Protune, and decode to Cineon if you like, that is better than encoding Cineon.

-l90 -- SI-2K curve
-l400 -- Protune 12 stop optimized
-l900 -- Protune 13 stop optimized

While the default Protune log curve and -l400 curve both store 13+ stops, it is just a matter of emphasis, and how much do you need to dig into the shadows at the mild expense of midtones and highlights.

[sergiocamara93]

-l90 -- SI-2K curve
-l400 -- Protune 12 stop optimized
-l900 -- Protune 13 stop optimized

While the default and -l400 both store 13+ stops, it is just a matter of emphasis, and how much you need to dig into the shadows, are the mild expense of midtones and highlights.

Thanks, David, I'll try those. I was using cineon since it's the only one which is not giving me the pink highlights after recovery (with 28th Build in 5D Mark III). I input the files directly into Resolve so I guess I cannot take advantage of the protune-encoding/cineon(or other options)-decoding since Resolve ignores them.

I think Resolve reads cineform metadata in a different way, I've tried to input the R G B color matrix you posted but colors were off so I'll try the log options and see how it goes. Let me know if you need any specific feedback/tests for the utility, happy to help.

[DANewman]

I have updated my batch tool to run raw2gpcf as an option. Have a look at http://www.magiclantern.fm/forum/index.php?topic=5557.0

I have a question for @DANewman, how are you writing the console output? I have tried several ways to capture the console output but failed. I want to continually read while the processing is going on but somehow the output is not released until the process is finished. I have no problems with all other tools I'm using: dcraw, raw2dng, ffmpeg etc. Is there something you could change with it so the output will be more friendly to catch?

Otherwise this is really good for fast generation of video files. Thanks.

Weird the console output will not flush without a '\n', and as the progress indicator is just a '.' per frame, you aren;t getting any flush output.  There is a workaround add the '-v' switch.

You will get output like this:
M:\cameras\CanonMagicLantern>raw2gpcf M0000000.RAW aa.mov -v
found 8 cpus
1920x1080 53frames 23.976fps
export frame 0
export frame 1
export frame 2
export frame 3
export frame 4
export frame 5
export frame 6
export frame 7
...

[DANewman]

Thanks, David, I'll try those. I was using cineon since it's the only one which is not giving me the pink highlights after recovery (with 28th Build in 5D Mark III). I input the files directly into Resolve so I guess I cannot take advantage of the protune-encoding/cineon(or other options)-decoding since Resolve ignores them.

I think Resolve reads cineform metadata in a different way, I've tried to input the R G B color matrix you posted but colors were off so I'll try the log options and see how it goes. Let me know if you need any specific feedback/tests for the utility, happy to help.

As you using Resolve, you will not be using the CineForm highlight recovery, use the equivalent in Resolve.  The pink highlights are part of the RAW image, as green channel clips first.  This is not an artifact the log curve you select, is might be harder to see with Cineon (on your 2.2 gamma monitor) but it is there just as strong.  The log curve we use is designed to distribute the codewords more evenly throughout the useable stops.

[sergiocamara93]

You mean that ACR and Cineon "neutralize" the magenta cast somehow by balancing the channels? That's interesting to know, I though it was a metadata handling error. I'll have to try and balance the channels then for this workflow.

[AndreasK]

Yes that pink highlight restauration is ACR magic.

@Samuel: I was surprised by the difference too, the BMC is definately sharper in that test. But keep in mind that those are very very fine lines and small fonts so in real world the .RAW is just fine

[AndreasK]

Ok, did some more digging with the patched and unpatched version for Resolve.
Remember the green cast on the unpatched version David? It's apparently a BMCC only problem. The 5D-CF-RAW files open just fine in Resolve in the unpatched version. They open with very little contrast I expect this to be the decoding curve I set when converting. Unfortunately there is no way to set the whitebalance as Studio changes are completely ignored BUT the whitebalance seems right, are you reading it from the .RAW file? It looks ok whereas the DNG in ACR always have that magenta shift?

Ok now for the quality comparsion. Resolve works lovely
As it opened very flat I added some contrast, thats all I did in Resolve, if you compare it to the ACR I opened with sharpening 0 and noisereduction 0 they almost look absolutely identical! I then installed the patch, chose ADV Detail 2 and reexported (with no contrast adjustements) from Resolve. Too bad there is no way to set the whitebalance/decoding curve for Resolve. Is there maybe a way to modify the .mov file instead of doing just metadata via Studio?





And well the BMCC images are completely of, but wrong topic for this forum

[Samuel H]

OK, so Resolve does a great job too, great to know! Thanks for the test.

The cineform sample has some color artifacts; was it debayered by the cineform software, or with the Resolve algorithm? If by CineForm, which debayering method did you use? The one I like best is Advanced Detail 1 (coincidentally, the one DANewman said was winning awards only 4 years ago), but if there's a way to use CineForm RAW compression and get an image that is as good as that Resolve sample, that's what I want!

[marten]

Weird the console output will not flush without a '\n', and as the progress indicator is just a '.' per frame, you aren;t getting any flush output.  There is a workaround add the '-v' switch.

I did try it now with -v and the output on the console is exactly as you describe. BUT the behavior is still the same when I try to catch the output. I'm not an expert on this, but could it be that on windows you need to end each line with an '\r\n", it seems like the \n only gives you a new line in the console but it will not flush until a \r. Would it be possible to output an \r\n when the -v flag is used?

It's really fast, love it!

[Northernlight]

Not CF Raw. The best you can get out of AE is a debayered 4:4:4 file at the moment.

Ok, but at some point the footage need to be demosaiced, and from the look of it, ACR gives you results which are in a completely different class than what GPCF currently is capable of.

[AndreasK]

The cineform sample has some color artifacts; was it debayered by the cineform software, or with the Resolve algorithm? If by CineForm, which debayering method did you use?

It's the same .mov! Resolve normally totally ignores the Cineform Studio processing and reads pure raw data. With that you get this flat but high quality image. But there is no way to change output curve or white balance.
The 3rd picture is with David's Resolve patch. What that patch does is force Resolve to act like any other windows application meaning get the debayered processed image from CineformStudio. All metadata settings in Cineform are shown this way and I used advanced detail 2 as debayer algorithm.

So the best quality would be to go via Resolve unfortunately I don't know how to change whitebalance/curve with that. That's my question to David, is there a way to edit the .mov because there IS some wb-information stored in the file why else would it be fairly accurate and not totally of

[Samuel H]

^ THAT. IS. AWESOME.

So, the image you labelled Resolve comes from a MOV file that is at least 5 times smaller than the stream of DNG files, and it plays back exactly how fast?

Finally a reason to get my paws on Resolve... At least to get an idea of what it can do, and what those limitations are (WB and curves).

I understand you can recode the DNGs as a CF-RAW mov file, edit them in Premiere (with real-time playback, choosing a low quality debayering method if needed), then export the whole project to be color-corrected and rendered in Resolve.
Seeing those results, this would be my preferred workflow for anything that doesn't have VFX.

[AndreasK]

Yep, just download it and see for yourself
As long as you export only 1920x1080 and don't need the Resolve-NR it's free.

Too bad the raw control is not implemented with cineform. With DNG you can set exposure/wb/shift, with R3D you can set a lot in Resolve directly
Really hoping to find a way to change wb/curve, I wonder how David's converter set's the WB from the .RAW

[Samuel H]

the problem may not be how David's converter sets the WB, but whether Resolve reads that metadata or just sets a default value

[AndreasK]

I just played with my BMC files once more. I have to transfer the RED/GREEN/BLUE values from the whitebalance panel in Cineform Studio to resolve to get a clean BMCC file.

The 5D-CF file loads just fine in Resolve, I have no idea why, David may know. And about the quality of that BMCC-debayer I'll continue in the other forum

[DANewman]

I did try it now with -v and the output on the console is exactly as you describe. BUT the behavior is still the same when I try to catch the output. I'm not an expert on this, but could it be that on windows you need to end each line with an '\r\n", it seems like the \n only gives you a new line in the console but it will not flush until a \r. Would it be possible to output an \r\n when the -v flag is used?

It's really fast, love it!

'/r' didn't work.  Calling fflush() didn't work.  No idea why the output can't captured, all standard printf()s.  It seem I need 4K output before anything to a redirected tool.

David

[DANewman]

the problem may not be how David's converter sets the WB, but whether Resolve reads that metadata or just sets a default value

Currently I don't believe Resolve reads any CineForm metadata, which is why there are these color differences. 

[AndreasK]

Yep but you do write something I guess? Why else is the 5D-CF file correct and BMCC totally green?

[DANewman]

Yep but you do write something I guess? Why else is the 5D-CF file correct and BMCC totally green?

No. Same data for both cameras, so this is a puzzle.

[AndreasK]

Yikes! But why is your CF whitebalance right and the raw2dng opened in ACR totally wrong?
Is there some camera wb-data transfered in your process? Maybe I should try set the wrong WB in camera and see if that wrong get's copied to CF.

[DANewman]

In the end .RAW file there is a single color matrix field, but not an as shot white balance.  The colormatrix is the weird XYZ to RAW mapping (reversed for our needs.)  I reverse this matrix, exact the camera calibration RGB sensor range differences (green tints) which I use as white balance, then generate a pure camera (after calibration RGB Gains) RAW to D50 sRGB matrix, which drives the saturation. All looks pretty after that.

[1%]

Does this have a1ex's banding fix?

[fatpig]

Hello guys, I like RAW2GPCF, unfortunately it shows a lot of dots, then says it is done, but no output file is generated.

What are the requirements for it to work?
I have the Codec installed.