Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - DANewman

#101
Thanks for everyone exploring the demosaicing options.  While it is assumed our current filters where selected for performance, that is not entirely true.  While there are some performance options, bi-linear and matrix 5x5, the others where the render grade filters of their day, finishing films from My Bloody Valentine to Slumdog Millionaire.  As computer power has increased even more advanced algorithms are being used. 

The problem we have is GoPro CineForm codec is not open source, while we are working on a SMPTE standard version of CineForm as VC-5, we can't use any GPL code -- although LGPL (lesser GPL v2.1 is fine as a dynamically linked library.)  Several of the offered links point to GPL code, which sadly prevents its use.  Stuck again.

One of the reasons the CineForm RAW workflow is so much easier than everything else, is the controllable demosaic is built into the decoder, effective making every tool RAW capable.  So I still look for a friendly license (MIT & LGPL) or an inexpensive commercial license that could be included in a free decoder distribution.
#102
Make you you did the following,

  "The contents of this zip file should be copied over the components in C:\Program Files (x86)\CineForm\Tools"

If it only have with RAW and not 4:2:2, then this is the fix for the issue you describe.  Make your you are not running any application that may be using the decoder, as that would prevent the component replacement from suceeding.
#103
This is want it should look like

M:\Cameras\Canon5DMkIII>raw2gpcf M0000043.RAW new.avi "E:\temp\temp.avi" -422 -c
-f25
1920x720 594frames 25.000fps
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
..................................
done


The frame rate should be automatically extracted and the image size.  What does yours report a size?  If you are using the free GoPro CineForm Studio it is limited to 1920x1080 for direct codec access.  The fact that is reports the size wrong is a worry.
#104
Quote from: Jake Segraves on June 03, 2013, 09:10:14 PM
A new version of the CineForm decoder that addresses crashing problems within Adobe products (and possibly Resolve) can be downloaded from here:

http://software.gopro.com/PC/new_CFHDEncoder.zip

Thanks Jake.

The contents of this folder should be copied over the components in C:\Program Files (x86)\CineForm\Tools

David
#105
Quote from: meditant on June 03, 2013, 06:30:04 PM
Hello,

Do you planned a mac version ?

Regards

Nothing official, just like it is not official for PC.  I mentioned to one of the Mac engineers as GoPro, he says "cool, sounds fun".  Hope he will consider it (on his weekend like I did for the PC.) ;)


#106
Quote from: 1% on June 03, 2013, 02:44:52 PM
http://www.qfpost.com/file/d?g=Ve9DzOGDb

I think that's the first dng in the series.

You are correct, the artifact while reduced, it is still present with other in demosaic settings.  This is due to the very high frequency nature of this image, and that the 5D doesn't have an optical low pass filter designed for video.  The current demosaic filters in CineForm are designed for native resolution bayer cameras (SI-2K, KineRAW S35,etc), not bayer scaled images, like that in the 5D.  Is this yet another reason I'm looking for new demosaic algorithms we can add, something closer to ACR would be nice. 
#107
Could you provide the source or a DNG frame.  Thanks
#108
Try it without the batching, it seems not all the switches are taking effect. 

RAW is one chroma value per pixel, and through demosaic can be 4:4:4.
4:2:2 is an average of two chroma values per pixel (YU or YV) requires a baked demosaic.
4:4:4 is three chroma values (R, G & B) requires a baked demosaic.

RAW is a natural (lossless) 3:1 compression over 4:4:4.  So compressing RAW CFA Bayer is the most efficient way to store any single sensor image.
#109
1%,

That looks like a bilinear demosaicing filter artifact.  Choose a different filter, try a -d2 switch or set you desired in Studio in the Workspace control field.
#110
The pink in the highlights is controlled by the "Sat. Clip Point" slider, in the Workspace area of CineForm Studio Premium.  The camera's green channel clip first, so you need to do some highlight reconstruction or dynamaic range extension, this is what Sat. Clip Point" does, it is just poorly named. 
#111
Quote from: fatpig on May 31, 2013, 05:17:11 PM
Hello guys, I like RAW2GPCF, unfortunately it shows a lot of dots, then says it is done, but no output file is generated.

What are the requirements for it to work?
I have the Codec installed.

If you request RAW or 444 and don't have a license to CineForm Studio Premium it might do that.  Try -422 to confirm you get output, as that doesn't require more than the free license.
#112
In the end .RAW file there is a single color matrix field, but not an as shot white balance.  The colormatrix is the weird XYZ to RAW mapping (reversed for our needs.)  I reverse this matrix, exact the camera calibration RGB sensor range differences (green tints) which I use as white balance, then generate a pure camera (after calibration RGB Gains) RAW to D50 sRGB matrix, which drives the saturation. All looks pretty after that.
#113
Quote from: AndreasK on May 30, 2013, 02:21:51 PM
Yep but you do write something I guess? Why else is the 5D-CF file correct and BMCC totally green?

No. Same data for both cameras, so this is a puzzle.
#114
Quote from: Samuel H on May 29, 2013, 01:30:15 PM
the problem may not be how David's converter sets the WB, but whether Resolve reads that metadata or just sets a default value

Currently I don't believe Resolve reads any CineForm metadata, which is why there are these color differences. 
#115
Quote from: marten on May 29, 2013, 11:11:51 AM
I did try it now with -v and the output on the console is exactly as you describe. BUT the behavior is still the same when I try to catch the output. I'm not an expert on this, but could it be that on windows you need to end each line with an '\r\n", it seems like the \n only gives you a new line in the console but it will not flush until a \r. Would it be possible to output an \r\n when the -v flag is used?

It's really fast, love it!

'/r' didn't work.  Calling fflush() didn't work.  No idea why the output can't captured, all standard printf()s.  It seem I need 4K output before anything to a redirected tool.

David
#116
Quote from: sergiocamara93 on May 29, 2013, 06:33:14 AM
Thanks, David, I'll try those. I was using cineon since it's the only one which is not giving me the pink highlights after recovery (with 28th Build in 5D Mark III). I input the files directly into Resolve so I guess I cannot take advantage of the protune-encoding/cineon(or other options)-decoding since Resolve ignores them.

I think Resolve reads cineform metadata in a different way, I've tried to input the R G B color matrix you posted but colors were off so I'll try the log options and see how it goes. Let me know if you need any specific feedback/tests for the utility, happy to help.

As you using Resolve, you will not be using the CineForm highlight recovery, use the equivalent in Resolve.  The pink highlights are part of the RAW image, as green channel clips first.  This is not an artifact the log curve you select, is might be harder to see with Cineon (on your 2.2 gamma monitor) but it is there just as strong.  The log curve we use is designed to distribute the codewords more evenly throughout the useable stops.
#117
Quote from: marten on May 28, 2013, 05:21:31 PM
With the new patch 0.4.0 (link in first post) you have it. Haven't got the top progress bar to work yet, they are not send out any breaks I can catch :(

You have to install the software according to the linked message

Add the '-v' switch to RAW2GPCF.exe, you should get more console output.
#118
Quote from: marten on May 29, 2013, 12:18:21 AM
I have updated my batch tool to run raw2gpcf as an option. Have a look at http://www.magiclantern.fm/forum/index.php?topic=5557.0

I have a question for @DANewman, how are you writing the console output? I have tried several ways to capture the console output but failed. I want to continually read while the processing is going on but somehow the output is not released until the process is finished. I have no problems with all other tools I'm using: dcraw, raw2dng, ffmpeg etc. Is there something you could change with it so the output will be more friendly to catch?

Otherwise this is really good for fast generation of video files. Thanks.

Weird the console output will not flush without a '\n', and as the progress indicator is just a '.' per frame, you aren;t getting any flush output.  There is a workaround add the '-v' switch.

You will get output like this:
M:\cameras\CanonMagicLantern>raw2gpcf M0000000.RAW aa.mov -v
found 8 cpus
1920x1080 53frames 23.976fps
export frame 0
export frame 1
export frame 2
export frame 3
export frame 4
export frame 5
export frame 6
export frame 7
...
#119
Quote from: sergiocamara93 on May 28, 2013, 11:03:43 PM
@DANewman Really great utility!!! Thanks a lot! I've tried the "raw2gpcf -444- -c -q5" with and without the cineform curve (MOV files for Resolve). The most information I get is with the -c command (without it the highlights are completely blown out). It's fast and seems quite reliable but color noise is worse than in ACR (debayer process I guess?) and the "highlights recovery" of the curve creates some posterization.

...
I mean the -lX command. I've no idea what value to use or what the effects are but I would like to try the Protune curve with this tool.

I expect you are have issues with Cineon as an encoding curve, it is not the optimal choice as it is not very efficient for using the available codewords.  Hightlight are not blown out, some are just above 1.0, in RAW they can be restored.  Open Studio and try reducting the gain or exposure, all the highlights are still there.  You can encode Protune, and decode to Cineon if you like, that is better than encoding Cineon.

-l90 -- SI-2K curve
-l400 -- Protune 12 stop optimized
-l900 -- Protune 13 stop optimized

While the default Protune log curve and -l400 curve both store 13+ stops, it is just a matter of emphasis, and how much do you need to dig into the shadows at the mild expense of midtones and highlights.
#120
Quote from: AndreasK on May 28, 2013, 10:21:54 PM
Ahh I forgot. Ok let's assume I run the version with no patch, how can I set whitebalance and basis settings in Resolve to get rid of that green cast?

I don't know Resolve at all.  The white balance values in Studio are linear gains that should be applied as linear channel gains in Resolve (same values should work.)  The color matrix then calibrates the sensor to Rec709/sRGB), but I'm not sure that Resolve supports a manually entered color matrix.
#121
Quote from: AndreasK on May 28, 2013, 06:58:47 PM
Is Resolve doing the demosaic not cineform? So no matter what I select in GoPro Studio Resolve handels it? Or does this behaviour change since I use your patched dll?

The patch you are running also switches the de-mosaic.  As the demosaic happens before color corrections, it is not possible for Resolve to do the de-mosaic and CineForm to the first color pass.   For the out of the box experience, Resolve does both the demosaic and color from CineForm RAW files.  The best solution for all would be to add a current generation de-mosaic filter to the CineForm decoder. CF Advance Detail 1, was award winning only 4 years ago, so we have some work to do.
#122
As the de-mosaic is not baked, I happy to hear suggestions on for alternative algorithms to add, making the CineForm RAW results even better. Currently you can use CineForm RAW (as .MOV) directly in Resolve, so you aren't limit to our de-mosaics today. 
#123
This is hardly an official company supported tool, just something I did on the side.  Yet GoPro does support transcoding other cameras, so I used the code base for the Vision Research Phantom transcoder as the .cine files are very like .RAW.  Don't expect this tool to become part of a regular software distribution. Yet notice how this tool requires you to have GoPro software, so it is a win-win I feel. ;)
#124
New build of RAW2GPCF (RAW to GoPro CineForm)

Direct download for RAW2GPCF:  http://miscdata.com/ML/RAW2GPCF.zip  (I replaced the odd link)
The old v1.00 build is now here  http://miscdata.com/ML/RAW2GPCFv100.zip

Unzip are install here: C:\Program Files (x86)\CineForm\Tools

To run:

open a shell, and RAW2GPCF at the prompt

M:\cameras\CanonMagicLantern>raw2gpcf
Usage: raw2gpcf inputfile.raw output.mov|avi [switches]
       -422- Encode to YUV 4:2:2 (default RAW.)
       -444- Encode to RGB 4:4:4 (default RAW.)
       -c  - use Cineon curve (black 95, white 685)
       -cbX- use Cineon curve with 'X' black point (default 95.)
       -cwX- use Cineon curve with 'X' white point (default 685.)
       -dX - use debayer type 1-softest thru 4-sharpest (default auto)
       -fX - the framerate override for AVI|MOV media.
       -gX - use gamma curve
       -iX - in point, start processing on frame X (default 0, disables audio.)
       -lX - use log curve of power X (GoPro Protune.)
       -oX - out point, e.g. -o10 would precess 0 thru 9 (default infinite.)
       -qX - the encoding quality for AVI|MOV (default 4, range 1-5.)
       -sX - skip every 'x' frames (default 0).
       -(x1,y1,x2,y2)  - window source Top Left (x,y) to Bottom Right (x2,y2)
  examples: raw2gpcf M00000004.raw NewRAW5.avi
            raw2gpcf new422.AVI M0000005.RAW -d1 -q3 -422
            raw2gpcf new444.AVI M0000005.RAW -d3 -q5 -444
            raw2gpcf V:\Conversions\newRAW.AVI E:\M0000052.RAW -l400 -d2 -q5
  RAW2GPCF version: 1.01 May 27, '13, (c)2013 GoPro.


Feature additions:
-444 : performs the demosaic and encodes to 4:4:4 (requires a GoPro Studio Premium license.)
-422 : performs the demosaic and encodes to 4:2:2 (requires a free install of GoPro CineForm Studio -- Premium not needed.) and is limited to 1920x1080 under the free encoder license.

Both these new modes are still RAW for color processing, only the demosaic is baked. While the 4:2:2 option will be popular for cost-conscious, the image development controls for RAW are more flexible in the Premium level GUI.

Also 4:2:2 and 4:4:4 modes are slower than the default RAW transcodes.  RAW has the greatest flexibililty as I do want to add more demoasicing options in the future.

Bugs:
This fixes a crash in the windowing switch.
#125
I now have my 5D3 running RAW, although with tiny images until I get new media.