Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - DANewman

#126
This code has too few legal range checks, so you can tell it so do things the codec can't support.  First you can't use odd numbers for the windowing as the image is CFA RAW, color pattern is an even number repeat -- technically I could switch the bayer phase to handle odd numbers -- currently values these are rounded.  Next the codec likes horizontal encodes of a 16 multiple and vertical of 2.  So this should work -(8,0,1928,960) -- except I introduced a bug it doesn't.  Fix coming.

I did have a look at the BMDFilm output curve, and it is a little odd.  It is likely has BMD sensor calibration stuff in it, the curve we use for Protune and others, mathematically models the mapping of linear to log and back again, the BMD curve don't fit our models. PanaLog, S-Log, C-Log, Protune etc. are all nice mathematical equations, but not BMD. If we were to use that curve natively, either a lot of code changes to the codec, or an approximation of the curve would produce (mild) color errors in the matrix, white balance and saturation (all the linear light operations.) 
#127
"... seems like I was able to get more detail by using raw2dng, then using CF Studio to convert the DNG files. "

This doesn't make sense, same process internally. Likely subtle metadata differences -- you can copy settings from one clip to another.

Cineon was left over from the tool I ripped the code from, I never really tested that path. The Cineon curve was not designed for compression, it is film density emulation.
#128
In the meantime, I need feedback on the current tool. 
#129
Quote from: iaremrsir on May 27, 2013, 02:33:44 AM
Not CF Raw. The best you can get out of AE is a debayered 4:4:4 file at the moment.

Yes, the resulting images will have baked color. Not a bad thing if you have all of the information you need to do your finish, but RAW in my opinion requires you to be able to restore back to everything the sensor read, giving you the greatest flexibility.   

I don't consider applying the demosaic as a necesary part of RAW. Consider the Protune CAM_RAW mode on GoPro HERO3 Black Edition (something I worked on,) that is about as raw as you can get in an H.264 file.  All the demosaic is applied in camera, but color matrix and white balance are applied in post, just like classic uncompressed RAW.

If there is enough interest, I will enable YUV or 444 development, without baking the color.  The only reason you might like 4:2:2 RAW, is that could run on the freely distributed GoPro CineForm Studio, just like Protune CAM_RAW works for GoPro users today.
#130
Here is version 1.0.

To run this tool you will need the Windows verison for GoPro CineForm Studio Premium or Professional installed -- either activated or working with a full 15-day trial (only need Premium, Pro is for 3D.)  The trials are downloaded from here: http://cineform.com/downloads

Direct download for RAW2GPCF:  http://miscdata.com/ML/RAW2GPCF.zip

Unzip are install here: C:\Program Files (x86)\CineForm\Tools

To run:

open a shell, and RAW2GPCF at the prompt

M:\cameras\CanonMagicLantern>RAW2GPCF
Usage: raw2gpcf inputfile.raw output.mov|avi [switches]
       -c  - use Cineon curve (black 95, white 685)
       -cbX- use Cineon curve with 'X' black point (default 95.)
       -cwX- use Cineon curve with 'X' white point (default 685.)
       -dX - use debayer type 1-softest thru 4-sharpest (default auto)
       -fX - the framerate override for AVI|MOV media.
       -gX - use gamma curve
       -iX - in point, start processing on frame X (default 0, disables audio.)
       -lX - use log curve of power X (GoPro Protune.)
       -oX - out point, e.g. -o10 would precess 0 thru 9 (default infinite.)
       -qX - the encoding quality for AVI|MOV (default 4, range 1-5.)
       -sX - skip every 'x' frames (default 0).
       -(x1,y1,x2,y2)  - window source Top Left (x,y) to Bottom Right (x2,y2)
  usage: raw2gpcf M00000004.raw New5.avi (using current paths)
         raw2gpcf V:\Conversions\newRAW.AVI E:\M0000052.RAW -l400 -d2 -q5
  RAW2GPCF version: 1.00, (c)2013 GoPro.


I was going to add batch conversion, but I thought I should get some feedback first.

The current tool extracts the RAW XYZ colormatrix data, and converts that into CineForm Active Metadata as the "Source" color matrix, and the white balance gains will auto correct for the sensor's sensitivity differences.  The highlight recovery (which many users can't find) is defaulted to 0.5 (seems to correct many highlight issues.)  The tool is very fast as no demosaic is required to produce CineForm RAW files.  If you convert from your Compact Flash source, the conversion will be at read speed with very little CPU usage (a on set ultrabook could do it.)

For a fast decode speed, I defaulted to be using the automode demosaicing which is a fast debayer (crappy bilinear) when decoding to 8-bit (QuickTime, Window Media Player) but reverts to a nicer image when there 10-bit or greater decodes requested (AE and Premiere float modes, etc.) You can set the demosaic in CineForm Studio Premium after conversion, just as you can with all the RAW parameters (the whole point of RAW.)  You can also default the demosaic to any setting you like (-dX switch) again this is not baked.

Here are mnteddy sample clips (thank you) encoded with the defaults (Filmscan 1, RAW, auto debayer.)  These are 10:1 compressed (images have lots of shallow depth of field which helps CineForm compression.) http://miscdata.com/ML/M0SamplesQ4.zip I also repeated these highly compressable images at Filmscan 2, yielding 6:1. http://miscdata.com/ML/M0SamplesQ5.zip  I did these both in MOV files so users from both platforms can try them out.
#131
Quote from: mnteddy on May 26, 2013, 08:44:45 AM
I've got 3 short shots uploaded here.

These are Tungsten, 1920x1080, ISO 500. Unfortunately, I've nuked all of my daylight RAWs I shot earlier today. Let me know if you'd like to see something specifically.

Any chance we'll see a shell tool for Mac to take RAWs to Cineform RAW MOVs? Thanks!


Thank you, these samples are prefect and working without any code changes.
#132
Any links for .RAW? I don't want to post the tool without testing a range of clips.
#133
I'm working on a Windows shell tool to convert .RAW directly to CineForm RAW .AVI or .MOV files.  My current test tool is working and converting to CineForm RAW at over 100 frames per second (old 2.6GHz i7 920 -- probably disk limited -- converting directly from CF Card will be as fast as you can read.)  The problem is I haven't yet set up my 5D3, and I only have one .RAW clip to test with.  Please provide links to hosted .RAW files for my testing before I post the tool.
#134
Quote from: 1% on May 18, 2013, 06:39:02 PM
Newest studio gives the option of 4:4:4/709/RGB in AE where is raw?

Black frame bug is from vegas using the CFHD decoder DLL and not VFW codecs. So some version of vegas work, some don't... I know its probably buggy for other things too, buggy in general.

AE is only 444, so we have to develop the RAW to 444 via the parameters you set in Studio.

Yes, Vegas seem to be forever a little buggy, it is a shame as we have such a very gooding work relationship with their engineering team.  While I like Adobe Premiere also, Sony Vegas natively implement our codec SDK, and this is cool.
#135
Quote from: AndreasK on May 18, 2013, 06:27:40 PM
David, Neoscene is 10bit tough right? I could use CF Pro with 4:4:4 so debayering with adoberaw and then using cineform normal not raw would be an option?

NeoScene is the old 4:2:2 10-bit tool.  GoPro Studio Premium is 4:4:4 and RAW and 12-bit.
#136
Quote from: 1% on May 18, 2013, 05:23:38 PM
Cineform studio... I dunno. The codec is good but always broken with sony vegas. In v12, preview at full best size causes black screen on some clips.

Codec is 10 bit so I guess I'm losing color now converting? I've yet to find an intermediate that is better in terms of file size and quality so I live with the bugs. :(

There is not been any CineForm issue with Vegas, but yes, they has been a Vegas issue will some third party codecs (not just ours.)  I surprised to hear the Vegas black frame bug still plagues anyone.

CineForm RAW hasn't been 10-bit since the SI-2K in 2007. Yet even in 2007, 10-bit log is good for 14-bit linear, it is still a dominate finishing format with 10-bit DPX files today.   CineForm RAW is currently 12-bit log, and with VC-5 (our SMTPE efforts to standard CineForm) we extend this for crazy superHDR cameras of the future.
#137
Quote from: Kuky on May 18, 2013, 05:19:14 PM
Color noise reduction is killing this workflow. Also it appears that debayer filter enhances also the color noise.

That is why I wouldn't use Advance Detail 3 filters, but some like it, and it was used on parts of Slumdog Millionaire. We didn't not yet add a denoise stage to the demosaic. 

Note: you are showing the old FirstLight tool which hasn't been updated in two years.  The development has moved to the GoPro CineForm Studio application.  They are many that still prefer the FirstLight interface, but slowly you can do more in the new tool.
#138
Quote from: Kuky on May 18, 2013, 04:12:00 PM
- there is a small bug when using regional settings in Windows (comma instead of point). 23.976 fps files will transcode insted to a monstrous 23976 fps file. I have to revert to English(US) region or reinterpret in AE.

- did not find a way to save colour matrix in Studio as a preset.

I'm not sure how to reproduce that speed bug.

We never thought there would be a need to save the color matrix in a preset, as these are the first CinemaDNGs I've seen that didn't have a color matrix included.
#139
Quote from: Kuky on May 18, 2013, 11:05:34 AM
I didn't find any of the demosaic options to be as good as the ACR one.
Advanced Detail 3 seem to work best but it's not on the same level.

Also going through cineform raw I didn't find any  options to recover highlights on the same level as ACR/LR.

Any feedback appreciated.

We do have a highlight recovery control, the slider in "Workspace" is called "SAT. CLIP PT".

I totally agree the demosaic in ACR is awesome, we are on the look out for an open source equivalent to add to our demosaic options.  We like having a good range of filters as playback speed is also an import factor.  You could edit with Advanced Detail 1 or 2 (I don't like 3) then switch to some (future) compute intensive demosaic for export.
#140
Quote from: Peter on May 18, 2013, 10:23:36 AM
Sounds good, what about a tool mlraw2cfraw or even import the raw file into Cineform Studio?

I haven't looked into details for want we could customize, create custom tool for transcoding would be straight forward.  I was just finding the current tool is working well with the DNG sample file someone shared (thank you).
#141
I should be asleep, instead I'm experiment with Canon 5D ML RAW in GoPro CineForm Studio.  Here is more setup, conversion and correction using GoPro CineForm Studio Premium.

Import files (look RAW):


My advanced conversion settings:


Add Color Matrix and White balance to the CineForm RAW clip:


Add a Preview LUT (ProtuneLook):