3D LUTs for MLV -> Prores?

Started by sgofferj, March 20, 2015, 08:13:20 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

sgofferj

Thanks to Danne, I have a working script to batch-convert MLV-files to Prores with ffmpeg. The result is pretty REC709-ish which does leave some room for correcting and grading but I'd prefer a little flatter image. I tried using MLRawViewer's LUTs for C-log, SLog, etc. but those are 1D LUTs and ffmpeg only works with 3D LUTs.
Does anyone know some good (free?) 3D LUTs for MLV -> Prores conversion? Or maybe somebody could spend a few mins and convert the MLRawViewer 1D LUTs to 3D?
18+ years Linux user, wolf-fan, hobby photographer and -filmmaker
EOS 6D, EOS 7D

Danne

In my automator workflow cr2hdr-r Andy_600 provided this workflow with three different luts. They are inside the download package and the intention is to use them with ffmpeg to ProRes workflow. Although lowres to suit ffmpeg lut handling they work very good indeed.

cr2hdr-r_5.3s_s(beta)
https://drive.google.com/file/d/0B4tCJMlOYfirV0gxNTlYQWZYTHM/view?usp=sharing

(Will soon upload a new version)

Examples in the thread
http://www.magiclantern.fm/forum/index.php?topic=13512.msg142552#msg142552

Andy600

I wouldn't recommend baking-in a lin-to-log transfer using a small 3D cube lut even using tetrahedral interpolation, because it a) does not map code values correctly and b) a 3D lut has inherent rounding errors.

The luts for cr2hdr are ok(ish) but only if you intend grading the raw image directly (i.e. not transcoded) - and if you are doing this you will get much smoother results using color correction tools. It would be much better if FFMpeg could use 1D luts (or even transfer functions at code level). Better still, a capable coder should be able to integrate OpenColorIO into FFMpeg and move all lut transfer/transform operations to a much more capable environment.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

Danne

Thanks Andy.
Recommended workflow? Exporting to prores ffmpeg applying 1d luts in post?

arrinkiiii


@Andy600

Since that we are here speaking about ML to ProRes with beaked LUT... I think some one or even me already have put this on the table but here the question... What is needed to have Cinelog V3.0 beaked in ProRes in MlViewer the app from Baldand ?  It's a open source no? Just need to make some code in MlViewer for work? 

Andy600

As it stands, FFMpeg is not the best solution for transcoding a log master. It has a good debayer and color tools but the reliance on small 3D cube luts (a current FFMpeg limitation) is very restrictive. Technical transforms/transfers (log-to-lin, lin-to-log and colorspace transforms) are exactly that - 'technical' and for that you need the ability to use high precision lookup tables, matrices and/or hybrid shaper luts.

Transcoding in Resolve, ACR/AE, Scratch, Speedgrade, baselight, Nuke etc is better for this kind of thing but obviously they cost money.

DaVinci Resolve Lite is free and much more than capable than FFMpeg when it comes to transcoding but then there are still issues with ML color matrices, lack of auto raw WB in Resolve (fundamental to some users), no NR (unless you invest in NeatVideo OFX) and unpublished BMD Film colorspace management that can make things difficult (BMD Film is effectively an unbound colorspace based on camera XYZ primaries  ;)).

Natron is another option -  it's basically an open source compositing app VERY similar to Nuke but it does have some very good color tools. Incidentally there is a non-commercial 'Free' version of Nuke coming soon - there is also the free Fusion Lite from BlackMagic. These are all capable but have vastly different workflows when it comes to transcoding and depends a lot on how deep you want to get into a new environment.

For a fast and free option, continued development of MLRawViewer is really the best way to go. The MLRV engine is really good but the app needs some work doing regarding colorspace management (sRGB based log colorspaces are not really a good idea), and there needs to be some improvements to MLV meta (i.e. color matrix development that goes beyond DCRaw). Unfortunately I'm a complete novice when it comes to coding but I'm learning as fast as I can.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

arrinkiiii


Yes, it would be very nice that MlViewer could output ProRes with the Cinelog V3.0 beaked. I don't understand code but i have friends that is their day job... but i don't know what i should tell them what to do. For sure that they don't understand nothing about color matrix, srgb, lut, etc... but they are good at coding. Baldand stop making updates in his app, i think mostly because exist MLVFS but is so different app and each one are good as it is, and the other reason is because maybe no ones jump/help on the code.

Maybe i can ask for a friend to help, since is a open source code but what i should tell them what to do?

Danne

Andy, there you go. Thanks for sharing.
WHat about mlrawviewer? Isn,t it using ffmpeg for ProRes conversion?

Andy600

@arrinkiiii - It can be done (as far as the log curve of Cinelog V3.0) but the V3 profiles contain much more information to shape color - specifically reducing saturation of certain hues because the default DNG forward matrices (white balanced XYZ to XYZ D50 illuminant) cause highlights to over saturate in red and blue channels. The further transform from V3.0 to Cinelog-C (the transform to a much wider gamut) is also needed or you end up losing or clipping color information when baking a log master (i.e. it's not a good idea to bake a transform to a log space using sRGB primaries).

If you use the Cinelog V3.0 transfer function in MLRV you will get a log signal but you will still get color issues in extreme highlights. If MLRV can be made to rebuild color information in clipped red and blue channels (by borrowing from green which usually has more information) then transform the colorspace to Cinelog-C primaries (and have a 'monitor only' lut slot or hardcoded transform to REC709 to aid visual exposure and WB correction) the results should be close to Resolve.

BUT - a full set of color matrices (embedded in meta) is also required or you'll get inaccurate color, especially shots under artificial lighting because the DCraw matrices only provide CIE XYZ to Camera RGB spectral sensitivity under a single illuminant (D65) - it needs metadata for another illuminant (usually tungsten) to provide 2 points for white point interpolation and accurate default white balance, plus ideally 2x forward matrices to map white balanced XYZ to PCS (XYZ D50) using Bradford chromatic adaptation (or the raw reader might default to another method of CA like Von Kries etc) - in short, these matrices are needed if you shoot under anything other than daylight - MLVFS and raw2cdng now have the full set.


Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

Andy600

@Danne - The issue is about lut handling. MLRV can use combinations of 1D and 3D luts and has slots for 3. Technically it can handle very complex transforms but the issues in my previous reply come into play. Also, ProRes in Windows is of course possible but it's not Apple ProRes that you get on a Mac and often fails QI. DNxHD (or even better, DNxHR) is a better option for Windows based users (for any commercial work).
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

Andy600

@arrinkiiii - my advice for your coding friends would be to first look at the source of MLVFS and raw2cdng (specifically how they embed color matrix meta data in DNGs) then look at integrating OpenColorIO into the app - OCIO is open source and there are Python bindings (MLRV is Python based) http://opencolorio.org/developers/index.html. This will take care of color management. OpenIO is also something else to look at as it will give more output options (EXR, DPX etc). Another valuable resource for anything color related is http://colour-science.org/

If they get that far I have a list of other things that could be done ;)

I'll be happy to advise on color management etc. As and when I get my own coding skills up to scratch I will contribute - I'm just tinkering at present.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

Danne

How far would this take me?

Dcraw applying AW or controlled wb through greycard workflow (cr2hdr-r), exiftool inserting the correct color matrice, exporting to prores444 using tetrahedral setting with ffmpeg? Will the file be considered for pro use?

sgofferj

I would consider Resolve Lite if there was a Linux version - and I mean a version that is INSTALLABLE on Linux and not a whole new Distro. I played around with it some time ago in a virtual machine but that's obviously no solution for serious use...
At the moment, I'm still very much on an enthusiast amateur- or hobby level and I mainly edit with kdenlive. I have been considering switching to Resolve as a BMPC4k ist somewhere on my wishlist and - at least according to the advertising - Resolve is capable of more than just color correcting, i.e. proper editing, at least basic stuff like transitions, chroma key, etc. But then there is also a very steep learning curve as the Resolve GUI is completely unintuitive and has nothing in common with any other software I know. Kinda like Blender in the graphics world :D.
And I would need to invest in another (maybe even 2?) big screen plus some serious NVidia power to be able to work fluently.
18+ years Linux user, wolf-fan, hobby photographer and -filmmaker
EOS 6D, EOS 7D

Andy600

@Danne - You need the embedded color matrices to be correct before white balancing. WB is derived from the color matrices - the single DCRaw color matrix can be used as fall-back but it really only works if the shot is in daylight. ColorMatrix1 and ColorMatrix2 are not intended for color either. Forward matrices assign the color with a DCP or ICC profile handling corrections or creating the 'look'.

A lin-to-log transform using a small 3D cube lut is prone to errors. Tetrahedral interpolation helps but this type of transform should really only be done with a lut that maps code values with hard numbers i.e. a 1D shaper. A high precision 1D lut is not dependent on an apps interpolation algorithms because the deltaE should be below the visible limit. A 3D lut relies on interpolation and this can result in deltaE above the visible threshold for contouring (banding), especially when the transform is so extreme i.e. mapping a lin-to-log/log-to-lin signal.

Most users will not see much of a difference between a precision 1D transform and a 3D interpolated transform but it's all in the numbers and is the difference between a 'that'll do' workflow and a professional one. There is a difference between hardcoding specific input/output code values and interpolation derived code values (it affects granularity) and this can affect all kinds of things later in the pipeline. It gets even worse if you try transforming a colorspace with a 3D lut - this is best done with a matrix + 1D shaper or big 3D cube (linear transform) + a shaper.

The ProRes bit is fine if your on a Mac but FFMpeg ProRes and other PC based variants of ProRes are not the same thing (although you'll be hard pressed to SEE a difference without pixel peeping). A lot depends on the broadcasters requirements. I've known great looking work to fail QC because it was encoded with Miraizon ProRes codecs (discontinued) and that was supposed to be better than FFMpeg ProRes.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

Danne

This is so interesting. Thanks

sgofferj

Quote from: Danne on March 20, 2015, 10:19:50 AM
In my automator workflow cr2hdr-r Andy_600 provided this workflow with three different luts. They are inside the download package and the intention is to use them with ffmpeg to ProRes workflow. Although lowres to suit ffmpeg lut handling they work very good indeed.
Thanks! The lin to rec709 LUT is pretty close to what I had in mind, comparing the result of ffmpeg without LUTs to that.
18+ years Linux user, wolf-fan, hobby photographer and -filmmaker
EOS 6D, EOS 7D

Danne

Also check out how to insert a "same level" aw from dcraw. Prevent variations in wb. It takes the aw from the first file and applies those settings to all the files.

dcraw -T -a -v -c ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_000000.dng 2>&1 | awk '/multipliers/ { print $2,$3,$4,$5 }' > /tmp/magic_l_MLV/mltpl_1

Wtemp=$(cat /tmp/magic_l_MLV/mltpl_1)



(example)

if ls *.wav
then
find . -maxdepth 1 -mindepth 1 -name '*.DNG' -print0 | xargs -0 dcraw -c -H 0 -6 -W -q 3 -r $Wtemp | ffmpeg -i *.wav -f image2pipe -vcodec ppm -r "$FPS" -i pipe:0 -c:v copy -c:a aac -strict experimental -vcodec prores_ks -pix_fmt yuv444p10 -n -r "$FPS" -vf lut3d=/usr/bin/bmcc_ixml_luts/Andy600_cr2hdr-r_luts/Andy600_lin_to_log.cube,lut3d=$lut3d_1_MLV_a,lut3d=$lut3d_2_MLV_a,lut3d=$lut3d_3_MLV_a ../${BASE}_1_$(date +%Y-%m-%d)_0001_C0000.mov



Also when used with sox you can get the audio perfectly matched corresponding to how many frames extracted from mlv_dump.

(prints frames to textfile)
mlv_dump --dng -o ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_ $FILE | awk '/Processed/ { print $2; }' > /tmp/magic_l/framecount_5 ;

    frct_5=$(cat /tmp/magic_l/framecount_5) ;
    rm /tmp/magic_l/framecount_5


(prints framerate to textfile) 
exiftool ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_000000.dng | awk '/Frame Rate/ { print $4; }' > /tmp/magic_l/framecount_fps_5
    FPS=$(cat /tmp/magic_l/framecount_fps_5)
    rm /tmp/magic_l/framecount_fps_5


(divides frames with frames per second, equals seconds) 
echo $frct_5/$FPS | bc -l | awk 'FNR == 1 {print}' > /tmp/magic_l/framecount_result_5
    frct_result_5=$(cat /tmp/magic_l/framecount_result_5) ;
    rm /tmp/magic_l/framecount_result_5


(prints the result here $frct_result_5 and transcodes a new wav file)
/usr/bin/sox_1 ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_.wav ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000.wav trim 0 $frct_result_5 ;
rm ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_.wav

sgofferj

Gotta play around with that. I do generally set WB with a graycard at the beginning of filming or if the light changes, even before each take. As for audio, I usually record externally, so I anyways have to sync that by hand.
18+ years Linux user, wolf-fan, hobby photographer and -filmmaker
EOS 6D, EOS 7D

Danne

Cool. So you use my "greycard" workflow or how do you set wb in dcraw to ProRes otherwise?

sgofferj

Actually, I don't... Footage looks still pretty good. Still in experimental phase, though.
#!/bin/bash

FPS=$1
SCRIPTPATH=$( cd $(dirname $0) ; pwd -P )
LUTSDIR=${SCRIPTPATH}/LUTS
if [ ${2} ]; then
  LUT="-vf lut3d=${LUTSDIR}/${2}.cube"
fi

OUTPUT=$(echo ${PWD##*/} | sed "s/\.MLV//g").mov

if [ ${FPS} ]; then
  dcraw -a -c -H 0 -6 -W -q 3 *.dng | \
  ffmpeg -f image2pipe -c:v ppm -r ${FPS} -i pipe:0 \
  -c:v prores_ks -profile:v 3 -vendor ap10 -pix_fmt yuv444p10 \
  ${LUT} -y -r ${FPS} ${OUTPUT}
else
  echo Give framerate as parameter!
fi
18+ years Linux user, wolf-fan, hobby photographer and -filmmaker
EOS 6D, EOS 7D

sgofferj

All recorded in MLV 1472x626 on my 6D with 24-105 f/4L with or without Vivitar 2x adapter. Used the above script to convert to Prores and except for the Magpie video where I pulled up Gamma a little, I didn't do any corrections in post. Upscaled to 1920x818 when rendered to h.264.

Edit: The WB is almost perfect. On my monitor, the weather sensor appears in exactly the same tone as it does to my naked eye.





18+ years Linux user, wolf-fan, hobby photographer and -filmmaker
EOS 6D, EOS 7D

Danne

Some nice action with the birds on the food table :). I do a lot of nature photography myself.
For stuck situations aw probably won,t change and mostly it might not even be noticable. If you want 100% control you know what to do.
Thanks for sharing.

sgofferj

Yeah, ML is amazing for this kind if stuff! Especially the crop mode helps a lot if you don't have crazy expensive tele glass.
18+ years Linux user, wolf-fan, hobby photographer and -filmmaker
EOS 6D, EOS 7D

Danne

This shot was made with regular and crop mode.


DeafEyeJedi

Love it!!!

In fact there are unique and different kinds of birds around here in Puerto Rico comparing to USA...

Will definitely go out do some non-crop & crop on them when I can...

Thanks for sharing!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109