MLP - Mac OSX batch processing workflow (former cr2hdr-r)

[aschille84]

Noise reduction sounds intersting!
One question, is the prores files produced from DNGs In MLP in linear color space by default?

[Danne]

Hi aschille84.

Settings defaults are

dcraw $icc +M $br $dpf_a $any $ga -c -6 -W -q 3 $r $wb

Where the important picture settings are
dcraw +M -H 2 -g  -6 -W -q 3

+M/-M     Use/don't use an embedded color matrix
-H [0-9]  Highlight mode (0=clip, 1=unclip, 2=blend, 3+=rebuild) Set to -H 2(slight higlight recovery. Works very well)
-g <p ts> Set custom gamma curve (default = 2.222 4.5) Gamma set to -g 2.4 0
-6        Write 16-bit instead of 8-bit This setting is not linear altough -H 2 moves towards linear output. For linear output use following
-4        Linear 16-bit, same as "-6 -W -g 1 1" Not used in MLP
-W        Don't automatically brighten the image
-q [0-3]  Set the interpolation quality


You can change these basic settings by removing the gamma and any_settings file in A_lut_hold folder.
 

[aschille84]

Thanks, Danne!
Ill try to get some results with this. Im interested in using this with color space transfrom latest resolve update.
So can I change the settings in MLV_RAW_dng_dcraw_any_command.txt to +M -H 2 -g  -4 -W -q 3 or do I have to change this in another file?

[Danne]

Well, some dcraw code settings might not apply since they are not put into variables.

Easiest would be to change in the source_code_MLP folder in following files(look at the bottom of each file). Put in the same code in all four files(multithreading)
fold_prores_01.txt
fold_prores_02.txt
fold_prores.txt
process_01.txt
After changing stuff in there double click A_move_content_to_workflow.command in the source_code_MLP folder and then install MLP after doing your changes. Don,t forget to migrate binaries with move_binaries.command before installing.


Alternative you do these changes straight inside an already installed MLP versionand test your results.
Go here ~/Library/Services/MLP.workflow/Contents/ and search for these files
fold_prores_01.command
fold_prores_02.command
fold_prores.command
process_01.command

Let me know how it works or I help you out later. Prepare for dark files and even darker with -H 2 selected .

[aschille84]

Ok, so I did a test.. Ended up making a custom LUT to match the DNGs in a ARRI Log-C timeline in DR12.5. Ungraded prores 444 to the left, slightly graded to the right.
The last clip is a HDR clip with the enfuse_correct_fps.txt setting. I couldnt get the scaling to work with the HDR workflow, not a big deal though.. Its low res 6D, but the colors are still great.

[Danne]

Cool stuff. HDR is great working with.
How are you using the scale MLV_RAW_MOV_dng_scale.txt file? I just tested changing to 2500k in by replacing scale=1920:-1, with scale=2500:-1, and it worked over here.

[Danne]

New version uploaded.

Created a menu selector in which one can add different settings in ffmpeg. It works with previewing and HDR as well. Report any bugs since I had to change a lot of scripts.
TO activate the menu add the FFmpeg_settings.txt file to A_lut_hold folder prior to running MLV/RAW, MOV, or folders with dng files. Add settings and when ready select run MLP.
You can also exit the menu before running MLP and do previewing and then run MLP again when ready.

[aschille84]

Cool stuff. HDR is great working with.
How are you using the scale MLV_RAW_MOV_dng_scale.txt file? I just tested changing to 2500k in by replacing scale=1920:-1, with scale=2500:-1, and it worked over here.

Thanks! I agree.
I plotted in the same resolution as the DNGs 1600x680 and it worked great with non-HDR footage. (Otherwise movs streched vertically).

But with the enfuse process the same scaling didnt work. (Thou the original resolution was different, and I kept the 1600x680 scaling ). Need to check again.

Edit: due to different res, not hdr-process

[Danne]

Strange.
Could you show me exact what you put in the A_lut_hold folder and I will try to reproduce.
Maybe even put up a test file(short one)?
Also, you should keep the 1 and only put in the width number scale=1600:-1

[aschille84]

Excited to try out, the denoise function! Will test it soon.

It checked the scaling again. When using scale=1600:-1, it scales the picture to 1600x1133 i. e 16:9. With different resolutions it does keep the fixed width, but scales any aspect ratio to 16:9.

If i plot the same resolution as the clip scale=1600:680, its okey for that clip. But not several clips with different resolutions. Workaround is to only process clips with same resolution, I guess.The clips used were: 1600x680 and 1280x582:

Link to MLVs:

https://www.dropbox.com/sh/gommijju5i2dmaf/AACt9JtJ7l8ZHp2lfcHvJtJha?dl=0

Folder:

https://www.dropbox.com/sh/fy1gio3lqjl9b07/AACCohFg5JWbB4yNl8oKSBs2a?dl=0

[Danne]

Thanks a lot for sharing your findings. I think I come to a solution.
Seems going from tiff images to .mov files keeps a pixel dimension tag in the exported file. Even though the image size shows 1600x680 the aspect ratio will still be 1600x1133. This tag cannot be erased with exiftool but ffmpeg has a setting that will let you alter aspect ratio. If you want 16:9 relation you write the following in the scale text document.
scale=1600:680,setdar=16:9,

Let me know if this works.
More info here
https://ffmpeg.org/pipermail/ffmpeg-user/2012-April/006201.html
I,ll update this post with more findings later.

scale=1600:680,setsar=1:1,
is even better. Might automate this one into the script.



*update
Seems to be a problem with this version of mlv_dump changing the default scale tag on regular fps files. This creates wrong aspect ratio on your files. Will check.
You filmed these clips in regular 1080p mode and not 720pmode?

[aschille84]

Thanks for helping out, you do a great job with this app, @Danne. Even for this very specific need ill come back with more findings, and maybe some travelogue summer vids!

[Danne]

Spent a few hours trying to understand some of the code around how to detect whether or not to add the default scale tag. Checking in ml_dng branch(dmilligan) he calculate the active area (width and height) from the raw buffer(dng)_info) and if height can be doubled before it meets max width it can be determined for squeezed footage.
Another assumption is <=720 and if that doesn,t match it continues to ask if rawW could be < 2000(width).
However. Since your footage orignally was filmed in 1600x680 mlv_dump wrongfully assumes it,s squeezed footage.
The solution right now is for you to either film an aspect ratio that won,t be taken for squeezed footage or run footage through mlvfs. Or export it back to correct aspects with ffmpeg.
To fix this we must have mlv_dump determine the active area from the raw source rather from the raw buffer like in mlvfs. I see what I can do here but I will need some help  .

[Danne]

Uploaded a new version.
I might have fixed the rare default scale tag problem. It has to be tested with some different cameras since I only have a 5D mark III right now.
Instead of checking for 2.0 I put in to check for 2.5 which could be the better number here.
if(aspect_ratio > 2.0 && rawH <= 720)
is changed to
if(aspect_ratio > 2.5)

How to verify
Testing with maybe three different aspect ratio settings going from low medium to highest and camera set to mv720 mode(slowmo) and develop to dng files. Open in acr or run in MLP. Footage should come out unsqueezed.
Or simply scroll through aspect ratio menu and calculate the settings addition Height*2.5. The numbers should be more than width setting if filming mv1080 and below if filming mv720 mode.

Looking forward to some lovely summer vids aschille84

[DeafEyeJedi]

Nice find @aschille84 and MUCH Thanks for the quick fix @Danne (even without help) and I'll definitely do the suggested test verification with slo-mo files in mv720 mode for unsqueezed footage. This will be a huge time saver as it usually take forever to do so in DR12.5 (I don't know of a short cut to fix AR for each MLV within timeline as copy and paste doesn't do the trick) and also this can be a bit quicker in AE anyway.

[Danne]

There is some setting for aspect ratio in resolve. At least there was before. When that was set one can work freely with squeezed dng files.

[DeafEyeJedi]

But what about working with a bunch of different footage (mv720 mixed with mv1080) together or something in that regard?

[Danne]

Then we have a problem I suppose

[bobbyshots2]

Hey,
Im curious if it would be possible to wirte a script to upscale the dng's to say 4k if possible?

[Danne]

It,s possible to do this already inside this file. MLV_RAW_MOV_dng_scale.txt.
Simply replace scale=1920:-1, to scale=4000:-1, and put the MLV_RAW_MOV_dng_scale.txt file into A_lut_hold folder prior to running MLP. Inside the file you can also choose between different scaling techniques. This will output a mov file to 4k. Upressing the actual dng files isn,t possible.

[bobbyshots2]

Ive been using ml raw for about a year now and i love the dynamic range that it gives me. I do alot of run and gun style shooting and picked up a sony a6300. I love the slog2 color and i want to match up color as best i can. MLP is the fastest workflow and most powerful that i have used to process my raw file.
Long story short ...
MLRAWviewer can convert to slog2. How can i use MLP or can that file be create in MLP with the other log files to convert my raw files to slog2 mov's?

Here is an example of a video i did with mixed footage from a6300 and 5dIII raw

[Danne]

Thanks for sharing Bobbyshots! Great vid
Could you try these luts?
Slog2
https://drive.google.com/file/d/0B4tCJMlOYfirOVU3eEZESXFIanM/view?usp=sharing

Also run them with MLV_RAW_dng_dcraw_any_command.txt for highlight recovery(knee)

[bobbyshots2]

Hey,
I came across this link where the guy upressed 1080 dng to 4k thats why i ask about up ressing.
I figured his process of using after effects to upressed you guys could do using chdr or repacking or some other step.
The file that you pointed to does the job but does it degraded the quality?does it create the MOV in 1080 then upres the MOV to 4k?dioes the  quality get degraded? Maybe only if you pixelpeep?


http://blueoceanproductions.com/blog/?p=89

[Danne]

It really depends on upscaling algorithms how degraded the picture is gonna be. Of course a higher quality source file will yield better results. One have to test and see what works or not.

[bassrebel]

Hi guys!

I feel extremely stupid for this... but here it goes:

In the HOWTO it says "2 - Extract the enblend-enfuse-4.0-mac.tar Move enfuse to Place_binaries_here folder 13,6 MB in size' But I don't seem to find a file named "enfuse" inside the extracted folder.

Well, I also need to mention that the file downloaded is enblend-enfuse-4.2.tar not 4.0

IDK if it makes any difference.

Any leads here? Is there a folder I need to move or what?

Thanks!