MLP - Mac OSX batch processing workflow (former cr2hdr-r)

Started by Danne, October 05, 2014, 04:09:34 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

DeafEyeJedi

Not sure if this would help because I don't know if dcraw auto produces the same 'auto' within ACR?

As shot WB within ACR:


Auto enabled WB within ACR:


*Next time I should use a grey card in this particular shot with tungsten light coming in from the lamp in the back behind him along with a 27" inch monitor from his right hand side that produces cooler (blue desktop wallpaper) colors so I set my 5D3 to 5100 in order to compensate for his face (although the background is too warm but NOT sure if this could be a factor regarding dcraw auto being 'confused' somehow?* <--- I also thought this scenario wouldn't be a good way to test dcraw auto as well?

WB Comparisons Part 2 ( cr2hdr-r / MLRV ):
https://vimeo.com/119781083

DNG's were converted via cr2hdr-r in case you were curious.

Also I've given up testing with this particular MLV file (has too many spanned files) so from here on out I'll be using smaller sizes in varieties (Dual-iSO, RAW/MLV, HDR Raw, etc)

One thing I thought it was strange was the fact that I somehow lost one of the two RAW files (shot in HDR Video) which were placed into the HDR_RAW_MLV folder before running app the 2nd time and it produced strange files. Like the name would change on one of the MLV file. Good thing these were just crappy footages I shot in HDR for beta testing purposes!  ;D
https://vimeo.com/119782661

Will try again today to see if I can produce this bug with all three different versions that I have (cr2hdr_r_2.6d, wb_no_short, _long) unless told otherwise if that would help narrow down this bug regarding HDR videos?

Thought we were allowed to shoot HDR videos in RAW/MLV and not just H.264, correct?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

Im still working on the HDR-raw workflow. The originals should move to a folder within Originals folder though. The movie file created will be deleted and the merged files will produce a new version. I have  a version coming soon.
Regarding dcraw wb I think mlv is set to some stuck aSshot numbers so migt look for a different solution. I was thinking of grabbing the first multiplier settings from dcraw auto and apply this to the prores file. Problem right now I cant get tee or awk to print out the numbers to a txt file altough theyre clearly showing in terminal. Strange.
You could test some short mlv files but maybe better wait for the next beta. It, s your call.
Thanks for feedback. Willcheck vids tomorrow.

DeafEyeJedi

No problem. Sounds good. I ended up just using all your ProRes (non dual-ISO) & instead used your DNG's (from Dual-ISO) and exported .mov's from MLRV and combined them all together in PP and currently exporting Dailies from the last BTS I shot to drop off to the client.

Also doing the same (on 2006 MBP) via MLVFS just to see the comparisons.

May be too large to handle though... (260+ GB total of footage)

Fingers are crossed!

lol
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

hehe. Yeah. Mlrv is the way too go. My app is still a big bunch of experiments :).  1.8 version is still to be trusted. I guess ;).
I got some outputs from dcraw going so gonna start experimenting with those numbers a bit tomorrow.

Danne

cr2hdr-r 2.7
https://drive.google.com/file/d/0B4tCJMlOYfirSGhONUszLUtYcEE/view?usp=sharing

- Linear auto white balance when creating ProRes files

The app picks the wb multipliers from the first file and applies that auto white balance setting to the whole file. SHould prevent from wb variations.

- HDR_RAW_MLV_MOV

as it says. Put all files you want to create hdr files from in this folder.

All else is running as before, dual iso and such.

The app should be more robust. Regarding MLV to ProRes they should run all kinds, coming from dual iso, with or without audio and so on. Also big files should work.

Raw to ProRes works but there is more to be done here. Huge files might be an issue.

Still working for getting some sort of lut applied to dualiso files.
Audio might need some rework.

This is beta ;)



Danne

Not really. Dual iso is still running in parent folder and is more or less automated. If it is not a dual iso file it will detect what it is and progress accordingly. HDR_RAW_MLV_MOV is an extra folder/feature to try out. Film in at least 48, 50 or 60 fps when you do :)

dubzeebass

What does it do? Is it like the interleaved HDR frames of yore? I'm just not familiar with HDR in the context of MLV.

Danne

It,s merging the frames as when filming in regular h.264 .MOV setting. Only difference you have to film in 48, 50 or 60 fps also in raw or mlv. When transcoding in post it merges the frames creating a new file in 30, 25, or 24 fps depending on original file. It also corrects the distorted aspect ratio.
It,s more or less for fun but Jerrykil created a nice script which also merges audio to the final file.


Danne

According to Andy600

QuoteA 3D lut for linear to log conversion is not recommended as it relies heavily on interpolation, doesn't map code values correctly and should only ever be used for end-to-end raw grading.


Shortcomings with ffmpeg lut implementation make me go for experimenting with the means available. A work in progress. Might not work in the end.

Experimenting with inserting a lut to dual iso to ProRes files. Since ffmpeg won,t take 1d luts( Thanks to Andy600 for checking out), I found one lut which I liked to start off with that actually worked. It is a lut called ML_RAW_to_LOG_LUT.CUBE which Andy600 created and put up in a forumlink here. In conjunction with curves in photoshop, a preset which also can be inserted to ffmpeg together with a lut, I think I got some progress in getting a good file to work with. HEre are some comparisons.

Examples are graded printscreens so quality only for fast viewing purposes.

Andy600 ML_RAW_to_LOG_LUT.CUBE


Andy600 ML_RAW_to_LOG_LUT.CUBE + curves preset(mine)


Andy600 ML_RAW_to_LOG_LUT.CUBE


Andy600 ML_RAW_to_LOG_LUT.CUBE + curves preset(mine)


Andy600 ML_RAW_to_LOG_LUT.CUBE


Andy600 ML_RAW_to_LOG_LUT.CUBE + curves preset(mine)










DeafEyeJedi

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

DeafEyeJedi

It's actually nice to be able to view footage with MLRV while your app is doing the work(s) in the background.

This feels so good... 8)





Quote from: Danne on February 18, 2015, 08:32:19 PM
According to Andy600

It is a lut called ML_RAW_to_LOG_LUT.CUBE which Andy600 created and put up in a forumlink here.

Do I locate this from LUT BANK on Andy's Cinelog's site?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

As for know there is no lut provided and the one I got is not optimal. Andy600 will hand me over a few alternatives to look into, after that we,ll see if it is practical enough to implement. Things that could occur is ffmpeg lut implementation won,t be good enough or if it takes too long rendering to ProRes with the lut inmplemented. LEt,s wait and see after I checked some luts when I get them.

DeafEyeJedi

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

cr2hdr-r_3.4
https://drive.google.com/file/d/0B4tCJMlOYfirRXd6YVVFQURMak0/view?usp=sharing


Hi guys!

- Two updates.

- Multithreading RAW/MLV to ProRes encoding. (My computer fan is singing) I leave no warranties if anything happens here! Be careful. My computer still working  8).
- Implemented a really fine lut, created by Andy600, when transcoding to ProRes with dual iso raw/mlv files.
The lut is specially made for use with cr2dhr dual iso movie files. There are drawbacks with ffmpeg lut implementation.

Important
Quote"3d luts are really not recommended for this type of thing because they have insufficient code values and produce rounding errors - colors can also be clipped because they are debayered in FFMpeg to sRGB (an output referred colorspace)"Andy600.

I am using ffmpeg with dng files piped from dcraw. White balance is set to auto. The philosophy here is for fast automated ready to use ProRes files coming from dual iso. I think it is achieved much thanks to Andys lut. You be the judge. Andy provides three luts in the packackage. Inserted to the script is the "Andy600_lin_to_log.cube" lut. Below some examples. Beutiful dynamic range.
Thanks a lot Andy600!

Check out the dynamic range coming from dual iso files. Amazing.

Going from dual iso original - Andy600_lin_to_log.cube - Andy600_lin_to_REC709.cube








































DeafEyeJedi

Woke up this AM and noticed 3.4 was still singing in my MBP (total 38 GB worth of footage) towards the end on the last few MLV files...

Looking forward to seeing results as soon as I get back into the hotel tonight!

[emoji108]
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

Nice! I,m working on multithreading HDR_RAW_MLV folder at the moment...

DeafEyeJedi

Nice!!! Also seems the proper order of renderings per 3.4 as it follows:

1) CR2 files
2) ProRes folder
3) RAW_MLV_Mov_HDR

Is this always going to be this way?

Asking because if I wanted to see HDR results right away -- then I would separate HDR into another parent folder & run a 2nd copy version of 3.4!

That way it'll render the HDR videos right away while the other will just do its thing...

Basically like run two separate 3.4's seamlessly at the same time... Worth it?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109


DeafEyeJedi

Fun Stuff from 3.6...

5D3 HDR-Video:
http://vimeo.com/120761009

5D3 Dual-ISO (100/1600):
https://vimeo.com/120761007

On this one I intentionally pushed the EV a bit too much but hey this is just for fun, right?

8)
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

DeafEyeJedi

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109


Danne

cr2hdr-r v 3.8b
https://drive.google.com/file/d/0B4tCJMlOYfirZEJGaFVoaFdrSnc/view?usp=sharing

This beta gives control over wb with the help of a greycard. When including a silent picture DNG of neutral white or grey the dcraw will measure the middle of the "greycard" silent DNG and apply that to the files you selected in a subfolder. This way you will have consistently control over white balance. Check the video.
Be careful, it is all beta, no warranties.

White balance control
https://www.youtube.com/watch?v=S5SAnakpum0&list=UUomeOeghS6wanMOCQ8BtH_A

Got an ok from Visioncolor to use the BMDFilm to VisionLOG-RAW.cube on regular RAW/MLV to ProRes files. Let,s test and see. It is in the version.

Nielebung

Such a fantastic app for MLV on our OSX machines. Running quietly and really fast in background, and i love the organization it does (MLVs in the "originals" File, that's great) !
Keep up the good work Danne !

Danne

Thanks. Glad you like it and use it.
It is all g3gg0, Alex and a bunch of other fine programmers that inspires me to create some good workflows so I can get most out of my filming.
Next up is lut support for prores and refining some other stuff :)