Author Topic: MLP - Mac OSX batch processing workflow (former cr2hdr-r)  (Read 414465 times)

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #400 on: May 14, 2015, 09:53:00 AM »
*FuN sTuFf*

70D shot in:
- RAW 60fps HDR mode iso 100/1600 (I know I could have done it in 48fps but wanted to push it for this test with 70D) hehe...
- added two 3D luts (provided in the HDR_RAW_MLV_MOV folder, no other post work)
02_cr2hdr-r_dan_log_01_warm_+05.cube
03_cr2hdr-r_dan_log_01_to_contrast_01.cube
- Also included a CR2 still picture from location with correct base white balance. (I made an error on this part realizing the CR2 was in the wrong WB so I will have to do another run with 7.6 tmw - my fault on that, D!)














Lots of aliasing w these 70D HDR files, eh?

Stand by for the 5D3 files (currently rendering on 7.6)  8)
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #401 on: May 14, 2015, 04:12:20 PM »
If you skip CR2 file you get auto WB. Also try to push the shadows even higher to get even more light from inside.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #402 on: May 15, 2015, 08:18:48 AM »
Thanks for inputs, D.

5D3 shot in:
- RAW 60fps HDR mode iso 100/1600
- added two 3D luts (provided in the HDR_RAW_MLV_MOV folder, no other post work done)
02_cr2hdr-r_dan_log_01_warm_+05.cube
03_cr2hdr-r_dan_log_01_to_contrast_01.cube













https://vimeo.com/127903841

Although the Auto WB does a good job but perhaps I should be able to grab a DNG with proper WB into the HDR folder (since I forgot to snap a CR2 photo for this test) correct?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #403 on: May 15, 2015, 08:26:20 AM »
Really nice looking files. Want to see more :).
You can retake a cr2 or silent DNG afterwards if you remember the settings or you have the same lighting.. If you grab a dng from the raw file itself it is the same as using auto white balance.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #404 on: May 15, 2015, 09:52:33 AM »
Thanks, D and on top of that I actually used a NIKKOR 24mm f/2.8 D with this test... Not too bad for a FF, huh? (Looks okay on 70D tho)...
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #405 on: May 17, 2015, 09:40:29 AM »
New upload, 7.7 first post.

- cr2hdr-r_speed_preview.workflow (former cr2hdr-r_speed_boost)
- some more fixes

Reworked speed_boost workflow service and added a preview possibilty. Right click on a file/files and select cr2hdr-r_speed_preview at the bottom.
Add luts and/or a CR2(white balance) next to the file/files. You will get one preview png file which you could also use as a starting point for creating complimentary luts in Davinci resolve for instance. Also added som luts and white balance luts which will be added when running a preview file the first time.
When cr2hdr-r_speed_preview is applied to the parent folder it gives the extra speed just as before with the speed_boost service.

*Don,t know if it,s an automator thing but folders can,t be named with spaces or dots or previews won,t work.

When selecting a file


When selecting parent folder

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #406 on: May 17, 2015, 10:35:33 AM »
Once again... Brilliantly done with the newly revamped speed boost service which also allows previews now -- simply amazing!

Thanks again as always, D!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #407 on: May 17, 2015, 10:36:21 AM »
Thanks mate :)

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #408 on: May 20, 2015, 07:20:48 PM »
Installed 8.0 and just shot a few more test shots (RAW HDR, Dual-ISO) on 5D3 w Nikkor 24mm 2.8 -- will post results hopefully before I leave for work.



Thanks, D!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #409 on: May 20, 2015, 08:23:26 PM »
Nice. I,m implementing an automation workflow with included multiple CR2 files in PRORES folder. More about that later :)

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #410 on: May 21, 2015, 10:03:10 PM »
Version 8.2, first post.

Trying out a feature.

- Automated CR2 white balance handling(PRORES folder only)

Workflow
1 - Take a preshot CR2 still image with chosen/correct white balance before filming. When changing white balance take a new chosen CR2 white balance picture before filming the next scene. Keep on going.
2 - When finished simply drop all mlv/raw files together with all preshot white balance cr2 stills. White balance from your CR2 will be applied according to time filmed and group the CR2 to your movie files, fully automated. Do not mix raw and mlv files when doing this. Not working with dual iso files handling.




DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #411 on: May 22, 2015, 05:29:23 PM »
Sounds so friggin awesome! Such a brilliant idea and actually makes shooting more fun and fluidly with the ability to snap a photo after each scene without hesitation. [emoji56]

Also it would be even more awesome if you were able to implement the ability to throw in RAW and MLV along with Dual-ISO files all together but I understand that it may be too complex w the coding.

Great job once again with your amazing work. Keep up the eagerness and it is much appreciated, D!

[emoji108]
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #412 on: May 24, 2015, 08:37:58 PM »
Version 8.4, first post

Calculations found out through dcraw and from discussions with Chmee(Thanks!) here
http://www.magiclantern.fm/forum/index.php?topic=5618.msg148135#msg148135
and here
http://www.magiclantern.fm/forum/index.php?topic=6779.0

 - Added white balance matching with dng files in parent folder

Works the same as with PRORES foldes. Snap a CR2 with white balance and then start to film. When changing white balance, snap another picture and then continue filming. Keep going. When finished drop the mlv/raw files in parent folder with CR2 files. It will automatically choose the preshot CR2 and the group of files filmed after taken the CR2 file. It then takes the next CR2 and moves on to the next group of movie files.

- As these CR2 files will interfere with dual iso CR2 conversion you have to write WB or wb to ONE of the white balance CR2 files. That way it will pass on the dual iso conversion and use these CR2 files specific with mlv/raw files. Don,t mix mlv and raw files as in the picture when using wb CR2 files.

example






DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #413 on: May 27, 2015, 10:07:45 AM »
*Fun StuFF 8.4*

Shot on 5D3 earlier this year in h.264 format (not RAW) 720p 60fps HDR video on 5D3 and converted files with 8.4 which worked well -- Thanks, D!



















https://vimeo.com/128966965
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #414 on: May 27, 2015, 10:48:06 AM »
Cool, thanks for sharing

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #415 on: May 27, 2015, 05:29:39 PM »
Up next will use these files again and export from AE with Pixlemotion plugin per your suggestion from earlier.

[emoji3]
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #416 on: May 29, 2015, 01:54:50 PM »
version 8.5 first post

Reworked the speed_preview workflow.
http://www.magiclantern.fm/forum/index.php?topic=13512.msg147712#msg147712

By right clicking on a raw/mlv/mov/MOV file you get one preview png frame from the chosen file/files works with following files.

- MLV/RAW regular files
- Dual iso movie files
- Higher fps, 48, 50, 60 fps (will autocorrect aspect ratio)
- HDR recordings - You have to put in the word HDR in the file for the preview to recognize it as a HDR file! example M09-1234hdr.MLV
- MOV, mov regular files (might be handy previewing 3d luts in general

Main usage is for a fast workflow with trying out 3d luts and checking CR2 white balance before taking of in the PRORES folder.
Simply add the chosen lut and/or the CR2 wb file next to the files and run the preview service.

Start up idea I had from dmilligan compression mlv tool which works with an automator service.

Code: [Select]
for f in "$@"
do

By adding above it works on selected files.

Then I added below for listing directory

Code: [Select]
cd "`dirname "$f"`"
Incorporated 3d lut, wb support, used mtime to only erase 30 sec old clips and used g3gg0, A1ex cool mlv_dump 
Code: [Select]
-f frames to extract only a few frames from the file. This and a lot of other stuff :)
All scripts are provided in the download package.

Added some more luts created in da vinci resolve. Will be automatically copied to folder when using the speed_preview on a file.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #417 on: June 04, 2015, 10:10:58 AM »
Just downloaded 8.8...

tempted to check out the new preview method!

 :D

Thanks, D!

*update*



This can be a fun feature (I often have to remind myself to delete png files after each previews otherwise it won't take effects if I were to leave them in) though not sure if this was to be expected?

It can sometimes be confusing when trying to decide which log/luts I used previously that looked good from last time I used the app (any suggestion on how to make myself familiar with them ) other than spending more time with it in order to speed up the workflow -- no pun intended! lol

Looking forward to see the results!

*cheers*
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #418 on: June 04, 2015, 11:20:17 AM »
Your welcome.

* Thanks for feedback. What would be a faster workflow with previewer?
Don, t understand the question with luts. I always use dan_log_01 or 02 as base log and then it is up to the user to add wb and/or contrast_03 lut.
HDR folder has dan_log_01 hardcoded in the script so no log is needed in that folder.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #419 on: June 04, 2015, 12:39:49 PM »
I guess I can easily forget which LUTS to use after applying the logs (either dan_log_01 or 02) then this is where I get a brain freeze (or writers block) if I may say...

so I guess a faster workflow would be if possible if I can just run preview again with different LUTs AFTER previously running it as oppose to having to delete the PNG files each time? (not that I do mind but it kind of slows down the thinking process a little bit) unless I'm missing something...

Does that make sense?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #420 on: June 04, 2015, 12:53:04 PM »
Yea, I had automatic png removal before. Might throw that back in there. Will think a bit.
Another issue is that preview doesn, t work with spaces in folder names. Annoying. Tried adding quotes in script but no succes.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #421 on: June 04, 2015, 01:21:15 PM »
I know I read that and didn't use spaces or dot which worked fine.

But with It does not.

Not that important comparing to others I would say.

Hmmm?

Great work though as I'm very pleased with having the ability to keep myself from exporting ProRes again only because I didn't Iike the look from the logs/LUTS I chose...

Overall it's an excellent addition & a great tool -- just needs a little refinement and it'll be gold!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #422 on: June 04, 2015, 02:26:04 PM »
Updated the previewer. Download in first post. On rerun of the speed_preview service png files will be erased. All png files within 20 minutes that is. Older files you have to manually delete.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #423 on: June 05, 2015, 11:44:36 PM »
Version 8.9

I added a batch_folder inside the dan_log_luts. Dan_log_luts folder is created when running speed_preview service on a raw/mlv file the first time. You can put in up to 10 files in then batch_folder and then run the speed_preview on a selected file. This will run all .cube files from inside the batch_folder and generate their names. Typically you start of with testing all your logs. Find one then go for lets say white balance luts, pick one, and finally run all your rec709(contrast) luts and pick one. You now have a combination of three luts tested and ready for batch run in the PRORES folder.

Might clear things a bit with short tut.


DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3401
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: cr2hdr-r OSX batch conversion app (dng, CR2 ,MLV, RAW, HDR video)
« Reply #424 on: June 06, 2015, 06:54:54 AM »
Now I'm starting to smell gold as we move on to 8.9 and can't wait to get my hands on this as soon as I get home from work!

Love how you're digging in like crazy with speed_preview especially the newly implemented batch_folder -- nice tutorial btw!

 8)
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109