Switch
Switch automator app is a multiprocessing program being able to process dualiso as well as non dualiso files coming from Magic Lantern generated CR2, DNG, RAW and MLV files. Different magic lantern related workflows are all being simplified through different self explanatory bash menus which all are started from within Main menu.
Following processing workflows are included:
* Process dual iso generated files from multiple cameras
* Process CR2, dng, RAW and MLV files(both dualiso and non dualiso)
* Export to ProRes4444 and proxy(FFmpeg, dcraw)
* Process other files such as mov, mts, mpeg etc to ProRes(X to ProRes menu)
* HDR processing from MOV files. (ffmpeg tblend averaging filter. 48, 50, or 60fps recommended)
* 3D lut support(cube)
* White balance correction
* Darkframe averaging automation(mlv_dump)
* Flatframe automation(mlv_dump)
* MLVFS workflow integration
* MlRawViewer integration(previewing)
* Multithreaded
* Full auto mode
The program itself is depending on cr2hdr(a1ex) and mlv_dump(g3gg0, a1ex), mlv_dump_on_steroids(Bouncyball), raw2dng(a1ex), dcraw (https://www.cybercom.net/~dcoffin/dcraw/)(dave Coffin) and FFmpeg (https://www.ffmpeg.org/)(Fabrice Bellard) for processing the different dualiso generated files. Processing includes white balance correction through exiftool (http://www.sno.phy.queensu.ca/~phil/exiftool/)(Phil Harvey), exiv2 (http://www.exiv2.org/)(Andreas Huggel) and bwfmetaedit (http://bwfmetaedit.sourceforge.net/)(FADGI) for audio metadata adding and if adobe dng converter is installed in applications folder Switch is able to include perfect correlation of white balance going from CR2 to dng files. Switch also includes a menu selector created in bash from where the user can select or deselect different settings inside Switch. Included in code as well is dfort pixel fixing lists for cameras with focus pixels such as eos m, 650D etc. https://bitbucket.org/daniel_fort/ml-focus-pixels
Also incorporated is a MLVFS workflow menu. For this to work and take effect you need MLVFS (http://www.magiclantern.fm/forum/index.php?topic=13152.0)(dmilligan) installed onto your system. In conjunction with MLVFS I also put in very fine movie player MlRawViewer (https://bitbucket.org/baldand/mlrawviewer)(baldand).
Switch works with different menus created in bash from where the user can select or deselect different settings inside Switch. The program is able to run all processes in a chain so just fill up the folder with various files and after selecting settings in the different menus simply run Switch and wait for it until it,s done.
A very special thanks to Bouncyball for dedicated support and groundbreaking development(mlv_dump_on_steroids, fpmutil, RAW to MLV converter) in general.
Source code
https://bitbucket.org/Dannephoto/switch-git/src
Mac version
https://bitbucket.org/Dannephoto/switch-git/get/b832bc4315e8.zip
Ubuntu version
SWITCH_LINUX.tar.gz (https://bitbucket.org/Dannephoto/switch/downloads/SWITCH_LINUX.tar.gz)
linux_source_code (https://bitbucket.org/Dannephoto/switch/branch/Switch_linux)
Linux_Forum_post (http://www.magiclantern.fm/forum/index.php?topic=15108.msg188867;topicseen#msg188867)
Main menu
(https://s1.postimg.cc/45zhxljy7/Screen_Shot_2017-07-18_at_01.09.31.png)
HOWTO
1. Move Switch to applications folder
2. Double click Switch and select a folder with dual iso CR2, DNG, dng or MLV files then navigate from the Main menu.
3. For compressing and matched white balanced Adobe dng converter needs to be installed into application folder. Switch works perfectly fine without ADC but white balance will be slightly off balance due to missing camera calibration information.
4. 3D luts can be added to both ProRes and proxy folders to further tweak the output.
Gatekeeper
To supress gatekeeper hold ctrl button down(mac OS Sierrra) while opening the application the first time. You can also change permissions from within privacy/security settings.
For coders
About working inside the source_code folder
Inside source_code folder there is a Build_dmg_package.command script. When double clicked it will copy and overwrite all .txt files over in source_code folder into the Switch one level up in source tree. After copying a mac dmg package will be created. Run chmod u=rwx on the script for full privilegies if needed.
MAIN
If you want to include changes done to Switch_MAIN.txt these have to be copied by opening up Switch in automator. Automator is installed in your system by default.
Thanks to: a1ex, g3gg0, bouncyball, Dave Coffin, Fabrice Bellard(FFmpeg community) Phil Harvey, Andreas Huggel, dmilligan, baldand, so-rose, Chmee, dfort, deafeyejedi.
Official GPL statement(Magic Lantern) (http://www.magiclantern.fm/forum/index.php?topic=13335.0/)
Interesting stuff... Hopefully for you to maintain two apps at once won't make you pull your hair out!!!
I'm literally done with the move from apt to house.
Now gonna get on my Mac mini and play with all your new toys online shortly...
Thanks for the surprises as always, D!
[emoji5]️
Na, this little sidedish hopefully won,t be much more than this :).
Is it only for Mac? If there is a Windows version, I would like to try it.
I only know how to do this for mac unfortunately.
*feedbacks*
Seems to work fine for CR2 files but does NOT seem to pick up for MLV's from 70D?
Funny thing is the first 3-4 minutes of the spinning ball in status bar showed "0%" and got me thinking that it wasn't working...
https://vimeo.com/126981665 (https://vimeo.com/126981665)
But then soon enough I was wrong! 8)
Should I run this again with 5D3 files?
Mlv?
It, s main purpose is to transcode cr2 files. Secondary, dualiso dng files. MLV and RAW is not in this workflow.
Thanks for testing
Oh that's right I don't know why I thought I read it in first post that it was for all Dual ISO files (but really it's only for photos as in regular & Dual ISO DNG's from FRSP as well as Dual-ISO Cr2's) is that right?
The purpose for this service workflow to speed up the pace of converting photos rather than wasting time looking for other files as in your other app w cr2hdr-r - correct?
Purpose is for dual iso stills, CR2 mainly. The speed on still CR2 conversion and functionality is more or less the same from cr2hdr-r and this workflow.
Copy that!
So if I Took a FRSP in dual ISO would that work or no?
Thanks!
Updated with latest cr2hdr from A1ex. Compiled by Kichetof. Thanks!
New upload. First post
- Abilty to also right click directly to a file or select multiple files (CR2,dng dual iso) and start conversion instead only from right clicking a folder.
Might be handy for fast checking a single or more files. Selecting a folder is still fastest (multithreaded)
(http://s29.postimg.org/a81cl0b93/Screen_Shot_2015_05_14_at_16_23_02.png)
If you right click on file(s) it isn't multithreaded but right click on a folder will be, correct?
Yes. Right click multiple files or a single file, not multithreading. If you want speed. Do your batch from a folder as usual.
Hello - I downloaded the dualiso_to_DNG service - then selected a folder where the dual-iso CR2s are located, and right click > Services > dualiso_to_DNG. I checked to ensure that the DCRAW file was copied to the /usr/bin (had to temporarily disable System Integrity Protection to successfully copy dcraw to /usr/bin).
I see the automator wheel on the top spinning for a second, then it disappears.
I opened the folder, and I do see ORIGINALS folder containing the CR2 files. However, no DNG files were created.
My guess is that this service couldn't find the cr2hdr file? Where can I download the cr2hdr (latest) file and where should I put it so this service can properly convert the dual-iso file to DNG?
OR perhaps this service is already 'retired'. is there a new service out there that replaces it?
Thank you.
Hi ciriuss925. Yes, It,s no longer maintained mainly due to E l capitan installation issues. I will put a note about it. Use MLP and check page 5 in the user guide. It should work straight away.
MLP
http://www.magiclantern.fm/forum/index.php?topic=13512.0
hello Danne - thank you so much. I am able to get the MLP installed and so far the dualiso DNG files are rendering quite beautifully!
Good to to hear :) The processed dng in MLP will bring in the correct white balance metadata as well so that,s yet another reason to use it.
I updated the first post with an updated automator app to be used with dualiso files (CR2, DNG, dng, MLV). Mainly for a still image workflow. It,s stand alone and I made an attempt to obtain exact white balance conversion going from CR2 to dng but the roadblock still persist.
http://www.magiclantern.fm/forum/index.php?topic=10265.msg118958#msg118958
A workaround is to have adobe dng converter installed which will make exiftool grab and calculate the camera calibration numbers into the dng AsShotNeutral tag. Fully automated.
Feel free to test and let me know how it works. All scripts are multithreaded.
I put in a lot into the scripts so now it will work with dual iso files as well as non dual iso files. Just put all files in the same folder and cr2hdr.app will know what is what. Download in the first post.
Lovely stuff, @Danne as per usual. Even the spanning files from Dual-ISO FRSP works flawlessly (big THANKS for that!) and will continue to use this wonderful tool for batch processing all my photos (including MLV stuff) which is insanely awesome!
Oh btw I like how you added the mlv_dump settings (ml) under the menu selection as one of the options before running your cr2hdr app -- complete genius!!! 8)
Thanks Deafeyejedi :). Always nice to have you aboard. Yes, think it,s great to be able to tweak command line settings in both binaries through a menu.
Question -- would it be ideal to make it possible to include some kind of custom settings for dcraw such as -H 2 -b 2 for MLV files and such?
All base settings in the menu are based on -H 2 excepts for rec709. I don,t really see not to do this since -H 2 are more or less scene referred(which is what raw footage wants) opposed to otherwise clipped highlights.
Been porting some more good stuff into cr2hdr.app. This workflow more or less outgrown its origin name and a1ex outstanding cr2dhr is now part of all kinds of other processing possibilities. Anyway. The main reason for this workflow is that it,s all menu based and supposed to be easier to work with than straight in command line or as in MLP putting files in a dedicated folder for activation.
Once you select a folder with files cr2hdr.app now starts with a main menu and then you take it from there.
The "select new output" feature works with MLV files for now but the rest will be implemented later on.
Main menu
(https://www.dropbox.com/s/di3yrwy5ntp915p/main.png?raw=1)
Darkframe automationnew upload
Spend a few hours with different kinds of automation around darkframe averaging with mlv_dump(g3gg0,a1ex). With imagemagick cr2hdr.app can read the rgb pixel and exposure level. This way I first automated HDR processing so that the mov file will automatically be detected as HDR or not.
With this in mind I also automated darkframe processing to make this much easier to integrate in a mlv workflow. The workflow is pretty straightforward. Record MLV footage MLV and see to it that you record a short darkframe with the cap on the lens. If you change settings film another darkframe to go with that footage etc. cr2hdr.app will create averaged files and then match those to the footage you filmed. It will match according to iso, camera model, frames per second and shutter settings.
Don,t film the darkframe longer than 8-10 seconds. 3-4 seconds will do.
The easy way to understand it:
1 - Record MLV footage as usual
2 - Record a short MLV file with the lens cap on(your darkframe footage)
3 - If you change settings film another darkframe that match the new settings
4 - Put all the files in the root folder select (A) from the main menu and then (r) to start cr2hdr.appcr2hdr.app will search for three different criterias to determine if it has a darkframe shot to work with. It checks if the file has less than 400 frames, that it is smaller than 1gb in size and finally it uses imagemagick to read the rgb pixel before it goes into averaging the file.
Here is a short demonstration.
Code starts at line 96
https://bitbucket.org/Dannephoto/cr2hdr/src/8395b3a49108eb2256871ed40bb559f3f6407038/source_code/cr2hdr_MAIN.txt?at=default&fileviewer=file-view-default
Added support for flatframe correction.
What,s a flatframe? Start reading here.
http://www.magiclantern.fm/forum/index.php?topic=17795.msg171607#msg171607
WHat does it do? The little I tested this out it will even out vignetting and other uneven anomalies. You can run both flatframe and darkframes in cr2hdr.app. I tested this on some clips filmed with canon eos m and it got rid of most of the focus pixels.
Darkframe averaging is automated but a flatframe is selected when choosing flatframe from the main menu. I,ll might put some workflow video when I get some time for it. Meanwhile feel free to test it out as is.
Flatframe script starting at line 97.
https://bitbucket.org/Dannephoto/cr2hdr/src/57b9a6af31cbc218ac67f975e0c925e30797fce2/source_code/cr2hdr_MAIN.txt?at=default&fileviewer=file-view-default
For future reference here is an example of how to work with both darkframe and flatframe subtraction through cr2hdr.app.
Been testing darkframes and flatframes and now I got some scripted automated solutions for both darkframes and flatframes in cr2hdr.app
http://www.magiclantern.fm/forum/index.php?topic=15108.msg146822#msg146822
Darkframe subtraction really cleans up color noise significantly on higher isos(example 800 iso) and flatframes evens out vignetting among other things. I ended up with a flatframe filmed at f5.6 while cat clip was filmed at f1.4.
A lot more to learn and test with in regard of creating the better flatframe. Here is an interested read about it. Note that I didn,t follow the rule.
https://photographingspace.com/how-to-create-dslr-and-ccd-flat-frames-for-astrophotography/
Original dng files herehttps://drive.google.com/file/d/0B4tCJMlOYfirS2RoRV9jUGVVOE0/view?usp=sharing
Original(https://s13.postimg.org/vfm3tnrjb/Screen_Shot_2016_11_19_at_23_00_46.png)
Darkframe subtraction added(https://s13.postimg.org/6l2lzl6p3/Screen_Shot_2016_11_19_at_23_00_54.png)
Darkframe+flatframe added(https://s13.postimg.org/da95fls13/Screen_Shot_2016_11_19_at_23_01_00.png)
Included in the root folder is one darkframe, one flatframe and one movie clip.
cr2hdr.app will automate processing the way it,s shown in the movie below.
Yesterday I started playing with the experimental Crop mode 3x3 720p in the canon eos m. I found out it,s actually possible to film this way with H.264 when raw movie tab is set to off. That also makes it possible to select 60fps 720p in canon menu. Now when filming HDR you,ll get a much cleaner image than otherwise, much closer to mv1080. The output will be skewed but that is easy to fix with ffmpeg. There is also the problem around focus pixels but hopefully this will either be fixed somehow with another raw stream or dfort,s pixel maps might work with a certain ffmpeg filter. I can,t say that they disturb in the footage right now but you might see them if you pixel peep.
I put in a feature in the scaler which is included in the X to ProRes menu. Select the appropriate one when exporting HDR material.
(https://www.dropbox.com/s/x0ryvcyvbi47ei5/scaler.png?raw=1)
Tutorial
https://youtu.be/S_jeLyA6LWI
Test footage
https://youtu.be/Suj-tknA3mI
The crop module versions I,m using are coming from dfort here. Use them at your own risk of course.
https://bitbucket.org/daniel_fort/magic-lantern/downloads/magiclantern-x-perimental.2016Nov23.zip
Quote from: Danne on November 24, 2016, 01:19:19 PM
Yesterday I started playing with the experimental Crop mode 3x3 720p in the canon eos m. I found out it,s actually possible to film this way with H.264 when raw movie tab is set to off...There is also the problem around focus pixels...
Please tell me you aren't seeing focus pixels when recording to H.264. They should only appear when shooting MLV/raw. Well, they shouldn't appear at all but we're dealing with it. Removing focus pixels from H.264 footage would be even more challenging than raw.
I,m sorry dfort.
Here is the original H.264 file.
https://drive.google.com/file/d/0B4tCJMlOYfiraHhURmdXV3N4d1E/view?usp=sharing
(https://s17.postimg.org/t46rvftzj/Screen_Shot_2016_11_24_at_18_17_32.png)
(https://s17.postimg.org/dwqsb324v/Screen_Shot_2016_11_24_at_18_16_52.png)
So this only happens with the crop_rec module, right? All the other H.264 video modes are fine? You said you can deal with them with ffmpeg?
I think this is only happening with crop module.
Well. There is a filter called delogo which works like dcraw neighburing pixel fix but I,m not sure this will work with multiple coordinates in any easy fashion. RIght now I actually include the deflate filter to the HDR footage in the crop rec module. It works good and it,s hard to see the pixels if at all on the exported footage.
MLVFS_workflow(download in first post)
Lately I,ve been trying to figure out how to make good use of the command line (http://www.magiclantern.fm/forum/index.php?topic=13152.msg174915#msg174915) options in MLVFS(dmilligan) and to find a fast way to develop my MLV files into folders with dng files. Needles to say MLVFS works with dfort pixel focus lists as well as with 10/12-bit files and is by far the fastest way to get into dng editing.
In the other script program MLP (http://www.magiclantern.fm/forum/index.php?topic=13512.msg130562#msg130562) I simply leech on the the automator created partition folders but in cr2hdr.app I have been porting most of the command line options into a bash menu selector. I choose to default into resolve folder setting.
Also different is that cr2hdr.app creates symlinks of the dng folders which is then put into your root folder. This makes it work seemlessly with the other functionalities in the program. This idea I had for a long time and after some discussion with so-rose, author of convmlv (http://www.magiclantern.fm/forum/index.php?topic=16799.msg163675#msg163675) I decided to give it a go. It seems to work well.
What cr2hdr.app does it creates two folders, name/of/your/root/folder_mount and name/of/your/root/folder _storage inside A_ORIGINALS folder and then goes from there creating the mlvfs mount and copies/creates the symlink folders. You can check the menu and the movie for further understanding.
Treat this workflow as bleeding edge and test it a lot with back up files before doing any important stuff with it.If you get any ideas feel free to share them here.
MLVFS workflow menu(https://www.dropbox.com/s/g82ls0u9943qzxv/mLVFS.png?raw=1)
The MLVFS_workflow menu is based around this script.
#root folder name
root=$(grep -o '[^/]*$' /tmp/DUALISO/"path_1")
kill $(cat /tmp/"$root"_pid) 1> /dev/null 2>&1 &
rm "$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS/"$root"_storage/*.IDX &>/dev/null
find "$(cat /tmp/DUALISO/path_1)"/ -maxdepth 1 -type l -delete
mkdir -p "$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS 1> /dev/null 2>&1 &
mkdir -p "$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS/"$root"_storage 1> /dev/null 2>&1 &
mkdir -p "$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS/"$root"_mount 1> /dev/null 2>&1 &
if ! ls "$(cat /tmp/DUALISO/path_1)"/*.MLV &>/dev/null
then
if ! ls "$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS/"$root"_storage/*.MLV &>/dev/null
then
echo > /tmp/no_files
fi
fi
if ! ls /tmp/no_files
then
mv -i "$(cat /tmp/DUALISO/path_1)"/*.M* "$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS/"$root"_storage &>/dev/null
cd ~/Library/Services/MLVFS.workflow/Contents/
./mlvfs "$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS/"$root"_mount --mlv-dir="$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS/"$root"_storage -f -o local -o volname=""$root"_mount" -o allow_other --resolve-naming $(cat "/tmp/"$root"_settings.txt" | sed -e 's/^[ \t]*//') &
echo $(ps aux | pgrep mlvfs) > /tmp/"$root"_pid
cd "$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS/"$root"_storage/
check=$(ls -r *.MLV | head -1 | cut -d "." -f1) &>/dev/null
cd "$(cat /tmp/DUALISO/path_1)"/
while ! ls -d "$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS/"$root"_mount/"$check"MLV_* &>/dev/null
do sleep 1
done
ln -s "$(cat /tmp/DUALISO/path_1)"/A_ORIGINALS/"$root"_mount/* "$(cat /tmp/DUALISO/path_1)"/ 1> /dev/null 2>&1 &
open "$(cat /tmp/DUALISO/path_1)"/
else
rm /tmp/no_files 1> /dev/null 2>&1 &
fi
Lately I,ve been adding some more stuff around darkframe automation(mlv_dump) and around mlvfs integration into cr2hdr.app. It,s now posible to have a storage folder containing already created(averaged) darkframes or add non averaged darkframes to the storage folder and they will be averaged. You activate this by selecting the storage folder before running any processing of mlv files. You can then leave this storage folder activated all the time or disconnect it from the Main menu the next time you open up cr2hdr.app.
Select (A) from Main menu to get to this:
(https://s23.postimg.org/ow39ez7qz/image.png)
The other thing is I noticed you can actually change metadata tags in the virtual dng files in mlvfs with exiv2(A.Huggel)http://www.exiv2.org/
I therefore added the possibility to run auto white balance on the virtual dng folders if anybody likes that. It,s reasonably fast despite the fact that it will iterate and apply tags to all dng files. It starts and will then run in background so you can keep working with other stuff meanwhile.
Also added is a reset scale tag. That is for bleeding edge material such as footage coming from crop_rec which are being read as stretched footage for the time being. If that is a problem you can now reset the metadata scale tag.
With xxd I added the possibilty to alter white level in 5D mark III from 15000 to 16383(White level maximum 14-bit) for some extra highlight information. This will only work if the tag is either 15000 or 16383 to begin with otherwise nothing will be changed in the file to prevent other cameras to get these tags. Xxd is altering the MLV files directly.
(https://s23.postimg.org/pkc3xx6gr/image.png)
Simply stunning @Danne and can't wait to get my hands on this gem of yours once again!
Thanks for making DarkFraming Avg at least one of your priorities. Been wanting this new storage feature for a long time and it absolutely was worth the wait!
Your welcome and please report how it works out for you.
I know this is a little off topic but felt this was worth sharing since it is actually incredible to be able to preview 10/12-bit files with MLRawViewer while running MLVFS within your cr2hdr.app which is a dream come true for many.
Here's what I was able to do with a quick test file shot in 10-bit from 100D.101:
https://vimeo.com/198240254
Thanks again @Danne for your never ending Magic!
Thanks for sharing your workflow. It,s not that strange really. both mlv_dump and MLVFS outputs 14/16 bit even if being recorded 10bit. That,s why MlRawViewer works. It can be done outside cr2hdr.app as well. cd into a virtual folder of dng files right click a dng file and choose to open with MlRawviewer.
Worth mentioning some recent progress on handling focus pixels coming from MLV files. Until now this has been taken care of only when piping through dcraw to different formats ProRes in MLP and cr2hdr.app leaving the dng file still full of pixels.
I,ve been discussing the possibilites together with Bouncyball about utilise cold pixel code in raw2dng, used in mlv_dump somehow to also account for x,y coordinates created in dforts focus pixel script for those cameras affected. This was possible and Bouncyball used his magic fingers and created som real nice list handling through mlv_dump. Now the vital piece of the puzzle is the actual coordinates generated in dfort,s focus pixel script which are used also in mlvfs so I married the "allbadpixel.map" code from Bouncyball and the focus pixel script from dfort and I let them both live and prosper in cr2hdr.app in a nice multithreaded way :). It,s all taken care of automagically so just run cr2hdr.app and let dng´s be baked.
In this version I also put in chroma smoothing code from raw2dng since cr2hdr cs code code seems to not work so good in blown highlights for now.
Code here
https://bitbucket.org/Dannephoto/magic-lantern/branch/ml-dng-unified_4_chroma_smooth
Focus pixel code(dfort)
https://bitbucket.org/daniel_fort/ml-focus-pixels
A picture that says it all. Dng file with and without allbadpixels.map file generated.
(https://s29.postimg.org/xgu0qgh2f/Screen_Shot_2017_02_02_at_15_06_23.png)
(https://s29.postimg.org/7mk80oh2f/Screen_Shot_2017_02_02_at_15_09_41.png)
Added a switch which will disable cr2hdr automation in mlv files. Could be used for picky files which sometimes will fool cr2hdr into thinking it,s actually a dualiso file when it,s not. Of course when the switch is set it won,t transcode movie dualiso dng files at all coming from MLV files.
Discussed here.
http://www.magiclantern.fm/forum/index.php?topic=5601.msg180229#msg180229
(https://s22.postimg.org/uc7czmq6p/Screen_Shot_2017_02_20_at_20_56_53.png)
(https://s22.postimg.org/45663o7wx/Screen_Shot_2017_02_20_at_20_57_05.png)
love this app. i keep getting pink footage though. do i need to change the black and white level in mlv_dump? sorry I'm a noob just really liking this app and i always get pink footage. using experimental builds. maybe thats why?
Nice to hear :).
What camera and firmware?
If black and white level related you can change these numbers in mlv_dump menu. You can test:
White level 16000
Black level 2048
https://vimeo.com/205001874 (https://vimeo.com/205001874)
eos m firmware 2.0.2. my proxies and 4444 prores come out fine but my raw dng folder always has strange color like neon pink skin. maybe I'm missing an add on? I might not even have mlv_dump I'm on a mac and I don't know what I'm doing lol. I only see it as a windows exe.
Thought you were using cr2hdr.app? Mlv_dump is incuded in there. Head over to the correct menu by selecting (m) from the main menu. In there you can set white and black levels.
ok so mlv_dump is already in cr2hdr. good to know. yeah tried the white and black settings you suggested still pink skin. you have an eos m that works fine? I'm using the 2-11 experimental build with h.264 proxie.. but I've been getting this pink skin thing since I've started using cr2hdr. with other builds as well. I have no idea what it is. just really love the 4444 proxies but i need to open the raw folder in davinci to adjust exposure and kick out my own proxies or do an online color correction on the raw material.
Preview set to hacked? This is really experimental stuff.
Best movie mode for eos m is the crop_rec module. Coninuous 10bit without proxy needed to be set.
I've tried lots of different things. I've had it set to hacked and gotten MLV files that open fine in mlvfs. I was using that before and it was working. Then it just stopped working when the experimental builds came out. Focus Pixels show up. I used mlvfs through your app and they are gone. Strange.. but I need actual files can't have ghost folders that unmount. So I wanted to use your app but it comes out pink.
Can you post a file which comes out pink? Hopefully not too big.
About focus pixels it can be problematic. crop_rec needs a special fpm map which dfort created and I,m using it in cr2hdr.app.
https://drive.google.com/open?id=0B_4dpBYObDvEY3B1V0I3bVRseTg
Actually, there is nothing wrong with the file. The previewer in mac is giving pinkish output but when opening the file in acr or transcodeing the file to ProRes in cr2hdr.app everything works as expected. Be aware that using proxy+MLV will create a file which has no focus pixel map that is currently working with that file. If you want something useful with eosm use crop_rec for the time being.
that file was 14bit regular magic lantern crop footage. not 3x3 720 or mv1080. and not recorded with proxies. so if you run it your dng folder doesn't come out pink? it comes out fine? I'm using resolve to open the dng folder after processing with cr2hdr. yeah the prores 4444 and proxies are good. anyway thanks. just can't seem to find a real good post processing workflow since 10 bit and 12 bit came out. i either get focus pixels and good color or no focus pixels and whack color.
Update
Disregard what,s written below. Just tested unified mlv_dump and the files where exported normally with preview looking good. Problem is with my version of mlv_dump. Will dig further.
*Update 2
Problem solved. The color matrice for eosm was totally off. Seems maybe off in mlvfs as well. Will upload a new version soon.
New version uploaded.
Ok, tested some more and there is something funny going on with the files on the eosm that you sent. Now this actually happens over here too on my eosm and since I only opened my files in acr and in cr2hdr.app lately I didn,t know that it affects resolve and preview there. It seems there is corruption of some kind in the preview image in the dng file.
It would help if you could film a sequence that you know causes corruption and then try film the with the same settings with an older build for the eosm. If you find a build with corruption free footage you can narrow down which build is causing the corruption. Or maybe the problem lays elsewhere. Hard to tell.
Older builds here.
https://builds.magiclantern.fm/EOSM-202-all-builds-changes.html
You can post your findings here.
http://www.magiclantern.fm/forum/index.php?topic=9741.1175
tried a 14bit file looks super clean! going to try 10 bit and 12 bit next. awesome! thanks
Hi.
I had issues with some random upper case .DNG frames inside a regular .dng sequence, which caused Resolve and AE to think that the sequence was corrupt. After the latest update when Danne added the force disable dualiso command, this was fixed, and I haven´t had that issue since then after doing several test shots.
But now I have another weird issue?! For some reason that I don´t understand, sometimes I get a split .mlv file, even thou I have disabled "skipped frames" inside of ML in my 5D mark III running 1.2.3 with the latest build. Meaning for about 1 file per card, usually one of the last files recorded on the card before it fills up (I have reserve card space enabled, as well as the EXFAT option checked and the card is formatted as EXFAT) at the end of the recording, I get those .M00 .M01 .M02 etc split .mlv files for some weird reason? And for some reason, the CR2HDR.app doesn´t convert those files properly and skips frames for some reason?
Look at this picture from the bin inside of Resolve. As you can see, all of my converted .mlv files, converted with the latest build of the CR2HDR.app, is shown as clips as ususal, except for the last clip that you can clearly see is not recognized as a clip, but a folder..
(http://i66.tinypic.com/rbcw95.png)
Inside this folder is as you can see in this next picture, the .dng sequence but with some missing frames, making Resolve or AE not treat it as a sequence of course since it has missing frames.
(http://i65.tinypic.com/v31st5.png)
So I check my .dng folder with all the frames extracted by the latest CR2HDR.app, and as you can clearly see, there are missing frames here:
(http://i63.tinypic.com/hx7vhx.png)
Just by being curious, I tried converting the same split .mlv file with another piece of software I have installed on my computer that I don´t use any more after reading here that it´s not supported by ML, and for some reason it converts properly without any issues, and I can see the clip as a clip inside Resolve:
(http://i63.tinypic.com/2vtcmd3.png)
And by looking in the converted .dng folder here, you can clearly see that there are no missing frames in the same exact .mlv file, just converted by a different piece of software:
(http://i68.tinypic.com/1zpqo8z.png)
So obviously the camera has recorded all the frames, as they apparently exists within the .mlv file, even thou it´s split. But the CR2HDR.app for some reason skips some frames, rendering the footage useless as it doesn´t come out right and the audio is messed up and doesn´t line up or sync when getting these missing frame sequences.
Has anyone else experienced this issue or know of a fix? After I heard that the other piece of software I used to use to convert my .mlv files to .dng sequences was not supported by ML, I obviously stopped using it instantly and searched for another way of converting my .mlv files on a MAC. And the "easiset" one I could find since I´m not a programmer or coder or have any knowledge whatsoever of using the terminal on a MAC, was the CR2HDR.app. I have no skills nor any knowledge of how to code, program or use the Terminal since I´m only a filmmaker.
I´ve tried both the regular latest 14 bit build, as well as the experimental 10-12 bit build, and it´s the same issue with missing frames on split .mlv files no matter if I use 14, 12 or 10 bit.
Anyone has any ideas, that would be deeply appreciated.
Kind regards
@steeth
Could you test this version?
https://bitbucket.org/Dannephoto/cr2hdr/downloads/cr2hdr_test.app.zip
Will do and will report back after testing. Thanks a lot Danne.
Hi again Danne. Just tested the version of the app you posted, but unfortunately it behaves the same way.. When trying to convert spanned .mlv files (M00, M01 etc), it skips frames during conversion for some reason? See this picture of the mlv file I try to convert to .dng:s. It´s a folder inside Resolve and not a clip, and you see all the missing frames. And now there wasn´t any audio extracted either for some reason?:
(http://i68.tinypic.com/2e3zi2f.png)
And here is a part of the .dng folder where you see the missing frames again:
(http://i65.tinypic.com/30u7rr9.png)
And again, I converted this same exact .mlv file with the other piece of software I have installed that is not supported (Rawmagic), which I again of course stopped using as soon as I found out it was not supported by ML, and it works perfectly in both Resolve and Ae, has all frames intact and also the audio extracted and synced:
(http://i68.tinypic.com/rmrbih.png)
So there´s obviously something strange going on? It´s very unfortunate that this happens as I have no control over when my camera makes these spanned files. I have my ML menu set to EXFAT, no spanned files across the cards and not to allow skipping frames, and reserve card space is checked.. And for some reason the very last file I record to when my card fills upp and the recording stops automatically saying "Recording stopped. Camera/Card really EXFAT?), always turns into a spanned file... Since there is no way of knowing when the card is about to be full since the space left on the card doesn´t update on the screen after each recorded clip, it´s completely impossible to know when the card is filled up until the camera stops recording by itself and gives this message. The only time the space left on the card is updated, is by restarting the camera, which is of course impossible to do between each take as it´s not practical.. So does anyone know why ML makes these spanned files when the camera stops recording by itself? And is there a fix for it so it doesn´t do this?
If you like to download and check the same .mlv file I just tried to convert, please let me know and I´ll send you a download link to the files.
Thanks again for your help, and I hope this is solvable as I of course don´t want to use an unsupported piece of software to convert my .mlv files.
Downloading is slow. Hard to diagnose but maybe you filled your card up and some MLV safe guard stop is trying to save the content or it could be some issue related to exfat and MLV handling. I,d stick to fat32 but not everyone agree on this.
If I manage to download the file I´ll check it out.
Hi everyone,
I'm pretty new as MagicLantern user, and up to now I admit that I simply used RAWMagic for converting mlv-RAW (no audio) into DNG. RAWMagic did the job A) maintaining the original white balance, B) fixing the vertical stripes, C) untouching the original shot's color curve. This is what I need. Because unfortunally, since I've changed my computer to a Sierra Hackintosh, RAWMagic stopped working (crashing on startup).
So I'm searching for a new app that does the same things. DeafEyeJedi said to me about Danne's cr2hdr for Sierra. I tried to open it, but it seems a lot articulated for me, so I'd need a very basic guide, please: as I said I simply need to convert mlv-RAW (no audio) into DNG for Davinci, maintaining the original white balance, fixing the vertical stripes, and untouching the original shot's color curve.
So, please, tell me if I'm wrong:
1. I select the folder with dual-iso and also NOT-dualiso .mlv (no audio) footage, because the app works for both, right?;
2. In order to obtain CinemaDNG output I must choose (m) mlv_dump, right?;
3. Having NON-dualiso footage, I choose (10) disable dualiso automation, right?;
4. NOT setting the black/white levels, and simply having Adobe DNG Converted installed, I will obtain a DNG conversion with the original white balance of the original shot, right?;
5. I don't understand (07) disable stripes etc. If NOT chosen, the app will always correct the vertical stripes issues of the original shots or not?
6. Is it necessary to add (01) no chroma smoothing to obtain an original-like conversion?
7. My doubt about the curves remains: does the app maintain the original curve of the original shot, or does it set a custom curve based on MlRawViewer setting?
Please, kindly confirm if I'm right or wrong.
Thanks everyone really much.
1. Correct.
2. Correct.
3. If that's the case, then yes.
4. Yes, but also keep in mind depending on which DSLR you use you may want to push white levels higher for more highlight recovery for 5D3.
5. Correct as the app does this automagically.
6. Why not try both and see which suits your needs.
7. Not sure if I understood this one correctly re: MLRV's settings? I'm pretty sure it's as normal as it can get especially with all the latest bells and whistles w raw2dng, cr2hdr, mlv_dump, etc...
@adrjork
The way you are gonna use it it,s very easy. After downloading you have to fix the gatekeeper issue
To supress gatekeeper hold ctrl button down(macOS Sierrra) while opening the application the first time. You can also change permissions from within privacy/security settings.
Once this is done I would do following
1 - Drag a folder containing MLV or RAW files straight onto cr2hdr.app
2 - Choose (r) from the main menu. (Correct white balance and colormatrices will be applied as well as support for Davinci Resolve)(Thanks Bouncyball, dmilligan)
The option to disable dualiso automation isn,t necessary. This option came to be when a user had one or two issues where cr2hdr thought a non dualiso file were dualiso. But if you want to stay sure just select this and also any other setting in mlv_dump you need.
Generally don,t change white and black balance. They are optimized already.
a1ex Vertical stripe correction code is applied as default.
Thank all of you SO much men! You save my day. And I'm honored of using your brilliant piece of code.
1. About my question on MlRawViewer's curves: this app let you choose a color curve (linear, log-C, sRGB, etc.). I think I prefer to make all the color changes later in Davinci, so I suppose it's better for me in this step to maintain the original curve of the original shot (right?) But which is the original curve? I use a Canon 5D3 and I've simply read in the Canon menu that the color space is set to sRGB:
does this mean that the original curve is sRGB?2. Anyway, if I simply use MlRawViewer as a viewer (
before converting with cr2hdr4Sierra) no curves should be applied to the final conversion (later with cr2hdr4Sierra), right?
3. Now let say that I have the same basic purpose (mlv-RAW without audio to DNG with original white balance, orig. curve, removing vertical stripes) BUT this time with dualiso footage, I simply avoid to select (10) disable dualiso automation, and that's all, right?;
4. Can this app convert also dualiso photos or only dualiso videos?
5. Do you confirm that I have simply to keep Adobe-DNG-Converter installed, without touching/setting it?
6.
Quote from: DeafEyeJedi on March 22, 2017, 06:15:15 AMkeep in mind depending on which DSLR you use you may want to push white levels higher for more highlight recovery for 5D3.
I use 5D3, so which is a good value to set this option?
Again, thank all of you SO much.
Welcome to the world of scripting :)
1. DNG files can be decieving looking like sRGB or rec709 and still contain a lot of not showing information. Since files will be 14bit (or 10/12) you can compress the signal to log and you,ll see the extra information that is still inside this raw file. logC, C-log, cinelog, cineon etc. However your question about changing any canon meny setting won,t affect your raw dng files and not the curve at all on the dng file.
The easy answer is. Process to dng files and the bring your folders to DaVinci resolve or wherever and continue from there.
2. If cr2hdr4Sierra means cr2hdr.app MlRawViever is only for previewing purposes. If you export with cr2hdr.app to ProRes files you can select different log settings from the Prores menu (p)
3. If you skip the disable dualiso automation cr2hdr.app will automatically detect if the file is a dualiso file or not and process accordingly. Means you can mix dualiso files with non dualiso files.
4. Yes, this was the main purpose with this app from the start. Select (d) from the main menu to check settings in cr2hdr or simply select (r) if you have CR2 files in your folder. It will find you files and process them. It,s only when it comes to CR2 files you benefit from having adobe dng converter installed.
5. Yes
6. Don,t change the settings, they are optimized already
If you want to have some real fun you can fill a folder with all your files(MLV,RAW,CR2,mov etc) then drag the folder on cr2hdr.app and select (a) for full auto mode and lastly hit (r) to run cr2hdr.app.
Dear Danne, you are really kind: your answers are very complete. Thanks for your time.
I think I understand all.
Perhaps just a couple of doubts on the 1st question about DNG and curves:
1. If I understand correctly, mlv-RAW 14bit files contain more information than visible via sRGB or rec709, so to see the extra information I have to move from sRGB/rec709 to a more convenient logC/Clog/etc. Is that right?
2. If so, MlRawViewer gives the possibility to change the curve of the mlv-RAW files, so with that app I can export logC-DNG or Clog-DNG etc. If I understand correctly, this is a way to handle - later in Davinci - a DNG full of information (if compared to sRGB-DNG). Another option is simply to convert mlv-RAW into DNG without touching the color curve, then import the DNG in Davinci and only at this point choose BlackmagicDesign as Color Space (in Camera Raw) to recover more information and start to grade. Then my question is: which is the best method to handle more information in Davinci? Converting mlv-RAW into a log-DNG (changing the curve and converting with MlRawViewer), or simply convert mlv-RAW into DNG whitout any curve change (with cr2hdr) in order to apply the Blackmagic Color Space in Davinci? (I think this is a workflow question.)
3. Very last question: with cr2hdr.app, the 14bit mlv-RAW is converted into a 16bit DNG, right?
Super thanks.
1. DaVinci resolve handles highlight recovery one way and adobe camera raw in another. You can check this for example
http://www.magiclantern.fm/forum/index.php?topic=13512.msg172443#msg172443
When shooting raw we want to pull back as much information possible in the file to work with. When viewing the actual dng file it shows in a viewable display referred format which already clipped highlights. But that extra info is still in there once you start adding logs or fiddling in adobe camera raw, resolve for instance which can handle 14bit files.
2. You don,t alter any curve info in the dng file itself. The metadata inside the dng file which resolve, ACR, premiere will read and use when applying white balance, white level color matrices etc not any log adding. When you add logs in MlRawViewer you are suppose to export this look into a prores file. If you export to dng logs are not included.
If you process to dng files it won,t matter if doing it with mlrawviewever or mlv_dump except that mlrawviewer compresses the dng and it won,t work with audio embedding in DaVinci resolve like with cr2hdr.app.
3. The files are still 14bit when using cr2hdr.app. But there is a version which outputs 16bit. Those two extra bits are not giving any extra info so you know.
https://bitbucket.org/Dannephoto/cr2hdr/downloads/cr2hdr_steroids.dmg
That version of mlv_dump is coming from Bouncyballs heavy work on porting dmilligan,s mlvfs code into mlv_dump
http://www.magiclantern.fm/forum/index.php?topic=18975.0
It's all very clear now, thanks a lot Danne!
Just curious: about 14bit-DNG from cr2hdr.app, you say...
Quote from: Danne on March 22, 2017, 09:17:47 PMThe files are still 14bit when using cr2hdr.app. But there is a version which outputs 16bit. Those two extra bits are not giving any extra info so you know.
My question is: if the two extra bits adding nothing, why making a specific version of the app that outputs 16bit-DNG? Having 2 bits more with no information is useful for what?
Thanks again.
New version up. Download in first post.
- Added support for RAWC metadata in mlv_dump.
- Removed hardcoded temporary fix
More about RAWC here
http://www.magiclantern.fm/forum/index.php?topic=17021.msg181639#msg181639
@adrjork
At some point premiere wouldn,t work with 14bit files but with 16bit I think. There are other reasons as well I,m sure. As long as the cdng tags are represented 14bit works just fine.
About white level and resolve. Maximum white point 14bit is 16383 and that is what 5D mark III is set to and this usually is good for bringing home most highlight information. This could however affect how highlights are handled in resolve. especially if adding different highlight recovery methods. If this seems to be looking strange you could try setting white level to 15000 in mlv_dump settings instead.
I understand. Very clear as usual, Danne.
And cr2hdr_steroid (the 16bit version) has the same functions as cr2hdr.app (14bit version)?
Thanks a lot.
I would take that question on the relevant thread here:
http://www.magiclantern.fm/forum/index.php?topic=18975.0
I havn,t updated to very latest in steroid version yet so I would stick with 14bit for now.
Thanks Danne for adding RAWC metadata support as well for removing the hardcoded temp fix. About friggin' time and seriously I had no idea how much more vital this app could get. Just wait for the steroid version and it's just going to be a magnificence piece of an
apparatus in the making and most of all it's become an essential part of my life for everyday needs when it comes to ML as a whole.
This is purely one of the greatest gift a person could ask for, seriously no joke in here ladies & gentlemen...
Quote from: Danne on March 22, 2017, 01:43:03 PM
If you want to have some real fun you can fill a folder with all your files(MLV,RAW,CR2,mov etc) then drag the folder on cr2hdr.app and select (a) for full auto mode and lastly hit (r) to run cr2hdr.app.
:)
Hi Danne, thanks for the software that I'm just discovering. I've noticed that it relies on MlRawViewer as well. I have some RAW files that have a lot of hot pixels and sadly there's a bug in baldand software that prevents the correction from sticking in the exported dngs (https://www.magiclantern.fm/forum/index.php?topic=9560.1725). In my opinion is the only big bug that mlrv has and if we still need it have it solved would be a big help.
Just throwing it here, if you don't have time to check it or a thing like this is too complex (I really don't know) forge it :)
Can you post a sample mlv or RAW with hot pixels?
Merged mlv_dump compressed_raw code into a version of cr2hdr.app(mlv_dump). Feel free to test compressed MLV files and other things around a1ex latest code.
http://www.magiclantern.fm/forum/index.php?topic=19300.msg182201;topicseen#msg182201
http://www.magiclantern.fm/forum/index.php?topic=18443.msg182186;topicseen#msg182186
cr2hdr_4k.app
https://bitbucket.org/Dannephoto/cr2hdr/downloads/cr2hdr_4k.dmg
Thanks so much @Danne
Not sure if its me, the new crop module, or cr2hdr but my first test of 1920x1080 48p resulted in pink clipped highlights.
Are you shooting compressed material? I use a1ex compressed_raw code mlv_dump in there but can you test with a version with mlv_dump directly as well?
https://bitbucket.org/Dannephoto/cr2hdr/downloads/mlv_dump
Run
mlv_dump --dng INPUT.mlv
Same result?
Download in first post.
To honor a1ex great lossless compression_raw work I put in his version of mlv_dump in cr2hdr.app together with the older one. I havn,t found out how to differ compressed raw material with metadata so the trick is to develop one dng file and if it turns out to be a compressed file cold pixel fix will totally max out showing 200000 found pixels. THis make scr2hdr.app choose the mlv_dump4k version instead. Otherwise cr2hdr.app will process mlv files as usual.
Don,t expect all mlv_dump functions to work here. compressed_raw still wip experimental
Don,t expect the 4k mlv_dump version to work exactly like the version of mlv_dump in cr2hdr.app going through the menu system since I,m using a slightlu modified version of mlv_dump.
if grep '200000' <<< "$(mlv_dump -m -f 1 --dng -o /tmp 2>/dev/null "$FILE" | awk '/cold pixels/ { print $6;}')"
then
#frame extraction in mlv_dump4k makes multihtreading possible. Thanks to so-rose for highlighting this possibility
frct=$(mlv_dump4k -f 0-1 --dng $mlv -o "$O2""${BASE}_1_$date"_ "$FILE" | awk '/Processed/ { print $2; }')
mlv_dump4k -f 2-$(($frct/4)) --dng $mlv -o "$O2""${BASE}_1_$date"_ "$FILE" &
mlv_dump4k -f $(($frct/4+1))-$(($frct/4+1+$frct/4)) --dng $mlv -o "$O2""${BASE}_1_$date"_ "$FILE" &
mlv_dump4k -f $(($frct/4+1+$frct/4+1))-$(($frct/4+1+$frct/4+1+$frct/4)) --dng $mlv -o "$O2""${BASE}_1_$date"_ "$FILE" &
mlv_dump4k -f $(($frct/4+1+$frct/4+1+$frct/4+1))-$frct --dng $mlv -o "$O2""${BASE}_1_$date"_ "$FILE"
else
#frame extraction in mlv_dump makes multihtreading possible. Thanks to so-rose for highlighting this possibility
frct=$(mlv_dump -f 0-1 --dng $mlv -o "$O2""${BASE}_1_$date"_ "$FILE" | awk '/Processed/ { print $2; }')
mlv_dump -f 2-$(($frct/4)) --dng $mlv -o "$O2""${BASE}_1_$date"_ "$FILE" &
mlv_dump -f $(($frct/4+1))-$(($frct/4+1+$frct/4)) --dng $mlv -o "$O2""${BASE}_1_$date"_ "$FILE" &
mlv_dump -f $(($frct/4+1+$frct/4+1))-$(($frct/4+1+$frct/4+1+$frct/4)) --dng $mlv -o "$O2""${BASE}_1_$date"_ "$FILE" &
mlv_dump -f $(($frct/4+1+$frct/4+1+$frct/4+1))-$frct --dng $mlv -o "$O2""${BASE}_1_$date"_ "$FILE"
fi
Any issues using the new mlv_dump for regular footage?
Maybe not but I,m using a specific branch which adds white balance correction(mlvfs style) and also some extra code from Bouncyball which works with externa pixel map files. There are more stuff. All changes are ported by Bouncyball and coming from mainly mlvfs code or Bouncyball himself.
https://bitbucket.org/Dannephoto/magic-lantern/branch/ml-dng-unified_6_RAWC_rec_trigger-latest
Havn,t really tested this at all but now we can test run compressed_raw material easily and check what works and not. Deafeyejedi did some short testing. Maybe he can put up his findings here.
Wish I had more time for this stuff 8)
New download.
Adjusted cold pixel fix number to 100000 or more since some files does not hit the maximum roof and falsely thinks it,s uncompressed footage. Thanks deafeyejedi for samples. Hopefully it will differ from compressed and non compressed files better now. If not, please report.
Thanks so much for the quick fixes. At first I thought it had something to do with the 2017-04-06 experimental build for 5D3.123 but doesn't seem to be.
I wanted to give ImageJ a go within your app and this is what I'm getting (even after placing the PointPicker inside the Plugin Folder).
(https://c1.staticflickr.com/3/2886/33762827942_6cbaf74786.jpg) (https://flic.kr/p/Trvjt9)
Also I noticed with the latest cr2hdr.app I wasn't able to get it to respond to 14-bit lossless MLV's in HDR 100/1600 from 5D3.
Here are some short samples in case you want to try on your end: https://mega.nz/#!D1lHhAbL!X_QKk046acE_oWG1k8UVc9Yqo1eOqQTjCd9s3bvv-eU
Reworked mlv_dump compressed raw detecting so hopefully even better finding out what is what now. Check with latest.
About your ImageJ issue I can,t see what,s going on. See to it that you have ImageJ in applications folder and so on.
Why when processing there's not the real time log in the same window of the options but the strange percentage thing in the menu bar? It feels really weird to me!
Also a small thing, when files are relocated on the A_ORIGINALS folder when converting RAW to MLV the R00 files etc are not moved, only the RAW one.
Time log in the same window? Could you elaborate?
The automator wheel you are referring to isn't telling more than that the application is running. It's not possible for automator to know how long processing will take from the included scripts. I guess that estimation could be done and put into an indicator but it won't be exact.
I'll check into also moving the R00 and so on to the originals folder. Thanks.
QuoteAlso a small thing, when files are relocated on the A_ORIGINALS folder when converting RAW to MLV the R00 files etc are not moved, only the RAW one.
Fixed!By the way. Thanks to Bouncyball for the RAW to MLV conversion function.
Quote from: Danne on April 12, 2017, 07:43:36 AM
Fixed!
By the way. Thanks to Bouncyball for the RAW to MLV conversion function.
Excellent catch @togg! Thanks for the reminder @Danne for us to have the ability to go through old RAW files from back in the days of early ML to convert them into MLV's and none of this could have been possible without @Bouncyball's work.
That was a fast fix, thanks!
@Danne I mean, it's strange that the app window disappear, I don't care that much about the estimation (although it would be usefull) but not seeing what the software is doing is strange.
@DeafEyeJedi Oh yes a big thank to Bouncyball! I've just discovered this feature thanks to Danne, I think a lot of people would like to know that something like this is possible. MLV Lite is a recent development so there are a lot of files from 5D2 owners out there.
I'm now converting 1.5 TB of files. I had to copy them from a 3TB drive that was full in order to do the conversion. Would it be difficult to throw in a destination folder option for the conversion from RAW to MLV? It would also avoid the fact that now the drive is reading and writing at the same time at half speed. For a big amount of files this basically mean one extra day of work for each of the two non necessary steps, more possible errors as well.
Thanks for feedback togg.
Quote@Danne I mean, it's strange that the app window disappear, I don't care that much about the estimation (although it would be usefull) but not seeing what the software is doing is strange.
I thought of this a while ago. I guess I could let a small terminal window put in a corner tell exactly what processing is going on while it´s running. I already do some apple script notfication messaging when user selects (a) full auto from the main menu.
QuoteWould it be difficult to throw in a destination folder option for the conversion from RAW to MLV?
I`ll have to check into this. Might be working.
*Update
Could not find a way to specify MLV output however you can trick raw2dng to think it,s processing file in another location by creating symbolic links to the output location and then you,ll end up with the MLV files wherever you set output. I see if I can create something useful here.
Thanks to you.
Yes exactly, the same terminal window. You already have the log file updated in real time anyway.
A small thing, the version number doesn't change right? I only see 1.2 in the finder informations.
@Danne Don't rush up the destination folder thing for me, I'm already on my path here anyway. I've already moved the files and the conversion is progressing, it would be usefull for other people.
Version history is always the same.
Would be good if you could test the specified output loction once it,s ready just to confirm working or not.
Thanks again for ideas and feedback.
Here we go. New version up. Specify output location for where to put MLV files when converting from RAW to MLV. Simply choose (2) to choose or create a new output folder. Works through symlinking.
Commit: https://bitbucket.org/Dannephoto/cr2hdr/commits/884bbada0050493a77aba013db7289d33d18617b
(https://s28.postimg.org/suqjwa4ul/Screen_Shot_2017-04-12_at_14.33.15.png)
Niiice. If I stop the conversion now (I'm doing the second batch, I've divided in 3) will it keep the files already converted or it will revert everything?
Files converted will stay converted.
Before stopping I would simply move all RAW files into A_ORIGINALS. By doing this cr2hdr.app will probably break or you can stop it by pressing the automator wheel if it doesn,t stop.
I posted yet another version just now which handles erasing symlinks after processing a little differently.
Havn,t had time to test with .R00 .R01 etc yet but should work.
*Update. Tested working with spanned file R00 assigned to alternate location.
Yey double speed as expected! I was so sad of treating my hard drive roughly :'(
It works with both single and spawned files. But there's a little bug. It creates aliasing of the RAW on the destination while converting, it deletes them when is a single file but doesn't when is a spawned one. Just a minor thing.
hehe, just fixed that quirk. Noticed it myself.
New version uploaded.
Nice that it works.
Love this app! :D
So fuckin' HAPPY that more users are digging this app. Been trying to publicize this for years. Going back from Pre-cr2hdr-r to MLP to present has been a great journey and I am honored to continue to ride on this wave of yours that just never ends...
Thanks @Danne for sticking with us through thick and THIN! 8)
Thanks guys. Appreciated. Happy times :D
the aliases were not deleted! I mean it really takes a couple of click to get them out but I wanted to let you know.
Thanks. Thought I fixed it. Sure you downloaded latest?
Could you upload your version of cr2hdr.app so I can check it? Lots of versions floating now :).
99.221.254 byte, should be the last one!
It works over here so not sure what,s going on unfortunately.
*Update. Was an issue with multiple spanned files. Fixed now.
Thanks for the kind update @Danne in regards to the MLV log countdown via Terminal.
(https://c1.staticflickr.com/3/2832/33942509342_26409409a9_o.png) (https://flic.kr/p/THoeuj)
It can be handy when working on other things and to be able to see this prompt window is awesome actually!
cool. I´m putting in a notifiyng countdown counter for most of the stuff in time. If downloading the downlod repository directly I just added the counter to RAW and dual iso CR2 processing as well. Idea from feedback from togg originally.
Quote from: Danne on April 17, 2017, 07:11:27 PM
I´m putting in a notifiyng countdown counter for most of the stuff in time. If downloading the downlod repository directly I just added the counter to RAW and dual iso CR2 processing as well. Idea from feedback from togg originally.
Nice. I did notice @togg was responsible for the concept. Great thinking @togg!
Also @Danne do you plan on doing the same for ProRes renderings as well?
QuoteProRes renderings as well?
Yes. I´m right now fooling around some idea that right now opens up a new notifier for every process being started but I will somehow make it stay within the same terminal window. Let´s see when I trick bash to do exactly what I want ;).
Having some issues with pink highlights in 50p 3x3 (old style - 1920x672).
Is this something to do with ML or the DNG processing?
Sample zip file:
https://drive.google.com/open?id=0B-GY59oq1k7eS1FaUTg2U2ZjWDQ
(http://i.imgur.com/JcbpT81.jpg)
Hi Beauchampy. You could try set white level to 15000 instead of 16200 ad see if that helps some. Then I think you have to be careful with highlight recovery here since there are a lot of blown highlights. Don´t think there´s anything wrong with ML or processing. Only difficult footage.
octave:1> a = read_raw('M14-1541_000000.dng');
octave:2> prctile(a(:), [1 5 10 50 90 99 99.9 99.99])'
2278 2846 3306 5634 8430 12558 12654 12710
=> try 12500.
Details: http://www.magiclantern.fm/forum/index.php?topic=5601.msg181926#msg181926
Steps to reproduce?
Slight bit of pink @ 15000
Much less pink (but still a tiny tiny amount) @ 12500
Thanks both!
@a1ex - this didn't used to happen with 14-bit, I could usually blow the highlights and they would recover as grey, unless I'm mistaken. Is this something to watch for when exposing & 12-bit?
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
I doubt it has to do with lower bit depth, but you need to show me the steps to reproduce the issue (preferably from a clean configuration, e.g. reset both Canon and ML to defaults).
Seems these may have to do with using 1080p 45/48fps 3x3 in prior to. I was able to get some of these pink cast around the blown highlights in recent tests with 14-bit lossless 3.5k afterwards.
Here are some samples. Will try to find a way to reproduce on my end and report my findings.
Short Samples:
https://mega.nz/#!C5VlyYpb!4HMSCgIqEPccAADnbensPlAjGIV_ZVwcHb2zZ8_-BWU
https://mega.nz/#!39tnlZjR!SAWzGp0ycr-iDivWRna99rjbxC0IJWweNT88lZsmzew
https://mega.nz/#!fkEzRYQI!V3GVfDG3a5FT2nu3QgAk9PtLXZU3Dq8Hk8mpkFKwy04
https://mega.nz/#!iwFDRDoa!yA-RPrd3fvFm8XHEaweGIJJZwzhSwU0cgXUhBj8NiBU
Thanks Danne for the CR2HDR April 18 build! I'm now gladly converting 3K mlv to dng on my mac. Rather than having to do it on my PC then transfer the dngs back to Mac.
Cool that a1ex mlv_dump version seems to play nicely in cr2hdr.app @shadowsenhance.
Uploaded a new version. Includes notifiers for core processing (mlv_dump, raw2dng, cr2hdr, dcraw/FFmpeg ProRes piping). When processing is done there will be an ending prompt specifying total processing time and also how long each process took.
ex.1
(https://s2.postimg.org/s8ux0qgmh/Screen_Shot_2017-04-21_at_09.13.04.png)
ex.2
(https://s2.postimg.org/ipl879t49/Screen_Shot_2017-04-21_at_09.11.20.png)
End prompt
(https://s29.postimg.org/h4dhn9npj/Screen_Shot_2017-04-21_at_09.24.22.png)
Digging all the new notifiers big time especially the end prompt -- really useful details and great job @Danne!
Great job on the latest build!
Just wanted to add, I noticed inside cr2hdr there was embedded both mlv_dump and mlv_dump4k. I was processing all 3K lossless stuff, and some MLVs processed okay, others gave a single corrupted frame and moved on to next MLV.
I figured for some reason maybe cr2hdr had used the non-experimental build for these, giving the corrupted result. So I replaced mlv_dump with mlv_dump4k (so both files are the same experimental build), ran cr2hdr again, and everything processed without a hitch.
So maybe the technique to determine whether to use mlv_dump or mlv_dump4k is not perfect yet. But as I am only shooting 3k/4k stuff, this workaround is fine for me.
Yes, I'm using information based on what happens when regular mlv_dump fail. Usually it stops after one file outputting something like "processed one frame" but this might not be the case always.
As it turns out Bouncyball just released recoded versions of mlv_dump and mlvfs I will put up a version which hopefully fixes the issues with having two versions.
Check out the latest from mr B here.
http://www.magiclantern.fm/forum/index.php?topic=19429.msg183756#msg183756
New version which supports compressed raw as well as regular 10/12/14bit raw files.
Download link (https://bitbucket.org/Dannephoto/cr2hdr/downloads/cr2hdr_lj92.dmg)
I "monkeywrenched" Bouncyball´s lj92 implemetation to work with cr2hdr.app. It won´t get me any prices but it seems to fly.
Code here:
https://bitbucket.org/Dannephoto/magic-lantern/branch/ml-dng-unified_6_RAWC_rec_trigger-lj92
Andrew Baldwin(MlRawViewer) worked out lj92 code
http://thndl.com/how-dng-compresses-raw-data-with-lossless-jpeg92.html
Code has been around and worked upon and now Bouncyball reworked the code so that it works with MLVFS and mlv_dump for steroids. Major leap here. Big thanks to Bouncyball. I somehow managed to get the code working with 14bit ml-dng(dmilligan) branch in cr2hdr.app.
The pros are obvious. Hopefully working with a1ex and g3gg0 stellar vertical stripes code and chroma smoothing, cold pixel fix etc. If testing please report your experience both good and bad on Bouncyball´s thread or here. Check if vertical stripes code is working and so on.
Bouncyball "updated MLVFS" thread
http://www.magiclantern.fm/forum/index.php?topic=19429.msg183756#msg183756
Do not miss out on downloading Bouncyball version of dmilligan´s state of the art MLVFS (https://bitbucket.org/bouncyball/mlvfs/downloads/) fully working with compressed raw.
Darkframe averaging seems not to be working with compressed raw yet.
Quote from: Danne on April 25, 2017, 10:43:29 PM
New version which supports compressed raw as well as regular 10/12/14bit raw files.
Download link (https://bitbucket.org/Dannephoto/cr2hdr/downloads/cr2hdr_lj92.dmg)
I "monkeywrenched" Bouncyball´s lj92 implemetation to work with cr2hdr.app. It won´t get me any prices but it seems to fly.
Code here:
https://bitbucket.org/Dannephoto/magic-lantern/branch/ml-dng-unified_6_RAWC_rec_trigger-lj92
Andrew Baldwin(MlRawViewer) worked out lj92 code
http://thndl.com/how-dng-compresses-raw-data-with-lossless-jpeg92.html
Code has been around and worked upon and now Bouncyball reworked the code so that it works with MLVFS and mlv_dump for steroids. Major leap here. Big thanks to Bouncyball. I somehow managed to get the code working with 14bit ml-dng(dmilligan) branch in cr2hdr.app.
The pros are obvious. Hopefully working with a1ex and g3gg0 stellar vertical stripes code and chroma smoothing, cold pixel fix etc. If testing please report your experience both good and bad on Bouncyball´s thread or here. Check if vertical stripes code is working and so on.
Bouncyball "updated MLVFS" thread
http://www.magiclantern.fm/forum/index.php?topic=19429.msg183756#msg183756
Do not miss out on downloading Bouncyball version of dmilligan´s state of the art MLVFS (https://bitbucket.org/bouncyball/mlvfs/downloads/MLVFS) fully working with compressed raw.
Darkframe averaging seems not to be working with compressed raw yet.
Can someone put this in laymans terms? Do I just need to update cr2hdr and it's much faster? Or does this require MLVFS as well? I know how to use both converters, just never used them together. DeafEye, where are you? You're good at explaining this stuff :D
Just do what you normally have done. Download and install. This time it's temporarily called cr2hdr.app_lj92 from the bitbucket download link above.
Re: MLVFS from within this app (assuming you've updated to the latest supercharged MLVFS (http://www.magiclantern.fm/forum/index.php?topic=19429.msg183756#msg183756MLVFS) from @bouncyball) - Once you get into main menu and press 'ml' to get into MLVFS menu then press 'A' to activate.
It should then also prompt up MLRawViewer (if you have it installed) as well. Once you're done previewing the footage, simply quit MLRV and yet cr2hdr.app will be standing by upon waiting for your next command request.
Installed everything, I'm missing a step to understand how to compress the MLV (if it is possible)!
edit: ok it seems it is not yet since martinhering is asking for it mmm
Question: Would it be long to make cr2hdr.app also works on OSX 10.6.8?
I have two systems on my Mac and 10.6.8 is my choice for the moment, so each time I test ML I have to restart. Don't ask ;-)
Is it not working on 10.6.8? What is happening when you run it?
Nothing, the window doesn't open.
Maybe you need to change security settings?
Could also be that you need to put in all scripts in your older automator version to make it work. Hard to tell.
Hi, I'm trying to process dual_iso with lossless compressino enabled (lj92) - first few frames are normal but then a lot of greenish frames until the end. Is this known issue?
(even if I disable lossless compression after few frames the rest are greenish)
I have tried to open it in Automator and step by step execute the tasks, all is going well until the last script which is understood but the cr2hdr window doesn't open.
Sorry, cannot help you here. If you get all scripts to work from inside automator it should probably work. MAybe if you copy all scripts and files into a vanilla build of an automator app it might work but there are other thkngs which might be troublesome here. HArd to say.
festr. Are regular dual iso files also problematic? Not compressed ones? Could you upload a short sample?
Edit: I have tried to turn off ls92 compression in MLV menu (the same result) so its not compression issue
This is first frame:
(https://s7.postimg.org/vb4mund6z/Sni_mek_obrazovky_2017-04-27_v_0.00.43.png)
this is second frame (bad) and all others are bad
(https://s9.postimg.org/udnn7sakv/Sni_mek_obrazovky_2017-04-26_v_23.58.00.png%5Dhttps://s9.postimg.org/udnn7sakv/Sni_mek_obrazovky_2017-04-26_v_23.58.00.png)
Upload a short file that exhibits the issue. I can take look.
Here it is: https://drive.google.com/open?id=0B650ZX3ln336QzlvVWhTanRiY2M
let me know how it decodes on yours machine
Thanks for the clip. It´s my version of mlv_dump. It seems to go bananas with your file. You can still use regular cr2hd.app and it will work.
Small suggestions, option 05 and 07 are a little bit strange to read. Are they active when selected or not? Could be improved to be less confusing.
(https://i.imgur.com/hKyNhfa.png)
It´s from the mlv_dump menu. But yes, things can always be clearer :)
cr2hd.app - but will it work with the compressed lj92 feature?
The issue is related to vertical stripes correction. Since I divide mlv_dump processing into 4 chunks working in parallell it seems only the first chunk using vertical stripes correction is producing nice output. This bug appears on latest lj92 version crop4k branch as well.
If I run processing in one chunk all looks ok but it seems to change vertical stripes algorithm for every new chunk when processing in parallell.
The fix is to disable vertical stripes correction with dualiso files. Can be done in mlv_dump menu (m)
Vertical stripes correction:
1.00000 1.00084 1.00200 0.98271 0.99992 1.00200 1.00189 1.01596
Frame0 : cold pixels found: 0
Vertical stripes correction:
1.00000 1.99995 1.00266 1.99995 0.99980 1.99995 1.00221 1.99995
Frame0 : cold pixels found: 0
Vertical stripes correction:
1.00000 1.99995 1.00204 1.99995 1.00047 1.99995 1.00127 1.99995
Frame0 : cold pixels found: 0
Vertical stripes correction:
1.00000 1.99995 1.00270 1.99995 1.00055 1.99995 1.00276 1.99995
Hi Dane, yes without stripes correction there is no problem - but there are vertical stripes now which I need to correct. Do you know how to fix it?
Could you upload a dng with stripes?
On a sidenote this issue could be of something related to mlv_dump function -f which works with frame exports. Since files ranging from 0 and forward seems play nicely a -f 100-200 does not. Question is how non dualiso files are affected or not.
Uploaded a new cr2hdr_lj92.dmg version. Download in first post.
Reason is I think I found the bug with vertical stripes correction when using the -f function in mlv_dump. This one only occurs when exporting let´s say mlv_dump -f 10-20 --dng. If outputting frames somewhere in the middle of the mlv file vertical stripes correction numbers seems way off as opposed to processing mlv_dump --dng or mlv_dump -f 0-100 --dng.
After the fix I got much more coherent numbers. Please check older numbers a few posts before.
Vertical stripes correction:
1.00000 1.25429 1.12299 1.33395 1.12294 1.14198 1.08633 1.20380
Frame0 : cold pixels found: 0
Vertical stripes correction:
1.00000 1.25465 1.13469 1.31161 1.11452 1.17938 1.09082 1.12737
Frame0 : cold pixels found: 0
Vertical stripes correction:
1.00000 1.25990 1.13376 1.31618 1.12531 1.18373 1.07240 1.12363
Frame0 : cold pixels found: 0
Vertical stripes correction:
1.00000 1.25998 1.12959 1.31648 1.12939 1.18280 1.10335 1.12971
Reached end of chunk 1/1 after 232 blocks
Processed 223 video frames
Done
Commit here.(Needs to be tested)
https://bitbucket.org/Dannephoto/magic-lantern/commits/be8d3a22ce28ec8c18bc07b43c1045204dc03d83
In short vertical stripes correction is called after /* finally save the DNG */ as opposed to before.
/* finally save the DNG */
if(!dng_save(frame_filename, frame_buffer, &dng_info))
{
print_msg(MSG_ERROR, "VIDF: Failed writing into .DNG file\n");
goto abort;
}
/* call raw2dng code, moved here because otherwise numbers are off for compressed raw */
if (fix_vert_stripes)
{
fix_vertical_stripes();
}
I have tried the latest cr2hrd_lj92.dmg but it does not remove vertical strips now (I have checked that I did not disabled it)
Well, you need to upload example dng files. Dual iso files in general doesn´t work as regular footage. Not even sure that dual iso files are supposed to apply vertical correction to them. Upload dng samples that shows your problem.
It is the same MLV file I have uploaded to dropbox. Here is the new DNG which exhibits vertical stripes:
(https://s17.postimg.org/cbwvkbxvj/Sni_mek_obrazovky_2017-04-27_v_13.14.59.png)
and here is for comparison previous version which removed vertical stripes (but it broken all other images)
(https://s7.postimg.org/vb4mund6z/Sni_mek_obrazovky_2017-04-27_v_0.00.43.png)
I want the dng files.
I have uploaded few DNGs here https://drive.google.com/open?id=0B650ZX3ln336MWJ1VDZISGNxTkk
the postimg.org changed resolution, here is the same picture where stripes are clearly visible:
(https://s12.postimg.org/6ewhz43y5/Sni_mek_obrazovky_2017-04-27_v_13.17.40.png)
Specify two dng files. One with stripes, the other one without stripes. For comparisons.
I have reverted to previous cr2hdr which has the green problem - first of two images are without vertical stripes - I have uploaded it to the same link under directory WithoutStripes
(please note that I'm using still the same .MLV which is also uploaded there
https://drive.google.com/open?id=0B650ZX3ln336MWJ1VDZISGNxTkk
Compared with g3gg0 latest code and there is a problem with my so called fix. Will have to tinker some more...
Thanks for sharing information on this. It´s only related to -f function, otherwise it works.
By the way. Is it happening on all your dualiso files or only this one?
I can try more footage but I'm convinced it will do for all. I was actually not using dualiso because of very long postprocess but this application with parallelisation and automation rocks so I wanted to use it :)
what is exactly -f and how I can remove it as a workaround?
Try a few more files. It seems to work for non dualiso files. I suspect it could be the file. If not I upload a version which skips processing in parallell. I uploaded the old version again.
this latest version converted first 56 frames OK (including vertical stripes removing) but the rest of 56-222 frames are green again
I know. I asked you to test with a few other dual iso files.
Ok, never mind. Found a file myself. Strange bug. Must check further.
I have just checked another iso file and it has the same exact problem, after 72 frames green frames happened.
Yes, thanks. I will investigate further. I think it has to do with raw2dng changes. I found when the problem started to occur in my version of mlv_dump.
Here we go. I nailed down the problem to some newer vertical stripes code. Inserted back the older code and now all is working fine again.
Hopefully... :P
all problems gone, good job! Thank you.
Yikes what a battle... You guys push each other to its limits and Danne was able to pull it off nicely at the end.
Thanks for pointing this out @festr!
Gonna shoot some short Dual-ISO lossless stuff and confirm this ASAP!
Great. Also uploaded an updated cr2hdr.app non compressed version.
New version uploaded. Compressed mlv_dump version is now merged with cr2hdr.app. Thanks to Bouncyball for setting s´things straight :P. Thanks g3gg0 and a1ex for groundbreaking features.
Lately I´ve been playing around with white balance trying to outsmart dcraw -a (averaging auto white balancing) with either some better auto wb handling or some better way to do it manually. After some attempts using imagemagick scripting with some fairly good results but in the end it wasn´t better then dcraw I finally settled with a manual workflow in conjunction with ImageJ, a java based application. so if you want to try this out install ImageJ straight to applications folder before proceeding.
ImageJ download
https://imagej.nih.gov/ij/
Workflow description:
Simply select x,y coordinates with the multipoint tool in ImageJ and save those to a txt file. cr2hdr.app will take care of the rest.
It still depends on dcraw but by selecting x,y coordinates manually you get more control over the result.
The interesting workflow here is that you will be able to select up to 10 white balance points and those will be calculated into a median value and converted into the AsShotNeutral tag in your dng file. A lot like spot metering but using the same idea with white balance.
Also note that usually your NLE doesn´t rewrite the actual white balance tag in the dng file but applies it´s own local metadata system. Personally I rather have the actual dng tag changed so white balance could be read in multiple nle´s.
Movie tutorial:
https://www.youtube.com/watch?v=o7iaef54Vxo&feature=youtu.be
Choose (13) in mlv_dump menu
(https://s29.postimg.org/ecr0bhqtj/image.png)
(https://s8.postimg.org/tw1hxagn9/image.png)
Work with the multipoint tool
(https://s1.postimg.org/zamdmrr4f/Screen_Shot_2017-05-06_at_11.19.16.png)
Select points(up to 10 points) and save the file next your MLV files. The format will be txt
(https://s8.postimg.org/5uulvu1tx/image.png)
You preview the new white balance by selecting (s) from the menu
(https://s8.postimg.org/er5dzrsg5/image.png)
Proceed to the next file (n) and apply white balance points and keep going as long as you like. When done select (q) and then run cr2hdr.app and it will process the dng files and apply the new AsShotNeutral tag to your dng files.
White balance files will be named with the extension .AW and cr2hdr.app will match the file with corresponding MLV
(https://s8.postimg.org/udwnd5685/image.png)
Quote from: Danne on May 06, 2017, 11:40:42 AM
New version uploaded. Compressed mlv_dump version is now merged with cr2hdr.app.
How do I output compressed DNGs?
Yet another version in first post.
Thanks(as always) to Bouncyball, for patching and adding vital parts to get g3gg0 latest lj92 code in mlv_dump to work with my version of ml-dng code(dmilligan). I added most of the code from here https://bitbucket.org/hudson/magic-lantern/branch/crop_rec_4k but Bouncyball tracked down parts which was causing black, corrupted frames and added some needed metadata to dng.c.
https://bitbucket.org/Dannephoto/magic-lantern/branch/ml-dng-unified_8_-Mr_B_compress
This means you now are able to output compressed dng files(how awesome is that!) through Andrew Baldwins lj92 code. Select (10) from within the mlv_dump menu.
Please report any findings, good or bad.
Happy processing.
Quote from: Danne on May 07, 2017, 09:29:27 AM
Yet another version in first post.
Please report any findings, good or bad.
Happy processing.
Cool, thanks. All good. 4.9Gb uncompressed file is down to 3.1Gb. Original 12Bit compressed MLV is 2.4Gb.
Awesomeness squig.
There is a problem with 10/12-bit files and mlv_dump with the latest crop_rec4k mlv_dump version. Until they are sorted out I´ll put the older version back up. It won´t compress dng files but it will work with all files.
I really like the compressing of dng/MLV files and would like to test it out in cr2hdr.app. For now it will not compress 10/12bit files but all the rest seems to work so therefore I will post a side testversion of cr2hdr.app
10/12bit files will still export but uncompressed. This is automated so even if you select -c, (10) in mlv_dump menu, compression will be neglected for only 10/12bit files.
https://bitbucket.org/Dannephoto/cr2hdr/downloads/cr2hdr_lj92_cmprs.dmg
(Added to universal cr2hdr.app build, link not working. Download in first post)
Happy testing.
I'm happy to say that cr2hdr is now nicelly integrated in my workflow! On the last period a lot of new things were added by Danne
1) you can pick destination folder.
2) dng compression almost 50% in size.
3) multithread (was already there maybe).
4) hot pixel map creation and usage.
5) batch processing of a series of folder.
6) countdown of files left to process.
7) A_ORIGINALS folder deleted when not needed, files stay in place.
On a usual day you can preview your MLV by launching MLVFS, then feeding the dngs to mlrawviewer. After that you can just launch the conversion to compressed dngs.
Thanks for feedback togg and testing, helping ironing out bugs(especially around multiple folder selections). It´s great to see the result of dng compression implemented by g3gg0(A.Baldwins code) and that you find good use of the badpixelmap handling(Bouncyball)
I will add some of the changes you highlight soon to main build and keep refining other processes when I find the time.
New download in first post.
- Reworked mlv_dump handling so that it runs multiple mlv files in parallell instead of the slowish extraction handling before. I tested against processing files lined up one by one opposed to parallelising processing and speed increase doubled when working with multiple mlv files.
- Also reworked how pixel map files are used with cameras such as eosm.
- Added a percent counter, going from 100% down to zero while extracting dng files.(togg feedbacked)
- Drag multiple folders onto cr2hdr.app containing files and cr2hdr.app will process what´s in those folders one by one.
- More goodies in the mlv_dump menu from recent code from g3gg0, amongst them "relaxed" mode if you check under "autopsy" setttings.
- HDR handling with dng files. Check ProRes menu(tblend filter)
Probably more stuff I can´t remember...
Feedback is welcome, both good and bad news.
just tried it.. seems really fast. thank you.
Cool, thanks for getting back.
Just uploaded a new version which makes it possible to preview any flavour of MLV. It works through symbolic linking which utilises MLVFS as a preview engine together with MlRawViewer. Thanks to dmilligan state of the art mlvfs now working with compressed raw(thanks Bouncyball for implementing A.Baldwin lj92 code) we can now preview any flavour of MLV.
It works like this.
- Once you have mlvfs installed you simply drag one or more MLV files to cr2hdr.app. This will build a temporary mlvfs structure and open up all MlRawViewer and builds thumbnails of all MLV files from within the folder. Once you close MlRawViewver the mlvfs structure is erased and ejected.
Some brute ejection (killall) so I renamed a mlvfs version to mlvfss so it doesn´t eject any regular mlvfs structures.
The script goes like this
#workaround which lets all files preview through MLVFS
killall mlvfss
rm -r /tmp/MLV_viewer
mkdir /tmp/MLV_viewer
mkdir /tmp/MLV_viewer/storage
mkdir /tmp/MLV_viewer/mount
ln -s "$(dirname "$(cat /tmp/DUALISO/path_1)")"/*.MLV /tmp/MLV_viewer/storage
cd "$path_2"Contents/mlvfs
./mlvfss /tmp/MLV_viewer/mount --mlv-dir=/tmp/MLV_viewer/storage -f -o local -o volname=/tmp/MLV_viewer/mount -o allow_other --resolve-naming &
"$path_2"Contents/bin/MlRawViewer.app/Contents/MacOS/mlrawviewer /tmp/MLV_viewer/mount &
while pgrep mlrawviewer
do sleep 1
done
killall mlvfss
rm -r /tmp/MLV_viewer
rm /tmp/DUALISO/path_1
New download.
Reworked a lot around darkframe automation. Now a setting to process dng files together with darkframe files for example. All automated and multithreaded of course. Just film a few short darkframes along with your regular footage and add them to the mix of MLV files and see the difference in picture quality when g3gg0´s implementation of darkframe averaging takes place. The darkframe average settings can be reached from Main menu then select (A).
"The select darkframe storage" is optional and the thinking around building a darkframe storage folder is to be able to use it statically and always set and ready for any matched MLV footage. Fill the selected darkframe storage with darkframe MLV files and if they´re not created yet they will be in the process.
(https://s17.postimg.org/4r8lklub3/Screen_Shot_2017-06-08_at_00.03.11.png)
Danne,
Can i convert DNG sequence (edited in Lightroom) into MOV (say, Prores 4444 and other formats) with your app?
Davinci Resolve doesn't see changes that i made with DNGs in Lightroom.
Adobe Premiere doesn't see DNGs. And i wouldn't like to work with TIFFs, because anyway i need to keep DNGs to be able to change them after.
After Effects correctly sees DNGs with changes, but there are problems with understanding that app - and it's very bulky itself and too slow - 8 min for 10 sec video.
I just need a simple conversion from edited DNGs to MOV.
Thanks!
Yes, you can export to ProRes4444 and proxys too if you like. Once you get it rolin´one easy way is to do following
1 - Drag a folder with mlv file onto cr2hdr.app
2 - From Main menu select (p) and hit enter to get into the prores menu
(https://s13.postimg.org/4m02hho3b/Screen_Shot_2017-06-11_at_09.05.21.png)
3 - Select an output flavour. In the example I selected cineon but any of the settings could be used. If you want proxys too just select from the proxy section further down.
(https://s13.postimg.org/nsd9ko4l3/Screen_Shot_2017-06-11_at_09.05.50.png)
4 - If done select (r) to start the application. First the MLV files will be turned into folders with dng files and after this exported to ProRes files.
There will be two folders created in which the ProRes files will be exported to. Let me now if it works out for you.
*For dng files edited in acr/lightroom check out the setting (22) in the prores menu.
Hey guys. I have a strange problem with the latest version of the CR2HDR app that I just downloaded the other day. When converting my regular .MLV files (14 bit, with sound on MAC 10.11.6), for some reason the audio for each .MLV file dissapears from the dng folder for each clip?! I can see the audio file clearly when it´s working on spitting out the .dng files, it´s the first file that appears when it starts converting, and after it´s done, the audio file normally jumps to the bottom of the .dng files in each folder. But for some reason, as soon as CR2HDR is done processing a .MLV file to .dng files, the audio file disappears and is simply gone and I can´t find them anywhere? Very strange?
In this first pic when the app is converting the file you can clearly see the audio file at the top of the .dng folder.
(http://i64.tinypic.com/2czbp0z.png)
In this pic the .MLV file has been finished and there is no audio file at the top as usual, since it normally goes down to the bottom.
(http://i67.tinypic.com/ri7r82.png)
And in this pic you can clearly see that there is no audio file at the bottom either.
(http://i63.tinypic.com/lhdsg.png)
I opened the converted .dng sequence in Resolve just to check and see if the audio was present anyway, but it wasn´t.
So I went back to an earlier version of the CR2HDR app that I fortunately had saved in my system that I´ve been using up until a couple of days ago when I decided to try out the latest version. This version is from around march-april 2017 (can´t remember exactly), and the exact same .MLV file converts properly with the audio saved and present in the .dng folder as usual..
It´s just a regular 14 bit .MLV file, no dual iso or anything else, and I didn´t do anything differently in the CR2HDR app between both conversions, I used the exact same settings. Am I doing something wrong? I´ve looked thru the app to see if there has been an option for sound added somewhere that I don´t know about, but couldn´t find anything.
Thanks for a respons, and if this has been asked and answered somewhere, I´m sorry to write this post, but I´ve searched and couldn´t find anything about this..
Kind regards
Thanks for telling. I check into it. Could be I updated ffmpeg. Gonna check.
By the way. Could you upload the LOG file?
If possible upload a sample MLV.
Uploaded a new version. Audio issue fixed.
Thanks @steeth
Awesome!! Thanks a lot Danne. Will download the new version and give it a try.
Thanks again and have a great weekend!
Danne,
thank you a lot for your reply and for personal message! I tried what you wrote but can't get the result, obviously i'm doing it wrong((. May i ask you to kindly write exact steps that i need to make in your app in order to turn DNG sequence into MOV Prores4444?
I edit original DNGs in Lightroom and export them in DNGs into another folder. These changed DNGs i need to convert to MOV Prores4444.
Also do i understand you correctly, that i can change only the first DNG in the sequence and the app will apply the same settings to all DNGs in the sequence? Is this what you meant? If yes, how can i do it — what exact steps in the app?
Thank you!
Hi again.
I just tested out the newest build and I´m sorry to say that the problem with the audio .wav file getting deleted by itself is still there? Tried on several various .MLV files recorded with audio, and it behaves exactly like the last build I´m afraid. Going back to the older build from around march/april that I use and converting the exact same .MLV files, and everything works as expected again, and the audio is there.. Wierd..
Hi. Thanks for reporting. Since it works over here and I can´t get any other user reporting that issue I need more info. Best would be if you could send me some sample files over. You can create sample files by selecting (17) in the mlv_dump menu.
It will also be helpful if you send me the LOG file over.
Could you try replacing ffmpeg from an older build and put it in a newer build? It's inside Contents folder inside cr2hdr.app.
Thanks.
*I would avoid the older versions since it will be laggy when processing larger mlv chunks.
New version up.
cr2hdr.app @togg noticed some serious speed delays when processing bigger batches of MLV files. This should be fixed now. Instead of passing through all files checked through dfort fpm.sh pixel list it will now only run the script if MLV files are coming from cameras affected by the focus pixel issue(eosm, 100D, 700).
automatic easy batch capture and spit out dngs fast. like rawmagic except it gets rid of focus pixels automatically. no settings. just batch capture to dng. you tell it where to output it. it knows if you're in 5x crop mode automatically, it knows if you shot 3x3. it just makes dngs...what do think? good idea? danne for the win.
Thanks man! Glad to hear when it comes to good use. Determing optional output path is great, I agree. Quite a lot of stuff that can be achieved wit a few workarounds through automator.
I also added so that last path chosen is rememebered. Simply double click cr2hdr.app and it will open up from last selected folder.
New version uploaded
https://bitbucket.org/Dannephoto/switch/downloads/Switch.dmg
I decided to rename cr2hdr.app to Switch.
This project started as a cr2hdr automation tool but quickly grew into a processing tool for a lot of ML related things. Mainly my focus has been around mlv_dump and different flavours around mlv_dump. Cr2hdr is still in there of course!
This program is based around command line switches set through bash menus so therefore the name Switch felt as good one.
In this upload I also implemented Bouncyball mlv_dump on steroids version along with the ongoing ml-dng version already inside Switch. User can switch between both versions through main menu. I´d say Bouncyball did pretty impressive work here building/refining g3gg0/a1ex/dmilligan code.
I reworked scripts so darkframe processing will work with both versions. Focus pixel handling is also working differently between the two versions but thanks to dfort´s focus pixel script it can handle both ways.
(https://s2.postimg.org/rfsrtamwp/Screen_Shot_2017-07-26_at_22.25.25.png)
Nice! I hope Nintendo doesn't mind :)
New version uploaded in first post.
- Added fpmutil(Bouncyball)
A continuation of dfort focus pixel script(for cameras like eosm, 100D, 700D) only recoded and refined as a c program. We got speed!
https://bitbucket.org/bouncyball/mlv-tools/overview
Running time command comparing creation time building the first croprec focus pixel file for eosm:
fmputil
real 0m0.021s
user 0m0.012s
sys 0m0.005s
fpm.sh
real 0m4.717s
user 0m10.848s
sys 0m2.304s
- Also added latest version of mlv_dump_on_steroids
http://www.magiclantern.fm/forum/index.php?topic=18975.0
I am keeping dfort´s trusty "steamboat" fpm.sh focus pixel script in my ml-dng version of mlv_dump for future history class while adding fpmutil to mlv_dump_on_steroids in Switch.
Enjoy and once again kudos Bouncyball.
Hey Danne,
Looks exciting!
I can launch Switch from the DMG, but when I copy to my applications folder I get 'The application "Switch" can't be opened".
I have "allow apps from anywhere" selected in my OSX security settings, and I'm on 10.11.6.
B
I tried to reproduce the issue on macOS Sierra but couldn´t. I assume it was working before? Could you try doing these things:
1 - Restart computer and then double click from /applications folder again
If that isn´t working do:
2 - Drag the folder on to "Switch". Will do the same as double clicking only it will open from within your folder directly
Now try step one again.
3 - You can also try downloading the unzipped repository and see if that helps
https://bitbucket.org/Dannephoto/switch/get/b69c25498d89.zip
Edit*
4 - Open up Switch in automator on your computer and save it. Then double click
On macOS Sierra I can´t open from the dmg file since gatekeeper complains about opening "First_out.app" which is a workaround I use to open up Switch when double clicking it.
Hello,
I'm planning to shoot a doc with 20hrs footage. I want to use 5DmkIII + compressed 14 bit 1920*1080 @24p and post-production workflow MLV=>CDNG (DNG if not possible)=>Proxys ProRes=>online edit & grading on CDNG/DNG (resolve 14)
I don't want to use MLVFS because it's too slow on Resolve. The only reliable app for transcoding seems to be switch (tested footage but no batch, tested MLV producer but no compressed 14 bits, tested rawmagic but outdated). What are the simplest settings in switch to transcode to CDNG/DNG ? Can I use full auto mode ? Or there isn't automode for compressed 14 bits ? Sorry, lot of questions and thank you in advance =)
I
Quote from: Danne on July 26, 2017, 09:52:07 AM
I tried to reproduce the issue on macOS Sierra but couldn´t. I assume it was working before? Could you try doing these things:
1 - Restart computer and then double click from /applications folder again
If that isn´t working do:
2 - Drag the folder on to "Switch". Will do the same as double clicking only it will open from within your folder directly
Now try step one again.
3 - You can also try downloading the unzipped repository and see if that helps
https://bitbucket.org/Dannephoto/switch/get/b69c25498d89.zip
Edit*
4 - Open up Switch in automator on your computer and save it. Then double click
On macOS Sierra I can´t open from the dmg file since gatekeeper complains about opening "First_out.app" which is a workaround I use to open up Switch when double clicking it.
All well after restart. Thanks Danne.
Great :)
@12georgiadis
Switch only outputs to cdng.
Choose either full auto or enter (p) to tweak prores settings. If you like to tweak mlv_dump settings you can do that too. Select either (m) or (ms) depending on what mlv_dump version you want to run. When done with all switching select (r) to run Switch.
Regarding compressed footage. Mlv_dump outputs both compressed and uncompressed cdng files automatically. If you need to decompress MLV files there are settings for this in the menus. Play around a little and get back on how it goes.
Thank you Danne. So, if I understand correctly, I can have an auto config from compressed MLV and batch export in compressed cdng ?
Yes, that should work just fine. Not sure if the dng comes out compressed or decompressed. Anyway. The dng file itself should behave exactly the same either way.
Once you are about to transcode you could write me and I help you set up a configuration if needed.
will this work on El Capitan (hackintosh)??
Finally made the switch to Switch and damn Danne did it again!!! :D
Once again thanks to Bouncyball and currently batch processing lossless DNG's w ease on 10.12.5 OS X.
Love the new percentage notification counter. Will play around with some more later.
Thanks, Danne for continuing to make bold decisions revolving around your app!
@nikfreak. Most probably yes. MacOS Sierra has a tougher gatekeeper system, that's the main reason I put the headline up. Older os should work fine. Would be nice to hear if you get it up and running.
@Deafeye. Welcome back dude!
Hopefully this is just me (or just the 100D files) but I ran it twice to be sure and seems that spanning files do spit out DNG's into respective folders but then doesn't clean up (into the A_ORIGINAL folder) properly for some reason after conversion is complete.
Here's a screenshot...
(https://farm5.staticflickr.com/4298/35348454224_f93230644b.jpg) (https://flic.kr/p/VRC4Vj)
Could this be a bug?
Thanks Danne. I'm making a couple of tests soon (low light, overexposure+sky etc.). I keep you in touch to get the best out of it. This is a killer app !
@12georgiadis
Great!
@deafeyejedi
I´m trying to reproduce with a bunch of spanned eosm files but it seems to work ok here. Are you exporting to a different folder location? Could you describe more in detail and also maybe send me your LOG file?
Quote from: Danne on July 26, 2017, 10:27:06 PM
@deafeyejedi
I´m trying to reproduce with a bunch of spanned eosm files but it seems to work ok here. Are you exporting to a different folder location? Could you describe more in detail and also maybe send me your LOG file?
After a thorough investigation on my end -- turns out it was my own user error, why? Because during this trip out to Vegas I only brought one card (128gb) w me and no MacBook pro or let alone a card reader. So what I did was deleted few MLV files from within File Manager in ML menu after running out of space towards the end and thus the reason why I had incorrectly assumed Switch wasn't properly spitting out ANYTHING for certain spanning files.
It was actually missing the original MLV's this whole time. Shame on me, I know. Sorry for the false alarm.
Btw if I can recall correctly that your app used to be able to do this, right? Once I kill the rendering for whatever reason and I decide to run it again even with movie files already spitted out -- it would continue to render overwriting the ones that were already done when it used to be able to skip the ones that are already done and go to the next new file that hasn't been spat out yet, makes sense?
I could swear your app was already capable of doing this but maybe just not with the latest release of Switch? Correct me if I'm wrong though, Danne!
No overwriting of ffmpeg prores files but with ae project files ut will.
Killing automator just freezes eveything from when it's killed but it's better than waiting for automator to finish since it actually runs long after turned off.
Quote from: Danne on July 28, 2017, 07:53:44 PM
No overwriting of ffmpeg prores files but with ae project files ut will.
That's what it was. Makes sense. Thanks and perhaps is there a reason for it not to work w ae project files atm?
Quote from: Danne on July 28, 2017, 07:53:44 PM
Killing automator just freezes eveything from when it's killed but it's better than waiting for automator to finish since it actually runs long after turned off.
I understand and am aware of this. Just had to because I had forgotten to do Dual-ISO process for the few MLV's that otherwise would have came out w stripes. So a restart was necessary for this situation.
I have tested several .mlv files shooted in Crop mode 1920 50/60 3x3 (1920x800 - 60fps), exported in .dng from Switch and imported in After Effects via Camera Raw, but each time I have tried to import them AE crashed. I have succeeded in importing them by deleting several on the last .dng images (10,20 ou more) of each files. Each time I have manually stopped the recording on my 5DMKIII (it hasn't stopped due to lack of speed). Anyone else experienced this?
Thanks,
S/.
Never tried this. Are the last dng files corrupted somehow? Are you getting the same results with mlv_dump_on_steroids and the regular mlv_dump version in Switch?
I've deleted the files because it was only some tests to see if the workflow was good and consistant. The last .dng seems all OK for me.
I have recorded new files today, and no problem at all. Strange...
Perhaps some error from AE or Camera Raw as I use a demo version from where I am for now.
I will record some new files tomorrow to see if I can reproduce the problem and try to see what is the origine (if not I could send you some files).
Thanks,
S/.
Hello Danne, thanks for giving Switch to the ML community. Your app is dope!
Love the fact that all is included.
A question about darkframe workflow, because I don't seem to understand it even if I watched your youtube videos.
The purpose is to reduce noise at big ISO right?
I have shot two files at 1920x800 - 60 fps - ISO 3200 - 14-bit lossless
The first one is a normal shot.
The second one has been made with same parameters but caps off and really short (200 frames).
I've put the two .mlv files inside a folder at the same hierarchy level, then run Switch.
I've tried to activate the darkframe average automation but nothing came from it.
Could you please explain in short lines how to get a result, a .dng file processed?
I don't know if it is even possible to do this or if you get two files to process in a comp ap. later...
Thanks a lot,
S/.
I think you need to decompress MLV files from lossless before doing darkframe automation. I think you select (11) in mlv_dump menu to decompress. In compression/decompression menu start by selecting (d) , can't remember if you add more before running (r).
Ok once MLV files are decompressed and ready you can keep all mlv files in the same folder. Select (A) from the main menu and you can choose (1) and then (r).
By the way. If you have trouble getting dng files at all from those files you can find the setting --relaxed in mlv_dump menu. There is some metadara sometimes missing in compressed footage, that is why.
Good luck and don't forget to post your LOG file if you have more issues :).
Yeah, you are right, the average file hadn't been created because of the 14-bit lossless. I have followed your way and all is OK now, thanks a lot.
The difference between a standard file and a processed one isn't astonished, but the noise is a little bit less pronounced and seems more random (a sort of visible "pattern" effect disappeared).
Thanks...
S/.
Cool.
Hi everyone,
I need your help, please.
I have a very basic question: I use cr2hdr only for mlv to CDNG task (no dual-iso), but I have a strange issue here: after converting mlv into dng I noticed that the very first frames have right colours (even if a bit darker then the original mlv), while from (let's say...) the 4th frame to the end the colours turn into a very dark and bluish tone... In other words, cr2hdr converts badly my 5D3 files.
I have simply set (m) mlv_dump settings, and (10) disable dualiso automation, then (r) run cr2hdr.
My .mlv files are without audio (like legacy raw video) even if made with a recent ML release.
I work on an hackintosh Sierra.
Any ideas?
Thanks a lot.
Upload two sample dng files, one good and one bad. If possible upload a MLV file.
Exiv2 is applying white level changes to the dng if you filmed with your camera set to auto white balance(awb). If it stops after the 4th file It might say why in the LOG file also. I need more info to help you. Right now it's guesswork.
Also download the latest. It's renamed to Switch so if you're lucky it might have been fixed.
I'm trying Switch right now, and it seems it's solving the problem.
Anyway, yes: I'm converting auto-balanced footage (awb) and this could be the problem.
I also set the camera (5D3) in AdobeRGB color space (instead of the default sRGB), could it be a problem for Switch?
(Thanks a lot as always!!!)
I think it was broken before for a while. Great that it works on your hackintosh.
Dear Danne,
I've completed a long test. I used all auto-balanced MLV files without audio (like legacy RAW files) on 5D3.1.2.3. It seems to me (but I could be wrong) that Switch tries to correct the WB in ALL the awb-files, and I must say that 9 out of 10 it rocks, while 1 out of 10 is (again) very-very dark and blue.
I'd have some question (I've read all the topic, but some doubt remains):
1. Does Switch work well with both sRGB or AdobeRGB files, or it works better with sRGB only?
2. Is my "dark-bluish issue" due only to the awb set of my footage, or could be I didn't properly install Adobe DNG Converter? (I did just drag&drop the app onto Applications folder);
3. Anyway, outputting DNG, the bluish file can be corrected in Davinci Resolve's RAW tab, right?
4. Which is the difference between (m) and (ms) in Switch's Main menu?
5. Once in (m), I usually added only (15) to disable dualiso (my files are not dual-iso) and then (r), but now I notice the "disable vertical stripes in highlights" option. Sorry for the silly question: are there usually vertical highlights stripes is 5D3 footage? (Because I never saw them, but it could be my inobservance);
6. Which is the difference in quality terms between dng vs comressed dng? And if I want to convert MLV into compressed dng, should I add (10) compress dng, or (11) compress MLV?
7. (12) option says "uncompressed exporting": what does it mean? If I don't add this option I should anyway convert a MLV into uncompressed 14bit dng, right? Or not?
8. What is MLVFS?
Sorry for this big bunch of questions :)
Really thanks for your very kind help.
A lot of questions. Out in the country but will answer a few.
2. Either camera is set to awb or some othr manually selected wb setting. If you have older processed dngs with different white balance just correct it as you do in nle.
3. ms is a specific branch from Bouncyball. Search for it in the forum. It's badass and that is why it's in there. Both m and ms outputs cdng files.
5. I wouldn't touch vertical stripes setting. It's always on.
6. Exactly the same quality but smaller in size. Works for 10bit 12bit files in ms version. Works with 14bit in m version. Compress mlv is for archival of mlv files.
7. Uncompressed. I never use it. Good for inspecting problematic files.
8. Use search
Now back to work. I'll check in later...
Many thanks for your useful reply!
It just remains the little doubt about AdobeRGB vs sRGB MLVs: Switch should handle the mlv-to-dng conversion as well, right?
In cam colorspace has no effect on your dng files. MLV contains raw data. You apply stuff in post. DNG spec itself contains color matrix information and your nle will default into some rec709(viewfriendly) output. You can change all of this with color management, luts etc to reach full raw potential but in post...
Quotewhile 1 out of 10 is (again) very-very dark and blue
You really need to provide sample files of this. Sample package? Easily created from the m or ms menu. Or just upload a dng file.
Thanks Danne,
HERE (https://www.dropbox.com/s/88n155cvcpxm0gi/dark_bluish.zip?dl=0) is a zip with: dark/bluish dng (just the first frame*) + correct image (a screenshot of MlRawViewer) + Log file.
(*Just the first frame because actually Switch corrects the former cr2hdr error, that was first frame correct vs the others uncorrect: now Switch works well for all the frames, or - 1of10 - it changes the colours of all the frames.)
Thanks for your time and attention.
@adjork
Thanks for posting info and dng file. If there still are issues with auto white balance on some of your files I would really like to know to be able to narrow down possible hard to find bugs. The questions are:
1 - After you processed your MLV files into dng files with Switch, what folders with dng files are not having auto white balance applied? You processed a lot of files so if there are multiple folders showing broken auto wb there might be a pattern?
2 - Can you post me a sample of a mlv file that you know is getting the wrong white balance applied in "Switch"? I would like the full sample package. Do following:
With only the problematic MLV file in a folder select (m) for mlv_dump menu then select (17) to build the sample file package.
Thanks
Quote+ correct image (a screenshot of MlRawViewer)
Actually MlRawViewer always outputs the same white balance which isn´t what the AsShotNeutral tag is showing in your dng file. Most likely your kelvin is set low around 2500-3000k in cam(in the blue dng file) and then you put your setting to AWB for auto white balance. WHat happens here is that your 2500-3000k kelvin numbers are registered in MLV therefore the blue output. Now Switch is the only program that applies its own auto white balance numbers to the dng files since the MLV format can´t store AWB. It´s either that or the blue output when opening up the dng file.
Switch now also works with linux(Ubuntu). I ported most of the processing scripts to Ubuntu. It works almost like the mac automator version but with the difference that the user needs to double click a desktop icon to open up a selection prompt. Just follow the HOWTO inside the download package to get it running. Big thanks to Bouncyball for early testing and feedback. Treat it as beta.
Ubuntu version
SWITCH_LINUX.tar.gz (https://bitbucket.org/Dannephoto/switch/downloads/SWITCH_LINUX.tar.gz)
linux_source_code (https://bitbucket.org/Dannephoto/switch/branch/Switch_linux)
(https://s4.postimg.org/daryp4h19/Screen_Shot_2017-08-20_at_17.09.56.png)
***Saving space for upcoming post***
Switch ported to Linux(see above post)
Looks good Danne!
Does it also run from the command line, over SSH? That would be awesome because I can remotely start the conversion process (using SSH) on pc's that are more suitable for doing those tasks.
What is a normal render time for near full HD to Prores 4444? I had a 20 minute clip that took 11 hours to render from DNG to Prores. Seems a bit long and impractical.
@ErwinH
Welcome to the world of switches :).
Ok, are we talking about mac or linux version here? 20 hours for a 20 minute clip sounds like something´s wrong. Or a very slow computer if on mac? Havn´t been able to test extensively on linux. FFmpeg isn´t exactly using all cores when creating the prores files so I parallellized processing to do 4 exports simultaneously for speed reasons.
SSH. I must admit I have no idea about this procedure. Sounds cool though so let me know if I can do something to help you out with this.
Thank you!
I was running the process on a Macbook Pro 13" 2015 i5. I know it's not the fastest, but it's workable.
The MLV is 45.76GB large and contains 31345 frames and it takes 50 minutes to extract the dng files from the MLV. The process of creating a prores movie takes ages ;) Maybe you can add the running times to the log.txt file?
The SSH part, is simple running the command from a console, terminal, like pressing ctrl+alt+f1 on a linux box. If it's possible to start the run command for a task from the command prompt would be very helpful. I am aware of the fact that multithreaded might be an issue.
It would also be helpful to change the number of parallel tasks, because working with MLV's that are 20 minutes long takes up a lot of disk space. 4x100 GB of temp data is a lot!
QuoteThe MLV is 45.76GB large and contains 31345 frames and it takes 50 minutes to extract the dng files from the MLV. The process of creating a prores movie takes ages ;) Maybe you can add the running times to the log.txt file?
I have total running time in the notifier window which should pop up when done. Are you thinking some estimating time in LOG.txt file or a revealing the frame by frame process in ffmpeg? I intentionally neglected this to not let the LOG grow too big.
About speed, I think the bottleneck is the pipe through dcraw. Higher quality is demanding. Might be some alternate prores codec which could help speed as well but probably will create bigger files. Could check into this.
QuoteThe SSH part, is simple running the command from a console, terminal, like pressing ctrl+alt+f1 on a linux box. If it's possible to start the run command for a task from the command prompt would be very helpful. I am aware of the fact that multithreaded might be an issue.
It could be possible to start Switch outside automator by double clicking a command script. The Switch_MAIN_DBG.command is more or less doing this but it needs paths which now are established through automator.
QuoteIt would also be helpful to change the number of parallel tasks, because working with MLV's that are 20 minutes long takes up a lot of disk space. 4x100 GB of temp data is a lot!
I could add a switch which would let the user choose number of processes manually. On a side note you can assign a different output location if you want to export to let´s say an external drive. Do this with (O) setting in main menu.
By the way. Are we talkning about MLV to dng processing or the dng to prores processing?
QuoteI have total running time in the notifier window which should pop up when done.
This summary would be nice to have in the LOG.txt file. You could add the runtime of a task after that task is done.
I am using external drives, because the internal drive is way to small on my MBPr. 256GB isn't that much once you start doing video, but still I think it's a good thing to be able to alter the amount of processes based on preferences, other tasks, etc.
MLV is 43GB.
DNG's are 88GB
Prores is 35GB
The Prores image is ok, but waiting 11 hours for it to build is unpractical and it's more convenient to use the DNG files instead of the Prores file.
Just finished some tests and there are alternatives which will speed up things considerably. I ran 101 frames in a dng folder and it goes like this:
Based on running 101 frames
With the settings right now highest FFmpeg quality and dcraw -q 3:
41 sec dcraw -q 3 FFmpeg -vcodec prores_ks -pix_fmt yuv444p10
Altering -q setting in dcraw
27 sec dcraw -q 0 FFmpeg -vcodec prores_ks -pix_fmt yuv444p10
New prores codec(422)
28 sec dcraw -q 3 FFmpeg -vcodec prores -pix_fmt yuv422p10le
14 sec dcraw -q 0 FFmpeg -vcodec prores -pix_fmt yuv422p10le
Pick your poison :).
In real life I think it´s actually pretty hard to see any difference here. Hard to say how color is affected with dcraw -q alterations. Anyhow. If there is any interest and some use for prores I could smack in some speed up alternatives in the prores menu.
Great! Thanks for looking for solutions! I even like the 422 codec a lot! 444 is way overkill for a lot of use cases. Mine is one of them.
If you want I'll give it a go on bigger files.
Just pushed fixes for
- Specifying thread amount
- Added total time to LOG.txt
I´ll put in the alternative codec and then you might wanna check quality and speed comparison for bigger files. Stay tuned...
Ok, here we go. New upload in first post. Following changes suggested from ErwinH added:
- Specify thread amount
- Add total time to LOG.txt
- Switch to much faster prores422 codec(less than half processing time)
Main menu (CP) for changing thread processing
(https://s30.postimg.org/5v7p8h40x/Screen_Shot_2017-08-23_at_20.45.06.png)
In ProRes menu (07) to switch codec
(https://s30.postimg.org/f49vilcwx/Screen_Shot_2017-08-23_at_20.45.21.png)
Note that I also switch from -q 3 to -q 0 in dcraw when selecting new codec. Do check the final output and compare.
5758 frames of 1360x764:
Using codec yuv422p10le
Total running time:
0 Days, 00 Hrs, 55 Min, 40 Sec
X to FFmpeg ProRes:
XxXx
mlv_dump:
0 Days, 00 Hrs, 08 Min, 24 Sec
raw2dng:
XxXx
dcraw_FFmpeg ProRes:
0 Days, 00 Hrs, 47 Min, 11 Sec
cr2hdr(CR2 files):
XxXx
Still quite a lot, 20-30 seconds of work for every second of footage.
This is with the old version (with a small change)
Try the latest version. I changed to prores instead of prores_ks and altered -q setting in dcraw which makes all the difference.
Can you also change the output directory to ProRes422 instead of ProRes4444? So you can keep different versions apart?
1522 frames 1:01
Total running time:
0 Days, 00 Hrs, 04 Min, 11 Sec
X to FFmpeg ProRes:
XxXx
mlv_dump:
0 Days, 00 Hrs, 01 Min, 22 Sec
raw2dng:
XxXx
dcraw_FFmpeg ProRes:
0 Days, 00 Hrs, 02 Min, 44 Sec
cr2hdr(CR2 files):
XxXx
That's a nice improvement! This way it's a lot easier to use the footage.
Can you also remove the DNG files after the run is complete?
Sure. Just check the quality and speed increase of the 422 files and if all is ok I do the folder change as well.
Quality check isn't my cup of tea. But the speed improvement is huge! 4:1 instead of 20:1.
Cool. Well, I did a sloppy check before but cannot spot any difference. I'll add the 422 folder in an hour or so. Thanks for feedback and ideas.
Can you also add something to clean up the temp dng files, so larger queue's don't need as much space as needed now.
Another improvement would be to finish one file at the time. First extract the dng's from the first file and convert it to ProRes, remove the dng's and then move on to the next file.
I´ll see if I could do deletion of dng folders possible.
Waiting for files before moving to the next is not a priority. More important to limit parallell processing and to keep the flow going for speed reasons. In the end files will all be sorted.
Isn´t the "set thread manually" to 1 doing exactly what you want? Processing one file at the time?
New version uploaded.
Added a switch for deletion of dng folders after ProRes exports. It´s at the bottom of th ProRes menu.
(https://s30.postimg.org/5nonppig1/Screen_Shot_2017-08-23_at_23.06.18.png)
QuoteAnother improvement would be to finish one file at the time. First extract the dng's from the first file and convert it to ProRes, remove the dng's and then move on to the next file.
Setting the threads to 1 in main menu (CP) and setting the (X) switch in ProRes menu would probably do exactly what you ask for.
Regarding deletion of dng folders. The switch can be set and run from within the ProRes menu. As soon as you leave the ProRes menu this switch will be reset. Reason for this is that there is a good chance you´ll forget it next time you start switch and then the damage is done.
For complexity reasons I´ll probably won´t add a prores422 folder. All files will end up in the ProRes4444 folder instead.
Nice updates, @Danne! :)
Quote from: Danne on August 23, 2017, 10:53:05 PM
...As soon as you leave the ProRes menu this switch will be reset. Reason for this is that there is a good chance you´ll forget it next time you start switch and then the damage is done.
+1
QuoteIsn´t the "set thread manually" to 1 doing exactly what you want? Processing one file at the time?
I think I messed up a setting skipping the conversion to ProRes, which gave the impression it was doing the MLV_dump first for every file, and after that doing the conversion to ProRes.
Quote from: Danne on August 23, 2017, 10:53:05 PM
As soon as you leave the ProRes menu this switch will be reset. Reason for this is that there is a good chance you´ll forget it next time you start switch and then the damage is done.
-1
What is the use for the DNG files after you have converted them to ProRes. You can quite quickly extract them from the MLV and they just take up space.
There could be cases when one saves and works strictly from folders with dng files. Not good practice to set permanent deletion flags hidden in submenus. If you really want to keep the flag set you can simply erase following lines inside the Menu.command:
#remove destructive switch
if [ -f /tmp/DUALISO/DEL_DNG ]
then
rm /tmp/DUALISO/DEL_DNG
fi
Another tip. You could build and create prores without the need to convert the MLV files by running MLVFS from inside the main menu (ml). Once in the menu select (A) and MLV files will be turned into dng folders. All MLV files will be placed in A_ORIGINALS folder inside two subfolders. Now you can move into (p) prores folder and start processing prores files. Once don open up Switch and go back to (ml) and:
you can then get back your MLV files via menu by doing (EJ) once you´re done or manually dig them out from the subfolder in A_ORIGINALS.
It will not be as fast as with already processed physical dng files but maybe fast enough.
MLVFS works, and saves a lot of space, but the development of a 2 minute clip takes 45 minutes.
dcraw / ffmpeg runs at 4-4.5 fps at the moment, on a slow virtual machine running Ubuntu Xenial, using extracted dng's. Storage is a samba mounted network storage, so that's not in it's advantage either.
My MBPr, using external USB3.0 SSD storage runs at approximately 3-3.5 fps using MLVFS.
More testing results:
Profile 2?
Debayer 0
find DNG/ -iname '*.dng' -print0 | sort -z | xargs -d "\n" -0 ~/Switch/bin/dcraw +M -c -6 -W -q 0 -w | ffmpeg -f image2pipe -vcodec ppm -r 25 -i pipe:0 -vcodec prores -pix_fmt yuv422p10le -n -r 25 M23-1452.mov
frame= 100 fps=4.4 q=-0.0 Lsize= 51746kB time=00:00:04.00 bitrate=105975.6kbits/s
Debayer 3
find DNG/ -iname '*.dng' -print0 | sort -z | xargs -d "\n" -0 ~/Switch/bin/dcraw +M -c -6 -W -q 3 -w | ffmpeg -f image2pipe -vcodec ppm -r 25 -i pipe:0 -vcodec prores -pix_fmt yuv422p10le -n -r 25 -profile:v 2 M23-1452_profile2_q3.mov
frame= 100 fps=2.4 q=-0.0 Lsize= 48464kB time=00:00:04.00 bitrate=99255.2kbits/s
Quality 3 (Hq)
Debayer 0
find DNG/ -iname '*.dng' -print0 | sort -z | xargs -d "\n" -0 ~/Switch/bin/dcraw +M -c -6 -W -q 0 -w | ffmpeg -f image2pipe -vcodec ppm -r 25 -i pipe:0 -vcodec prores -pix_fmt yuv422p10le -n -r 25 -profile:v 3 M23-1452_profile3.mov
frame= 100 fps=4.4 q=-0.0 Lsize= 77486kB time=00:00:04.00 bitrate=158690.4kbits/s
Debayer 3
find DNG/ -iname '*.dng' -print0 | sort -z | xargs -d "\n" -0 ~/Switch/bin/dcraw +M -c -6 -W -q 3 -w | ffmpeg -f image2pipe -vcodec ppm -r 25 -i pipe:0 -vcodec prores -pix_fmt yuv422p10le -n -r 25 -profile:v 3 M23-1452_profile3_q3.mov
frame= 100 fps=2.4 q=-0.0 Lsize= 70703kB time=00:00:04.00 bitrate=144800.6kbits/s
Prores_ks
Quality 2
Debayer 3
find DNG/ -iname '*.dng' -print0 | sort -z | xargs -d "\n" -0 ~/Switch/bin/dcraw +M -c -6 -W -q 3 -w | ffmpeg -f image2pipe -vcodec ppm -r 25 -i pipe:0 -vcodec prores_ks -pix_fmt yuv422p10le -n -r 25 -profile:v 2 M23-1452_proresks_profile2_q3.mov
frame= 100 fps=2.1 q=-0.0 Lsize= 35701kB time=00:00:04.00 bitrate=73114.9kbits/s
find DNG/ -iname '*.dng' -print0 | sort -z | xargs -d "\n" -0 ~/Switch/bin/dcraw +M -c -6 -W -q 3 -w | ffmpeg -f image2pipe -vcodec ppm -r 25 -i pipe:0 -vcodec prores_ks -pix_fmt yuv422p10le -n -r 25 -profile:v 3 M23-1452_proresks_profile3_q3.mov
frame= 100 fps=2.1 q=-0.0 Lsize= 53619kB time=00:00:04.00 bitrate=109811.1kbits/s
Prores_ks 4444
find DNG/ -iname '*.dng' -print0 | sort -z | xargs -d "\n" -0 ~/Switch/bin/dcraw +M -c -6 -W -q 3 -w | ffmpeg -f image2pipe -vcodec ppm -r 25 -i pipe:0 -vcodec prores_ks -pix_fmt yuv444p10 -n -r 25 M23-1452_proresks_4444.mov
frame= 100 fps=1.8 q=-0.0 Lsize= 79918kB time=00:00:04.00 bitrate=163672.1kbits/s
On the MBPr
100 frames read from MLVFS vs 100 frames read from storage:
Storage:
frame= 100 fps= 11 q=-0.0 Lsize= 50995kB time=00:00:03.96 bitrate=105491.5kbits/s speed=0.446x
MLVFS:
frame= 100 fps=6.6 q=-0.0 Lsize= 50995kB time=00:00:03.96 bitrate=105491.5kbits/s speed=0.261x
New version in first post
Added:
- MOV proxy/wav file integration when paired with corresponding MLV file.
Follow_this_post (http://www.magiclantern.fm/forum/index.php?topic=19300.msg189157#msg189157)
Work like this:
1 - Film with proxy setting on while recording MLV files to camera
2 - Add MLV files and MOV proxy files to a folder and run Switch
What will happen:
Switch will match the corresponding MLV/MOV proxy and then cut out the black frames from the MOV file. In the process it will also generate a wav file which will be matched with the dng files to the corresponding MLV file. Additional metadata information will be injected to the wav file, needed for davinci resolve.
I tried to match files also if they are named slightly different. This seems to be the case when the camera prefix preset is running. Typically you would get a match like this:
2U0A7621.MLV
IMG_7621.MOV
as opposed to:
2U0A7621.MLV
2U0A7621.MOV
Anyway. Switch should be able to handle both scenarios.
What else:
When all processing is done there will be a "cleaned" MOV proxy file next to your DNG folder. The original MOV file will be moved to A_ORIGINALS.
Commits:
https://bitbucket.org/Dannephoto/switch/commits/183ac6b2096ead0bb7d637f7c73ff95183dc2d10
https://bitbucket.org/Dannephoto/switch/commits/f84a76a6d2016705adb6e5a386cf5591b2731afd
Replaced ffprobe dependency with good old exiftool(Phil Harvey)
Thanks to a1ex for script feedback and very cool MOV proxy recording setting.
Thanks Lars Steenhoff for testing and feedback.
Quote from: Danne on August 30, 2017, 05:49:29 PM
Typically you would get a match like this:
2U0A7621.MLV
IMG_7621.MOV
as opposed to:
2U0A7621.MLV
2U0A7621.MOV
How to reproduce? I get 3U3B for both MLV and MOV...
Switching to sd card (2) in canon setting
File name setting to default IMG_ in canon menu
MLV comes out with Preset code prefix 2U0A
MOV comes out with prefix IMG_
Of course changing the File name setting to Preset code fixes it...
Is canon select card (2) setting reset to CF card upon restart? I know preferred card in ML menu is set to CF card but what if I want to keep the MOV proxy workflow going without going back to canon menu again?
Okay, my default is 3U3B (after clearing Canon settings). Indeed, if you change this prefix from Canon menu, the new one is not recognized by ML (but it's not IMG_ either). Should add some more tests about this (currently this is tested in api_test.lua for photos only, without changing Canon setting).
The "preferred card" setting in ML changes Canon card at startup - it does nothing else. MLV from mlv_lite always goes to CF if you have one inserted, otherwise SD. H.264 always goes to the card selected in Canon menu.
So, just leave "preferred card" off (it's off by default).
Ok, so reproduced. Good. I also had to reset some configurations to get the behaviour reproduced over here.
So, just leave "preferred card" off (it's off by default)
preferred card to off fixes it however Preferred card CF is the default upon installation over here...
Danne, incredible workflow !
With this update, I think proxy can work as offline/online workflow editing
1) MLV/proxy => switch => clean proxy and sound with dng synched
2) edit with clean proxys for editing
3) lock edit => XML => Resolve => conform by name => online edit with synched dng
I'll try to test but that can save a lot of time and storage space
Hopefully yes. Please test and report back how it goes :).
Ok, crossfinger ;-)
@Danne, @Alex, MLV are recording fine in CF by default, but for H264 proxy, I have to select SD card again in canon menu each time I turn off/on the camera. Is there a special setting to keep it in memory ?
Set preferred card CF in ML menu to off.
http://www.magiclantern.fm/forum/index.php?topic=15108.msg189323#msg189323
By the way. Will refine proxy process to this:
http://www.magiclantern.fm/forum/index.php?topic=19300.msg189351#msg189351
Ok it worked for SD/CF management by disabling the function in ML menu.
New download.
I rewrote proxy script parts. Should now match precisely since calculating ending part derived from actual amount of dng frames.
http://www.magiclantern.fm/forum/index.php?topic=19300.msg189351#msg189351
Commit:
https://bitbucket.org/Dannephoto/switch/commits/13104b4d77d3a78656dcd28c0c10a62c7d7427d7
Danne, where is the option to sync proxy/sound/mlv in switch ?
No option. Automated. Just include your proxy files together with your MLV files and run Switch the normal way.
http://www.magiclantern.fm/forum/index.php?topic=15108.msg189314#msg189314
Also:
I just uploaded a new version which supports (C) changing to an alternate output folder.
Hello Danne,
I would like to replace all my old MLV workflow with Switch to speed up the process...
My concern for the moment is the White Balance, how can I manage it like in CR for example where I can tweek Temperature and Tint?
Is it possible to do this from Switch?
Second question, I use Cinelog-C (http://www.cinelogdcp.com/) to convert my MLV footages.
Is it possible to apply it directly from Switch as I don't know if it is some sort of LUT or not (the file extension is .dcp for the 5DMKIII)?
Thanks for any help on this.
S/.
Cool. Hope you´ll like it.
You can use DCP profiles on your dng files. In ACR/lightroom that is. All tags needed are provided in dng metadata.
Regarding white balance. White balance from camera is added. Then you need to refine it in your nle.
There is one way to create wb coordinates. You will need imageJ installed and then a good portion of patience as it´s about picking points on a tmp tiff file, go to next pick and save point etc. You find that workflow in (m) menu (mlv_dump at the bottom). Nice thing here is that white balance is actually transformed into actual dng AsShotNeutral numbers.
If you plan on exporting to prores from within Switch I suggest you try one of the provided logs.
Ok, I did a test with a folder containing .MLV (comp raw) + proxy with sound. I just ran switch without touching any options. It created me 5 types of files :
1) X proxy (APR LT or Proxy 36) black frames
2) X 444 (APR 444) black frames
3) APR LT or Proxy 36 no black frames no sound
4) APR 444 no black frames no sound
5) CDNG no sound
Is there a way to get a trimmed (no reencoding) canon H264 .mov + sound w/o blackframe ? I'd like to test it to edit offline, and then make an online edit on resolve with CDNG using MLVFS. It could be a fast and lightweight workflow.
Quote from: 12georgiadis link=topic=15108.msg189435#msg189435
Is there a way to get a trimmed (no reencoding) canon H264 .mov + sound w/o blackframe ? I'd like to test it to edit offline, and then make an online edit on resolve with CDNG using MLVFS. It could be a fast and lightweight workflow.
Sounds like a good workflow
QuoteIs there a way to get a trimmed (no reencoding) canon H264 .mov + sound w/o blackframe ?
Do following:
1 - Fill a folder with proxy H.264 and MLV files
2 - Open Switch and in the first menu select (R) to reset Switch.
3 - From the same menu select (r) to run Switch
What do you get?
It should be as straight as selecting (r) from the first menu.
The proxies and the mlv files are named the same right?
I'm not getting the Wav file, I'm using the test video that you also have Danne, any idea what I could be doing wrong?
And also a request, can you make an option to leave the original footage in place to where it is and only make a output directory with the new files?
The wav file should be inside the dng folder next to your dng files.
You mean leave the original MOV alone and move the cleaned MOV to a specific proxy folder?
Yes leave footage alone and make only new output folders with new files.
For example if I have a compact flash card mounted, I can select the folder and it would not touch the files, only create new folder on my computer and save all new files to my ssd.
I could not find the Wav file in the DNG folder with the latest build ( like 25 min ago )
I'll have a look within a couple of hours.
Could you test with another file as well?
Cool, I wil do more tests
Tested and your file is working over here. Sure you didn´t run your prores file without the audio? Also check through the whole folder make sure you didn´t miss the wav file somewhere.
If you test with another file and it´s still not working I need to see the LOG.txt file.
Here's the log file, I looked for the wav but it seems it was not created
ls: *.RAW: No such file or directory
ls: *.RAW: No such file or directory
File not found: /tmp/DUALISO/what_cam_000000.dng
File not found: /tmp/DUALISO/what_cam_000000.dng
File not found: /tmp/DUALISO/what_cam_000000.dng
rm: /tmp/DUALISO/what_cam_000000.dng: No such file or directory
ls: /tmp/DUALISO/what_cam_lock: No such file or directory
ls: *.mlv: No such file or directory
ls: *.mlv: No such file or directory
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
/tmp/DUALISO/DUALISO
cat: /tmp/_X_Proxy: No such file or directory
cat: /tmp/_X_Proxy: No such file or directory
cat: /tmp/_X_ProRes4444: No such file or directory
cat: /tmp/DUALISO/FLATFRAMES: No such file or directory
cat: /tmp/DUALISO/FLATFRAMES: No such file or directory
[ERROR] Failed to open file ''
cat: /tmp/DUALISO/FLATFRAMES: No such file or directory
cat: /tmp/DUALISO/FLATFRAMES: No such file or directory
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: ''
- Rewrite MLV
- Output only one frame with averaged pixel values
- Output into 'ft__'
[ERROR] Failed to open file ''
rm: *_.wav: No such file or directory
cat: /tmp/DUALISO/FLATFRAMES: No such file or directory
mv: : No such file or directory
ls: *.mlv: No such file or directory
[ERROR] Failed to open file 'ft_*.MLV'
mv: rename avg_*.MLV to A_ORIGINALS/avg_*.MLV: No such file or directory
mv: rename ft_*.MLV to A_ORIGINALS/ft_*.MLV: No such file or directory
ls: *.mlv: No such file or directory
ls: *.mlv: No such file or directory
cat: /tmp/DUALISO/badpixelMLVab: No such file or directory
cat: /tmp/DUALISO/badpixelMLVac: No such file or directory
cat: /tmp/DUALISO/badpixelMLVad: No such file or directory
5D Mark
953A9793.MLV
ls: *.mlv: No such file or directory
ls: /tmp/DUALISO/mlv_dump_TIME: No such file or directory
ls: *.mlv: No such file or directory
cat: /tmp/DUALISO/MLVFILESab: No such file or directory
cat: /tmp/DUALISO/MLVFILESac: No such file or directory
cat: /tmp/DUALISO/MLVFILESad: No such file or directory
cat: /tmp/output: No such file or directory
cat: /tmp/output: No such file or directory
mv: rename 953A9793.M* to 953A9793_1_2017-08-27_0001_C0000/953A9793.M*: No such file or directory
mv: rename 953A9793.WB to 953A9793_1_2017-08-27_0001_C0000/953A9793.WB: No such file or directory
ls: *953A9793.map: No such file or directory
ls: *.xls: No such file or directory
cat: /tmp/mlv_dump_settings: No such file or directory
cat: /tmp/output: No such file or directory
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: '953A9793.MLV'
- Decompressing before writing DNG
- Enforcing 14bpp for DNG output
- Convert to DNG frames
- Output into '953A9793_1_2017-08-27_0001_C0000_'
File 953A9793.MLV opened
File 953A9793.M00 not existing.
Processing...
Vertical stripes correction:
1.000 1.000 1.000 0.999 0.999 1.002 1.001 1.005
Frame0 : cold pixels found: 0
Reached end of chunk 1/1 after 328 blocks
Processed 315 video frames at 25.00 FPS (12.60 s)
Done
ffmpeg version 3.3 Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
configuration: --prefix=/Volumes/tempdisk/sw --as=yasm --enable-gpl --enable-version3 --enable-pthreads --disable-ffplay --disable-ffserver --disable-shared --enable-static --enable-libvpx --disable-decoder=libvpx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libx265 --enable-libxvid --enable-zlib --enable-avfilter --enable-fontconfig --enable-libfreetype --enable-libass --enable-libvidstab --enable-libsnappy --enable-filters --enable-postproc --enable-runtime-cpudetect --disable-indev=qtkit --disable-indev=x11grab_xcb
libavutil 55. 58.100 / 55. 58.100
libavcodec 57. 89.100 / 57. 89.100
libavformat 57. 71.100 / 57. 71.100
libavdevice 57. 6.100 / 57. 6.100
libavfilter 6. 82.100 / 6. 82.100
libswscale 4. 6.100 / 4. 6.100
libswresample 2. 7.100 / 2. 7.100
libpostproc 54. 5.100 / 54. 5.100
Invalid duration specification for ss: -i
ffmpeg version 3.3 Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
configuration: --prefix=/Volumes/tempdisk/sw --as=yasm --enable-gpl --enable-version3 --enable-pthreads --disable-ffplay --disable-ffserver --disable-shared --enable-static --enable-libvpx --disable-decoder=libvpx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libx265 --enable-libxvid --enable-zlib --enable-avfilter --enable-fontconfig --enable-libfreetype --enable-libass --enable-libvidstab --enable-libsnappy --enable-filters --enable-postproc --enable-runtime-cpudetect --disable-indev=qtkit --disable-indev=x11grab_xcb
libavutil 55. 58.100 / 55. 58.100
libavcodec 57. 89.100 / 57. 89.100
libavformat 57. 71.100 / 57. 71.100
libavdevice 57. 6.100 / 57. 6.100
libavfilter 6. 82.100 / 6. 82.100
libswscale 4. 6.100 / 4. 6.100
libswresample 2. 7.100 / 2. 7.100
libpostproc 54. 5.100 / 54. 5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '953A9793.MOV':
Metadata:
major_brand : qt
minor_version : 512
compatible_brands: qt
encoder : Lavf57.71.100
Duration: 00:00:13.21, start: 0.000000, bitrate: 30078 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 28550 kb/s, 25 fps, 25 tbr, 25k tbn, 50k tbc (default)
Metadata:
handler_name : DataHandler
timecode : 17:54:07:22
Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)
Metadata:
handler_name : DataHandler
Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s
Metadata:
handler_name : DataHandler
timecode : 17:54:07:22
Output #0, mov, to 'n953A9793.MOV':
Metadata:
major_brand : qt
minor_version : 512
compatible_brands: qt
encoder : Lavf57.71.100
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 28550 kb/s, 25 fps, 25 tbr, 25k tbn, 25k tbc (default)
Metadata:
handler_name : DataHandler
timecode : 17:54:07:22
Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)
Metadata:
handler_name : DataHandler
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 315 fps=0.0 q=-1.0 Lsize= 46407kB time=00:00:12.58 bitrate=30210.0kbits/s speed= 247x
video:44034kB audio:2364kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.017785%
cat: /tmp/output: No such file or directory
ffmpeg version 3.3 Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
configuration: --prefix=/Volumes/tempdisk/sw --as=yasm --enable-gpl --enable-version3 --enable-pthreads --disable-ffplay --disable-ffserver --disable-shared --enable-static --enable-libvpx --disable-decoder=libvpx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libx265 --enable-libxvid --enable-zlib --enable-avfilter --enable-fontconfig --enable-libfreetype --enable-libass --enable-libvidstab --enable-libsnappy --enable-filters --enable-postproc --enable-runtime-cpudetect --disable-indev=qtkit --disable-indev=x11grab_xcb
libavutil 55. 58.100 / 55. 58.100
libavcodec 57. 89.100 / 57. 89.100
libavformat 57. 71.100 / 57. 71.100
libavdevice 57. 6.100 / 57. 6.100
libavfilter 6. 82.100 / 6. 82.100
libswscale 4. 6.100 / 4. 6.100
libswresample 2. 7.100 / 2. 7.100
libpostproc 54. 5.100 / 54. 5.100
953A9793_1_2017-08-27_0001_C0000_.wav: No such file or directory
rm: 953A9793_1_2017-08-27_0001_C0000_.wav: No such file or directory
2017-09-01 11:41:36 953A9793_1_2017-08-27_0001_C0000.wav: File does not exist
cat: /tmp/output: No such file or directory
mkdir: : No such file or directory
cat: /tmp/output: No such file or directory
mv: rename 953A9793.M* to /Users/thunderbook/Desktop/sync/black/cut/A_ORIGINALS/953A9793.M*: No such file or directory
cat: /tmp/A_cr2hdr_settings.txt: No such file or directory
cr2hdr: a post processing tool for Dual ISO images
Last update: ab1e90c on 2015-11-24 09:50:19 UTC by a1ex:
cr2hdr: moved safeguard from median_int_wirth to kth_smallest_i...
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
Input file : 953A9793_1_2017-08-27_0001_C0000_000000.dng
Camera : Canon EOS 5D Mark III
Full size : 1920 x 1080
Active area : 1920 x 1080
Doesn't look like interlaced ISO
Doesn't look like interlaced ISO
[ERROR] Failed to open file '/Users/thunderbook/Desktop/sync/black/cut/953A9793.MLV'
cat: /tmp/output: No such file or directory
ls: *.RAW: No such file or directory
ls: *.RAW: No such file or directory
ls: /tmp/DUALISO/FULL_AUTO: No such file or directory
wc: _1_2017-09-01_0001_C0000/pix_fix.txt: open: No such file or directory
wc: _1_2017-09-01_0001_C0000/pix_fix.txt: open: No such file or directory
cat: /tmp/FFmpeg_settingsPR: No such file or directory
cat: /tmp/FFmpeg_settingsPR: No such file or directory
cat: /tmp/FFmpeg_settings: No such file or directory
ls: *.CR2: No such file or directory
ls: /tmp/DUALISO/FULL_AUTO: No such file or directory
ls: *.CR2: No such file or directory
ls: *.DNG: No such file or directory
ls: *.dng: No such file or directory
[8;16;35t[3;955;0tNo matching processes belonging to you were found
[H[2JTotal running time:
[1m[30m0 Days, 00 Hrs, 00 Min, 11 Sec(B[m
X to FFmpeg ProRes:
[1m[30mXxXx(B[m
mlv_dump:
[1m[30m0 Days, 00 Hrs, 00 Min, 07 Sec(B[m
raw2dng:
[1m[30mXxXx(B[m
dcraw_FFmpeg ProRes:
[1m[30mXxXx(B[m
cr2hdr(CR2 files):
[1m[30mXxXx(B[m
[1m[31m(q) Exit(B[m
You didn´t try another proxy file with corresponding MLV?
If still trying the same file I need to check the MOV you´re using. It still has the black frames when included with the MLV? Can you upload the exact MOV you´re using?
Yes I did another test with another video and its fine, I don't know whats wrong with the previous video.
Ah I found it, sorry I was using the .mov file with already had the black frames cut of.
User error.
Could you upload it? As asked before. Are you using the prores file?(there is no audio in it)
Are you using a file that has been "cleaned" already?
I uploaded a new version for any rare occasion when there is no black frames in the beginning of a file. Probably never happwns but still...
Ok, good saw your answer. You can try this latest version and see if it works on a cleaned file.
Yes the latest version works also with the already cut files, Thanks for the update
And if you can have a look at not changing the source files directory that would be good for my usage.
I always get paranoid if my footage gets moved around and I don't know where it ends up.
this makes me do a backup first that takes more space, so I prefer it to be untouched and only create new files in subfolders / custom folder
I understand. It could be done like this.
MLV folder is generated and both MLV and MOV are moved inside this folder. When processing is done the cleaned proxy is moved to a proxy folder, MOV original goes back to origin place and MLV is moved to A_ORIGINALS.
You could also use menu setting (C) and assign a new output folder. This way both MOV and MLV stay untouched in its origin folder.
Ok custom output folder is good enough for how I will use it (C)
It does make a temp folder originals but it deletes it after, seems like a redundant step?
Thanks
Yes, it's more like a safety step since there could be files wandering in and out of that folder. Hard to keep track of all processrs nowadays :). If it's empty when all processing is done it is removed.
Danne, it's working now, thanks. I tried a test in Davinci but the conformation isn't working. The CDNG's timecode is reset to 00:00:00:00 and the .mov and cdng don't have the exact same name (aditionnal count for DNG in the name). The conformation can work if the two sources share same reelname + same timecode. Do you think we can add .mov timecode, add reelname and copy it to CDNG ?
Oh my, it never ends ;D.
I'll take a look. Stay tuned.
Do you by any chance have any links to workflows which includes linking proxies like you describe? I never do this myself.
Regarding reelname. Please print out an example how your files looks now after processed in Switch and then how they should be named to work with resolve.
Quote from: Danne on September 01, 2017, 04:55:37 PM
Do you by any chance have any links to workflows which includes linking proxies like you describe? I never do this myself.
Regarding reelname. Please print out an example how your files looks now after processed in Switch and then how they should be named to work with resolve.
Ok Danne, it seems more easier than I expected. I have a successfull test connecting a ProRes and CDNG (from Black magic camera). The two clips only need to match the same timecode. No reelname to add, and no problem with DNG source clip name.
My source file from canon proxy mov had a TC : 19:44:38:03
The DNG processed in Switch had a TC : 00:00:00:00
There are two solutions to get a successfull conformation : we can copy the mov proxy TC to the DNG during processing. Or reset TC from canon proxy mov to 00:00:00:00 during the black frame removing process.
To make a conformation test, here is the workflow :
1) one folder containing MLV + mov proxy
2) run switch to get new files w/o black frame
3) open free demo version of FCPX. File => Import proxy file, drag drop in the timeline. Make some cuts, trim and change order of the clip
to make a sort of film edit
4) Go to file => export XML (whatever version)
5) Open resolve 14 (free)
6) import DNG
7)File => import XML
8 when there is the pop up menu = untick "automatically import source clips into media pool"
9) if resolve relinks to the DNG and the timeline is similar to what you did in FCPX, it's matching.
Note : we can then replicate the workflow on any NLE (open source, Premiere, Avid...) using XML. Maybe AAF can work too.
If you need a screencast, I can provide one
Are you actually getting synched MOV files with cdng files in resolve from following those steps described above? If you edit stuff in fcpx and export your xml, how can resolve now where to link?
Maybe a screenrecord will help :).
I will be busy today but as soon as I have time I will experiment with this. If anybody else wants to get their hands dirty feel free to fire up exiftool, ffmpeg, exiv2 or whatever needed to modify and get a correct relinking...
You can try to modify first and last time code tag in the cdng sequence to have the same start and ending like in the MOV file. Install exiftool and do for instance this:
exiftool "-TimeCodes=17:54:07.22" 953A9793_1_2017-08-27_0001_C0000_000000.dng
Last dng:
exiftool "-TimeCodes=17:54:20.15" 953A9793_1_2017-08-27_0001_C0000_000306.dng
This is just my example based on wht the MOV is telling my in resolve. Have no idea how it works.
New version.
- Added timecode start of 00:00:00:00 in MOV file to match the dng time code in resolve(needs testing)
- Some refactoring
commit:
https://bitbucket.org/Dannephoto/switch/commits/e926c347e5b7fd72d6cd2d7702e2fc7fedab3d68
All recent MOV file seems to match in and out timecode. All files but the testfiles with the leaves from Lars Steenhoff. Good file to have.
Hi Danne, I just arrived in bangalore so I'll test it on my new footage tonight. Thanks !
Envoyé de mon iPhone en utilisant Tapatalk
Danne, I made successfull test in Resolve! I tried both with the CDNG generated by switch and the CDNG from MLVFS (this one without sound of course). TC + name are matching. So now, we can edit offline with the H264 ML proxy in any NLE, export XML to send in/out cut info and relink in Resolve. In Resolve, you just have to import the DNG and after that to import the XML and untick import media (which import the proxy...).
To save more space and time, if you add an option to only reset the TC of the MOV proxy (without generating CDNG in the same time), we could edit and just transcode the necessary clips in DNG at the end (it's called a "consolidate").
I think it is a solid step forward for indie production !
Cool!
Generate only proxy with corrected tc and cleaned from black frames without developing MLV could be an option in the menu. We still need the corresponding MLV files though to achieve the correct end frame.
Quote from: Danne on September 02, 2017, 05:52:40 PM
Cool!
Generate only proxy with corrected tc and cleaned from black frames without developing MLV could be an option in the menu. We still need the corresponding MLV files though to achieve the correct end frame.
Yes, of course we need MLV. This option Would be Nice.
By the way, is there a tool in switch that can remove hot pixels from h264 proxys ? I also noticed pink hightlight with 10/12 bit
Envoyé de mon iPhone en utilisant Tapatalk
Here´s an example using the delogo filter in ffmpeg.
http://www.madpanic.tv/blog/files/fixing-dead-pixels-with-ffmpeg.html
Using this is probably meaning you need to reencode the file itself and then the much better proxy solution would be to create proxys inside Switch. That will also cure any highlight issues.
Excellent article ! However, as you said, it means reencoding, so it's better to edit straight with H264 proxy with dead pixels or making proxies from MLV in Switch. Thanks anyway !
New version
- added support for creating "cleaned" proxies only without processing of MLV files(multithreaded of course 8))
You find the setting in both (m) mlv_dump and (ms) mlv_dump_on_steroids menus at the bottom. If not selected proxy+MLV processing works as before.
(https://s26.postimg.org/rj0kl9jvt/Screen_Shot_2017-09-03_at_18.54.45.png)
- refined black detection with two more thresholds
Very simple:
first_black=$(ffmpeg -i *"$MOV" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
if [ x"$first_black" = x ]; then
first_black=$(ffmpeg -i *"$MOV" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.04" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
if [ x"$first_black" = x ]; then
first_black=$(ffmpeg -i *"$MOV" -to $snippet -vf "blackdetect=d=0.1:pix_th=0.08" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
fi
fi
commit:
https://bitbucket.org/Dannephoto/switch/commits/e0a8ad7a67468ebb3f32a6bfd19876be4a4692a5
beautiful ! I'll test it !
I made some average calculations for 10 hours of footage :
rec. 1080p23.976 MLV raw (not comp) + CDNG transcode = 5.6TB av.
rec. 1080p23.976 MLV raw comp + H264 proxys = 1.940TB av.
Nice.
Could you check for this issue? MLV_lite is broken in the middle of the file in later versions of crop_rec_4k. Some metadata missing.
http://www.magiclantern.fm/forum/index.php?topic=19300.msg189583#msg189583
Also if things are actually working, what build are you using? I think it affects longer takes around more than 2 or 3 minutes long.
I found out my version of mlv_dump has been buggy. Fixed it and uploaded a new version.
Tested new version with proxy only mode and it works perfectly.
for storage space : 10hrs footage MLV comp + proxy H264 + proxy H264 black removed = 2.1TB av.
I think it's good to keep original proxy.
Great. Thanks for testing with bulk footage.
QuoteI think it's good to keep original proxy.
All originals are kept into A_ORIGINALS right?
Quote from: Danne on September 04, 2017, 09:48:41 AM
Great. Thanks for testing with bulk footage.All originals are kept into A_ORIGINALS right?
Yes, I advice people not to delete content in A_ORIGINALS
Doubt: why does the counter go beyong 100% when converting only one file? it seems like 100 represents only one chunck of the whole video, a single file, if this is the case is there a way to have it fixed somehow? not that important just a thing :)
QuoteDoubt: why does the counter go beyong 100% when converting only one file?
Well, the reason for the inaccuracy is actually partly because of your reports a few months ago. We tested two scenarios of which one was slowing down MLV batches several minutes because of the frame accuracy issue.
It´s either:
for FILE in `ls -A1 *.MLV *.mlv | grep -v 'avg_\|ft_' 2>/dev/null`; do
$mlv_dump -v "$FILE" | awk '/Processed/ { print $2; }' >> /tmp/DUALISO/MLVprogress_bar3
doneor faster but less accurate(but still pretty accurate)
for FILE in `ls -A1 *.MLV *.mlv | grep -v 'avg_\|ft_' 2>/dev/null`; do
$mlv_dump -v "$FILE" | awk '/Frames/ { print $3; exit}' >> /tmp/DUALISO/MLVprogress_bar3
doneGrabbing the "Processed" number requires all files processed before giving you the more accurate frame number. This will severely slow down overall processing. Fetching "Frames" however does not since it´s already in the header metadata. Why the inaccuracy? My guess is that the frame number tag is an estimate depending on size of the MLV file and aspect ratio recorded. Probably also depending on if recorded lossless/10/12/14bit as well. So speed is more important than accuracy in this case imo. Still nice to see that there even was "Frames" tag in there...
oooh! right, yea absolutelly. speed is more important.
New version uploaded.
Got word from dfort who´s tested a 30 minute MLV with included proxy MOV file. MOV files are of course cut in 4gb chunks so I added support for this in Switch. It will cat the files and then run as before.
I noticed that all MLV files are treated individually. Even .M00, .M01 etc so this made the percent counter count way above 100% before since I missed to include the frames for .M00, .M01 etc. This is hopefully fixed. Maybe @Togg could have a go here.
Also added some more codec settings rather than only ProRes output in the ProRes menu.
Commits:
https://bitbucket.org/Dannephoto/switch/commits/59905ecc1368a8ea2a29417397a2c36aaa355b22
https://bitbucket.org/Dannephoto/switch/commits/dc648a19d878b27a59bcabc9911bd8c0ffdea0c6
https://bitbucket.org/Dannephoto/switch/commits/ee3dfa54f36600beb257802706c098c4fdcba9f9
New version.
Been testing refining under the radar. Mostly been feedbacked from dfort. Firstly I fixed proxy cleaning with all kinds of scenarious, short, long, different prefixes and spanned files. Another issue was that the cleaning scripts would consistently cut a few frames too many at the end on longer takes. The solution here seems to be to use -vframes rather than the other math I was doing before. A1ex pointed this out a while ago but I didn´t put this to some real tests until today when feedbacked from dfort.
Commits here:
https://bitbucket.org/Dannephoto/switch/commits/all
tried the one linked on the first page. went really fast i got pretty excited. looked into my dng folders and nothing in there. switched back to previous version of switch (pulled it out of the trash) and that one worked fine dngs back in the folders all good. so i don't know might be broken.
Do you have the LOG file anywhere?
What camera are you using, how are the files recorded? What are your settings when processing?
Just tested over here and couldn´t reproduce with files from a 5D mark III. Eosm files seems ok as well.
Speed increase will mostly affect spanned MLV files and only when checking metadata information in the very start of the program.
heres a log and link to it https://drive.google.com/file/d/0B_4dpBYObDvENnd2V2VCY3ZOaFU/view?usp=sharing (https://drive.google.com/file/d/0B_4dpBYObDvENnd2V2VCY3ZOaFU/view?usp=sharing)
Strange. Can you send me a samle of the MLV file? You can create samples from the mlv_dump menu (17) create a sample files package
For those of us on an openzfs filesystem on OS X (am I the only one?), Switch (as did cr2hdr before it) produces prores files with all the frames running in random order. This is pretty cool to look at, but is definitely not as useful for normal purposes as standard sequential output. This can be fixed by simply adding a '-s' flag to the 'find' commands in the FFmpeg_produce_0*.command files. All this flag does is make explicit that find returns files in 'lexicographical order', which appears to also mean numerical order.
Here's an example of one of the invocations of 'find' from those files, with the '-s' flag inserted:
Quote#export ProRes4444
find -s "$out""$out3""$name""$sl". -maxdepth 1 -iname '*.dng' -print0 | xargs -0 dcraw +M $h2 $o $S \
-c -6 -W -q 3 $gam $wb $pix $br | ffmpeg -loglevel warning $wav1 -f image2pipe -vcodec ppm -r $fps -i pi\
pe:0 $sd -vcodec prores_ks -pix_fmt yuv444p10 -n -r $(echo $fps / 2 | bc -l) $tbl$cin$cin_01$cin_02$cin_\
03$cin_01b$cin_02b$cin_03b$scale$denoise$sharpen "$out""$out2""$date"_ProRes4444/"$name".mov
I've done the search and replace on all instances of 'find "' to change them to 'find -s "' in my local copies of FFmpeg_produce_0*.command, and now my prores output is watchable. I'd suggest this as a fix for the main branch of Switch but I don't know if the '-s' argument works in non-OSX versions of 'find'.
Great find @zachnfine!
Quote from: zachnfine on September 19, 2017, 04:37:18 AM
I'd suggest this as a fix for the main branch of Switch but I don't know if the '-s' argument works in non-OSX versions of 'find'.
You know you could submit a pull request in Bitbucket so anyone who wants to test out your suggestions can easily do so and give feedback. Though knowing Danne he is probably already trying out the '-s' flag.
Gonna check. It's actually the second time someone a user encounter this. Last time was in MLP, former project I worked upon. Oh, it´s the same user :). Welcome to Switch then.
i tried to make a sample. https://drive.google.com/file/d/0B_4dpBYObDvETUtRSHZxVWVJZGs/view?usp=sharing (https://drive.google.com/file/d/0B_4dpBYObDvETUtRSHZxVWVJZGs/view?usp=sharing)
Processing of that sample works just fine over here with the latest version. I suggest you try to process it one more time. If it doesn´t work try restarting your computer and run it again, and then report back.
Thanks for testing.
Quote from: dfort on September 19, 2017, 06:37:39 AM
You know you could submit a pull request in Bitbucket so anyone who wants to test out your suggestions can easily do so and give feedback. Though knowing Danne he is probably already trying out the '-s' flag.
Ah, interesting. I figured there was some sort of standard procedure. Have never used bitbucket.
On a different note -- I think the "-s" flag for sort order is a feature of BSD find but not of GNU find. So "-s" will work for OS X and BSD varietals, but likely not for Cygwin or Linux.
According to the man page '-s' tells 'find' to traverse the directories in lexicographical order. It appears that this just happens by default without the flag present with HFS+ filesystems, but OpenZFS filesystems are an exception. I suppose there might be others.
maybe its because I'm on el cap...
New upload
HDR CR2 automation
ML has a really good HDR bracketing feature. Even canons own bracket function works really well. There is also a really nice open source project called HDRmerge, which can output merged dng files straight from CR2 without any need for intermediate tiffs for instance:
http://jcelaya.github.io/hdrmerge/
For this to work you need to install HDRmerge into the applications folder.
Usage:
Fill a folder with bracketed CR2 files and Switch will group and then use HDRmerge to output merged dng files. Switch uses exiftool to find the time gap between the brackets, sends them to a list then process to dng files multithreaded.
In main menu select (h) HDR processing(CR2)
Then
(https://s26.postimg.org/4w88o6855/Screen_Shot_2017-09-25_at_21.25.03.png)
Progress bar while grouping CR2 brackets
(https://s26.postimg.org/69zvjh7eh/Screen_Shot_2017-09-25_at_21.25.26.png)
HDRmerge processing the brackets
(https://s26.postimg.org/7b048loe1/Screen_Shot_2017-09-25_at_21.25.57.png)
Thanks @Danne for reviving HDR back once again and this time it is definitely a game changer when it comes to this HDRmerge automation. Can't wait to try it out! :D
I've been using Switch to process MLV files for a documentary project and came up with an issue that really isn't a problem with the app but it is something that people shooting mlv_lite with H.264 proxies should be aware of.
Out of four days of shooting that amounted to a few hundred clips and several hours (estimated about 300 shots, 4 hours) there were only two clips where the length of the raw footage didn't match up with the trimmed H.264 proxy. Being the perfectionist that he is Danne tried his best but the issue isn't with Switch--it turned out that sometimes the H.264 recording stops before the MLV. One short clip was off by 1 frame and the other, an 8 minute shot that spanned two H.264 files was off by 2 frames.
The way that H.264 proxies are recorded they are black at the head and tail. Switch can create new proxies to the same length as the raw video. I learned that Switch uses the black at the head of the proxy and checks the length of the raw file to make sure the lengths match perfectly. Of course if the H.264 cuts out early there's not much Switch can do about it.
Ok--so there's a slight chance of having a H.264 proxy short by 1 or 2 frames but shooting MLV with H.264 proxies is probably the most practical way to shoot. At least if you're shooting regular 1920x1080 with a 5D3. I doubt this will work with other frame sizes or other cameras. Here are the advantages:
1 - You only need to activate the mlv_lite module. No need for mlv_snd to record audio or mlv_play (raw_tweak?) -- you can't playback in real time in camera with that anyway. However, you can play back the H.264 proxies in the camera just fine which is very helpful in the field.
2 - You can start editing before the MLV files are processed. There's an option in Switch that allows you to trim the H.264 proxies without having to process the MLV files. While not as fast as simply copying the files off the card, it does cut down the wait time between shooting and editing. As an example, on a day that I shot 51 clips totaling 31.5 minutes it took only 17.5 minutes to trim the H.264 proxies compared to processing the MLV files to DNGs which took 4 hours. Yes, you can also use MLVFS or whatever other post workflow you want, just remember that the audio must be extracted from the H.264 proxies and Switch does that for you.
3 - Recording on two cards simultaneously. On the 5D3, which is the only camera this workflow is practical, can be setup to record H.264 on the SD card and MLV on the CF card. I used 128GB cards so the SD card had ML loaded and I didn't lose any setting because I could shoot all day without having to swap out that card. The CF cards could hold about 45 minutes of 14-bit lossless but I would switch out CF cards frequently just in case something get corrupted. Glad to say it has never happened.
Here's the best part about shooting this way--my partner in this project is comfortable working with H.264 and isn't quite convinced that Magic Lantern is worth all the extra work. This way we can have it both ways. At the end of the post we have the option of color grading the raw video files.
Ok, there are drawback too. My biggest gripe is that LiveView goes dark until it starts recording. That makes it hard to see if your shot is lined up on a run and gun shot and I missed several shots because it does seem to take longer before recording starts, about 4 seconds or so.
One other point worth considering when shooting lossless compression. If the scene has a lot of contrasty detail and you open up the aperture (like to check focus) it could overwhelm the compression. The console opens, shows a bunch of error messages and ML saves crash logs. Restarting clears things up but if you are having problems simply switch off the compression. Yeah, the files will take up more room but the CF card has no problems keeping up shooting uncompressed 14-bit 1920x1080 23.976 FPS.
Good summary Dfort. I think this workflow is necessary for all the cameras if it's possible with lower bitrates. This is the best way to preview and to edit straight off the camera.
Very nice follow up Dan.
12georgiadis, you should download the latest version. Tweaked thresholds a bit...
Ok, thank you Danne, I'm gonna test it soon
This is very useful practical information, Dfort! Thank you.
I have a question to Danne: Is it possible to compile Switch for Windows x64 too?
Hi IDA_ML. I started porting most of this stuff to linux subsystem for windows. Do you have the possibility for this on you Windows system? Another solution for proxy matching is to continue working with batch_mlv and put it in there.
It would be really nice if we could make a nice GUI for this workflow.
even better totally automatic use the new pixel stuff bouncy ball is working on and just have it batch mlv to dng. no options just works. thats my dream. But seriously thank you this is the only software that works well for me and i'm always using it.
@Lars Steenhoff
GUI, always wanted to do this. Then again, I can´t find a way building one to fit bash scripts. The path right now is to instead focus on keeping everything updated and multi processing. A lot of functions implemented. Will take a while porting it to a gui.
@Teamsleepkid
It´s on my todo list putting in Bouncyball´s latest mlv_dump version both in Switch and batch_mlv. Even so, right now Switch is using Bouncyball´s fpmutil, a side tool doing exactly what latest mlv_dump_on steroids is doing. Just select (ms) in main menu and you´ll be using mlv_dump_on_steroids...
I'm not an expert but it seem Q is used for apps that are cross platform
https://en.wikipedia.org/wiki/Qt_(software)
Maybe this can be an option?
I guess its a lot of work
https://www.cocoawithlove.com/2010/11/cocoa-application-for-running-scripts.html
Or in Cocoa?
Thanks. Been through those links but never had the time to dig in. If you have some knowledge in this field I'd be happy to assist you explaining what's going on in Switch. Could start porting mlv_dump for instance.
Quote from: Danne on October 11, 2017, 09:15:42 PM
Thanks. Been through those links but never had the time to dig in. If you have some knowledge in this field I'd be happy to assist you explaining what's going on in Switch. Could start porting mlv_dump for instance.
I'd be happy to help contribute this project in getting @Danne's scripting work into a GUI wrapper.
I'm with you on this one @Lars Steenhoff!
Video
https://www.youtube.com/watch?v=pJwvPtD4vRI
Slides
http://macadmins.psu.edu/files/2017/07/psumac2017-141-Bringing-bash-to-Xcode-1kc5g1b.pdf
I found a simple tutorial for adding bash scripts in Xcode
I don't wan't spam the thread with GUI stuff, I just think this might be a good way to start working on in Xcode.
@DeafEyeJedi Do you have Xcode installed?
@Danne Do you think you could set up a Switch GUI repo in git where we can work on an Xcode project together.
Wow, looks good. Thanks. Don´t worry about spamming. Just put in any info you find on this.
I create a repository on bitbucket right? Same as git? Will check some more tomorrow, but seems interesting enough.
Nice!
We will set up a repo for hosting the project and we can start connecting scripts and adding layout ideas.
The only thing we need to find out is if we can interface with the scripts from Xcode.
I'm playing around in Xcode, following the tutorial.
(http://www.larssteenhoff.com/forum_pics/switchGUItest2.png)
What a great start already. Loving how the foundation layout looks from Xcode! 8)
Will need to read up some more on the tutorial. Thanks for jump starting on this @Lars Steenhoff!
How about setting up the repository on Bitbucket. That's where Magic Lantern and Switch live. Makes it easier when submitting pull requests.
https://bitbucket.org/hudson/magic-lantern
https://bitbucket.org/Dannephoto/switch
that looks seriously nice. easy to use. really great.
@Dfort. Yes on bitbucket
@Danne you can set up a repo on bitbucket and I will make a commit
Switch_GUI project
https://bitbucket.org/Dannephoto/switch_gui/overview
(https://s1.postimg.org/6o7cj3bqlb/Screen_Shot_2017-10-12_at_09.28.13.png)
Here we go...
Initiated by Lars Steenhoff
http://www.magiclantern.fm/forum/index.php?topic=15108.msg191535#msg191535
I created a project in mercurial. Will this work Lars? I also added you to the project. You should be able to put up stuff immediately. Please have a look.
You seem to have gotten fast into this workflow around xcode and it already looks very promising. Feel free to start off any way you like. Personally I would start my focus on building the functions for mlv_dump and get those working before adding other stuff but that´s only my opinion.
Sidenote:
Nothing against cinelog but it is not included in Switch which is gpl open source based.
Yes great, I will add the xcode project.
I was just playing with the Gui elements in xcode, Your right, now its time to setup a first real case.
As for cinelog, I would imagine we can support a user folder with luts. So we don't include cinelog in the build.
Nice.
I include some logs inside Switch. They are converted to 3D luts which in no way is optimal for 1D logs. Unfortunately ffmpeg can´t work with 1D luts so it´s a workaround.
Looking forward to whatever pops up into this project. Will help and dig in as much as I can...
I created a branch develop and added the code project there
You can open the project in Xcode with Switch.xcodeproj
and press play to see the first version.
Right now it's just the GUI layout and the app icon.
The layout is in progress.
The script AppDelegate.applescript is the one that can interact with the bash scripts.
Great Lars. Inspiring. Like your SwitchIcon. Will dive in tonight. Working...
Indeed it is inspiring. Here's how it looks running Switch.xcodeproj via Xcode for those who are curious.
http://vimeo.com/237937004
Good work! looks really functional and pretty
I like GUIs :D
DeafEyeJedi,
I cannot see it. It's an embedded video, isn't it? Could you please post a link to it? Thank you.
It´s a work in progress. Started today:
http://www.magiclantern.fm/forum/index.php?topic=15108.msg191556#msg191556
Thank you so much, Danne! I will be looking forward to a Win x64 version. I wish I could help but I have no coding skills whatsoever!
Keep up the good work, guys!
While I also am a big fan of GUIs, I would like to pose this observational warning.
GUI Builders, in what I would guess to be an attempt to make things Neat & Tidy, can
end up Hiding Functionality Behind Doors/Pull Downs.
Even though I'm Not a savvy Terminal User, I Really Like the Simplicity of Switches
present iteration in that all the Major Functionality Choices are Upfront & Visible in the
Main Menu. Thus I'm hoping that the GUI will contain a similar Main Menu.
Huge applauds for this effort.
Great to hear Olivrat. My exact intention with Switch.
With that said I welcome the initiative to build a nice gui here and who knows. Might be even faster than terminal ;)
Just saw of the GUI, very nice. Will be easier to spot what doesn't make sense flow wise. Within this new GUI vibe maybe it should be fixed the ambiguity about "vertical stripes" etc etc
Will the GUI be Put Up in the 1st Post when ready for use?
We just managed to execute Switch main command from xcode gui. That doesn't mean it's ready by no means. But it works. When there is something useful I put it out. Meanwhile it would be interesting to know what is considered more useful in Switch? There is quite a lot of processes to choose from...
For me
1 Proxy workflow that cuts the black frames from the h264 is very useful. ( and adds WAV to the dng folder )
2 Getting the best mlv to dng conversion. ( black frame averaging etc)
Agreed Lars. Let's stick to those features and skip the rest. At least for now.
Really looking forward to this switch app with a GUI, Tried using the normal switch app, but could never really figure it out (user error) lol. Great work Danne and Co.
Switch now skips the intermediate step asking for crop_rec when running footage from cameras with focus pixels. That is, it must find the RAWC metadata or else you'll get the queries again.
Thanks to A1ex and g3gg0. RAWC has been in there for a while...
thats really great. nice move
More like a great save by @Danne!
Kudos out to @a1ex & @g3gg0!
I have a problem running Switch. I'm running it on macOS Sierra, installed by just moving it to /Applications as directed. As far as I can tell from the original post on the thread, there are no othe
Running it, I get the pop-up window asking for the source folder, as expected. then this:
ls: /tmp/DUALISO/crop_rec?: No such file or directory
ls: /tmp/DUALISO/RAW_demolish: No such file or directory
ls: /tmp/DUALISO/what_cam_lock: No such file or directory
/Applications/Switch.app/Contents/Menu.command: line 1976: syntax error near unexpected token `>'
/Applications/Switch.app/Contents/Menu.command: line 1976: ` exec &> >(tee -a "$(cat /tmp/DUALISO/samples | head -1 | cut -d "." -f1)"_samples/"$(cat /tmp/DUALISO/samples | head -1 | cut -d "." -f1)"_LOG.txt >&2 )'
[Process completed]
...as well as a forever-spinning gear icon in the menubar, displaying as 80% completed.
Needless to say, there is no crop_rec?, RAW_demolish, what_cam_lock or samples in /tmp/DUALISO/.
What there is, is:
ls -l /tmp/DUALISO
total 32
-rw-r--r-- 1 sacha wheel 26 11 Nov 12:21 DBG_path
-rw-r--r-- 1 sacha wheel 1 11 Nov 12:21 DUALISO
-rw-r--r-- 1 sacha wheel 0 11 Nov 12:21 MOD
-rw-r--r-- 1 sacha wheel 0 11 Nov 12:21 fpss
-rw-r--r-- 1 sacha wheel 0 11 Nov 12:21 image_size
-rw-r--r-- 1 sacha wheel 32 11 Nov 12:21 path_1
-rw-r--r-- 1 sacha wheel 35 11 Nov 12:21 path_2
-rw-r--r-- 1 sacha wheel 0 11 Nov 12:21 what_cam.txt
To be honest, I haven't tried to debug the Menu.command file, but I ask if anyone has had this problem, before I try to read through the file looking for where it fails.
Thanks.
Thanks for posting about your issue. Does the problem occur when double clicking Switch? What happens if you drag a folder onto Switch?
You can place Switch wherever. Not necessary to run it from applications folder. I should change that description part.
You can also try erasing the tee command which seems to be causing this issue. I think it can be fixed with another syntax. Had this issue when trying it in a gui project.
By the way. Is your terminal set working bash per default? Could be related to this as well. If erasing the tee command fixes it the issue will probably occur in two more places. We'll see.
The menu.command is located inside contents folder within Switch.
Hi Danne,
It's the exact same error if I drag into Switch.
Commenting out the line brings me to another error further down (another tee command):
/Applications/Switch.app/Contents/Menu.command ; exit;
ls: /tmp/DUALISO/crop_rec?: No such file or directory
ls: /tmp/DUALISO/RAW_demolish: No such file or directory
ls: /tmp/DUALISO/what_cam_lock: No such file or directory
/Applications/Switch.app/Contents/Menu.command: line 2751: syntax error near unexpected token `>'
/Applications/Switch.app/Contents/Menu.command: line 2751: ` exec &> >(tee -a "$(cat /tmp/DUALISO/samples | head -1 | cut -d "." -f1)"_samples/"$(cat /tmp/DUALISO/samples | head -1 | cut -d "." -f1)"_LOG.txt >&2 )'
[Process completed]
Commenting out line 2751 does actually take me to the menu, where everything appears to work - I'm testing to see if there are any obvious problems.
And, yes, very good question, I've changed my shell to zsh, I haven't tried it in Bash and expect that it will work since you've tested exclusively in Bash, right?
Thanks
Yes, it's most likely an issue with zsh and I havn't bothered to fullproof it but I'll check into some bash fix even if set to zsh.
Thanks for sharing info here.
Yes it must be (an issue with zsh). I can't see why without testing, since all of the commands on those lines work independently. Perhaps the problem is in the piping, routing or in character escaping. In either case, I'm not wedded to zsh, so it's a non-issue.
I've run a complete test on a Dual-ISO file (interlaced), and Switch works just fine, creating the expected .dng files, then going over them all, applying a1ex's algorithm and renaming them to .DNG files.
The only trouble I've run into is where the process fails, and it's whenever the camera is pointed at a very bright light source. Then the files are left untouched. I suspect this is no different to the camera going into panic when in 14-bit lossless, at ISO 100, and it's pointed at a very bright scene which it can't adequately compress.
Thanks again for the quick help, Danne.
Thanks for the update, @Danne! ;)
Thanks deafeyejedi.
I updated Switch with the latest changes to cr2hdr from here:
http://www.magiclantern.fm/forum/index.php?topic=7139.msg194062#msg194062
If testing check for flicker(or the absence of it)
Hi small update to the GUI has been done,
now its possible to select input folder and output folder, I'm now going to investigate how to connect the buttons in the gui the to switch command for input and output.
https://bitbucket.org/Dannephoto/switch_gui/overview
https://vimeo.com/246417987
Looking good Lars. Since you managed to select both input and output folders it should be fairly easy to send the paths to a specific location like in Switch. Try following:
1 - After input folder is selected have xcode write a file with the actual input path and send the file accordingly:
mkdir -p /tmp/DUALISO #creates the path directory
echo $PWD > /tmp/DUALISO/path_1 #sends the current directory to a file called path_1
2 - do the same for the selected output path but send it accordingly:
echo $PWD > /tmp/output #sends output directory to a file named output
We now have both input and output set as in Switch. Both examples needs you to be inside the actual folder you selected(that´s what echo $PWD does, it gets the current directory path).
There is one more location that needs to be set in this application for it to be stand alone. We need to know where to find the binaries and subscripts. For that to work in automator there are a lot of roundtripping to be done so I´m thinking there are better ways for this in xcode. In Switch itself all scripts and binaries are located inside the Contents folder.
Question. Do you need the Select Input Folder button? Wouldn´t the buttons Input Folder and Output Folder be sufficient?
Right now the input code is like this:
on inputfolder_(sender)
choose folder
quit
end inputfolder_
https://developer.apple.com/library/content/documentation/LanguagesUtilities/Conceptual/MacAutomationScriptingGuide/PromptforaFileorFolder.html
If I look at the dev docs I see what is returned is for example
--> Result: alias "Macintosh HD:Users:yourUserName:Desktop:"
I think I need to add the result to a variable and then write the variable result to the switch folder right?
Will this write path_1?
Quote from: Lars Steenhoff on December 08, 2017, 01:28:18 PM
on inputfolder_(sender)
choose folder
do shell script "/bin/bash -s <<'EOF'
mkdir -p /tmp/DUALISO
echo $PWD > /tmp/DUALISO/path_1
EOF"
quit
end inputfolder_
Or this?
Quote from: Lars Steenhoff on December 08, 2017, 01:28:18 PM
on inputfolder_(sender)
choose folder
quit
do shell script "/bin/bash -s <<'EOF'
mkdir -p /tmp/DUALISO
echo $PWD > /tmp/DUALISO/path_1
EOF"
end inputfolder_
What we´re doing is putting in shell scripts inside apple scripts inside xcode :P
The quit command was not needed as it just quits the app :o
I tried this :
on inputfolder_(sender)
choose folder
do shell script "/bin/bash -s <<'EOF'
mkdir -p /tmp/DUALISO
echo $PWD > /tmp/DUALISO/path_1
EOF"
end inputfolder_And got this:
Quote-[AppDelegate inputfolder:]: /bin/bash: line 3: EOF: command not found (error 127)
Then I tried to put it in brackets and got this
QuoteClass FIFinderSyncExtensionHost is implemented in both /System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit (0x7fff87bdfb68) and /System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride (0x114496cd8). One of the two will be used. Which one is undefined.
on inputfolder_(sender)
choose folder
--do shell script ("/bin/bash -s <<'EOF'mkdir -p /tmp/DUALISO echo $PWD > /tmp/DUALISO/path_1 EOF")
end inputfolder_
https://stackoverflow.com/questions/46999695/class-fifindersyncextensionhost-is-implemented-in-both-warning-in-xcode-si
Its nothing serious.
I will try again
Sounds good. Gonna be limited access this weekend but will check in from time to time. Good luck Lars.
New version uploaded
- added a time estimation notifier along with the percentage counter.
https://bitbucket.org/Dannephoto/switch/commits/04f46bb1147f177155904208dbc8fc09525dfe9f
(https://s18.postimg.org/4zj8vn1k9/Screen_Shot_2017-12-11_at_15.46.19.png)
Great to have estimates on percentage and time, It will be good to tap into those for the gui too
I think so too. Just added some auto adjustments while processing to the code so that time estimation will stay closer to the truth.
https://bitbucket.org/Dannephoto/switch/commits/4a0093889fa0132f14e7eb7ec7d36a1bce9e10a6
New version.
Added -s option to find to fix this:
http://www.magiclantern.fm/forum/index.php?topic=15108.msg190108#msg190108
Thanks zachnfine and deafeyejedi for reports.
@Danne
Is this "New Version" the 1 that is linked to in the 1st Post?
Sure wish Version Numbers could be Attached to all Versions of ML Builds.
ORR ~ DeanB
Yes, first post.
Updated Switch. Download in first post:
-fpmutil(https://bitbucket.org/bouncyball/mlv-tools) updated and also added new fpm files in mlvfs workflow in Switch(thanks to dfort and Bouncyball and theBilalFakhouri)
http://www.magiclantern.fm/forum/index.php?topic=16054.msg195837#msg195837
Hi Danne, come back to it after sometime! Switch is the only one that really worked, others crashed. I didn't update in a while so this can't be a real bug report, out of 25 files (240GB - 2.5 hours to do it) last two didn't extract to dng: M20-1734 and M20-1739. I've now done them specifically and it worked. The faulty log: https://pastebin.com/FYQi6W7V The good one for the last two: https://pastebin.com/uUtrSvJD
Downloading the new version now!
Hi togg. Do they extract when processed individually? Did you run mlv_dump on steroids?
Quote from: Danne on January 21, 2018, 01:40:42 PM
Hi togg. Do they extract when processed individually? Did you run mlv_dump on steroids?
yes it worked when done individually, second log. no it wasn't mlv_dump on steroids. Anyway don't think about it it was an old version, I just pointed it out to come back on the thread :)
New version in first post. Mainly I introduced a (b) bash section into main menu. These scripts will fall into a category of independently running bash scripts and probably will ask user for downloading a dependency or two.
(https://s14.postimg.cc/c6gnyvprl/Screen_Shot_2018-02-26_at_16.57.21.png)
Latest bash scripts are working with HDR workflows for files like MOV, CR2, JPG, tif and a plethora of open source tools are being used. Latest scripts are about this:
https://www.magiclantern.fm/forum/index.php?topic=21678.msg197821;topicseen#msg197821
(https://s14.postimg.cc/8muq8wfa9/Screen_Shot_2018-02-26_at_16.46.51.png)
If choosing (cr2) HDR CR2,JPG,tif automation(hugin/exiv2/HDRmerge/enfuse/ffmpeg)
(https://s14.postimg.cc/wdu3r0n75/Screen_Shot_2018-02-26_at_16.46.58.png)
@ Danne
Will Switch work in High Sierra?
Yes.
Hey Danne!
Good to see you have kept up with the development of cr2hdr (Now switch).
I haven't been around much, been shooting raw on a c200, but using my 5d3 as a b-cam tomorrow and cant wait to give this a try.
Do these DNGs work in premiere?
Cheers
B
I believe so. Long time since I checked but should be fine. Let me know otherwise.
Now possible to simply drag CR2 or DNG dualiso files onto Switch and it will start using cr2hdr right away:
https://bitbucket.org/Dannephoto/switch/commits/20dcc4188cf1b913168279aa08cd3d4f67c554fc
Download in first post
Will work multiprocessing style(4 processes in parallell).
It´s also possible to drag jpg/JPG,/tif/tiff/png files and you´ll get some options to transcode into smaller tif format. Good for posting here for instance.
(https://s7.postimg.cc/4x7g5dyyj/Screen_Shot_2018-04-12_at_11.50.20.png)
Sorry for this silly question, but I have a doubt:
Time ago I had installed an old Switch. Then yesterday I installed Exiftool. And then today I downloaded the updated Switch for overwriting the old one. Only at this point I read that Switch uses Exiftool, so my question is: could this 2 installations (Switch and Exiftool) be in conflict? Should I uninstall Exiftool?
Thanks a lot.
@adrjork
Not at all. You can install any version of exiftool since Switch is using its own version (PATH is set to Contents folder inisde of Switch).
Perfect! Thank you so much Danne!
@Danne
Thank you for this exciting work on Switch.
I was used to MLRV exporting to .mov and adding a curve (log-c, c-log or slog2) to then go straight into Premiere Pro to edit and grade using 70d. After now playing with EOSM raw I've found switch gets rid of focus pixels, which MLRV can't do on its own - huge thanks.
My question is, to get the same effect as before, do I just select Rec709 - Clog or Rec709 - LogC before export to ProRes to get the same solution from Switch? Will it net the same result? Sorry if I have missed something here. I think the reference to rec709 had thrown me.
@bakersdozen
Not sure how far you got yet but exporting to any of the colorspace formats and view them should be a good starting point. Did you select (p) in main menu and then pick one of the formats.
(https://s14.postimg.cc/3r17hp1ch/Screen_Shot_2018-04-23_at_08.15.54.png)
Yes I did, but yours looks a little different. I must check for the latest version I guess as I am still seeing option for rec709 to LogC etc. Your screenshot now makes more sense. If I am understanding correctly, linear means what it would look like without adding any curves, just a straight export to ProRes .mov? Beautiful. Thanks mate. Any ideas to add slog2 by any chance? What is cineon, XYZ and aces by the way?
Linear in this case is dcraw -H 2 which is linear with added -H 2 highlight reconstruction. This setting is the closest I could get to acr.
Aces and XYZ are colorspaces called from dcraw as well. Plenty to read about these two but philosophy here is to retain as much color as possible to be used in post. However output mov is reduced to 10bit so color is affected(maybe not visible by eye). Using these two is probably better with full 14bit as the dng itself which isn´t possible right now.
The untouched, passthrough setting is the (04) rec709 but this isn´t the best setting imo since dng output to rec709 will in this case clip highlights.
About slog2, I could check into adding this.
Thank you Sir! This and your last post have helped me understand emmensly. Much appreciated. You have certainly got me pointed back in the right direction. Slog2 would be a cool addition, if at all possible.
Did you check into Mlv app by the way? Think it runs slog2.
Oh, and see to it you are selecting (p) not (o) in main menu to get to dng to prores menu.
Thanks again, yes my issue was wrong menu selection. Confirming P works well for me now after some guidance. mlv app is good, but didn't fix focus pixels.
Hm, that is interesting as Switch should be using the same focus pixel lists. Think it will help Mlv app developers if you post them an example file to check.
Just to make sure we are talking of the same app:
https://www.magiclantern.fm/forum/index.php?topic=20025.0
Danne, i don't know if it's the right thread, but since Switch uses MlRaw Viewer 1.5, does anybody know how can i convert MLV to MOV (or DNGs) with the same look (filter/settings/grading - whatever it's called) as MlRaw Viewer 1.5 shows with it's default c-log? I just like the default look that this app gives.
I have Switch, cr2hdr, Colorcast 0.5 and Footage for Mac (i use these two), also Resolve, Adobe Premiere and After Effects.
I can try to achieve this with Colorcast and Footage for Mac, but i can't get the same results, i feel like i'm doing it worse than MlRaw Viewer 1.5 does and i'd like to do it quickly.
OR
What does MlRaw Viewer 1.5 does with the file besides giving it c-log? So i could repeat it in Colorcast or Footage for Mac?
Thank you!
(https://thumb.ibb.co/bR5Wgx/Screen_Shot_2018_04_29_at_9_37_25_PM.png) (https://ibb.co/bR5Wgx)
I can´t really say what you should do here. Switch is exporting Log-C through piping in dcraw and ffmpeg. What do you see when you export these files and wht is wrong with them coming from Switch @whysodifficult?
Came back to magic lantern and switch (1.2? latest) after kind of a long break, I have a couple of new questions to get in touch again with it :)
- Batched 22 clips, 174GB. Simple dng export with jpeg compression, it took 2 hours. Is it in line with what you're having? I remembered it being faster mmm
- Audio of the last clip doesn't export, M29-1323.
- Skipped naming or exporti a frame in a moment where there're corrupted frames in the original file, M28-1741.
- The countdown was completelly off at the end of the process, negative numbers or something like that.
the log: https://pastebin.com/fWaHdWyM
Time sounds about right. Compression will take longer then without. The indicator can of course be counting better. Not very reliable and I´ll see if I can fix it. The other stuff is too hard to tell anything about. Seems it doesn´t have to do with Switch...
I wonder if it is possible to output dualiso stills without any interpolation in Switch? I am ok with resolution loss and if i get it right it will help get rid of banding even at extreme settings. Is it so?
Cheers! :)
I am sorry but I don't really get what you want? Please feel free to elaborate.
I found some lines in dualiso pdf:
5. Mixing the two exposures: "So, why not just use the
ISO 1600 data everywhere, besides the top 4 stops? Because resolution matters too, and so does
aliasing and moire."
5.1 Recovering the full resolution in midtones:
"As you can see from Figure 7, the result with the smooth transition is a little on the soft side (but
without any banding). The midtones created without any interpolated pixels (Figure 7, bottom)
have the same resolution as a normal picture (without dual ISO), but has some slight banding
artifacts."
"...in this case you'll have to live
with a softer image, obtained only with curve (9), which does not have any banding."
That makes me think that banding in shadows and highlights appears while recovering full resolution midtones. Is it correct?
Is it possible to trade half resolution to have zero banding?
I hope I managed to make it a bit more clear. :)
I don't think you can get around this(havn't checked the pdf) but you could ask in the dualiso thread about this or simply read it through. I think most of the stuff has been tested and discussed more than twice already :)
Will try that. Thanks! :)
Hello.
I am writing this in Google Translator because I am not good at English.
I think the switch is a great program. Thank you for your development.
I am using 5d mark3 and magiclantern-crop_rec_4k.2018Mar10.5D3113.
In the meanwhile, the monitor of 5d is small, and it is inconvenient to shoot. So I recently connected my smartphone to the camera and tried shooting with the DSLR Controler app. But there are various problems. An error often occurs during shooting, which makes stable shooting difficult. And there is also a problem with dng conversion.
After shooting today, I converted my mlv file to dng using the latest version of the switch. I reset the switch to default by using (R) first. Then, after selecting (ms), I ran (r). Some files were successfully converted, but some did not.
This is my mlv file.
case (1)
https://drive.google.com/open?id=1G43xO-7ni3ynm997KmB_utRPoOb1-W9t
After converted to dng, with an oblique oblique line on the screen and strange colors.
(https://thumb.ibb.co/cnLwco/Screen_Shot_2018_06_10_at_11_36_32_PM.png) (https://ibb.co/cnLwco)
case (2)
https://drive.google.com/open?id=1IuOQTsPbHdSBZDEFBp3b--rxaWrDKtQW
This file has only a small wav file (626bytes) in the folder after the conversion and no dng file at all.
Here in my log
https://drive.google.com/open?id=1xgPV8Oegmb0zFtxmkJFctPB5TdDU7Mwc
I have played files with the above problem with MlRawViewer.app (1.5.0). Files play well in MlRawViewer.app.
I have converted the file to dng from MlRawViewer.app (1.5.0). I imported it from Da Vinci Resolv, and the screen looks clean. But the sound can not be played back. Import wav separately and place it on the timeline to hear the sound.
Today I took 35 mlv files and converted them into switches, of which there are 3 files with strange colors like case (1), and 9 files with no dng file like case (2).
I think it is caused by linking an app called DSLR Controler with 5d mark3, but I would like to use a smartphone to use focus picking and zebra on a bigger screen. Or I would prefer to see a larger magic lantern screen directly on my smartphone. Is there anyone who can suggest a good way I do not know?
Thank you for reading and I wish you all happy. Thank you.
Cool. Have a little less time atm. Could you try exporting your troubling files again but this time set the switch to relaxed in (ms) menu?
Thank you.
I tried relaxed mode export. Here is screenshots.
case 1 is same. Nothing changed.
This is first frame of case 1.
(https://s19.postimg.cc/69p7g7cqr/Screen_Shot_2018-06-11_at_7.24.32_AM.png) (https://postimg.cc/image/fu8u3322n/)
This is second frame of case 1.
(https://s19.postimg.cc/4j66e5agz/Screen_Shot_2018-06-11_at_7.24.57_AM.png) (https://postimg.cc/image/ljp2mtni7/)
MlRawViewer 1.5.0 plays well.
(https://s19.postimg.cc/v48pa3xj7/Screen_Shot_2018-06-11_at_9.17.03_AM.png) (https://postimages.org/)
case 2 is little better. It converted. but video and audio is separated.
(https://s19.postimg.cc/bme1trnmb/Screen_Shot_2018-06-11_at_7.36.26_AM.png) (https://postimg.cc/image/pstsozyhb/)
(https://s19.postimg.cc/5lgcwp8pv/Screen_Shot_2018-06-11_at_7.36.33_AM.png) (https://postimg.cc/image/5y7r2vqzj/)
MlRawViewer 1.5.0 plays well.
(https://s19.postimg.cc/7d9bs17mr/Screen_Shot_2018-06-11_at_9.21.15_AM.png) (https://postimages.org/)
PS: I noticed a few moments ago, but the number of frames displayed by MlRawViewer is different from the one converted to a switch. If possible, I would like to know how to convert when converting from MlRawVeiwer to dng, which is able to import the latest version of DaVinci Resolves without problems.
I'm using DaVinci Resolve 15 beta 2 now.
Beta 3 and 4 has has trouble, green and magenta cast when import dngs.
(https://s19.postimg.cc/et8ldenhv/Screen_Shot_2018-06-03_at_7.43.15_PM.png) (https://postimages.org/)
Thank you.
Hi freely.Thanks for posting clear examples.
I tried your file M10-1749.MLV. It exports if selecting (15) relaxed(skip blocks that are erroneous) in ms menu.
(https://s22.postimg.cc/oysby0w75/Screen_Shot_2018-06-11_at_06.52.20.png_500px.jpg)
The error is related to this:
File /Users/dan/Desktop/Timer_A/freely/M10-1749.M00 not existing.
Processing...
[ERROR] LJ92: Open failed (-1)
Reached end of chunk 1/1 after 229 blocks
Processed 89 video frames at 23.98 FPS (3.71 s)
Done
In mlv_dump.c you can see that it has to to do with decompressing. Some block that is erroneous, setting relaxed mode skips abortion when that block comes along:
#ifdef MLV_USE_LJ92
lj92 handle;
int lj92_width = 0;
int lj92_height = 0;
int lj92_bitdepth = 0;
int lj92_components = 0;
int ret = lj92_open(&handle, (uint8_t *)frame_buffer, read_size, &lj92_width, &lj92_height, &lj92_bitdepth, &lj92_components);
/* this is the raw data size with 16 bit words. it's just temporary */
size_t out_size = lj92_width * lj92_height * sizeof(uint16_t) * lj92_components;
if(ret == LJ92_ERROR_NONE)
{
if(verbose)
{
print_msg(MSG_INFO, " LJ92: Decompressing\n");
print_msg(MSG_INFO, " LJ92: %dx%dx%d %d bpp (%d bytes buffer)\n", lj92_width, lj92_height, lj92_components, lj92_bitdepth, out_size);
}
}
else
{
print_msg(MSG_ERROR, " LJ92: Open failed (%d)\n", ret);
if(relaxed)
{
goto skip_block;
}
goto abort;
}
Why are you getting this error? Proabably the reason you mention yourselves. Now I´ll check your other file if it gets downloaded today :P.
By the way. I can also recommend Mlv App for previewing exporting to dng etc:
https://www.magiclantern.fm/forum/index.php?topic=20025.0
Last I tried MlRawViewer dng exports metadata wasn´t top notch (cdng specwise) so maybe that relates to issues in DaVinci Resolve.
Thank you for your reply.
I have tried several things first with M10-1746, including what you recommended. The method I have found so far is to import the mlv file from the MLV App and uncheck "Enable RAW Correction".
(https://s19.postimg.cc/jsexup1w3/Screen_Shot_2018-06-11_at_3.21.47_PM.png) (https://postimages.org/)
Then select "Export Selected Clips" from the File menu. Then, when I import the file from Da Vinci Resol, both video and audio are played back simultaneously. I do not see anything like color casts.
I also tried M10-1749, which is exported but the message in the conversion process is different from M10-1746. When the converted dng is imported into Da Vinci, the resolution information is "2048x1556".
(https://s19.postimg.cc/nc0vkicbn/Screen_Shot_2018-06-11_at_3.37.25_PM.png) (https://postimg.cc/image/iddd5z8in/)
When I check the information of the dng file in the finder, it appears as "1920x818", and it seems that some misinformation is transmitted.
In the last few weeks, as I was searching and testing various information, what I was feeling was that I wanted to know how to stably shoot with a magic lantern on a bigger screen. I feel it is unstable to shoot a magic lantern with a DSLR controller. I wonder how you can shoot with a bigger screen like a smartphone, with the focus adjusted exactly.
Danne Thank you for your kindness and good luck.
QuoteI wonder how you can shoot with a bigger screen like a smartphone, with the focus adjusted exactly.
Something out of the hdmi output? A small monitor maybe?
I use magic zoom on the actual liveview window. For me it works really good and it can be hide and unhide with half shutter pressing.
Danne Thank you for your suggestion.
I tried Magic Zoom. It will be quite helpful when taking fixed shots. However, when shooting a slight motion shot, I could not move the zoom box during shooting, so it seemed a bit difficult to control. Still useful information thanks.
Purchasing an hdmi monitor is burdensome for now, so I hope to find a way to fix it with my smartphone if possible.
I'm testing DSLR Coltroller again. When you shoot with DSLR Coltroller, these characters appear on the camera lcd. I do not know the meaning of this code, but I do not know if I can get help from this.
(https://s19.postimg.cc/jd8pyiz2b/20180611_232428.jpg) (https://postimg.cc/image/6lujs0pa7/)
Thank you very much.
Mainly means something isn't optimal, I can't say what exactly. I would probably stay off dslr controller and raw recording but it's your call. Good luck!
I've messed with DSLR Controller and Raw... definitely not ideal. I believe it wasn't possible to trigger video via the app, but it was possible to see the phone screen during recording. Could be wrong as this was a long time ago, on my 5d2.
Normal video recording, however, works fine. I believe DSLR controller has a 1:1 pixel feature, too.
(https://thumb.ibb.co/jwLA4p/Capture_d_e_cran_2018_09_13_a_15_31_23.jpg)[/url]
Hello Danne, I'm cleaning 10 hours of h264 proxy from 5DMKIII, and switch is running. But it shows me this parse error message (see the picture)
What does it mean ?
log :
https://pastebin.com/sYV9guek
Edit : I batch renamed h264 files to correspond to the MLV's name. It's still showing errors...
Edit2 : just 5 shots worked and now they are named like this : FY5A0516_1_2017-08-30_0001_C0000
It's good if we generate DNG files via MLVFS. But if we relink with the new XML option from MLV, we could keep only the original name. I don't understand what's wrong with my files.
Good morning, everyone,
I have a problem with the amazing workflow H264 proxy + MLV.
I have to process a lot of h264 proxy files to remove the black frames/pink frames at the beginning of the shot in order to get clean h264 files that start with the same 1st frame as the MLVs. With Switch, this function worked on some files and I had done a lot of tests in September 2017 (see: https://www.magiclantern.fm/forum/index.php?topic=15108.msg189532#msg189532). The workflow consisted of:
1) run switch to remove the black/pink frames from the beginning of the MOV H264 proxy by generating a new file without transcode and whose TC is reset to 00:00:00:00:00 to match the MLV TC
2) edit offline with the h264 proxy.
3) At the end of the editing, select the MLV files that correspond to the files edited in H264 proxy. Then develop them with MLVFS to generate on the fly CNDG
4) import via XML and grading in Resolve
This saved a lot of disk space and time.
The first issue is that step 1) does not work on all files, whether with an old build or the latest build. The file keeps the pink/black frame from the beginning even after it has been processed.
In addition, with Danne, we tried to set up an even sexier workflow when it is combined with MLV app. It would be:
1) run switch to remove the black/pink frames from the beginning of the MOV H264 proxy by generating a new file without transcode and whose TC is reset to 00:00:00:00:00:00 to match the MLV TC
2) edit in offline with the h264 proxy.
3) At the end of the offline editing, Export an FCPXML from the timeline (or convert via Resolve a Premiere/AAFAvid XML to FCPXML), import it into MLV app via the FCPXML import option. This will output the list of mounted shots and import the MLV files corresponding to the edited h264 proxy.
4) do the pre-correction via MLV app and export to ProRes 4444 C-log
5) import into Resolve via FCPXML which will reconnect to ProRes 4444 C-log.
6) Grading and export your master
7) take some popcorn and admire the masterpiece 😎
The time, performance and disk space savings would be enormous!
So the problem for now is really how to detect properly the black/pink frames from the beginning and cut them by an automation. Danne thinks there's no need to cut the black frames at the end of the clip to reconnect the clips during the conformation phases (just the black frame/pink frames at the begining). This is why the latest switch versions allow you to skip frame counting to cut the black/pink frames of the MOV H264 Proxy, which speeds up the process enormously.
If anyone has any idea of the problem, don't hesitate!
@Dfort maybe?
I'm trying to run Switch in ChromeOS as a Linux container. I'm running into this error:
(yad:708): Gdk-WARNING **: /build/gtk+3.0-NmdvYo/gtk+3.0-3.22.11/./gdk/x11/gdkwindow-x11.c:5573 drawable is not a native X11 window
It successfully opens up a file-picker, but I just get a long list of "No such file or directory"s
Any thoughts? Would appreciate any help. Getting this running smoothly in a container would make my life on the go so much easier.
Sorry, can't help atm but you should try google the issue and maybe you'll fix it yourself.
I get it! I'm a linux n00b so a lot of basic stuff goes over my head.
Looking around a bit, it seems someone solved a similar issue by launching their program with something like
GDK_BACKEND=x11 ~/Switch/bin/Switch_MAIN.sh
This fixed the error I was having, I was prompted to pick a file like before, but then pretty much the same stuff followed. I'm only pasting a few lines because it goes on for a while:
ls: cannot access '*.MLV': No such file or directory
ls: cannot access '*.mlv': No such file or directory
/home/username/Switch/bin/Switch_MAIN.sh: line 80: /home/username/Switch/bin/mlv_dump: Permission denied
ls: cannot access '*.MLV': No such file or directory
ls: cannot access '*.mlv': No such file or directory
/home/username/Switch/bin/Switch_MAIN.sh: line 85: /home/username/Switch/bin/mlv_dump: Permission denied
Appreciate any ideas anyone may have.
@12georgiadis
Quote from: 12georgiadis on October 05, 2018, 10:03:26 AM
latest switch versions allow you to skip frame counting to cut the black/pink frames of the MOV H264 Proxy, which speeds up the process enormously.
Pink frames will not count as black frames so naturally they will not be cut out. Do you get a lot of files with pink starts?
You say that some files works and others don´t in your workflow. So which ones are working fully and which ones are not? Only the pink proxy files or are there others? If possible upload one file that is fully working, and one file that is not for comparison testing.
Is the two first steps working with all proxy scenarios on a working file?
Quote1) run switch to remove the black/pink frames from the beginning of the MOV H264 proxy by generating a new file without transcode and whose TC is reset to 00:00:00:00:00 to match the MLV TC
2) edit offline with the h264 proxy.
Please test following with a working proxy file and corresponding MLV:There are three workflows in Switch. One is automated and activated simply without doing anything in any menu settings(First test). Next processing test would be activating
(24) MLV+proxy(outputs only MOV(default MOV+MLV) in mlv_dump on steroids then run Switch menu and in the following menu select
(r) Continue without modifications. Last test would be selecting
(S) Skip MLV indexing(faster but proxy end part is not cut)
@jpegmasterjesse
Quote from: jpegmasterjesse on October 06, 2018, 06:49:53 AM
This fixed the error I was having, I was prompted to pick a file like before, but then pretty much the same stuff followed. I'm only pasting a few lines because it
Are you selecting a folder with MLV files to develop into dng or what are you trying to do? I don´t think you are supposed to select the MLV file itself but the actual folder.
You could try fix the permission error. In terminal write:
cd /home/username/Switch/bin/
sudo chmod o+r mlv_dumpenter
rerun Switch again on a folder with files.
Quote from: Danne on October 06, 2018, 07:53:33 AM
@12georgiadis
Pink frames will not count as black frames so naturally they will not be cut out. Do you get a lot of files with pink starts?
You say that some files works and others don´t in your workflow. So which ones are working fully and which ones are not? Only the pink proxy files or are there others? If possible upload one file that is fully working, and one file that is not for comparison testing.
Is the two first steps working with all proxy scenarios on a working file?
Please test following with a working proxy file and corresponding MLV:
There are three workflows in Switch. One is automated and activated simply without doing anything in any menu settings(First test). Next processing test would be activating (24) MLV+proxy(outputs only MOV(default MOV+MLV) in mlv_dump on steroids then run Switch menu and in the following menu select (r) Continue without modifications. Last test would be selecting (S) Skip MLV indexing(faster but proxy end part is not cut)
@Danne,
I had my 5DmkIII back to make some tests. I used the last build and the new workflow is working well ((S) Skip MLV indexing(faster but proxy end part is not cut)), very fast and it cuts the black. I think the pink frames belongs to a bugged build but it's fixed in newer versions.
I still have to test the offline/online MLV app conformation and see if it's ok or a problem to have the black at the end. I expect no problem. Thank you again for the support and to have helped with an even faster workflow with the new skip MLV indexing option !
Cool, let me know how it goes :).
Quote from: 12georgiadis on October 05, 2018, 10:03:26 AM
@Dfort maybe?
Looks like you got it figured out. I used a similar workflow when Danne first added the proxy workflow feature. What I like is that it is pretty much like shooting plain H.264. Realtime in camera playback with audio (H.264 files) is a great confidence check. I was getting very picky with a couple of files that didn't match exactly (out of several hundreds) but that was due to the card reaching capacity. Overall it a great workflow though probably only useful for 3x3 sampling at 1920x1080. Haven't tested other resolutions.
Sent from my iPhone using Tapatalk
Quote from: dfort on October 07, 2018, 04:45:25 AM
Looks like you got it figured out. I used a similar workflow when Danne first added the proxy workflow feature. What I like is that it is pretty much like shooting plain H.264. Realtime in camera playback with audio (H.264 files) is a great confidence check. I was getting very picky with a couple of files that didn't match exactly (out of several hundreds) but that was due to the card reaching capacity. Overall it a great workflow though probably only useful for 3x3 sampling at 1920x1080. Haven't tested other resolutions.
Sent from my iPhone using Tapatalk
I agree with you and now that Switch is superfast by skipping number count, it's even more reliable. I don't see any problem with other resolution because for what I know you can do offline/online from 720p Alexa proxy and reconnect to 2K raw. you just need the same name, tci+tco
Is it confirmed working skipping number count?
Yes working !
That is good news.
Anyone have any pointers for porting Switch to Android? Thinking primarily for DUAL-ISO photo conversion - I don't think I want to mess around with MLV's on a smartphone. Although....
Quote from: Lars Steenhoff on December 08, 2017, 11:20:57 AM
Hi small update to the GUI has been done,
now its possible to select input folder and output folder, I'm now going to investigate how to connect the buttons in the gui the to switch command for input and output.
https://bitbucket.org/Dannephoto/switch_gui/overview
https://vimeo.com/246417987
Have you tried CocoaDialog (v2)
I was using it in for raw2dng.app (https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dng.app.0.13.zip) years ago, with platypus as wrapper fo the scripts.
You can see inside the pkg it's not loked up so it's easier to see how it works:
rv=`CocoaDialog fileselect \
--text "Select some files and/or directories to convert or scan" \
--with-directory $HOME/Images/ \
--with-extensions .422 .MOV .CR2 .SH .JPG .jpg .WAV .LOG .AVI .RAW .DNG .XMP .MLV .UFR\
--select-directories \
--select-multiple`
if [ -n "$rv" ]; then
...
fi
With xcode could probably made nicer, but maybe this helps.
Quote from: jpegmasterjesse on April 09, 2019, 04:57:28 AM
Anyone have any pointers for porting Switch to Android? Thinking primarily for DUAL-ISO photo conversion - I don't think I want to mess around with MLV's on a smartphone. Although....
Switch works on Linux too?
So you can try to start it with termux...
Hum, seems to use sudo so you need to have a rooted phone, and so far even with it gives problems.
Sorry for my basic-user questions:
1. I saw that in MLVFS workflow White level can be set at 15000 or 16383. In Davinci RAW tab there is an Highlight Recovery option. Is it the same thing as setting 15000/16383 in Switch, or the two things works together?
2. Switch give me both the options of rendering regular DNG, compressed DNG and on-the-fly DNG (via MLVFS). Is it possible that for playback speed in Davinci, the fastest option is regular rendered DNG over compressed DNG over MLVFS? And if so, is it possible that grading could be slightly different if applied to compressed DNG instead of regular DNG?
3. Believe me, I don't want make a SW-vs-SW challenge: I'm just curious to know which are the differences between compressing DNG with Switch and with SlimRAW (both are in this forum and both are loved by users, and both have a similar function, so I suppose both have great PROs and perhaps are useful in different situations).
Thanks a lot.
I tried Switch for the 1st time without problem on a ubuntu studio 18.10. It installed Yad , as planned. With a serie of pictures, dualiso, from the same Eos5DmkII, it doesn't work at all. I look at the log file & there, i don't understand the problem... The Switch files are in my Home directory. I followed the instructions. Can you help me?
The log file:
Quotels: cannot access '*.MLV': No such file or directory
ls: cannot access '*.mlv': No such file or directory
[ERROR] Failed to open file ''
ls: cannot access '*.MLV': No such file or directory
ls: cannot access '*.mlv': No such file or directory
[ERROR] Failed to open file ''
[8;23;55t[3;450;0t[3J[H[2J =========
[1mMain menu(B[m
---------
[1moutput: [34m(B[m
[1mdarkframe storage:[34m[34m(B[m
[1mafplayer: [34m(B[m
[1m[31m(m) mlv_dump settings(B[m[1m(MLV)(B[m
[1m[31m(ms) mlv_dump_on_steroids settings(B[m[1m(MLV)(B[m
[1m[31m(p) ProRes output(B[m[1m(MLV,RAW,dng)(B[m
[1m[31m(d) cr2hdr dualiso processing(B[m[1m(CR2)(B[m
[1m[31m(C) select new output folder(B[m[1m(MLV,RAW,dng,mov)(B[m
[1m[31m(R) reset Switch to defaults(B[m
[1m[31m(A) darkframe average automation(B[m[1m(B[m
[1m[31m(F) flatframe average automation(B[m[1m(B[m
[1m[31m(q) exit Switch(B[m
[1m[31m(r) run Switch(B[m
Please enter your selection number below:
ls: cannot access '/home/dim/Switch/config/tmp/FLATFRAME_A': No such file or directory
[8;16;63t[3;410;100t[3J[H[2J[1mWould you rather select flatframe and darkflat manually?[31m
Y/N?(B[m[3J[H[2J[8;4;50t[3;200;50tSelect your flatframe and a darkframe if needed
[8;23;55t[3;450;0t[3J[H[2J =========
[1mMain menu(B[m
---------
[1moutput: [34m(B[m
[1mdarkframe storage:[34m[34m(B[m
[1mafplayer: [34m(B[m
[1m[31m(m) mlv_dump settings(B[m[1m(MLV)(B[m
[1m[31m(ms) mlv_dump_on_steroids settings(B[m[1m(MLV)(B[m
[1m[31m(p) ProRes output(B[m[1m(MLV,RAW,dng)(B[m
[1m[31m(d) cr2hdr dualiso processing(B[m[1m(CR2)(B[m
[1m[31m(C) select new output folder(B[m[1m(MLV,RAW,dng,mov)(B[m
[1m[31m(R) reset Switch to defaults(B[m
[1m[31m(A) darkframe average automation(B[m[1m(B[m
[1m[31m(F) flatframe average automation(B[m[1m(B[m [1m[32madded!(B[m
[1m[31m(q) exit Switch(B[m
[1m[31m(r) run Switch(B[m
Please enter your selection number below:
[8;33;50t[3;410;0t--amaze-edge --cs2x2 --same-levels
--amaze-edge --cs2x2 --same-levels
--amaze-edge --cs2x2 --same-levels
grep: /home/dim/Switch/config/A_cr2hdr_cmpr.txt: No such file or directory
grep: /home/dim/Switch/config/A_cr2hdr_cmpr.txt: No such file or directory
ls: cannot access '/home/dim/Switch/config/cpuboost': No such file or directory
[3J[H[2J ===============
[1m[31mcr2hdr settings(B[m
---------------
[1mnote! Reenter selection to erase!
[1moutput: [34m(B[m
[1mInterpolation methods(B[m
[1m(a)(B[m Amaze edge(default) [1m[32madded!(B[m
[1m(m)(B[m Mean23(faster)
[1mChroma smoothing(B[m
[1m(2)(B[m apply 2x2(default) [1m[32madded!(B[m
[1m(3)(B[m apply 3x3
[1m(5)(B[m apply 5x5
[1m(di)(B[m disable chroma smoothing
[1mFlicker handling(B[m
[1m(s)(B[m same-levels(avoid flicker) [1m[32madded!(B[m
[1mSpeed increase(B[m
[1m(B)(B[m CPU boost
[1m[31m(E) erase all settings(B[m
[1m[31m(d) Main menu(B[m
[1m[31m(q) exit Switch(B[m
[1m[31m(r) run Switch(B[m
Please enter your selection number below:
[8;16;95t[3;410;100t
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: '/home/dim/Pictures/2019/20190911/FLATFRAMES'
- Rewrite MLV
- Output only one frame with averaged pixel values
- Output into 'ft__FLATFRAMES'
File /home/dim/Pictures/2019/20190911/FLATFRAMES opened
File /home/dim/Pictures/2019/20190911/FLATFRAM00 not existing.
Processing...
Reached end of chunk 1/1 after 0 blocks
Processed 0 video frames at -nan FPS (-nan s)
[ERROR] Number of averaged frames is zero. Cannot continue.
[ERROR] Failed to rewrite header in .MLV file
Done
rm: cannot remove 'ft*.wav': No such file or directory
ls: cannot access '*.MLV': No such file or directory
ls: cannot access '*.mlv': No such file or directory
ls: cannot access '*.MLV': No such file or directory
ls: cannot access '*.mlv': No such file or directory
cat: /home/dim/Switch/config/tmp/MLVFILESaa: No such file or directory
cat: /home/dim/Switch/config/tmp/MLVFILESadcat: /home/dim/Switch/config/tmp/MLVFILESab: No such file or directory
: No such file or directory
cat: /home/dim/Switch/config/tmp/MLVFILESac: No such file or directory
wc: _1_2019-09-21_0001_C0000/pix_fix.txt: No such file or directory
wc: _1_2019-09-21_0001_C0000/pix_fix.txt: No such file or directory
cat: /home/dim/Switch/config/FFmpeg_settingsPR: No such file or directory
cat: /home/dim/Switch/config/FFmpeg_settingsPR: No such file or directory
cat: /home/dim/Switch/config/FFmpeg_settings: No such file or directory
ls: cannot access '*.CR2': No such file or directory
ls: cannot access '*.DNG': No such file or directory
ls: cannot access '*.dng': No such file or directory
Quote from: dim on September 21, 2019, 01:08:41 PM
I tried Switch for the 1st time without problem on a ubuntu studio 18.10. It installed Yad , as planned. With a serie of pictures, dualiso, from the same Eos5DmkII, it doesn't work at all. I look at the log file & there, i don't understand the problem... The Switch files are in my Home directory. I followed the instructions. Can you help me?
The log file:
Got the same yesterday. Worked last time I tried. Ubuntu 18.04.
Nothing changed from linux version(which was created looong time ago so could need refinements. Feel free to help) so If you use the same workflow as "Worked last time I tried" at least we can be sure nothing´s wrong with Switch.
More info/testing/digging needed.
Thank you, Danne! I use Switch in a xfce Terminal. I take screenshots & copy the folders images & Switch in a dropbox. I tried dualiso2hdr & the others alone, it works with those in folder Bin .
https://www.dropbox.com/sh/wktvznb98nxuymw/AADKn1F7EcieDYmXXJ0aEpIKa?dl=0 (https://www.dropbox.com/sh/wktvznb98nxuymw/AADKn1F7EcieDYmXXJ0aEpIKa?dl=0)
Sorry, unable to help here. I don´t use linux myself but maybe someone else can help.
Maybe it can be as simple as renaming .cr2 to .CR2 before converting. I also don´t see why there´s darkframe storage added for dualiso? Maybe start fresh?
Thank you Danne! The problem is the suffix .cr2 to rename to .CR2!...
hi! installed macOS Catalina, and I can't reinstall Switch, different questions showing up and I can't get it to work. https://cdn1.imggmi.com/uploads/2019/10/8/d7a2b47bf331b969d53f6304e3176a1e-full.png (https://cdn1.imggmi.com/uploads/2019/10/8/d7a2b47bf331b969d53f6304e3176a1e-full.png)
https://www.google.com/amp/s/www.cyberciti.biz/faq/change-default-shell-to-bash-on-macos-catalina/amp/
Another issue will probably affect all 32bit apps. Think Catalina is strictly 64bit architecture.
I can also confirm that Catalina definitely broke Switch due to being 32-bit architecture and unable to support.
I'm pretty sure @Danne's aware and already hard at work on updating this asap. No rush, though! 8)
https://vimeo.com/366523191
*Edit*
Take 2 per @Danne's suggestion which was to open Switch under Automator and save as a new app but didn't pan out either...
https://vimeo.com/366531067
In the meantime you can use the 64-bit mlv_dump as per Danne's modification (https://www.magiclantern.fm/forum/index.php?topic=7122.msg221204#msg221204). I posted a copy of it on my downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/).
Thanks for pointing this out @dfort -- will definitely give this a run right now.
Will let you know results as soon as I know.
*edit*
I just swapped out the mlv_dump file with the new one. Reran Switch and it's asking me to fix few things.
https://vimeo.com/368373524
Looks like I'm gonna have to go back to the tutorial page. Stand by!
I'm not having any luck with this so far. Any hints?
The tee command not working indicates terminal not defaulted to bash anymore. Look a few posts back and see if you can get it to open in bash.
Uploaded a new version of Switch. But raw2dng, cr2hdr and one of mlv_dump binaries are not working due to 32bit architecture? This affect Catalina version on mac. Mlv_dump on steroids works and it should start and show menus etc. If not try changing default shell back to:
chsh -s /bin/sh
dualiso conversion will not work.
Uploaded a version with raw2mlv conversion tool included(Ilia3101). It will only be active when on mac os catalina and when dragging CR2 file onto Switch. This will make it possible to open up for instance dualiso CR2 files straight into Mlv app for further processing. Please check it out:
Commit:
https://bitbucket.org/Dannephoto/switch/commits/7f1f4cabefc0b45c2338b3e1e99130f801a0357c
Booyah... This is all feeling like 2014 again. Love this @Danne even tho it may very well be a bit Catastrophy but hey it works!
Even better to be able to play with Dual-ISO photos within MLV_App is actually godsend. Just seems a bit slow tho. :)
Here I was able to somewhat duplicate the look of Ektar 100 LUT's (https://www.magiclantern.fm/forum/index.php?topic=19338.msg182487#msg182487Ektar%20100%20LUT's) with an old photo taken in ISO 100/1600 below...
Though I was under the impression that it would be ideal to have this done w Cineon Profile but I couldn't get it to look right.
Used Reinhard 3/5 & Alexa Wide Gamut RGB as Tonemapping Function/Processing Gamut options instead.
Applied the ML Cinelog-C to Ektar 100 TWISTED LUT on top w opacity set @ 80% or so within MLV_App.
The prompt messages while running Switch especially the final beep is indeed remarkable! :-*
https://vimeo.com/369062404
Tiff sample:
(https://live.staticflickr.com/65535/48964278748_34e532f450.jpg) (https://flic.kr/p/2hANJby)
just realised I need to add date and time to raw2mlv output!
Quote from: Ilia3101 on October 27, 2019, 02:05:21 AM
just realised I need to add date and time to raw2mlv output!
Yep. Thanks @Ilia3101! :)
Possible to also also the exif data as well?
It works! thanks all! when I export the picture it will end up .mov, what shall it be .gif or? how to change that?
Quote from: Macke1 on October 27, 2019, 12:32:05 PM
It works! thanks all! when I export the picture it will end up .mov, what shall it be .gif or? how to change that?
What are you doing? Mlv App exports to many formats, why choose mov?
When you plan your answer make sure to include a screen recording of what´s going on.
ok, sorry I found it now
Quote from: DeafEyeJedi on October 27, 2019, 02:13:45 AM
Yep. Thanks @Ilia3101! :)
Possible to also also the exif data as well?
I already add info about camera, so some exif/metadata is there. But I will add lens info and time/date now.
Quote from: Ilia3101 on October 27, 2019, 01:54:53 PM
I already add info about camera, so some exif/metadata is there. But I will add lens info and time/date now.
Gotcha. Good to know. Just thought it wasn't included because MLV_App wasn't showing any info after it's been processed through raw2mlv via Switch. Hmm?
Hence the video of screen recording I posted above.
Ah yes I see the video. Not much metadata is being added yet. Just enough for MLV App to know what camera it is :D
Cool that it works fully anyway. Outputting to dng would be nice seems deafeye tried but corrupted.
Also uploaded a new Switch version just now. Handles drag/drop better now.
Been enjoying the luxury of Dual-ISO CR2's being spat out via Switch w raw2mlv to view them files within MLV_App, however, as previously stated that if we export as DNG it's corrupted — any hints what could be the cause of this?
Good and bad news! :o
The bad is the fact that I am still unable to export Dual-ISO photos when running on Catalina... Here's the error code output via Switch:
Last login: Thu Nov 7 18:03:18 on ttys000
/Users/appleuser/Desktop/Switch.app/Contents/progress_bar.command ; exit;
DeafEyeJedi@Apples-Macintosh-10 ~ % /Users/appleuser/Desktop/Switch.app/Contents/progress_bar.command ; exit;
cat: /tmp/DUALISO/FFmpeg_ALL: No such file or directory
grep: /tmp/DUALISO/prores_TRAP: No such file or directory
grep: /tmp/DUALISO/samples: No such file or directory
grep: /tmp/DUALISO/CR2LIST: No such file or directory
grep: /tmp/DUALISO/LISTaa: No such file or directory
ls: /tmp/DUALISO/HDR_MOV: No such file or directory
ls: /tmp/DUALISO/HDR_CR2: No such file or directory
/Users/appleuser/Desktop/Switch.app/Contents/progress_bar.command: line 1237: syntax error near unexpected token `>'
/Users/appleuser/Desktop/Switch.app/Contents/progress_bar.command: line 1237: `exec &> >(tee -a "$(cat /tmp/DUALISO/path_1)"/LOG.txt >&2 )'
[Process completed]
The good is that I've scored a decent mint condition 13" MBP (mid-2010) from OfferUp.com. It came with El Captain pre-installed. :P
Upgraded to High Sierra (later I will do this trick (https://www.youtube.com/watch?v=v9sE_FhSdT8/trick) to get MoJave) and can already confirm to be spitting out Dual-ISO photos like normal via Switch.
Now I've dedicated this very vanilla 13" MBP to be only for compiling purpose. This old beast can sing it's praise while humming nicely!
I can't fathom mixing in with my personal stuff on a MBP together with compiling environments (don't ask why ;)) and now I should be golden going forward.
Happy Compiling! ;D
I uploaded a new version again. As you can see the tee command is breaking again. I guess you didn´t do this as suggested before?
chsh -s /bin/sh
Anyway. I took out the breaking command and uploaded a new version. Maybe works again.
Or are you trying to run cr2hdr? Only thing working with CR2 files atm is dragging them onto Switch and converting them to MLV files. Also explained before.
Quote from: Danne on November 08, 2019, 06:00:35 AM
I uploaded a new version again. As you can see the tee command is breaking again. I guess you didn´t do this as suggested before?
chsh -s /bin/sh
Whoops. Figured I was missing something. :/
Quote from: Danne on November 08, 2019, 06:00:35 AM
Or are you trying to run cr2hdr? Only thing working with CR2 files atm is dragging them onto Switch and converting them to MLV files. Also explained before.
Simply trying to run cr2hdr alone via Switch is what I'm aiming for. I just want to fast process Dual-ISO CR2's into DNG's (lossless) and be able to play with them in Adobe.
I find this workflow most efficient and while I still don't mind dragging into Switch for raw2mlv to use w MLV_App but doing Dual-ISO photo work is dreadfully slow.
Colors are beautiful and shouldn't be justifiable though.
To be able to go either way would be feasible. Jm2c.
Thanks for the quick update. I'll give it a try on Catalina.
Unless you update cr2hdr to work with 64bit architecture this won't work. Drag drop to mlv only atm.
By the way. After converting a CR2 to MLV file in Mlv App to dng it opens just fine in adobe camera raw:
(https://i.postimg.cc/qBtXB3Dh/Ska-rmavbild-2019-11-08-kl-07-06-18-png-scaled.png)
Also:
"One feature not in Switch yet is that raw2mlv will cat and output multiple files into one big mlv file if chained up in command line. In practice this means we can collect all dualiso CR2 files into one big mlv, import it into Mlv App and then export every dualiso file indivually into dng files. Works, just tested."
https://www.magiclantern.fm/forum/index.php?topic=20025.msg222180#msg222180
Quote from: Danne on November 08, 2019, 06:49:13 AM
Unless you update cr2hdr to work with 64bit architecture this won't work. Drag drop to mlv only atm.
Updated Switch. Here's a screen record while running on Catalina. Seems broken in both ways?
Guess opening Switch alone seems to show menu like normal and process but then it spits out 0 Bytes size files.
Also same if I were to drag and drop MLV to utilize raw2mlv then it shows 0 Bytes file size as well. What gives?
Gonna try take two on the 13" MBP (running MoJave) and will report my findings then. Thanks again @Danne!
Quote from: Danne on November 08, 2019, 07:13:12 AM
By the way. After converting a CR2 to MLV file in Mlv App to dng it opens just fine in adobe camera raw:
Wow. Did I read that, right? You took DNG files spat out from MLV_App (which settings did you use under export within MLV_App)?
Quote from: Danne on November 08, 2019, 07:13:12 AM
Also:
"One feature not in Switch yet is that raw2mlv will cat and output multiple files into one big mlv file if chained up in command line. In practice this means we can collect all dualiso CR2 files into one big mlv, import it into Mlv App and then export every dualiso file indivually into dng files. Works, just tested."
https://www.magiclantern.fm/forum/index.php?topic=20025.msg222180#msg222180
I would love this feature when it becomes avail. Sounds very exciting to be able to do this method. Definitely bad ass! 8)
You keep testing things I just pointed out is not going to work. Only drag and drop, hopefully you get it this time.
Also if files are acting strange, what to test first? Maybe skipping odd names on folders? Or what did you test when you noticed they came out 0 bytes?
Well, thanks for feedback anyway.
Any chance to see a Windows version.
Would be awesome to process Time Lapse Photo sequences processed, as a series of stills.
Sorry, mac only atm. About to push a first version of a dedicated tool for mac users. Hopefully some windows coder manage to at least compile the raw2mlv binary.
Hi. Sorry to be a booger but I just decided to try running CR2's (drag n drop) on Mac Mini running MoJave (10.14.6) and is now spitting out 0 bytes for the MLV files after being converted from CR2's (Both non and Dual-ISO's) and I've also tried re-installing Switch from OP just to be sure I'm running the latest.
*EDIT*
Again, I had to re-phase to your infamous slogan... I can't have any spaces within the Folder's name. Any way to fix this? ;D
I tend to have spaces in between words for certain photo projects. If not, no boggie. I'll just use ' _ ' instead when needed.
At least it works. Ha. Thank you always, @Danne!
Switch (nor any other apps) are converting my dual-iso CR2's, its putting them into the not_dualiso folder. (Catalina 10.15.1) Has anyone experienced this?
cr2hdr is 32bit only. Catalina is 64bit only. YOu can transcode CR2 files into MLV files with raw2mlv and then process these in Mlv App. Maybe even mlv_dump will chew them and spit out dng files for all I know.
Quote from: Danne on December 13, 2019, 09:21:17 PM
cr2hdr is 32bit only. Catalina is 64bit only. YOu can transcode CR2 files into MLV files with raw2mlv and then process these in Mlv App. Maybe even mlv_dump will chew them and spit out dng files for all I know.
Cheers!
New Switch version uploaded in first post
Updated cr2hdr to work with Catalina 64bit system only:
Commit
https://bitbucket.org/Dannephoto/magic-lantern_jip-hop/commits/91e06676ce736908b6cca7912329e65189a1f204
Thanks to @names_are_hard who took the time to find the erratic line.
Needs testing through, did not have time to do more than drag drop cr2 onto Switch.
dependencies
brew install dcraw
brew insatll exiftool
Dude... this is such a relief to be able to process over 40 CR2's of Dual-ISO from within the Menu in no time! 8)
No longer stuck with the drag 'n drop mode for raw2mlv only. It's SO nice to be able to choose one or the other.
Thanks, @Danne for the quick fix after @name_are_hard conquered the erratic line. BIG thanks for that!
Hi!
I wonder if it is possible to process dual_iso cr2 at half resolution to eliminate stripes using Switch?
Thanks!
Hi everyone, sorry if the question is a bit stupid, but I can't convert MLV to Prores only with Switch.
In this case I'm not interested into converting MLV to DNG, but only to Prores proxy.
I tried:
-Output is exactly where I want to save the converted Prores files;
-On main menu I add (p) for Prores output, then I add (12) for rec709, then (r) for run.
The result on target folder: NO Prores proxy file, but DNG files!
Then I tried:
-Same things as previous test;
-On main menu I add (m) for mlv_dump, then I add (19) MLV+proxy, then (r) run.
The result is nothing...
Then I tried:
-Same things;
-On main menu (o) then (08) then (r) run.
The result is a little window with "Claening proxys", but nothing seems happening (CPU not working...)
May you kindly help me? Thanks
Been a while since I used Switch. Noticed proxy exports has an issue. What works is the regular prores exporter. You could test Select (p) from main menu then select (04) for rec709 output.
If there´s an MLV it will first convert into dng files then start encoding to prores. Switch cannot pipe MLV to prores directly but created dng intermiediates to work from.
Thanks Danne (my hero!)
Quote from: Danne on May 12, 2020, 10:22:54 AMIf there´s an MLV it will first convert into dng files then start encoding to prores. Switch cannot pipe MLV to prores directly but created dng intermiediates to work from.
This is yet the answer I was looking for: actually I was searching for an app that converts MLV-to-Proxy without producing DNGs (for lack of space) and faster than MLV App.
Up to now, I've noticed that the only app that can convert MLV-to-Proxy a little faster than MLV App is MLVFS+Davinci-optimized-media. BUT... Davinci optimized media are sort of weird/folderish/internal files, and if I disconnect the MLVFS-virtual-drive then I lose the preview image on the timeline footage...
So... it seems MLV App is the only way to go at the moment... (but it takes 8 days for a basic-bilinear MLV-to-Proxy conversion...)
Do you know if there is a Mac app that can do the job via GPU? It should be a faster solution, right? Expecially for something like producing proxies.
Actually, there's built in mlvfs support in Switch. In main menu select (ml) Then select (A) to activate virtual dng files and go from there.
Thanks again Danne.
Quote from: Danne on May 12, 2020, 11:55:08 AM
Actually, there's built in mlvfs support in Switch. In main menu select (ml) Then select (A) to activate virtual dng files and go from there.
I had seen it. Unfortunately in my current case mlvfs seems not to be a solution because all the (too many) original MLV files are in a relatively old (and noisy) external RAID. I would be oriented keeping my external-draisine (dRAIDsine could be an appropriate neologism) "safely"
off during the looong time of the preliminary video-editing in Davinci I'll have to face. So I'd prefer having indipendent proxies on a faster (and smaller) internal drive. The only problem is the huge amount of time for the conversion... I'm on Mac, but I've read something about a Win software that should be very fast in MLV-MOV conversion since it's GPU-based (for proxy-things should be great). I'm hoping into something similar for Mac too... :)
For proxy workflow you will still need dng files in the end either virtually or physically exported.
For fast conversion in Switch you select these options in Prores output menu:
(https://i.postimg.cc/X7jbqsbF/Screenshot-2020-05-13-at-05-07-47-png-scaled.png)
(https://i.postimg.cc/BZc3Jdd4/Screenshot-2020-05-13-at-05-07-55-png-scaled.png)
Now select (04) or something else from prores4444 output. Since (07) is set to prores 422 and dcraw quality reduced it will export faster and produce smaller files.
(https://i.postimg.cc/15jjSB5y/Screenshot-2020-05-13-at-05-13-16-png-scaled.png)
Don´t see why you don´t want the dng folders if you are using a proxy workflow in resolve. If you want direct exports MLV to proroes simply use mlv app. You can start two or three Mlv app apps in prallell to speed up conversion. You could also erase dng files after prores files has been created. Or create proxies from mlvfs.
EDIT: Also fixed the proxy not working situation from before
Updated Switch. Included mlvfs workflow now has the latest pixel maps from dfort and also working with 1x3(anamorphic) mlv files.
Dear Danne, THANKS (really) for your guide and your patience!
Quote from: Danne on May 13, 2020, 05:18:03 AMYou can start two or three Mlv app apps in prallell to speed up conversion.
That's VERY interesting:
sorry I can image it's a dummy question, but how can I run 2 or 3 MLV App apps in parallel? Some script from the Terminal? Or via Switch? (Up to now I can simply run a single window of that app...)...Now I know: Script Editor :) :) :) Great advice, Danne, thanks a lot!
Quote from: Danne on May 13, 2020, 05:18:03 AMFor proxy workflow you will still need dng files in the end either virtually or physically exported. [...] Don´t see why you don´t want the dng folders if you are using a proxy workflow in resolve.
[I want to give you an asnwer about this, but I honestly warn all the people in the forum that the following section is boring, not-technical and unuseful, so
you can happily ignore it :)] Actually I don't plan to follow a regular proxy workflow in Davinci, but simply use a prores proxy version of my MLVs, because my current project in based on about 3500 MLVs that means around 34 TB stored in a single 8-drives external RAID6 thunderb.2 full as an egg, and in my 2nd external RAID I haven't enough free space to store all the converted-DNGs. Surely I could use MLVFS but it means keeping the 1st RAID always on, that's I'd avoid for various "safety" reason (first of all the content's safety). It's a huge project to handle for a newbie total-indie like me, mainly because I have NO screenplay: in fact the footage is a sort-of-documentary thing simply based on a concept, not on a story. My "freely-artistic" approach (clearly this is a non-business thing) is "discovering" the form of the film (the screenplay) directly from the "timeline-lab", where I hope to understand which (FEW!) shots to keep, and which others (many, I hope) to cut out. For the few "survivors", I finally will do the actual MLV-to-DNG convertions, and this selected DNG-footage will be placed onto a pretty fast internal 4-NVMe 8TB RAID (should be sufficiently roomy). Then I'll redo the editing following the former proxies-based editing. It's not actually a workflow... i.e. it would not have been strictly efficient for an eventual paid job ;)
Switch is already running multiprocessing 4 parallell exports so if you want dng exports you could use it. Should be fairly fast.
With mlv app you need to make 4 copies of the program and you can run those at the same time.
Dear Danne, I've made a test (perhaps could be useful sharing this):
My hackintosh has an old i7-6950X Extreme 3GHz 10-cores.
Two tests:
1. A single MLV App converting a list of 4 different 27GB MLVs to MOVs in about 37': CPU usage is 1750-1800% (Activity Monitor) and all the cores seem working equally (iStat Menus);
2. Four MLV App instances running in parallel, each one converting 1 single 27GB MLV to MOV in 19-20': each instance's CPU usage is around only 400% and all the cores seem working equally.
So, it seems there is a strange discrepancy: the CPU usage of a single App running in "solo" is more than the sum of the CPU usage of 4 apps running in parallel (so a single App seems to be more efficient because it use more CPU), BUT 4 Apps do the job in about half the time (even if using less CPU...)
It's a bit strange, isn't it?
Yes, pretty strange. Maybe you should run eight copies 8).
Also test proxyexports with Switch. Might work fast too. Proxy issue is now fixed by the way.
Uploaded a new version. Proxy exports now are way faster than before. Typically a 5minute clip will take around 1minute.
Also note that scaling down output will fasten exports even more. Watch out for speeding tickets...
Sorry guys, surely a stupid question: I've downloaded the last Switch and now, converting MOVs into DNGs, it deletes the MLV originals (the A_ORIGINALS folder is empty). I reset Switch to defaults, but it continues to delete the MLV originals...
How can I avoid that?
Thanks in advance.
MOV to DNG?
Maybe you didn't wait for the dng files to complete. In that case mlv file is still inside the folder with the dng files. Hard to say what's going on with so little information.
Hi Danne, sorry I meant MLV to DNG! I made various tests: MLV (dual-iso) to DNG. Running the task, MLV disappears almost immediately from its native folder. After the conversion is terminated, MLV is not in the A_ORIGINALS folder neither...
Never happened with old versions of Switch... It's strange.
Find that hard to believe. Please choose a short file and run a screen recording from start to finish. My own eyes have to see this.
If true this needs to be fixed asap.
Mea culpa, Danne: yes, the original MLV file is still in the folder of all the dngs. Anyway, what actually happens is that Switch creates the A_ORIGINALS folder, then immediately after starting the conversion it moves the original dual-iso MLV file into the dngs' folder. When Switch is near to have finished, it removes the A_ORIGINALS folder. Then arrives the "finish" window.
I believed that at the end of the conversion I'll have found the MLV into the A_ORIGINALS folder (like when I used old versions of Switch, if I remember correctly). Anyway, is it all normal?
Screen recording...
Quote from: Danne on May 31, 2020, 04:27:05 AM
Screen recording...
Here we are :)
https://youtu.be/dpOBfOc5seU
Try keeping the original name of the file. Renaming adding punctuations, white space or other strange symbols will often mess things up.
Test done, you are right: re-changing back the name to the original form Mdd-hhmm.MLV solves the issue: now the original MLV goes finally in the A_ORIGINALS folder. I tried importing into Davinci both DNG-folders: the one converted from the Mdd-hhmm.MLV file, and the other one converted from the renamed mess-mess.mess.MLV file. The result seems pretty the same (I can't notice any difference in preview, nor in scopes). But you wrote:
Quote from: Danne on May 31, 2020, 09:28:18 PM
Renaming adding punctuations, white space or other strange symbols will often mess things up.
And that worried me a bit: do you mean simply that original MLV will be placed into the DNGs folder? (That's honestly a minor problem.) Or do you mean that something in the DNGs themselves (code, list order) could be messed up? (That would be a pretty dramatic thing because I should rename my MLVs to their original name, that's a rapid thing in itself but implies loosing my detailed system of identification for all that amount of MLVs...)
Second question: I converted MLV to DNGs once in Switch (leaving all default), and once in MLV-App specifing Cineon tonemapping and Alexa gamut. I imported both results in Davinci: in RAW Panel I set both with BMD Film in Color Space and Gamma. Since I was using DNG files, I naively expected to see the same visual result, instead I noticed a huge difference. So, the question is simply: which is the "right" way of converting the MLVs? Should I specify a profile (BMD Film, since the target is Davinci?) or leaving all default (MLV-App's default gamut is Rec.709, but which is Switch's default gamut?)
Thanks a lot for your help.
You have to test and draw your own conclusions. For instance. What symbol messed up your workflow before? Thrn proceed to see if you can rename as you wish after this. If it looks ok, it probably works.
Quote from: Danne on June 01, 2020, 06:43:29 AMIf it looks ok, it probably works.
This last thing referred to the second question?
What happens in resolve has nothing with converting to do. Check google or create a new post if you can't find your answer in here already.
Quote from: Danne on June 01, 2020, 07:30:22 AMCheck google or create a new post if you can't find your answer in here already.
MY bad, sorry, I can't find in Google anything about how Resolve handles dng(from-mlv) with different tonemapping...
But actually my question was mainly oriented on Switch: which is its default tonemapping? Is there one?
And more generically: converting to DNG (which I'd do with Switch), which is the "most advisable" (if there's one) profile/tone to make the conversion? Or, being DNG, it's insignificant? (Here I mentioned Davinci only because, if it's insignificant, why I saw a huge difference in the same MLV converted in DNG with two different tonemappings?)
Another Switch specific: I saw it can handle CR2 for dual-iso or HDR. I shooted a normal timelapse in CR2s with MagicLantern. Can Switch handle/convert timelapse too?
No time to explain dng specifics, sorry.
Regarding timelapse sequencing I usually convert my CR2 files with Ilia3101 raw2mlv tool. I wrapped it into an app here:
https://bitbucket.org/Dannephoto/raw2mlv/src/master/
Download:
https://bitbucket.org/Dannephoto/raw2mlv/downloads/raw2mlv.dmg
When converted into mlv it´s just a matter of importing to mlv app and off you go.
@adjork DNG or any kind of raw has no tonemapping of any kind, ever, at all. What are you talking about?
If you saw difference it was for other reasons, please explain more.
Quote from: Danne on June 01, 2020, 09:55:55 AMRegarding timelapse sequencing I usually convert my CR2 files with Ilia3101 raw2mlv tool.
Fantastic, I didn't know that software. Thank you!
Quote from: ilia3101 on June 01, 2020, 11:41:49 AMDNG or any kind of raw has no tonemapping of any kind, ever, at all. If you saw difference it was for other reasons, please explain more.
Thank you so much for your kind reply ilia3101. Yes, as I knew DNGs should not have tonemapping, but I saw a big visual difference importing 2 dng-folders into Davinci. These 2 dng-folders are two different conversions of the same MLV: one made with Switch (leaving all default) and the other one made with MLV-App with Cineon tonemapping and Alexa gamut applied. I imported into Davinci both the dng-folders, and in RAW Panel I set both with BMD Film in Color Space and Gamma. I expected to see the same visual result, instead I noticed a huge difference: the one converted with MLV-App (with Cineon tonemapping and Alexa gamut) has midtones strongly brigther and less saturated than the one converted with Switch (default). Why? Why does Davinci interpret these 2 dngs in different ways? And which is the right thing to do?
Thanks a lot.
It makes no sense of what you are describing. Again, a screen record will probably reveal the answer. Also upload dng files, one from each program.
Now the problem is clear (not really the solution): the case of the MLV converted into two DNGs (Switch-default vs MLVApp-cineon) which are seen differently by Davinci, does make sense because it happens only when the MLV is a somehow faulty dual-ISO.
HERE (https://youtu.be/5QffZCwMPjk) the screen record.
And HERE (https://drive.google.com/file/d/15YK2B4lvH6xKhRs0KN2vJIN3J3sIUa9T/view?usp=sharing) the two first-frames of the DNG-folders.
And finally HERE (https://drive.google.com/file/d/1czdKUAc5L4ua24b_qzruXuLWPM8yn3ku/view?usp=sharing) the original problematic dual-ISO MLV.
Thanks a lot.
https://youtu.be/5QffZCwMPjk
There you go. With proper info all is clear as crystal.
Quote from: Danne on June 03, 2020, 06:26:20 AMWith proper info all is clear as crystal.
Well... not "all" actually: now it's clear that the problem is the faulty dual-ISO MLV, but it's not clear (to me) how to fix it (or IF it's possible to fix it...) Anyway, this is not directly a Switch problem, so I'll try to migrate my question to the proper dual-ISO topic.
Thanks again :)
Question about Switch vs MLVFS:
If I understand correctly, MLVFS should do on-the-fly what Switch does saving a new dng-folder. If that's true, the following two things should be identical: making an mlv-to-dng conversion via Switch, or copying&pasting a "virtual" dng-folder from the MLVFS' virtual-drive to another "real" place.
Is that right?
I'm asking this because I noticed that MLVFS maintains the complex naming structure I used for my MLVs (with dots and dashes), and Davinci seems handle it without problems.
Everything doesn't work everywhere. Just fix your naming structure or get your hands dirty and fix my code part.
Adding punctuation to a folder is never a good idea in general imo.
Of course Danne (and I'll never do it again in future).
But just to know: do you confirm that when you copy&paste a dng-folder from the MLVFS'virtual-drive to another "real" folder-place is exactly like doing a conversion with Switch (or MLVApp)?
No. Why would it. Three different programs, with code variations.
Hi Danne, thanks for you work!
I cant get the new cr2hdr implementation to work - when i drop a CR2 onto Switch the notifications say the cr2hdr shortcut is running and complete but no output is created.
I've downloaded the latest Switch and installed dcraw and exiftool and I'm running Catalina 10.15.5. Do you know why this is? Are there any other dependancies i need?
Thanks :)
Checked the log file?
Please upload an example CR2 file as well.
All dependencies are included inside Switch. It might reveal what´s not working in the log file.
EDIT:
Wait. I changed cr2hdr to an older version before. Probably breaking 64bit. Will try and fix it.
Ok, I couldn´t resist merging the latest from cr2hdr into the 64bit version fixed by @names_are_hard and it seems it is working. Needs to be tested:
Commit:
https://bitbucket.org/Dannephoto/magic-lantern_jip-hop/commits/72d1ff90759372d95126f5739947cb5f4272fcfa
As I am on a very slow connection I suggest @adamk71 download the binary directly and replace cr2hdr inside Content folder inside Switch and test again:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/cr2hdr
If working it will confirm @names_are_hard changes will work for later 64bit os systems on mac.
@a1ex, sorry, ml source and Switch need updating again :P
Just tried latest Switch and working perfectly, thank you!
Great. I have a few more changes going in but will have to wait for a better connection.
Confirmed latest Switch works flawlessly in Mac OS 10.15.5 (Hackintosh) & 10.15.3 (MBP). Thanks for the update, @Danne!
So I really love this! I used it today to convert some 5D3 RAW footage at work (just some stuff I was playing around with), and I was able to take 1080 footage, upscale, add sharpening and noise reduction, and the shots looked just like our Panasonic Varicam LT. A bunch of shots from this shoot will make it to air in our show.
I wanted to go home and do the same thing, but unfortunately I have a windows environment, and I haven't found a tool that can do the same upscale, sharpening and noise reduction as Switch.
Any chance this will ever get a windows version?
Sounds good. No windows version planned on my part. Did you try Mlv App maybe? Works cross platform.
I spent about 4 hours last night going through every program I could put my hands on, and unfortunately either the software isn't ass good (or reliable), or there is a learning curve I couldn't beat...
To give you a background, I spent maybe 2 hours at work playing with the different softwares, and Switch was by far the best outcome. There is a slight learning curve with the menu system, but once you find settings that work, it's a breeze.
I used MlRawViewer in the past, and it's a good solution for converting, but it doesn't upscale and include the denoise and sharpening options
I used Mlv App last night and I tried about 5-6 different Sharpening and denoise options, but none of them kept the details, they just muddied the image up, and smoothed out edges. Needless to say, I couldn't get the results I got on Switch. In addition, it froze doing a batch at about 12%, so I consider that unreliable.
Switch essentially gave my 5dIII new life, because the 1080 RAW image that came out of it could be upscaled and could sit side by side with an actual 4K image out of a production camera. I'd really like to have the opportunity to work with those files or anything I shoot at home. Unfortunately all my processing power is the PC ecosystem.
The only other options I have is to try Linux, but I'm not at all familiar with that ecosystem, so I would have to get over a steep learning curve...
Hehe, cool. Yes, I like the dcraw + ffmpeg options too in Switch. Only way in windows would be trying to get it working in linux subsystem but that is a pain really.
I tried using this on Catalina, but maybe I am missing something. Can Switch be used to convert DNGs back to MLV, or to DNG again, but apply the Force Vertical Stripes processing upon output? I can't seem to figure it out.
Thanks!
Todd
Please don't double or triple post. I already answered you in another thread.
Quote from: Danne on August 13, 2020, 08:09:17 PM
Please don't double or triple post. I already answered you in another thread.
Understood, but they were for 3 different solutions: MLV App, raw2mlv, Switch. None of them work on a few of these DNGs, unfortunately.
Unable to download Switch. Bitbucket link tells that "Repository unavailable" :(
PS Maybe someone can share previously downloaded app?
Download full repo here, including a Switch version:
https://bitbucket.org/Dannephoto/switch-git/get/b832bc4315e8.zip
Is there a M1 chip optimized version?
Didn´t test this on M1 chip. s it working?
Hi there! I recently got a new M1 macbook, and have been searching for a GUI-less way to convert MLV files to dngs on the command line. Previously I'd been using a precompiled mlv_dump executable that ran only on Linux, and I had to set up a crazy workflow using Docker containers to get the output I needed. Searching around the internet for a better way led me to this thread.
I realize this is only a small component of what Switch is meant to do, but may I ask how you're compiling mlv_dump on a mac? I'm finding it hard to find the source code for this utility, and not a lot of info for how to get it working on an M1 mac.
I'm happy to try to contribute if I can (I'm only medium-adept at C coding), fundamentally I'm just looking for a way to do this one small little task.
Thank you!
I'm using a MacBook Pro with Intel silicon (not M1), and I have tried this on macOS Monterey 12.1, and it doesn't work. I'm using the .app from the .zip in the link a couple posts above. I am also using the latest ML lua fix build on a 60D (magiclantern-lua_fix.2020Dec28.60D111) if that makes any difference.
Here is the log:
ls: *.RAW: No such file or directory
ls: *.RAW: No such file or directory
-: line 112: /Applications/Switch.app/Contents/raw2dng: Bad CPU type in executable
File not found: /tmp/DUALISO/what_cam_000000.dng
File not found: /tmp/DUALISO/what_cam_000000.dng
File not found: /tmp/DUALISO/what_cam_000000.dng
rm: /tmp/DUALISO/what_cam_000000.dng: No such file or directory
ls: /tmp/DUALISO/what_cam_lock: No such file or directory
ls: *.MLV: No such file or directory
ls: *.mlv: No such file or directory
-: line 126: /Applications/Switch.app/Contents/mlv_dump: Bad CPU type in executable
ls: *.MLV: No such file or directory
ls: *.mlv: No such file or directory
-: line 135: /Applications/Switch.app/Contents/mlv_dump: Bad CPU type in executable
I think things just need to be updated to work with whatever changes happened in Monterey, but I do not know enough to guess, really.
Hope this helps.
Quote from: boredmanicrobot on January 12, 2022, 06:38:23 AM
I'm using a MacBook Pro with Intel silicon (not M1), and I have tried this on macOS Monterey 12.1, and it doesn't work. I'm using the .app from the .zip in the link a couple posts above. I am also using the latest ML lua fix build on a 60D (magiclantern-lua_fix.2020Dec28.60D111) if that makes any difference.
Did you try this after disabling SIP?
Hello, I've tried switch on my m1 Mac 💻 with Monterrey. I must to inform something: cr2hdr work just with cr2 files, and raw2dng doesn't work, so you may use adobedng converter, with cr2 Dual ISO pictures you must to force Dual ISO and disable dng compression. (adobedng needs to be installed) then just run it.
First install of Switch (2023Jan25) on Ventura and the first time I fire it up I get this just after selecting folder with MLV files:
(https://raw.githubusercontent.com/vastunghia/vastunghia.github.io/main/ss.png)
That is: "Menu.command" can't be opened because (null) is not allowed to open documents in Terminal
Seems to be a pretty common issue with Ventura. There are a few workarounds suggested over the net. Will try a few. Just wanted to signal this.
Ciao
Sergio
How do we uptade the focus pixel maps in Switch ? I've copied the files provided by Bilal for crop mood in the mlvfs folder of Switch but I still see the focus dots in Resolve. Am I missing a step or an option that has to be activated when using mlvfs in order to get rid of thes focus dots ?
There should be a place for maps in there somewhere? They are not found in there?
The files are in the that folder but I still see the dots on the image. Is it supposed to work automatically (like in MlvApp) or is ther an option to activate in the Terminal before running Switch ?
Should work if they are named correctly. Not dualiso right?
Do the maps work in Mlv App?
Hi Danne,
Sorry for responding months later.
The problem occurs on files shot in 3x3 1736x738 @57fps.
I've put the same fpm files in the the mlvfs folder of the Switch app than those present in the MLV App folder. It's working for the other files in different modes, but not for these slowmo shots.
Int hte terminal it says :
Loading focus pixel map '80000331_1672x2068.fpm'...
Loading focus pixel map '80000331_3144x1336.fpm'...
Loading focus pixel map '80000331_1808x766.fpm'...
I guess the last one is the one needed for 3x3 1736x738 ?
Hard for me to know what is going on. Are those files working exporting in for example Mlv App?
You could also test exporting to dang with this
https://bitbucket.org/Dannephoto/switch_mini-git/downloads/
Switch mini. Works better maybe?
What I'm after is if it works with any other program or not.
I've downloaded Switch mini and it doesn't work. It works with MLV App.
Here are two dng exported from the same mlv file, once exported with switch mini and once exported with MLV App :
https://drive.google.com/drive/folders/1pJmH8oWDFI3w088MRxfQOJnm5Bnoyz4K?usp=sharing (https://drive.google.com/drive/folders/1pJmH8oWDFI3w088MRxfQOJnm5Bnoyz4K?usp=sharing)
Could you upload a Mlv file and your fpm file?
Ok I will, but how do I know which fpm file is the one needed ?
Here is a screenshot of the fpm files, left is in MLV App, right is in Switch.
https://drive.google.com/open?id=1qrHKqZxBMYXMJTsHNAI3HJ9ItJP-otR5&usp=drive_fs (https://drive.google.com/open?id=1qrHKqZxBMYXMJTsHNAI3HJ9ItJP-otR5&usp=drive_fs)
I've actually copied the fpm files from MLV App over to Switch. That's why there is a folder called "_old fpm" in the "mlvfs" folder of Switch, I've kept the initial fpm files. That's perhaps a problem ?
You can see which fpmi is active with your Mlv file in mlvapp. Go from there.
I just bought an used M1 mac mini and I get the same results.
In MLV app 80000331_1808x766.fpm is used for that file.
In Resolve, using Switch, I see in the Terminal window that the same map 80000331_1808x766.fpm is loaded but I still see the focus dots in these slow-mo shots.
Look. To be able and help I need a short mlv sample and the actual map file uploaded somewhere. Not a dng, not screenshots or mores descriptions of the issue.
That is available here : https://drive.google.com/drive/folders/1pJmH8oWDFI3w088MRxfQOJnm5Bnoyz4K (https://drive.google.com/drive/folders/1pJmH8oWDFI3w088MRxfQOJnm5Bnoyz4K)
How did you create your focus pixel map? It is very small in size.
Easiest way to produce a map is by grabbing something nearby in Ml App and then rename it. I did and it works in both Switch and Mlv App.
https://bitbucket.org/Dannephoto/switch-git/downloads/80000331_1808x766.fpm
Hi Danne,
Thanks for the map and your time.
I got mine from Bilal's bitbucket (https://bitbucket.org/bilal_fakhouri/crop-mood-builds/downloads/Crop_mood_focus_pixels_maps_-_EOS_M.zip (https://bitbucket.org/bilal_fakhouri/crop-mood-builds/downloads/Crop_mood_focus_pixels_maps_-_EOS_M.zip))
I don't know how to do pixel maps.
I'll try with your's tonight.
Thanks Danne, problem solved !