Switch for macOS Catalina/Linux (former cr2hdr.app)

[Danne]

Actually, there is nothing wrong with the file. The previewer in mac is giving pinkish output but when opening the file in acr or transcodeing the file to ProRes in cr2hdr.app everything works as expected. Be aware that using proxy+MLV will create a file which has no focus pixel map that is currently working with that file. If you want something useful with eosm use crop_rec for the time being.

[Teamsleepkid]

that file was 14bit regular magic lantern crop footage. not 3x3 720 or mv1080. and not recorded with proxies. so if you run it your dng folder doesn't come out pink? it comes out fine? I'm using resolve to open the dng folder after processing with cr2hdr. yeah the prores 4444 and proxies are good. anyway thanks. just can't seem to find a real good post processing workflow since 10 bit and 12 bit came out. i either get focus pixels and good color or no focus pixels and whack color.

[Danne]

Update
Disregard what,s written below. Just tested unified mlv_dump and the files where exported normally with preview looking good. Problem is with my version of mlv_dump. Will dig further.


*Update 2
Problem solved. The color matrice for eosm was totally off. Seems maybe off in mlvfs as well. Will upload a new version soon.

New version uploaded.


Ok, tested some more and there is something funny going on with the files on the eosm that you sent. Now this actually happens over here too on my eosm and since I only opened my files in acr and in cr2hdr.app lately I didn,t know that it affects resolve and preview there. It seems there is corruption of some kind in the preview image in the dng file.
It would help if you could film a sequence that you know causes corruption and then try film the with the same settings with an older build for the eosm. If you find a build with corruption free footage you can narrow down which build is causing the corruption. Or maybe the problem lays elsewhere. Hard to tell.
Older builds here.
https://builds.magiclantern.fm/EOSM-202-all-builds-changes.html

You can post your findings here.
http://www.magiclantern.fm/forum/index.php?topic=9741.1175

[Teamsleepkid]

tried a 14bit file looks super clean! going to try 10 bit and 12 bit next. awesome! thanks

[steeth]

Hi.

I had issues with some random upper case .DNG frames inside a regular .dng sequence, which caused Resolve and AE to think that the sequence was corrupt. After the latest update when Danne added the force disable dualiso command, this was fixed, and I haven´t had that issue since then after doing several test shots.

But now I have another weird issue?! For some reason that I don´t understand, sometimes I get a split .mlv file, even thou I have disabled "skipped frames" inside of ML in my 5D mark III running 1.2.3 with the latest build. Meaning for about 1 file per card, usually one of the last files recorded on the card before it fills up (I have reserve card space enabled, as well as the EXFAT option checked and the card is formatted as EXFAT) at the end of the recording, I get those .M00 .M01 .M02 etc split .mlv files for some weird reason? And for some reason, the CR2HDR.app doesn´t convert those files properly and skips frames for some reason?

Look at this picture from the bin inside of Resolve. As you can see, all of my converted .mlv files, converted with the latest build of the CR2HDR.app, is shown as clips as ususal, except for the last clip that you can clearly see is not recognized as a clip, but a folder..



Inside this folder is as you can see in this next picture, the .dng sequence but with some missing frames, making Resolve or AE not treat it as a sequence of course since it has missing frames.



So I check my .dng folder with all the frames extracted by the latest CR2HDR.app, and as you can clearly see, there are missing frames here:



Just by being curious, I tried converting the same split .mlv file with another piece of software I have installed on my computer that I don´t use any more after reading here that it´s not supported by ML, and for some reason it converts properly without any issues, and I can see the clip as a clip inside Resolve:



And by looking in the converted .dng folder here, you can clearly see that there are no missing frames in the same exact .mlv file, just converted by a different piece of software:



So obviously the camera has recorded all the frames, as they apparently exists within the .mlv file, even thou it´s split. But the CR2HDR.app for some reason skips some frames, rendering the footage useless as it doesn´t come out right and the audio is messed up and doesn´t line up or sync when getting these missing frame sequences.

Has anyone else experienced this issue or know of a fix? After I heard that the other piece of software I used to use to convert my .mlv files to .dng sequences was not supported by ML, I obviously stopped using it instantly and searched for another way of converting my .mlv files on a MAC. And the "easiset" one I could find since I´m not a programmer or coder or have any knowledge whatsoever of using the terminal on a MAC, was the CR2HDR.app. I have no skills nor any knowledge of how to code, program or use the Terminal since I´m only a filmmaker.

I´ve tried both the regular latest 14 bit build, as well as the experimental 10-12 bit build, and it´s the same issue with missing frames on split .mlv files no matter if I use 14, 12 or 10 bit.

Anyone has any ideas, that would be deeply appreciated.

Kind regards

[Danne]

@steeth
Could you test this version?
https://bitbucket.org/Dannephoto/cr2hdr/downloads/cr2hdr_test.app.zip

[steeth]

Will do and will report back after testing. Thanks a lot Danne.

[steeth]

Hi again Danne. Just tested the version of the app you posted, but unfortunately it behaves the same way.. When trying to convert spanned .mlv files (M00, M01 etc), it skips frames during conversion for some reason? See this picture of the mlv file I try to convert to .dng:s. It´s a folder inside Resolve and not a clip, and you see all the missing frames. And now there wasn´t any audio extracted either for some reason?:



And here is a part of the .dng folder where you see the missing frames again:



And again, I converted this same exact .mlv file with the other piece of software I have installed that is not supported (Rawmagic), which I again of course stopped using as soon as I found out it was not supported by ML, and it works perfectly in both Resolve and Ae, has all frames intact and also the audio extracted and synced:



So there´s obviously something strange going on? It´s very unfortunate that this happens as I have no control over when my camera makes these spanned files. I have my ML menu set to EXFAT, no spanned files across the cards and not to allow skipping frames, and reserve card space is checked.. And for some reason the very last file I record to when my card fills upp and the recording stops automatically saying "Recording stopped. Camera/Card really EXFAT?), always turns into a spanned file... Since there is no way of knowing when the card is about to be full since the space left on the card doesn´t update on the screen after each recorded clip, it´s completely impossible to know when the card is filled up until the camera stops recording by itself and gives this message. The only time the space left on the card is updated, is by restarting the camera, which is of course impossible to do between each take as it´s not practical.. So does anyone know why ML makes these spanned files when the camera stops recording by itself? And is there a fix for it so it doesn´t do this?

If you like to download and check the same .mlv file I just tried to convert, please let me know and I´ll send you a download link to the files.

Thanks again for your help, and I hope this is solvable as I of course don´t want to use an unsupported piece of software to convert my .mlv files.

[Danne]

Downloading is slow. Hard to diagnose but maybe you filled your card up and some MLV safe guard stop is trying to save the content or it could be some issue related to exfat and MLV handling. I,d stick to fat32 but not everyone agree on this.
If I manage to download the file I´ll check it out.

[adrjork]

Hi everyone,
I'm pretty new as MagicLantern user, and up to now I admit that I simply used RAWMagic for converting mlv-RAW (no audio) into DNG. RAWMagic did the job A) maintaining the original white balance, B) fixing the vertical stripes, C) untouching the original shot's color curve. This is what I need. Because unfortunally, since I've changed my computer to a Sierra Hackintosh, RAWMagic stopped working (crashing on startup).

So I'm searching for a new app that does the same things. DeafEyeJedi said to me about Danne's cr2hdr for Sierra. I tried to open it, but it seems a lot articulated for me, so I'd need a very basic guide, please: as I said I simply need to convert mlv-RAW (no audio) into DNG for Davinci, maintaining the original white balance, fixing the vertical stripes, and untouching the original shot's color curve.
So, please, tell me if I'm wrong:
1. I select the folder with dual-iso and also NOT-dualiso .mlv (no audio) footage, because the app works for both, right?;
2. In order to obtain CinemaDNG output I must choose (m) mlv_dump, right?;
3. Having NON-dualiso footage, I choose (10) disable dualiso automation, right?;
4. NOT setting the black/white levels, and simply having Adobe DNG Converted installed, I will obtain a DNG conversion with the original white balance of the original shot, right?;
5. I don't understand (07) disable stripes etc. If NOT chosen, the app will always correct the vertical stripes issues of the original shots or not?
6. Is it necessary to add (01) no chroma smoothing to obtain an original-like conversion?
7. My doubt about the curves remains: does the app maintain the original curve of the original shot, or does it set a custom curve based on MlRawViewer setting?

Please, kindly confirm if I'm right or wrong.

Thanks everyone really much.

[DeafEyeJedi]

1. Correct.
2. Correct.
3. If that's the case, then yes.
4. Yes, but also keep in mind depending on which DSLR you use you may want to push white levels higher for more highlight recovery for 5D3.
5. Correct as the app does this automagically.
6. Why not try both and see which suits your needs.
7. Not sure if I understood this one correctly re: MLRV's settings? I'm pretty sure it's as normal as it can get especially with all the latest bells and whistles w raw2dng, cr2hdr, mlv_dump, etc...

[Danne]

@adrjork
The way you are gonna use it it,s very easy. After downloading you have to fix the gatekeeper issue

To supress gatekeeper hold ctrl button down(macOS Sierrra) while opening the application the first time. You can also change permissions from within privacy/security settings.

Once this is done I would do following

1 - Drag a folder containing MLV or RAW files straight onto cr2hdr.app
2 - Choose (r) from the main menu. (Correct white balance and colormatrices will be applied as well as support for Davinci Resolve)(Thanks Bouncyball, dmilligan)

The option to disable dualiso automation isn,t necessary. This option came to be when a user had one or two issues where cr2hdr thought a non dualiso file were dualiso. But if you want to stay sure just select this and also any other setting in mlv_dump you need.
Generally don,t change white and black balance. They are optimized already.
a1ex Vertical stripe correction code is applied as default.

[adrjork]

Thank all of you SO much men! You save my day. And I'm honored of using your brilliant piece of code.

1. About my question on MlRawViewer's curves: this app let you choose a color curve (linear, log-C, sRGB, etc.). I think I prefer to make all the color changes later in Davinci, so I suppose it's better for me in this step to maintain the original curve of the original shot (right?) But which is the original curve? I use a Canon 5D3 and I've simply read in the Canon menu that the color space is set to sRGB: does this mean that the original curve is sRGB?

2. Anyway, if I simply use MlRawViewer as a viewer (before converting with cr2hdr4Sierra) no curves should be applied to the final conversion (later with cr2hdr4Sierra), right?

3. Now let say that I have the same basic purpose (mlv-RAW without audio to DNG with original white balance, orig. curve, removing vertical stripes) BUT this time with dualiso footage, I simply avoid to select (10) disable dualiso automation, and that's all, right?;

4. Can this app convert also dualiso photos or only dualiso videos?

5. Do you confirm that I have simply to keep Adobe-DNG-Converter installed, without touching/setting it?

6.

keep in mind depending on which DSLR you use you may want to push white levels higher for more highlight recovery for 5D3.

I use 5D3, so which is a good value to set this option?

Again, thank all of you SO much.

[Danne]

Welcome to the world of scripting

1. DNG files can be decieving looking like sRGB or rec709 and still contain a lot of not showing information. Since files will be 14bit (or 10/12) you can compress the signal to log and you,ll see the extra information that is still inside this raw file. logC, C-log, cinelog, cineon etc. However your question about changing any canon meny setting won,t affect your raw dng files and not the curve at all on the dng file.
The easy answer is. Process to dng files and the bring your folders to DaVinci resolve or wherever and continue from there.

2. If cr2hdr4Sierra means cr2hdr.app MlRawViever is only for previewing purposes. If you export with cr2hdr.app to ProRes files you can select different log settings from the Prores menu (p)

3. If you skip the disable dualiso automation cr2hdr.app will automatically detect if the file is a dualiso file or not and process accordingly. Means you can mix dualiso files with non dualiso files.

4. Yes, this was the main purpose with this app from the start. Select (d) from the main menu to check settings in cr2hdr or simply select (r) if you have CR2 files in your folder. It will find you files and process them. It,s only when it comes to CR2 files you benefit from having adobe dng converter installed.

5. Yes

6. Don,t change the settings, they are optimized already


If you want to have some real fun you can fill a folder with all your files(MLV,RAW,CR2,mov etc) then drag the folder on cr2hdr.app and select (a) for full auto mode and lastly hit (r) to run cr2hdr.app.

[adrjork]

Dear Danne, you are really kind: your answers are very complete. Thanks for your time.

I think I understand all.
Perhaps just a couple of doubts on the 1st question about DNG and curves:

1. If I understand correctly, mlv-RAW 14bit files contain more information than visible via sRGB or rec709, so to see the extra information I have to move from sRGB/rec709 to a more convenient logC/Clog/etc. Is that right?

2. If so, MlRawViewer gives the possibility to change the curve of the mlv-RAW files, so with that app I can export logC-DNG or Clog-DNG etc. If I understand correctly, this is a way to handle - later in Davinci - a DNG full of information (if compared to sRGB-DNG). Another option is simply to convert mlv-RAW into DNG without touching the color curve, then import the DNG in Davinci and only at this point choose BlackmagicDesign as Color Space (in Camera Raw) to recover more information and start to grade. Then my question is: which is the best method to handle more information in Davinci? Converting mlv-RAW into a log-DNG (changing the curve and converting with MlRawViewer), or simply convert mlv-RAW into DNG whitout any curve change (with cr2hdr) in order to apply the Blackmagic Color Space in Davinci? (I think this is a workflow question.)

3. Very last question: with cr2hdr.app, the 14bit mlv-RAW is converted into a 16bit DNG, right?

Super thanks.

[Danne]

1. DaVinci resolve handles highlight recovery one way and adobe camera raw in another. You can check this for example
http://www.magiclantern.fm/forum/index.php?topic=13512.msg172443#msg172443
When shooting raw we want to pull back as much information possible in the file to work with. When viewing the actual dng file it shows in a viewable display referred format which already clipped highlights. But that extra info is still in there once you start adding logs or fiddling in adobe camera raw, resolve for instance which can handle 14bit files.

2. You don,t alter any curve info in the dng file itself. The metadata inside the dng file which resolve, ACR, premiere will read and use when applying white balance, white level color matrices etc not any log adding. When you add logs in MlRawViewer you are suppose to export this look into a prores file. If you export to dng logs are not included.
If you process to dng files it won,t matter if doing it with mlrawviewever or mlv_dump except that mlrawviewer compresses the dng and it won,t work with audio embedding in DaVinci resolve like with cr2hdr.app.

3. The files are still 14bit when using cr2hdr.app. But there is a version which outputs 16bit. Those two extra bits are not giving any extra info so you know.
https://bitbucket.org/Dannephoto/cr2hdr/downloads/cr2hdr_steroids.dmg
That version of mlv_dump is coming from Bouncyballs heavy work on porting dmilligan,s mlvfs code into mlv_dump
http://www.magiclantern.fm/forum/index.php?topic=18975.0

[adrjork]

It's all very clear now, thanks a lot Danne!

Just curious: about 14bit-DNG from cr2hdr.app, you say...

The files are still 14bit when using cr2hdr.app. But there is a version which outputs 16bit. Those two extra bits are not giving any extra info so you know.

My question is: if the two extra bits adding nothing, why making a specific version of the app that outputs 16bit-DNG? Having 2 bits more with no information is useful for what?

Thanks again.

[Danne]

New version up. Download in first post.
- Added support for RAWC metadata in mlv_dump.
- Removed hardcoded temporary fix

More about RAWC here
http://www.magiclantern.fm/forum/index.php?topic=17021.msg181639#msg181639

@adrjork
At some point premiere wouldn,t work with 14bit files but with 16bit I think. There are other reasons as well I,m sure. As long as the cdng tags are represented 14bit works just fine.
About white level and resolve. Maximum white point 14bit is 16383 and that is what 5D mark III is set to and this usually is good for bringing home most highlight information. This could however affect how highlights are handled in resolve. especially if adding different highlight recovery methods. If this seems to be looking strange you could try setting white level to 15000 in mlv_dump settings instead.

[adrjork]

I understand. Very clear as usual, Danne.

And cr2hdr_steroid (the 16bit version) has the same functions as cr2hdr.app (14bit version)?

Thanks a lot.

[Danne]

I would take that question on the relevant thread here:
http://www.magiclantern.fm/forum/index.php?topic=18975.0
I havn,t updated to very latest in steroid version yet so I would stick with 14bit for now.

[DeafEyeJedi]

Thanks Danne for adding RAWC metadata support as well for removing the hardcoded temp fix. About friggin' time and seriously I had no idea how much more vital this app could get. Just wait for the steroid version and it's just going to be a magnificence piece of an apparatus in the making and most of all it's become an essential part of my life for everyday needs when it comes to ML as a whole.

This is purely one of the greatest gift a person could ask for, seriously no joke in here ladies & gentlemen...

If you want to have some real fun you can fill a folder with all your files(MLV,RAW,CR2,mov etc) then drag the folder on cr2hdr.app and select (a) for full auto mode and lastly hit (r) to run cr2hdr.app.

 

[togg]

Hi Danne, thanks for the software that I'm just discovering. I've noticed that it relies on MlRawViewer as well. I have some RAW files that have a lot of hot pixels and sadly there's a bug in baldand software that prevents the correction from sticking in the exported dngs (https://www.magiclantern.fm/forum/index.php?topic=9560.1725). In my opinion is the only big bug that mlrv has and if we still need it have it solved would be a big help.

Just throwing it here, if you don't have time to check it or a thing like this is too complex (I really don't know) forge it

[Danne]

Can you post a sample mlv or RAW with hot pixels?

[Danne]

Merged mlv_dump compressed_raw code into a version of cr2hdr.app(mlv_dump). Feel free to test compressed MLV files and other things around a1ex latest code.
http://www.magiclantern.fm/forum/index.php?topic=19300.msg182201;topicseen#msg182201
http://www.magiclantern.fm/forum/index.php?topic=18443.msg182186;topicseen#msg182186


cr2hdr_4k.app
https://bitbucket.org/Dannephoto/cr2hdr/downloads/cr2hdr_4k.dmg

[beauchampy]

Thanks so much @Danne

Not sure if its me, the new crop module, or cr2hdr but my first test of 1920x1080 48p resulted in pink clipped highlights.