Dual ISO - massive dynamic range improvement (dual_iso.mo)

[Filanto]

I'm trying to find an active link for the software that converts dual iso to a useable format for a MacBook. So far all the links are dead minus the Lightroom plugin,  which I do not have and don't want to pay $120 a year for.

[Danne]

https://www.magiclantern.fm/forum/index.php?topic=15108.0
Drag a cr2 file on the icon. Be sure to get pass gatekeeper on first time install.

[Levas]

I'm having a problem using dual iso with crop_rec presets on the 6d.

I've recorded some clips with dual iso and my own made crop_rec presets, but there is something weird going on.
Not all frames are exactly the same, colorwise.
The difference is not very apparent in full scale 1:1. But you can see a slightly hue difference, some pictures are slightly warmer, colors are slightly different.
This is how the thumbnails look in rawtherapee, these are unprocessed dual iso files, the thumbnails don't represent the real colors seen in the 1:1 picture, it's exagerated, but it gives a hint there is something going on every other frame.
This was recorded with 2480x1396 at 25 fps preset dual iso 100/1600


Now when I use my 2688 x 1296 at 25 fps preset, I've get something very alike, but it happens 2 frames and then switches for 2 frames, same is visible in da vinci resolve in the thumbnails:



It is very apparent in the thumbnails of non processed dual-iso movie files. When looked at the image 1:1, you can't hardly tell the difference.
The color hue is slightly off or something.
checked black levels, white level and stuff with exiftool, but there is no difference in the files 

Tested dual-iso with plain 5x zoom mode, no crop_rec preset and nothing weird going on, all thumbnails are the same, no difference 

So the problem is when using the crop_rec presets on the 6d. The presets are not exactly 25.000 fps is it a timing problem ?

Anyone any idea what is going, and maybe how to fix it ?

Here's a 100mb sample MLV with 30 frames:

https://drive.google.com/open?id=1JiQVH6VEZCAf23QkihsHuWm5rUBzh9NZ

[Levas]

Ok, thank you all for your massive help   

Dived some more into the problem described in the post above. And it appears to be related to the B-timer, you can fix the above within 4 units of the B-timer setting.

I can't imagine the 6d is the only cam that had this problem in the crop_rec presets.
You can check your presets, if you enable dual iso and a preset, the ML-preview should give a image with static dual-iso line/bars...if the dual-iso lines/bars move around...your preset is not 100% accurate for dual-iso, you can fix this by adjusting the B-timer, should be right within 4 units.
A good preset gives static dual-iso lines/bars in ML preview.

Gonna post an updated crop_rec module in the 6d topic now.

So be sure to check out your presets and see if you have static dual-iso lines/bars with dual-iso.

[Danne]

Interesting. The slight hue difference issue has been discussed from time to time. So if lines are correct output should be too? Gonna check into this one some more on my eosm when I can.

[Levas]

Never noticed any problems with the presets without dual-iso.
But I can imagine that it also gives minor, not much visible issues there (never noticed them actually  )
The processing of the dual-iso files with cr2hdr probably makes the difference more visible.

[Danne]

Probably only affects dualiso files and in post processing.

[Danne]

Pretty cool. you can actually see timers go into static lines while changing a-timer on the fly.

[Danne]

Unfortunately I get bad side effects also when I set my preset to static:
https://www.magiclantern.fm/forum/index.php?topic=20025.msg226701#msg226701

In this case I was actually getting more consistent results with how the a-timer was set before.

Edit: I suspect 10bit file issue. I better jump off this wagon before the can of worms is reopened...

Edit: tested 12bit, already flicker free. Stay off 10bit on eosm while doing dualiso.

[2blackbar]

WOuld dual exposure be more impressive ? IS that possible so instead of different isos , it captures different shutter speeds ? FRom over 1000 to 40000000000 theres kinda no difference in shutter speeds so that might look fine ?
OR am i lame and late and its already been done ?

[adrjork]

Hi, I'm migrating here my question from other topics (Switch and MLVApp) since I've found out that my problem are my somehow faulty dual-ISOs. After the conversion with Switch I can clearly notice the horizontal lines and an heavy flickering. I've tried also MLVApp and the result is even more weird: horizontal lines, flickering, but also bluish tint and vertical lines. The funny thing is that if I set a tonemapping in MLVApp, even if I'm converting to DNG, the tonemapping seems influencing the final DNG, so that in Davinci I can see the two DNGs (from Switch and from MLVApp) as completely different!
Images speak louder than words, so HERE the screen record of the two conversions.
HERE the first-frames of the two DNG-folders.
And finally HERE the original problematic dual-ISO MLV.
I'd like to know how to fix my dual-ISOs (and IF it's possible to do that).
Thanks really a lot.

[DeafEyeJedi]

Thank for sharing your videos of what you were experiencing re: Dual-ISO. Now I would like to ask which build was this shot with?

Also would you mind sharing some short MLV samples if you still have them. Would be nice to have a play with. Appreciated @adrjork!

[adrjork]

I would like to ask which build was this shot with? Also would you mind sharing some short MLV samples if you still have them.

Thank to YOU DeafEyeJedi and to all the guys here for your kind support.
My build is the old Nightly.2017Feb12.5D3123 (it's so old because I started a project in 2017 and it's still in progress...)
I haven't “short” original MLVs (all my MLVs, expecially dual-ISO MLVs, are 1 minute long and up). What I can do is re-linking THIS 6GB file, and adding 3 other “shortened” MLVs (I shortened 3 long MLVs with MLVApp hoping this is the correct workflow: I uploaded and cut-out each MLV at frame 100, then I un-checked the RAW Correction panel, then I exported DNGs uncompressed, then I used “Transcode and Import” function). HERE the link for these 3 short dual-ISO MLVs.
Thanks for your help, I deeply appreciate it.

[masc]

The funny thing is that if I set a tonemapping in MLVApp, even if I'm converting to DNG, the tonemapping seems influencing the final DNG, so that in Davinci I can see the two DNGs (from Switch and from MLVApp) as completely different!

I can guarantee that tonemapping is not applied for DNG export in MLVApp, also if dualiso is used. Whatever you did, it must have another reason. But this is 100% impossible.

There is no unique rule how to preprocess a dual iso picture. There are different algorithms out there, and even if you change just a tiny little thing on such an algorithm you'll get a different result. The current algorithms are made for current builds. So if you use an old build, you should have more luck with an old processing tool.

[yokashin]

.dng files are different ...

M11-1011_1_2018-09-11_0001_C0000_000000.DNG
Temperature 4450
Tint +30

M12-1011_1_2018-09-11_0001_C0000_000000.dng
Temperature 5150
Tint +17

Photoshop on Windows 10

[masc]

.dng files are different ...

M11-1011_1_2018-09-11_0001_C0000_000000.DNG
Temperature 4450
Tint +30

M12-1011_1_2018-09-11_0001_C0000_000000.dng
Temperature 5150
Tint +17

What do you want to say with that numbers? These are two different clips. Many values in metadata are different. Who knows how ACR interprets metadata... Camera's WB values are not saved in the MLV, so they won't exist in the DNG.

[yokashin]

What do you want to say with that numbers? These are two different clips. Many values in metadata are different. Who knows how ACR interprets metadata... Camera's WB values are not saved in the MLV, so they won't exist in the DNG.

I thought they were frames from the same film but from different programs ...

[adrjork]

I thought they were frames from the same film but from different programs ...

Yes indeed: the original MLV is the same, but converted to DNG via 2 different apps (Switch and MLVApp).

[kayman1021]

I took the 6GB file to experiment
Experiment 1
Opened in MLVApp, exported some frames as Uncompressed DNG.
RawTherapee saw the dng-s as raw files, but they were filled with 0 values.
Opened with my homemade tool i was able to see the values in the files. This tool ignores any metadata apart from resolution and raw values.
These dng-s cannot be processed with cr2hdr app directly. Converting with Adobe DNG converter usually solves this problem, but it faild to convert.

Experiment 2
The original file is uncompressed, made a compressed version with MVLApp. Exporting the compressed file's frames they could be successfully opened in RawTherapee. Valid values.
These DNG frames can be successfully processed with cr2hdr application. I processed them as group of 250 frames,4 times paralelled, default options
The processed frames can be stiched into a single MLV file with raw2mlv application.

I suspect there is some problem with the metadata/info fields in the original file. When frames exported, those info get tranferred into the dng, causing bad file.
Compressing the mlv file somehow solves this.
Done on Win10/x64

Here is the processed file
google drive link

[adrjork]

Incredible! Thank you really a lot, kayman1021.
Unfortunately I can't replay your “Experiment 2”... So please, may you explain to me Experiment 2 like if you are talking to a baby?
1. You uploaded the original MLV file into MLVApp and unchecked Enable RAW Correction? (Or the Correction remains enabled?)
2. In Export Settings you set codec MLV, Compress, uncheck Export audio, then clic Close (or Export audio is enabled?)
3. At this point, you pressed Export selected clip.
4. Now you uploaded the new compressed MLV into the same MLVApp and again unchecked RAW Correction enabled, right?
5. In Export Settings you set codec DNG Uncompressed with Davinci Resolve naming scheme, and Export audio unchecked, right?
6. At this point, you pressed Export selected clips, and so you obtained the DNGs folder.
7. Now you pointed Switch to the DNGs folder, and chose “(d) cr2hdr dualiso processing(CR2)”, then “(f) Force dualiso processing”?
Please, explain your detailed workflow.
Thanks a lot.

[kayman1021]

1-4. is correct
5. In MLVApp i used Default Naming Scheme.(I renamed the files manually in 7., i think it does not matter)
6. is correct
7. I used cr2hdr.exe app on windows. I have no knowledge about Switch.
Drag&dropping all dng onto cr2hdr.exe give me a "filename or extension too long" error, so i renamed the files. The cause however wasn't this, but the list of filenames is too long to drag&drop.
cr2hdr works on a single core/thread, and i have a skylake i5-6400/4core. I made a few directories, each containing the exe, and 250 dng files
I drag&dropped each directory's 250 file onto the EXE, i let 4 processes run at once, giving full cpu usage

cut&pasted the DNG files to a common single folder
The other app is raw2mlv.exe for making a single MLV from the DNGs
This one too shows "filename or extension too long" error on drag&drop
However, this can be worked around from command line. LINK
opening a command prompt in the dng folder, and running raw2mlv.exe *.dng -o output.mlv gave a single mlv file

MLVApp for compressing the original, raw correction off
MLVApp for extracting the frames as DNG, raw correction off
cr2hrd.exe for processing dual iso to single iso
raw2mlv for stiching the dng frames into a single mlv
btw this problem reminds me when i recorded dualIso anamorphic videos. MLVApp detected anamorphic video, and set the Transformation/Width Stretch to 3.0x
The frames exported this way were not recognizeable to cr2hdr, but when i set the stretch back to 1.0x, the exported frames could be processed. I think that too is stored somehow in the dng files

[masc]

opening a command prompt in the dng folder, and running raw2mlv.exe *.dng -o output.mlv gave a single mlv file

This step can also be done in MLVApp, if you don't like the command line.

btw this problem reminds me when i recorded dualIso anamorphic videos. MLVApp detected anamorphic video, and set the Transformation/Width Stretch to 3.0x
The frames exported this way were not recognizeable to cr2hdr, but when i set the stretch back to 1.0x, the exported frames could be processed. I think that too is stored somehow in the dng files

Stretching metadata is saved in DNG, when exporting with MLVApp.

[adrjork]

MLVApp for compressing the original, raw correction off
MLVApp for extracting the frames as DNG, raw correction off
cr2hrd.exe for processing dual iso to single iso
raw2mlv for stiching the dng frames into a single mlv

Incredibly, I CAN'T obtain the same result... Still flickering, still lines... And I tested all in Windows, following step by step!
So, please kayman1021 may you very kindly make a couple of things for me?
1. Please, tell me the version of MLVApp you are using;
2. Please, link an archive with cr2hdr.exe and raw2mlv.exe you are using.
Thanks in advance.

[Danne]

@kayman1021
Upload an unprocessed dng file after your roundtrip and also a processed one to show your result. I doubt lines are gone. Maybe you are referring to simply being able to process the file without the force option?

[adrjork]

@kayman1021:
As I wrote before, I tried your method without obtaining your clean result. I really don't know why, since I replicated step by step your guide.
The only explanation to this is that probably you are using a peculiar version (an old one?) of cr2hdr.exe
I tried everything, even a super-long workflow with Lightroom and LRTimelapse...
At the moment the best&fastest result I obtained is based on a petty trick:
1. In MLVApp I upload the uncompressed MLV, and I set Off Focus dots, Off Bad pixels, Off Vertical stripes, whilst Chroma smooth 2x2 and Dual ISO forced on Preview;
2. I export uncompressed DNGs;
3. In Davinci the first node is for RAW panel settings, then the second is a layer-node in Color mode: the “under” layer completely desaturated (B/N) and with a strong deflicker, while the “above” layer is a coloured still exported from a “grab” still. Adding just a bit of blur, the dirty trick works, but only because the shot is pretty motionless.
Conclusions: this is NOT the way to go...