Dual ISO - massive dynamic range improvement (dual_iso.mo)

[Audionut]

I think we can count on a1ex making it maximum IQ out of the box 

These should make it faster.

Code Select Expand

{ &interp_method, 1, "--mean23",      "average the nearest 2 or 3 pixels of the same color from the Bayer grid (faster)

{ &chroma_smooth_method, 0, "--no-cs",       "disable chroma smoothing" },

{ &fix_bad_pixels, 0, "--no-bad-pix",       "disable bad pixel fixing (try it if you shoot stars)" },

{ &use_fullres,     0, "--no-fullres",       "disable full-resolution blending" },

{ &use_alias_map,   0, "--no-alias-map",     "disable alias map, used to fix aliasing in deep shadows" },

{ &use_stripe_fix,  0, "--no-stripe-fix",    "disable horizontal stripe fix" },

[Walter Schulz]

@a1ex: German language support, please!

Semi-overexposed = "Halb-überbelichtet"? Urks! "Scheinbare Überbelichtung"?
Deep shadow = "Tiefer Schatten"? ... naja ...
Dynamikbereich (src1), Dynamikbereich (src2) = "Quelle 1", "Quelle 2". No clue what's meant by that.

Other finetuning needed? Open to suggestions!

Ciao
Walter

[Walter Schulz]

More help needed. Pretty sure I messed up this one:

TTAmaze=use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
TTAmaze=Verwende einen De-Mosaik-Zwischenschritt (AMaZE), gefolgt von kantenbetonter Interpolation (Standardeinstellung)

Ciao
Walter

[citytrader]

Finally I've got an answer from Photomatix for the issue of magenta cast, this is their email:

"appears that the issue comes from the fact that the DNG files produced by Magic Lantern are missing two DNG tags that Photomatix needs in order to be able to interpret the color information of the DNG data. The Magic Lantern people would need to make their firmware write these two tags before Photomatix Pro would be able to support those files.

These are the missing tags:

CFAPlaneColor tag 50710
CFALayout tag 50711

Once these two tags are added it may be that Photomatix Pro will be able to immediately handle the files, or we may need to take another look at that time to try and correct the reading once the information is available."

Could be possible to add that two tags!?

[a1ex]

These tags are optional according to the DNG spec, so... yeah, better let the free software guys work around problems in nonfree programs.

Anyway, adding these tags is quite easy, you only have to edit chdk-dng, fill the missing tags, and try it. Since I don't use Photomatix, I'll let you do this part and submit a pull request if it works.

[Marsu42]

These tags are optional according to the DNG spec, so... yeah, better let the free software guys work around problems in nonfree programs.

Well, you cannot really call it a problem since they're free to depend on official, but optional tags if they require them for one reason or another and all camera out there excluding ml write them - which seems to be the case or they'd have added a workaround by now.

Still, it unfortunately shows that ml still seems to be considered a "hack", not widespread and not to be taken seriously enough to add consideration for ml features (dual_iso, mlv) into commercial, closed-source software

Anyway, adding these tags is quite easy, you only have to edit chdk-dng, fill the missing tags

I would have thought adding static exif tags would best be done in cr2hdr esp. since as I just learned the silent dng don't contain much exif data anyway?

[a1ex]

When reading an optional tag, the proper thing to do is to use its default value, as described in the DNG spec.

Do you disagree with this?!

[Marsu42]

When reading an optional tag, the proper thing to do is to use its default value, as described in the DNG spec. Do you disagree with this?!

Right, if it does have a default value (which I didn't know, if I'd read long lists I'd start with the exiftool help page :-p) then clearly the photomatrix guys are to blame - I'd suggest the user in question replies to their "support" e-mail as such, suggesting to add the default fallback to their program. In reality of course, adding these to cr2hdr might be the realistic solution

[citytrader]

Right, if it does have a default value (which I didn't know, if I'd read long lists I'd start with the exiftool help page :-p) then clearly the photomatrix guys are to blame - I'd suggest the user in question replies to their "support" e-mail as such, suggesting to add the default fallback to their program. In reality of course, adding these to cr2hdr might be the realistic solution

Hello, so what do you suggest me to do!?... is this a mistake of Photomatix?

[Audionut]

<Dark_Shikari> you can never go wrong by abiding by the spec, since if someone breaks the spec you can just blame them instead
<pengvado> fair enough. I'll allow the patch on the premise of passing the blame.

Hello, so what do you suggest me to do!?... is this a mistake of Photomatix?

Tell photomatrix to abide by the spec.  You'll probably have more luck fixing the problem yourself, see here, and here for hints.

This is a handy reference.

[Marsu42]

Hello, so what do you suggest me to do!?... is this a mistake of Photomatix?

I wouldn't mention the word "mistake" :-) but since you're in customer e-mail support contact with them anyway, there's no harm telling them that they're not observing the dng specification as there are default values for the tags in question when they don't exist in the file. Falling back to the default value would be the proper thing to do and easy to accomplish as well.

[Walter Schulz]

Hello, so what do you suggest me to do!?

Download Adobe DNGconverter 8.4, convert your ML-DNGs to whatever-DNGs and give them a try in Photomatix.

[Marsu42]

Prey, is there any drawback if converting the dual_iso unprocessed cr2 to dng (non-lossy of course) and *then* put them through cr2hdr?

Storing images as dng instead of cr2 always saves a few mb (mostly because the jpeg preview is dropped), and this can add up to a lot of saved space looking at my growing cr2 dual_iso archive. Alternatively, I'm still looking for a tool that erases the thumbnail from a cr2 and leaves the rest untouched :-p

[Audionut]

AFAIK, there is no tool to remove thumbnails.  Probably because so many things rely on the embedded thumbnail.

cr2hdr relies on the actual data, it doesn't care about exif or the file extension.  As long as it's CR2/DNG 

edit:  Actually, a1ex likes robust solutions so much, that cr2hdr doesn't care about the extension at all.   

[Marsu42]

Probably because so many things rely on the embedded thumbnail.

They do? Other than in-camera preview I never found anything that uses it, it's just a waist of storage space. The one reason to keep it around is because you can extract it from the cr2 if the raw data is broken for some reason, but that's hardly any good with dual_iso shots.

Thanks for the answer concerning cr2/dng though, in this case I'll convert my cr2 to dng as any other cr2-processing programs like dxo cannot handle dual_iso files anyway.

Edit: fyi: the dng files are nearly 20% smaller than cr2.

[Audionut]

They do? Other than in-camera preview I never found anything that uses it

Probably because most things don't advertise the fact that they do.     It's significantly faster to display the embedded preview, for a preview, then to decode the entire data stream.

Thanks, re: space.  Personally, I prefer to stick to the original format and just upgrade HDDs, since they're cheap enough.  I should probably convert all the dual ISO CR2s though. 

[a1ex]

Prey, is there any drawback if converting the dual_iso unprocessed cr2 to dng (non-lossy of course) and *then* put them through cr2hdr?

I actually do that. For archiving, I delete the dual ISO output files and keep the unprocessed dng's.

On Windows, be careful because they have the same extension, so the unprocessed one will probably get overwritten. On Linux, Adobe DNG converter outputs lowercase dng's, and cr2hdr outputs uppercase DNG's => no conflicts.

To solve this on Windows, one way would be to rename the unprocessed .dng back to .cr2, since dcraw doesn't care about the extension. Ugly, but gets the job done.

[Marsu42]

On Windows, be careful because they have the same extension, so the unprocessed one will probably get overwritten. [...]
To solve this on Windows, one way would be to rename the unprocessed .dng back to .cr2, since dcraw doesn't care about the extension. Ugly, but gets the job done.

Yes, I already noticed that, quite a catch since it destroys the original data w/o even a warning. Couldn't you just add an auto-rename or output path option to cr2hdr to solve this once and for all for everyone (yes-i-could-submit-a-patch)?

[Audionut]

I'm getting lower case output here from Adobe, on a windows (8.1) machine.

Yes, I already noticed that, quite a catch since it destroys the original data w/o even a warning. Couldn't you just add an auto-rename or output path option to cr2hdr to solve this once and for all for everyone (yes-i-could-submit-a-patch)?

There is an option to not overwrite existing data:  https://bitbucket.org/hudson/magic-lantern/commits/7fad96785821644a05dfb9d4212791e6e1bc7706

[Marsu42]

There is an option to not overwrite existing data:  https://bitbucket.org/hudson/magic-lantern/commits/7fad96785821644a05dfb9d4212791e6e1bc7706

But even if you have to opt-in so you have to be already aware of the problem, and then it's no good because it would refuse converting the dual_iso original dng... the .cr2 rename method is imho rather hackish. I'd rather rename them to .mld or something :-p

I'm getting lower case output here from Adobe, on a windows (8.1) machine.

In theory, you can configure any windows ntfs volume to be case-sensitive, I did this because it helps mingw before I finally moved to cygwin. But very few users do that, usually windows is case-insensitive and .dng overwrites .DNG and vice versa.

[Audionut]

Indeed it does.  Adobe will prefix and not overwrite, but the output from cr2hdr will overwrite lowercase dng's.

But even if you have to opt-in so you have to be already aware of the problem, and then it's no good because it would refuse converting the dual_iso original dng... the .cr2 rename method is imho rather hackish. I'd rather rename them to .mld or something :-p

I understand and can see the cause for concern.

Personally, I would segregate with folder management.  Originals in one folder, processed in another.  BarracudaGUI is great for this.  But I'm used to being pedantic with file management. 

[Marsu42]

Personally, I would segregate with folder management.  Originals in one folder, processed in another.

I'm doing that right now with cr2 dual_iso originals on a backup hd and converted dng on a live hd right now. The nice thing about this solution that I just need to back up the non-dual_iso files from the live hd and am not duplicating data.

But folder management doesn't work if cr2hdr insists on overwriting the source .dng and you cannot rename the target or set an output folder ... cr2hdr simply needs an output path option. Or am I missing something here?

[Audionut]

I think you might be alone on that one.  All of the GUI's provide that option, which fits in with this philosophy.

My bold for emphasis.

Instead, I'm trying to build a community that does not just consume whatever we give to them, but I want this community to actually participate in the development process, help each other, and share the knowledge. We gave you some free software, we gave you a proof of concept that you found useful, and now we expect you all to take this software at the next level, and let us build upon your work, in the same way as you have built upon ours. I'm quite far from this utopian goal though, but this is the direction I want ML to go.

[arturochu]

for people that have more than 500 files and cannot use same levels, i'd recommend using exiftoolGUI, way easier and fast.

[Luiz Roberto dos Santos]

Hi,
I have beem using Color filters for B&W photography, especificaly the 091 red filter, with CPL. Well, I receive very strips out of the cr2hdr-20bit, like the image bellow (--no-cs --full-res).
I put some files here. I think it will be very difficult to fix it, because the cr2hdr need to colors to interpret the photo, or not? Any ideia how to fix it?

Example frame here.