Author Topic: Dual ISO - massive dynamic range improvement (dual_iso.mo)  (Read 2182222 times)

Legushka

  • New to the forum
  • *
  • Posts: 25
  • Rus
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1525 on: October 30, 2013, 10:15:59 PM »
2'Alex
I'm experimenting with some noise reduction, with an algorithm along these lines...

uses windowsXP 32
does not work
works
Canon eos 60d, firmware 1.1.1/sigma 10-20 + ML v2.3.RC2 + Nodal Ninja NN4

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1526 on: October 30, 2013, 10:21:50 PM »
First impression.

Does a better job of maintaining detail in textures (probably due to less aggressive denoise), but muddles the noise in flat textures.

LR - Luma 50 - Chroma 25


Default denoise cr2hdr


edit:  No noise reduction

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1527 on: October 30, 2013, 10:33:07 PM »
@Legushka: can you upload the CR2? I have no errors or warnings in valgrind.

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1528 on: October 30, 2013, 10:38:07 PM »
The CR2 and processed JPG from LR if you want target practice.

https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_46A2224.CR2
https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/Lightroom.jpg

edit:  Good test shot for the severe color artifact on edges also.

I remember using NL means way back in '06 on video.  IIRC, it did a good job in the temporal domain, of course back then it was dog slow.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1529 on: October 30, 2013, 10:46:38 PM »
Yes, the algorithm is old and simple, but very slow. The papers talk about computing each pixel as a linear combination of *all* other image pixels (which is really crazy in practice, but useful to present the mathematical foundation behind it, theoretical limits and other stuff like that).

Here's a PhD thesis on this subject: http://www.mi.parisdescartes.fr/~buades/publicacions/tesi.pdf

(see chapter 5 for pixel peeping)

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7654
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1530 on: October 30, 2013, 10:50:50 PM »
Don,t know how to change command line but made denoising, color fix and some sharpening in lightroom of my old original you uploaded :) Looks like this

Lightroom denoised




Original




Denoiced monte carlo style, 2 alternatives














a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1531 on: October 30, 2013, 11:00:20 PM »
In a terminal, type this:

cr2hdr-denoice --denoice IMG_1234.CR2

or, on Mac/Linux:

wine cr2hdr-denoice.exe --denoice IMG_1234.CR2

and you get a denoised DNG.

N/A

  • Hero Member
  • *****
  • Posts: 576
  • Dreaming in 14 bit
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1532 on: October 30, 2013, 11:46:56 PM »
Here's a tough situation that Dual Iso handled pretty exquisitely.

7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7654
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1533 on: October 30, 2013, 11:57:54 PM »
Updated my comparisons with two examples of denoiced pictures. I say lightroom is a little bit better here, what do you think?
see Reply #1530

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1534 on: October 31, 2013, 12:04:44 AM »
One has chroma noise and the other doesn't. Can you adjust them so they have similar noise levels?

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7654
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1535 on: October 31, 2013, 12:14:02 AM »
Like this? It actually cleans up nicely with denoiced cr2hdr. Could maybe work

denoiced

not denoiced


settings for both

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1536 on: October 31, 2013, 12:21:48 AM »
Sort of. Chroma noise seems similar now, but luma noise is much higher in the second one.

If you can measure the standard deviation of the noise, for example in the triangle between the table top and legs, it may help. To compare them, I think you need to have equal noise levels in both images, so you can judge texture details. Or, have the same amount of details, but that's difficult to quantify. If you have both of them different, it's difficult to draw a conclusion.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7654
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1537 on: October 31, 2013, 12:41:47 AM »
Trying this
Denoiced

Not denoised

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7654
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1538 on: October 31, 2013, 12:48:23 AM »
Here I tried to get them look similar by sharpen the denoiced to look the same as the other

denoiced

No denoice

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1539 on: October 31, 2013, 01:00:52 AM »
If you have PS, make a selection of a flat texture and expand the histogram to advanced.  This will show the std deviation.

I'll post some more examples later tonight.

Marsu42

  • Contributor
  • Hero Member
  • *****
  • Posts: 1557
  • 66d + flashes
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1540 on: October 31, 2013, 05:41:30 PM »
Now the big question: how does this compare with state-of-art denoising software? (Lightroom, Neat Image, Noise Ninja, whatever you use). I don't have license for any of these, so I don't know if I'm reinventing the wheel or not.

Btw: For the best current nr algorithm try DxO Pro Optic's v9's PRIME noise reduction - slow as hell, but for noisy images at high iso one step in front of Adobe ACR. The advantage of other 3rd party nr plugins like Noise Ninja afaik is not that they are overall better at nr, but just that they are"smarter" and try to figure out which parts need more nr and which don't.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1541 on: October 31, 2013, 06:10:31 PM »
From their description, it sounds very similar to my approach. So yeah, I'm reinveting the wheel.

4 stops sounds impressive; for that, for every output pixel they need to average 256 other pixels if they don't do any weighting, or more if they use weights for averaging.

Marsu42

  • Contributor
  • Hero Member
  • *****
  • Posts: 1557
  • 66d + flashes
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1542 on: October 31, 2013, 06:16:56 PM »
From their description, it sounds very similar to my approach. So yeah, I'm reinveting the wheel.

But as oss :-) ... here's the relevant thread from CR with some samples: http://www.canonrumors.com/forum/index.php?topic=17701.msg328534#msg328534

Beware: DxO's PRIME only works with raw cr2 pictures, so if anyone wants to use it don't concert your cr2 to raw dng...

4 stops sounds impressive; for that, for every output pixel they need to average 256 other pixels if they don't do any weighting, or more if they use weights for averaging.

This is probably the reason why their algorithm takes ages to process the picture - but it really doesn't matter because for your very best shots you'd be happy to wait (or process overnight) as long as the result is an improvement.

tjaja

  • Freshman
  • **
  • Posts: 83
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1543 on: October 31, 2013, 06:26:08 PM »
First off: All contributors to magiclantern thanks alot.
Question:
Is there a way to use cr2hdr-histmatch3 in combination with raw video?

If I use raw2dng It already process and interpolate the dual iso raw file.

Thanks in advance
5DFree, 550D, canon 50mm 1.4

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1544 on: November 01, 2013, 03:17:37 AM »
From their description, it sounds very similar to my approach. So yeah, I'm reinveting the wheel.

Looks like a lot of marketing BS to me. 
For instance, they talk about ISO 51,200 but show an example at ISO 25,600.  Also, the example image seems to contain mostly color noise only.  Color noise is easy to deal with, it's the luma noise that creates the problems (if you respect fine details), and in my tests, ISO 25600 creates a ton of luma noise.

As for reinventing the wheel.  Well your solution is free.

Legushka

  • New to the forum
  • *
  • Posts: 25
  • Rus
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1545 on: November 01, 2013, 10:23:40 AM »
after cr2 -> DNG lost the white balance settings in the DNG
Canon eos 60d, firmware 1.1.1/sigma 10-20 + ML v2.3.RC2 + Nodal Ninja NN4

Marsu42

  • Contributor
  • Hero Member
  • *****
  • Posts: 1557
  • 66d + flashes
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1546 on: November 01, 2013, 10:39:18 AM »
after cr2 -> DNG lost the white balance settings in the DNG

Known problem, see some posts above - currently you have to manually correct the wb in dual_iso files, at least that's what I (have to) do with ACR.

Edit: Of course this can be tricky - either shoot a non dual_iso frame to get the wb or use a gray or combined gray/color card next to tehe subject for calibration, the latter is a good idea anyway in difficult lighting.

Speaking of which: Is it necessary to shoot a frame to get the awb - or can/could ml get the current awb from live view (feature request) and display it in the info bar or somewhere else?

orim

  • New to the forum
  • *
  • Posts: 22
  • 50D / 70D
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1547 on: November 01, 2013, 01:40:43 PM »
again, please (post #1499):

Am I really the only one, who gets DNG files from not dualISO CR2 files?

painya

  • Senior
  • ****
  • Posts: 321
  • 6d 50d 550d 5dc
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1548 on: November 03, 2013, 06:25:37 AM »
Didn't think I should make a bug report out of this as it is not directly ML related. Does anyone else have issues with processing too many photos with cr2-hdr-histmatch3? My limit is 30. Last page someone mentioned having issues, but I wasn't sure if it was with this. Also quick question, does this version have denoice (denoising)? Maybe this is the wrong thread, if so point me to the right one.

PS. It would be nice if on the front page all of the awesome new versions were being put up.

Thanks,
Painya
Good footage doesn't make a story any better.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1549 on: November 03, 2013, 07:16:24 AM »
I'll try to update it these days. Sorry, I'll be busy with non-ML stuff till the end of the year.