Dual ISO - massive dynamic range improvement (dual_iso.mo)

Started by a1ex, July 16, 2013, 06:33:50 PM

Previous topic - Next topic

0 Members and 4 Guests are viewing this topic.

Legushka

2'Alex
Quote from: a1ex on October 30, 2013, 09:51:23 PM
I'm experimenting with some noise reduction, with an algorithm along these lines...

uses windowsXP 32
does not work
works
Canon eos 60d, firmware 1.1.1/sigma 10-20 + ML v2.3.RC2 + Nodal Ninja NN4

Audionut

First impression.

Does a better job of maintaining detail in textures (probably due to less aggressive denoise), but muddles the noise in flat textures.

LR - Luma 50 - Chroma 25


Default denoise cr2hdr


edit:  No noise reduction

a1ex

@Legushka: can you upload the CR2? I have no errors or warnings in valgrind.

Audionut

The CR2 and processed JPG from LR if you want target practice.

https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_46A2224.CR2
https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/Lightroom.jpg

edit:  Good test shot for the severe color artifact on edges also.

I remember using NL means way back in '06 on video.  IIRC, it did a good job in the temporal domain, of course back then it was dog slow.

a1ex

Yes, the algorithm is old and simple, but very slow. The papers talk about computing each pixel as a linear combination of *all* other image pixels (which is really crazy in practice, but useful to present the mathematical foundation behind it, theoretical limits and other stuff like that).

Here's a PhD thesis on this subject: http://www.mi.parisdescartes.fr/~buades/publicacions/tesi.pdf

(see chapter 5 for pixel peeping)

Danne

Don,t know how to change command line but made denoising, color fix and some sharpening in lightroom of my old original you uploaded :) Looks like this

Lightroom denoised




Original




Denoiced monte carlo style, 2 alternatives














a1ex

In a terminal, type this:

cr2hdr-denoice --denoice IMG_1234.CR2

or, on Mac/Linux:

wine cr2hdr-denoice.exe --denoice IMG_1234.CR2

and you get a denoised DNG.

N/A

Here's a tough situation that Dual Iso handled pretty exquisitely.

7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

Danne

Updated my comparisons with two examples of denoiced pictures. I say lightroom is a little bit better here, what do you think?
see Reply #1530

a1ex

One has chroma noise and the other doesn't. Can you adjust them so they have similar noise levels?

Danne

Like this? It actually cleans up nicely with denoiced cr2hdr. Could maybe work

denoiced

not denoiced


settings for both

a1ex

Sort of. Chroma noise seems similar now, but luma noise is much higher in the second one.

If you can measure the standard deviation of the noise, for example in the triangle between the table top and legs, it may help. To compare them, I think you need to have equal noise levels in both images, so you can judge texture details. Or, have the same amount of details, but that's difficult to quantify. If you have both of them different, it's difficult to draw a conclusion.

Danne


Danne

Here I tried to get them look similar by sharpen the denoiced to look the same as the other

denoiced

No denoice

Audionut

If you have PS, make a selection of a flat texture and expand the histogram to advanced.  This will show the std deviation.

I'll post some more examples later tonight.

Marsu42

Quote from: a1ex on October 30, 2013, 09:51:23 PMNow the big question: how does this compare with state-of-art denoising software? (Lightroom, Neat Image, Noise Ninja, whatever you use). I don't have license for any of these, so I don't know if I'm reinventing the wheel or not.

Btw: For the best current nr algorithm try DxO Pro Optic's v9's PRIME noise reduction - slow as hell, but for noisy images at high iso one step in front of Adobe ACR. The advantage of other 3rd party nr plugins like Noise Ninja afaik is not that they are overall better at nr, but just that they are"smarter" and try to figure out which parts need more nr and which don't.

a1ex

From their description, it sounds very similar to my approach. So yeah, I'm reinveting the wheel.

4 stops sounds impressive; for that, for every output pixel they need to average 256 other pixels if they don't do any weighting, or more if they use weights for averaging.

Marsu42

Quote from: a1ex on October 31, 2013, 06:10:31 PM
From their description, it sounds very similar to my approach. So yeah, I'm reinveting the wheel.

But as oss :-) ... here's the relevant thread from CR with some samples: http://www.canonrumors.com/forum/index.php?topic=17701.msg328534#msg328534

Beware: DxO's PRIME only works with raw cr2 pictures, so if anyone wants to use it don't concert your cr2 to raw dng...

Quote from: a1ex on October 31, 2013, 06:10:31 PM
4 stops sounds impressive; for that, for every output pixel they need to average 256 other pixels if they don't do any weighting, or more if they use weights for averaging.

This is probably the reason why their algorithm takes ages to process the picture - but it really doesn't matter because for your very best shots you'd be happy to wait (or process overnight) as long as the result is an improvement.

tjaja

First off: All contributors to magiclantern thanks alot.
Question:
Is there a way to use cr2hdr-histmatch3 in combination with raw video?

If I use raw2dng It already process and interpolate the dual iso raw file.

Thanks in advance
5DFree, 550D, canon 50mm 1.4

Audionut

Quote from: a1ex on October 31, 2013, 06:10:31 PM
From their description, it sounds very similar to my approach. So yeah, I'm reinveting the wheel.

Looks like a lot of marketing BS to me. 
For instance, they talk about ISO 51,200 but show an example at ISO 25,600.  Also, the example image seems to contain mostly color noise only.  Color noise is easy to deal with, it's the luma noise that creates the problems (if you respect fine details), and in my tests, ISO 25600 creates a ton of luma noise.

As for reinventing the wheel.  Well your solution is free.

Legushka

after cr2 -> DNG lost the white balance settings in the DNG
Canon eos 60d, firmware 1.1.1/sigma 10-20 + ML v2.3.RC2 + Nodal Ninja NN4

Marsu42

Quote from: Legushka on November 01, 2013, 10:23:40 AM
after cr2 -> DNG lost the white balance settings in the DNG

Known problem, see some posts above - currently you have to manually correct the wb in dual_iso files, at least that's what I (have to) do with ACR.

Edit: Of course this can be tricky - either shoot a non dual_iso frame to get the wb or use a gray or combined gray/color card next to tehe subject for calibration, the latter is a good idea anyway in difficult lighting.

Speaking of which: Is it necessary to shoot a frame to get the awb - or can/could ml get the current awb from live view (feature request) and display it in the info bar or somewhere else?

orim

again, please (post #1499):

Am I really the only one, who gets DNG files from not dualISO CR2 files?

painya

Didn't think I should make a bug report out of this as it is not directly ML related. Does anyone else have issues with processing too many photos with cr2-hdr-histmatch3? My limit is 30. Last page someone mentioned having issues, but I wasn't sure if it was with this. Also quick question, does this version have denoice (denoising)? Maybe this is the wrong thread, if so point me to the right one.

PS. It would be nice if on the front page all of the awesome new versions were being put up.

Thanks,
Painya
Good footage doesn't make a story any better.

a1ex

I'll try to update it these days. Sorry, I'll be busy with non-ML stuff till the end of the year.