Dual ISO - massive dynamic range improvement (dual_iso.mo)

Started by a1ex, July 16, 2013, 06:33:50 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

golem

Hi,

I have been playing with dual-iso and FullRes-SilentPic for the last view days to shoot time-lapse.
Now I would like to add ETTR to the process but I have 2 problems.

#1 ETTR does not have a "fastest shutter" setting to prevent it from using values that make no sense with FullRes-SP (<1/10 or 1/30)

#2 It seems that ETTR looks always at the entire picture to set the ISO and Shutterspeed. It would be cool if ETTR could be set to only measure the darker half of the DualISO Pics to be able to shoot bright sky and very dark parts in one image with DualISO 100/1600.
Canon EOS 60D

barepixels

Just curious did you do ETTR FullRes-SP silent in DNGs  or MLV

When you did DUAL ISO FullRes-SP only, didn't your shutter speed change a lot through out the shoot? If you did in DNG then you would't have EXIF info.  But you may have notice/remember
5D2 + nightly ML

golem

Quote from: barepixels on July 27, 2014, 07:50:37 PM
Just curious did you do ETTR FullRes-SP silent in DNGs  or MLV

I havent used ETTR until now - cam is shooting with ETTR in .MLV right now. The dual ISOs where recorded as .MLV (200+ frames, splited .MLV files with up to 6GB = 4+2GB)

Quote from: barepixels on July 27, 2014, 07:50:37 PMWhen you did DUAL ISO FullRes-SP only, didn't your shutter speed change a lot through out the shoot? If you did in DNG then you would't have EXIF info.  But you may have notice/remember

With "DualISO only" you mean without ETTR? I dont know - I would asume not - but in one test I saw some flickering but havent looked into it any further till now.



edit:
a video of my tests is here: http://www.magiclantern.fm/forum/index.php?topic=12802.0

its the 3rd scene without clouds that flickers
Canon EOS 60D

dmilligan

Quote from: golem on July 27, 2014, 06:00:22 PM
It would be cool if ETTR could be set to only measure the darker half of the DualISO Pics

If ETTR doesn't "measure" the bright part of you picture, that part will just be blown highlights. If that's actually what you want, then set the highlight ignore to 50%.

golem

Quote from: dmilligan on July 27, 2014, 09:29:04 PM
If ETTR doesn't "measure" the bright part of you picture, that part will just be blown highlights. If that's actually what you want, then set the highlight ignore to 50%.

Thats exactly what I want - I don´t need the full resolution in the bright clouds.
Right now I do this manually by not overexposing with 100 ISO and recording with 100/800

I thought about the 50% but that only works if the brighter image is 100% (all pixels) overexposed.
Canon EOS 60D

a1ex

When used with Dual ISO, ETTR measures the darker sub-image only.

golem

Quote from: a1ex on July 28, 2014, 08:00:40 AM
When used with Dual ISO, ETTR measures the darker sub-image only.

Yes, now it works - not sure what went wrong in my first test - thanks.
Canon EOS 60D

handbanana

Any chance to ever get this working in Raw video mode for the 50D and 5D2?

chris_1975

Hi there. Thanks for DUAL ISO !!
Sorry, but I have since yesterday problems to convert the images correctly.
I have the RAW image in the dropbox.
The DNG file after conversion still has the lines can be seen. I have tried everything, but unfortunately can not find the error.

It was photographed with ISO 100/400
Bulb Timer 2min
Dark Frame

Thank you for your help and thank you for the great work that makes all your !!

Greetings
Christian

https://www.dropbox.com/sh/ocp8xl7q6u7a4jd/AABJQ7-LciZ3QFwmjTtCN2G-a

Edit:
The RAW was converted with the current cr2hdr.exe files with old files I still had, with the Lightroom plugin and without (ie directly in the folder)

glubber

@chris_1975:

It looks like you have chosen an extreme exposure situation, so cr2hdr-algorithm picks a wrong rendering-choice.

I tried with some cr2hdr-betas and it worked out fine:
Jpgs developed with ufraw

latest cr2hdr:


cr2hdr-amaze-edge5:


cr2hdr-20bit:


Btw. from my experience Dual-ISO doesn't improve a longtime exposure at night


I copied below the output of the command window for further research by a pro ;D


E:\ufrawconvert>cr2hdr sepp.cr2
cr2hdr: a post processing tool for Dual ISO images

Last update: d0ac769 on 2014-01-23 10:13:39 UTC by alex:
cr2hdr: experimental white level routine with two separate levels ...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default
)

Input file      : sepp.cr2
Camera          : Canon EOS 5D Mark III
Full size       : 5920 x 3950
Active area     : 5796 x 3870
Black borders   : 124 left, 80 top
Black level     : 1965
ISO pattern     : BddB RGGB
White levels    : 15301 15300
Noise levels    : 10.82 8.34 8.34 10.74 (14-bit)
Black delta looks bad, skipping correction
ISO difference  : 1.78 EV (344)
Black delta     : 0.00
Dynamic range   : 10.64 (+) 10.27 => 12.05 EV (in theory)
Looking for hot/cold pixels...
Hot pixels      : 3080
Cold pixels     : 162
AMaZE interpolation ...
Amaze took 6.84 s
Edge-directed interpolation...
Semi-overexposed: 0.00%
Deep shadows    : 51.67%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 5.9 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 53.43 (16-bit), ideally 37.05
Dynamic range   : 9.96 EV (cooked)
Black adjust    : -5
Output file     : sepp.DNG
    1 image files updated

----------------------------------------------------------------------------------------------


E:\ufrawconvert>cr2hdr-20bit sepp.cr2
cr2hdr: a post processing tool for Dual ISO images

Last update: 26b7a08 on 2014-05-06 20:04:08 UTC by a1ex:
cr2hdr: experimental option to embed the original raw into the DNG

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default
)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (def
ault)

Input file      : sepp.cr2
Camera          : Canon EOS 5D Mark III
Camera model    : Canon EOS 5D Mark III
Full size       : 5920 x 3950
Active area     : 5796 x 3870
Black borders   : 124 left, 80 top
Black level     : 2048
ISO pattern     : BddB RGGB
White levels    : 15212 13794
Noise levels    : 10.81 8.32 8.34 10.73 (14-bit)
ISO difference  : 1.98 EV (395)
Black delta     : 1.98
Dynamic range   : 10.63 (+) 10.09 => 12.07 EV (in theory)
Looking for hot/cold pixels...
Hot pixels      : 1210
Cold pixels     : 156
AMaZE interpolation ...
Amaze took 3.40 s
Edge-directed interpolation...
Semi-overexposed: 0.58%
Deep shadows    : 82.23%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 5.6 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 143.94 (20-bit), ideally 140.87
Dynamic range   : 12.51 EV (cooked)
Black adjust    : -46
AsShotNeutral   : 0.62 1 0.55, 4127K/g=1.20 (gray max)
Output file     : sepp.DNG
sepp.DNG        : updating EXIF


-----------------------------------------------------------------------------------------

E:\ufrawconvert>cr2hdr-amaze-edge5 sepp.cr2
cr2hdr: a post processing tool for Dual ISO images

Last update: af9a60f on 2013-10-24 18:44:01 UTC by alex:
cr2hdr: always use last 8 lines from top optical black bar (should fi...


Input file     : sepp.cr2
Canon EOS 5D Mark III detected
Full size      : 5920 x 3950
Active area    : 5796 x 3870
White level    : 12500
Black borders  : 124 left, 80 top
Black level    : 1965
ISO pattern    : BddB RGGB
Noise levels   : 10.82 8.34 8.34 10.74 (14-bit)
Estimating ISO difference...
ISO difference : 2.00 EV (399)
Black delta    : 5.90
Interpolation  : amaze-edge-chroma2x2-alias
AMaZE interpolation ...
Amaze took 6.79 s
Edge-directed interpolation...
Semi-overexpo'd: 0.64%
Deep shadows   : 53.94%
Dynamic range  : 10.30 (+) 9.93 => 11.92 EV (in theory)
Matching brightness...
Horizontal stripe fix...
Looking for hot/cold pixels...
Hot pixels     : 877
Cold pixels    : 171
Full-res reconstruction...
ISO overlap    : 5.3 EV (approx)
Half-res blending...
Chroma filtering...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level    : 9.79 (16-bit), ideally 9.53
Dynamic range  : 12.07 EV (cooked)
Black adjust   : -8
Output file    : sepp.DNG
    1 image files updated


        
EOS 550D // Sigma 18-200 // Sigma 18-70 // Canon 10-18 STM

chris_1975

THX Glubber...

now i tried it with "cr2hdr-hot" ... it works .... thanks...

regards chris

Audionut

The long drawn out discussion about learning how to process dual ISO files, has been moved here:  http://www.magiclantern.fm/forum/index.php?topic=13028.0

Danne

Hi! I see a bug in cr2hdr when converting from dualiso dng to regular dng.
I,m wondering if there is anything to do regarding this issue?
https://bitbucket.org/hudson/magic-lantern/issue/2091/cr2hdr-dual-iso-3x-zoom-mode-strange
I,ve been trying to get some answers before but with little success. Also wondering if where I,ve placed this issue-report on bitbucket is the right place?


Thanks
/D

KurtAugust

Quote from: kichetof on May 07, 2014, 06:36:39 PM
$ ./cr2hdr
cr2hdr: a post processing tool for Dual ISO images

Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
cr2hdr: print a message when overwriting the output file


Download cr2hdr-20bit for Mac only with last commit 30a5132 ;)

@a1ex thanks for having solved the warnings on compilation!

I am totally hooked on how good Dual ISO MLV's look on a Canon 5D3 to get nicer highlights, but I've broken my teeth on this for several days now and I'm dying to see the latest improvements in quality and less flicker in my files. I can't get this version to work on my mac and I'm very sorry to ask for help.

dcraw & exiftool are installed. The version from 4/10/13 works as expected.

It would be great if there was a complete package out that that people can run without any further installations. I'm trying every approach and I understand if a developer doesn't feel like pushing every update to the mac platform, but perhaps there's someone out there to help me out. Or point to other useful threads. Of course you can also reply here:
http://www.magiclantern.fm/forum/index.php?topic=13028.0

Quote from: dubzeebass on May 07, 2014, 07:46:20 PM
Many thanks!

Care to provide a step by step or checklist of all the stuff that should be installed in the tool chain to get this working.
www.kurtaugustyns.com @HetRovendOog

a1ex

If you teach me how to cross-compile a Mac binary from Linux, I can add it to the nightly build server. I wanted to include the postprocessing tools on that page for a long time.

KurtAugust

I'm sorry. I can not be of too much much help there (yet). Sorry.

On the other hand, a small step in the that direction has been taken as I got the 20bit version of cr2hdr to work. And very happy about that!

The steps I took:
-Move to a computer with a different OS. OS X 10.9.4 instead of OS X 10.6.8 (the cr2hdr didn't run on 10.6.8 )
-Installed dcraw with this tutorial: http://vkphotoblog.blogspot.be/2014/05/dcraw-921-for-os-x-mavericks-users.html
(including xcode installation) (interesting read too)
-Move cr2hdr to the same folder as dcraw is in. Allows you to just type the cr2hdr command without specifying a directory.
-Install exiftool from here http://www.sno.phy.queensu.ca/~phil/exiftool/
-Run the program from the terminal
-Say fuuuuuuh...!

I will see if I can now also make it work on a OS X 10.6.8 machine.
www.kurtaugustyns.com @HetRovendOog

kichetof

Quote from: a1ex on August 25, 2014, 03:54:03 PM
If you teach me how to cross-compile a Mac binary from Linux, I can add it to the nightly build server. I wanted to include the postprocessing tools on that page for a long time.

Hi a1ex, I found OSXCross maybe it will works to compile it ! :) (another way)
If you need some help/files, I'll send to you !

Quote from: KurtAugust on August 25, 2014, 10:25:42 AM
I am totally hooked on how good Dual ISO MLV's look on a Canon 5D3 to get nicer highlights, but I've broken my teeth on this for several days now and I'm dying to see the latest improvements in quality and less flicker in my files. I can't get this version to work on my mac and I'm very sorry to ask for help.

dcraw & exiftool are installed. The version from 4/10/13 works as expected.

If you run cr2hdr into the Terminal, could you post the output ? And what's you dcraw version ? 9.21 works for cr2hdr 20bit

KurtAugust

Quote from: kichetof on August 26, 2014, 11:56:10 AM
If you run cr2hdr into the Terminal, could you post the output ?
Illegal instruction

Quote from: kichetof on August 26, 2014, 11:56:10 AM
And what's you dcraw version ? 9.21 works for cr2hdr 20bit
9.19 works too for cr2hdr 20bit on 10.9.4 , but not on os x 10.6.8
www.kurtaugustyns.com @HetRovendOog

kichetof

Quote from: KurtAugust on August 26, 2014, 02:48:01 PM
Illegal instruction

Very strange ! It works well on my 2  mac (10.9.4 too)

Quote from: KurtAugust on August 26, 2014, 02:48:01 PM
9.19 works too for cr2hdr 20bit on 10.9.4 , but not on os x 10.6.8

To works on my mac, I need dcraw 9.21.

Quote from: OSXCross
From OSXCross:
Note: libc++ requires Mac OS X 10.7 or newer!

Maybe this is the reason, I don't know... I compiled on Mac OS X 10.9.3

Levas

tried dual iso video in normal video mode a few times (so I didn't use crop mode)
I always end up with visual horizontal stripes with a slightly brightness difference. (And yes I processed my files  ;) (with CR2HDR v2.0)
And I don't use extreme values, mostly iso 100/800

Is it possible to get normal results with dual iso when not in crop mode ?
Or is impossible because the lineskipping and dual iso won't give a normal RGGB pixel readout in the same iso value ?

a1ex


Levas

Ofcourse I will show some examples  ;)

See the png's in this folder (It's not showing the entire frame, but I think it shows what I'm talking about)

https://drive.google.com/folderview?id=0B1BxGc3dfMDaZWxfLTlVQmxnU1k&usp=sharing


a1ex

How did you get these png's? Were they straight out of the camera?

Levas

Those png's are screenshots of the processed frames (mlv-dump osx -> CR2HDR -> RawTherapee -> screenshot...)
I'm uploading some more files to the same folder now.
An original frame out of mlv-dump osx (renamed to CR2), and the same frame processed by CR2HDR (the dng).

This is recorded using the nightly build for 6d from March 15, 2014, using this build still a lot because it gives me the option for FPS override to 35fps.




a1ex

Can you try with cr2hdr-20bit?

(you may have to run it under Wine, or maybe kichetof has a Mac binary here)