Author Topic: Dual ISO - massive dynamic range improvement (dual_iso.mo)  (Read 2157503 times)

golem

  • New to the forum
  • *
  • Posts: 8
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2225 on: July 27, 2014, 06:00:22 PM »
Hi,

I have been playing with dual-iso and FullRes-SilentPic for the last view days to shoot time-lapse.
Now I would like to add ETTR to the process but I have 2 problems.

#1 ETTR does not have a "fastest shutter" setting to prevent it from using values that make no sense with FullRes-SP (<1/10 or 1/30)

#2 It seems that ETTR looks always at the entire picture to set the ISO and Shutterspeed. It would be cool if ETTR could be set to only measure the darker half of the DualISO Pics to be able to shoot bright sky and very dark parts in one image with DualISO 100/1600.
Canon EOS 60D

barepixels

  • Member
  • ***
  • Posts: 134
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2226 on: July 27, 2014, 07:50:37 PM »
Just curious did you do ETTR FullRes-SP silent in DNGs  or MLV

When you did DUAL ISO FullRes-SP only, didn't your shutter speed change a lot through out the shoot? If you did in DNG then you would't have EXIF info.  But you may have notice/remember
5D2 + nightly ML

golem

  • New to the forum
  • *
  • Posts: 8
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2227 on: July 27, 2014, 08:59:47 PM »
Just curious did you do ETTR FullRes-SP silent in DNGs  or MLV

I havent used ETTR until now - cam is shooting with ETTR in .MLV right now. The dual ISOs where recorded as .MLV (200+ frames, splited .MLV files with up to 6GB = 4+2GB)

When you did DUAL ISO FullRes-SP only, didn't your shutter speed change a lot through out the shoot? If you did in DNG then you would't have EXIF info.  But you may have notice/remember

With "DualISO only" you mean without ETTR? I dont know - I would asume not - but in one test I saw some flickering but havent looked into it any further till now.



edit:
a video of my tests is here: http://www.magiclantern.fm/forum/index.php?topic=12802.0

its the 3rd scene without clouds that flickers
Canon EOS 60D

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2228 on: July 27, 2014, 09:29:04 PM »
It would be cool if ETTR could be set to only measure the darker half of the DualISO Pics

If ETTR doesn't "measure" the bright part of you picture, that part will just be blown highlights. If that's actually what you want, then set the highlight ignore to 50%.

golem

  • New to the forum
  • *
  • Posts: 8
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2229 on: July 27, 2014, 09:50:57 PM »
If ETTR doesn't "measure" the bright part of you picture, that part will just be blown highlights. If that's actually what you want, then set the highlight ignore to 50%.

Thats exactly what I want - I don´t need the full resolution in the bright clouds.
Right now I do this manually by not overexposing with 100 ISO and recording with 100/800

I thought about the 50% but that only works if the brighter image is 100% (all pixels) overexposed.
Canon EOS 60D

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2230 on: July 28, 2014, 08:00:40 AM »
When used with Dual ISO, ETTR measures the darker sub-image only.

golem

  • New to the forum
  • *
  • Posts: 8
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2231 on: July 28, 2014, 03:17:56 PM »
When used with Dual ISO, ETTR measures the darker sub-image only.

Yes, now it works - not sure what went wrong in my first test - thanks.
Canon EOS 60D

handbanana

  • Freshman
  • **
  • Posts: 88
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2232 on: August 09, 2014, 04:51:22 AM »
Any chance to ever get this working in Raw video mode for the 50D and 5D2?

chris_1975

  • New to the forum
  • *
  • Posts: 8
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2233 on: August 14, 2014, 08:30:41 AM »
Hi there. Thanks for DUAL ISO !!
Sorry, but I have since yesterday problems to convert the images correctly.
I have the RAW image in the dropbox.
The DNG file after conversion still has the lines can be seen. I have tried everything, but unfortunately can not find the error.

It was photographed with ISO 100/400
Bulb Timer 2min
Dark Frame

Thank you for your help and thank you for the great work that makes all your !!

Greetings
Christian

https://www.dropbox.com/sh/ocp8xl7q6u7a4jd/AABJQ7-LciZ3QFwmjTtCN2G-a

Edit:
The RAW was converted with the current cr2hdr.exe files with old files I still had, with the Lightroom plugin and without (ie directly in the folder)

glubber

  • Member
  • ***
  • Posts: 118
  • 550D
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2234 on: August 14, 2014, 02:32:53 PM »
@chris_1975:

It looks like you have chosen an extreme exposure situation, so cr2hdr-algorithm picks a wrong rendering-choice.

I tried with some cr2hdr-betas and it worked out fine:
Jpgs developed with ufraw

latest cr2hdr:


cr2hdr-amaze-edge5:


cr2hdr-20bit:


Btw. from my experience Dual-ISO doesn't improve a longtime exposure at night


I copied below the output of the command window for further research by a pro ;D

Code: [Select]
E:\ufrawconvert>cr2hdr sepp.cr2
cr2hdr: a post processing tool for Dual ISO images

Last update: d0ac769 on 2014-01-23 10:13:39 UTC by alex:
cr2hdr: experimental white level routine with two separate levels ...

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default
)

Input file      : sepp.cr2
Camera          : Canon EOS 5D Mark III
Full size       : 5920 x 3950
Active area     : 5796 x 3870
Black borders   : 124 left, 80 top
Black level     : 1965
ISO pattern     : BddB RGGB
White levels    : 15301 15300
Noise levels    : 10.82 8.34 8.34 10.74 (14-bit)
Black delta looks bad, skipping correction
ISO difference  : 1.78 EV (344)
Black delta     : 0.00
Dynamic range   : 10.64 (+) 10.27 => 12.05 EV (in theory)
Looking for hot/cold pixels...
Hot pixels      : 3080
Cold pixels     : 162
AMaZE interpolation ...
Amaze took 6.84 s
Edge-directed interpolation...
Semi-overexposed: 0.00%
Deep shadows    : 51.67%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 5.9 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 53.43 (16-bit), ideally 37.05
Dynamic range   : 9.96 EV (cooked)
Black adjust    : -5
Output file     : sepp.DNG
    1 image files updated

----------------------------------------------------------------------------------------------


E:\ufrawconvert>cr2hdr-20bit sepp.cr2
cr2hdr: a post processing tool for Dual ISO images

Last update: 26b7a08 on 2014-05-06 20:04:08 UTC by a1ex:
cr2hdr: experimental option to embed the original raw into the DNG

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default
)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (def
ault)

Input file      : sepp.cr2
Camera          : Canon EOS 5D Mark III
Camera model    : Canon EOS 5D Mark III
Full size       : 5920 x 3950
Active area     : 5796 x 3870
Black borders   : 124 left, 80 top
Black level     : 2048
ISO pattern     : BddB RGGB
White levels    : 15212 13794
Noise levels    : 10.81 8.32 8.34 10.73 (14-bit)
ISO difference  : 1.98 EV (395)
Black delta     : 1.98
Dynamic range   : 10.63 (+) 10.09 => 12.07 EV (in theory)
Looking for hot/cold pixels...
Hot pixels      : 1210
Cold pixels     : 156
AMaZE interpolation ...
Amaze took 3.40 s
Edge-directed interpolation...
Semi-overexposed: 0.58%
Deep shadows    : 82.23%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap     : 5.6 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 143.94 (20-bit), ideally 140.87
Dynamic range   : 12.51 EV (cooked)
Black adjust    : -46
AsShotNeutral   : 0.62 1 0.55, 4127K/g=1.20 (gray max)
Output file     : sepp.DNG
sepp.DNG        : updating EXIF


-----------------------------------------------------------------------------------------

E:\ufrawconvert>cr2hdr-amaze-edge5 sepp.cr2
cr2hdr: a post processing tool for Dual ISO images

Last update: af9a60f on 2013-10-24 18:44:01 UTC by alex:
cr2hdr: always use last 8 lines from top optical black bar (should fi...


Input file     : sepp.cr2
Canon EOS 5D Mark III detected
Full size      : 5920 x 3950
Active area    : 5796 x 3870
White level    : 12500
Black borders  : 124 left, 80 top
Black level    : 1965
ISO pattern    : BddB RGGB
Noise levels   : 10.82 8.34 8.34 10.74 (14-bit)
Estimating ISO difference...
ISO difference : 2.00 EV (399)
Black delta    : 5.90
Interpolation  : amaze-edge-chroma2x2-alias
AMaZE interpolation ...
Amaze took 6.79 s
Edge-directed interpolation...
Semi-overexpo'd: 0.64%
Deep shadows   : 53.94%
Dynamic range  : 10.30 (+) 9.93 => 11.92 EV (in theory)
Matching brightness...
Horizontal stripe fix...
Looking for hot/cold pixels...
Hot pixels     : 877
Cold pixels    : 171
Full-res reconstruction...
ISO overlap    : 5.3 EV (approx)
Half-res blending...
Chroma filtering...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level    : 9.79 (16-bit), ideally 9.53
Dynamic range  : 12.07 EV (cooked)
Black adjust   : -8
Output file    : sepp.DNG
    1 image files updated

        
EOS 550D // Sigma 18-200 // Sigma 18-70 // Canon 10-18 STM

chris_1975

  • New to the forum
  • *
  • Posts: 8
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2235 on: August 14, 2014, 05:33:53 PM »
THX Glubber...

now i tried it with "cr2hdr-hot" ... it works .... thanks...

regards chris

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2236 on: August 18, 2014, 10:26:12 AM »
The long drawn out discussion about learning how to process dual ISO files, has been moved here:  http://www.magiclantern.fm/forum/index.php?topic=13028.0

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7553
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2237 on: August 22, 2014, 08:59:56 AM »
Hi! I see a bug in cr2hdr when converting from dualiso dng to regular dng.
I,m wondering if there is anything to do regarding this issue?
https://bitbucket.org/hudson/magic-lantern/issue/2091/cr2hdr-dual-iso-3x-zoom-mode-strange
I,ve been trying to get some answers before but with little success. Also wondering if where I,ve placed this issue-report on bitbucket is the right place?


Thanks
/D

KurtAugust

  • Freshman
  • **
  • Posts: 73
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2238 on: August 25, 2014, 10:25:42 AM »
Code: [Select]
$ ./cr2hdr
cr2hdr: a post processing tool for Dual ISO images

Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
cr2hdr: print a message when overwriting the output file

Download cr2hdr-20bit for Mac only with last commit 30a5132 ;)

@a1ex thanks for having solved the warnings on compilation!

I am totally hooked on how good Dual ISO MLV's look on a Canon 5D3 to get nicer highlights, but I've broken my teeth on this for several days now and I'm dying to see the latest improvements in quality and less flicker in my files. I can't get this version to work on my mac and I'm very sorry to ask for help.

dcraw & exiftool are installed. The version from 4/10/13 works as expected.

It would be great if there was a complete package out that that people can run without any further installations. I'm trying every approach and I understand if a developer doesn't feel like pushing every update to the mac platform, but perhaps there's someone out there to help me out. Or point to other useful threads. Of course you can also reply here:
http://www.magiclantern.fm/forum/index.php?topic=13028.0

Many thanks!

Care to provide a step by step or checklist of all the stuff that should be installed in the tool chain to get this working.
www.kurtaugustyns.com @HetRovendOog

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2239 on: August 25, 2014, 03:54:03 PM »
If you teach me how to cross-compile a Mac binary from Linux, I can add it to the nightly build server. I wanted to include the postprocessing tools on that page for a long time.

KurtAugust

  • Freshman
  • **
  • Posts: 73
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2240 on: August 25, 2014, 05:54:46 PM »
I'm sorry. I can not be of too much much help there (yet). Sorry.

On the other hand, a small step in the that direction has been taken as I got the 20bit version of cr2hdr to work. And very happy about that!

The steps I took:
-Move to a computer with a different OS. OS X 10.9.4 instead of OS X 10.6.8 (the cr2hdr didn't run on 10.6.8 )
-Installed dcraw with this tutorial: http://vkphotoblog.blogspot.be/2014/05/dcraw-921-for-os-x-mavericks-users.html
(including xcode installation) (interesting read too)
-Move cr2hdr to the same folder as dcraw is in. Allows you to just type the cr2hdr command without specifying a directory.
-Install exiftool from here http://www.sno.phy.queensu.ca/~phil/exiftool/
-Run the program from the terminal
-Say fuuuuuuh...!

I will see if I can now also make it work on a OS X 10.6.8 machine.
www.kurtaugustyns.com @HetRovendOog

kichetof

  • Contributor
  • Senior
  • *****
  • Posts: 469
  • Take a beer and enjoy it!
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2241 on: August 26, 2014, 11:56:10 AM »
If you teach me how to cross-compile a Mac binary from Linux, I can add it to the nightly build server. I wanted to include the postprocessing tools on that page for a long time.

Hi a1ex, I found OSXCross maybe it will works to compile it ! :) (another way)
If you need some help/files, I'll send to you !

I am totally hooked on how good Dual ISO MLV's look on a Canon 5D3 to get nicer highlights, but I've broken my teeth on this for several days now and I'm dying to see the latest improvements in quality and less flicker in my files. I can't get this version to work on my mac and I'm very sorry to ask for help.

dcraw & exiftool are installed. The version from 4/10/13 works as expected.

If you run cr2hdr into the Terminal, could you post the output ? And what's you dcraw version ? 9.21 works for cr2hdr 20bit

KurtAugust

  • Freshman
  • **
  • Posts: 73
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2242 on: August 26, 2014, 02:48:01 PM »
If you run cr2hdr into the Terminal, could you post the output ?
Illegal instruction

And what's you dcraw version ? 9.21 works for cr2hdr 20bit
9.19 works too for cr2hdr 20bit on 10.9.4 , but not on os x 10.6.8
www.kurtaugustyns.com @HetRovendOog

kichetof

  • Contributor
  • Senior
  • *****
  • Posts: 469
  • Take a beer and enjoy it!
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2243 on: August 26, 2014, 04:39:16 PM »
Illegal instruction

Very strange ! It works well on my 2  mac (10.9.4 too)

9.19 works too for cr2hdr 20bit on 10.9.4 , but not on os x 10.6.8

To works on my mac, I need dcraw 9.21.

Quote from: OSXCross
From OSXCross:
Note: libc++ requires Mac OS X 10.7 or newer!

Maybe this is the reason, I don't know... I compiled on Mac OS X 10.9.3

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1741
  • 6d - Nightly build user
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2244 on: August 27, 2014, 06:49:17 PM »
tried dual iso video in normal video mode a few times (so I didn't use crop mode)
I always end up with visual horizontal stripes with a slightly brightness difference. (And yes I processed my files  ;) (with CR2HDR v2.0)
And I don't use extreme values, mostly iso 100/800

Is it possible to get normal results with dual iso when not in crop mode ?
Or is impossible because the lineskipping and dual iso won't give a normal RGGB pixel readout in the same iso value ?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2245 on: August 27, 2014, 06:57:44 PM »
I don't see any stripes ;)

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1741
  • 6d - Nightly build user
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2246 on: August 27, 2014, 07:12:10 PM »
Ofcourse I will show some examples  ;)

See the png's in this folder (It's not showing the entire frame, but I think it shows what I'm talking about)

https://drive.google.com/folderview?id=0B1BxGc3dfMDaZWxfLTlVQmxnU1k&usp=sharing


a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2247 on: August 27, 2014, 07:44:16 PM »
How did you get these png's? Were they straight out of the camera?

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1741
  • 6d - Nightly build user
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2248 on: August 27, 2014, 08:00:25 PM »
Those png's are screenshots of the processed frames (mlv-dump osx -> CR2HDR -> RawTherapee -> screenshot...)
I'm uploading some more files to the same folder now.
An original frame out of mlv-dump osx (renamed to CR2), and the same frame processed by CR2HDR (the dng).

This is recorded using the nightly build for 6d from March 15, 2014, using this build still a lot because it gives me the option for FPS override to 35fps.




a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2249 on: August 27, 2014, 08:08:27 PM »
Can you try with cr2hdr-20bit?

(you may have to run it under Wine, or maybe kichetof has a Mac binary here)