Magic Lantern Forum

Developing Magic Lantern => Modules Development => Topic started by: a1ex on July 16, 2013, 06:33:50 PM

Title: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on July 16, 2013, 06:33:50 PM
Time for the first real image quality improvement in ML history :)

Here's a trick that samples half of the sensor at ISO 100 and the other half at ISO 1600, for example. If you mix these two, you can get almost the entire dynamic range the sensor is capable of (around 14 stops). 5D3/7D only. Now it works on most cameras!

There are no motion artifacts; not even a difference in motion blur for the two exposures. So, it works well for fast moving subjects.

(http://acoutts.com/a1ex/diso/014.jpg) (http://www.magiclantern.fm/forum/index.php?topic=7402.0) (http://acoutts.com/a1ex/diso/dual_iso_test.jpg) (http://www.magiclantern.fm/forum/index.php?topic=7402.msg63104#msg63104) (http://acoutts.com/a1ex/diso/TFzi2Uk.jpg) (http://www.magiclantern.fm/forum/index.php?topic=7402.msg71370#msg71370) (http://acoutts.com/a1ex/diso/fireworks.jpg) (http://www.magiclantern.fm/forum/index.php?topic=7139.msg65307#msg65307) (http://acoutts.com/a1ex/diso/9255_594194790623658_495677599_n.jpg) (http://www.magiclantern.fm/forum/index.php?topic=7139.msg74445#msg74445)

Please post your samples here -> Thread photos dual ISO (http://www.magiclantern.fm/forum/index.php?topic=7402)

More samples on Flickr (tag: Dual-ISO) (http://www.flickr.com/search/?q=dualiso)
Tutorial - Guide to using Dual ISO Mode on your Canon 5D Mark III (http://rohidassanap.wordpress.com/2013/08/10/guide-to-using-dual-iso-mode-on-your-canon-5d-mark-iii/)
Open Source Brings High-End Canon 5D MK III Dynamic Range Closer to Nikon D800 (http://www.zdziarski.com/blog/?p=2792)
Dual ISO vs Exmor: on CanonRumors (http://www.canonrumors.com/forum/index.php?topic=22230.0), reddit (http://www.reddit.com/r/photography/comments/2d6qiw/exmor_vs_dualiso_full_writeup/), POTN (http://photography-on-the.net/forum/showthread.php?t=1393255) and FredMiranda (http://www.fredmiranda.com/forum/topic/1316361).

Code
- dual_iso module (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso)
- recent nightly builds (http://builds.magiclantern.fm/#/) should have this working on most cameras

Technical doc
- dual_iso.pdf (http://acoutts.com/a1ex/dual_iso.pdf) (in-depth description of how it works)
- See also: ADTG and CMOS discussion (http://www.magiclantern.fm/forum/index.php?topic=6751)

Warning
This code changes low-level sensor parameters. In the technical doc you can see how this method messes with the feedback loop for optical black, for example. Since the original implementation, many people have tried it (including timelapse), with very few side effects reported (e.g. more hot pixels than usual on long exposures). However, that's not a guarantee.

We don't pay for repairs. Use it at your own risk.


Usage
- Select one ISO from Canon menu, the other from Dual ISO submenu.
- Expose to the right for the lower ISO (usually ISO 100). Maybe darken 1 stop from there.
- Start at ISO 100/800.
- Try a larger gap for really extreme situations, or a smaller gap for better midtone detail (less aliasing).

Histograms, ETTR, playback tricks

- ETTR integrates with dual ISO (just enable both). It will use dual ISO only in high dynamic range scenes. Adjust the SNR limits to let the algorithm know what exactly is a "high dynamic range scene" and to fine-tune the balance between midtone/shadow noise and highlight aliasing.

- Raw zebras: weak markers appear where only one of the two ISOs is overexposed (likely to get aliasing), strong markers appear where both ISOs are overexposed (there's no more detail in these areas).  See here for examples. (https://www.magiclantern.fm/forum/index.php?topic=12096.0#post_dualzebras)

- Raw histogram will use the darker exposure. Consider the brighter one as some sort of shadow recovery.
- JPEG zebras/histogram are totally unreliable.
- In photo LiveView you will see the ISO from Canon menu. In movie mode, you will see horizontal lines.
- Playback in 1x will show some sort of HDR preview. At certain zoom levels, you'll be able to see only one of the two exposures, at very low resolution.
- The dynamic range indicator from the histogram is not accurate. (solved!)

Postprocessing

- Reference converter: cr2hdr.c (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso/cr2hdr.c) (Windows: cr2hdr.exe (https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/)). It requires dcraw and (optional) exiftool in your executable path (they are in the zip). Drag and drop the CR2 files over cr2hdr.exe.

- For RAW video files, use the following commands in a terminal:
Code: [Select]
mlv_dump clip.mlv --dng --no-fixcp --no-stripes
cr2hdr --same-levels *.DNG

- GUI frontends:
   - Lightroom plugin (http://www.magiclantern.fm/forum/index.php?topic=11056) (WIN/MAC)
   - BarracudaGUI (http://www.magiclantern.fm/forum/index.php?topic=9102) (WIN)
   - Script for multiple instances of cr2hdr (http://www.magiclantern.fm/forum/index.php?topic=8520.0) (WIN, VBScript)
   - Mac GUI for cr2hdr: OSX_cr2hdr_app.zip (https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip) and OSX_cr2hdr.zip (https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr.zip)
   - MLV to DNG Batch Converter (http://www.magiclantern.fm/forum/index.php?topic=10198.0) (WIN, video only)
   - MLP (http://www.magiclantern.fm/forum/index.php?topic=13512) (MAC)

- To get a natural HDR look: try my automatic color grading script (http://www.magiclantern.fm/forum/index.php?topic=7022).

Price to pay
- Half resolution in highlights and shadows
- Aliasing and moire - of course, in highlights and shadows
- You can no longer check critical focus when zooming in

Samples

This is actually a video frame from 5D Mark III. Here's the DNG (http://acoutts.com/a1ex/000000.dng).
Bottom-left: ISO 100. Top-right: ISO 100/1600.
Credits: Luke Neumann.

(http://i.imgur.com/I0gYqqD.jpg)

The next one is a still photo from 5D Mark III. Here's the DNG (http://acoutts.com/a1ex/0R0A0640-fullres-soft.DNG) and the original CR2 (http://neumannfilms.mediafire.com/download/6tvihparmfwfbch/0R0A0640.CR2). ISO 100/1600.

(http://i.imgur.com/ezz8sAR.jpg) (http://acoutts.com/a1ex/0R0A0640-fullres-soft.jpg)




Questions? Read the pdf (https://bitbucket.org/hudson/magic-lantern/downloads/dual_iso.pdf) first. The mystery should clear up, I hope.

That's it. Looking forward to your samples. (http://www.magiclantern.fm/forum/index.php?topic=7402)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Andy600 on July 16, 2013, 06:39:28 PM
a1ex - this is amazing! Super impressed.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: driftwood on July 16, 2013, 06:47:18 PM
Excellent work a1ex.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: iaremrsir on July 16, 2013, 07:14:27 PM
 :o Mother of God, this is amazing! You guys never fail to impress!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: platu on July 16, 2013, 07:15:56 PM
Wow... so clever!   And so useful!!  Thank you for your work on this and for the in depth technical docs.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ilguercio on July 16, 2013, 07:18:43 PM
It's all very interesting but it's a shame that you can only use the trick on the 5DIII and sort of on the 7D.
I guess that this research could easily lead into other interesting things, being able to control the sensor readout is a BIG thing.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: critycal on July 16, 2013, 07:24:37 PM
Is there any possibility to port it to the 5DII
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: D.L. Watson on July 16, 2013, 07:33:00 PM
That's too bad it only works on the 5D Mark III. Oh well, loving the images out of my 50D (http://"https://vimeo.com/70206717").
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 16, 2013, 07:33:46 PM
This is too crazy. Who could ever imagine. The mother of all updates, with raw video to. E x c e l l e n t!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Stedda on July 16, 2013, 07:43:07 PM
Amazing, thank you for more great features!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: anchoricex on July 16, 2013, 07:43:48 PM
Please, for everyones sake people, don't come in here and ask "when can we expect this for <insert canon model here>"

The 5dmkiii is a highly capable camera and with it's hardware it makes sense that many of these new developments are coming forward on this platform. I, myself, moved on from my 550d to the 5dmkiii because of the future course of Magic Lantern developments.

This is an insane development to come in the midst of all the raw development. So much new stuff to go forward with, I'd imagine it's going to be quite some time before a stable release is on the horizon
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: johnha on July 16, 2013, 07:45:09 PM
Incredible work. The white paper was illuminating. Thank you A1ex!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jaybirch on July 16, 2013, 07:50:32 PM
Words fail me.... Amazing.

Look forward to testing this out
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: rmihai81 on July 16, 2013, 07:52:26 PM
Nice work my friend!!!!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ilguercio on July 16, 2013, 07:54:51 PM
The way i understand it is that no further moire or aliasing is introduced in raw while recording vs stills.
This is amazing.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: discocalculi on July 16, 2013, 07:55:09 PM
Awesome! Should buy myself some new cameras instead of lenses. Looks promising.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Stedda on July 16, 2013, 07:55:47 PM
Please, for everyones sake people, don't come in here and ask "when can we expect this for <insert canon model here>"

Amen, just reading the PDF should answer that question. Other cameras aren't sold with the same level of hardware as the higher end cameras.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Kabuto1138 on July 16, 2013, 08:03:50 PM
Amazing, I had to read this 3 times so I can make sure what I was reading was what think it is.  Holly crap.  Is official, 5dIII is a mini Alexa.

BTW Alex, I can't access the PDF you posted in Dropbox.  It gives me an error.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Frank209 on July 16, 2013, 08:05:19 PM
Error (509)
This account's public links are generating too much traffic and have been temporarily disabled!



This just means that we all love you A1ex!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jc on July 16, 2013, 08:08:27 PM
sounds an amazing feature!!! so useful for HDR stills with motion in them .. and for video might be even more amazing
I can't however get any of the links to work? they say :

Error (509)
This account's public links are generating too much traffic and have been temporarily disabled!

Could the files be hosted elsewhere please?

James
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: wintix on July 16, 2013, 08:22:40 PM
i took the liberty of mirroring it:

http://linuxdingsda.de/~wintix/dual_iso.pdf
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: duncanidaho25 on July 16, 2013, 08:33:33 PM
wow!  wish i had a 5D3 and not a 5D2.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Rick M on July 16, 2013, 08:35:06 PM
Awesome work; I don't have a 5DIII, so wont' be able to enjoy the cool stuff like this, but it's definitely a sign of progress that you guys are getting deeper into how the sensors are working and what not.

Random question that has to do with the sensor workings: On lower end models, would it be possible to reduce moire/anti-aliasing by modifying the sensor read outs or something similar? I know the 5DIII is typically moire/anti-aliasing free, but I'm not quite sure how or why that is. I know there's also the VAF filters available for lower end cameras, but still, with this type of manipulation I'm curious if there would be a way to reduce this common problem on lower end models.

Thanks for the awesome work you guys do and any time it takes to answer my question (which might be really foolish).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 16, 2013, 08:39:30 PM
Magical !!!!!!

Although the bad side effects (low resolution moire aliasing) are serious defects.

Just thinking loudly ..
What if you try with alternating ISO line by line and use no interpolation just scale the not-burned hiISO pixels by low/hi (zero point should be the Black level). Then a good highlight recovery algo can make the job it knows best ...
This could decrease the DR expansion a bit (1 stop ??) but I think denoise algos can make a good job there also .. I am thinking of denoise at raw level like those at the "RAW" tab of RawTherapee
 
Line by line interlacing will give the opportunity to use hi ISO on the weak line depending on the WB (red for daylight, blue for tungsten) ..   
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: haemma on July 16, 2013, 08:41:03 PM
Amazing Work! Thanks for your effort.  :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dhilung on July 16, 2013, 08:57:59 PM
Brilliant work!

Interesting black-level drift analysis and correction.

Btw, I also put a mirror at http://dhilung.com/vault/ml/dual_iso.pdf (http://dhilung.com/vault/ml/dual_iso.pdf)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: hetfanatic on July 16, 2013, 09:30:33 PM
This would be very interesting to try out with my 7d. So far ML is amazing (even though it's not developed like 5Ds etc) and this would be so great to have. At least for fun. However, I'm not sure, since I'm not a developer of any sort, how to get this working on my 7D? Is it possible at this stage? If anyone can help, I'd be thankful.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: daisermac on July 16, 2013, 09:37:39 PM
Very cinematic images - can't wait to see the first video footage. A1ex you are a genius - probably you will soon be abducted by some spy agency to work for their super secret decoding missions...
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jrumans on July 16, 2013, 09:46:13 PM
Anxious to test this out, tried putting these files on my card but I know I am missing a step.  I guess we need to wait for Lourenco to compile for 5diii users?

https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Stedda on July 16, 2013, 09:47:17 PM
Anxious to test this out, tried putting these files on my card but I know I am missing a step.  I guess we need to wait for Lourenco to compile for 5diii users?

https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso

You're missing autoexec.bin
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jrumans on July 16, 2013, 10:11:29 PM
You're missing autoexec.bin

Thanks for the heads up.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Steve Kahn on July 16, 2013, 10:18:12 PM
Would love to see a video of this!  Link?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ml-images on July 16, 2013, 10:23:12 PM
Hi,

Impressive work! One little question, I read in your pdf that the native ISOs are 100, 200 and so on, but some videos demonstrated that in video mode there is much less noise (aka native ISOs) on 160, 320, 640, ... at least on the 7D (growing noise order is 160, 320, 640, 100, 200, 400, 800, 1250, 125, 250, 500, 1000, 1600, 2500, 2000, 3200, 4000, 5000, 6400). Is there any logical explanation I'm missing here?

Regards, Michel
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Redrocks on July 16, 2013, 10:27:01 PM
@ml images Were those videos talking about ML RAW or just plain old h264? I think A1ex mentions in his PDF that that does not apply here.


"As expected, intermediate ISOs like 160 or 250 do not cause any changes in ADTG/CMOS configuration.
These ISOs are obtained by applying some digital gain to the raw data acquired
at the nearest full-stop ISO, and this gain is configured from the DIGIC register 0xC0F08030
(SHAD GAIN). In LiveView, the gain is only applied to the YUV image (it does not affect the 14-bit
raw data at all), but in photo mode, the gain is burned into the raw data. Don’t ask me why."
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ml-images on July 16, 2013, 10:41:58 PM
@Redrocks indeed I'm talking Canon H264 here, not ML files. So my understanding is that Canon does change ADTG/CMOS configuration for 160, 320, 640 and doesn't for 100, 200, 400 which are obtained in software post processing. Does this sound correct?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: legreve on July 16, 2013, 10:42:32 PM
I'm curious.... How much does the 5D mk III sensor differ from the sensor of fx the Alexa?

Could one force the data handling and interpretation from the Alexa down on the 5D chip?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: eatbuckshot on July 16, 2013, 11:15:13 PM
@Redrocks indeed I'm talking Canon H264 here, not ML files. So my understanding is that Canon does change ADTG/CMOS configuration for 160, 320, 640 and doesn't for 100, 200, 400 which are obtained in software post processing. Does this sound correct?

I believe native ISOs refer to the available analog ISO gains to the sensor, which are the 100,200,etc. (set in the CMOS #0 register)

The ADTG/CMOS configuration can adjust only in those native increments, and the third stop gains are applied digitally after the nearest native ISO settings have been set in the configuration.

If I recall correctly, some videos that demonstrated the lower noise in 320, 640, etc were due to the fact that these are negative digital gain applied to the closest ISO of 400, 800, etc... which on our canon dslrs result in better shadow noise due to the fact that it is effectively "exposing to the right" then stopping down.  Even the PDF says that Canons are notorious for having bad shadow recovery.  This is why 125, 250, 500 yield higher noise since they are taking ISO 100,200,400 and digitally gaining +1/3 EV
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: horshack on July 16, 2013, 11:22:00 PM
It's interesting that Canon implemented this capability in their sensor. I believe the technique is patented by Eric Fossum, the inventor of the active-pixel CMOS sensor. Here is one reference to the idea on dpreview; there are other references but I couldn't find them quickly: http://www.dpreview.com/forums/post/32010565
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dadinio13 on July 16, 2013, 11:43:01 PM
after installing this we can remove it and return to the previous build ?

or the sensor settings are modified "for ever" ?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: togg on July 16, 2013, 11:53:38 PM
Impressive. For real.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Redrocks on July 17, 2013, 12:15:45 AM
Most Online Today: 564. Most Online Ever: 564 (Today at 07:09:43 PM)

That's about twice the previous high and it looks like this has gotten everyone's attention. An old 4k thread got bumped the other day and I'd just been reading about how Arri are reluctant to move on from 2k, obviously to squeeze more money out of what they already produce, but also they feel that the current tech hasn't been fully exploited. Now this, The 5d3's shelf life just extended a few years.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: hjfilmspeed on July 17, 2013, 01:04:28 AM
Hmmmmm half rez in shadows and highlights plus some moire. That might be ruff on video. Would be less of and issue on stills though. This is really interesting though. Amazing work!!!! Didnt even think this kind of thing was possible. I thought the same about raw vid from a dslr tho.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: payne on July 17, 2013, 01:16:13 AM
Does it seem even remotely possible that this type of functionality could later be accessed for stills in APS-C lines that share a cut down version of the same sensor (if not the exact same sensor) as the 7D such as T2i and 60D?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: crazyrunner33 on July 17, 2013, 01:35:04 AM
Does it seem even remotely possible that this type of functionality could later be accessed for stills in APS-C lines that share a cut down version of the same sensor (if not the exact same sensor) as the 7D such as T2i and 60D?

Probably not; the 5D Mark III has an 8 channel readout along with the 7D, the others have a 4 channel readout from the sensor. 
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on July 17, 2013, 01:45:16 AM
@a1ex,

would a 7D with VAF have minimal aliasing and moire?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: mesebar2 on July 17, 2013, 01:50:36 AM
You are a Genius.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 17, 2013, 02:52:20 AM
I can't get the windows executable to work.

dcraw is working fine, but cr2hdr reports "no such file or directory".

Code: [Select]
cr2hdr D46A8015.CR2

Input file     : D46A8015.CR2
'D46A8015.CR2': No such file or directory
Error: D46A8015.CR2

Wonder if it's related to this?

Quote
snprintf(dcraw_cmd, sizeof(dcraw_cmd), "dcraw -v -i -t 0 '%s' > tmp.txt", filename);
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jkdjedi on July 17, 2013, 03:16:12 AM
 :)ANy Links to this MAgic?? Would like to try this cool hack.  8)  (Canon 7D)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 17, 2013, 03:20:30 AM
I've got a 5D3 build here (https://www.dropbox.com/sh/xiksvwbehzmycq4/QbuV17WuE5).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jkdjedi on July 17, 2013, 03:28:48 AM
Thanks Audionut, I wish I had the 5DMiii  :(

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: aaphotog on July 17, 2013, 04:13:01 AM
I've got a 5D3 build here (https://www.dropbox.com/sh/xiksvwbehzmycq4/QbuV17WuE5).
Everything goes into the 'modules' folder, except for the autoexecbin which goes to the root folder of the card, right?

Also, when this newest build is on my card, there is a possibility to shoot normal video(without the double iso thing), right?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: crazyrunner33 on July 17, 2013, 04:17:45 AM
Everything goes into the 'modules' folder, except for the autoexecbin which goes to the root folder of the card, right?

Also, when this newest build is on my card, there is a possibility to shoot normal video(without the double iso thing), right?

Sounds right for the file locations.  I believe H.264 video shouldn't be changed and you should be able to disable the module, but there's only one way to find out.  I would experiment with it tonight, but I don't dare try something this new the day before a big shoot.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on July 17, 2013, 04:26:01 AM
Hopefully reg addresses are the same on 6D. I need a reliable way to figure those out.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: rod1000rj on July 17, 2013, 07:08:25 AM
Dear A1ex, your team work is effortless, generous and FREE. I wonder how come we see people here complaining a new funcionality does work on this or that model. I here to say, I deeply thankfull for the work you do even if my camera is not able to new funcionalities just because there are people like you and your team putting hours, knowledge and inteligence to the benefit of all of us. THANK YOU MAGIC LANTERN!!!!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: imme on July 17, 2013, 07:40:26 AM
Amazing stuff as always.

A1ex  hacking sensor to its roots is really good. Can I expect RGB Raw output from sensor now?

I want three RAW files for one click. Red RAW, Blue RAW and Green RAW. Sensor recording for only one color per RAW file and producing 3 RAW files.

We have Foveon sensor technology from Sigma that has 3 layered sensor. Each layer is dedicated to just one color. Can we mimick it with ML now.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: MA Visuals on July 17, 2013, 07:50:17 AM
For those wanting to see a test of the Dual ISO feature in use, I put together a video comparing normal raw video and video shot using the new Dual ISO feature.  More details and testing methods used can be found at the Vimeo link... http://vimeo.com/70459941 (http://vimeo.com/70459941).

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: aachrisg on July 17, 2013, 07:59:53 AM
Wow, this is quite exciting and impressive. How dysfunctional of canon to not exploit the fact that they'd basically made a camera which can capture a true HDR image in one shot.

It should be possible to do some quite sophisticated denoising of the iso1600 image guided by the data in the iso100 one.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 17, 2013, 08:09:25 AM
Updated cr2hdr.exe, works for me under Wine. The dependencies (dcraw and exiftool) are included in the zip file.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: VisualPursuit on July 17, 2013, 09:05:36 AM
I've got a 5D3 build here (https://www.dropbox.com/sh/xiksvwbehzmycq4/QbuV17WuE5).

Does this one work on firmware 1.2.1?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Lichtgestalt on July 17, 2013, 09:27:26 AM
Dears, I'm sorry for maybe stupid question, but I really want to know..
Is it possible to read one shot twice? I mean - read sensor at iso 100, save it to buffer, and immediately read the same data at iso 1600? For CCD sensors it is definitely impossible, but for CMOS sensor it is not so obvious. So could you please help me?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sebamax on July 17, 2013, 09:31:56 AM
Hi,
is possible postprocessing dual iso's files on mac with raw2dng?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Walter Schulz on July 17, 2013, 09:38:20 AM
is possible postprocessing dual iso's files on mac with raw2dng?

Yes, has to be consumed with WINE.
http://www.magiclantern.fm/forum/index.php?topic=5404.msg34763#msg34763

Ciao, Walter
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: squig on July 17, 2013, 09:41:49 AM
Dears, I'm sorry for maybe stupid question, but I really want to know..
Is it possible to read one shot twice? I mean - read sensor at iso 100, save it to buffer, and immediately read the same data at iso 1600? For CCD sensors it is definitely impossible, but for CMOS sensor it is not so obvious. So could you please help me?

What about 720p 48p? A1ex would that be a way to get around the lineskipping?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: squig on July 17, 2013, 09:43:44 AM
Yes, has to be consumed with WINE.
http://www.magiclantern.fm/forum/index.php?topic=5404.msg34763#msg34763

Ciao, Walter

Yep wine for now until scrax has the time to update the mac raw2dng.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sebamax on July 17, 2013, 09:49:35 AM
Grazie Walter,
think will be possibile compiling an update to raw2dng ?
I hope in scrax
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: lagunak47 on July 17, 2013, 09:58:27 AM
isnt the whole point of raw to record the entire RAW image the sensor sees? and isnt iso on the canons just digital gain? So how if this gaining any DR over just shooting raw if iso has no effect on raw?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 17, 2013, 10:00:26 AM
Ouch.

(https://dl.dropboxusercontent.com/u/34113196/ML/D46A8019.jpg)


Reducing the image size to around 33% all but eliminates the problems in the tree.  But the power line is beyond repair.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 17, 2013, 10:05:51 AM
Here, a different interpolation method may give different results. Try choosing another one (in cr2hdr.c, there are some defines at the top).

I guess plain averaging will handle this one a bit better than medians.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Redrocks on July 17, 2013, 10:27:37 AM
isnt the whole point of raw to record the entire RAW image the sensor sees? and isnt iso on the canons just digital gain? So how if this gaining any DR over just shooting raw if iso has no effect on raw?

The real point of RAW is to be working with uncompressed footage in post. My understanding is this technique is like hdr at the photosite level and of course ISO affects RAW - try using footage shot at 6400+
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 17, 2013, 10:33:15 AM
Yep wine for now until scrax has the time to update the mac raw2dng.
Oh man, Wine again :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: squig on July 17, 2013, 10:46:24 AM
Oh man, Wine again :)

I think I'll wait it out.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 17, 2013, 11:08:07 AM
I think I'll wait it out.

Keeping my fingers crossed for Scrax to come through. This is beautiful work!!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dariSSight on July 17, 2013, 11:24:46 AM
Can you use this module with the 5D Mark II and where do I find it?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Walter Schulz on July 17, 2013, 11:28:06 AM
First post in this thread, third line. End of it.

Ciao, Walter
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: edwmotion on July 17, 2013, 11:32:15 AM
Good news for the 7D owners. Thank you Alex, we appreciate your efforts  ;)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: kgv5 on July 17, 2013, 11:32:41 AM
Hopefully reg addresses are the same on 6D. I need a reliable way to figure those out.

Go get them 1%  :D
Finally some hope for the 6d owners :)
With almost the same specs and iso performance as 5d3 maybe you could port this in some time. Fingers crossed.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Dns on July 17, 2013, 11:44:45 AM
No hope for poor people like me(550D)? Finger Crossed :(
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Walter Schulz on July 17, 2013, 11:51:32 AM
Read the White Paper and cry. You're not the only one, that's for sure.
Also linked in the first post of this thread: http://acoutts.com/a1ex/dual_iso.pdf
See 3.1

Ciao, Walter
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Braam544 on July 17, 2013, 02:23:53 PM
Help me, I need a little tutorial to install this update. thank you guys for helping me.

Thank you

B.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: crazyrunner33 on July 17, 2013, 02:42:17 PM
Help me, I need a little tutorial to install this update. thank you guys for helping me.

Thank you

B.

Keep an eye out on the nightly builds and follow the instructions for installing the RAW video builds.  If you have difficulty doing so, then it would be probably be best to wait for an official release.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: swinxx on July 17, 2013, 03:04:16 PM
Thats so great! Will this work with magicraw?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: hbr on July 17, 2013, 03:04:56 PM
Thank you! Finally some good news for 7D owners! Is there any build to test out for 7D?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 17, 2013, 04:08:30 PM
First of all, wow, wow, wow!

A few questions. I may have missed this in reading through the doc, so apologies if so. Is it correct that when in this mode, that highlights will always be half-res regardless of the DR of the scene? If this is so, it sounds like we need to be careful when shooting a flat scene (low DR) and exposing to the right. You could end up with a substantial portion of the frame in half-res.

And if that's true, is it possible to perform the dual-ISO trick only as much as the DR of the frame calls for? For lower-DR scenes it seems like maintaining highlight fidelity should be the first priority. As DR increases, I'd want to sacrifice shadow resolution first, and then highlight resolution last. Or maybe the other way. Or maybe symmetrically.

Also, is there something special about 4 stops? I mean how about the option to run 100/6400 for a full 16 stops, understanding that the extra 2 bits will also be at half-res?.

I can't wait to start shooting with this. The few videos I've seen posted so far are like miracles. Soon we'll all be lifting shadows bravely and chuckling over "noise reduction". Which reminds me, care to SHARPEN? Don't mind if I do.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: brapodam on July 17, 2013, 04:24:51 PM
No hope for poor people like me(550D)? Finger Crossed :(
I'm by no means an expert on this, so take my words with a pinch of salt (or a tablespoon for that matter)  :)

They need to program the ISO for each separate amplifier circuit for this trick to work. If the "poor men's cameras" don't have separate amplifier circuits, they have to find another way to implement this, like maybe somehow finding a way to hack a single amplifier circuit to alternate ISO every two lines, if it's even possible.

I don't know if it has a lot to do with the channel readouts though; all I've read about that was that it helped with higher readout speed and hence higher drive speed and better contrast detect AF.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Steven on July 17, 2013, 04:50:40 PM
It sounds like it was possible to use this feature also as an on sensor gradual nd filter without the resolution loss even.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ajay on July 17, 2013, 06:10:10 PM
Thank again for another brilliant addition to the firmware for the 5DM3. I did some extensive still-shooting with the mod this morning...the dynamic improvement is incredible -- well done!

Just a few comments using the cr2hdr program (These are not complaints/criticisms, just letting you know what I experienced.):

1. Moire is extensive with bird feathers.
2. I'm seeing color artifacts around small, over-exposed areas. (By small, I mean points that are maybe 10 x 10 pixels or less.)

If anyone wants to see samples, let me know.

AJ
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 17, 2013, 06:14:22 PM
Yes, some samples will be useful.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: horshack on July 17, 2013, 06:19:46 PM
Alex, here is some feedback about the technique from theSuede:
http://www.fredmiranda.com/forum/topic/1227550/0#11686217
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: bp on July 17, 2013, 06:20:33 PM
A1ex, I know you don't need any more ego stroking, but gotta say, you're a flippin genius man.  Did some testing with this last night.

The extent to which you can push the RAW settings (shadows, highlights, exposure) around without noticeable noise is ASTOUNDING.  For video, however, the aliasing is noticeable enough to push this feature into the "only in extreme emergencies" category, for me at least.  And that's not meant to be a criticism in any way - when you skip lines, that's what will happen.  I will be watching for future developments on this, if you get any brilliant ideas on how to eliminate the aliasing, this would be downright unfair to every other camera on the market

By the way, I was looking around for the "Donate" link - would like to toss more into the kitty.  But couldn't find it anywhere.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ajay on July 17, 2013, 06:49:02 PM
Yes, some samples will be useful.

a1ex...Here are some 100% cropped images showing the artifacts. The first image shows extensive moire in the feather detail:

(http://iwishicouldfly.com/iwishicouldfly/images/moire1.jpg)

The second image shows the color halo effect. (Close-up of an orange.) This reminded me of the halo problem that Canon corrected when both the 5DM2 and 5DM3 were first introduced. It usually reared its ugly head with night-astro shots of stars or xmas lights at night:

(http://iwishicouldfly.com/iwishicouldfly/images/color_halo.jpg)

If you want the originals, let me know where I can upload them.

Thanks again for all your hard work. --AJ
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 17, 2013, 07:04:20 PM
@horshack,

I don't know who theSuede is (he seems to know some stuff though), but please ask him to read the PDF. I'm pretty sure he didn't.

In deep shadows I only use data from the higher ISO, therefore his averaging formula does not apply here.

@ajay,

Of course I need the original files. Use some file sharing service.

@all,

For aliasing, there's nothing you can do to remove it completely; but you can reduce the artifacts by using a better interpolation algorithm. Even the debayering method you use in your raw software matters, so try to fiddle with it.

Or, use less aggressive settings (e.g. ISO 100/400) to minimize the areas where aliasing may appear.

Here's an alternate cr2hdr/raw2dng, with the mean23 interpolation algorithm (see the source code for details). It has more jagged edges in the Batman shot and my other test shots, but should handle trees and power lines a bit better.

cr2hdr_mean23.exe (http://acoutts.com/a1ex/cr2hdr_mean23.exe)
raw2dng_mean23.exe (http://acoutts.com/a1ex/raw2dng_mean23.exe)

You can further reduce shadow aliasing by increasing the overlap amount (you'll get more noise). See the paper and the source code for details.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 17, 2013, 07:09:40 PM
First of all, wow, wow, wow!

A few questions. I may have missed this in reading through the doc, so apologies if so. Is it correct that when in this mode, that highlights will always be half-res regardless of the DR of the scene? If this is so, it sounds like we need to be careful when shooting a flat scene (low DR) and exposing to the right. You could end up with a substantial portion of the frame in half-res.

And if that's true, is it possible to perform the dual-ISO trick only as much as the DR of the frame calls for? For lower-DR scenes it seems like maintaining highlight fidelity should be the first priority. As DR increases, I'd want to sacrifice shadow resolution first, and then highlight resolution last. Or maybe the other way. Or maybe symmetrically.

Also, is there something special about 4 stops? I mean how about the option to run 100/6400 for a full 16 stops, understanding that the extra 2 bits will also be at half-res?.

I can't wait to start shooting with this. The few videos I've seen posted so far are like miracles. Soon we'll all be lifting shadows bravely and chuckling over "noise reduction". Which reminds me, care to SHARPEN? Don't mind if I do.

I hope the below data will answer your questions.  :) ;)

Based on DXO screen-DR measures

Code: [Select]
5DIII
       screen  per ISO  Total Gain
  ISO    DR     Gain    from base ISO
 0100   10.97    0.00    0.00
 0200   10.87    0.90    0.90
 0400   10.69    0.82    1.72
 0800   10.41    0.72    2.44
 1600    9.94    0.53    2.97
 3200    9.23    0.29    3.26
 6400    8.30    0.07    3.33
12600    7.48    0.18    3.51

7D
       screen  per ISO  Total Gain
  ISO    DR     Gain    from base ISO
 0100   11.12    0.00    0.00
 0200   11.08    0.96    0.96
 0400   10.76    0.68    1.64
 0800   10.10    0.34    1.98
 1600    9.02   -0.08    1.90
 3200    8.26    0.24    2.14
 6400    7.09   -0.17    1.97
12600    6.22    0.13    2.10   

As we can see there is no reason to go over ISO 800 for 7D or ISO 3200 for 5DIII. Because we gain negligible DR while we loose a 1 stop range of the "perfect midtones" (where all pixels are used so no resolution loss no artifacts) to the problematic hi and low ranges where only half the pixels are in use.

IMHO for 5DIII I would not sacrifice the range of 1 stop of full sensor use for 1/3 stop DR increase at ISO 3200 not even for the 1/2 stop gain from ISO 1600.

Lets look at it from another side .. for example the 100-1600 case .. the calculated 3 stop gain is valid if we use all the pixels.
In theory, by using only half of the sensor at darks we should have 0.5stop more noise there than a full 1600 ISO shot, so the net gain should be 2.5 stops.

What we see in the samples is not only the better sensor behavior at ISO1600 but also a kind of simplistic denoise (by interpolation - averaging) and the excellent holistic approach of ML team regarding black point regulation, channel imbalance and FPN elimination.

It will be interesting to see the differences between using 100-800 vs 100-1600 ranges with 5DIII ... especially after using a mild denoise in ACR or Rawtherapee ..
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Tinnunculus on July 17, 2013, 07:40:31 PM
Would it, if possible, increase DR if the 70D dual photo-diodes were used separately with different iso?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jaybirch on July 17, 2013, 08:58:46 PM
This is about as good as i can get it. I was working between ISO 400 and 1600.... Exposed heavily to the right (I tried about 10 exposures, this was best). Some pretty heavy colour moire and stair stepping in the highlights... But the mids and shadows are pretty good.

(http://s15.postimg.org/f4im1z8tj/400_1600_800_exp.jpg) (http://postimg.org/image/f4im1z8tj/)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: NateVolk on July 17, 2013, 09:08:53 PM
This is great alex!  I'm happy to help test with my 5d3, but I'm a mac guy and having some difficulties post processing the files.  Read the pdf, downloaded the zips from page one, but then what?  Sorry for the stupid ?'s...
I searched for a tutorial on using the cr2hdr.exe and the draw.exe but no luck.  Thanks anyone for some direction!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: horshack on July 17, 2013, 10:33:11 PM
@horshack,

I don't know who theSuede is (he seems to know some stuff though), but please ask him to read the PDF. I'm pretty sure he didn't.

In deep shadows I only use data from the higher ISO, therefore his averaging formula does not apply here.

Hi Alex, theSuede is a software developer specializing in high-end color workflow products I believe. He is very knowledgeable on most areas of imaging technology but especially on sensors and color. I'm not sure if he's read through your PDF.

I read through the PDF and I saw the section on deep shadows where you only use the ISO 1600 data, indicating "corresponding data from ISO 100 will be just noise". The ISO 100 lines will have more read noise and tossing them out will improve DR but at the same time will reduce photons collected by half, reducing the shot-noise SNR of those shadows from 22.93:1 to 16.21:1, based on some quick calculations assuming the deep shadows start at around 7EV below ISO 100 saturation. As you descend further into the shadows the read noise represents an increasing percentage of total noise. The cross-over point between read noise and shot noise is about -7EV (ISO 12,800 equiv), so even starting in the deep shadows the shot noise represents 50% of the total noise, so discarding the ISO 100 lines is still significant to the total noise.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sarangiman on July 17, 2013, 10:40:29 PM
Wonderful work, & great white paper!

The funny thing about all this is that this is all a workaround largely b/c of the ADCs being on a separate chip from the image sensor (as surmised by ChipWorks). Essentially: your technique is trying to recover the dynamic range of the CMOS sensor itself, not the sensor & the entire signal processing chain. This'd all be pretty unnecessary on a D600/D800, where there's not much difference between the two (although the higher ISO exposure could help for very very deep shadows that might otherwise suffer from quantization error at ISO 100). The actual CMOS sensors on Canons have a similar DR to that of the best Sony sensors, if we believe the numbers from DXO/Sensorgen.info (though, this doesn't take into account FPN; I'm not sure exactly where in the chain FPN is introduced).

Do you know what the difference between the top & left black bars is? I've seen this & often wondered.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sarangiman on July 17, 2013, 10:45:50 PM
I read through the PDF and I saw the section on deep shadows where you only use the ISO 1600 data, indicating "corresponding data from ISO 100 will be just noise". The ISO 100 lines will have more read noise and tossing them out will improve DR but at the same time will reduce photons collected by half, reducing the shot-noise SNR of those shadows from 22.93:1 to 16.21:1

Hi horshack-- why would you 'reduce photons collected by half'? The exposure at the sensor plane is not changed; there is only one exposure. The ML hack is simply changing what you do with that exposure data, & so should not affect the SNR save for actually increasing it in shadows by reducing the effects of downstream noise (after ISO amplifier) in the ISO 1600 exposure.

The focal plane exposure (in terms of shutter speed/aperture) remains the same for both ISO 100 & ISO 1600 exposures; it's not like two separate exposure are being made (where the ISO 1600 exposure would have 4 stops less exposure which, yes, would increase the effects of shot noise).

Let me know if I'm misunderstanding you.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: platu on July 17, 2013, 11:32:06 PM
See here an additional comparison of Normal raw video "Denoised" vs Dual ISO raw video.  https://vimeo.com/70511941 (https://vimeo.com/70511941)

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 17, 2013, 11:45:34 PM
Whoa, the moire on high contrast vertical edges is pretty bad.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 18, 2013, 12:06:11 AM
Just found a bug in the interpolation code near clipped highlights. Updated converters in first post.

Before (half-res ISO 1600 interpolated):
(http://acoutts.com/a1ex/median-fix-before.jpg)

After (half-res ISO 1600 interpolated):
(http://acoutts.com/a1ex/median-fix-after.jpg)

Please remember that in video mode it's skipping 6 lines at a time, and in photo mode it's skipping only 2 at a time. So my suggestion is to try to post photo samples too; videos are not the strong point for this method.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 18, 2013, 01:23:05 AM
The artifact you fixed looks a lot like the "serrated" edges in the video in the previous post. In light of your note regarding video versus photo use of the feature, will this fix improve the video case nonetheless?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 18, 2013, 01:55:28 AM
The ISO 100 lines will have more read noise and tossing them out will improve DR but at the same time will reduce photons collected by half, reducing the shot-noise SNR of those shadows from 22.93:1 to 16.21:1

Can you explain the relationship between resolution and SNR?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 18, 2013, 02:20:31 AM
I just did a test using fine details (fur and grass) to demonstrate the line-skipping.  This is of course a "worst-case" video, but it should help to show the interlacing.

Dual-ISO: http://youtu.be/w4PxA1KcUFA
Single-ISO: http://youtu.be/yiMdOVfCgJ0
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Marsu42 on July 18, 2013, 02:56:50 AM
As with a lot of recent new features I'm wondering "Why should I care, will this eventually available on the xyz (replace with your camera model)?"

It'd be great if this information would make it in news announcements w/o the need to flip through dozens of pages, because you never know if some features *require* a dual-digic or new camera (like dual iso 7d/5d3) or if these are just the first models that get it.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ShootingStars on July 18, 2013, 03:07:45 AM
I'm sick and tired of these "wish it was on XYZ model" posts.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 18, 2013, 03:12:29 AM
It'd be great if this information would make it in news announcements w/o the need to flip through dozens of pages, because you never know if some features *require* a dual-digic or new camera (like dual iso 7d/5d3) or if these are just the first models that get it.

The information is linked in the pdf in the first post.

Asking people to read is not an overly unreasonable expectation imo.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on July 18, 2013, 03:12:57 AM
Code: [Select]
/* 00:00:02.660613  */ CMOS
/* 00:00:02.660658  */     00 0593 404520BC
/* 00:00:02.660702  */     01 0000 404520BE
/* 00:00:02.660742  */     02 0225 404520C0
/* 00:00:02.660782  */     03 0000 404520C2
/* 00:00:02.660820  */     04 04F0 404520C4
/* 00:00:02.660859  */     05 0433 404520C6
/* 00:00:02.660898  */     06 0000 404520C8
/* 00:00:02.660937  */     07 0000 404520CA
/* 00:00:02.660974  */     08 000A 404520CC

CMOS regs for 6D I hope... maybe this will work for ADTG too.

It did! ADTG shutters for all!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Marsu42 on July 18, 2013, 03:48:11 AM
The information is linked in the pdf in the first post. Asking people to read is not an overly unreasonable expectation imo.

Thanks for the information, given with the usual finesse if I may be so bold to comment. I'd even assist you in your reading efforts: the pdf is linked with the description "in-depth description of how it works" which is exactly not where I'd expect to find the very basic information "will this come to my camera" - that's why I've suggested to provide this in a more obvious way, i.e. neither buried in a pdf nor somewhere in the thread.

For everyone else: No, it's not coming to cameras except 7d/5d3 :-\
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on July 18, 2013, 03:54:08 AM
Quote
No, it's not coming to cameras except 7d/5d3

Lol, don't be so sure.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: driftwood on July 18, 2013, 04:56:12 AM
Dual ISO artefacts with mean23 but much less than with standard current raw2dng.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: squig on July 18, 2013, 05:55:57 AM
Dual ISO artefacts with mean23 but much less than with standard current raw2dng.

Wow!

Did you get a 100/1600 shot?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Ryan Lightbourn on July 18, 2013, 07:07:55 AM
This is great alex!  I'm happy to help test with my 5d3, but I'm a mac guy and having some difficulties post processing the files.  Read the pdf, downloaded the zips from page one, but then what?  Sorry for the stupid ?'s...
I searched for a tutorial on using the cr2hdr.exe and the draw.exe but no luck.  Thanks anyone for some direction!

2nd that...also on Mac.  Got some shots, just can't figure out how to make them look normal in AE.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 18, 2013, 08:54:01 AM
@driftwood,

Can you upload 1 second (or less) of raw video (the RAW file, not the DNGs), where mean23 gives better results than the default median?

In all my tests, median was a lot better on edges (see my bugfix above - that's an interpolated image, 2 lines from the sensor, 2 interpolated). I did not shoot power lines, I don't like them :P
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: squig on July 18, 2013, 09:10:36 AM
I did not shoot power lines, I don't like them :P

Who does?

From what I've seen so far 100/400 has a lot more DR, has Samuel run his DR test?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Iain on July 18, 2013, 12:34:02 PM
I hope this is helpful, sorry if it is not.

I downloaded the CR2 and DNG to have a play with and I found that by denoising the noisy rows in the shadow areas before demosiacing and then demosicing all rows some artefacts are reduced.

This is my version created from the cr2 (via dcraw then G'MIC) and then the DNG version (via UFRAW).

(https://dl.dropboxusercontent.com/u/10782279/ML.gif)


The denoising is a simple median filter on the noisy rows. It is applied to 3 consecutive pixels of the same colour EG R1,R2 and R3 and also on G1,G2 and G3 and so on.

R1 G1 R2 G2 R3 G3

I repeated the filter 3 times.

I used my own demosaicing method. I'm not a programmer, I just play with scripting in G'MIC , but I thought this might be useful.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: araucaria on July 18, 2013, 02:25:05 PM
Lol, don't be so sure.
That would be great!

Anyway, what happened to iso-less? The increased DR is great for the guys tied to canon but the really interesting thing was that iso-less thing.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: hetfanatic on July 18, 2013, 03:31:20 PM
I'd like to test these for 7d but I'm still not sure how to implement / install it :(
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Stedda on July 18, 2013, 03:45:29 PM
You need to compile the code for yourself at this point... there are tutorials here on the site.

or you must wait for either a forum user to host their compiled code of the Devs to release an updated Alpha.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 18, 2013, 03:53:01 PM
Regarding shadow aliasing: which of these 3 pics would you choose, and why?

http://acoutts.com/a1ex/shadow/a.jpg
http://acoutts.com/a1ex/shadow/b.jpg
http://acoutts.com/a1ex/shadow/c.jpg

Just for your reference, to get an idea of the noise levels:
ISO 100 half: http://acoutts.com/a1ex/shadow/iso100.jpg
ISO 1600 half: http://acoutts.com/a1ex/shadow/iso1600.jpg
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Seanc on July 18, 2013, 04:36:03 PM
I prefer 'b' because it has more texture detail than 'c', and tames the chroma noise that's objectionable in 'a'.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: horshack on July 18, 2013, 04:44:12 PM
Hi horshack-- why would you 'reduce photons collected by half'? The exposure at the sensor plane is not changed; there is only one exposure. The ML hack is simply changing what you do with that exposure data, & so should not affect the SNR save for actually increasing it in shadows by reducing the effects of downstream noise (after ISO amplifier) in the ISO 1600 exposure.

The focal plane exposure (in terms of shutter speed/aperture) remains the same for both ISO 100 & ISO 1600 exposures; it's not like two separate exposure are being made (where the ISO 1600 exposure would have 4 stops less exposure which, yes, would increase the effects of shot noise).

Let me know if I'm misunderstanding you.

For the shadows the ISO 100 lines are being discarded from the composite raw, which reduces the light-capturing surface area of the sensor in half.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 18, 2013, 05:02:33 PM
I'd choose c. The first two have too much chroma noise in the shadows. I think texture detail between b and c are comparable.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: driftwood on July 18, 2013, 05:23:08 PM
a1ex: Not at home right now so can't send you original raws. Here's my latest screenshots shoot produced today showing the processing of the same RAW file fed thru each of the methods: mean23 v median6 v shadow100.

https://www.dropbox.com/sh/4rpnyayri6n1jrk/2iMOz1n5Nl/mean2rb3g%20450%20avg%20hot%20pixels%20per%20frame.jpg

https://www.dropbox.com/sh/4rpnyayri6n1jrk/osCxpEq1xz/median6-1500%20avg%20frame%20hot%20pixels.jpg

mean23 averaging 450 hot pixels per frame.

median6 1500 per frame.

Shadow100 averages 1000 hot pixels per frame on the same tested RAW file.

EDIT: UPDATED (now with shadow100) picture showing the major differences of each against each other:
 https://www.dropbox.com/s/2h7x59girnbxdpu/mean23_median6_shadow100.png

As to your A,B,C, question the grain in A to C goes from courser to finer chroma noise. Most people are going to choose the aesthetically more pleasing C, but I'd probably hunch for B. There's a tad more shadow detail in A which could be processed in post.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 18, 2013, 05:39:52 PM
Here's how I've got the 3 pictures:

a) should have no shadow aliasing (shadows were handled as if they were midones). To get similar results, use this raw2dng: http://acoutts.com/a1ex/raw2dng_shadow100.exe

b) is a 50% blend between a) and the high ISO image. Less noise, a little aliasing. http://acoutts.com/a1ex/raw2dng_shadow050.exe

c) is 25% a) and 75% high-iso. Therefore, even less noise, but a little more aliasing. http://acoutts.com/a1ex/raw2dng_shadow025.exe

By default, shadows used data from high ISO only. This gets minimal noise, but it seems to have serious problems with aliasing.

Of course, now the formula from Horshack's friend should be valid for case a).

@MA Visuals: can you repeat your test with these 3 raw2dng's? Just post-process the same video with them.

Latest cr2hdr uses 50% blend (link in first post).

@ajay and @audionut: can you develop your test shots (the same pictures that you posted earlier) with latest cr2hdr?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on July 18, 2013, 05:48:38 PM
For the 3 images and choosing with a relaxed eye, and not scrutinizing/pixel peeping the image,  C is more "pleasing" than the rest, its less noisy.

It might not be the same choice with a different subject matter and composition.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: mannfilm on July 18, 2013, 06:04:34 PM
Read the PDF, maybe I missed this. Regarding 50% vertical rez loss in dark and highlights. So on either end we're getting line doubled 1920 X 540  instead of true 1920 X 1080? And whats the range of the lower rez? If we are getting 14 stops, is the low  rez sections in the bottom 4 stops and upper 4 stops or what? You guys are the new gods of film. 
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 18, 2013, 06:10:27 PM
Figure 8 shows where you get full resolution.

With my latest modifications, you also get almost full resolution in deep shadows, but with more noise (see my previous post).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: driftwood on July 18, 2013, 06:12:12 PM
https://www.dropbox.com/s/2h7x59girnbxdpu/mean23_median6_shadow100.png (https://www.dropbox.com/s/2h7x59girnbxdpu/mean23_median6_shadow100.png)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Toffifee on July 18, 2013, 06:31:02 PM
I'd pick picture C, but I could go B.

C obviously has less noise, it also doesn't seem to soften up the details.
Aliasing seems about the same, A a tiny bit better, but it's really noisy compared to the other two.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 18, 2013, 06:47:04 PM
C because it has the least noise and artifacts.  Although it has more aliasing around the top silver edge of the camera on the table and the table edge, and more color noise in the silver edges (of the camera).  There is more perception of detail in the table in B, but it looks to be the luma noise.

A has by far the least aliasing.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: MA Visuals on July 18, 2013, 06:48:13 PM

@MA Visuals: can you repeat your test with these 3 raw2dng's? Just post-process the same video with them.


Rendering now... will upload the results shortly.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: driftwood on July 18, 2013, 06:49:42 PM
median6 v mean23 v shadow25 v shadow 50 v shadow 100

https://www.dropbox.com/s/noa57vx3h6g7r0y/shadow25_50_100test.png (https://www.dropbox.com/s/noa57vx3h6g7r0y/shadow25_50_100test.png)

I really like shadow 50 together with the less purple artifacting in mean23
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 18, 2013, 06:58:47 PM
Nice, which one do you like from Driftwood's test?

Another tip: in crop video mode, it will skip only 2 lines at a time (instead of 6). Therefore, you can expect a lot less aliasing.

You can also record at higher vertical resolution (1320) and then resize it to something lower. It should hide some artifacts.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on July 18, 2013, 07:06:40 PM
my choice would be either 25 or 50.
50 has less aliasing.
25 has less noise

(https://fbcdn-sphotos-g-a.akamaihd.net/hphotos-ak-ash3/1005186_541996815857388_1702224196_n.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ajay on July 18, 2013, 07:28:21 PM

@ajay and @audionut: can you develop your test shots (the same pictures that you posted earlier) with latest cr2hdr?

A1ex ... I took the two images that I sampled before using the latest cr2hdr and the halo effect is completely gone from the orange. The second image still has moire caused by the feathers, but the false colors are reduced quite a bit. Definitely better with both images.

AJ
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: GregoryOfManhattan on July 18, 2013, 07:40:46 PM
have tried out the dual_iso module with raw2dng at 7351be3d3c0c
sequences processed through resolve.
room was so dark i moved the Base ISO up to 800 and that may not have been enough.
did no camera preparation before this event - so i can't confirm that the comparison clips had significant Ev offsets applied.

dual iso mode seemed to work at all resolutions i tried - here's a 3586x1320 DNG
http://50.56.67.113/ml-17july13/000012.dng
http://50.56.67.113/ml-17july13/1927-3584x1320-000012.jpg - same thing as an RPP processed jpg at 1/2 scale

(http://50.56.67.113/ml-17july13/1927-3584x1320-000012.jpg)

3.5K did not seem to work as well as builds from last week or at least the usual small and memory hacks didn't work as well when i tried them - only got 1-2 seconds of 3.5K.  2.2K seemed continuous.

had one lock up where i needed to pull the battery

which repository has the code with median23 and other options? will have time to test this weekend.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 18, 2013, 07:50:04 PM
To choose other options, just flip the conditional defines from the source code (INTERP_*). For shadow aliasing, adjust f_shadow.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: GregoryOfManhattan on July 18, 2013, 08:20:26 PM
To choose other options, just flip the conditional defines from the source code (INTERP_*). For shadow aliasing, adjust f_shadow.
great we can make our own custom blends.
modules/lv_rec/raw2dng.c
modules/dual_iso/cdr2hdr.c

if a clip is dual_iso will the current raw2dng always display the HDR info and the EV offset?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: MA Visuals on July 18, 2013, 08:44:11 PM
Ok... here's the new comparison using shadow100, shadow50, and shadow25.  I enabled the download link on Vimeo for those wanting a more accurate representation of the footage. 

Shadow 50 has my vote as a set it and forget it default. 

Shadow 100 is too noisy... Shadow 25 has too much aliasing for me to use the footage... Shadow 50 has minor aliasing but much less noise than Shadow 100.  On the other hand... Shadow 100 combined with a light pass of de-noising in post would work as well.

Vimeo link... https://vimeo.com/70574082 (https://vimeo.com/70574082)

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: mauerfuchs on July 18, 2013, 08:52:25 PM
Is the dual iso method similar to the canon video raw strategy?
http://blog.abelcine.com/2012/11/05/what-exactly-is-canon-raw/
I bet some of the C500 developers who are watching ML Raw development remember some of these steps very well.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: driftwood on July 18, 2013, 09:16:32 PM
A look at the artefacts of each of the test interp modes. Same dual ISO DNG opened 'As Shot - 5500 / Tint +17' (RawMagic and v0.13 OSX raw2dng no interp modes and opened with a tint of +66 then changed to +17).

https://www.dropbox.com/s/a3wd7s9sh5ud2sv/interp_artifact_test.png
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: JackDaniel412 on July 18, 2013, 09:20:41 PM

Dual ISO test on 5D3 (ISO 100 & ISO 1600), post-processing using "raw2dng_mean23" and After Effects.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: portmantoad on July 18, 2013, 10:38:07 PM
ok so I have a few thoughts here.

when using this in a video mode (60p as an example), could you alternate which exposure you use each row for each frame? that way you'd have the full resolution midtones output as it is now, as well as what is essentially a iso100 60i output and a iso1600 60i output. I would imagine losing some temporal resolution is probably better looking than losing half of your vertical resolution.

the other idea is, although I have much less hope for this to be feasible, is it required to choose the exposure per row or can it be adjusted on an individual photosite basis? Because if that's the case we could take a page out of fujifilm's X-Trans filter strategy and set up the sampling in a pseudo-random fashion. That would make the aliasing less obvious and could actually be kindof cool and filmic.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 18, 2013, 10:44:36 PM
I can alternate the exposures, but the postprocessing will be a lot more complex. It should help with highlight aliasing (because the shadow one is pretty much solved).

In some videos, the exposure pattern is moving, and in others it's fixed. I don't know yet why this happens.

I don't know how to alternate ISO for every single line - if you look in the paper, all I do is changing a register to low-high or high-low. The ADTG chip does the rest.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: driftwood on July 18, 2013, 10:55:37 PM
a1ex: In the dual ISO raw2dng interpolation, what ways are you looking at improving the accuracy of returned values and reducing artefacts? Ah I see you're already answering this.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sarangiman on July 18, 2013, 11:19:38 PM
For the shadows the ISO 100 lines are being discarded from the composite raw, which reduces the light-capturing surface area of the sensor in half.

Thanks Horshack-- funny how the obvious sometimes escapes me. Although, b/c they're interpolating between the two ISO exposures, you can set up the algorithm to only use the ISO 1600 file where read noise trumps shot noise contributions significantly. But I see what you're saying -- when you do that, you pay a cost due to shot noise contributions (vs. exposing the whole image at ISO 1600, for example).

But typical 'engineering' DR calculations using SNR of 1 as the lowest signal don't consider shot noise, correct? Therefore, one could still say that the engineering DR is extended to FWC/read noise @ISO 1600, which is ~14EV. If you're using a higher SNR cutoff of, say, 20, then the DR is much more modest (considering the increase in shot noise contributions you mention).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: MA Visuals on July 19, 2013, 02:24:50 AM
I updated my original test to use the optimized version of RawToDNG (raw2dng_shadow050.exe).  Aliasing is noticeably reduced now. Thanks Alex.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 19, 2013, 03:05:07 AM
I'm finding it difficult to properly set exposure, though I'm sure this will improve as code revisions progress.  This RAW converted with the Shadow 50 executable is particularly special.

(http://s2.postimg.org/dr71uroix/Untitled.png)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jkdjedi on July 19, 2013, 03:30:04 AM
Sorry if I missed it...but....Is there a noob guide on how to implement this hack on the 7d? My guess is that you'll need the original Magic Lantern firmware and just copy and paste to some bin files..??? I know, I know...we're not worthy..but please, point us in the right direction..Thank You. :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: mucher on July 19, 2013, 03:42:19 AM
I was able to download the full-res dng file, and open it with GIMP 2.8, and didn't find much artifacts, and, when shrinked to 1920x1280, the picture was beautiful, but I couldn't get that much DR as the whitepaper did.

Couldn't wait to see when this good thing, plus raw video, plus 10 bit downbit raw video, will come to 7D. And I had a dream last night shooting that Dual-raw video with 7D. This is serious stuff.

I personally appreciate Alex's great job and his daring thought, which I think quite fun, and the whitepaper is really well written(though a little beyond me). However, I do notice that this setting only works with iso100, if it can be extended to other ISOs will be a genuine plus.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: brapodam on July 19, 2013, 03:42:33 AM
Sorry if I missed it...but....Is there a noob guide on how to implement this hack on the 7d? My guess is that you'll need the original Magic Lantern firmware and just copy and paste to some bin files..??? I know, I know...we're not worthy..but please, point us in the right direction..Thank You. :)
You'll either have to compile yourself, or wait for someone to release the updated autoexec.bin for 7D. You'll need both the dual_iso modules and the autoexec.bin
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 19, 2013, 03:42:39 AM
I'm finding it difficult to properly set exposure,

Set your base ISO for the highlights and the recovery ISO for the shadows.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jkdjedi on July 19, 2013, 03:45:17 AM
Thanks Brapodam.. I guess I'll have to wait then... :(  :'(
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 19, 2013, 08:36:41 AM
@dubzeebass: can you upload a sample? (less than 1 second of video, if possible).

Did it work better with a previous version?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dariSSight on July 19, 2013, 09:38:35 AM
Why can't the 5D Mark II get Dual ISO function, what the difference in 7D that allow it to function?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sebamax on July 19, 2013, 09:46:57 AM
Hi guys,
to work with raw2dng_shadow050.exe on wine in osx,
when the terminal tells me "usage:

Z:\Users\Seba\Desktop\Raw\raw2dng_shadow050.exe file.raw [prefix]

 => will create prefix000000.dng, prefix0000001.dng and so on."

What should I do?

Thanks
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Walter Schulz on July 19, 2013, 09:53:00 AM
Copy your RAW video file to a location you like. For you it will do in Z:\Users\Seba\Desktop\Raw\ and drag and drop this file onto raw2dng_shadow050.exe. After this step you will get a bunch of files in this very folder/directory for further processing.

Ciao, Walter
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: brapodam on July 19, 2013, 09:58:16 AM
Copy your RAW video file to a location you like. For you it will do in Z:\Users\Seba\Desktop\Raw\ and drag and drop this file onto raw2dng_shadow050.exe. After this step you will get a bunch of files in this very folder/directory for further processing.

Ciao, Walter
That works fine in Windows, I'm not sure about whether you can just use it like that if you have to run it using wine.

Normally what I'd do would be to run
Quote
wine ./raw2dng_shadow050.exe file.raw


Of course replace file.raw with whatever the filename is.

If you want a prefix to your dng files then put the prefix after the filename
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Walter Schulz on July 19, 2013, 10:02:14 AM
Thanks for the correction!

Ciao, Walter
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sebamax on July 19, 2013, 10:27:33 AM
Thanks Guys,.....perfect
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 19, 2013, 11:18:10 AM
Can,t wait for macversion converter so installed virtualbox and used Alex raw2dng converter. Some serious iso improvements here. Question. I get this famous pinkish cast when converting, how can I avoid it? Filmed in 3x crop mode but it shows otherwise as well.
Thanks!
//D
(http://s12.postimg.org/65736pk09/image.jpg) (http://postimg.org/image/65736pk09/)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 19, 2013, 11:20:20 AM
Upload a short sample. I don't see it in any of my test files.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: driftwood on July 19, 2013, 11:59:04 AM
a1ex, have you tried sampling more than 1 frame (i.e. two) during the interpolation and if so were there no averaging improvements? I know it would be more taxing but Id be interested to know what your findings were.

How about frame x {1 second in} (settled) - do the math - return to frame 0 / begin?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 19, 2013, 12:02:14 PM
I have to align the frames first, and since they contain motion, it's not trivial.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 19, 2013, 12:19:59 PM
Sent you a pm with a short clip Alex. Sometimes 3x crop mode works fine with no pink cast, others it don,t. Hot pixels numbers skyrocket in the converter when pinkish. Normal filmings seems to work fine.

*update, just tried a regular 1920x1080 clip and pinkish unfortunately. Seems inconsistent on my camera. Or with the conversion.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 19, 2013, 12:29:26 PM
Updated raw2dng.exe in the first post, seems to be OK now.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 19, 2013, 12:33:57 PM
wow, that,s fast. Gonna try it out immedietaly
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 19, 2013, 12:43:14 PM
Did you change the raw2dng.exe? I still get the skyrocket high hot pixel numbers and pinkish cast as before.
Thanks
/D

*update, worked with my latest moviefiles. Will try in 3x mode an also do a comparison. Will post it later tonight
Thanks
//D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 19, 2013, 12:55:40 PM
Works here. Cache issues when downloading?

Uploaded the same thing with another name: http://acoutts.com/a1ex/raw2dng_whitefix.exe
md5: 3af31ca290c8e4cb3e3013cf134a16f6
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 19, 2013, 12:57:30 PM
Nice Alex! Worked perfectly :)
I,ll post a comparison tonight.
Sweet
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: bumkicho on July 19, 2013, 03:50:19 PM
Is raw2dng_whitefix the latest raw2dng that handles dual iso raw files? Will it handle normal raw files as well?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: driftwood on July 19, 2013, 05:28:11 PM
yes.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sebamax on July 19, 2013, 05:56:36 PM
I did a short test
the lighting conditions it was hardly

https://vimeo.com/70633755
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ShootingStars on July 19, 2013, 07:08:50 PM
ISO100 looks the best there. What's the difference between external and internal?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sebamax on July 19, 2013, 07:31:19 PM
ISO 100 is better but the DR is much lower than dual ISO. Using ETTR: external exposure the shutter was 800 (ISO 100-F2.8) and internal exposure the shutter was 50 (ISO 100 F2.8).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 19, 2013, 08:07:15 PM
What shutter speed did you use for dual ISO?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: horshack on July 19, 2013, 08:40:53 PM
But typical 'engineering' DR calculations using SNR of 1 as the lowest signal don't consider shot noise, correct? Therefore, one could still say that the engineering DR is extended to FWC/read noise @ISO 1600, which is ~14EV. If you're using a lower SNR cutoff of, say, 20, then the DR is much more modest (considering the increase in shot noise contributions you mention).

Right, SNR goes down but if one uses a 1:1 SNR (like DxO does) for DR calculations then then SNR reduction doesn't affect the DR number.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 19, 2013, 09:20:25 PM
A little testing with the dual iso feature. Real cool stuff going on. Feel free to comment.

//D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 19, 2013, 10:37:26 PM
ISO 100 is better but the DR is much lower than dual ISO. Using ETTR: external exposure the shutter was 800 (ISO 100-F2.8) and internal exposure the shutter was 50 (ISO 100 F2.8).

I don,t quite get this. Clearly you get the best results here in your movie "iso 100 raw expo internal ettr raw2dng".
Could you explain what your settings were exactly.
Thanks!
//D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 19, 2013, 11:03:34 PM
Pixel peepers: which of these 3 would you choose?

http://acoutts.com/a1ex/bright1.dng
http://acoutts.com/a1ex/bright2.dng
http://acoutts.com/a1ex/bright3.dng

Both are ISO 1600 half interpolated.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: driftwood on July 19, 2013, 11:44:55 PM
bright3 better edge. check the curvature of the lamp. bright1 inconsistent. bright2 good balance.

Yep for me bright3 detail wins (see blind edges too). bright1 slightly better in highlight detail - see blind chord. (right click images into new window to analyse). bright2 somewhere in the middle.

(http://www.driftwoodproductions.co.uk/img/testresult.png)

(http://www.driftwoodproductions.co.uk/img/testresult2.png)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 20, 2013, 12:00:08 AM
I think bright 1 is nice. The blinds seem smooth and the camera is less aliased than the others.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 20, 2013, 03:19:33 AM
For the window, bright 1.  Smoothest edges, least artifacting.

For everything else, bright 2.  Bright 1 is a no show, to much aliasing. 

Bright 3 has more artifacts.  Check the difference between 2 and 3 on the full res samples in the areas I have highlighted below.

(https://dl.dropboxusercontent.com/u/34113196/ML/bright3.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 20, 2013, 03:34:22 AM
Hi,

Thank to Audionut for supplying the binary install for the dual iso module. Uploaded and now sailing to go. Facing a challenge though and need desperate help. The raw2dng that I downloaded cannot be activated since windows 8 sees it as a threat if some sort and is locking it on my PC. Any idea how to deactivate the security to get the program running?

Second, how do I record stills with the dual iso module? Cold someone post the workflow?

Thank you again and to A1ex too.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sebamax on July 20, 2013, 09:54:07 AM
I don,t quite get this. Clearly you get the best results here in your movie "iso 100 raw expo internal ettr raw2dng".
Could you explain what your settings were exactly.
Thanks!
//D
Hi,
I wanted to try an extreme situation of light.
In the first shoot with the raw module and ETTR module, I pointed out of the window and ETTR suggested me ISO 100-shutter 1.000 F2.8. In the second shoot I pointed inside the room: ISO 100 shutter 60- F2.8.
With dual ISO module (ISO 100-1.600) with ETTR I pointed out of the window and it suggested me ISO 100 shutter 800-F2.8. I processed all in ACR using Highlight -100 and shadow +100.
With dual ISO module it was a little bit difficult using viewfinder to focus.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 20, 2013, 11:13:45 AM
Pixel peepers: which of these two?

http://acoutts.com/a1ex/highlight/a.jpg
http://acoutts.com/a1ex/highlight/b.jpg
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sebamax on July 20, 2013, 11:33:01 AM
"a" seems more clear in  the curtains
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 20, 2013, 12:08:55 PM
I agree 'a' seems to be better (crispier).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 20, 2013, 12:09:50 PM
A1ex: How do I take still with the dual_iso functionality?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 20, 2013, 12:23:02 PM
To take stills, press the shutter button, then read the first post. Picture quality should be RAW.

So... you all say it's better to have sharper false detail and aliasing artifacts?

To me, it seems better to have less artifacts, even if the highlights are softer. I've updated the converters with the new algorithm, it should print "mean23-vsmooth3". Can you run some tests?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jaybirch on July 20, 2013, 02:13:05 PM
To me, it seems better to have less artifacts, even if the highlights are softer.

100% Agreed.

Softer images, i can work with. Noticeable artefacts are a no go.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 20, 2013, 02:17:41 PM
My vote for the smoother picture with less artefacts definitely.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Stedda on July 20, 2013, 02:32:04 PM
I vote less artifacts.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 20, 2013, 02:55:20 PM
My first Dual ISO production. Really am proud for having worked it out!

Link: https://www.dropbox.com/s/i51u04c44yb3t9w/first%20%20dyau%20iso%20video.mov
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 20, 2013, 03:19:17 PM
My first Dual ISO production. Really am proud for having worked it out!

Link: https://www.dropbox.com/s/i51u04c44yb3t9w/first%20%20dyau%20iso%20video.mov

A still photo using the dual iso functionality

https://www.dropbox.com/s/botulropqxbsjk9/SANDALS%20DUAL%20ISO%20MAGIC%20LANTERN.tif
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jaybirch on July 20, 2013, 03:41:28 PM
Starting to get close to a usable settings. This was shot ISO 200-800 .... Exposure was pretty centred, to avoid too much highlight aliasing. Click for 1080p

(http://s18.postimg.org/gmtabf4d1/200_800.jpg) (http://postimg.org/image/gmtabf4d1/)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 20, 2013, 03:54:44 PM
Yes, that's the ideal use case. Main subject covered by both ISOs (full detail), highlights and shadows as out-of-focus backgrounds (no aliasing).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jaybirch on July 20, 2013, 04:07:27 PM
yep... I did a rack focus and once the highlights come into focus, it gets a bit messy, but not too bad.

For now, i think protecting the highlights as much as possible is the best way to go, avoiding artefacts.... Then pulling out the deep shadows. Probably not getting quite such a DR boost, but it gives a more usable image.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: GregoryOfManhattan on July 20, 2013, 04:14:07 PM
yes - good to know better use cases may not be the extremes and
to consider the value (thematically and exposure wise) of what's in field of focus.

is raw2dng in unified supposed to be able to process dual iso clips?
tried 5bad65ff8cb8 and it isn't converting a dual shot clip - instead it creates banded DNGs.

easy enough to download a different version.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 20, 2013, 04:19:26 PM
Yes, raw2dng from here is from unified.

Can you upload a short clip?

HDR clips are recognized from the first frame, by computing the ISO difference - so if that frame is dark, it probably won't work.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: GregoryOfManhattan on July 20, 2013, 04:28:46 PM
very underexposed - so good to know and explains why i have clips which when running raw2dng do not show the EV HDR correction info - though this is the first time i'm seeing the banded DNG.

uploading now - if you want to see - try these in a couple of minutes when upload is complete
http://50.56.67.113//ml-20july13/M20-0958.RAW 32.7MB

http://50.56.67.113//ml-20july13/000000.dng 2.7MB
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 20, 2013, 04:32:23 PM
A1ex, for video could you alternate 100<>1600 each frame and then de-interlace the highlights and shadows in order to address the aliasing? It could turn the extreme spatial artifact into a slight temporal one.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 20, 2013, 04:39:47 PM
Yes, feel free to write down the math for deinterlacing. Alternating ISO is trivial.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 20, 2013, 04:57:36 PM
Rather than filling line-skipped highlight/shadow pixels with synthesized values, fill them with values from the previous frame.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 20, 2013, 05:03:40 PM
Continue until you get a precise mathematical description of what you said.

Hint: it's 100x more complex than what I did so far.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 20, 2013, 06:03:52 PM
Gotcha. Just wondering what tricks could be used if ISOs were alternated.

Btw I've noticed in the past that when lifting RAW video shadows to the extreme, the noise "crawls" through the frame. I think it was diagonally down and to the left. It reminds me of how the alias patterns are crawling here. I don't understand why there is a barber pole effect during a locked-down shot. That's what I would expect if ISOs were alternating (without de-interlacing).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 21, 2013, 12:03:30 AM
Another video ... I used the Shadow 050 one again and I think it gives the best results.

Is there any way to implement this in such a way that each sensor records individually to the CF card?  Obviously it'd be double the data rate (ergo half the write speed per channel on the CF bus) but it could be a start?

https://www.youtube.com/watch?v=nGBRDbLeIgk
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 21, 2013, 12:04:29 AM
@dubzeebass: can you upload a sample? (less than 1 second of video, if possible).

Did it work better with a previous version?

a1ex, it appears to be just the frames at the beginning of the video that do that interlaced pink highlight thing.  It's akin to the judder that some people get, myself included, when viewing a rendered RAW file - the first few frames repeat back and forth and then it continues normally.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 21, 2013, 12:32:04 AM
Hi! Need some help what to do with my developed cr2 dual iso file. I throw the file on dcraw.exe and then to cr2hdr.exe which creates somekind of.ppm file. This opens fine in photoshop and it contains the info from the darker iso. The lines are still visible. Am I not supposed to get a combined file of the two iso,s?
Thank you!
//D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 21, 2013, 12:33:22 AM
@dubzeebass: that was a bug solved a while ago, turn on sync beep if you still use an old version.
@Danne: first cr2hdr, then dcraw.

I have some major updates to the interpolation algorithm. It's too late here to post nice comparisons and stuff, but you can do that for me.

Basically, this update gets the noise levels from the original samples in out-of-focus areas and minimizes aliasing in high-detail areas.

http://acoutts.com/a1ex/shadow/000000.jpg

compare with http://acoutts.com/a1ex/shadow/b.jpg (shadow50) for example
or with http://acoutts.com/a1ex/highlight/b.jpg (highlight blurring)
or with http://acoutts.com/a1ex/shadow/median6.jpg (the first algorithm)

There's no denoising; the algorithm is just selecting the cleaner signal source (ISO 1600) in low-detail areas. There's some slight blurring in aliased highlight areas, but it has no impact on noise (it's only used to reduce the artifacts).

Now I think the algorithm is suitable for some really nice shots (photos+videos), not just aliased keyboards in black caves ;)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 21, 2013, 12:43:03 AM
@dubzeebass: that was a bug solved a while ago, turn on sync beep if you still use an old version.

I use the latest build for 5Dmk3 from Lourenco, will turn on Beep and see if that sorts it out.

Now I think the algorithm is suitable for some really nice shots (photos+videos), not just aliased keyboards in black caves ;)

The new one looks way better.  Still that chroma aliasing ... I wonder is it possible to pull the colour from surrounding pixels on anything that's got a jaggie and has an RGB level near clip while other levels are relatively low for that area?  Might not work if something is supposed to be jaggie ... but could help with the chroma aliasing.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 21, 2013, 12:54:25 AM
nice Alex, with the blurring!
Still didn,t get the cr2 to combine. I drop the the file first on cr2hdr wich opens up, than on dcraw, creates somekind of ppm-file. When viewed in 100% the horisontal lines are showing. Should I get a dng?
Thank you for your patience.
//D

*update. Got everything to work eventually. Thanks
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 21, 2013, 01:00:20 AM
a1ex can you link the cr2hdr executable separately to save some bandwidth.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 21, 2013, 01:01:48 AM
Still didn,t get the cr2 to combine. I drop the the file first on cr2hdr wich opens up,

You should get the DNG after dropping the CR2 onto the cr2dng app.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 21, 2013, 01:06:14 AM
Hi audionut. Thanks for helping. cr2dng app? Do you mean cr2hdr? Can,t find any cr2dng
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 21, 2013, 01:29:39 AM
Hi audionut. Thanks for helping. cr2dng app? Do you mean cr2hdr? Can,t find any cr2dng

Yes that's what he meant, available in the original post.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 21, 2013, 01:31:27 AM
Got it to work. Had to move the whole package to the virtual box desktop. rather than using the virtual drive.
Thanks for your patience. THis is gonna be fun :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 21, 2013, 01:35:16 AM
Got it to work. Had to move the whole package to the virtual box desktop. rather than using the virtual drive.
Thanks for your patience. THis is gonna be fun :)

Nice one.. I'm just getting my VM environment up and running again after failing with WINE and WINEBottler to get the EXE to run.  Post some samples when you get em!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 21, 2013, 02:52:31 AM
Hi Alex. Playing around with the smooth3 version. Is that the updated version of the combining tools?? With cr2 it seems to be a real winner. Havn,t done anything special yet but its wonderful to have such clean shadows. Will try to post something tomorrow. I,m away and on a real slow bandwith for a couple of days but I,ll try to post a picture.
Did you by the way update the conversion tools for film yet? I got some vertical pink stripes but probably due to an old version? Is whitefix still the latest raw2dng.exe version?
Thanks man.

//D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: squig on July 21, 2013, 04:20:51 AM

I have some major updates to the interpolation algorithm. It's too late here to post nice comparisons and stuff, but you can do that for me.

Basically, this update gets the noise levels from the original samples in out-of-focus areas and minimizes aliasing in high-detail areas.

http://acoutts.com/a1ex/shadow/000000.jpg

There's some aliasing on the coffee table that isn't in the shadow 50 shot. Can you post up a 100/400 shadow 50 shot for comparison. Now that you have the FPN under control the noise doesn't bother me, with a bit of chroma noise reduction it will look like grain.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 21, 2013, 07:00:03 AM
That shot is just a tricky shot from EOSHD - here's his original rendering: http://www.eoshd.com/uploads/5d3-hdrx.jpg

I can remove the shadow aliasing completely, but right now I'm only cleaning 50% of it, because the noise gets pretty bad. Still lots of room for tweaking (but you need to compile yourself, there are no user parameters yet).

Here's the spot you need to change:

Code: [Select]
            if (b0 < white && d < white)
                f = MAX(f, c * FIXP_ONE / 2);

change to f = MAX(f, c * FIXP_ONE); to remove all the shadow aliasing, and maybe tweak CONTRAST_MAX to fine-tune stuff.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 21, 2013, 09:13:17 AM

I can remove the shadow aliasing completely, but right now I'm only cleaning 50% of it, because the noise gets pretty bad. Still lots of room for tweaking (but you need to compile yourself, there are no user parameters yet).

Here's the spot you need to change:

Code: [Select]
            if (b0 < white && d < white)
                f = MAX(f, c * FIXP_ONE / 2);

change to f = MAX(f, c * FIXP_ONE); to remove all the shadow aliasing, and maybe tweak CONTRAST_MAX to fine-tune stuff.

I think that as soon as FPN is suppressed (and cr2dng does a beautiful job on this) then the optimal mix depends on the workflow. I suppose that with ACR (and it's very effective denoise in use) a user would be better to have a DNG with less aliasing at the cost of more noise and then denoise in ACR.
The same with Rawtherapee which has some nice tools for demosaic, denoise, false color suppression ...
With Resolve with it's primitive demosaic and denoise the needs are different.

User parameters would be a nice option.

 
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: squig on July 21, 2013, 09:53:17 AM
(but you need to compile yourself, there are no user parameters yet).

Hehe, yeah like that's ever gonna happen.   :D

Yeah noise is easily dealt with in ACR without much of a detail penalty, and besides if you want anything to look like film you still have to light. Samuel mentioned that 100/400 still gives you 13.5 stops and only the top and bottom 2 stops have aliasing vs top and bottom 4 with 100/1600. With only a half a stop DR penalty to substantially reduce the aliasing 100/400 looks like the best option so far.

Is shadow 50 100/1600 4+4 stops aliased?

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 21, 2013, 10:08:03 AM

.... Samuel mentioned that 100/400 still gives you 13.5 stops and only the top and bottom 2 stops have aliasing vs top and bottom 4 with 100/1600. With only a half a stop DR penalty to substantially reduce the aliasing 100/400 looks like the best option so far.


What you describe is for 100/800 not 100/400 .. http://www.magiclantern.fm/forum/index.php?topic=7139.msg59971#msg59971
And I also think 100/800 is optimal.

At 100/400 the DR increase is 1.25 less than 100/1600.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 21, 2013, 11:40:59 AM
Hi Audionut could you please help with the latest binary of the revised module code. Do not know how to do it myself. Running the dual ISO module that you updated last.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Digital Corpus on July 21, 2013, 12:08:46 PM
If I got off my arse and compiled a 7D copy from the repo, I'd host it. I have no time for extra projects though and this would fall into that category. Hosting isn't a problem when I've pushed through 1 TB in 3 weeks off my home connection and Verizon hasn't cared...
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on July 21, 2013, 03:24:12 PM
How do you find which bits are iso and which bits are flag bits?

50D:

Code: [Select]
16C - 3200
16C - 1600
1FC - 1000
1B4 - 500 -320
94  - 250
04    100

10 11 0 1 1 00
11 11 1 1 1 00
11 01 1 0 1 00
01 00 1 0 1 00
  1 00

Also FRAME_CMOS_ISO_SIZE

6D reg0 doesn't move... have to check the stubs:

Code: [Select]

iso 100 - 0
800 - 33
1600 -44
3200 -55
6400 - 77
128k - 77
256K - 77


/* 00:00:05.898461  */     03 0000 40450E08
/* 00:00:25.234033  */     03 0033 40450E3E
/* 00:00:55.186093  */     03 0044 40450E50
/* 00:01:23.931307  */     03 0055 40450E62
/* 00:01:39.858203  */     03 0077 40450E74

 11 0011
100 0100
101 0101
111 0111
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ap1hk on July 21, 2013, 07:31:19 PM
Hi,

I took still photo mostly and didn't use ML before as I thought ML was for movie mostly.

I believe this update is amazing and drove me to try ML finally. So far my test did show some improvements in DR but depending on situations sometimes the difference are not very apparent.

I would also like to report some suspected bugs when using the Dual_ISO module for photo.
1. Auto ISO doesn't work well with relative setting of Recovery ISO. It will give very badly under-exposed photo (e.g. I usually set at +4EV but even setting to negative the symptom is the same.)
2. The "alternative frames only" option will fail for continuous shots and sometimes it seems just completely stop taking @ dual_ISO at all after a while.
3. The converted DNG always shows a WB of 5500K and tint=17 when I imported them into Lightroom, independent of the original WB at shot.

Regards,

Peter
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 21, 2013, 09:16:13 PM
1. Try a fixed recovery ISO (not relative).

2. It looks at file number (IMG_1234) and burst is not supported. Once you know that, it's completely predictable.

3. Check if exiftool is working (post a conversion log).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jpaana on July 21, 2013, 09:47:49 PM
Using cr2hdr from today (as of https://bitbucket.org/hudson/magic-lantern/commits/00ea74a66945b9626e0fe53807232af831d62269) compiled on OSX, I get a crash on one particular file:

Input file     : IMGL9830.CR2
Full size      : 5920 x 3950
Active area    : 5796 x 3870
Black borders  : 124 left, 80 top
ISO pattern    : BddB RGGB
Interpolation  : mean23-vsmooth3-contrast

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0xaa5e3408
0x00006967 in hdr_interpolate () at cr2hdr.c:1126
1126                   dark_smooth[x + y*w] = ev2raw[(raw2ev[a] + raw2ev[ b] + raw2ev[ b] + raw2ev[c]) / 4];
(gdb) p a
$1 = -30

Other files I've tried so far convert fine so I put the file up at https://www.dropbox.com/s/ruv13bn6ibkzm08/IMGL9830.CR2
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 21, 2013, 10:19:11 PM
For some reason, dcraw was writing a pixel value of 65506 (come on, for 14-bit data, the range is 0-16383...). This also breaks my calculations (even the robust statistics part). The image is read correctly though.

I've converted the shot by hacking the source code a bit, but before providing a fix, I want to understand exactly what's going on.

http://acoutts.com/a1ex/IMGL9830.DNG
http://acoutts.com/a1ex/IMGL9830.jpg
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jpaana on July 21, 2013, 10:30:23 PM
For some reason, dcraw was writing a pixel value of 65506 (come on, for 14-bit data, the range is 0-16383...). This also breaks my calculations (even the robust statistics part). The image is read correctly though.

I've converted the shot by hacking the source code a bit, but before providing a fix, I want to understand exactly what's going on.

http://acoutts.com/a1ex/IMGL9830.DNG
http://acoutts.com/a1ex/IMGL9830.jpg

Excellent, thank you!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 21, 2013, 10:46:36 PM
/* 00:00:05.898461  */     03 0000 40450E08
/* 00:00:25.234033  */     03 0033 40450E3E
/* 00:00:55.186093  */     03 0044 40450E50
/* 00:01:23.931307  */     03 0055 40450E62
/* 00:01:39.858203  */     03 0077 40450E74

This one looks really good.

Code: [Select]
        ?_CMOS_ISO_START = 0x40450E08;
        ?_CMOS_ISO_COUNT = at least 7; // from ISO 100 to 6400
        ?_CMOS_ISO_SIZE  =  18;

        CMOS_ISO_BITS = 4;
        CMOS_FLAG_BITS = 0;
        CMOS_EXPECTED_FLAG = 0;

If it works, the theory about 8-channel readout is invalidated (the 6D has 4 channels afaik).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on July 22, 2013, 01:34:56 AM
I get error 1 in photo & LV, reg not 0? So 7D + 5d3 = 8 channels and everything else = 4?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ap1hk on July 22, 2013, 03:28:31 AM
1. Try a fixed recovery ISO (not relative).

2. It looks at file number (IMG_1234) and burst is not supported. Once you know that, it's completely predictable.

3. Check if exiftool is working (post a conversion log).

Thanks.
1. Yes, autoISO with fixed recovery ISO or fixed ISO with relative ISO both okay. Just to report the combo I find not working.
2. IC. Thx.
3. I am using Mac OS and I compile cr2hdr myself. I found that the my exiftool is a perl interface to Image::ExifTool. There is no error message from stdout of cr2hdr, e.g.

Input file     : 5D3_6476.CR2
Full size      : 5920 x 3950
Active area    : 5796 x 3870
Black borders  : 124 left, 80 top
ISO pattern    : dBBd RGGB
Interpolation  : mean23-vsmooth3-contrast
ISO difference : 2.97 EV (784)
Black delta    : 2
ISO overlap    : 6.0 EV (approx)
Hot pixels     : 1055692
Output file    : 5D3_6476.DNG
    1 image files updated

For conversion log, do you mean those tmp.txt at conversion?
I will check if exiftool really work later.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 22, 2013, 06:38:35 AM
I get error 1 in photo & LV, reg not 0?

From the dumps, it's register 3, right? (I didn't use the original dump format, but some custom hooks)

Try overriding it manually (just change that memory address so it reads 0x3 instead of 0x33, for example).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 22, 2013, 09:44:00 AM
Some comparison results with cr2 files. No question you get much cleaner shadows with less noise in dual iso. It,s pretty amazing to say the least. Done with "smooth" version. I think I used 100-1600 in dual iso. Can,t remember but I think I even went 100-3200 on another example and got nice results on this particular view.

(http://s7.postimg.org/hk7ssvzwr/IMG_5713dngorig.jpg)
dng original(dual iso)

(http://s18.postimg.org/ypkzg6dl5/IMG_5713dng.jpg)
dng dual iso processed in lightroom

(http://s17.postimg.org/82m70avr3/IMG_5711.jpg)
regular cr2 processed in lightroom (not dual iso)

(http://s21.postimg.org/kix56r993/IMG_5713_2.jpg)
dng pressed shadow (dual iso)

(http://s13.postimg.org/68x56wxxj/IMG_5711_3.jpg)
cr2 regular pressed shadow
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 22, 2013, 11:01:07 AM
Here,s an example where I went between I believe 100-3200 or even further. I,ll post a exif from the conversion app which show the ev settings. It,s like "turning on the lights"

(http://s18.postimg.org/5694vcqq1/IMG_5716_2.jpg)
dng original(dual iso)

(http://s8.postimg.org/4xbiiam1x/IMG_5716.jpg)
dng processed in lightroom(dual iso)

(http://s23.postimg.org/hx5z8h80r/IMG_5716_4.jpg)
Crop 1, not very sharp due to handshake but clean!(dual iso)

(http://s15.postimg.org/uofag6dkb/IMG_5716_3.jpg)
Crop 2(dual iso)


(http://s21.postimg.org/6kgioe0t3/Ska_rmavbild_2013_07_22_kl_01_51_47.png)
ev settings
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Mike Bluestone on July 22, 2013, 11:35:16 AM
Dual ISO works well, great new function. Problem I'm having, which may be normal, is that when I activate Dual ISO the display is really "wavy", almost like the scan lines from out of sync artificial lights, but worse.
Would like to know if this is to be expected or is there a problem with my set up specifically?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 22, 2013, 11:39:05 AM
Read the first post. All info there
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 22, 2013, 01:59:17 PM
ISO 100/800
(https://dl.dropboxusercontent.com/u/34113196/ML/D46A8690.jpg)



Base ISO 100 shot with same post processing.
(https://dl.dropboxusercontent.com/u/34113196/ML/D46A8686.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 22, 2013, 02:04:49 PM
@Danne: try processing the same shots with the latest cr2hdr. You should get much cleaner shadows (so I shouldn't have to read the text to know which is which ;) )

In particular, the black spots should no longer be there.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 22, 2013, 02:25:49 PM
Hi Audionut could you please load the binary for the latest dual USO module. I am currently using the one you created last.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 22, 2013, 02:31:08 PM
a1ex maintains an updated cr2hdr binary in the first post. (http://www.magiclantern.fm/forum/index.php?topic=7139.msg59687#msg59687)

There doesn't appear to be any updates to the module since my last build.  Only the cr2hdr binary.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 22, 2013, 02:41:51 PM
Two examples with crops in extreme dual iso, 100-6400. As you can see it works fairly well and it blows the native cr2 miles, miles away. Even tried a 100-12800 iso but had to much magenta issues in certain mid-whites. Settings processed the same in lightroom for both the crg and dual iso dng. Had to drag the exposure slider all the way to the right with the dual iso-file. Probably could have gone a little further but lightroom stopped me from trying ;).

(http://s15.postimg.org/6uyu70fu3/IMG_5760_64k_3.jpg)
dng 100-6400 original


(http://s12.postimg.org/ljv8h7xtp/IMG_5760_64k.jpg)
dng 100-6400 processed

(http://s9.postimg.org/7z0ytnp5r/IMG_5760_64k_2.jpg)
dng 100-6400 cropped

(http://s23.postimg.org/g1nu4jx3v/IMG_5763_3.jpg)
cr2 iso 100 original

(http://s18.postimg.org/p6fzbj4g9/IMG_5763.jpg)
cr2 iso 100 processed

(http://s18.postimg.org/hy8xzzh3d/IMG_5763_2.jpg)
cr2 iso 100 crop

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 22, 2013, 02:47:04 PM
@Alex. Appears to be a new hdr converter from Alex. I will reconvert the pics in time. I,m on the countryside and slooow bandwidth. Today uploads took me two hour to complete.
Amazing work Alex.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Stedda on July 22, 2013, 03:17:51 PM
Had to drag the exposure slider all the way to the right with the dual iso-file. Probably could have gone a little further but lightroom stopped me from trying ;).

Using the brush tool you can apply layered effects that would exceed 100%. You may already know this though.  :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: pipino on July 22, 2013, 04:18:50 PM
It seems a great idea!

which FW I've to load for double_iso on my 5DMK3?

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: JackDaniel412 on July 22, 2013, 05:39:56 PM
Highlight problem with last raw2dng.

(http://dcmakecreative.altervista.org/ML/DNG000000.jpg)

DNG file

http://dcmakecreative.altervista.org/ML/DNG000000.dng (http://dcmakecreative.altervista.org/ML/DNG000000.dng)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 22, 2013, 05:41:34 PM
Yes, feel free to write down the math for deinterlacing. Alternating ISO is trivial.

I now realize that alternating ISOs would likely affect midtones and cause them to shimmer as well if not de-interlaced. I suppose that simplifies the problem because the whole frame would need to be de-interlaced. So without having an alternating ISO video to play with, I tried an experiment in Premiere to see what it looks like to temporally smear video one frame. I did it by copying a set of clips, layering them on top of the originals, shift by one frame, and set opacity to 50%. The goal was for the output be the average of every two input frames.

The result looks good. Although there are temporal artifacts reminiscent of ML HDR video, static areas are crisp and actually cleaner because noise gets averaged down.

So the question is, couldn't such averaging of alternating dual ISO frames, whether performed by RAW2DNG or manually in an editor, treat the aliasing issue?

I'd be happy to experiment manually if I could generate some alternating ISO samples.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 22, 2013, 05:47:24 PM
@JackDaniel412: record 0.5 seconds of video showing the problem, and upload the RAW file. The DNG is processed and doesn't help.

@stevefal: this patch should alternate ISO exactly the way you suggested: http://acoutts.com/a1ex/alt_iso.patch

You can play with it, see if you get improvements, and I'll probably add it as an option in menu.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Erik Krause on July 22, 2013, 07:22:15 PM
Two examples with crops in extreme dual iso, 100-6400.

Very impressive! Could you provide the 100-6400 iso dng for download. I'd like to try for myself. I need to decide whether I buy a 5D3...
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 22, 2013, 08:16:04 PM
I,ll give it ago later. Will try to find a faster upload spot :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 22, 2013, 09:25:43 PM
this patch should alternate ISO exactly the way you suggested: http://acoutts.com/a1ex/alt_iso.patch

You can play with it, see if you get improvements, and I'll probably add it as an option in menu.

I'm not set up to build - could never get my Windows setup to work. Also, I'm still stumped by the crawling aliasing artifacts in dual-ISO video. It looks like they are on a three frame cycle. Do these have to crawl? Even if the artifacts were flipping due to alternating ISO, my two-frame blending wouldn't work as long as the beats are crawling that way in the first place.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 22, 2013, 09:34:48 PM
I'm not yet sure when crawling occurs; it appeared in one test shot from Luke Neumann, but it didn't appear in all of my other test shots...

Maybe it has to do with FPS, PAL/NTSC or something like that.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 22, 2013, 09:55:34 PM
I'm not sure if we're talking about the same thing. I haven't seen one dual-ISO video where the alias beats aren't moving - self-crawling moire.

Doesn't it have something to do with the BBdd, dBBd, ddBB cycle I see when converting? Both are three frame cycles.






Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 22, 2013, 10:06:30 PM
Yes, you see it in the bbDD pattern. For me, only that one clip from Luke Neumann is crawling; everything recorded by me, and the Batman clip (also from Luke Neumann) are stationary. The clip from EOSHD is also stationary.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 22, 2013, 10:17:11 PM
Ok, I see a few that don't crawl. But everything I shoot crawls like crazy, just like the one above.

I'm shooting 1920x818, 23.976, C1, no FPS override, non-zoom.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on July 22, 2013, 10:24:25 PM
If I got off my arse and compiled a 7D copy from the repo, I'd host it. I have no time for extra projects though and this would fall into that category. Hosting isn't a problem when I've pushed through 1 TB in 3 weeks off my home connection and Verizon hasn't cared...

Please, if you can make a compile version for the 7D. Want to trie   :-\ 


Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on July 22, 2013, 10:38:10 PM
Another with crawl. Switch to 1080p and look at the bottom of the fence towards the end:

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 22, 2013, 11:33:28 PM
Small converter update:

- fix crashes in cr2hdr with some unusual pics
- fix red "zebras"
- nicer highlight handling (aliasing artifacts should be blurred)
- minor tweaks
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 23, 2013, 01:53:25 AM
- nicer highlight handling (aliasing artifacts should be blurred)

Old vs New.

(https://dl.dropboxusercontent.com/u/34113196/ML/old.jpg)(https://dl.dropboxusercontent.com/u/34113196/ML/new.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 23, 2013, 03:42:13 AM
A1ex - I understand that the dual functionality is in development stages and hence most of the stuff will be things only developers can play with or comprehend. I am not a techie but try to keep up.

Have the raw2dng figured out thanks to Mixer2 and 1% from their contributions on the EOS M forum. Mixer2 has us really spoilt with his video casts and detailed explainations and steps.

Have the dual iso working on my 5D Mark III. Loving it.

Struggling to understand a major step. If I want to record in 100 /800 iso which one should I set on camera and which on the ML interface?

I first tried 100 on the camera and 800 on ML. But get better results if I do it the other way around. Is that the way this is supposed to work? Doing it the 800/100 way gives me better histogram accuracy and highlight /shadow feedback while recording.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 23, 2013, 04:29:18 AM
Another with crawl. Switch to 1080p and look at the bottom of the fence towards the end:


Hahaha, that's my video ... and my wife ... and my backyard! #winning
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 23, 2013, 04:30:55 AM
Small converter update:

- fix crashes in cr2hdr with some unusual pics
- fix red "zebras"
- nicer highlight handling (aliasing artifacts should be blurred)
- minor tweaks

Is the EXE posted in the OP kept up to date?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 23, 2013, 06:28:46 AM
Quote
I first tried 100 on the camera and 800 on ML. But get better results if I do it the other way around. Is that the way this is supposed to work? Doing it the 800/100 way gives me better histogram accuracy and highlight /shadow feedback while recording.

Is there any difference in the end result? Can you post a comparison?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: enliten on July 23, 2013, 09:46:17 AM
a1ex and stevefal... Are you guys talking about one frame, having line 1,3,5,7... at iso 100 and 2,4,6,8... at iso 1600, then on the next frame switching them?

if this is the case, couldn't you shoot at 60fps and and in post production get a 30 fps average of 2 frames, eliminating the moire?

let me know if i'm way off the mark here.

Also has anybody compiled the 7D binary yet?

-Ben
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 23, 2013, 12:43:02 PM
Is there any difference in the end result? Can you post a comparison?

To my eye there was a difference - both in output and the way the camera reacted. Unless, by the time I swapped the ISO allocation, I had become better at playing around with this stuff and therefore began to imagine there was a difference. Will post the comparison tonight / tomorrow since I will have to look for a suitably contrastry situation to drive home the point.

By the way, I pointed the camara directly at the sun today and was able to get some awesome DR using the dual iso interface. The camera without the dual iso module activate threw some lovely blue clouds but a very shadowy foreground. 

Will post the results tonight.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 23, 2013, 12:53:35 PM
Converted with the latest converter from Alex. Better sharpening algorithm which works best for most dualiso stuff. When converting the 100-6400 I get some artefacts pronunciation with the latest converter. Probably from the image itself.THere are som white minimal dots from the roompicture. You can look for yourself. Probably gonna use the older converter for bigger iso intervals and for most other shots using the new one.
Crap, I,m n the woods in the middle of the sun. Can,t get the pics right :). One last try

(http://s2.postimg.org/vcfws3t7d/IMG_5716.jpg)
original

(http://s2.postimg.org/xv1lssexl/IMG_5716new.jpg)
new converter

(http://s2.postimg.org/kgoiwr89l/IMG_5716old.jpg)
old converter

(http://s2.postimg.org/46yctuxll/IMG_5760b_3.jpg)

(http://s2.postimg.org/dc6pnq109/IMG_5760b_new.jpg)
new converter

(http://s2.postimg.org/6pk1ujjbt/IMG_5760b_old.jpg)
old converter

(http://s2.postimg.org/ig2kpo1i1/IMG_5760new.jpg)
new converter

(http://s2.postimg.org/62pqirbtl/IMG_5760old.jpg)
old converter

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 23, 2013, 02:13:06 PM
The old converter from here was different from the one used for the previous set (http://www.magiclantern.fm/forum/index.php?topic=7139.msg60936#msg60936) (the shadow artifacts are no longer there). Was it median6 (the very first version)?

I don't see any quality loss with the new converter. If anything, the sharpness was set too high when postprocessing the picture.

I'd also try these DNGs with this script (http://www.magiclantern.fm/forum/index.php?topic=7022). Would be nice if you could upload one, maybe when you come back.

Can you do now a comparison between ISO 100 and ISO 100/1600?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 23, 2013, 02:38:48 PM
Hi Alex! I see what I can do. Used the latest converter from the zipfile in the first post. Soon getting back to civilisation to do some more comparison. Gettin on it asap, probably tomorrow
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 23, 2013, 04:09:17 PM
This update should be pretty close to the ideal auto ISO option - for my needs.

https://bitbucket.org/hudson/magic-lantern/commits/373d8b5b0e69

You expose for highlights, it takes care of the shadows. It's pretty much impossible to do the other way, because you can't figure out how much a picture is overexposed if you look at the histogram.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 23, 2013, 08:48:28 PM
New conversion tool from Alex works very good. Check out the difference and also the possibilities that the dualiso module gives.
Happy peeping!

Feel free to download the dual-iso cr2 rawfiles here. Both of them in 100-1600iso
https://docs.google.com/file/d/0B4tCJMlOYfirVWtwbDNpSXBMM2c/edit?usp=sharing

(http://s23.postimg.org/4ms54criz/IMG_5714_iso800full.jpg)
full res

(http://s23.postimg.org/twmwrfyaj/IMG_5715_iso1600old.jpg)
old converter

(http://s23.postimg.org/wbesbvejf/IMG_5715_iso1600new.jpg)
new converter

(http://s22.postimg.org/cvzqpigdt/IMG_5760_iso6400full.jpg)
full res

(http://s22.postimg.org/z97hcbhbl/IMG_5762_1600old.jpg)
old converter

(http://s22.postimg.org/tmb4euesx/IMG_5762_iso1600new.jpg)
new converter
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: daancalo2013 on July 23, 2013, 09:07:16 PM
hello I have a 5d mark 3 and want to know how do I install dual iso, video tutorial please.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Braam544 on July 23, 2013, 10:34:09 PM
hello I have a 5d mark 3 and want to know how do I install dual iso, video tutorial please.


- It is a good idea, help us. Por favor chicos ayudarnos ?

(http://tomhanks.chez.com/film16.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: nickazg on July 24, 2013, 01:10:46 AM
If you are incapable of searching google on how to install ML, you clearly aren't ready for it.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: brapodam on July 24, 2013, 01:54:10 AM
hello I have a 5d mark 3 and want to know how do I install dual iso, video tutorial please.
I'm not going to make a video tutorial because I don't feel it's needed, plus I don't have a 5D3.

Anyways (5D3 users please correct me if I'm wrong, I don't have a 5D3).
1. Follow the instructions in this thread to install ML http://www.magiclantern.fm/forum/index.php?topic=2602.0
2. After you're done doing all those stuff, download everything here: https://www.dropbox.com/sh/xiksvwbehzmycq4/QbuV17WuE5 (I just linked this from the first post in this thread)
3. Replace autoexec.bin on your SD card with the one you downloaded in step 2
4. Create a folder under the ML folder on the SD card, and call it "modules" (without the quotation marks)
5. Copy everything else you downloaded from step 2 into that folder
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 24, 2013, 02:05:05 AM
There are some video tutorials here too http://www.magiclantern.fm/forum/index.php?topic=6099.0

I've updated the dropbox folder with the latest build.

@a1ex, with auto shadow recovery enabled in photo LV, ML menus become laggy.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 24, 2013, 02:41:51 AM
It is looking WAY BETTER a1ex!  I am really impressed with the quality... however the vertical line problem is back.  I've edited the picture below for clarity and if you look closely at the video in 1080p you'll see the lines in the upper-right corner.  Image full-size is here: http://postimg.org/image/j93dgsinh/

(http://s22.postimg.org/j93dgsink/Converted000105.jpg)

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 24, 2013, 02:57:34 AM
I am just BLOWN AWAY by the extra Dynamic Range for still photography.  Wow!

(http://s22.postimg.org/aavomq7ap/MG_3448.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 24, 2013, 03:10:54 AM
@a1ex, with auto shadow recovery enabled in photo LV, ML menus become laggy.

Audionut I just downloaded the builds you posted and I can't find the auto shadow recovery under the Dual ISO menu.  Any idea?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 24, 2013, 03:21:52 AM
It only works in photo LV.  In dual ISO, hit set on recovery ISO and scroll all the way to bottom.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 24, 2013, 04:58:19 AM
One more ... this time comparing 100 to 6400 and then combined.  This was purposefully a very taxing image to capture with two light sources, diagonal lines, and very dark and very light parts of the image.

(http://s24.postimg.org/p0icqwj1f/MG_3455_2.jpg)

It can't be stressed enough that this development is groundbreaking and frankly I think a1ex for prez!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Steve Kahn on July 24, 2013, 08:08:01 AM
So is there a difference in dual iso images for still photography vs DNGs?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 24, 2013, 08:14:40 AM
@dubzeebass: for banding, you started the video with a very underexposed image. Banding correction is computed from the first frame only, so if you start from the sky, it should be a lot better.

You may try something really extreme now... maybe a photo from a cave outside? or maybe something like this (http://www.youtube.com/watch?v=fjtGenTSfX8) or this (http://vimeo.com/36863089)?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 24, 2013, 03:14:41 PM
you started the video with a very underexposed image

What's the best method to ensure both ISOs are picking up the maximum signal without clipping?  Histogram I guess?  I was using the spot meter (%) but I am thinking that may not be good enough.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 24, 2013, 04:00:20 PM
Set your base ISO for highlights (Zebras/Histogram/AutoETTR), and enable the needed recovery (Histogram/Zebras).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 24, 2013, 06:00:43 PM
Set your base ISO for highlights (Zebras/Histogram/AutoETTR), and enable the needed recovery (Histogram/Zebras).

Cheers.  I notice 3 colours in the zebras: white, green, and red.  Red is obv overexposed, white is under ... what does green signify?  Back on topic after I promise!

#a1exforpres
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jaybirch on July 24, 2013, 10:18:20 PM
The still images are starting to looking really nice... a bit of edge fringing, but usable

http://postimg.org/image/5s5d2fkx5/full/
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 24, 2013, 10:38:18 PM
did you try to defringe chromatic abberations in lightroom or another application? I got rid of most of it. It,s a marvellous thing, dual-iso
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jaybirch on July 24, 2013, 11:26:18 PM
Yea, that did a very good job of clearing it up. Resized down to 4K (and 2K), it looks pretty much perfect.

http://postimg.org/image/ot17d1ytd/full/
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: fcpk on July 25, 2013, 12:00:01 AM
would anyone have an archive of the cr2hdr?
The one on first page isn't working with the test pictures I took yesterday. It complains it's not an interlaced file.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 25, 2013, 12:02:57 AM
Upload a sample.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 25, 2013, 12:20:30 AM
would anyone have an archive of the cr2hdr?
The one on first page isn't working with the test pictures I took yesterday. It complains it's not an interlaced file.

The "not interlaced" is related to cr2 files that,s not photographed with dual iso I believe. Try a dual-iso file, works splendid
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: fcpk on July 25, 2013, 12:52:08 AM
I'm pretty sure it is a dual iso, there's clear interlacing in the raw when I preview it, with two different ISO visible
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: AnotherDave on July 25, 2013, 01:26:38 AM
The dynamic range looks good, but the samples I have seen look pixelated and I wonder what-then is the point of using it at all...
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 25, 2013, 01:53:49 AM
The dynamic range looks good, but the samples I have seen look pixelated and I wonder what-then is the point of using it at all...

It's a work in progress ... and has already improved by leaps and bounds.  Oh ye of little faith.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 25, 2013, 02:30:17 AM
Cheers.  I notice 3 colours in the zebras: white, green, and red.  Red is obv overexposed, white is under ... what does green signify?

Dotted white is underexposed.  Each other color represents over exposure in that channel.  Black represents over exposure in all channels.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 25, 2013, 03:00:39 AM
I've uploaded a 7D build.

See here: http://www.magiclantern.fm/forum/index.php?topic=7139.msg61920#msg61920
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: premini on July 25, 2013, 03:25:14 AM
Hi guys, ive installed the dual iso distro on 7/19 and, for some clips i am getting this error in the raw2dng conversion:: "Bright/dark detection error"

    if (is_bright[0] + is_bright[1] + is_bright[2] + is_bright[3] != 2)
    {
        printf("Bright/dark detection error\n");
        return 0;
    }

Looking at the code it seems that the algorithm finds clipped brights in both ISOs and returns with an error, but im just guessing.
Is there something i can do to save those videos since raw2dng is leaving the frames "interlaced"?
Thanks!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: feureau on July 25, 2013, 04:50:22 AM
I've uploaded a 7D build here (https://www.dropbox.com/sh/yp6dl9i104i1j2g/9Ta_9074YP).

I have no idea if it works, or if it will make your camera explode.  I merely built the source.
I don't even know which modules will work.  Obviously Dual_ISO should.

No warranty is implied or otherwise.  If your house burns down, you're on your own.

No warranty noted.

How do I install this? The latest 7D Alpha 2 has no modules.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 25, 2013, 05:04:13 AM
No warranty noted.

How do I install this? The latest 7D Alpha 2 has no modules.

Create a module folder inside the ML folder.  Copy everything except the autoexec.bin file to the module folder.

There doesn't appear to be a devkit released for the 7D.  I'm pretty sure you might need this first, and that's out of my hands. See here:  http://www.magiclantern.fm/forum/index.php?topic=3974.msg61662#msg61662
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on July 25, 2013, 05:14:34 AM
I guess that I am one of those "luna"s.  I load Audionut's autoexec.bin with Alpha 2 into 7D, nothing happens. I mean really nothing, the ML didn't load into camera. I download everything in Audionut's 7D build directory and put them all in CF card's root directory. And start up the camera again. Nothing happens, again. I don't visually see that my camera is damaged neither.

@a1ex
@1%
@g3gg0

It's not possible to do nothing for we trie the Dual ISO on the 7D ?



Dual ISO - massive dynamic range improvement for 5D3 and 7D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 25, 2013, 06:25:05 AM
I am having an awesome time doing after dark street photography with the new dual iso stuff.

Struggling a bit with video though since the files are heavy and rendering takes a lot of time. I have developed a sort of work around to get a better preview of the files before these are sent to raw2dng.

I bascially, first secure a preview on rawnaizar.exe. This tells me the files I want to work with first. The ones that I mark out, I copy to my harddisk and breakout in raw2dng_whitefix.exe. Then the files are directly sent to AE CS6 to render.

A few photo samples shot yesterday and processed exclusively on ACR 8.1:

http://www.flickr.com/photos/[email protected]/9360108468/

http://www.flickr.com/photos/[email protected]/9360245550/

http://www.flickr.com/photos/[email protected]/9357757481/

http://www.flickr.com/photos/[email protected]/9360352442/

http://www.flickr.com/photos/[email protected]/9357878171/

Must have shot the Mapletree signage a million time since it is on my way back home from office. Nothing special ever came out of it till I met Mr. Dual Iso. Rest is history as they say.

I am really proud of the red cup picture. This was a transparent discarded plastic glass lying on a bed of fallen leaves. There was not much reflected light. Mr. Dual Iso came to my rescure. I then exposed the picture in ACR 8.1 and coloured the glass red so that it could standout. ACR 8.1 is now capable of much more with Mr. Dual Iso coming to its aid. They should now call this ACR 801.1 or something since the new functionality is a game changer. Thanks to all developers and a special bow of respect to A1ex.

Tell me what you folks think.

   
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 25, 2013, 07:39:50 AM
Could someone please teach me how to upload the picture directly on the magiclantern forum rather than pasting a link.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Walter Schulz on July 25, 2013, 07:44:03 AM
Code: [Select]
[img]http://farm6.staticflickr.com/5510/9360108468_2404c0c2e8_h.jpg[/img]
Ciao, Walter
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 25, 2013, 07:55:41 AM
You will need to link a thumbnail so that the embeded image size is less then 900 pixels as per the forum rules (http://www.magiclantern.fm/forum/index.php?topic=934.msg1141#msg1141).

Any further discussion about image linking should go in this thread (http://www.magiclantern.fm/forum/index.php?topic=6061.0) please.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: fcpk on July 25, 2013, 10:39:44 AM
sent in pm:)

Upload a sample.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 25, 2013, 10:47:42 AM
You will need to link a thumbnail so that the embeded image size is less then 900 pixels as per the forum rules (http://www.magiclantern.fm/forum/index.php?topic=934.msg1141#msg1141).

Any further discussion about image linking should go in this thread (http://www.magiclantern.fm/forum/index.php?topic=6061.0) please.

Thanks everyone. Audionet noted. 
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 25, 2013, 11:00:18 AM
This shot was taken keeping the bright sun in the frame. Post processed in ACR 8.1.

(http://www.flickr.com/photos/[email protected]/9359793024/)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 25, 2013, 11:15:32 AM
(https://www.dropbox.com/s/qe12wdg58plz2qv/saisha%20balcony%20west%20bay.mp4?m)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 25, 2013, 12:07:12 PM
@Alex. Trying to understand dual iso and movie mode. WHen taking pictures I get noisefree cr2 files however when filming with the exact same settings I can,t get the same results after conversion. Am I doing something wrong? Providing a link to the movie clip and some comparisons.
Settings on camera was manual mode shutter 1/50 dual iso 200-6400. Magic lantern 22nd juli build

Link movie 0,5s
https://docs.google.com/file/d/0B4tCJMlOYfirQmtHenZNbTNkVU0/edit?usp=sharing

example still
(http://s10.postimg.org/ah2kydy55/IMG_5849.jpg)

(http://s10.postimg.org/i5zfa3ymx/test3still.png)

example movie
(http://s8.postimg.org/64wxusz5h/000000.jpg)

(http://s8.postimg.org/xqzpfhiid/test3movie.png)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 25, 2013, 02:54:50 PM
@Alex. Found out what was going on sort of. By reversing the dual iso from 100-6400 to 6400-100 the conversion tool "gets it right" in moviemode. I believe it does some kind of inverted conversion right now. A bug? Tested a short clip with magic lantern from 16th juli, same problem.

example 200-6400 (regular)
(http://s8.postimg.org/rrshpavhx/000000.jpg)

example 6400-200 (reversed)
(http://s22.postimg.org/ejc8si169/000000.jpg)


Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jaybirch on July 25, 2013, 04:03:33 PM
By reversing, do you mean you set the camera's ISO to 6400 and the ML ISO to 200?

Or something in the raw2dng settings?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 25, 2013, 04:09:03 PM
I set the camera to the higher iso and then proceed to the dual-iso settings. There I see to it that the higher iso is to the left and the lower to the right. One way or the other, If the other one does,nt work try reversed :)

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dadinio13 on July 25, 2013, 05:25:38 PM
raw2dng.exe stop extracting frames at frame 493 with all raw files i had tested.
anyones noticed that?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 25, 2013, 05:34:10 PM
A1ex - I understand that the dual functionality is in development stages and hence most of the stuff will be things only developers can play with or comprehend. I am not a techie but try to keep up.

Have the raw2dng figured out thanks to Mixer2 and 1% from their contributions on the EOS M forum. Mixer2 has us really spoilt with his video casts and detailed explainations and steps.

Have the dual iso working on my 5D Mark III. Loving it.

Struggling to understand a major step. If I want to record in 100 /800 iso which one should I set on camera and which on the ML interface?

I first tried 100 on the camera and 800 on ML. But get better results if I do it the other way around. Is that the way this is supposed to work? Doing it the 800/100 way gives me better histogram accuracy and highlight /shadow feedback while recording.

I agree with Danne. Precisely my observation. Pointed this to A1ex. He asked for samples but could not produce any since I was travelling.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sebamax on July 25, 2013, 06:19:14 PM
I try 3200/100 ISO with zebra raw on (raw2dg_whitefix)
I noticed:

24p    http://www.youtube.com/watch?v=p2Q24us23w8

25p    http://www.youtube.com/watch?v=_JG2RwWjTG4

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 25, 2013, 06:25:42 PM
What did you notice? Why didn,t you film in 1920x1080p?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sebamax on July 25, 2013, 06:30:16 PM
Hi,
in 24p more flicker
in 25p no flicker
I filmed in 1920X1080
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 25, 2013, 07:08:01 PM
Ok, I see. Interesting.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: kinematicdigit on July 25, 2013, 08:48:17 PM
How did you compile this? I'm using gcc and getting all sorts of errors?

Thanks.
1. Yes, autoISO with fixed recovery ISO or fixed ISO with relative ISO both okay. Just to report the combo I find not working.
2. IC. Thx.
3. I am using Mac OS and I compile cr2hdr myself. I found that the my exiftool is a perl interface to Image::ExifTool. There is no error message from stdout of cr2hdr, e.g.

Input file     : 5D3_6476.CR2
Full size      : 5920 x 3950
Active area    : 5796 x 3870
Black borders  : 124 left, 80 top
ISO pattern    : dBBd RGGB
Interpolation  : mean23-vsmooth3-contrast
ISO difference : 2.97 EV (784)
Black delta    : 2
ISO overlap    : 6.0 EV (approx)
Hot pixels     : 1055692
Output file    : 5D3_6476.DNG
    1 image files updated

For conversion log, do you mean those tmp.txt at conversion?
I will check if exiftool really work later.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 26, 2013, 12:50:59 AM
One more comparison. This time between raw and dual-iso in movie-mode. two extracted dng,s from raw and dual-iso raw-moviefiles. Even if dual-iso benefits most from high contrast scenes there is still one or two steps better dynamic range with dual-iso even in more evenly lit situations. Pretty amazing since regular raw is already amazingly good.

(http://s16.postimg.org/arjkc4jd1/1600.jpg)
Original raw 1600 iso

(http://s16.postimg.org/9drxgtk3p/6400_1600.jpg)
Original dual-iso 6400-1600 iso

(http://s16.postimg.org/do6lcep6t/1600processed.jpg)
Original raw processed in lightroom

(http://s16.postimg.org/5eupxzvgl/6400_1600processed.jpg)
Original dual-iso processed in lightroom

(http://s16.postimg.org/87nxi0vt1/isoraw.jpg)
Cropped orignal raw flattened to show iso-grain

(http://s16.postimg.org/4a0nsm8zp/isodualiso.jpg)
Cropped original dual-iso to show iso-grain
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 26, 2013, 04:03:45 AM
@ Danne,

Can you upload the RAWs of this last comparison please.

I think something is badly wrong with the single-ISO sample.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 26, 2013, 04:10:39 AM
One more comparison. This time between raw and dual-iso in movie-mode. two extracted dng,s from raw and dual-iso raw-moviefiles. Even if dual-iso benefits most from high contrast scenes there is still one or two steps better dynamic range with dual-iso even in more evenly lit situations. Pretty amazing since regular raw is already amazingly good.

Danne, I cannot spot the difference between the single & dual iso versions except in the last 2 crops. There has to be something I am missing. Could you please Dropbox the unprocessed RAW's.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 26, 2013, 06:46:52 AM
There is hardly any visible difference in the first pictures as you can see which is a good thing. It is when you start to push the shadows out that you start to see a subtle difference, This gets more visible when you shoot in contrasty situations but there is still a difference, slightly.
The movie raws are about 350mb each, to big to upload from where I sit atm. You can play around with two extracted dng-files if you like :). Link below.
https://docs.google.com/file/d/0B4tCJMlOYfird3RMV0l5c2c0RzQ/edit?usp=sharing

@Ilias G. I don,t see anything wrong with the single iso-file?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 26, 2013, 08:56:18 AM
I've managed to hack the source to work with 7D.

edit:  removed, to many problems.

Don't bug the devs about my build.  I've created it for the sole reason of people with 7D's being able to test this feature.  It comes with no warranty or support (but I'll help where I can).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Alex Roman on July 26, 2013, 09:52:48 AM
There is hardly any visible difference in the first pictures as you can see which is a good thing. It is when you start to push the shadows out that you start to see a subtle difference, This gets more visible when you shoot in contrasty situations but there is still a difference, slightly.
The movie raws are about 350mb each, to big to upload from where I sit atm. You can play around with two extracted dng-files if you like :). Link below.
https://docs.google.com/file/d/0B4tCJMlOYfirTTRETlpaQnZVMG8/edit?usp=sharing

@Ilias G. I don,t see anything wrong with the single iso-file?

Danne, it's amazing, thanks. Video is getting better and better (whitefix?)... Only some blur in highlights so it seems to begin to be pretty usable now. Is that interpolation becomes less noticiable with smaller interval (i.e 400-100)?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 26, 2013, 10:19:48 AM
Have not done any serious filming yet but it could be nice in certain situations. Nighttime, citylights, maybe real estate filming getting the highligts from windows and so on. When taking stills it works really well.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Alex Roman on July 26, 2013, 10:32:53 AM
Certainly is a huge step taking stills! On the other hand, i think the real revolution comes with (pretty) usable dual iso video function -which from what i see is taking giant steps in fews days-

Looking forward to see more dual iso clips (no that agressive interval ones?)  8)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Alex Roman on July 26, 2013, 10:35:46 AM
...maybe real estate filming getting the highligts from windows and so on. When taking stills it works really well.

I also see amazing serious cinema 14-stops DR beautifully rendered imagery...  ;)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 26, 2013, 10:45:33 AM
When I made this comparison with 100-6400 and regular 100 iso one realize there has to be more shadows regained than 2 extra stops. By dividing in such a big interval i,d argue that you suddenly gained about 4-5 extra steps. I was able to lift the shadows to the extent I couldn,t go any further with the lightroom exposure slider. Could have used the brushes though but still.
These shots are even converted with the old converter.

(http://s15.postimg.org/6uyu70fu3/IMG_5760_64k_3.jpg)
dng 100-6400 original


(http://s12.postimg.org/ljv8h7xtp/IMG_5760_64k.jpg)
dng 100-6400 processed

(http://s9.postimg.org/7z0ytnp5r/IMG_5760_64k_2.jpg)
dng 100-6400 cropped

(http://s23.postimg.org/g1nu4jx3v/IMG_5763_3.jpg)
cr2 iso 100 original

(http://s18.postimg.org/p6fzbj4g9/IMG_5763.jpg)
cr2 iso 100 processed

(http://s18.postimg.org/hy8xzzh3d/IMG_5763_2.jpg)
cr2 iso 100 crop
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Pelican on July 26, 2013, 11:52:58 AM
There is a working build for the 7D in the 7D alpha2 topic.
The produced files (ISO 100/1600 and ISO 1600/100):
http://pel.hu/down/100_1600_7D.CR2
http://pel.hu/down/100_1600_7D.DNG
http://pel.hu/down/1600_100_7D.CR2
http://pel.hu/down/1600_100_7D.DNG



Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dadinio13 on July 26, 2013, 11:59:37 AM
Last raw2dng.exe stops at frames 493 for all my footages with the error : mallocxels.
look at this :
(http://img442.imageshack.us/img442/4373/245y.png)
i have downloaded raw2dng.exe many times i was thinking it was a browser cache issue but is not...
any ideas ? please !
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 26, 2013, 12:34:08 PM

The movie raws are about 350mb each, to big to upload from where I sit atm. You can play around with two extracted dng-files if you like :). Link below.
https://docs.google.com/file/d/0B4tCJMlOYfirTTRETlpaQnZVMG8/edit?usp=sharing

@Ilias G. I don,t see anything wrong with the single iso-file?

Thanks for the samples.

The problem I see in your 200% crop of the single ISO file is a magenta cast at the darks of the sculpture and some clipping. Which is much less present with my conversions using Rawtherapee.

What was the lighting during the shots ? I have a hard time finding a correct WB.
Was it consistent in color and intensity ?. Because if it was a candle we cannot come at a valid conclusion comparing only two single frames.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 26, 2013, 01:19:09 PM
How can you whitebalance rawmovies. Rawmovie converted with latest raw2dng.exe. Dngs provided in the link. The test was aimed for noisecomparison. Thanks for watching :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: yanone on July 26, 2013, 01:24:56 PM
Hi everyone,

somewhere here in this thread I read that the 5D3 produces less noise in dark areas with a higher ISO, therefore the dark areas are recovered from high ISO while light areas are recovered from low ISO.

I did some tests today and filmed the same scene (without Dual ISO) in ISO steps from 100 to 3200 and adjusted the exposure accordingly.
After running the DNGs through ACR in AE I can't see this effect. The lowest ISO still has the least noise in recovered dark areas. Starting at ISO 800 there's a noticeable shift towards magenta in the shadows.

In other words: I don't understand the Dual ISO. What did I miss?
As I see it lowest ISO is best for highlights and for shadows.

In the sample images you see the background in dark shadow under a roof, front in sun light.

(https://dl.dropboxusercontent.com/u/5752966/Permanent/ML/Bildschirmfoto%202013-07-26%20um%2013.13.20.png)
ISO 100

(https://dl.dropboxusercontent.com/u/5752966/Permanent/ML/Bildschirmfoto%202013-07-26%20um%2013.13.35.png)
ISO 800

(https://dl.dropboxusercontent.com/u/5752966/Permanent/ML/Bildschirmfoto%202013-07-26%20um%2013.13.42.png)
ISO 3200
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 26, 2013, 02:13:36 PM
Hi Yanone, I think the point is missed here. In normal circumstances (read 10 bit and below situations) you should use the plain RAW mode for still and video. Extreme situations (read 11 bit and above) require extreme treatment. This is what Dual-ISO is all about. Faced with a high Dynamic Range situation (facing the sun, bright reflection), it is better to use this functionality to capture and recover most of the shadows and highlights that would otherwise be lost forever. It is just a special tool to be used in special situations.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 26, 2013, 02:26:53 PM
Hi everyone,

somewhere here in this thread I read that the 5D3 produces less noise in dark areas with a higher ISO, therefore the dark areas are recovered from high ISO while light areas are recovered from low ISO.

I did some tests today and filmed the same scene (without Dual ISO) in ISO steps from 100 to 3200 and adjusted the exposure accordingly.

In other words: I don't understand the Dual ISO. What did I miss?
As I see it lowest ISO is best for highlights and for shadows.


(Faulty step in bold).

You missed the fact that to take the effect of cleaner darks with hi ISO you have to keep the exposure constant between shots .. and exactly is what dual-ISO does ..
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 26, 2013, 02:51:46 PM
IliasG could you elaborate the concept in some more detail.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 26, 2013, 03:00:18 PM
How can you whitebalance rawmovies. Rawmovie converted with latest raw2dng.exe. Dngs provided in the link. The test was aimed for noisecomparison. Thanks for watching :)

To make fair noise comparisons we have to compare shots at the same exposure (light inconsistency can give faulty conclusions) and the compared images been at the same contrast, saturation, WB and detail levels ...

Your uploaded dual ISO DNG is named 6400-100 .. wasn't your shot at 6400-1600 ??

Can you declare what lighting you used ??.
And upload the first frames of each session if possible, I would like to have them because the Black Level data is calculated on the start of each session.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 26, 2013, 03:05:57 PM
Wrote wrong. Dual iso was shot 6400-1600, regular shot 1600, same lighting, same settings on the camera same processing in lightroom. Could upload the first dual iso dng later today. The other dng is the first one I believe? I used a lightbulb
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: yanone on July 26, 2013, 03:27:22 PM
Hi Yanone, I think the point is missed here. In normal circumstances (read 10 bit and below situations) you should use the plain RAW mode for still and video. Extreme situations (read 11 bit and above) require extreme treatment. This is what Dual-ISO is all about. Faced with a high Dynamic Range situation (facing the sun, bright reflection), it is better to use this functionality to capture and recover most of the shadows and highlights that would otherwise be lost forever. It is just a special tool to be used in special situations.

Right. I do consider my sample scene an extreme lighting situation. Front in bright sunlight with reflections, background in dark shadow. The background used to be almost black in the undeveloped DNG, front used to be very bright.

You missed the fact that to take the effect of cleaner darks with hi ISO you have to keep the exposure constant between shots .. and exactly is what dual-ISO does ..

Hmm, I see. Now I'm curious to actually try the Dual ISO. If I get a noticable difference, I'll post an update.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 26, 2013, 03:38:27 PM
IliasG could you elaborate the concept in some more detail.

Alex explains it in http://acoutts.com/a1ex/dual_iso.pdf page 1-2 and gives some links ..

http://www.guillermoluijk.com/article/nonoise/index_en.htm (it's a dead link for the moment, let's hope for recovery)
http://www.dpreview.com/forums/post/28749589
http://www.dpreview.com/forums/post/28750076

The explanation for the decreased of noise at the dark areas at hi ISOs is that there is electronic noise at the late stages (transfer lines, DAC) after ISO amplification which noise is constant. If we feed the DAC with low signal (ISO100) lets say the noise is equal to this signal (SNR 1.0). If we feed it with amplified signal (ISO 1600) then the signal is 16 times stronger while the noise remains constant so SNR is 16.0 .. This with an ideal sensor and first stage electronics with no noise. In practice there is first stage noise also (but lower than late stage) so the SNR becomes around 8.0 and we gain 3 stops DR instead of 4 ..   

 
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: yanone on July 26, 2013, 03:55:01 PM
Here's the scene with Dual ISO 100/1600.
Indeed, the dark background looks fantastic. But it's about the only detail that looks nice in this picture. (Same workflow: raw2dng.exe (July 22nd) -> AE -> ACR)  :-\

(https://dl.dropboxusercontent.com/u/5752966/Permanent/ML/Bildschirmfoto%202013-07-26%20um%2015.43.53.png)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Alex Roman on July 26, 2013, 04:32:21 PM
Ouch!  :(
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 26, 2013, 05:58:53 PM
@Ilias

updated my link now containing the first dngfile from both of the examples. base-iso 1600 for both files. 6400-1600 on the dual-iso. (reversed due to converting issue, still base-iso 1600 though). Lighting and camerasettings identical for both shots.

https://docs.google.com/file/d/0B4tCJMlOYfird3RMV0l5c2c0RzQ/edit?usp=sharing

//D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on July 26, 2013, 06:01:10 PM
There is a working build for the 7D in the 7D alpha2 topic.
The produced files (ISO 100/1600 and ISO 1600/100):
http://pel.hu/down/100_1600_7D.CR2
http://pel.hu/down/100_1600_7D.DNG
http://pel.hu/down/1600_100_7D.CR2
http://pel.hu/down/1600_100_7D.DNG

THANKS Pelican !!! U ROCK  :D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Shizuka on July 26, 2013, 06:46:29 PM
@Ilias

updated my link now containing the first dngfile from both of the examples. base-iso 1600 for both files. 6400-1600 on the dual-iso. (reversed due to converting issue, still base-iso 1600 though). Lighting and camerasettings identical for both shots.

https://docs.google.com/file/d/0B4tCJMlOYfird3RMV0l5c2c0RzQ/edit?usp=sharing

//D

Do you even gain any sort of benefit doing dual iso with recovery ISOs above 1600 (APS-C, 5D2) / 3200 (5D3+) - also known as the point where the read noise no longer decreases?

The theory suggests that 100/[200,400,800,1600] only make sense, and 100/3200 only if you've got a 5D3.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 26, 2013, 07:19:02 PM
check out my picture of a 100-6400 some posts up. Worked rather well.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 26, 2013, 09:16:57 PM
Thanks for the samples. :)

check out my picture of a 100-6400 some posts up. Worked rather well.

It did worked.
But the point is that the difference between 100-3200 vs 100-6400 is very small  in noise quality (0.09 stops in favor of 100-6400) while the difference in artifacts and resolution - detail is significant in favor of 100-3200.

Keep in mind that the result you see using 100-6400 is not only due to the cleaner ISO 6400 but also due to better processing by cr2hdr (black point calibration, channel matching ..) and a kind of denoise ( the data averaging by median filters) which get activated even when the ISO range is small.

You will need tests 100-6400 vs 100-3200 vs  100-1600 vs 100-800 (or 1600-3200 vs 1600-6400) combined with the appropriate for each case denoise by your raw converter to have useful data for a correct strategy for dual-ISO shots.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 26, 2013, 09:25:47 PM
Of course one can spend a whole week comparing. Actually tried 3200 1600 and 12800.  100-6400 worked best. Fully aware of the algorithms with the converter creating the hdr and its importance. My tests so far are a blend of pseudoscience combined with vacation and just having fun ;)
By the way Ilias, did you look at the noisetest dng,s?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jpaana on July 26, 2013, 09:52:02 PM
One small issue I noticed with the converter is that some tags, noticeably GPS coordinates, don't get transferred from CR2 to DNG, but adding -all:all to the exiftool command line fixed that for me:
Code: [Select]
diff -r 728e571a1276 modules/dual_iso/cr2hdr.c
--- a/modules/dual_iso/cr2hdr.c Wed Jul 24 16:58:18 2013 +0200
+++ b/modules/dual_iso/cr2hdr.c Fri Jul 26 22:42:56 2013 +0300
@@ -247,7 +247,7 @@
                 raw_info.white_level /= 4;
 
                 char exif_cmd[100];
-                snprintf(exif_cmd, sizeof(exif_cmd), "exiftool -tagsFromFile \"%s\" \"%s\" -overwrite_original", filename, out_filename);
+                snprintf(exif_cmd, sizeof(exif_cmd), "exiftool -tagsFromFile \"%s\" -all:all \"%s\" -overwrite_original", filename, out_filename);
                 int r = system(exif_cmd);
                 if (r != 0)
                     printf("Exiftool didn't work\n");
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 26, 2013, 10:11:00 PM
..
By the way Ilias, did you look at the noisetest dng,s?

Yes, and in this comparison the single ISO shot matches better the Dual ISO one. The difference is smaller than with the first pair, no weird casts etc.

@ Alex
Do you consider instead of calculating "black data" in camera just save the side optically black pixels for better calculations later with raw2dng ?. For these black data 12bit are enough as it is now and if there is a possibility to subtract the black offset 8bit will be OK (for BL at 2048 subtract 2048-128= 1920 so 256 will cover the range +/- 128).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: budafilms on July 27, 2013, 10:01:03 AM
Hi A1ex!
Your warning about use this method is strong than others advices. I will start a big project - movie - And I need to know wich is the real problem about work with this - I did some shots and no problem yet. So, it's Temperature? Card? Nikon people watching Canon Dual Iso quality?

 ;)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on July 27, 2013, 11:06:04 PM

@A1ex

I try the dual iso on the 7D but i take 1 or 2 pictures and show up the #70 error and didn't record any file to the card. Any ideas or solution for this?

Thanks for all, it bring a new breeze to the  7D owners.
 
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 28, 2013, 02:19:56 AM
@A1ex

I try the dual iso on the 7D but i take 1 or 2 pictures and show up the #70 error and didn't record any file to the card. Any ideas or solution for this?

Don't bug the devs about my build.  I've created it for the sole reason of people with 7D's being able to test this feature.  It comes with no warranty or support (but I'll help where I can).

I don't know why some are getting errors.  I would suggest resetting the camera settings for a start, only shooting raw, no liveview shooting and only using base ISO's (100, 200, 400, 800, etc).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: feureau on July 28, 2013, 03:07:03 AM
I don't know why some are getting errors.  I would suggest resetting the camera settings for a start, only shooting raw, no liveview shooting and only using base ISO's (100, 200, 400, 800, etc).

Using fresh installs from pelican's EOSCard, and no live view: my 7D crashed at ISO 800 and 400 and 100. The crash at ISO 100 is worst. The camera just died and removing and reinserting battery wouldn't revive it. Left it batteryless for a few minutes. Works again.

It would shoot a frame or two, then I could keep firing the shutter, but nothing gets written to card.

None of the crashes produces crash log though. :(
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on July 28, 2013, 03:09:42 AM
I don't know why some are getting errors.  I would suggest resetting the camera settings for a start, only shooting raw, no liveview shooting and only using base ISO's (100, 200, 400, 800, etc).

Sorry Audionut, thanks for the alert and for the build  :D 

If my words don't belong were feel free to delete or to move to the post of the 7D (new ports).

In short words,

-My last picture that i take just freeze de camera, only the red light stay on (forever, evan if i turn of the camera) and in the upper scrren of the camera it's say "busy". I toke the battery off. Its possible to be a format card problem?

-Wend i manage to take a photo in dual iso, i review the photo in camera and i make a zoom to the maximum that the camera  can and i see only in the screen/photo lines. Do i need to convert this photo? Or is some error?

 Thanks one more time Audionut.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 28, 2013, 04:19:06 AM
Just checkout the dynamic range of the photograph (must be more than 14bit after some post processing):

http://www.flickr.com/photos/[email protected]/9379336159/lightbox/

However, the [email protected]_Whitefix is causing some issues in the initial frames in video especially when the iso ratio is high (especially 100-6400 and higher). The alternating horizontal lines between the low and high iso show up even after rendering. Any idea what I am doing wrong with the workflow.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 28, 2013, 06:04:02 AM
An interesting article on the cost of going RAW for full lenght features (120 minutes of the final product):

http://www.redsharknews.com/post/item/885-preparing-to-shoot-raw?page=all
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ap1hk on July 28, 2013, 10:12:36 AM
How did you compile this? I'm using gcc and getting all sorts of errors?

I downloaded the entire ML source and then go to the modules directory and compile the cr2hdr alone by "make cr2hdr".
It seems to work but I am not 100% sure. I compared with the result of wine cr2hdr.exe and didn't see much differences in the end picture so I assume this is going to work. I don't like using wine unless really necessary.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 28, 2013, 11:53:49 AM
@ap1hk. See you got something going on mac. Why not share it?  A lot of guys out there in desperate need of a mac conversion tool :)
//D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ap1hk on July 28, 2013, 12:12:28 PM
@ap1hk. See you got something going on mac. Why not share it?  A lot of guys out there in desperate need of a mac conversion tool :)
//D

I am not sure if my compilation of cr2hdr is 100% okay.  :-\

And cr2hdr is just a small issue but to get dcraw (and maybe exiftool as well) working on Mac, one has to install either Macport or Homebrew which actually could be very troublesome. However if one was using wine already then probably they have one of these unix ports installed.

Anyway, I don't mind sharing it if anyone wants it.  :)

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on July 28, 2013, 12:46:37 PM
@Audionut

Can you upload again the files in the dropbox (its empy) for the 7D users that solve the problem of crashing ?

Thank you  :D

I already manage to get yours last build and install but still crash with the 70error after a few photos and the pink/magenta still there wend use the cr2hdr.

What we can do?


(7D)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: xmd5a on July 28, 2013, 09:16:16 PM
I think that pink highlights is a result of incorrect processing of clipped channels in some cases by cr2hdr.
"--force" option is needed because cr2hdr does not always detects that cr2 is interlaced. Option for reversed order of fields needed too.
One more comparison. No noise reduction (only "Remove dot noise" option), RPP. Aperture, shutter speed and processing settings are the same. Unbelievable clear shadows. (7D)
https://dl.dropboxusercontent.com/u/4624786/dual-iso_test2.1.jpg
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on July 28, 2013, 10:45:27 PM
I think that pink highlights is a result of incorrect processing of clipped channels in some cases by cr2hdr.
"--force" option is needed because cr2hdr does not always detects that cr2 is interlaced. Option for reversed order of fields needed too.
One more comparison. No noise reduction (only "Remove dot noise" option), RPP. Aperture, shutter speed and processing settings are the same. Unbelievable clear shadows. (7D)
https://dl.dropboxusercontent.com/u/4624786/dual-iso_test2.1.jpg

Its possible, with me is the same problem. In camera and computer the  preview of the cr2 of the photo seems to be ok, then wend use cr2hdr its just appear the pink/magenta in the highlights... In my case i think its not clipped channels, wend i toke my photos i have the preoccupation to not clip (overexpose) the photos but still appear the magenta/pink color in the highlights  wed use the cr2hdr.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: cosmo.lv on July 28, 2013, 11:50:28 PM
Alex - how about 5D mkII is any chance to get dual iso on that? :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 29, 2013, 12:15:57 AM
Some random examples of dual iso cr2 images processed in lightroom. Nothing to compare to but the original exposure.

(http://s21.postimg.org/7eggror3b/IMG_6003.jpg)
(http://s21.postimg.org/dqvm1iu5j/IMG_6003_2.jpg)
(http://s21.postimg.org/z4hn6d5bb/IMG_6010.jpg)
(http://s21.postimg.org/spe0vy97r/IMG_6010_2.jpg)
(http://s21.postimg.org/9o96g6pev/IMG_6105.jpg)
(http://s21.postimg.org/v9e93sm5j/IMG_6105_2.jpg)
(http://s21.postimg.org/jzlj8uh47/IMG_6106.jpg)
(http://s21.postimg.org/qfuhyxpnr/IMG_6106_2.jpg)
(http://s21.postimg.org/9e1nwuasn/IMG_6128.jpg)
(http://s21.postimg.org/8ebd14vmv/IMG_6128_2.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sletts02 on July 29, 2013, 02:24:33 AM
I don't suppose this will work in 'Auto' movie mode will it?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: SuperJackRabbit on July 29, 2013, 07:52:57 AM
really amazing!
and I wonder if the imagine could be as sharp as the original or not..
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Alex Roman on July 29, 2013, 11:30:28 AM
Hey Danne, thanks for these; look awesome. Quick question: are they downscaled video frames or just full res pics? Also, which ISO interval on these? (artifacts are minimal)

thnx
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 29, 2013, 02:05:46 PM
Hi Alex. They,re fullres pics. Good stuff :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on July 29, 2013, 04:37:51 PM
Feature request: can we have any Dual-ISO pictures prefixed with some sort of indicator so that it's easier to sort?  I took 400 pictures this weekend, 37 of them were Dual-ISO and it was a pain to go through and pull out the original CR2s.  If they're prefixed and sorted by Name, it'd be easier cuz it'd go PREFIX-FILENAME.CR2, PREFIX-FILENAME.DNG, etc.

Just a thought.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Alex Roman on July 29, 2013, 06:52:58 PM
Hi Alex. They,re fullres pics. Good stuff :)

I see. Thanks ;)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on July 30, 2013, 06:39:43 AM
Folks, any idea how to recover the exif data of a photo taken using dual_iso. Want to see what settings was the camera using when I took the shot.

You may have noticed the filmy grain look (as opposed to noise on a DSLR) when shooting with dual_iso. There are situations that don't need the dual functionality since the DR is not too high but I still cannot get myself to not using dial_iso. I just love the results. ACR 8.1 handles RAW so much better when shot with multiple iso.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sletts02 on July 30, 2013, 07:15:47 AM
Any update on if you can shoot 'auto' video exposure and dual iso raw?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Magic 7D on July 30, 2013, 02:46:51 PM
check out my picture of a 100-6400 some posts up. Worked rather well.

First post.. So please bear with me if I seem dumb.
But Danne I'm very impressed with your pictures! Would you like to share how managed to render them? I've been trying for a few days now almost full time to get some good pics out of this Dual ISO function, but always get the horrid pink magenta highlights from cr2hdr and even some green stuff when under exposed, don't know what I'm doing wrong.. Did you use cr2hdr? Or how did you process your images can't wait to get a convenient workflow for this amazing function. And can't thank everyone working with it enough!!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 30, 2013, 02:58:49 PM
Yup, used cr2hdr. In the nightpictures there is some green and red moiré or something. Only minor though. In the daylight pictures almost none. When filming a lot more green/red issues appear. Mostly 100-1600 or 100-3200 used in the pics. Clean the pictures in lightroom to get rid of purple fringing
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 30, 2013, 04:37:20 PM
There is a working build for the 7D in the 7D alpha2 topic.
The produced files (ISO 100/1600 and ISO 1600/100):
http://pel.hu/down/100_1600_7D.CR2
http://pel.hu/down/100_1600_7D.DNG
http://pel.hu/down/1600_100_7D.CR2
http://pel.hu/down/1600_100_7D.DNG

In these files, the white level is close to 13000; in my 7D test sample it's 15382. Normally you get these values with intermediate ISOs (160, 320).

If you change the white level in cr2hdr.c to 13000 instead of 15000, these files will be converted without pink highlights. Now trying to figure out how to get this value from exif or what settings to use to get raw values higher than 15000.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 30, 2013, 05:35:34 PM
Updated the converters (download from first post):

cr2hdr:
- fix memory leak, tested with 100 GB of pictures
- white level autodetection (should fix pink highlights)
- exiftool fix from jpaana, not tested but looks OK

raw2dng:
- fix memory leak, not tested (should be OK with large clips)

I did not address the black level issues yet.

If you are having issues with image quality, please upload the raw files (*.CR2 or *.RAW).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 30, 2013, 08:52:56 PM
Some more examples of what dual iso can achieve. The shadows pop out like never before. Mostly use interval between 100-1600 and then use ETTR to find exposure settings. Works perfect most of the time. From a visit in Copenhagen.
Due to bandwith reasons and uploaded images being too big i decided to post a youtubelink containing the stills instead. Enjoy.


Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Magic 7D on July 30, 2013, 09:04:39 PM
Some more examples of what dual iso can achieve. The shadows pop out like never before. Mostly use interval between 100-1600 and then use ETTR to find exposure settings. Works perfect most of the time.

It's incredible!
Sorry if this is a noob question, but what is ETTR...? I was actually going to ask you how you know what exposure to do before.

@a1ex thanks a thousand for your hard work!!! How can I send you a cookie?  :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 30, 2013, 09:10:03 PM
ETTR is a great function that require some testing out. It,s in the menu. You can search on the forum on how to work with it.
I just hit ETTR on and then proceed to dual iso where I set the interval lets say 100-1600 or 100-3200. WHen I hit the set button the mirror flips and the ETTR is making its magic (calculating exposure to the right). Don,t be surprised if the iso and shutter changes. I believe the intervals stays the same though. Not sure. Than take the picture.
You could always do it manually by selecting the first iso for the highlights and then the other iso for shadows. If there,s a lot of contrast usually between 100-1600 or bigger works nice. Even 100-6400 could work sometimes.
Yup, it,s kind of magical. Feels like another camera atm :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Walter Schulz on July 30, 2013, 09:12:58 PM
@Magic 7D: Next time use the search function, please.
http://www.magiclantern.fm/forum/index.php?topic=5693.0

Ciao, Walter
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 30, 2013, 09:15:44 PM
In these files, the white level is close to 13000; in my 7D test sample it's 15382. Normally you get these values with intermediate ISOs (160, 320).

If you change the white level in cr2hdr.c to 13000 instead of 15000, these files will be converted without pink highlights. Now trying to figure out how to get this value from exif or what settings to use to get raw values higher than 15000.

Some Canon models (I think all APS-C with 18Mp) have lower white levels for ISO 100 than the rest ISOs. For 7D it's around 13500 for ISO 100 and 15300 for the rest "integer ISOs".

It would be very useful if you find a way to declare this from *.CR2 and it's exif :)

The sad thing is that some channels (B and G2 in my samples) have raw values higher that this white clipping point making the detection difficult even if we have burned areas ..
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 30, 2013, 09:36:33 PM
dcraw uses 13584 (hardcoded). And yes, since the white points changes with ISOs, it makes things difficult; for now, I just use the smallest value from burned areas, if it's above 10000. Probably a bit overkill, maybe 13000 should be a better starting point?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sletts02 on July 30, 2013, 11:29:58 PM
It would be fantastic to see some video examples of this. Perhaps indoors with windows to see how it can compensate for blown windows/doors.

Also, could someone try recording in auto exposure movie mode and report back it it works with dual ISO - I will be forever grateful!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 30, 2013, 11:33:57 PM
Right now, auto exposure won't work in raw video, because the digital ISO gain is not applied to raw data.

But if you include the post deflicker algorithm in raw2dng, you may even get something usable. Worth trying IMO.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on July 30, 2013, 11:34:03 PM
dcraw uses 13584 (hardcoded). And yes, since the white points changes with ISOs, it makes things difficult; for now, I just use the smallest value from burned areas, if it's above 10000. Probably a bit overkill, maybe 13000 should be a better starting point?

Dcraw is ineffective regarding WP, it uses a single value for every ISO and runs into the known problem, either pink highlights (ISO 100-160-320-...) or 1/3 stop burned highlights.

The best solution is to have a lookup table with the correct clipping point for every case. Keep in mind that we have data scaling (and higher WP) even when shooting at wide apertures (wider than f/2.8) with Canon glass ..

Adobe DNG converter uses 13000 as WP for 100-160-320 .. and 15000 for 200-400-800 .. For 250-500... I think it goes up to 16000.
But takes no care for wide apertures so in this case up to 1/3 stop highlights get lost ..

I do think that if you use a single WP then 13000 is a safe bet. Although you loose 1/3 stops of highlights for ISO 200-400-.. and maybe more for 250-500.. and when using wide apertures.

13584 is on the limits for ISOs 100-160-320-.. because when there is raw denoise (like "black frame subtraction") then we take not a single clipping point but a distribution extending -/+ 100 raw levels from the center value .. it's the same case as in Guillermo's  study .. http://www.guillermoluijk.com/tutorial/satlevel/index.htm
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 30, 2013, 11:40:42 PM
Well, I'd use 13000 or higher. If there are no burned areas, the only side effect should be picture being a little brighter than normal, and maybe a few clipped pixels (and highlight recovery should catch them). If there are burned areas, it will use the true white level.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on July 31, 2013, 04:18:44 AM
Updated the converters (download from first post):

cr2hdr:
- fix memory leak, tested with 100 GB of pictures
- white level autodetection (should fix pink highlights)
- exiftool fix from jpaana, not tested but looks OK

raw2dng:
- fix memory leak, not tested (should be OK with large clips)

I did not address the black level issues yet.

If you are having issues with image quality, please upload the raw files (*.CR2 or *.RAW).

The new cr2hdr works very good, no more pink highlights in the photos taking with the 7D. Tomorrow will take decent photos.

Thank you a1ex  :D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: feureau on July 31, 2013, 08:43:08 AM
Btw, is there a way to make a drag-and-drop a bunch of cr2 to convert to DNR using cr2hdr? (batch convert)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on July 31, 2013, 08:44:48 AM
On Linux I run cr2hdr *.CR2

No idea if this works on Windows, but try it.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on July 31, 2013, 10:53:34 AM
Tried a little quick and dirty test filming in 3x crop mode with a 14mm samyang lens. I used 6400-100 setting in a very dark kitchen. Surprisingly I can,t find any moiré or aliasing. The picture is totally clean! Check out the movielink. Genious work.


Also a link to my still pictures made with dual iso on youtube.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on July 31, 2013, 11:34:20 AM
Btw, is there a way to make a drag-and-drop a bunch of cr2 to convert to DNR using cr2hdr? (batch convert)

Drag and drop a bunch of them on the executable.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: feureau on July 31, 2013, 11:42:33 AM
Drag and drop a bunch of them on the executable.

Thank yoU! :D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: lourenco on August 01, 2013, 03:50:58 AM
These are not extreme examples, but show practical use for me at the track. 100-800 ISO. 


(http://trackshotsphotos.smugmug.com/photos/i-6wwwBjv/0/L/i-6wwwBjv-L.jpg)

(http://trackshotsphotos.smugmug.com/photos/i-kDp4MHm/0/L/i-kDp4MHm-L.jpg)

(http://trackshotsphotos.smugmug.com/photos/i-gvHxBdq/0/L/i-gvHxBdq-L.jpg)

(http://trackshotsphotos.smugmug.com/photos/i-k9BGZ4V/0/L/i-k9BGZ4V-L.jpg)

It was a big help for this shot.
(http://trackshotsphotos.smugmug.com/photos/i-m88f7Q9/0/L/i-m88f7Q9-L.jpg)

(http://trackshotsphotos.smugmug.com/photos/i-zVsdvWm/0/L/i-zVsdvWm-L.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ShootingStars on August 01, 2013, 04:27:50 AM
Nice shots! Is it me or does it seem these "merged" shots seem kinda unrealistic? Or is it because we have rarely seen this. Looks like HDR. What's the difference?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on August 01, 2013, 11:35:45 AM
Would be very nice if you can upload some DNG files (or CR2 if you still notice aliasing artifacts).

Aliasing seems under control these days.  Mainly false colors.
(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/falsecolor.jpg)

There is a little aliasing on his left elbow (https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/D46A9993.CR2).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 01, 2013, 11:38:10 AM
@Audionut. Is it green fringing? Could it be removed in lighroom?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on August 01, 2013, 11:40:19 AM
Yeah LR does remove it.  It requires fairly heavy defringing though which can cause artifacts.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 01, 2013, 11:42:25 AM
I see. Nice work, Looking forward to more examples using this great function :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 01, 2013, 11:43:19 AM
@Alex. I uploaded three cr2 with smaller issues on them. I don,t expect anything, think it works good as is. Provided in the link three cr2-files and a word-document with issuedescription.
https://docs.google.com/file/d/0B4tCJMlOYfirc1NZbVhJR2tMNHM/edit?usp=sharing
Thanks Alex
//D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on August 01, 2013, 12:33:28 PM
It's a bird? It's a aeroplane? It's a bulldozer?  NO! Its A1ex and the rest of the ML gang   ;D 

BIG THANK YOU for fixing the 70error for the 7D.




-7D IS BREATHING AGAIN -
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: pit3k on August 01, 2013, 02:37:05 PM
Quick test for sharpness: 100% middle crop of 7d with 17-55/2.8. 17mm @ 2.8

No dual iso (iso 100):

(http://img713.imageshack.us/img713/5523/w7ij.jpg) (http://imageshack.us/photo/my-images/713/w7ij.jpg/)

Uploaded with ImageShack.us (http://imageshack.us)

Dual iso 100/1600:

(http://img843.imageshack.us/img843/3016/ennf.jpg) (http://imageshack.us/photo/my-images/843/ennf.jpg/)

Uploaded with ImageShack.us (http://imageshack.us)

cr2 and raw files: https://www.dropbox.com/sh/tw3lo4kblurabkn/zjZGCfABR5 (https://www.dropbox.com/sh/tw3lo4kblurabkn/zjZGCfABR5)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 01, 2013, 03:15:36 PM
That's why you should try to cover your main subject with both ISOs, and keep the extra range for highlight/shadow recovery (e.g. detail in clouds).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: pit3k on August 01, 2013, 03:29:25 PM
That's why you should try to cover your main subject with both ISOs, and keep the extra range for highlight/shadow recovery (e.g. detail in clouds).

I think in this situation this wasn't the case. Light was even on the desk with no over or under exposure.

I think the cr2hdr needs a little bit of tweaking to leave alone the mids, and work with only  highlights and shadows.

Or maybe its exposure problem. What is av mode exposing for, 100 iso? Shouldn't I expose for around 640?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on August 01, 2013, 04:15:45 PM
I wonder if the posts about how it looks too "HDR" are just because we're not used to this sort of latitude coming from the camera.  I was watching Ray Donovan last night and also True Blood a couple of nights ago and I paused a couple of times where there was a lot of DR in the scene and it frankly looks like HDR.

Both the Alexa and film shoot at 14 stops and pausing such an image looks like the HDR pictures from the camera.  Just my $0.02!

-Ben
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 01, 2013, 05:35:08 PM
I think the cr2hdr needs a little bit of tweaking to leave alone the mids, and work with only  highlights and shadows.

Did you read my paper?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: pit3k on August 01, 2013, 06:11:59 PM
Did you read my paper?

Now I have, my bad mate. Your work is amazing.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: stevefal on August 01, 2013, 07:00:29 PM
It would be cool if the latest dynamic range features could be combined to provide a better and more convenient high-bit-depth still image. This maybe a blue-sky idea, but oh well.

First to clarify a term, "HDR" is casually used to describe two different things: 1) image data with high bit-depth (16+ bit) per channel, and 2) an image that was captured with high bit-depth and then compressed (curves, microcontrast etc), to reveal detail on low-contrast devices, sometimes with garish results - hence, "that looks unrealistic, like HDR".

Savvy photographers want the latitude of high bit-depth images in order to create the best possible result in post. So in-camera "HDR" features that generate an 8-bit JPEG from multiple exposures defeat the purpose.

Bracketing discreet still shots for the purpose of creating high bit-depth negatives in post is better, but the drawback is artifacts due to motion between shots.

So the question is, would it be possible to improve on the dual-exposure model by creating a new ML High-Range mode that does the following:

- shoot through viewfinder (ideally) a la still mode
- two iso-bracketed exposures during one shutter actuation - a la rawrec and miracle-happens-here
- two shots are combined in-camera - mids from first and highlights/shadows from second - kinda a la dual-iso modulo a1ex genius
- save 16-bit DNG that opens directly in ACR

This approach would have some drawbacks:
- lower still burst rate
- highlight/shadow artifacts due to motion

and advantages:
- pristine IQ for still scenes - no aliasing
- less severe motion artifacts than dual-shutter bracketing
- no workflow before ACR

versus current dual-iso advantages:
- higher burst rate
- no artifacts due to motion

and drawbacks
- always highlight/shadow artifacts due to interlace aliasing
- workflow steps before ACR

If the in-camera part was impossible or way too slow, combining could be done in workflow while keeping the IQ advantages.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on August 01, 2013, 07:14:32 PM
Some tests

NORMAL

(http://imageshack.us/a/img837/1861/r3mz.jpg)


DUAL ISO

(http://imageshack.us/a/img407/5785/80qu.jpg)

This is wanderful and never show up the 70error  :D

I see some things happen that i don't know if it's normal

1- Wend i back home to see the photos, i see in the structure of the CF card (in canon folders) that after i take a few photos he creates another folder and put the rest of the pictures there.

2-Wend i pick-up all the photos and drag to the cr2hdr, he fail always the first pictures. The rest of the pictures, he transcribe good and make the dng file. But almost in the end of each file, wend cr2hdr are doing is work, its show up a message saying that the exiftool didn't work, like the image below. Evan with this error the cr2hdr make is job and make the dng that i can open in photoshop for making the right tweek.


(http://imageshack.us/a/img441/9287/uod5.jpg)


3- A problem that i come across is that is hard for make the right exposure.  Didn't, yet, get the best way to exposure.

4- Also noted that the photos make a strange border, but can be because of the jpg. I can crop wth no problem. Just for let you guys  know about this.


Beside this DUAL ISO is one more great function in the ML arsenal =D.

THANK YOUU !!!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on August 01, 2013, 09:41:11 PM

2-Wend i pick-up all the photos and drag to the cr2hdr, he fail always the first pictures. The rest of the pictures, he transcribe good and make the dng file. But almost in the end of each file, wend cr2hdr are doing is work, its show up a message saying that the exiftool didn't work, like the image below. Evan with this error the cr2hdr make is job and make the dng that i can open in photoshop for making the right tweek.

Try running cr2hdr in a directory with no spaces just off the root directory.  It works for me like that.  You'll see the weird letters; for some reason it doesn't like spaces in the directory name (exifinfo.exe).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on August 01, 2013, 10:12:53 PM
Try running cr2hdr in a directory with no spaces just off the root directory.  It works for me like that.  You'll see the weird letters; for some reason it doesn't like spaces in the directory name (exifinfo.exe).

Much appreciated dubzeebass, thank you.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Sky_cleaner on August 02, 2013, 12:36:36 AM
Thanks to ALL for this wonderfull module - dual ISO! But how to work with resulted  .CR2   on Mac OS?  Can anyone help Mac users?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dadinio13 on August 02, 2013, 01:50:03 AM
Somes tests again :

(https://lh3.googleusercontent.com/-Vole3VqBpEw/Ufruw-mKe6I/AAAAAAAAAD0/Q_9t2QkRf_0/w1336-h892-no/HL9A2300.jpg)
(https://lh3.googleusercontent.com/--hNgniMt3Og/Ufrrn_50yII/AAAAAAAAADs/a47J1f2LXsk/w1336-h892-no/HL9A2297.jpg)
(https://lh5.googleusercontent.com/-ZiL9q7Kss40/Ufrzjc7c7GI/AAAAAAAAAFg/qhWKVklxB-k/w1336-h892-no/HL9A2336.jpg)

:)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: xmd5a on August 02, 2013, 09:36:04 PM
Here is a link to problematic cr2 (waves in the center of the picture painted in the colors of the rainbow)
https://dl.dropboxusercontent.com/u/4624786/_MG_8792.CR2
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 02, 2013, 10:32:37 PM
This should help you understand better how it works and how to expose the pictures:

http://acoutts.com/a1ex/split_MG_8792.dng
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: daancalo2013 on August 02, 2013, 11:39:59 PM
Here is a link to problematic cr2 (waves in the center of the picture painted in the colors of the rainbow)
https://dl.dropboxusercontent.com/u/4624786/_MG_8792.CR2

I am new but eager to aprendre, here I leave another example
http://we.tl/BeTdsXJe4G
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 02, 2013, 11:55:39 PM
Here I'm afraid the only solution is to cover the palm trees with both ISOs. Remember that aliasing can't be removed in software (Nyquist theorem), only the artifacts can be somewhat masked.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on August 03, 2013, 12:29:26 AM
Here I'm afraid the only solution is to cover the palm trees with both ISOs. Remember that aliasing can't be removed in software (Nyquist theorem), only the artifacts can be somewhat masked.


Quem sabe, sabe

who knows, knows
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sletts02 on August 03, 2013, 02:48:00 AM
Any video examples yet? It'd be great to see some =)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on August 03, 2013, 05:21:46 AM
This should help you understand better how it works and how to expose the pictures:

http://acoutts.com/a1ex/split_MG_8792.dng

Undoubtedly that the picture from bottom have the right exposure. But how you get at that settings?  (7D) 
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 03, 2013, 07:58:58 AM
Basically, what needs to have full detail or it's likely to get aliasing must be covered by both ISOs. In LiveView, check the exposure with both ISO values and zebras.

In these cases, less extreme settings like 100/400 should be very helpful.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on August 03, 2013, 11:05:11 AM
Basically, what needs to have full detail or it's likely to get aliasing must be covered by both ISOs. In LiveView, check the exposure with both ISO values and zebras.

In these cases, less extreme settings like 100/400 should be very helpful.

Indeed, thank you, will try more pic's.

I think with the silent picture burst in the 7D (the beginning of raw=) it will be more easy... if we get the ETTR working, no?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 03, 2013, 11:09:33 AM
ETTR should be already working. Be careful though - with dual ISO it exposes for the least amount of noise and ignores moire.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: hamishniven on August 03, 2013, 11:28:11 AM
Awesome news.
Ok, for the dumbo here, how do I use the CR2toHDR.exe file?
Is there any documentation as I'm a photographer not a programmer.

Thanks
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: pit3k on August 03, 2013, 11:56:31 AM
Awesome news.
Ok, for the dumbo here, how do I use the CR2toHDR.exe file?
Is there any documentation as I'm a photographer not a programmer.

Thanks

Drag and drop cr2 files to the cr2tohdr.exe.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on August 03, 2013, 12:19:34 PM
ETTR should be already working. Be careful though - with dual ISO it exposes for the least amount of noise and ignores moire.

Massive!!!

But do you know if the ETTR are in the new version of Pelican? For now we just got the silent burst pictures, almost 2 seconds of video.

Going post a video with that in the 7D treat.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 03, 2013, 12:23:28 PM
If it's not, copy it from some other nightly. Modules are portable by design.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on August 03, 2013, 12:33:33 PM
If it's not, copy it from some other nightly. Modules are portable by design.

Alright, super duper easy  :D 

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on August 03, 2013, 03:21:10 PM
@A1ex

I have download the 5DII ETTR module from https://bitbucket.org/a_d_/magic-lantern/downloads (the first zip file), this is correct?

Also have the autoexpo.mo, it will work in the 7D?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 03, 2013, 03:32:54 PM
I'm experimenting with some algorithms that should handle aliasing a bit better. Here's an experimental cr2hdr: http://acoutts.com/a1ex/cr2hdr_exp.exe

Can you compare it with the one from first post? I'm interested especially in cases where the old converter gets better results.

Main difference: in aliased areas I use chroma blur (luma is no longer blurred), and the algorithm uses a stronger bias towards full-resolution mode (which is only possible in midtones, where the subject is covered by both ISOs).

Here are my before and after samples (CR2's from Danne). Click for full-res jpegs:

(http://acoutts.com/a1ex/IMG_6706_before_crop.jpg) (http://acoutts.com/a1ex/IMG_6706_before.jpg) (http://acoutts.com/a1ex/IMG_6706_after_crop.jpg) (http://acoutts.com/a1ex/IMG_6706_after.jpg)

(http://acoutts.com/a1ex/IMG_6701_before_crop.jpg) (http://acoutts.com/a1ex/IMG_6701_before.jpg) (http://acoutts.com/a1ex/IMG_6701_after_crop.jpg) (http://acoutts.com/a1ex/IMG_6701_after.jpg)

For 100% aliasing-free results, don't forget to cover your main subject with both ISOs. There's no other way.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 03, 2013, 03:57:27 PM
Wow! Big difference in aliasing. I,ll try some comparisons conversions soon. On the road...
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 03, 2013, 05:13:32 PM
Found something surprisingly similar to this method:

http://image-sensors-world.blogspot.tw/2013/01/sony-explains-hdr-video-mode-in-its.html
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: robinlee on August 03, 2013, 06:11:36 PM
Hi all, newbie here but I have done all the installations... on some test shots I've done, when viewing at 100% I can see some horizontal lines as opposed to normal exposed images. Is there any reason for this?

I tried cr2hdr.exe but seems that the converter does nothing just open and close in 1/2 secs... any thoughts?

Here is a sample, this is 100% crop, exposed to highlight to prevent pink/magenta at highlight area, dual ISO 100/1600...

(http://farm6.staticflickr.com/5519/9430444662_0a8836103f_o.jpg) (http://www.flickr.com/photos/robinsslee/9430444662/)
_38A6284 (http://www.flickr.com/photos/robinsslee/9430444662/) by Robin SS Lee (http://www.flickr.com/people/robinsslee/), on Flickr

Thanks
Robin
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arrinkiiii on August 03, 2013, 06:29:15 PM
Hi all, newbie here but I have done all the installations... on some test shots I've done, when viewing at 100% I can see some horizontal lines as opposed to normal exposed images. Is there any reason for this?

I tried cr2hdr.exe but seems that the converter does nothing just open and close in 1/2 secs... any thoughts?

Thanks
Robin

The lines is the interlaced ISO. You need to "de-interlaced" using cr2hdr.exe

Put the cr2hdr.exe in the same folder that your pictures. Click and drag the pictures to the cr2hdr.exe and you are done. It will create the DNG files.

Read the first post... and the rest of it.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 03, 2013, 06:29:51 PM
Keep the folder with the cr2 with no spaces, short name otherwise it won,t convert.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: robinlee on August 03, 2013, 07:20:47 PM
Thanks for your help arrinkiiii and Danne... :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Shield on August 04, 2013, 03:30:31 AM
Shot some raw video today and when unpacking with the raw2dng.exe it gets about halfway and then:

"Error: mallocixels"

E:\RAW\5>raw2dng m03-2004.raw
Resolution  : 1920 x 1080
Frames      : 881
Frame size  : 3629056 bytes
FPS         : 23.976
Black level : 2079
White level : 15000
Processing frame 1 of 881...
Vertical stripes correction:
  1.000  1.000  1.009  0.970  0.987  0.997  0.985  1.036

ISO pattern    : ddBB
Interpolation  : mean23-vsmooth3-contrast
ISO difference : 6.03 EV (6516)
Black delta    : 89
ISO overlap    : 3.0 EV (approx)
Processing frame 494 of 881... [HDR BBdd]Error: mallocixels]

Any ideas?

5d3, Aug 2 build.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Shield on August 04, 2013, 04:08:35 AM
Figured it out - was using "Bachelor" to unpack the raw files, and it had an older version of Dcraw.exe in it.  Problem fixed.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: melihozbek on August 04, 2013, 11:04:38 AM
Hi, I am ashamed to ask this as a software developer, but I couldn't manage to get the modules menu on 7D.. What did I do wrong?

1. I downloaded the ML from http://upload.g3gg0.de/pub_files/17248a00956f1e932457094756b2a3ba/magiclantern_7D_203_Alpha2.zip
2. I downloaded modifications from the post #1 of this thread. ANd put modules in ML/MODULES folder.
3. I managed to get the ML Menu but no Dual Iso module, or menu.

Sorry to interrupt  :(


Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 04, 2013, 11:51:15 AM
There is a thread specific for the 7D. Try the search function
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Sky_cleaner on August 04, 2013, 12:43:16 PM
http://www.magiclantern.fm/forum/index.php?topic=3974.975
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 04, 2013, 12:58:31 PM
Quote
For 100% aliasing-free results, don't forget to cover your main subject with both ISOs. There's no other way.

@A1ex: Downloaded the ch2hdr exp version. The results are much better. Don't understand what you mean by "don't forget to cover your main subject with both ISOs"
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 04, 2013, 01:10:23 PM
For full resolution and no moire, you should have detail with both ISOs.  Where one of them gets clipped, you will get half-resolution. If the subject doesn't have fine detail, half-resolution will not be noticeable; but if it has, you will get moire.

In shadows, this is less of a problem; you still have detail in both ISOs, but one of them has too much noise. That detail is usually enough to reduce aliasing, and the algorithm takes care of that. In highlights, if it's clipped, there's zero detail, so you just can't bring back the missing lines.

So, you have to be careful what is covered by both ISOs (OK) and what is covered by only one (risk of moire).

Try zebras with the higher ISO in LiveView: if the main subject is not overexposed, should be OK. Background highlights can be overexposed; if they are not too sharp (e.g. clouds), you will not see the aliasing at all.

This should also help: https://bitbucket.org/hudson/magic-lantern/commits/84d446e1e06487fdf9a6cd1e3969ac717c84651c
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jaybirch on August 04, 2013, 01:55:18 PM
any chance of the new algorithm working with RAW2DNG too?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 04, 2013, 02:27:44 PM
There,s a nice app here that works for mac with dual iso.
http://www.magiclantern.fm/forum/index.php?topic=7266.msg62587#msg62587
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: davidcl0nel on August 04, 2013, 06:07:43 PM
DualIso works well on 5D3, thank you for this feature! With the first tests the detail loss of structures was a little bit to much, but with newer converters it is much better. But the shadow recovery works very very good.

But I have a little "other" request. I install the ML-version from the dropbox onto the CF-Card (not SD), and it's working good. Yesterday I want to format the CF, because of 300+ pics, and the format menu cleans the whole card, with ML. Grrr, I had to copy it again with the card reader.

With the ML-version of my 60D I know the "feature", which reads the whole ML-directory (and autoexec) into the RAM, formats the card, and rewrite ML back to the card. Is this feature broken or is it deactivated on purpose?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 04, 2013, 06:21:58 PM
That part needs to be reverse engineered almost from scratch because of the double card slot. Same for the installer.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 04, 2013, 06:56:18 PM
A little testing with Alex new aliasing changes in the experimental cr2hdr_.exe and the most recent one from the first post.
A little more jagged edges in the aliasing parts in some motifs but it also completely removes aliasing from other motifs in the experimental. Hard to tell what is best, I,m leaning towards the experimental, especially from the pic of the old woman and of course from the store. Here,s some pics. I strived to get the motif with both isos.
The originals can be downloaded from this link (119mb 4 pictures)
https://docs.google.com/file/d/0B4tCJMlOYfirRlFjZ3ljVEtDc0U/edit?usp=sharing

(http://s9.postimg.org/e9v70rpu7/1bnew_1_av_1.jpg)

experimental
(http://s9.postimg.org/6ndmy53m7/2bnew_1_av_1.jpg)

recent
(http://s9.postimg.org/nmmldeetr/2bold_1_av_1.jpg)

(http://s9.postimg.org/c2qycukjz/1new_1_av_1.jpg)

experimental
(http://s9.postimg.org/iqiylpeof/2new_1_av_1.jpg)

recent
(http://s9.postimg.org/48lrdpndb/2old_1_av_1.jpg)

experimental
(http://s9.postimg.org/5g4ajo2vj/4new_1_av_1.jpg)

recent
(http://s9.postimg.org/k4f6ur3bj/4old_1_av_1.jpg)

(http://s9.postimg.org/z6w0j6d8v/3cnew_1_av_1.jpg)

experimental
(http://s9.postimg.org/sy63buqa7/4cnew_1_av_1.jpg)

recent
(http://s9.postimg.org/4gdznynpr/4cold_1_av_1.jpg)

Magentaissue in some pics from the experimantal build, not seen in the converter from the first post.
I also only manage to convert 35 pictures, cr2 with the converter from the first post and the it quits. This don,t happen with the experimental build.
(http://s9.postimg.org/rmyc3duof/magenta_1_av_1.jpg)

If any of these are of interest as originals please tell me which and I,ll upload those.

*post updated with link to the originals(see above)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 04, 2013, 06:59:45 PM
Yes, upload them. These are very good for my test shot collection.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 04, 2013, 07:39:34 PM
@Alex
Post updated with originals
https://docs.google.com/file/d/0B4tCJMlOYfirRlFjZ3ljVEtDc0U/edit?usp=sharing
Thanks!
//D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: NateVolk on August 04, 2013, 09:30:36 PM
@Danne - What causes the magenta issue in the highlights?  I have a few pics I can't get that to go away on...
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 04, 2013, 09:42:24 PM
Download cr2hdr_exp.exe again and upload the CR2's if you still have trouble. It was hot pixels in overexposed areas tricking the white level autodetection code.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: NateVolk on August 05, 2013, 05:49:31 AM
Thanks A1ex,  That fixed the magenta highlight issues, but still major ailiasing.  I probably exposed way wrong and didn't keep things covered by both iso's.

One thought I had that might make things easier shooting is to be able to reverse the iso's.  Meaning shoot at iso 1600 and use iso 100 as recovery for the highlights.  I did a real estate shoot where if I exposed to not clip the highlights at iso 100 and set the dual iso to 1600 to save the mid tones, the live view was basically black and I couldn't frame my shots.  If I switched to 1600, then I could see the scene more properly exposed.  I would then switch back to 100 to shoot...

Maybe your script is already that smart, I haven't tested it yet.  I also have found that for video to come out correct, I have to reverse the settings as well...

Thanks for all your accomplishments, it really is mind boggling!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on August 05, 2013, 07:17:58 AM
@NateVolk

You can shoot with the higher ISO and then select the highlight recovery ISO in the dual-ISO menu.  You can use the +-EV settings also to either consistently use +whatever EV for recovery or consistently -whatever EV.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: AdamTheBoy on August 05, 2013, 07:36:04 AM
I'm having an issue I would like help with please, I don't think I've seen this before on the forum and I couldn't find it with a search.

A number of my pictures will begin processing only to say "file doesn't look like dual ISO"

The images are definitely dual ISO so I'm not really sure where to start troubleshooting, any suggestions?  Thanks very much.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 05, 2013, 10:15:04 AM
For the 1000th time: if you are having issues and you don't upload some CR2 files, there's nothing I can do to help you.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: AdamTheBoy on August 05, 2013, 10:35:51 AM
I'm sorry about that, that should have been obvious to me.  Here's the .cr2 file. https://www.dropbox.com/s/s2tew00wftvlef4/7O4C8804.CR2
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 05, 2013, 11:14:02 AM
Uploaded a fix (download cr2hdr_exp.exe again). Can you check with your other pics?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on August 05, 2013, 11:23:05 AM
a1ex can you add something to the filename to differentiate dual-ISO's? 

I shot 1000 photos the other day with dual-ISO on alternate frames thinking that would be a great idea.  1 back-up with each dual-ISO shot.  It worked great, and I love the results of +2EV shadow recovery, but processing them was a pita, simply because they take up space otherwise, if you don't delete the original CR2's.

Canon uses different filenames for AdobeRGB and sRGB color spaces.  Perhaps you could hook into that?

Once everything gets closer to RC, I would like to suggest the option of deleting the original CR2's after being processed by cr2hdr.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 05, 2013, 11:33:28 AM
My workflow is to drag all the files on cr2hdr: it will autodetect which shots were dual and which not. Sure, it takes a bit of time. My autoexposure script uses DNG by default, so it's quite straightforward: I just run both tools, leave it crunching numbers for a few hours, and I get nearly-finished pics.

For changing file names, I can add an option to shoot dual ISOs in adobe RGB, and normal pics in sRGB, for example. Or, I can change the first 4 letters (suggestion?)

I would not suggest deleting the CR2 files. The algorithm is improving, so if you reprocess the same thing a month later, you already get better results.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on August 05, 2013, 11:37:37 AM
I was selecting everything over 26000kb to drop on the exe.  Still takes time to process 000's of photos though. 

The first 4 letters are a good idea.  DUAL0001.CR2  :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 05, 2013, 01:50:19 PM
Small problem with this approach: they are no longer sorted (you need to rename them back in post so you can watch them in the same order). Plus, syncing the file names with the alternate option (semi-burst) is tricky (it's too easy to mis-label the pictures, because capturing and saving are done at different times).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on August 05, 2013, 03:04:04 PM
Hmmmm.  I would be happy if their was a cmd option to move processed files.  Including non dual-ISO files put through the exe.
I could dump the CF onto an SSD and have all the processed files moved to storage.  Not sure of the usefulness of this for others though.

The seperation of processed and original dual-ISO's is what I'm chasing.  I can't think of any easier ways.

edit:  And thanks.  Just noticed the commit.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: robinlee on August 05, 2013, 10:33:10 PM
Hi Alex, can you post the download link for this on your first page please?

Thanks
Robin

Uploaded a fix (download cr2hdr_exp.exe again). Can you check with your other pics?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 05, 2013, 10:39:05 PM
Small update for cr2hdr_exp.exe: better black level handling. This is only noticeable in extremely underexposed shots (e.g. when you need to push the shadows at +10 EV).

Before and after (CR2 from AdamTheBoy):
(http://acoutts.com/a1ex/7O4C8804_before.jpg) (http://acoutts.com/a1ex/7O4C8804_after.jpg)

For grading: I've developed the DNG at +4 EV ... +12 EV in 1 EV increments and ran the shots through enfuse. The image was quite underexposed, next time try using ETTR ;)

Just for fun: here's how it would have looked at ISO 100. Good luck denoising that.
(http://acoutts.com/a1ex/7O4C8804_dark.jpg)

For higher-res crops, download the CR2 and grade it yourself. This update was about color cast in shadows, not about fine detail.

I'll update it on the first page and in raw2dng when I'll solve the remaining issues in highlights.

@IliasG: you around? can you check this and see if you get better results?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on August 06, 2013, 03:35:29 AM
Small update for cr2hdr_exp.exe: better black level handling. This is only noticeable in extremely underexposed shots (e.g. when you need to push the shadows at +10 EV).

Before and after (CR2 from AdamTheBoy):
http://acoutts.com/a1ex/7O4C8804_before.jpg http://acoutts.com/a1ex/7O4C8804_after.jpg

For grading: I've developed the DNG at +4 EV ... +12 EV in 1 EV increments and ran the shots through enfuse. The image was quite underexposed, next time try using ETTR ;)

Just for fun: here's how it would have looked at ISO 100. Good luck denoising that.
http://acoutts.com/a1ex/7O4C8804_dark.jpg

For higher-res crops, download the CR2 and grade it yourself. This update was about color cast in shadows, not about fine detail.

I'll update it on the first page and in raw2dng when I'll solve the remaining issues in highlights.

@IliasG: you around? can you check this and see if you get better results?

Yep, around and just checked. Much better about color cast at darks, but at a cost. The noise is harsh and it's not points (easy to denoise) but like candles. And I think that there is too much black clipping.

As I see here the Black Level in the CR2 file is around 2032-2034 for the low ISO lines and 2062-2066 for the hi ISO lines. How do you normalize for this ?. (In the meantime I have to read again your pdf I suppose ..).
This shows clearly that Canon tries to normalize at the desired average of 2048 as BL but how exactly ??. Is it just a data sliding up or down or a multiplication ?.

It happens this shot is with f/1.8 so there should be data scaling (due to wide aperture) and the White Level must be around 16000 ... the the DNG's exif says 49600 !! ?? even lower than 4*13000 ...

In the case of white level I still vote for a Lookup table.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on August 06, 2013, 04:06:06 AM
@a1ex,

Took this dual ISO shot while i was testing burst mode. I know a res chart is not the best for testing dynamic range which is the benefit of using dual ISO but I figured the behavior of aliasing and moire can be better observed and tweaked in a repeatable test environment.

Just hoping it helps.

From dual iso shot 200 cam 1600 recovery. o=sharpening in acr

(http://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-ash3/1102430_354578578006266_222939689_o.jpg)

From normal cr2 same settings at 200 iso

(http://fbcdn-sphotos-d-a.akamaihd.net/hphotos-ak-prn2/1097275_354578574672933_1186918413_o.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on August 06, 2013, 07:40:03 AM
If dual iso can be ported to raw video in the 7D, it sure would be much better in handling moire and aliasing. :)

Here's a chart from burst mode compared to dual iso in my previous post.


(http://fbcdn-sphotos-f-a.akamaihd.net/hphotos-ak-prn2/1149779_354583941339063_760796991_o.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 06, 2013, 08:11:20 AM
@Ilias: I split the raw file in 4 sub-images (group by y mod 4) and bring the black level to 2048 for each one (scaling to keep the old white level and taking care of drift). Then I run the interpolation routines. At the end, I average the black level again (to compensate for what the interpolation might do to noise), and it still finds a few units (for this image, 3 units).

The output DNGs are 16 bit. Not sure if the extra 2 bits have anything useful in them or just rounding noise (but at least they open pretty fast in ufraw compared to CR2's).

@Ted: the result with resolution chart will vary greatly with exposure (so, for best results, you should be careful when exposing). Yes, the VAF will help a lot for video (and I'm sure it will work on 7D too).

In the mean time... do you have a VAF for 7D? if so, can you shoot the test chart in photo mode with dual ISO at some extreme setting? (where aliasing appears normally)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 06, 2013, 12:31:44 PM
I'll update it on the first page and in raw2dng when I'll solve the remaining issues in highlights.


Hi Alex is the raw2dng now updated on the first page?

Wondering if it would be possible to do audio with dual ISO active during video taking on my 5D Mark 3.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sletts02 on August 06, 2013, 01:12:07 PM
Is there any reason for no video examples?
Not being rude, just wondering if there is a technical reason  :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: NateVolk on August 06, 2013, 07:57:07 PM
Probably I'm doing it wrong, but I haven't gotten any good video samples yet....  I'll try again today :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ShootingStars on August 07, 2013, 04:39:49 AM
I can't tell the difference between this and HDR from the photos POV, not the technical aspect POV. Anyone mind telling me?  :D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 07, 2013, 08:07:49 AM
I can't tell the difference between this and HDR from the photos POV, not the technical aspect POV. Anyone mind telling me?  :D

http://www.magiclantern.fm/forum/index.php?topic=7139.0

Read the pdf embedded towards the very end of the very first post by A1ex.

Dual_ISO is HDR on steroids - literally. While the results may appear similar since it is bonding of different EV's as is the case with HDR. But that is where the similarity ends.

Conceptually, both come from different planets. For a good HDR yeild, you would need more a few photos  with the same output at different EV's (3 or 3+ is ideal). For Dual_ISO, all the action happens on the same picture but in alternating horizontal lines - each alternate bright horizontal line holds the highlights; and the underexposed ones hold the shadows (pulled up to a point where the overall picture is correctly exposed). To acheive this each line representing the highlights and shadows are drawn from two different ISO's (hence the name Dual_ISO).

The results are stunning. The camera easily handles RAW/ CR2 in Dual_ISO mode. The grain is superb and unlike the sensor noise you would get on a normal RAW frame process to pull up the shadows in post. The range you get is about DR (14-16bit) - basically a 3 EV stop improvement. This functionalilty works both on photo and video.   

My take on this is simple. For normal situations shoot normal. For abnormal situations shoot Dual_ISO.

Plus, the grain is something to die for.

Just look out for the grain on this link: http://www.flickr.com/photos/[email protected]/9449868520/
 
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ShootingStars on August 07, 2013, 07:43:04 PM
Thanks for the explanation!  :) Will go test it out in abnormal situations.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: xmd5a on August 07, 2013, 09:14:10 PM
I think it is better to cut 2 or 3 pixel lines/rows on top and left side.
(https://dl.dropboxusercontent.com/u/4624786/dual_iso_cut.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: optik on August 07, 2013, 10:22:17 PM
Is there a thread somewhere explaining how to do this through wine for us Mac users or maybe I am missing something im stuck  ???
I am using August 2 build from lourenco. I've also read that Son of Batch for Mac now supports dual ISO is that for video only?
Also I will be shooting a wedding this weekend and would love to use this feature , that being said the only thing that I have besides the Mac Pro is a laptop that has XP on it and every time I try to drag a file on the app it opens then closes.  :-\
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 07, 2013, 11:34:12 PM
Another update regarding aliased areas, for cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe). This time I've tried a very difficult shot - the CR2 from daancalo2013:

cr2hdr from first post:
(http://acoutts.com/a1ex/6X8A9415_before.jpg)

previous cr2hdr_exp:
(http://acoutts.com/a1ex/6X8A9415_after.jpg)

today's update:
(http://acoutts.com/a1ex/6X8A9415_after2.jpg)

Still not perfect, but I doubt it can get much better than this.

Pixel peeping is pretty hard, who'd like to run some tests to make sure it handles pretty much everything better than the older one?

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Rai2121 on August 07, 2013, 11:54:24 PM
Hello everyone,

I've been following all the recent developments with the ML team over the last few months, trying to read everything there is. It's a really exciting time for Canon DSLR owners and I want to give a huge thanks to all the developers for their hard work bring this to all of us common folk.

Here are some examples of a few Dual ISO shots that I've taken so far on my 7D. These are both 100/1600. I'm still trying to get the hang of it, but it is an amazing feature.

I'm using the cr2hdr.exe from A1ex's Reply #469

(http://i.imgur.com/HjYkh4q.jpg)
(http://i.imgur.com/BITsX1z.jpg)
(http://i.imgur.com/FqYRrkK.jpg)
(http://i.imgur.com/JDztUQI.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sletts02 on August 08, 2013, 12:04:18 AM
Itching itching itching to see video samples. My line of work is interior real estate video and if it means reliable high dynamic range to compensate for the interior and exterior then I am sold on getting a 5D mkiii  ;D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: weigertj on August 08, 2013, 01:15:02 AM
Hi a1ex,

Just an idea...
would it be possible or practical to emulate a graduated ND filter in a new dual ISO mode? Let's say the top 1/3 of the image is in ISO 100, the bottom 2/3 is in ISO 800/1600, etc. You could move up and down the transition line with the joystick and the transition could be hard or soft (variable). You could emulate even a reverse ND for sunset shooting.
This would be a great advantage for landscape photographers who use filters to reach the proper results. And in this case we wouldn't have to deal with resolution loss and aliasing, etc. Only the transition area could be more problematic by using alternating lines of ISOs as today. So basically we would narrow down the usage of the current dual ISO mode with alternating lines to a small horizontal segment of the image, while the other parts of the image would be in 2 different ISOs.
I am curious about your thoughts on this.

Great work, anyway!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sletts02 on August 08, 2013, 01:26:28 AM
Hi a1ex,

Just an idea...
would it be possible or practical to emulate a graduated ND filter in a new dual ISO mode? Let's say the top 1/3 of the image is in ISO 100, the bottom 2/3 is in ISO 800/1600, etc. You could move up and down the transition line with the joystick and the transition could be hard or soft. You could emulate even a reverse ND for sunset shooting.
This would be a great advantage for landscape photographers who use filters to reach the proper results. And in this case we wouldn't have to deal with any resolution loss and aliasing, etc. Only the (soft) transition are could be more problematic with using alternating lines of ISOs.  I am curious about your thoughts on this.

Great work, anyway!

That would be amazing!

On another note, I am tossing up between the 5D2 and 5D3 at the moment. Will the Dual ISO ever come to the 5D2?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: weigertj on August 08, 2013, 01:39:06 AM
That would be amazing!

On another note, I am tossing up between the 5D2 and 5D3 at the moment. Will the Dual ISO ever come to the 5D2?

Unfortunately only the 5D3 and 7D have the hardver for it, it's impossible on other models. Go for the 5D3!  ;)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on August 08, 2013, 01:45:11 AM
Hi a1ex,

Just an idea...
would it be possible or practical to emulate a graduated ND filter in a new dual ISO mode? Let's say the top 1/3 of the image is in ISO 100, the bottom 2/3 is in ISO 800/1600, etc. You could move up and down the transition line with the joystick and the transition could be hard or soft (variable). You could emulate even a reverse ND for sunset shooting.
This would be a great advantage for landscape photographers who use filters to reach the proper results. And in this case we wouldn't have to deal with resolution loss and aliasing, etc. Only the transition area could be more problematic by using alternating lines of ISOs as today. So basically we would narrow down the usage of the current dual ISO mode with alternating lines to a small horizontal segment of the image, while the other parts of the image would be in 2 different ISOs.
I am curious about your thoughts on this.

Great work, anyway!

its been asked already.

http://www.magiclantern.fm/forum/index.php?topic=6335.msg50620#msg50620
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sletts02 on August 08, 2013, 02:49:28 AM
Unfortunately only the 5D3 and 7D have the hardver for it, it's impossible on other models. Go for the 5D3!  ;)

Well that settles that. There was a 5D2 with battery grip/batteries for $1100AUD. But I think the extra $2200AUD will be worth this feature if it's stable and workable.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 08, 2013, 03:08:46 AM
Another update regarding aliased areas, for cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe)


Does the link above have the latest update?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: NateVolk on August 08, 2013, 03:17:44 AM
@optik re: running on xp

I don't know why, but to run on xp, you have to right click and open with. Otherwise it doesn't work. Also, we can't batch by dragging onto, makes processing a bunch of images a giant pain...
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on August 08, 2013, 03:18:54 AM
its been asked already.

http://www.magiclantern.fm/forum/index.php?topic=6335.msg50620#msg50620

It's a different question.

Instead of scanning alternate lines, scan the top half (1/3rd whatever) at ISO whatever, scan the rest at another ISO.  Make it graduated too ;)

a1ex mentioned that he is only tweaking a register (for dual-ISO).  There isn't enough hardware control (currently) to adjust the scanning procedure.

@Canon eos m
You could try downloading it and looking at the timestamps!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: NateVolk on August 08, 2013, 03:19:34 AM
Itching itching itching to see video samples. My line of work is interior real estate video and if it means reliable high dynamic range to compensate for the interior and exterior then I am sold on getting a 5D mkiii  ;D

Me too, if you find any good samples, post them up!  I haven't gotten it worth selling. Yet :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dubzeebass on August 08, 2013, 05:29:55 AM
Me too, if you find any good samples, post them up!  I haven't gotten it worth selling. Yet :)

Here you go! http://bit.ly/11NFGnh (http://bit.ly/11NFGnh)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: optik on August 08, 2013, 05:47:05 AM
@optik re: running on xp

I don't know why, but to run on xp, you have to right click and open with. Otherwise it doesn't work. Also, we can't batch by dragging onto, makes processing a bunch of images a giant pain...


I guess I do have the option to install and run boot camp Windows or should I just use something like parallels or VMware
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: eduperez on August 08, 2013, 11:38:56 AM
Hi a1ex,

Just an idea...
would it be possible or practical to emulate a graduated ND filter in a new dual ISO mode? Let's say the top 1/3 of the image is in ISO 100, the bottom 2/3 is in ISO 800/1600, etc. You could move up and down the transition line with the joystick and the transition could be hard or soft (variable). You could emulate even a reverse ND for sunset shooting.
This would be a great advantage for landscape photographers who use filters to reach the proper results. And in this case we wouldn't have to deal with resolution loss and aliasing, etc. Only the transition area could be more problematic by using alternating lines of ISOs as today. So basically we would narrow down the usage of the current dual ISO mode with alternating lines to a small horizontal segment of the image, while the other parts of the image would be in 2 different ISOs.
I am curious about your thoughts on this.

Great work, anyway!

You can always emulate a GND from a "properly" (no highlights blown out) photograph in the computer; problem is that raising the shadows produces too much noise. But now that you can have files with a massive dynamic range, it is probably worthy trying again. At the end of the day, the results are going to be roughly the same, and much more versatile.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 08, 2013, 12:30:31 PM
@Alex
Latest cr2hdr seems to be the best so far. I,m really impressed. Comparison as follows. These are crops to emphasize on defects and improvements.
Beta(first converter)
beta0.1(next one)
beta0.2(latest converter, also the best one)

beta
(http://s13.postimg.org/tsvftiz3r/beta.jpg)

beta01
(http://s13.postimg.org/xgb6zl7av/beta0_1.jpg)

beta0.2(compare the bottom of the sign)
(http://s13.postimg.org/cuwf7oppz/beta0_2.jpg)

beta
(http://s13.postimg.org/kjoo5o0tz/beta_b.jpg)

beta0.1
(http://s13.postimg.org/5qelynign/beta0_1b.jpg)

beta0.2(pay attention to the right arm)
(http://s13.postimg.org/omkaihkc7/beta0_2b.jpg)







Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: weigertj on August 08, 2013, 12:35:13 PM
You can always emulate a GND from a "properly" (no highlights blown out) photograph in the computer; problem is that raising the shadows produces too much noise. But now that you can have files with a massive dynamic range, it is probably worthy trying again. At the end of the day, the results are going to be roughly the same, and much more versatile.

I will definitely will try to shoot a sunset landscape in dual ISO. Expose for the sky in ISO 100 and bring up the shadows in post.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: hookah on August 08, 2013, 01:17:48 PM
Gooooddd work Alex !!!!!!!!!!!!!!!!!! 
I,m VERY impressed..
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Magic 7D on August 08, 2013, 02:02:13 PM
Wow A1ex! This is just getting better and better, so thankful for all your hard work. Noticed that the colours are a lot more vibrant with the latest version of cr2hdr too.

Now what I'm aobut to say is just dreaming..  :)
Imagine if we could controll every second line vertically too, wouldn't this make it even more powerfull, or maybe better yet in a bayer pattern with 3 different ISO values like 100, 800, and 1600 just imagining it would bring even more DR to the final image. Now as I said it's just imagination on my side and I don't think it's possible. And maybe it wouldn't even help.. but then again who thought dual ISO would be possible before A1ex revealed it to us. :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 08, 2013, 05:19:12 PM
Another update regarding aliased areas, for cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe).

Hi Alex,

Could you please advise what is wrong with my cr2hdr_exp.exe post processing.

The original RAW Dual_ISO DNG shot with my 5DM3 is just fine.
https://www.dropbox.com/s/m86dyhnrbtajhi5/_G1A7637.CR2

Then when I processed it on the latest cr2hdr_exp.exe, I see big huge white patches all over the picture.
https://www.dropbox.com/s/rf44z0tqy4sl4ub/_G1A7637.DNG

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 08, 2013, 05:29:25 PM
Ah, those tiny hot pixels.

It seems better with my latest copy (download the exe again), but there are still a bunch of them. I'll see what I can do, never seen that in my test samples.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 08, 2013, 05:46:42 PM
Grrreat!

Where can I get the latest version of ch2hdr.

Second, I downloaded enfuse after reading you used it for HDR type work. Any idea if enfuse will process CR2/
DNG files or will this require conversion first to TIFF/ JPEG?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: naturalsound on August 08, 2013, 05:52:26 PM

Pixel peeping is pretty hard, who'd like to run some tests to make sure it handles pretty much everything better than the older one?

I just realized that the new version works great removing the reddish shadows. Unfortunately there are some artifacts near highlights which may be caused by the anti-aliasing? Each highlight is surrounded by a ring of horizontal lines which are about three to four pixels long.
(http://www.naturalsound.de/MagicLantern/IMG_8485_exp.jpg)
I hosted the .cr2 at: http://www.naturalsound.de/MagicLantern/IMG_8485_ML.CR2 (http://www.naturalsound.de/MagicLantern/IMG_8485_ML.CR2)
(I may have to remove the link if traffic gets huge).

Update: This CR2 is of an EOS 7D

The last experimental version did not show the lines and rendered the highligts clearly. Of course there was the reddish shadow cast.

I really appreciate your work!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 08, 2013, 06:05:53 PM
@Nikon eos m: this was ISO 12800/400? Here, the hot pixels are grouped, and my algorithm assumes they are isolated. Try not going higher than 3200.

To download, lookup my earlier posts (it's the same link). To merge with enfuse, I use this: www.magiclantern.fm/forum/index.php?topic=7022

naturalsound: here, the ISO response seems highly nonlinear, so the algorithm has trouble matching the brightness. Probably caused by aliasing, not yet sure.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: naturalsound on August 08, 2013, 06:33:21 PM
Hi a1ex,

I forgot to mention the fact that this is an EOS 7D picture, but I guess you already noticed from the EXIF.
I have some additional RAW files in case you need another one.

Thank you for your fast response. I think it is a minor issue and will probably only happen in these very high DR situations. Maybe some pixels bleed because of the very high photon count and create the artifacts?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 08, 2013, 06:35:09 PM
Alex I call myself Canon eos m and not Nikon eos m - not that it matters, it is just that this a Canon forum and it is embarrassing as hell to have some one refer to me as a Nikon fellow :-)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on August 08, 2013, 06:50:18 PM
@a1ex


using aug 3 cr2hdr_exp
(https://fbcdn-sphotos-h-a.akamaihd.net/hphotos-ak-prn2/1093880_355572661240191_1837277124_o.jpg)

using aug 5 cr2hdr
(https://fbcdn-sphotos-b-a.akamaihd.net/hphotos-ak-frc3/1090950_355572857906838_595474699_o.jpg)

using aug 7 cr2hdr
(https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-ash3/1091175_355572994573491_396787713_o.jpg)

great improvement with color aliasing.

edit: used 7D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 08, 2013, 06:59:08 PM
Quote
Alex I call myself Canon eos m and not Nikon eos m

When using dual ISO it's like having a Nikon sensor ;)

@ted: can you upload the CR2 for this shot?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on August 08, 2013, 07:10:06 PM
@a1ex,

compressed it and sent it to your email.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 08, 2013, 07:16:06 PM
@Nikon eos m: this was ISO 12800/400? Here, the hot pixels are grouped, and my algorithm assumes they are isolated. Try not going higher than 3200.

To download, lookup my earlier posts (it's the same link). To merge with enfuse, I use this: www.magiclantern.fm/forum/index.php?topic=7022

naturalsound: here, the ISO response seems highly nonlinear, so the algorithm has trouble matching the brightness. Probably caused by aliasing, not yet sure.

Alex got myself the latest cr2hdr.exe from page 1 of this thread. Like you said, magically, almost all hot pixels gone (at least the big and bothersome ones).

Same picture processed with the latest cr2hdr.exe
https://www.dropbox.com/s/vys1cux6yi7iege/kevin%27s%20workshop.JPG
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 08, 2013, 07:19:10 PM
That's the old converter, which does not check whether the hot pixels are isolated or not. So it just corrects them blindly.

The problem is that it's too easy to mistake the hot pixels for aliasing. So, for now, I don't have a fix that solves both problems.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 08, 2013, 07:22:43 PM
That's the old converter, which does not check whether the hot pixels are isolated or not. So it just corrects them blindly.


Do you mean the one on page 1 is the 'old convertor'? If it is, where do I get the latest one?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 08, 2013, 07:24:30 PM
Another update regarding aliased areas, for cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: AdamTheBoy on August 08, 2013, 08:02:54 PM
Today's update keeps freezing as soon as it reaches the "looking for hot pixels" point.  Am I the only one with this issue? It seems to happen regardless of which image I try.  Thanks again A1ex for all the awesome work, it's exciting to see it get better and better.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 08, 2013, 08:12:41 PM
Can you upload a CR2? Shouldn't happen.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 08, 2013, 08:16:10 PM


That is the one I used the first time and ended with a lot of hot pixels. Will try to deploy it once again for another few pictures and see what happens! It is already 2:00 am here in Singapore. Good night!

BTW saw this awesome video today: http://nofilmschool.com/2013/07/magic-lantern-firmware-canon-linuxtag/

You folks rock!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 08, 2013, 08:22:30 PM


Alex, re-checked. It is only on this the photo I posted and only if I use the ch2hdr_exp.exe and not when I am using the ch2hdr.exe taken from the first page of this thread.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: AdamTheBoy on August 08, 2013, 08:28:01 PM
Sure thing, here's one that I tried https://www.dropbox.com/s/4r0pfxn9kaepl87/7O4C8813.CR2

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Alex Roman on August 08, 2013, 09:49:03 PM
I'm a bit confused. It seems that the last (7Aug) A1ex build has a huge step forward regarding antialiasing (which is really awesome BTW) but... how does this translates into video performance? Does it has huge advantage as well? Video samples anyone?

Thnx.

A1ex: i see, thank you. Looking so forward for it.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 08, 2013, 10:30:53 PM
It's not yet updated for video.

Updated the experimental cr2hdr (http://acoutts.com/a1ex/cr2hdr_exp.exe) again, this time fixed the brightness matching on 7D (test shot from naturalsound - the sensor response is linear, it was just my algorithm too sensitive to noise) and also fixed some buffer overflow errors.

I'm also on the way to fixing the black banding from 7D at ISO 100 (like this (http://www.magiclantern.fm/forum/index.php?topic=7402.msg64607#msg64607)). How many of you did you notice it? Do you have more samples? Did you notice similar problems on other cameras?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: naturalsound on August 08, 2013, 11:11:29 PM
Great work - Thank you!

I would not even have dreamed my 7D could deliver that clean shadows!
(http://www.naturalsound.de/MagicLantern/fireworks.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: IliasG on August 08, 2013, 11:42:11 PM

I'm also on the way to fixing the black banding from 7D at ISO 100 (like this (http://www.magiclantern.fm/forum/index.php?topic=7402.msg64607#msg64607)). How many of you did you notice it? Do you have more samples? Did you notice similar problems on other cameras?

I hope this helps a bit .. http://www.dpreview.com/forums/post/51945370

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 09, 2013, 12:12:30 AM
Maybe not the best test when it comes to dual iso but a little comparison between 3x cropmode and regular raw using a corrected 8mm peleng and a 20-35mm wideangle shot with dual iso.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: sletts02 on August 09, 2013, 01:21:59 AM
Maybe not the best test when it comes to dual iso but a little comparison between 3x cropmode and regular raw using a corrected 8mm peleng and a 20-35mm wideangle shot with dual iso.


Looks promising! Unfortunate that you didn't have a contrasty blue sky to get a good feel for the range. If possible, would you be able to do a quick test where you shoot the interior of a room with windows so we can see how it handles the typically blown highlights from windows? That would be ultra helpful please and thank you  :-*

EDIT: Just what I was after - Thanks heaps! I'm buying the 5D3 immediately now haha. It's just a shame that shipping for CF cards will take a week or two from the states.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 09, 2013, 02:29:00 AM
Nice balcony and neighborhood  :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Greg on August 09, 2013, 03:10:29 AM
5D III:
5796 x 3870 = 22,4MPx
3870 / 2 = 1935
2892 x 1935 = 5,6MPx Ultra High Quality ;D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 09, 2013, 08:01:24 AM
Well, it's not that easy.

In many practical cases you won't notice the detail loss (this happens when the aliased areas are out of focus backgrounds, or when the missing data is guessed correctly).

In some extreme cases (brick walls maybe?) aliasing might be so bad that it will be visible at 720x480 too. But the same is true about 1080p video with line skipping.

Anyway. The latest updates have a heuristic that I believe it puts a limit of how bad the aliasing can get (it was the key to solving the palm tree shot):
Code: [Select]
                /* if the brighter copy is overexposed, the guessed pixel for sure has higher brightness */
                fullres[x + y*w] = f < white_darkened ? f : MAX(f, dark[x + y*w]);

Anyone with good math skills can come up with a proof?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: makpat on August 09, 2013, 08:32:04 AM
Hi,

I was playing a bit around Dual ISO with my 7D and took few shots.
Made DNG with cr2HDR. But somehow dont know the workflow ahead as if I open the dng in ACR it shows very dark like 100 iso shots.

Sharing my CR2 as well as DNG folder link here if someone can guide how to achieve the finest dual iso process.
https://www.dropbox.com/sh/xz77sv2bfow98s4/dZZmUIMnkZ
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on August 09, 2013, 09:06:50 AM
@A1ex.

Since you have the cr2 of the test I did, you can check yourself what I noticed on the latest experimental you posted.
I compared it with the last 3 prior versions and I noticed that maybe your "hot pixel fix" might be getting too aggressive already as
I am now beginning to see dark spots coming out in the shadow areas which I havent seen in the previous cr2hdr builds. Check in the area I circled.
This pic I'm posting is already compressed so maybe you can verify with the cr2 I sent.

(https://fbcdn-sphotos-g-a.akamaihd.net/hphotos-ak-ash3/1149377_355793761218081_1129444886_o.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 09, 2013, 09:10:53 AM
These are cold pixels and they are no longer corrected. I'll take a look.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 09, 2013, 05:08:12 PM
I think I've got it working on 5D2: https://bitbucket.org/hudson/magic-lantern/commits/b3999097eed7

Also, since the experimental converter handles almost all cases better than the old one, I've updated the converter from the first post (so cr2hdr.exe now has the latest algorithms).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: batca.dragos on August 09, 2013, 05:26:18 PM
So does the 5D2 have a 8-channel readout after all a1ex ?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 09, 2013, 05:34:10 PM
It doesn't need an 8-channel readout, but two separate analog amplifiers. The paper says:

Quote
Unfortunately, most other Canon cameras follow a different pattern for the CMOS #0 register,
without any obviously duplicate fields.
...
We can now speculate that cameras with 8-channel readout have two separate amplifier circuits,
and you can program the ISO for each one separately.

Translation: at that time I knew how to program the cameras with 8-channel readout (because it was obvious) and I had no idea how to program the others, or whether they will work or not. Now I've discovered the missing bit from 5D2 and it works.

I still have no idea about the other cameras. I'd say the 6D has 99% chances to work, since 1% almost got it working, but I did not check the new trick on any other camera. Follow the ADTG thread (http://www.magiclantern.fm/forum/index.php?topic=6751) and give it a try.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: batca.dragos on August 09, 2013, 05:36:08 PM
Understood now  ;D. Congrats on the find.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on August 09, 2013, 05:47:28 PM
A1ex,

is there a compiled 5d2 dual iso module I can test?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ilguercio on August 09, 2013, 05:51:11 PM
It doesn't need an 8-channel readout, but two separate analog amplifiers. The paper says:

Translation: at that time I knew how to program the cameras with 8-channel readout (because it was obvious) and I had no idea how to program the others, or whether they will work or not. Now I've discovered the missing bit from 5D2 and it works.

I still have no idea about the other cameras. I'd say the 6D has 99% chances to work, since 1% almost got it working, but I did not check the new trick on any other camera. Follow the ADTG thread (http://www.magiclantern.fm/forum/index.php?topic=6751) and give it a try.
50D has got 2, from my pictures.
Right?
http://chdk.setepontos.com/index.php?topic=6161.180
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on August 09, 2013, 05:54:36 PM
50D, photo mode:

Code: [Select]
/* 00:00:04.078841  */ CMOS
/* 00:00:04.078911  */     00 0004 404B548E

/* 00:00:14.214297  */ CMOS
/* 00:00:14.214376  */     00 0094 404B549C

/* 00:00:26.551046  */ CMOS
/* 00:00:26.551116  */     00 01B4 404B54AA

/* 00:00:47.349123  */ CMOS
/* 00:00:47.349194  */     00 016C 404B54C6


Code: [Select]
       PHOTO_CMOS_ISO_START = 0x404B548E; // CMOS register 0000 - for photo mode, ISO 100
        PHOTO_CMOS_ISO_COUNT =          5; // from ISO 100 to 12800
        PHOTO_CMOS_ISO_SIZE  =         14; // distance between ISO 100 and ISO 200 addresses, in bytes

        CMOS_ISO_BITS = 3;
        CMOS_FLAG_BITS = 2;
        CMOS_EXPECTED_FLAG = 0;
is_5d2 = 1;   
 

This produces Iso1 != iso2 right after its set. Using size of 12 causes err reg not 0. Setting cmos expected flag like 5DII errors on the flag.

*HiNT. Seems the addresses when cmos updates all regs together are not what we're looking for... its when reg 0 is updated by itself.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 09, 2013, 06:03:31 PM
For 50D, these are the values for ISO 200/400/800/1600?

First we need to know what bits do what. You can set the values manually, e.g. *(uint16_t*)0x404B548E = whatever. Only after you are sure about the meaning of each bit (at least the two ISO fields), you can configure the constants in dual_iso and see the errors.

For experimenting, just change the memory values manually.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on August 09, 2013, 06:29:22 PM
On 50D they change on the middle ISOs.. like 160, 320, etc. I think somewhere at the begining of the thread I did what went with what. 04 is 100-160 tho. I went 160, 320, 640, 1250. I think I didn't do the last one.

4 I think is only 100
94 is        100 10 100
1B4         110 110 100
16C         101 101 100
So which ones are likely to be ISO.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Elmeri on August 09, 2013, 09:51:42 PM
could it be
004          000 000 100
094         010 010 100
1B4         110 110 100
16C         101 101 100
so is it       iso iso xxx
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on August 09, 2013, 10:49:40 PM
Is it ok to report tests and bugs on dual iso for 5d2 here? or should I start another thread for 5d2 dual iso?

did some tests on the dual ISO and heres some quirks.

1. after activating dual iso, i can take 1 shot with it then the following shots are normal cr2. I did not turn on alternate.
I sometimes take the card out and back before I can take a dual iso shot. Sometimes reboot cam will work sometimes not.
or I will download the cr2 via usb cable. after downloading i will take another shot, they will not be in dual iso. looking at ML menu dual iso is still at ON.

2. changing the recovery ISO has a bug, after entering submenu i can change it to 3200 for example. exiting sub menu I can see the the dual iso reads 100/1600 still.

I have no problems using dual iso with my 7D.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on August 10, 2013, 03:45:11 AM
6D photos, all ISOs starting with 100-128k.. there is no 66?

03 0000 40450E08
03 0011 40450E1A
03 0022 40450E2C
03 0033 40450E3E
03 0044 40450E50
03 0055 40450E62
03 0077 40450E74

100 0 1
100 0 10
110 0 11
100 0 100
101 0 101
111 0 111


Seems like ISO 0 ISO

50D seems ISO ISO 100

so how to set up flag bits, etc for this.. also on 50D the reg address never changes in LV.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Audionut on August 10, 2013, 07:42:46 AM
Also, since the experimental converter handles almost all cases better than the old one, I've updated the converter from the first post (so cr2hdr.exe now has the latest algorithms).

The cr2hdr in the first post is timestamped 30/07/13?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Danne on August 10, 2013, 03:53:21 PM
Hi @Alex! Tried some dualiso and some birds. The converter seems to have trouble converting certain lighting conitions such bright backgrounds with smaller subjects lika flying duck for instance? The first converter managed to convert this but not the latest

https://docs.google.com/file/d/0B4tCJMlOYfirWmVDd1pZVlYwU2M/edit?usp=sharing

Is the conversion dependent on motif? Have some pictures of an eagle , rather tiny in the original but could be cropped after conversion if converted. Seem the next latest of the converters could do some of the files the other two converters can,t handle but not all of them.

*update. Added the exiftool, didn,t think it was necessary but seems to convert ok now, still not able to convert linked example though.
Another strange thing is that the converter creates an extra dng_original file when exiftool included? Exact same size as the converted dng-file.
*update2. The converter gives random results. Sometimes it doesn,t recognize the cr2,s as interlace iso, but retrying, suddenly it works?

(http://s15.postimg.org/ni9ccddsr/extra.png)

Thanks
//D
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 11, 2013, 04:13:57 AM
Is there crop mode functionality on the 5DM III shooting video?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: mauerfuchs on August 11, 2013, 02:25:58 PM
Whats the difference to just use the RAW function in 50fps mode in combination with the older HDR function that records two different frames. I mean...wouldn't it help the antialiasing a bit? The motionblur now may be the new old problem...
i got a 5d mark III.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: swinxx on August 11, 2013, 02:43:44 PM
hello ml team!

is it possible to name the dual iso shots automatically (e.g. DI_99A6430.CR2 .. because in the mac finder it is not possible to see which cr files are dual isos and which are not.. ?

a really satisfied ml user..  :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 11, 2013, 06:56:23 PM
Wow, not a single sample from 5D2?!

See you in roughly 2 weeks (maybe earlier if I get net access).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on August 11, 2013, 07:32:20 PM
6D actually looks doable for LV and photo, maybe need to try to set dual ISO manually. I'm just not sure if its

ISO1 0 ISO2 in ISO1 reg address
or

ISO2 0 ISO1 in ISO2 address, etc.

I guess I have to experiment.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 11, 2013, 07:51:56 PM
What happens if you set *(uint16_t*)40450E08 = 0x50 or 0x05?

I'd say ISO_BITS=4, FLAG_BITS=0.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on August 11, 2013, 08:00:22 PM
I just took off some of the checks and got some interesting raw shots. It looks striped. Looking at the CR2s and then will just try to set it like that.

I kept getting error 1 and reg = 3... then got error 5 that prev iso was less than 10.


Here is one from 6D.
http://www.qfpost.com/file/d?g=KSwEmqWs8 (http://www.qfpost.com/file/d?g=KSwEmqWs8)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: robinlee on August 11, 2013, 08:01:25 PM
Hi,

When I do the conversion the exiftool always say didn't work for some reason  ???
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on August 11, 2013, 08:17:37 PM
Ok, after conversion the pic looks normal. So now I'll try setting the 0x50, 0x05, etc. It *looks* like its working but I'd need some scene to obviously test on. Both in LV and photo it does the same thing. Opening original CR2 is filled with black horizontal lines... is this how it should be?

0x50 turns everything solid black.... bummer, i should try 0x70 but from filming in lv a second ago it does appear to be working. The shadows lit up when I turned on dual iso... of course have the undulating black lines in the preview. The way it could be not working if its just set  iso 1600 and the black really is just black.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 11, 2013, 08:24:58 PM
Yep, it's working.

My dcraw doesn't know the black border sizes for 6D; maybe fix it and send a patch to Dave Coffin? or maybe the latest one already fixed it.

Some checks may not be needed; they are mostly to make sure the address doesn't change over time (and if it does, to prevent unwanted side effects).
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on August 11, 2013, 08:26:47 PM
I think the latest latest dcraw it was fixed. First version of it and raw-therapee I could see the black borders and before the meta data method counted them like that.

So bitbucket took the upload and never listed it, now upload box is gone again... sdlfjasdljfklasjdfklasdjflasjdfl


http://www.filedropper.com/6ddualiso-fullpack
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 11, 2013, 08:54:47 PM
Quote
2. changing the recovery ISO has a bug, after entering submenu i can change it to 3200 for example. exiting sub menu I can see the the dual iso reads 100/1600 still.

This is not a bug, max analog ISO in 5D2 seems to be 1600. Maybe I should remove the higher ISOs from menu.

Quote
The converter seems to have trouble converting certain lighting conitions such bright backgrounds with smaller subjects lika flying duck for instance?

I'm afraid it needs robust linear regression in estimate_iso (plain least squares is too sensitive to outliers). Can you find a simple method for doing that? Exif info won't do, I need precise fitting.

Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on August 11, 2013, 09:01:12 PM
So for 50D how to handle the binary 100 at the end? Is that the CMOS expected flag? I think 100 is 4dec
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 11, 2013, 09:06:04 PM
Yeah, the post from Elmeri makes sense. 3 bits for flag and 3 for ISO?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on August 11, 2013, 09:11:22 PM
It doesn't do ISO FLAG ISO FLAG that way? It will work just 3, 3 and expected flag of 4?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: a1ex on August 11, 2013, 09:18:39 PM
The current algorithm only knows ISO ISO FLAG, and the 50D seems to follow this pattern. The flag is not really used (it's just preserved and used for the internal checks).

The exact meaning of the ISO field doesn't really matter (it's copied from the higher ISO to the lower ISO). So far it had ascending values. As long as it repeats, it should work.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: oddname on August 11, 2013, 09:58:13 PM
Will the 5D2 be able to do dual-iso raw video like the mk3 or just too old hardware?
If I read right that the mk3 can do that :)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Francis on August 11, 2013, 10:09:28 PM
Working on the 5D2 fine for me. I'm trying to figure out how to put it to the test. I'll post some samples in the coming days.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: xNiNELiVES on August 11, 2013, 10:41:40 PM
Working on the 5D2 fine for me. I'm trying to figure out how to put it to the test. I'll post some samples in the coming days.

Will test any builds posted for 5D2. I suppose this thread's opening post should be updated, along with the title?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: 1% on August 11, 2013, 10:47:19 PM
5DII has multiple regs in LV? 50D probably won't work in LV as its just one REG, I can try to set the spacing to 0.

It doesn't do anything in LV so far but photo works.

cr2hdr doesn't take wildcards in windows but on linux its working, odd.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jgerstel on August 12, 2013, 04:49:11 AM
Hi, just installed dual iso andy600 build on my 50D, but can not find any CR2HDR tool for Mac OSX. Is there any CR2HDR or alternative tool for mac?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: arturochu on August 12, 2013, 05:32:50 AM
Hey Alex, when is the new raw2dng with new algorithm for dual isos coming? just shot around 140 gb pure dual iso raw video crazy stuff, haha.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Francis on August 12, 2013, 07:09:58 AM
Building the cr2hdr tool in a mac environment is super easy. Just follow the instructions for downloading the source repository and a prebuilt tool-chain in the 'getting started with development' thread. Then make in the dual_iso modules folder. It builds both the module and cr2hdr.

Here is the binary built as of Aug 11th. cr2hdr for Mac (http://avlmotofoto.com/cr2hdr)

This is a command line version. I would suggest making a symbolic link to where ever you save it in /usr/bin so you can run it in whatever folder you save your images. You also are going to need to install dcraw. I installed it using MacPorts.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ted ramasola on August 12, 2013, 08:15:12 AM
@a1ex

did a test similar to the 7D this one with the 5D2.

100_1600

(https://fbcdn-sphotos-d-a.akamaihd.net/hphotos-ak-ash3/1150633_357070077757116_1735920732_o.jpg)

100 iso only no adjustments.

(https://fbcdn-sphotos-h-a.akamaihd.net/hphotos-ak-prn2/1102615_357072681090189_124992235_o.jpg)

tell me if you need the cr2 files.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: daancalo2013 on August 12, 2013, 08:37:21 AM
Hi Alex look what happens in many of my photos
Raw http://we.tl/bcZHyu0dW1
http://www.flickr.com/photos/[email protected]/9490490087/lightbox
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: dlrpgmsvc on August 12, 2013, 09:23:22 AM
5DII has multiple regs in LV? 50D probably won't work in LV as its just one REG, I can try to set the spacing to 0.

It doesn't do anything in LV so far but photo works.

cr2hdr doesn't take wildcards in windows but on linux its working, odd.

Many compliments for this astounding achievement 1% ! For the LV problem, if it can be of help, I can dig inside rom to find more ADTG regs, if this route has not been squeezed dry yet...
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: jgerstel on August 12, 2013, 12:48:35 PM
Building the cr2hdr tool in a mac environment is super easy. Just follow the instructions for downloading the source repository and a prebuilt tool-chain in the 'getting started with development' thread. Then make in the dual_iso modules folder. It builds both the module and cr2hdr.

Here is the binary built as of Aug 11th. cr2hdr for Mac (http://avlmotofoto.com/cr2hdr)

This is a command line version. I would suggest making a symbolic link to where ever you save it in /usr/bin so you can run it in whatever folder you save your images. You also are going to need to install dcraw. I installed it using MacPorts.

Thanks a lot, but I get error:

studios-mac-pro:canon studio$ ./cr2hdr IMG_0155.CR2

Input file     : IMG_0155.CR2
sh: dcraw: command not found

got Xcode and gcc tools, I will try to compile on local
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: oscaroo on August 12, 2013, 03:17:37 PM
WEeeeeeeeeee!
Works on the 6D
*happy.

Thank you all.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Francis on August 12, 2013, 04:00:35 PM
Thanks a lot, but I get error:

studios-mac-pro:canon studio$ ./cr2hdr IMG_0155.CR2

Input file     : IMG_0155.CR2
sh: dcraw: command not found

got Xcode and gcc tools, I will try to compile on local

You need dcraw. As I said, I used MacPorts to install it. Macports (http://www.macports.org/) is a command line based package installation system for OSX that is similar to Linux package systems like APT.
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: MikeinCusco on August 12, 2013, 04:38:53 PM
Any chance of dual ISO for the 50D?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Silkway on August 12, 2013, 04:50:00 PM
Hi
i just installed the latest build from Andy600 tragic lantern for dual ISO on canon 50d
and this is what i got

is this image is as it supposed to be?
the horizontal zebra lines?

or am i doing something wrong?

thank you anybody for quick answer!
(http://i024.radikal.ru/1308/46/ac18eeefa2de.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ilguercio on August 12, 2013, 04:57:46 PM
You haven't read how the Dual ISO feature works and how to process it, right?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Silkway on August 12, 2013, 05:01:47 PM
oh yeah sorry i have read the first post

i downloaded that 3 files tools

and i try to drag and drop CR2 file on the cr2hdr.exe icon

but the command dial says "it seems like it is not interlaced ISO"

and shuts down...

here what i got
(http://s019.radikal.ru/i612/1308/e9/407824467458.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: ilguercio on August 12, 2013, 05:17:01 PM
Conversion works for me with my 6D files but i am not sure about the en result.
@A1ex:does the cr2hdr need to be changed with data from other cameras?
I would love to upload the dual iso file now but it's not possible at the moment.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Silkway on August 12, 2013, 05:32:56 PM

modified

ah no it works fine! it just doesnt pick up underexposed pictures.
(http://s003.radikal.ru/i204/1308/0e/2e24f283cdad.jpg)
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Andy600 on August 12, 2013, 05:51:25 PM
oh yeah sorry i have read the first post

i downloaded that 3 files tools

and i try to drag and drop CR2 file on the cr2hdr.exe icon

but the command dial says "it seems like it is not interlaced ISO"

and shuts down...

[spoiler]here what i got
(http://s019.radikal.ru/i612/1308/e9/407824467458.jpg)[/spoiler]

Did you shoot in live view? If so, it doesn't work. Use photo mode only.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Silkway on August 12, 2013, 06:03:37 PM
Did you shoot in live view? If so, it doesn't work. Use photo mode only.

no dear sir Andy600

i shot via viewfinder of canon 50d

but it seems like there is not enough info for processing on too dark images. thats a reason it didnt convert. but other images who shot on bright areas with good highlights it is just perfect working!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on August 12, 2013, 06:27:40 PM
Don,t shoot in too extreme situations. I got the same problems when shooting birds in very bright areas such as skies. Otherwise it,s fine
Title: Re: Dual ISO - massive dynamic range improvement
Post by: dubzeebass on August 12, 2013, 09:02:32 PM
Can I please request that the EXE for CR2HDR be updated in the OP?  Seems like it's an earlier build.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: akumiszcza on August 13, 2013, 12:46:39 AM
I've noticed green cast in shadows and highlights after converting dual iso cr2 into dgn on Canon 50D.

Normal case:
(http://i.imgur.com/4LAE0J5.jpg)

without Dual ISO:
(http://i.imgur.com/WiFTupz.jpg)

Extreme case:
cr2hdr.exe:
(http://i.imgur.com/htMTg7g.jpg)
cr2hdr_exp.exe:
(http://i.imgur.com/8MSQ4yg.jpg)

Is there some trick to remove green cast in ACR (in normal, not "extreme" cases at least)? Split toning won't do the trick. Setting WB sets it either for highlights and shadows or for middle parts.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 13, 2013, 01:21:45 AM
Yea, I had some problem with green cast on 50D. I think some of the problem might be that 50D uses iso 100, 320, 640, etc NOT 100, 200, 400, 800, etc. Have to check it out and port to 600D and *maybe* eosM if the module loads with CMOS logging enabled.

Title: Re: Dual ISO - massive dynamic range improvement
Post by: REDcineX on August 13, 2013, 01:30:30 AM
Hi all.

Really new to this and I have a "dumb" question. I'm on Mac (Mountain Lion) I have the source code and pre-built toolchain. Could someone point me where to go from there. Not quite sure what to do with it to use my 5D2 .cr2 files and get cr2hdr working.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Francis on August 13, 2013, 02:27:35 AM
Hi all.

Really new to this and I have a "dumb" question. I'm on Mac (Mountain Lion) I have the source code and pre-built toolchain. Could someone point me where to go from there. Not quite sure what to do with it to use my 5D2 .cr2 files and get cr2hdr working.

I just wrote up something http://www.magiclantern.fm/forum/index.php?topic=7645
Title: Re: Dual ISO - massive dynamic range improvement
Post by: REDcineX on August 13, 2013, 03:12:20 AM
Error: Port hg not found
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Silkway on August 13, 2013, 03:17:42 AM
oh yeah indeed!
there is a green stuff going on on canon 50d dual iso...

and i dont know if it is suitable hmm...

in some way its not so bad of course

but would be nice to see the solution...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Francis on August 13, 2013, 03:34:33 AM
Error: Port hg not found

My mistake.

port install mercurial
Title: Re: Dual ISO - massive dynamic range improvement
Post by: SenorClean on August 13, 2013, 03:48:35 AM
Working great on my 6D - well done all!

If you look closely in extreme areas you can see some fringing/jaggies/moire - see my 100% crop here -> http://img19.imageshack.us/img19/3097/tbje.jpg

BUT it is a small price to pay for the results.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ilguercio on August 13, 2013, 03:57:18 AM
Which ISO did you set in camera and which in ML menu?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: REDcineX on August 13, 2013, 04:17:48 AM
Aww..

"-bash: cr2hdr: command not found"

To back track a little further. When I do this line and of course the same line for raw2dng:
"Type 'make' then 'sudo ln -s cr2hdr /usr/bin/cr2hdr'."

Terminal says: "make: *** No targets specified and no makefile found. Stop."
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 13, 2013, 04:24:38 AM
Download/build it. and run ./cr2hdr

600D photo mode is working. 600D has real iso 3200. LV too.

I can make inside the raw_rec and dual_iso module folder and modules + binaries come out.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: REDcineX on August 13, 2013, 04:49:51 AM
The only other thing I might have done wrong was download the toolchain (mac version), renamed it gcc, and placed it inside the magic-lantern folder in Downloads. I used all your commands, Francis, after that to keep it consistent.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Francis on August 13, 2013, 07:04:45 AM
Sounds like you weren't in the right folder to build cr2hdr. You have to be in the directory containing the dual ISO code. It is in magic-lantern/modules/dual_iso
Title: Re: Dual ISO - massive dynamic range improvement
Post by: REDcineX on August 13, 2013, 07:29:56 AM
When I type "./magic-lantern/modules/dual_iso"
Terminal replies "-bash: ./magic-lantern/modules/dual_iso: is a directory"

I type "make" after cd to /dual_iso
Terminal replies "[ GCC      ]   cr2hdr
[ GCC      ]   cr2hdr
[ GCC      ]   cr2hdr
~/Downloads/magic-lantern/gcc/bin/arm-none-eabi-gcc-4.7.3  -Os -mthumb-interwork -march=armv5te -D__ARM__ -Wp,-MMD,./dual_iso.d -Wp,-MT,dual_iso.o -nostdlib -fomit-frame-pointer -fno-strict-aliasing -I../../include  -Wall -W -Wno-unused-parameter -Wno-unused-function -Wno-implicit-function-declaration -Wno-missing-field-initializers -Wno-format -Wdouble-promotion -ffast-math -fno-trapping-math -fsingle-precision-constant -fshort-double -std=gnu99 -Winline -I../../platform/all -I../../platform/all/include -I../../src -I../../picoc   -DMODULE -DMODULE_NAME=dual_iso    -c -o dual_iso.o dual_iso.c
/bin/sh: /Users/Admin/Downloads/magic-lantern/gcc/bin/arm-none-eabi-gcc-4.7.3: No such file or directory

I enter "sudo ln -s cr2hdr /usr/bin/cr2hdr"
Terminal responds "ln: /usr/bin/cr2hdr: File exists"

Title: Re: Dual ISO - massive dynamic range improvement
Post by: sletts02 on August 13, 2013, 10:28:46 AM
I know previously we said that 'auto' video mode would not be possible with dual iso as iso changes were needed.

Is it possible to have dual iso with 'auto video' exposure mode, but just changing shutter-speed/aperture? This would be a massive, massive help for my line of work!

In other news, my 5D3 arrived today! Just waiting for Transcend 1000x to arrive on Monday now.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 13, 2013, 10:31:15 AM
/bin/sh: /Users/Admin/Downloads/magic-lantern/gcc/bin/arm-none-eabi-gcc-4.7.3: No such file or directory

You need to make sure you have the arm toolchain https://launchpad.net/gcc-arm-embedded
And edit your makefile to point to the location where you store it. 

Take further discussion to a developing help thread please.  Here are some threads that should help.
http://www.magiclantern.fm/forum/index.php?topic=991.0
http://www.magiclantern.fm/forum/index.php?topic=6783.0
http://www.magiclantern.fm/forum/index.php?topic=6425.0
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 13, 2013, 05:59:48 PM
600D, its not perfect. I saw some artifacts.

http://www.filedropper.com/600ddualiso
Title: Re: Dual ISO - massive dynamic range improvement
Post by: CFP on August 13, 2013, 07:58:35 PM
600D, its not perfect. I saw some artifacts.

http://www.filedropper.com/600ddualiso
I just tried it on my 600D and it works fine! Just amazing!

It's so impressive to see how much progress has been made in the last time. Such insane stuff.
First 14-Bit RAW video and now almost 3 steps more dynamic range! What's next? :o
This really is MAGIC!

Thank you very much for posting this build. It is freaking awesome that Magic Lantern gets developed since 2009 and still it looks like the biggest treasures inside the cameras aren't even found yet. Who would have thought that any DSLR can handle RAW video? Even 2K RAW video! Or that a Canon camera could get the dynamic range of a Nikon sensor?

I am really curious to see how this firmware add-on will change in the future. It is by far the coolest thing you can download from the internet.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mk11174 on August 13, 2013, 08:19:25 PM
600D, its not perfect. I saw some artifacts.

http://www.filedropper.com/600ddualiso
If this works on the 600D doesnt it mean the 550D should have the ability as well?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: sletts02 on August 13, 2013, 09:39:23 PM
First 14-Bit RAW video and now almost 3 steps more dynamic range! What's next? :o
Rumor has it they have cracked 3D Video, should be out in a week or two.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: manniac on August 13, 2013, 11:25:40 PM
I understand, that dual iso is for RAW recording only.
Is there a chance it will become available for H264 encoded recordings later, or is this not possible?

The raw video workflow is really time consuming, and if dual iso was possible with H264 directly in camera, this would be a fantastic time saver..
Title: Re: Dual ISO - massive dynamic range improvement
Post by: dlrpgmsvc on August 13, 2013, 11:37:39 PM
If this works on the 600D doesnt it mean the 550D should have the ability as well?

Read the last 1% quote on our old 550D raw video thread: http://www.magiclantern.fm/forum/index.php?topic=5582.new#new
It seems possible... we must investigate asap !  ;D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jgerstel on August 13, 2013, 11:54:15 PM
Hi,

For me it is also new, but got dual iso tool working. Please use next guide for general setup:

Also install XCode and command line tools:
Xcode menu > Preferences > Downloads > choose "Command line tools" > Click "Install" button

Then ML stuff using next guide:

http://magiclantern.wikia.com/wiki/Magic_Lantern_Development_on_Mac

Then you also need to download dcraw and exiftool:

http://www.sno.phy.queensu.ca/~phil/exiftool/
http://lightzoneproject.org/content/mac-version-new-dcraw-available

Then download sources from Bitbucket, I use Andy's.

Goto dual iso dir and type make. It will give the cr2hdr tool.
If you only need cr2hdr and don't want arm tooling (it is complaining about arm then), you can remove dual_iso.c. Make will give some error, but the cr2hdr will be created anyway and works :-)

root# cr2hdr IMG_0155.CR2

Input file     : IMG_0155.CR2
Full size      : 4832 x 3228
Active area    : 4770 x 3177
White level    : 13327
Black borders  : 62 left, 51 top
Black level    : 995
ISO pattern    : dBBd GBRG
Estimating ISO difference...
ISO difference : 4.03 EV (1628)
Black delta    : 2.30
Interpolation  : mean23-chroma5x5-contrast
Matching brightness...
Looking for hot pixels...
Hot pixels     : 18316
Full-res reconstruction...
Alias filtering...
Dark chroma filtering...
Building contrast map...
Dilating contrast map...
Smoothing contrast map...
ISO overlap    : 5.0 EV (approx)
Black adjust   : 1
Output file    : IMG_0155.DNG
    1 image files updated

Cheers
Jorgen
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jgerstel on August 14, 2013, 12:15:30 AM
Hi

Tested the 50D "green" issue. Looks like Windows version has the " green " issue for 50d image, but custom build on Mac shows normal image. Please see comparison of same image

green Windows build:
http://www.gerstel.org/images/canon/dual_iso/IMG_0155win.jpg
(http://www.gerstel.org/images/canon/dual_iso/IMG_0155win.jpg)
OK Mac 50D build:
http://www.gerstel.org/images/canon/dual_iso/IMG_0155mac.jpg
(http://www.gerstel.org/images/canon/dual_iso/IMG_0155mac.jpg)
50D Mac cr2hdr Tool can also be downloaded:
http://www.gerstel.org/images/canon/dual_iso/cr2hdr

Then you also need to download dcraw and exiftool:

http://www.sno.phy.queensu.ca/~phil/exiftool/
http://lightzoneproject.org/content/mac-version-new-dcraw-available
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on August 14, 2013, 01:14:39 AM
Hi A1ex, 1%, Audionut,

I get this horrible black border and overall tone (especially on the skin) when I convert RAW to DNG using cr2hdr_exp:

Sample: http://www.flickr.com/photos/[email protected]/9503180650/

Picture processed on ACR 8.1.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: v8rrc on August 14, 2013, 08:34:02 AM
Any chance of this on the 60D - can I help investigate?
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: Canon eos m on August 14, 2013, 05:01:22 PM
Another update regarding aliased areas, for cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe). This time I've tried a very difficult shot - the CR2 from daancalo2013:


Hi A1ex,

Is this is the latest version of cr2hdr or the one on the OP?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: bart on August 14, 2013, 09:34:14 PM
600D, its not perfect. I saw some artifacts.

http://www.filedropper.com/600ddualiso

This one works pretty well. Very nice work!
Title: Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
Post by: optik on August 14, 2013, 11:26:57 PM
Hi A1ex,

Is this is the latest version of cr2hdr or the one on the OP?

I too would like to know ;)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ed.jenner on August 15, 2013, 12:19:33 AM
Try both and see.   :)

I haven't been following this from the beginning, but just tried it and the latest version seems pretty good.

However, I guess because my brain works strangely, intuitively I would use it the other way around - set 100 ISO in the Dual ISO menu and shoot the shot at ISO1600.  So I tried this and although the results were reasonable, they were quite different and not as good as shooting 1600/100 with exactly the same aperture and SS.

Is this to be expected?  Simplistically I would have though it would be identical since it's just the ordering that is different.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: teo770 on August 15, 2013, 12:35:27 AM
Anyone compared dualISO with a regular 2 expo HDR picture ?
Cause dualISO is fine, you don't need a tripod (!) but it costs in aliasing and resolution.

As read in posts, dualISO enhance pic's DR.
Anyone knows how DR can be estimated from a HDR (ISO shift, not aperture shift) of 2/3/5 pics ?

On the video side, would be interesting to test DualISO with Mosaic Engineering filter...
About aliasing mainly.

Update: Spent a few hours in Photomatix Pro, mixing 2 or 3 expo "classical" HDRs..... (ISO shift)
i never get even close to the result of DualIso....

How are mixed the 2 expo in cr2hdr ???
It's a fusion, of course, not a tone map but i sould miss something !

Someone knows ?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 15, 2013, 01:55:04 AM

On the video side, would be interesting to test DualISO with Mosaic Engineering filter...
About aliasing mainly.


Dial ISO sample I posted here is on 5d2 WITH mosaic filter.

http://www.magiclantern.fm/forum/index.php?topic=7139.msg66196#msg66196
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ed.jenner on August 15, 2013, 03:13:44 AM
Anyone compared dualISO with a regular 2 expo HDR picture ?
Cause dualISO is fine, you don't need a tripod (!) but it costs in aliasing and resolution.


I would also say for me it looks like not going to be good enough for a high quality landscape shot, but perhaps fine for many other applications.  Now if I could get two full res raw files, one at ISO100 and one at ISO1600 from a single exposure, that would be a real game-changer for times when multiple exposures won't work.  Not sure what you are doing in Photomatix, but the DualISO will reduce noise

I will likely use it when I would normally blow out highlights rather than when the highlights are my primary concern.

Like many of these techniques you need to figure out when it's going to work for you.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ilguercio on August 15, 2013, 04:22:09 AM
Photomatix is something else, if you want true extended dynamic range you should use Enfuse(as a plugin for LR) to add up the exposures.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ed.jenner on August 15, 2013, 05:26:25 AM
I would also say for me it looks like not going to be good enough for a high quality landscape shot,
OK, quoting myself, but just trying this on the 7D it's really good.  Seems better than on the 5DII with whatever workflow I used yesterday.  Need to test more on the 5DII, it might be something I'd use for 'serious' shooting where I might want to make a 20x30 print.

Not sure what you are doing in Photomatix, but the DualISO will reduce noise
Just to test, I used the HDR software I prefer and the two shot option is better (without any tone mapping) as expected.

I'm having a little problem with getting the colors to come out right, but it's not a showstopper, just a little extra PP.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: NateVolk on August 15, 2013, 06:23:15 AM
The photo side progress has been good.  I'm still getting mixed results, the same as @ed.jenner.  Switching the iso's weirds things up.  I always end up with an image that is about 3 stops underexposed.  The shadows come back pretty nice most of the time, and the highlights are great.

Here's the best I can get with video.  Mind you ,I'm on a mac, and not smart enough to run the command line stuff, so I converted this with son of batch app for osx.  I toyed with the exposure a bit, the first one has no clipped highlights in the live view preview, the second one has some in the clouds.  I boosted both the shadows to 100 and held the highlights to -100 just to see how far I can push it.  Now if someone can help make the upgrades that a1ex made to the cr2hdr to the video end, we would have something amazing!!!

Title: Re: Dual ISO - massive dynamic range improvement
Post by: arturochu on August 15, 2013, 06:38:32 AM
Here's the best I can get with video.  Mind you ,I'm on a mac, and not smart enough to run the command line stuff, so I converted this with son of batch app for osx.  I toyed with the exposure a bit, the first one has no clipped highlights in the live view preview, the second one has some in the clouds.  I boosted both the shadows to 100 and held the highlights to -100 just to see how far I can push it.  Now if someone can help make the upgrades that a1ex made to the cr2hdr to the video end, we would have something amazing!!!

i did about 140 gb of dual iso video material, i haven't processed it yet because of the outdated raw2dng. waiting for the upodate too.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: packgrad2k1 on August 15, 2013, 06:39:12 AM
First off, my apologies if this is covered elsewhere. I've dug and dug, and can't find anywhere this is explicitly answered: is DualISO working on the 5DII or 50D for video?

I've gotten it to work well with stills (though I have the same issue another poster reported that no matter what I set the recovery ISO to, it will not go above 1600 on my 5DII or 50D) but I see no difference between RAW video shot with DualISO on and off for either the 5DII or 50D. Since I'm using a mac, exclusively, it could also be that Son of Batch just doesn't deal with DualISO raw video -- Son of Batch is the only RAW -> DNG converter for mac I've seen that supposedly supports DualISO.

That's all for now. I greatly *greatly* appreciate your time, and my hat is off to all the folks working on ML for the amazing work to date on this project. ML RAW now gives me a way to process and grade that makes sense to my photo-centric brain.  ;D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: PressureFM on August 15, 2013, 09:07:58 AM
First off, my apologies if this is covered elsewhere. I've dug and dug, and can't find anywhere this is explicitly answered: is DualISO working on the 5DII or 50D for video?

I've gotten it to work well with stills (though I have the same issue another poster reported that no matter what I set the recovery ISO to, it will not go above 1600 on my 5DII or 50D) but I see no difference between RAW video shot with DualISO on and off for either the 5DII or 50D. Since I'm using a mac, exclusively, it could also be that Son of Batch just doesn't deal with DualISO raw video -- Son of Batch is the only RAW -> DNG converter for mac I've seen that supposedly supports DualISO.

That's all for now. I greatly *greatly* appreciate your time, and my hat is off to all the folks working on ML for the amazing work to date on this project. ML RAW now gives me a way to process and grade that makes sense to my photo-centric brain.  ;D

Then the answer should be pretty straightforward. No Dual ISO in video mode on the 5D Mark II, which has been said many times before now  ;)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: arrinkiiii on August 15, 2013, 09:38:09 AM
The photo side progress has been good.  I'm still getting mixed results, the same as @ed.jenner.  Switching the iso's weirds things up.  I always end up with an image that is about 3 stops underexposed.  The shadows come back pretty nice most of the time, and the highlights are great.

Here's the best I can get with video.  Mind you ,I'm on a mac, and not smart enough to run the command line stuff, so I converted this with son of batch app for osx.  I toyed with the exposure a bit, the first one has no clipped highlights in the live view preview, the second one has some in the clouds.  I boosted both the shadows to 100 and held the highlights to -100 just to see how far I can push it.  Now if someone can help make the upgrades that a1ex made to the cr2hdr to the video end, we would have something amazing!!!


Very nice image. I can see some moire and maybe some fringe in the top of the cabane...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 3pointedit on August 15, 2013, 10:12:46 AM
I understand, that dual iso is for RAW recording only.
Is there a chance it will become available for H264 encoded recordings later, or is this not possible?

The raw video workflow is really time consuming, and if dual iso was possible with H264 directly in camera, this would be a fantastic time saver..

I am believe that the compression would kill the fine detail needed to reconstruct the HDR image. Also there is a current HDR solution for h264 shooting, with temporal errors instead of spatial errors.

However if the interlacing could be conserved through compression I'm sure you could comb the fields out for processing.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 15, 2013, 03:31:39 PM
I thought of shooting XXiso/100 too. In the day time yea, 100/1600 but at night you have the opposite problem and want to expose backwards... ie you still get lights, etc. But I think the converter isn't set up for it.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: NateVolk on August 15, 2013, 08:55:12 PM
Very nice image. I can see some moire and maybe some fringe in the top of the cabane...

Yeah, on the computer it looks pretty bad.  Straight up RAW is still the best option.  You could process the dual iso better to remove the fringing, but the amount of work added is huge.

The stills converter works if you flip the iso's, but it produces very different results from the 100/1600 at the same settings.  I'm still having a real hard time understanding how to properly expose the image.  Any tips?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 15, 2013, 10:45:29 PM
7D test shots on dual ISOs.

Here are four shots, unlighted room with res chart and open window noontime. Processed the shots to get a "good" image indoors and try to recover as much info from the window. I was trying see the noise patterns and also if i reverse the recovery if there is a difference.
Took 4 shots;
1. 100 iso
2. 1600 iso
3. 100-1600 dual iso
4. 1600-100 dual iso

shot 1 - 100 iso
https://fbcdn-sphotos-c-a.akamaihd.net/hphotos-ak-prn2/1150512_358773894253401_433820467_o.jpg
shot 2 - 1600 iso
https://fbcdn-sphotos-e-a.akamaihd.net/hphotos-ak-frc3/1146262_358773880920069_1154231185_o.jpg
Shot 3 - 100_1600 iso
https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-ash3/1172292_358773887586735_1710098679_o.jpg
shot 4 - 1600_100 iso
https://fbcdn-sphotos-g-a.akamaihd.net/hphotos-ak-prn1/1172556_358774064253384_781772650_o.jpg
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 16, 2013, 09:05:33 AM
Great test Ted, can you upload the CR2 files for it? (or at least the 100/1600 one).

Also @Canon eos m: need the CR2 from here: http://www.magiclantern.fm/forum/index.php?topic=7139.msg66734#msg66734

I've recompiled cr2hdr.exe and re-uploaded it, just in case. No code changes. Do you still have issues?

Quote
Update: Spent a few hours in Photomatix Pro, mixing 2 or 3 expo "classical" HDRs..... (ISO shift)
i never get even close to the result of DualIso....

Try Zero Noise: http://www.guillermoluijk.com/software/zeronoise/index.htm

It should use a similar blending method (though I didn't try it, I've only read their articles and borrowed the technique).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 16, 2013, 10:03:07 AM
I've recompiled cr2hdr.exe and re-uploaded it, just in case. No code changes. Do you still have issues?

That fixed it.  Ta.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 16, 2013, 02:35:24 PM
So which algo was the best? From the repo I get v-23-contrast, from the EXE in the first post I had v22-smooth or something like that.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on August 16, 2013, 02:53:28 PM
Great test Ted, can you upload the CR2 files for it? (or at least the 100/1600 one).

Also @Canon eos m: need the CR2 from here: http://www.magiclantern.fm/forum/index.php?topic=7139.msg66734#msg66734

I've recompiled cr2hdr.exe and re-uploaded it, just in case. No code changes. Do you still have issues?

Try Zero Noise: http://www.guillermoluijk.com/software/zeronoise/index.htm

It should use a similar blending method (though I didn't try it, I've only read their articles and borrowed the technique).

Looking for it. Will send you soon as I track it down or will send another CR2 with similar blackening ... have a lot with this same issue.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on August 16, 2013, 03:59:06 PM

I've recompiled cr2hdr.exe and re-uploaded it, just in case. No code changes. Do you still have issues?


Some issue with ch2hdr.exe download from the OP - the black command prompt screen flashes for a second and then dies out. The ch2hdr_exp.exe works fine.

Zero Noise also has problems is asking for a missing MSSTDFMT.DLL file.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Doyle4 on August 16, 2013, 04:22:59 PM
Whilst batch converting mac im getting this:

Input file     : /Users/doyle/Desktop/DCIM/100EOS5D/IMG_9937.CR2
Full size      : 5792 x 3804
Active area    : 5634 x 3753
White level    : 15713
Black borders  : 158 left, 51 top
Black level    : 973
ISO pattern    : BddB GBRG
Estimating ISO difference...
ISO difference : 4.22 EV (1860)
Black delta    : -8.79
Interpolation  : mean23-chroma5x5-contrast
Matching brightness...
Looking for hot pixels...
Hot pixels     : 2232
Full-res reconstruction...
Alias filtering...
Dark chroma filtering...
Building contrast map...
Dilating contrast map...
Smoothing contrast map...
ISO overlap    : 4.8 EV (approx)
Black adjust   : -5
Output file    : /Users/doyle/Desktop/DCIM/100EOS5D/IMG_9937.DNG
sh: -c: line 0: unexpected EOF while looking for matching `"'
sh: -c: line 1: syntax error: unexpected end of file
Exiftool didn't work

But dng's have been created and seem to be ok as a file that was updated? noticed it does the same on windows too when batch converting,

Cheers guys, and great work again!

Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 16, 2013, 05:24:08 PM
Great test Ted, can you upload the CR2 files for it? (or at least the 100/1600 one).

@a1ex

emailed cr2

ted
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Ozgur on August 16, 2013, 08:00:42 PM
Does Dual Iso still only work for Still images on Eos 7D?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 16, 2013, 08:08:16 PM
Does Dual Iso still only work for Still images on Eos 7D?

yes.
Title: Re: Dual ISO - CR2HDR
Post by: frogcement on August 17, 2013, 12:33:27 AM
I've recompiled cr2hdr.exe and re-uploaded it, just in case. No code changes.


Just wondering if/when there will be a build that takes command line switches? Or does it already do this and I am missing it?
Cheers....
--
K.C.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 17, 2013, 12:57:05 AM
5D mkII test shots on dual ISOs.

The latest cr2hdr works in removing greenish cast from shadow areas.
Unlighted room with res chart and open window noontime. Processed the shots to get a "good" image indoors and try to recover as much info from the window. Compared the noise patterns and also tested with VAF filter to control color aialing and moire.
3 shots;
1. 100 iso
2. 100-1600 dual iso
3. 100-1600 dual iso with VAF

1.100 iso
https://fbcdn-sphotos-e-a.akamaihd.net/hphotos-ak-ash3/1150526_359389720858485_1133039032_o.jpg

100-1600 dual iso
https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-prn2/1146938_359389874191803_1768168050_o.jpg

3. 100-1600 dual iso with VAF
https://fbcdn-sphotos-c-a.akamaihd.net/hphotos-ak-frc3/1093863_359389800858477_834816052_o.jpg
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on August 17, 2013, 03:59:08 AM
Could someone point me to the link to the latest cr2hdr.exe update?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 17, 2013, 04:20:10 AM
Could someone point me to the link to the latest cr2hdr.exe update?

first post.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on August 17, 2013, 01:35:18 PM
Ted did that already but it doesn't work for me - whereas the previous on ch2hdr_exp.exe works just fine.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on August 17, 2013, 01:50:08 PM
first post.

The file does not convert instead cr2hdr.exe generates this tmp file:

Filename: C:\Users\USer\Downloads\cr2hdr August 16 2013 download\cr2hdr\_G1A8405.CR2
Timestamp: Wed Aug 14 18:42:17 2013
Camera: Canon EOS 5D Mark III
ISO speed: 12800
Shutter: 1/98.7 sec
Aperture: f/2.5
Focal length: 35.0 mm
Embedded ICC profile: no
Number of raw images: 1
Thumb size:  5760 x 3840
Full size:   5920 x 3950
Image size:  5796 x 3870
Output size: 5796 x 3870
Raw colors: 3
Filter pattern: RGGBRGGBRGGBRGGB
Daylight multipliers: 2.125175 0.943985 1.338680
Camera multipliers: 1259.000000 1024.000000 3666.000000 1024.000000
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 17, 2013, 07:06:30 PM
The file does not convert instead cr2hdr.exe generates this tmp file:

Filename: C:\Users\USer\Downloads\cr2hdr August 16 2013 download\cr2hdr\_G1A8405.CR2
Timestamp: Wed Aug 14 18:42:17 2013
Camera: Canon EOS 5D Mark III
ISO speed: 12800
Shutter: 1/98.7 sec
Aperture: f/2.5
Focal length: 35.0 mm
Embedded ICC profile: no
Number of raw images: 1
Thumb size:  5760 x 3840
Full size:   5920 x 3950
Image size:  5796 x 3870
Output size: 5796 x 3870
Raw colors: 3
Filter pattern: RGGBRGGBRGGBRGGB
Daylight multipliers: 2.125175 0.943985 1.338680
Camera multipliers: 1259.000000 1024.000000 3666.000000 1024.000000

make sure the folder where the cr2hdr.exe and the cr2 files have a simple file name, no dashes and special characters also try to move the said folder to the root of the drive.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on August 17, 2013, 07:41:41 PM
make sure the folder where the cr2hdr.exe and the cr2 files have a simple file name, no dashes and special characters also try to move the said folder to the root of the drive.

Worked!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: sdesign on August 18, 2013, 04:36:06 AM
has there been any simple tutorial or video help, for this?  I'm getting single pics with lines, new to this, not to ML though, HELP!!!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: sdesign on August 18, 2013, 04:42:58 AM
my tmp file is empty, as well
Title: Re: Dual ISO - massive dynamic range improvement
Post by: arrinkiiii on August 18, 2013, 06:13:17 AM
has there been any simple tutorial or video help, for this?  I'm getting single pics with lines, new to this, not to ML though, HELP!!!

The single pics (cr2) with lines that your are getting is the dual iso (interlaced iso), you need to click and drag to cr2hdr.exe and he will give you dng's, already with the dual iso merged.

The dng´s must be in the same folder than the cr2hdr.exe
Title: Re: Dual ISO - massive dynamic range improvement
Post by: D.L. Watson on August 18, 2013, 10:55:15 AM
Attended a wedding and tested the duel iso for my 50D. Some beautiful dynamic range.

(https://dl.dropboxusercontent.com/u/9746153/Site%20Images%20%26%20Files/duels.jpg) (https://dl.dropboxusercontent.com/u/9746153/Site%20Images%20%26%20Files/duell.jpg)

When nighttime hit, I tried to see if I could do something like 800/12800 but even though it was selected, it still defaulted back to 800/1600. Maybe a bug? (I know it would be incredi-noise, but I just wanted to see how bad it would be).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: nanomad on August 18, 2013, 11:03:49 AM
Dual iso works with "analog" ISO only. 12800 is definitely digital
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Asufeld_ins on August 18, 2013, 11:17:05 AM
Hi everyone and thx for this feature.

I use ML from many month and i take recently a new feature for Dual ISo.

But i have a problem.

When i can put my CR2 on cr2hdr.exe

Many of them don't work , the windows open just 1 second and shutdown.

Why a lot of my Cr2 work but not everything of this ?

( sorry for my english .... )
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on August 18, 2013, 11:31:50 AM
Hi everyone and thx for this feature.

I use ML from many month and i take recently a new feature for Dual ISo.

But i have a problem.

When i can put my CR2 on cr2hdr.exe

Many of them don't work , the windows open just 1 second and shutdown.

Why a lot of my Cr2 work but not everything of this ?

( sorry for my english .... )

Please look for post #633, #644 and #645 for help.

Best is to save the file in the root director. Name the directory cr2hdr. Download the zip in the directory and unzip. Save the cr2 files from the camera in the cr2hdr unzip directory where the cr2hdr.exe files have been extracted. It should work without a problem.  :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Asufeld_ins on August 18, 2013, 11:40:48 AM
Thx for answers

I read it before you post for my.

I have extract on c:/ch2hdr

and i taking ch2hdr_exp.exe too.

I clear my TMP file

And is the same problem.

My DSLR is a 50D. Have a feature ( ch2hdr ) Just for it or is the same for all ?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on August 18, 2013, 12:00:01 PM
Thx for answers

I read it before you post for my.

I have extract on c:/ch2hdr

and i taking ch2hdr_exp.exe too.

I clear my TMP file

And is the same problem.

My DSLR is a 50D. Have a feature ( ch2hdr ) Just for it or is the same for all ?

Same for all.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: fanous on August 18, 2013, 12:46:19 PM
Hi Guys,
New firmware is great, but...
After I install it (booting from CF card) my 7D eat energy very fast. Batteries (two in battery pack) are on half of capacity after taking around 150 photos :(
I have ML firmware on one CF card and I using it when I need Dual ISO. If the "Dual ISO" card is in my 7D the back red LED is blinking once a 2 second period. If I need normal shooting I place into my 7D another card (no red LED blinking). In both cases the consumption of energy is very fast. One more think, when I turn camera on there is about 2 second delays then camera is ready to work...
For me energy and readiness is priority number one. Can I remove booting from my 7D? Can you help me, please?
Thank you for your response

PS Sorry for my bad English

Fanous
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Walter Schulz on August 18, 2013, 01:09:00 PM
If the "Dual ISO" card is in my 7D the back red LED is blinking once a 2 second period.

Described in the 7D installation procedure thread: Prefs -> Warning for bad setting -> Q -> Turn all 3 warning items OFF

Can I remove booting from my 7D? Can you help me, please?

- Download http://pel.hu/ML/bootoff.bsdiff
Follow the steps creating 7Dbooton.fir but create 7Dnoboot.fir
See http://www.magiclantern.fm/forum/index.php?topic=3974.msg64342#msg64342 for detailed instructions.

Ciao
Walter
Title: Re: Dual ISO - massive dynamic range improvement
Post by: fanous on August 18, 2013, 01:36:58 PM
Thank you Walter
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Luzestudio on August 18, 2013, 01:44:42 PM
600D, its not perfect. I saw some artifacts.

http://www.filedropper.com/600ddualiso

Thanks a lot 1%, trying right now, the results looks incredible! Thanks for keep integrating the updates into the 600D.

Title: Re: Dual ISO - massive dynamic range improvement
Post by: akumiszcza on August 18, 2013, 03:14:10 PM
Attended a wedding and tested the duel iso for my 50D. Some beautiful dynamic range.

(https://dl.dropboxusercontent.com/u/9746153/Site%20Images%20%26%20Files/duels.jpg) (https://dl.dropboxusercontent.com/u/9746153/Site%20Images%20%26%20Files/duell.jpg)

When nighttime hit, I tried to see if I could do something like 800/12800 but even though it was selected, it still defaulted back to 800/1600. Maybe a bug? (I know it would be incredi-noise, but I just wanted to see how bad it would be).

Hi. What ISOs and converters did you use for this shot? I've just checked with latest tragic lanter build and latest cr2hdr (for windows) from first post here and still get greenish shadows.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: oddname on August 18, 2013, 04:03:07 PM
I agree about greenish tint, using latest cr2hdr with my 5D2 many images are greenish and no color correction can get rid of it, any attempt
just adds some other color on the greens.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: sdesign on August 18, 2013, 07:28:32 PM
getting very frustrated, shooting pictures with my 50d, newest tragic lantern, aug17th, keep getting nogo saying something about interlaced something rather.  Simple files names and all, still weird nogo's, HELP!!!!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Andy600 on August 18, 2013, 07:34:19 PM
getting very frustrated, shooting pictures with my 50d, newest tragic lantern, aug17th, keep getting nogo saying something about interlaced something rather.  Simple files names and all, still weird nogo's, HELP!!!!

Are you using the latest cr2hdr.exe? http://www.magiclantern.fm/forum/index.php?topic=5586.msg67977#msg67977 (http://www.magiclantern.fm/forum/index.php?topic=5586.msg67977#msg67977)

Are you sure you actually have Dual ISO enabled?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: D.L. Watson on August 18, 2013, 09:13:09 PM
Hi. What ISOs and converters did you use for this shot? I've just checked with latest tragic lanter build and latest cr2hdr (for windows) from first post here and still get greenish shadows.

I used ISO 100 and 1600 with the latest cr2hdr for Windows. They converted beautifully for me.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: D.L. Watson on August 18, 2013, 09:14:01 PM
Dual iso works with "analog" ISO only. 12800 is definitely digital

Then why is it even an option in the Duel ISO settings?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: akumiszcza on August 18, 2013, 09:21:39 PM
I used ISO 100 and 1600 with the latest cr2hdr for Windows. They converted beautifully for me.

I've checked again with latest build and latest cr2hdr compiled by Andy600. It looks like "extreme" cases (with blown highlights) are still converted wrong by it (green), but otherwise it's a huge improvement (most photos are nice).

Example of wrong processing:
https://mega.co.nz/#!lMUQ1Syb!bUPbAzlUBwnqWgjGER5eyGBbZ3JptHNwzZKRlXJsgkE (CR2)
https://mega.co.nz/#!IFtzGYqB!fEk-PEU91rsOyq6bGxSwYA9_IOJUFLRaTpbMKgl7lnY (DNG)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 18, 2013, 09:39:38 PM
Different cameras have different isos... 128K I don't think anything has native, maybe 5d3 or 1dc/x.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: D.L. Watson on August 18, 2013, 11:13:45 PM
Different cameras have different isos... 128K I don't think anything has native, maybe 5d3 or 1dc/x.

That makes sense. Thanks for the clarification.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: sdesign on August 19, 2013, 05:30:25 AM
Yes i'm actually using it and its giving me a quick error that i cannot read fast enough and closes
Title: Re: Dual ISO - massive dynamic range improvement
Post by: engardeknave on August 19, 2013, 08:46:51 AM
For those of you with a window just flashing when you drag-drop onto the .exe, try putting the files you want to convert right in the same folder, right off the root of your drive, ie. c:\tmp. There was a problem with the path when I tried dragging directly from the CF card at first. (Failing that, maybe try running cmd.exe and run the program so you can see the error.)

Does anyone know the best way to expose with dual ISO in the daytime? Specifically I mean: under or over? The photo seems way brighter after taking the shot than in LV. Am I going to get better highlight or shadow recovery? I'm going to do a shoot tomorrow and I won't have time to read/experiment and figure this out beforehand.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: akumiszcza on August 19, 2013, 10:33:42 AM

Does anyone know the best way to expose with dual ISO in the daytime? Specifically I mean: under or over? The photo seems way brighter after taking the shot than in LV. Am I going to get better highlight or shadow recovery? I'm going to do a shoot tomorrow and I won't have time to read/experiment and figure this out beforehand.

Select ISO 100 and set ISO 100/1600 in Dual ISO settings. Expose to get highlights properly (not cut). Best would be exposing to the right. 1600 ISO will take care of shadows.
Most often setting EV-2 in Av while taking a shot works fine.

I did not check 1600/100 method though, but I think proper exposure will be trickier (you won't be sure if highlights are ok). Also 200/800 might get much less noise if the dynamic range is not too great.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Doyle4 on August 19, 2013, 12:46:48 PM
As the Highlights and Shadows are half resolution,

could there be an option in the cr2hdr app to make the other iso image half resolution to match the resolution of the shadow and highlights? people may wonder why i would like this but i noticed Dual Iso only works on RAW setting no sRAW1 or sRAW2, resize is an option in photoshop or other editing software but i would be halving the resolution again in the highlights and shadows.

cr2hdr could produce two images with this option enabled, one is the final image that the program does now and a 2nd image has been scaled to match the resolution of the halved highlights and shadows.

Cheers :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Walter Schulz on August 19, 2013, 01:30:23 PM
The question is: Is it broken?

I'm not so sure about this feature (if it blocks ML developers "resources"). I learned mid tones have big influence on sharpness reception (and more). Look what LR/ACR ruler Clarity does and this is done (AFAIK) mostly by midtone manipulation. Therefore I don't have a big problem with lacking resolution in shadows/highlights.

At the moment this all is theoretical and has to be verified/falsified in the field with some valid examples.

Maybe it is just the other way round and diminished highlight/shadow resolution will stick out like a sore thumb when viewed by Mr. and Mrs. Common.

Ciao, Walter
Title: Re: Dual ISO - massive dynamic range improvement
Post by: sdesign on August 19, 2013, 03:33:47 PM
im really sorry to bother againg, but the root trick amongst every other trick, doesnt work.  I've changed filenames to be simpler, downloaded the latest and greatest cr2hdr and all it does is still flash and complain about some not interlaced or something like that, i cant read it so fast, as it comes up.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Walter Schulz on August 19, 2013, 03:55:04 PM
Old admin trick: Make a video at highest possible frame rate! ;-)

Ciao, Walter
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 19, 2013, 04:00:26 PM
Or open a cmd line and process the cr2 manually.  The cmd line will remain open when it's finished processing and you will be able to see the error message.

Path_to_cr2hdr\cr2hdr.exe D46A0001.CR2
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Doyle4 on August 19, 2013, 04:45:52 PM
The question is: Is it broken?

I'm not so sure about this feature (if it blocks ML developers "resources"). I learned mid tones have big influence on sharpness reception (and more). Look what LR/ACR ruler Clarity does and this is done (AFAIK) mostly by midtone manipulation. Therefore I don't have a big problem with lacking resolution in shadows/highlights.

At the moment this all is theoretical and has to be verified/falsified in the field with some valid examples.

Maybe it is just the other way round and diminished highlight/shadow resolution will stick out like a sore thumb when viewed by Mr. and Mrs. Common.

Ciao, Walter

I see what your saying :)
Unfortunately the mkii cant do Dual Iso Video, but i know what you mean about mr n mrs common not been able to notice, cheers for replying.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: oddname on August 19, 2013, 04:54:12 PM
Is it a limitation or what causes the mk2 not to be able to do video with this?
Final verdict or just atm? =)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Asufeld_ins on August 19, 2013, 06:52:01 PM
Are you using the latest cr2hdr.exe? http://www.magiclantern.fm/forum/index.php?topic=5586.msg67977#msg67977 (http://www.magiclantern.fm/forum/index.php?topic=5586.msg67977#msg67977)

Are you sure you actually have Dual ISO enabled?

Oh yeah thank you.

the old CR2HDR.exe don't work.
But your Link is OK !!

Good job and Thank you ^^


Edit: New feature on your download page from this day.
Tragic.Lantern-2.0-Andy600.Build.2013Aug19.50D109

What's is new ?
( sorry for my english ... )
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Doyle4 on August 19, 2013, 08:01:49 PM
Is it a limitation or what causes the mk2 not to be able to do video with this?
Final verdict or just atm? =)

not 100% sure my self, i know the mkiii and 600D do DI video.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: engardeknave on August 19, 2013, 11:48:54 PM
Select ISO 100 and set ISO 100/1600 in Dual ISO settings. Expose to get highlights properly (not cut). Best would be exposing to the right. 1600 ISO will take care of shadows.
Most often setting EV-2 in Av while taking a shot works fine.

I did not check 1600/100 method though, but I think proper exposure will be trickier (you won't be sure if highlights are ok). Also 200/800 might get much less noise if the dynamic range is not too great.

Thanks, I used this and I think it worked out well. May or may not post examples later after I'm done processing everything.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: engardeknave on August 19, 2013, 11:57:03 PM
I did a shoot just now alternating between advanced bracket for HDR and dual ISO. I did this by toggling each in junkie mode. When I got home I found that all the file names were "DUALXXXX.CR2", even the files that were part of a bracket with dualiso disabled from the menu. The files are definitely non-dualiso shots, but they are named incorrectly.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: TLN on August 20, 2013, 09:57:33 AM
I did a shoot just now alternating between advanced bracket for HDR and dual ISO. I did this by toggling each in junkie mode. When I got home I found that all the file names were "DUALXXXX.CR2", even the files that were part of a bracket with dualiso disabled from the menu. The files are definitely non-dualiso shots, but they are named incorrectly.

Exactly.

I've tried latest ML build + dual ISO module.

Started doing shots in pairs: 1st with dualISO off, 2nd with dualISO on.

I got one file named IMG_xxx1.cr2 and then all the files named DUALxxx2.cr2 and so on.


This is why I registered there, but you posted it first. :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 20, 2013, 10:00:48 AM
I got one file named IMG_xxx1.cr2 and then all the files named DUALxxx2.cr2 and so on.

I've pointed it out to a1ex.  Hopefully he can debug it.

https://bitbucket.org/hudson/magic-lantern/commits/8bd01d4dbb46386e50d4f97b71b7148a29571764
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Doyle4 on August 20, 2013, 12:05:56 PM
Im getting same problem, DualIso name stays, iv also noticed when you have the option to shoot one normal one dual sometimes it will take either two normal and two dual iso missing the normal/dual iso and replacing it with vice versa, anyone else had that?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: X-RAY on August 20, 2013, 01:42:50 PM
I shot some tests with Dual ISO on my 5D2 under real world conditions and in a challenging szenario. But what can say ... just a great addition and so powerful. :-)
This really isn't a scientific test, but I just wanted to try how good it is for my purposes. A sunrise seems quite challenging. As I understand, the best way to shoot Dual ISO is to expose for the highlights and to get them almost clipping. But not that much as I normally shoot. And then I hoped for the power on the dark side. ;-)
The unprocessed version:
http://www.flickr.com/photos/dark-ness/9552218087/

Just some slight adjustments:
http://www.flickr.com/photos/dark-ness/9552216865/

And a quick processed version:
http://www.flickr.com/photos/dark-ness/9552220807/


Does anybody know what those white dots are (there are just a few)? Seems to be some processing error. Are those in some way dead pixel that show up much more through cr2hdr?

And I also have just a few other shots that have strong magenta highlights. No correction possible through white balance.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: oddname on August 20, 2013, 05:51:41 PM
X-RAY how did you expose? ETTR-like or just normal exposure for the highlights?



AND for the ones that know, will video support be available for the 5D2 or is just a model limitation that says "Never"? =)
Hate to sound like a broken record, but fun to know if one is thinking about a mk3 like I am, dealbreaker if the 5D2 will get the support too eventually.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: akumiszcza on August 20, 2013, 06:38:44 PM

Im getting same problem, DualIso name stays, iv also noticed when you have the option to shoot one normal one dual sometimes it will take either two normal and two dual iso missing the normal/dual iso and replacing it with vice versa, anyone else had that?

I'm having same problems with DUAL (_UAL for AdobeRGB) file names. As for one normal/one dual shots, I've noticed it does not work for continuous shots. Probably you took double photos, then stopped, then double again. Each of those doubles will be the same type, so in this case it will be 2 normals and two duals for instance. And probably all named dual...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Mirazimov on August 20, 2013, 07:46:27 PM
How to use this function? I have the Canon 6D. I downloaded the latest version of the firmware build. Found a download module features DUAL ISO / But I do not understand how it works. In the photo just strip. What am I doing wrong?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 20, 2013, 07:52:05 PM
How to use this function? I have the Canon 6D. I downloaded the latest version of the firmware build. Found a download module features DUAL ISO / But I do not understand how it works. In the photo just strip. What am I doing wrong?

Read the first post of this thread Mirazimov.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: TLN on August 20, 2013, 08:43:22 PM
BTW, after some experiments with DualISO and ETTR, I found that I'm missing some options in ETTR module.

There's exposure settings, and it have an option like -0.5EV, -1EV, -2, -3 and -4.  With dual ISO i'd like to have something like 0EV, +1EV and may be +2EV as well.  This will make a brighter shots, and can be useful with DualISO. Without it, i shoud pull my blacks alot to see the difference.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: X-RAY on August 20, 2013, 10:15:51 PM
@ oddname
I didn't activate ETTR in ML but my approach was quite the same.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: oddname on August 21, 2013, 02:15:47 PM
Using dual ison on my 5D2 and this is what I get when done with the postprocessing.

http://img823.imageshack.us/img823/2647/nqrs.jpg

Is it really supposed to be this greenish?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: X-RAY on August 21, 2013, 09:46:20 PM
This seems to be just an incorrect whitebalance setting. Are the EXIF information in the DNG file correct (like aperture, shutter speed...)?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: robinlee on August 21, 2013, 10:18:04 PM
I am using one of the cr2hdr_exp.exe and it always give different exposure results even though my images were exposed manually and in the same light setting.

Another issue is sometimes it has blue overcasts on shadow too.

Plus it doesn't always convert the dualiso raw file and sometimes it need few attempts before it detects the dualiso raw files.

A1ex, any chance for you to look into this.

Thanks
Robin
Title: Re: Dual ISO - massive dynamic range improvement
Post by: engardeknave on August 21, 2013, 10:18:50 PM
White balance has been completely erratic with dual ISO. I just tried copying all the exif tags from the .cr2 to the .dng with exiftool (presumably what cr2hdr already does) and the white balance was still off.

Not to suggest that it really matters.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: robinlee on August 21, 2013, 10:37:53 PM
Hi engardeknaye,

Do you mind to PM me on how to copy exif tags from *.cr2 to *.dng with exiftool please?

Many thanks.
Robin

White balance has been completely erratic with dual ISO. I just tried copying all the exif tags from the .cr2 to the .dng with exiftool (presumably what cr2hdr already does) and the white balance was still off.

Not to suggest that it really matters.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Doyle4 on August 22, 2013, 01:04:42 AM
Iv noticed white balance can be massively out also on my MKii, to the point even auto white balance on PS can get confused and even with custom is difficult to fix, i use ML auto white balance now, haven't tested since then though.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: engardeknave on August 22, 2013, 06:18:56 AM
Hi engardeknaye,

Do you mind to PM me on how to copy exif tags from *.cr2 to *.dng with exiftool please?

Many thanks.
Robin

I just did one: exiftool -tagsfromfile img_1234.cr2 img_1234.dng

But I think cr2hdr already does this.

See http://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html#copying_examples (http://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html#copying_examples)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: stevefal on August 22, 2013, 08:42:01 AM
I'm going to commit blasphemy and say I think Dual ISO is a dead end for video. Some of the samples out there have amazing dynamic range, but the aliasing is just terrible.

Having HDR with no time-based artifacts may be worth it for stills, but even there, bracketing is an option with even more range, and it works fine in many cases.

I say this because I really wish that RAW video was perfected and truly field-ready. RAW video was already drop-dead awesome before dual-ISO came along, and yet there's still plenty of work before it's really usable. Solving audio, external displays, cropping overlays, status indication UI, file format, time-code, exposure/focus aids, workflow and, of course, stabilizing the system are all worthwhile if not critical. Finishing them is a job.

I think dual-ISO video is intriguing, but a huge "feature-creep" on RAW video, and it would be best to ship the first before becoming consumed by the second. ML RAW video could mean a revolution in film-making, but it's essentially still a prototype.

How about getting RAW video done? Think of the films, the press, the accolades! It's that good.

I'm happy to help any way I can.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: kgv5 on August 22, 2013, 11:27:43 AM
Dual ISO for video is a very usefull tool. And like every tool it has specific task to do. It is not usable in every situation though, for me it is great in cooperation with HDR function.
If I want to have high dynamic range with lots of detail - i use HDR. It gives motion artifacts so i use it when there is no fast camera movement. If I want high dynamic range in motion - i use dual ISO. Aliasing and moire can be less visible with motion blur and of course when background is out of focus, i tested it also with some chroma blur and it gives quite good results in decreasing artefacts.
It is great to have such awesome tools, they are still work in progress and i don't feel it is a dead end.
Even RED's HDRx has some quirks with motion so seems there are no way to have an ideal solution.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: oddname on August 22, 2013, 11:49:52 AM
This seems to be just an incorrect whitebalance setting. Are the EXIF information in the DNG file correct (like aperture, shutter speed...)?

My result is after "correcting" White balance.
Will check the dng when possible.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: X-RAY on August 22, 2013, 03:44:25 PM
My result is after "correcting" White balance.
Will check the dng when possible.

I looked into your uploaded file and I got it perfectly right with just the white balance eyedropper in ACR (I think on the curtain in the front should give the result).
Or do you mean those agressive greens in the tree? These may be chromatic aberrations.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 22, 2013, 04:03:14 PM
Or do you mean those agressive greens in the tree? These may be chromatic aberrations.

Fringe artifacts from the aliasing.  Defringe it in ACR/LR.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: stevefal on August 22, 2013, 04:50:37 PM
Aliasing and moire can be less visible with motion blur and of course when background is out of focus, i tested it also with some chroma blur and it gives quite good results in decreasing artefacts.

Even if dual-ISO video has value in those cases, I still wish that RAW video could be finished and shipped first. Shooting RAW provides massive value compared to in-camera video, and without the huge caveat and complication of aliasing.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 22, 2013, 05:47:48 PM
Even if dual-ISO video has value in those cases, I still wish that RAW video could be finished and shipped first. Shooting RAW provides massive value compared to in-camera video, and without the huge caveat and complication of aliasing.

In your opinion!

Besides, it's not like work has stalled on raw recording, g3gg0 is doing some awesome work on it here (http://www.magiclantern.fm/forum/index.php?topic=7122.0).  That's a massive job going on there, coding an entire container format.  How long does it take the MPEG to deliver a new format.  Proposals, drafts, deciding who gets the biggest percentage of the patent pool.  Years!
That includes a ton of stuff you mentioned in your previous post.

P.s.  Take further discussion into another thread please.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: stevefal on August 22, 2013, 06:01:51 PM
Of course my opinion! Also agree the file format etc. work is fantastic!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: robinlee on August 22, 2013, 06:31:10 PM
Thanks but still have no joy with this, cr2hdr always show exiftool don't work...  :(

I just did one: exiftool -tagsfromfile img_1234.cr2 img_1234.dng

But I think cr2hdr already does this.

See http://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html#copying_examples (http://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html#copying_examples)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 22, 2013, 07:10:48 PM
hello,

I downloaded the latest version for 5D2 (31302bbc6461.zip) and the corresponding cr2 to dng tool.
I see repeatable (non-random) noise patterns at dark areas (that were amplified with the use of Shadows slider (Camera Raw 8.1) in the resultant DNG file.
I have a screenshot at 200% but in "Attachments and other options" I do not see an option to upload a photo. I'll try dropbox (if I still have an account...)

https://www.dropbox.com/s/rvfwb2an3kxukb0/5D2_DUAL_ISO_100_1600.JPG
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 22, 2013, 07:32:23 PM
I have a screenshot at 200%.

Upload the CR2 also.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 22, 2013, 07:34:00 PM
As a start can you read the uploaded screenshot ?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 22, 2013, 07:35:44 PM
As a start can you read the uploaded screenshot ?

a1ex can't bugfix screenshots  ;)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 22, 2013, 07:38:27 PM
a1ex can't bugfix screenshots  ;)
Very funny!  ;D I asked however if you can read the dropbox link (as a start) because ... I just reactivated and I hadn't use it for sharing links ever!

CR2 will follow in 5 mins...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 22, 2013, 07:40:19 PM
Yeah it's fine.  :D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 22, 2013, 07:43:22 PM
Yeah it's fine.  :D

https://www.dropbox.com/s/rjjpbcloe0dbfvl/IMG_0200test.CR2
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Doyle4 on August 22, 2013, 08:33:05 PM
i spotted couple of them on my images, sometimes blue or red
Title: Re: Dual ISO - massive dynamic range improvement
Post by: engardeknave on August 22, 2013, 10:41:13 PM
Guys listen, I think all RAW video development should be put on hold for now because I personally don't need it. I know this might be a controversial point of view, but if you'd stop making video so much, I'm sure you'd see things my way.

More seriously, dual ISO is awesome. It's perfect for shooting stills outside where one exposure sometimes doesn't have enough range but stuff is blowing around too much for an HDR bracket. I've already started using it during every shoot. This feature is infinitely more important to me than anything that involves video.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: blainesuque on August 23, 2013, 01:12:20 AM
Can someone list a good post workflow for Dual Iso video? is it the same as RAW Video?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: stevefal on August 23, 2013, 01:53:01 AM
Guys listen, I think all RAW video development should be put on hold for now because I personally don't need it. I know this might be a controversial point of view, but if you'd stop making video so much, I'm sure you'd see things my way.

More seriously, dual ISO is awesome. It's perfect for shooting stills outside where one exposure sometimes doesn't have enough range but stuff is blowing around too much for an HDR bracket. I've already started using it during every shoot. This feature is infinitely more important to me than anything that involves video.

Fair point, but continued on another thread: http://www.magiclantern.fm/forum/index.php?topic=7839 (awaiting moderator approval)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 23, 2013, 02:19:41 AM
DUAL ISO concept is awesome indeed. It was THE reason I loaded the latest ML in my camera! However, I was taken aback by the pattern problem I reported (Reply #714). But that is why this is (a great) work in progress. I am also not interested in video but at the same time I respect others who want video too.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on August 23, 2013, 05:52:27 PM
Any chance of this on the 60D - can I help investigate?

*Bump* - I'd also really like to use this on my 60d, any tutorial for finding the required camera specific values, ideally w/o the risk of bricking it? Or is anyone else already on it with more experience like the people who also managed to put raw video on the 60d?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: n8ben on August 24, 2013, 03:16:56 PM
I just wanted to include my experience using Dual ISO somewhere for others that might be having the same struggle with the conversion to dng.

I am running Windows 8 64 bit which might be my problem :P.

I downloaded the cr2hdr zip folder from this post. I unzipped into the same folder as my dual ISO cr2 image file. I dragged the cr2 image onto cr2hdr and it launched, but it seemed to stop only part way through processing the file. It produced 2 files:

tmp.pgmCamera
tmp.txt

I searched for references to pgmcamera in an attempt to find a solution but found none, at first I thought I had the wrong copy of the .exe. In the end I had to run cr2hdr from the command prompt in order to produce a dng file I could use. After navigating to the correct folder just type:

cr2hdr.exe image.cr2

Where image.cr2 should be replaced with the name of the image you want to process.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on August 24, 2013, 06:50:48 PM
Folder issue? Keep the name short without spacing. Drag and drop produces dng-files.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: robinlee on August 24, 2013, 11:41:55 PM
Do you have dcraw.exe on the same folder as well?

R

I just wanted to include my experience using Dual ISO somewhere for others that might be having the same struggle with the conversion to dng.

I am running Windows 8 64 bit which might be my problem :P.

I downloaded the cr2hdr zip folder from this post. I unzipped into the same folder as my dual ISO cr2 image file. I dragged the cr2 image onto cr2hdr and it launched, but it seemed to stop only part way through processing the file. It produced 2 files:

tmp.pgmCamera
tmp.txt

I searched for references to pgmcamera in an attempt to find a solution but found none, at first I thought I had the wrong copy of the .exe. In the end I had to run cr2hdr from the command prompt in order to produce a dng file I could use. After navigating to the correct folder just type:

cr2hdr.exe image.cr2

Where image.cr2 should be replaced with the name of the image you want to process.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: aseek on August 25, 2013, 08:40:04 AM
I've been using dual iso on 50D on win 8 x64 for about a week, have not seen the green tint you guys showed in your samples. I use ML wb and correct in post

I've noticed the need to use some slight defringing.

When dragging multiple cr2s over cr2hdr.exe it only processes 30 or so images. Here's what happens afterwards: (http://i.imgur.com/uXHL3tO.png) (30 more of them get processed when re-attempted)

here are some sample dngs with the latest tragiclantern build (aug 19?) and cr2hdr (with no exif data, exiftool errors out for me):  https://www.dropbox.com/sh/wwmu3uuz6l1e7wd/4qazwI0KH1

I'm thrilled with dual iso, did not expect it to work so well on my aged 50D.

some edited results:
http://31.media.tumblr.com/2dae2151ca0235b14ede9500e55fc888/tumblr_mrtf6cWRtY1qculbmo1_1280.jpg
http://24.media.tumblr.com/899d4bf349e0f74d40cf4d28c7f71b40/tumblr_ms0xaypHSO1qculbmo1_r1_1280.jpg
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 26, 2013, 08:32:27 PM
https://www.dropbox.com/s/rjjpbcloe0dbfvl/IMG_0200test.CR2
hello, on the generated dng (from the above CR2) I also noticed that the repeatable pattern I reported can be observed not only in shadows but in mid tones as well. Generally it can be seen everywhere, it is just easier to see it in shadows. You may have to use 300% or 400% to see it but it is there  :(
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ed.jenner on August 26, 2013, 08:51:21 PM
I've seen the same thing on all my files.   Still gives nicer images in some situations than not using it though.

Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 26, 2013, 09:14:41 PM
It's caused by the bias towards more shadow details and less aliasing. The first cr2hdr was minimizing the noise, but people pointed out the aliasing issues very quickly. Instead of nice pictures in extreme lighting conditions, first samples were power lines, trees and dark keyboards...

Here's the lowest shadow noise that you can get from this shot: IMG_0200test_bright.dng (http://a1ex.magiclantern.fm/bleeding-edge/isoless/IMG_0200test_bright.dng)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 26, 2013, 09:42:26 PM
Could the cr2hdr.exe obtain an optional parameter switch that will allow the caller to choose the bias?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 26, 2013, 09:53:23 PM
I'd like something that chooses near-optimal settings every time. But I think I'll add something that outputs the components before blending (e.g. the ISO 100 half, the ISO 1600 half, the full-res image and so on).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 26, 2013, 09:56:41 PM
In contrast to the dng I sent I cannot open the bright dng either with CS4 or CS6.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 26, 2013, 10:21:59 PM
Re-uploaded, should be OK now.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 26, 2013, 10:28:15 PM
The new .dng  was a disaster. Pink or Magenta  instead of white everywhere! Maybe a WB problem but I am not sure that  it can be fixed (in contrast to the original .dng which was easily fixable)

However, I do not think I see these patterns now at the shadows!!!

But it is unusable. Even with patterns the previous was better  :(
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 26, 2013, 10:30:08 PM
In shadows?!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 26, 2013, 10:36:08 PM
In shadows?!
The new .dng  was a disaster. Pink or Magenta  instead of white everywhere! Maybe a WB problem but I am not sure that  it can be fixed (in contrast to the original .dng which was easily fixable)

However, I do not think I see these patterns now at the shadows!!!

But it is unusable. Even with patterns the previous was better  :(
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 26, 2013, 10:38:38 PM
... I've only showed you the theoretical limit for shadow noise and you complain about all sorts of stuff?!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 26, 2013, 10:42:44 PM
... I've only showed you the theoretical limit for shadow noise and you complain about all sorts of stuff?!
OK! I was afraid that you had changed the cr2exe !!!!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 26, 2013, 10:44:55 PM
But still very IMPRESSIVE. If only the changes you demonstrated could be applied to the shadows with no other effects Nikon would file for bankruptcy!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on August 26, 2013, 11:07:11 PM
LOGAN57 on the 7d thread uploaded a dual iso cr2 mac converter app. The thread can be found here http://www.magiclantern.fm/forum/index.php?topic=7752.msg68051;topicseen#msg68051
I put it up on google drive for anyone interested.

cr2hdr_mac

https://docs.google.com/file/d/0B4tCJMlOYfirUW1GRGw5bk02Q0E/edit?usp=sharing

*Actually the compiled stuff was from A.D here https://bitbucket.org/a_d_/magic-lantern/downloads/
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Greg on August 27, 2013, 02:16:31 PM
500D test :

ISO 100
(http://imageshack.us/a/img27/8718/l9i1.jpg)

DUAL ISO 100-1600
(http://imageshack.us/a/img836/9485/jr96.jpg)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a59192 on August 27, 2013, 04:22:23 PM
500D test :

ISO 100
DUAL ISO 100-1600


Will the 500D how to use Dual ISO?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 27, 2013, 05:17:31 PM
Updated cr2hdr (first post):

- better handling of shadow aliasing (thanks Ted for this resolution chart (http://www.magiclantern.fm/forum/index.php?topic=7139.msg67164#msg67164))
- robust line fitting when estimating ISO (median is doing wonders, fixes the duck from Danne (http://www.magiclantern.fm/forum/index.php?topic=7139.msg65662#msg65662) and some banding issues)
- fix some tricky overexposed shots (like the CR2 from akumiszcza (http://www.magiclantern.fm/forum/index.php?topic=7139.msg68042#msg68042))
- better handling of hot/cold pixels in some extreme situations (e.g. the CR2 from canon eos m (http://www.magiclantern.fm/forum/index.php?topic=7139.msg65236#msg65236)) - but still has some subtle artifacts
- reduced some shadow noise patterns (e.g. the CR2 from tron (http://www.magiclantern.fm/forum/index.php?topic=7139.msg69260#msg69260))

Also I've added some dynamic range analysis (estimated from how much noise you have in the optical black areas). The formula is just log2(full well capacity / noise stdev) and ignores the autocorrelation of the noise (which I don't know how to handle), so it's probably grossly inflated.

Sample before/after:
(http://acoutts.com/a1ex/100_1600-1750-2400_before.jpg) (http://acoutts.com/a1ex/100_1600-1750-2400_after.jpg)

If you still have problems with image quality, upload the CR2.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: noisyboy on August 27, 2013, 05:23:00 PM
That's a massive improvement dude! Cool  8)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 27, 2013, 05:26:10 PM
Alex thanks and congratulations! I will try to make some tests during the weekend.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 27, 2013, 05:39:07 PM
A1ex,

Seeing how much better color aliasing and moire is handled by tweaking the cr2hdr code, and also in my tests on the 5dmkII WITHOUT the VAF filter,
that dual iso image is better with aliasing compared to a Raw video and h264 video without vaf,

Would it be possible that you can FIX once and for all the issues with aliasing and moire that plagued these cameras by implementing this algorithms even without dual iso on the raw video image?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 27, 2013, 05:47:40 PM
hello, I run it against my CR2 (the one you have access to) and it crashed  :(

It initiated the Visual Studio debugger so I let it run and then captured the debugger screenshot that shows the offending instruction.

https://www.dropbox.com/s/2qtnuft59et5yfi/ch2hdr-27-8-crash.JPG

The last message printed was Looking for hot/cold pixels...

Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 27, 2013, 05:48:14 PM
I can only fix aliasing where there's some information about the missing pixels (and that's what I'm doing here).

For example: let's say the higher ISO is completely overexposed. This means the missing pixels are for sure higher than white level minus ISO difference (for 100/1600, that's white minus 4 stops). This info is essential for guessing what's there (it was the key for solving the palm tree shot).

In 5D2, there's no info about what might be in the skipped lines, so, according to Nyquist theorem, this is something that can't be fixed in post.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: AdamTheBoy on August 27, 2013, 05:54:49 PM
I love Dual ISO, it really can produce some stunning imagery!  I'm excited to test the newest processing method on raw video but I don't know if it's been adapted for video yet, is that further down the line?  Still, this is really awesome.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 27, 2013, 05:57:46 PM
I can only fix aliasing where there's some information about the missing pixels (and that's what I'm doing here).

For example: let's say the higher ISO is completely overexposed. This means the missing pixels are for sure higher than white level minus ISO difference (for 100/1600, that's white minus 4 stops). This info is essential for guessing what's there (it was the key for solving the palm tree shot).

In 5D2, there's no info about what might be in the skipped lines, so, according to Nyquist theorem, this is something that can't be fixed in post.

Is the pattern for skipping lines constant? I mean do we have the information on which lines from the sensor is being skipped? Is this not constant so you can apply an algorithm for resolving that missing line?

Maybe just maybe if you apply your dual iso algorithm tweaked a bit for  non dual iso shot that compensates not for an overexposure but the similar then maybe it would be better in guessing it than how the current debayer is handling it.

I will email to you a dng of full resolution chart properly framed without the vaf.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 27, 2013, 06:02:30 PM
@AdamTheBoy: not yet. Dirty trick: you can try to process the video with a tool that doesn't know about dual ISO, then drag the DNG files over cr2hdr. This may not work with the latest one, because it relies on the black bars (not present in raw video files), but with the older one (or cr2hdr_exp, look it up in the thread) should work just fine.

@ted: it reads one line, skips 2, and I don't know how to change it. I think it can be shifted somehow with ADTG registers, but this area is still a mistery.

To hide color aliasing where I can't recover it, I do some median filtering, but I believe all serious converters do it (dcraw does, I took the algorithm from there).

@tron: solved, download again.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 27, 2013, 06:20:11 PM
Solved indeed :) Many thanks.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: AdamTheBoy on August 27, 2013, 06:41:02 PM
https://www.dropbox.com/s/d3n7qd18i1b653b/M232154J.000378.JPG

https://www.dropbox.com/s/20mhvdfwbnfmfp1/M232154J.000378.dng

https://www.dropbox.com/s/j60135pvsx09bb8/M232154J.000380.dng

Here's a raw video still processed using the latest cr2hdr!  I'd say it's looking pretty damn good, I wanted to try a particularly alias prone shot to see how well the new techniques were working.  I can't wait to see where this goes.

I also put up an unprocessed and processed DNG to check out if you'd like.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: swinxx on August 27, 2013, 06:41:46 PM
can someone please post a link to a working conversion version for raw video, cause i can not make it work.. with that one i have.. and i can only find the newest one in the first post.
thx you so much. greets. sw
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 27, 2013, 06:48:55 PM
@a1ex,

Heres a test comparing aug 16 vs aug 27 crshdr.

(https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-prn2/1291846_364137963716994_324677966_o.jpg)

numbers correspond to numbered circles in attached image.

1. aug 27 better in handling fine frequency lines, better in handling aliasing and moire.

2. bright pixels popping out not shown in previous version

3. dark pixels/noise? popping out not present in previous.

Emailed to you cr2 of this image
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 27, 2013, 06:54:40 PM
Yeah, these dark pixels are easily mistaken for aliasing (and viceversa). Not that easy to find a good balance between the two.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 27, 2013, 07:07:06 PM
Here is the same set up but I used a VAF filter.

Used aug 16 build.

Obviously the VAF plays a key physical component in this approach, but those without it can go for either versions?

this version offers control over hot pixels, + vaf cuts down moire and aliasing.

Is there a middle ground?

(https://fbcdn-sphotos-h-a.akamaihd.net/hphotos-ak-ash4/1266495_364143190383138_129722111_o.jpg)

Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 27, 2013, 07:23:27 PM
For the troublesome image, here are the components that are being blended:

fullres (http://acoutts.com/a1ex/ted_fullres.dng) - best detail, least aliasing, but noisy in shadows
fullres_smooth (http://acoutts.com/a1ex/ted_fullres_smooth.dng) - the above, but with chroma filtered (used in highlights, where one ISO is overexposed)
halfres (http://acoutts.com/a1ex/ted_halfres.dng) - least noise, but aliased; not used directly
halfres_smooth (http://acoutts.com/a1ex/ted_halfres_smooth.dng) - the above, with chroma filtered; used in deep shadows if the algorithm thinks it's unlikely to have aliasing

The likelyhood of getting aliasing is computed from the difference between fullres and halfres images (and filtered a little).

In this image, the resolution chart detail is quite dark (close to the ISO 100 noise floor), so the algorithm chooses the less noisy version. Also, there's strong banding in the fullres shot (didn't see it that bad in the other shots). Also, the ISO 100 noise is a bit higher than normal (sensor got hot?)

At a closer look, the two exposures are not properly matched (ISO 100 shows a strong green cast in shadows), so the root of the problem is when matching the ISOs. The sensor response is probably nonlinear, and my algorithm assumes it's perfectly linear.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Greg on August 27, 2013, 08:29:53 PM

Will the 500D how to use Dual ISO?

You need to compile the module from the repository.


Canon dual iso without moire? ;)
http://image-sensors-world.blogspot.se/2013/08/canon-files-for-dual-range-column.html
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 27, 2013, 08:44:45 PM
If it has 20 stops of DR, I'm sold :D

I've tried to shoot a moonrise timelapse, and dual ISO struggled a bit. Didn't develop the shots yet, but I had to clip the moon in order to see the other stuff on the screen (not just pure black and street lamps).

@ted: I'm afraid the 5D2 has a nasty behavior in the optical black area. Here's the ISO 100 exposure after black correction:

(http://acoutts.com/a1ex/ted_5d2_dark)

The ISO 1600 exposure is fine, but it couldn't be matched with the ISO 100 one because of this.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 27, 2013, 09:22:54 PM
I think I prefer the half res smooth.

Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 27, 2013, 10:40:50 PM
I think I've solved it with some aggressive nonlinear correction. The two exposures are now adjusted at each bright raw level; before, I've used a gain (ISO) and an offset (black correction).

I've uploaded it to cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe), since it suffers from overfitting in some cases (e.g. Danne's duck).

BTW, even if it looked like a hot pixel problem, I didn't touch that code at all.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 27, 2013, 10:46:42 PM
BOOM!!

You nailed it A1ex!

Clean shadow areas, very minimal aliasing and moire!

If that can be achieved in raw video, I'd throw away the VAF filter!!

(https://fbcdn-sphotos-b-a.akamaihd.net/hphotos-ak-frc3/1262462_364215747042549_1577653041_o.jpg)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on August 27, 2013, 11:16:12 PM
Gotta try this out asap on some old files. Awesome!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 27, 2013, 11:26:11 PM
Danne's duck working again (a simple outlier check did the trick); all my pixel peeping tests passed (I have a script that runs it through 40 difficult test shots collected from the forum).

Also solved the problem from aseek (tested with 500 pics and still going strong).

=> updated cr2hdr from first post.

Title: Re: Dual ISO - massive dynamic range improvement
Post by: Shutterlab Creative on August 27, 2013, 11:48:44 PM
____URGENT____

Can someone PLEASE link me to where I can find the newest working download of CR2DNG? I was foolishly experimenting with this and then forgot to switch it off before a paid shoot. I have to process these photos tonight, Please help!

This will be for 5D3 CR2 files. I'm not sure if that makes a difference in which conversion version I will use.

Thanks for your help.

EDIT:

I've tried using the link to the CR2HDR.exe on the first page of this thread, but each time I drop the .CR2 files on the executable file command prompt opens very briefly and then nothing happens. What am I doing wrong? 
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on August 28, 2013, 12:16:50 AM
folder name, keep it without spaces
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Oedipax on August 28, 2013, 12:26:23 AM
cr2hdr is running great and my shots look even better with this new version. However, exiftool doesn't seem to be working. It's in the same folder as cr2hdr, do I need to do something different?

Here's how it looks (Windows 7 64bit running in VMware on OS X):

(http://i.imgur.com/h0h4OGq.png)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Shutterlab Creative on August 28, 2013, 12:36:07 AM
I have named my folder with the copied CR2 files "processing" without any spaces. It's created a blank text file (tmp.txt) but I can't get these to process correctly. Does CR2HDR create DNG copies of the CR2 files automatically?

I was using the July 22nd build for the firmware just fyi.

Title: Re: Dual ISO - massive dynamic range improvement
Post by: Oedipax on August 28, 2013, 12:38:00 AM
Shutterlab, are you dragging your cr2's over from another folder? You have to have cr2hdr, dcraw and exiftool in the same folder as the cr2 files you're dragging onto it for it to work.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Shutterlab Creative on August 28, 2013, 12:43:51 AM
Ahhh. I was hoping it was something simple. Thanks for the tip!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 28, 2013, 01:34:20 AM
Also on win7 make sure to unblock all of these programs, cr2hdr can't run exif tool if its restricted.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 03:27:50 AM
hello, I have noticed that in the specific CR2 I had sent although dark shadows do not show these repeatable patterns, I can see them in  mid tones (well actually less deep shadows). I have tested using the cr2hdr.exe from the first post and the _exp variance too.

(I must remind myself how the shadows would look with the Shadows slider close to maximum though in a normal Canon raw file though to keep things in perspective)

https://www.dropbox.com/s/7ilrw9bouph1t5u/cr2hdr_27aug_0708pm.JPG
https://www.dropbox.com/s/gvwfedjsujpx1rk/cr2hdr_28aug_1222am.JPG
https://www.dropbox.com/s/01osc2bj3ud8rka/cr2hdr_exp_28aug_0332am.JPG

I have uploaded the above screenshots (you have the .CR2) to see what I mean and for comparison...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Oedipax on August 28, 2013, 03:41:40 AM
Also on win7 make sure to unblock all of these programs, cr2hdr can't run exif tool if its restricted.

Not sure if I'm missing a step - I went into properties for exiftool just now and set "Full Control" for every account option I could find. I ran it as Administrator as well. Looking at that screenshot though it looks like it just can't find the file because something weird is happening with the name.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: ted ramasola on August 28, 2013, 03:47:16 AM
Not sure if I'm missing a step - I went into properties for exiftool just now and set "Full Control" for every account option I could find. I ran it as Administrator as well. Looking at that screenshot though it looks like it just can't find the file because something weird is happening with the name.

try to move the folder containing all the cr2 and the cr2hdr files is in the root of your drive not in another folder.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 28, 2013, 07:29:47 AM
@tron: this time you are oversharpening the picture. I don't see anything bad in my jpeg.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mk11174 on August 28, 2013, 07:49:20 AM
Can someone please add the 550D diff code to the commit?
Code: [Select]
diff -r ee7c743dfb14 modules/dual_iso/dual_iso.c
--- a/modules/dual_iso/dual_iso.c Thu Aug 08 23:03:02 2013 +0200
+++ b/modules/dual_iso/dual_iso.c Wed Aug 28 01:44:59 2013 -0400
@@ -91,6 +91,7 @@
 static int is_50d = 0;

 static int is_6d = 0;

 static int is_500d = 0;

+static int is_550d = 0;

 static int is_600d = 0;

 static int is_700d = 0;

 

@@ -742,6 +743,29 @@
         CMOS_FLAG_BITS = 3;

         CMOS_EXPECTED_FLAG = 0;

     }

+   else  if (   streq(camera_model_short, "550D")       )

+    { 

+               

+        FRAME_CMOS_ISO_START = 0x40695494; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)

+        FRAME_CMOS_ISO_COUNT =          6; // from ISO 100 to 3200

+        FRAME_CMOS_ISO_SIZE  =         30; // distance between ISO 100 and ISO 200 addresses, in bytes

+

+//  00 0000 406941E4  = 100

+//  00 0024 406941F6  = 200

+//  00 0048 40694208  = 400

+//  00 006C 4069421A  = 800

+//  00 0090 4069422C  = 1600

+//  00 00B4 4069423E  = 3200

+

+        PHOTO_CMOS_ISO_START = 0x406941E4; // CMOS register 0000 - for photo mode, ISO 100

+        PHOTO_CMOS_ISO_COUNT =          6; // from ISO 100 to 3200

+        PHOTO_CMOS_ISO_SIZE  =         18; // distance between ISO 100 and ISO 200 addresses, in bytes

+

+        CMOS_ISO_BITS = 3;

+        CMOS_FLAG_BITS = 2;

+        CMOS_EXPECTED_FLAG = 0;

+                is_550d = 1;   

+    }

     else if (streq(camera_model_short, "600D"))

     { 

         /*

Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 28, 2013, 07:52:09 AM
Can someone please add the 550D diff code to the commit?

Pull request?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mk11174 on August 28, 2013, 08:20:08 AM
Pull request?
Yes please, I dont know how to do that, sorry.  :-[
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 28, 2013, 08:34:22 AM
Yes please, I dont know how to do that, sorry.  :-[

It's very easy for small lines of code like this.  Give me a few mins and I'll do a quick tut.
It can all be done via web browser  :D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a.d. on August 28, 2013, 08:43:31 AM
@Oedipax
you may want to try my OSX release with gui? (https://bitbucket.org/a_d_/magic-lantern/downloads)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mk11174 on August 28, 2013, 08:43:55 AM
It's very easy for small lines of code like this.  Give me a few mins and I'll do a quick tut.
It can all be done via web browser  :D
cool, sounds good  :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Oedipax on August 28, 2013, 09:16:04 AM
@Oedipax
you may want to try my OSX release with gui? (https://bitbucket.org/a_d_/magic-lantern/downloads)

Thank you! Didn't know anyone was working on this.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on August 28, 2013, 09:27:16 AM
cool, sounds good  :)

Here you go :)

http://www.magiclantern.fm/forum/index.php?topic=7940.msg70958#msg70958

a1ex has already merged your code that I used to create the tutorial.
https://bitbucket.org/hudson/magic-lantern/commits/2a3faad36479e6bb576cd03ba10ec2a6649fe803
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 10:49:12 AM
@tron: this time you are oversharpening the picture. I don't see anything bad in my jpeg.
I hadn't  changed the sharpening in any case. I had just restored WB to auto (instead of as shot) to make the image realistic (to match the wall color).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mk11174 on August 28, 2013, 12:28:03 PM
Here you go :)

http://www.magiclantern.fm/forum/index.php?topic=7940.msg70958#msg70958

a1ex has already merged your code that I used to create the tutorial.
https://bitbucket.org/hudson/magic-lantern/commits/2a3faad36479e6bb576cd03ba10ec2a6649fe803
Ok, but still, thanks for the tut, at least it will help in future.  :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: f3000 on August 28, 2013, 12:56:10 PM
Maybe the hot dot remove algorithm is a bit aggressive and do harm to the shadow detail,can this be improved?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: pavelpp on August 28, 2013, 01:02:40 PM
Is dual ISO already supported in raw2dng GUI app for Mac?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: robinlee on August 28, 2013, 02:09:59 PM
During conversion, why the exiftool always say didn't work when I have them in every folder for conversion?

 ???
Title: Re: Dual ISO - massive dynamic range improvement
Post by: glubber on August 28, 2013, 04:30:04 PM
During conversion, why the exiftool always say didn't work when I have them in every folder for conversion?

 ???

In my case (Win7-64bit) the exiftool worked after doing the convertion in a rootfolder ("C:\DualIso" for example instead of "C:\Pictures\2013\2013-08-28").
Title: Re: Dual ISO - massive dynamic range improvement
Post by: X-RAY on August 28, 2013, 04:58:51 PM
Wow, the new CR2HDR is very nice for me.
I had one problematic file with magenta patterns in the shadows, but with the new version they're totally gone.
Not just the magenta but also those patterns in the dark. At this point it should be obvious, that we have to save the original CR2-Files for the future.

For demonstration.
The CR2-file:
https://dl.dropboxusercontent.com/u/35226956/DualIso/DUAL4390.CR2

This was processed with the old version (Shadows +100)
https://dl.dropboxusercontent.com/u/35226956/DualIso/DUAL4390_CR2HDR_OLD.jpg

This with the new version (same Settings)
https://dl.dropboxusercontent.com/u/35226956/DualIso/DUAL4390_CR2HDR_08_28.jpg

@ Alex
But there is a small issue. Is there a difference in the embedded Camera Profile? With the old CR2HDR it was "Adobe Standard" and with the new one "Embedded". Did you include something there?
The thing is...do you see this strange looking sun-corona? On the Adobe Standard-Profile this looked perfectly normal and on the embedded profile thats a little ugly. ;-)
But thats just a minor issue.
Nice work!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 28, 2013, 05:15:06 PM
I don't touch the exif info, except from copying it from the CR2. Try adding (or deleting) exiftool, to see if it makes any difference.

I also don't see anything strange in the sun, other than a tiny brightness change...

The image is quite oversharpened for my taste.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 05:52:25 PM
@alex: it is not oversharpened.

https://www.dropbox.com/s/cp954uq8f3reg0r/dual_iso_28-8-0810AM.JPG

I have used Adobe's defaults. You can see them in the screenshot.

I have used the latest cr2hdr (28-8-0810AM) but all the latest look alike.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 28, 2013, 06:04:28 PM
Updated cr2hdr again:

- small differences in shadow aliasing and hot pixel handling
  (7D test chart perfectly clean, 5D2 chart - which was more underexposed - almost there)
- should fix exiftool in long folders
- a little less aggressive with nonlinear corrections (not visible on my test shots, but should be less likely to get artifacts in tricky cases)

Before/after, 7D chart:
(http://acoutts.com/a1ex/100_1600-1750-2400_after.jpg) (http://acoutts.com/a1ex/100_1600-1750-2400_after2.jpg)

Before/after, 5D2 chart:
(http://acoutts.com/a1ex/100_1600_dots-2100-2500_before.jpg) (http://acoutts.com/a1ex/100_1600_dots-2100-2500_after.jpg)

(all compared with the first update from yesterday)

I chose not to fully correct the 5D2 chart (even if it's possible), because in very deep shadows, noise is easily mistaken for detail (and when it happens, it looks very ugly).

@tron: Adobe defaults have too much sharpening, look it up in the raw video section.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 06:44:36 PM
@a1ex:

https://www.dropbox.com/s/e9yfrvk1odyen41/dual_iso_28-8-0810AM-no-sharpening.JPG
https://www.dropbox.com/s/2ss9i46tb9touf5/dual_iso_28-8-722PM-no-sharpening.JPG

The above are with sharpening set to zero. You can see from the name the version of cr2hdr that was used.

Although Sharpening cannot always be as low as that I'd like your opinion on the above (regarding artifacts)

P.S Is that Canikon instead of the real camera name necessary ? Or you are using  it temporarily to distinguish between versions of the cr2hdr?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 28, 2013, 07:07:25 PM
Here's how it looks in ufraw. I don't see anything bad, a little noise is normal.

http://acoutts.com/a1ex/img0020test.png

Here, the camera name for your shot is Canon EOS 5D Mark II. Try changing it from exiftool and see if it makes a difference.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 07:21:42 PM
Here's how it looks in ufraw. I don't see anything bad, a little noise is normal.

http://acoutts.com/a1ex/img0020test.png

Here, the camera name for your shot is Canon EOS 5D Mark II. Try changing it from exiftool and see if it makes a difference.
It's the "Unique Camera Model" that is added to dng that has value "Canikon". It is displayed by Adobe Camera Raw as you can see in the screenshot that corresponds to the latest cr2hdr.exe
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 28, 2013, 07:31:57 PM
OK, change it and report.

exiftool -UniqueCameraModel="Nikanon" foo.dng
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 07:56:04 PM
OK, change it and report.

exiftool -UniqueCameraModel="Nikanon" foo.dng
Yes it was changed in ACR 8.1 It says: Camera Raw 8.1 - Nikanon

Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 28, 2013, 07:57:37 PM
Change it to the real thing, see if it makes any difference in output.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 08:01:13 PM
Change it to the real thing, see if it makes any difference in output.
You mean to set it to Canon Eos 5D Mark II

?? Actually it seems it has been added. So I was thinking to delete it!

But if you need another experiment I can set the value that is displayed by ACR with the original .CR2

EDIT: Done it displays the new value.

I delete it and it displayed: Digital Negative

I have saved the .dng however (no problem)

Why is this happening?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 28, 2013, 08:04:09 PM
To delete it: exiftool -UniqueCameraModel= foo.dng
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 08:09:30 PM
And ... the specific tag was added back...

The previous cr2hdr exe did not set this tag to Canikon though...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 08:10:04 PM
To delete it: exiftool -UniqueCameraModel= foo.dng
Yes, thanks I have deleted and it displayed: Digital Negative

Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 28, 2013, 08:12:09 PM
100% sure it did.

The question is: what value is best? There was a guy who said there are differences in output caused by this tag.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 08:14:13 PM
100% sure it did.
YES!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a.d. on August 28, 2013, 08:18:33 PM
cr2hdr from this morning still showed hot pixel in dark black area, but the latest cr2hdr works great. :D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: X-RAY on August 28, 2013, 08:19:17 PM
@ Alex
I tried to process my file through the old and the new cr2hdr on the exact same path directly under the root directory. Also with and without the exiftool.
But nothing changes ... in the old DNG are all camera profiles listed and in the new DNG only the "Embedded".

And the sun looks more pleasant on the "Adobe Standard" camera profile. Of course I know that a camera profile is just a "look" but I don't know if that was intended.

https://dl.dropboxusercontent.com/u/35226956/DualIso/DUAL4390_OLD.DNG
https://dl.dropboxusercontent.com/u/35226956/DualIso/DUAL4390_NEW.DNG
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 08:19:41 PM
100% sure it did.

The question is: what value is best? There was a guy who said there are differences in output caused by this tag.
I wonder: in the previous version had you just copied the camera model in this tag?

Do Adobe created DNGs have this tag or not?

Having the same value as the camera model however seems to display the same that ACR's displays for .CR2 files
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 28, 2013, 08:56:19 PM
Maybe it's this? http://www.magiclantern.fm/forum/index.php?topic=7139.msg62068#msg62068

@X-RAY: exiftool is not working for you.

Updated cr2hdr to set UniqueCameraModel to the real thing. Make sure exiftool is working.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 28, 2013, 09:45:31 PM
It works fine regarding UniqueCameraModel  :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Greg on August 28, 2013, 09:48:27 PM
Updated cr2hdr to set UniqueCameraModel to the real thing. Make sure exiftool is working.

(http://imageshack.us/a/img834/3085/cmfi.jpg)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Kuvaldoff on August 28, 2013, 11:48:34 PM
IMHO, the best way to open and convert DNG, is Luminance HDR - "Open HDR image".
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a59192 on August 29, 2013, 01:10:35 AM
You need to compile the module from the repository.


Canon dual iso without moire? ;)
http://image-sensors-world.blogspot.se/2013/08/canon-files-for-dual-range-column.html


Not quite understand ~? Refers (dual_iso.c) do? I can not find 500D available module (dual_iso.mo)? Consult advanced ...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 29, 2013, 04:56:50 AM
@A1ex: 

I took again the same picture DUAL_ISO 100/1600.  I used the Aug 28 10:07 PM cr2hdr.exe version.
I quite liked this time the result NO ARTIFACTS :) (even with default sharpening!)

DUAL ISO was not necessary for that specific image but that was what I could shoot late at night at home at 5:00 AM but this is only a test!!
 
Unfortunately, I noticed white and black pixels that did not exist in original CR2.

I have sent you the raw, dng and screenshot files.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 29, 2013, 12:20:33 PM
@alex: Any chance adding back black/white pixel fixing without changing anything else? (especially the latest aliasing tweaks which proved superb)  :)

In addition, if there are interconnected issues for example if fixing something could damage something else (I am speaking generally not necessarily for this feature) maybe cr2hdr could be made to accept switches for the users to choose.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 29, 2013, 12:42:43 PM
Yep, solved most of them.

A few of these white pixels are in the high-ISO images (and I only correct the ones from low ISO). With a lower setting (100/800 or 100/400) it shouldn't happen.

Also, the latest cr2hdr handles much better the files with missing or incorrect black correction info (e.g. raw video frames and those shots that were displaying a high value for "Black delta").

Here's a video frame from 6D, from nandoide (http://www.magiclantern.fm/forum/index.php?topic=7945.msg71110#msg71110):

Before / after solving black delta / after solving hot pixels, compared to cr2hdr from last week (since yesterday's one failed at black correction):

(http://acoutts.com/a1ex/M24-1127-600-50_before.jpg) (http://acoutts.com/a1ex/M24-1127-600-50_afterblack.jpg)  (http://acoutts.com/a1ex/M24-1127-600-50_after.jpg)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 29, 2013, 05:12:22 PM
@a1ex:

I have checked the latest cr2hdr (29-aug 1-17PM) and I have found sets of black and white pixels some of which look more complex (include color pixels):

https://www.dropbox.com/s/vsh84lmuihmqq1t/cr2hdr-29aug-117PM-1.JPG
https://www.dropbox.com/s/2kocv2kc6vup2p4/cr2hdr-29aug-117PM-2.JPG

EDIT: In the same DUAL0364.CR2 you already have
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 29, 2013, 05:24:47 PM
I'm afraid I can't solve these (they are not hot pixels, but large white patches). They are located where the high ISO is overexposed, so there's no hint whether they may be hot pixels or specular highlights.

That's what your sensor is outputting. Use less aggressive settings.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 29, 2013, 05:31:23 PM
What do you mean aggressive settings?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 29, 2013, 05:36:50 PM
You can compare with IMG_0363.CR2

Nothing aggressive...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 29, 2013, 05:40:50 PM
Something like this:

(http://i482.photobucket.com/albums/rr186/GinaDevon/AP-Cooling-System-03.jpg)

ISO 100/1600 is the absolute maximum on 5D2, how's that non-aggressive?! especially at 10 seconds of exposure...

Ever tried to do a long exposure at ISO 1600? how does it look?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 29, 2013, 05:44:30 PM
You imply hot pixels?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 29, 2013, 05:46:53 PM
I was looking at the previous .dng when I realized that I had shot at less than 1 sec.
I wonder how it will behave if I force long exposure noise reduction...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: arrinkiiii on August 29, 2013, 05:47:10 PM
What phot is that???  Aggressive !!! Me like it !!! It's water cooling ?!?!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 29, 2013, 05:54:52 PM
Yeah, from here: http://stargazerslounge.com/topic/146756-1100d-cold-finger-sensor-cooling-with-tec-and-water-cooling/

Long expo NR is worth trying, since it subtracts a dark frame.

Exiftool says 10 seconds for both 363 and 364.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 29, 2013, 05:58:00 PM
Something else: Temperature changed from 3100 (tint +3) to 4800 (tint -9) after conversion
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 29, 2013, 05:58:58 PM
I will do the same experiment tonight with long exposure NR  and I will let you know...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 29, 2013, 06:05:53 PM
Color matrix is hardcoded from 5D3. I thought exiftool will take care of it, but this info is not in the CR2 (it's hardcoded in dcraw).

Don't know an easy way to fix it. Maybe IliasG?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: arrinkiiii on August 29, 2013, 06:19:25 PM
***
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 29, 2013, 06:23:07 PM


Can anything be done for bad pixels like this?

Already post converted by raw2dng... originals are raw files like 500Mb-1GB

http://www.filedropper.com/m28-2304-000333

http://www.filedropper.com/m28-1958-000178
Title: Re: Dual ISO - massive dynamic range improvement
Post by: arrinkiiii on August 29, 2013, 06:32:57 PM
 

<3 
Title: Re: Dual ISO - massive dynamic range improvement
Post by: arrinkiiii on August 29, 2013, 06:42:53 PM
@a1ex

Impressive modding  :D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 29, 2013, 06:44:34 PM
Quote
Can anything be done for bad pixels like this?

Try extracting the interlaced DNGs with an older raw2dng (or delete the HDR code from it). Then, decompose them by enabling the debug flags in cr2hdr (those files with "dark.dng", "bright.dng", "fullres.dng" and so on).

If the hot pixels are not isolated, they are not corrected because they may be mistaken for aliasing. So, a hot pixel is only corrected if it's clearly brighter than its neighbours.

edit: I've just removed the old processing code from raw2dng, because:

- raw format is near end of life
- quality is much better with cr2hdr
- the cr2hdr workflow isn't that hard (extra step: select all, drag and drop)
- the algorithm is still improving, so it's easier to maintain a single way for doing things
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 29, 2013, 06:54:26 PM
The hot pixels show up as purple instead of red. It never corrected them (dual or not), esp the ones from FPS override. not sure why, they don't move and are very obvious. Also the color noise/banding in the shadows.

I'll have to try to get all of the files separated and see what is produced.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 29, 2013, 07:05:38 PM
There's no temporal analysis in the hot pixel algorithm, so they may not be that obvious mathematically.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 29, 2013, 07:11:05 PM
I'll have to find the worst offenders and break them up/post them. The noise is easier to get rid of than the hot pixels afterwards.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: rufustfirefly on August 29, 2013, 07:43:26 PM
I've been building Linux and Windows binaries for cr2hdr, as the code seems to be changing pretty often. They're available here, if anyone wants them, tagged with the date and Mercurial revision:

https://bitbucket.org/rufustfirefly/magic-lantern/downloads
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a59192 on August 29, 2013, 08:36:19 PM
I've been building Linux and Windows binaries for cr2hdr, as the code seems to be changing pretty often. They're available here, if anyone wants them, tagged with the date and Mercurial revision:

https://bitbucket.org/rufustfirefly/magic-lantern/downloads

Thanks! rufustfirefly sharing, very helpful to me.  :D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: swinxx on August 29, 2013, 11:24:48 PM
tried to convert some video dngs (son of a batch) in vmware windows 7 with the latest cr2hrd but got an error..
so what can i do now?

black subtrackt didn´t work
bright dark detection didnt work
iso blending didnt work..

greets. sw
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 29, 2013, 11:29:34 PM
You can read this: http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Francis on August 30, 2013, 04:55:34 AM
tried to convert some video dngs (son of a batch) in vmware windows 7 with the latest cr2hrd but got an error..
so what can i do now?

black subtrackt didn´t work
bright dark detection didnt work
iso blending didnt work..

greets. sw

Sounds like you are lacking dcraw.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 30, 2013, 07:54:01 AM
That would print dcraw didn't work or something like this.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: swinxx on August 30, 2013, 09:01:57 AM
i´m sorry alex.

i just recorded a dual iso enabled video raw file with the latest audionaut 5dmk3 build , then converted it with son of a batch onto my mac 10.8.4
then i had a directory with all the dng files onto my main hdd. with a vmware windows 7 machine i copied all the files into my desktop into a cr2 directory and then i selected those files (10 of them) and dragged them onto the latest cr2hrd.exe file..
with the cr2 photos it works..

thx. and sorry for my bad description, sw
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 30, 2013, 09:15:11 AM
Did you read my link? I know it's long, but... I have no idea what's going on. At least upload one DNG ;)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: swinxx on August 30, 2013, 09:54:38 AM
dear alex.
thx, i wil upload when i am at home.. (in a few hours.)

greetz.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: tron on August 30, 2013, 11:05:32 AM
@a1ex: Night experiment with same theme, exposure, lighting, shutter and aperture with Long Exposure NR set to ON.

No big changes. Strange patches continued to exist. Same image (no dual iso) with NR set to OFF at ISO 1600 at 10sec has very few and unobtrusive hot (of various colors) pixels.

Can we have a choice whether to activate white and/or black pixel fixing? 

P.S I can send you a CR2 of course but it is VERY similar.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: swinxx on August 30, 2013, 07:33:57 PM
here is the link to the dng sequence, and to the raw file.
i had no success to convert it to a normal dng seuquence because of the error (seen in the screenshot. ) thx

https://www.dropbox.com/sh/wzh6q2994ve7i6d/kKL5Whmuln
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 30, 2013, 07:49:03 PM
Okay, will fix.

Meanwhile, try a more aggressive setting (not 100/200).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Karmaschinken on August 30, 2013, 08:42:49 PM
Same thing here, can´t cr2hdr the dngs extracted from raw vid. Thanks a lot for fixing, Alex!  :-*
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Karmaschinken on August 30, 2013, 09:27:14 PM
Sorry, I suppose this is my mistake, obviously the DUAL ISO is not yet implemented for 5D2, right? Can´t wait to have it on my cam soon.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on August 30, 2013, 09:41:34 PM
Sorry, I suppose this is my mistake, obviously the DUAL ISO is not yet implemented for 5D2, right?

Dual ISO should work on the 5d2 for photos, what we're really waiting for is for it to reach the 60d :-p
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 30, 2013, 09:51:35 PM
Look in the ADTG thread ;)

(I actually took the first dual ISO photo today on my 60D with the ADTG GUI module)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on August 30, 2013, 10:05:32 PM
I actually took the first dual ISO photo today on my 60D with the ADTG GUI module

I didn't know this thread, but: Yippeeee!!! :-))) Thanks, it's really frustrating to see such a good feature ported everywhere else :-p
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Karmaschinken on August 30, 2013, 10:23:20 PM
 :'(

I bought a faster CF card some hours ago only to be able to record dual iso raw vids. I really love the style, I like the intro from Lars von Triers film "Melancholia", I would be very happy to be able to record footage that comes close to that.

Well, I suppose I will have to wait...  ::)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on August 31, 2013, 02:10:18 AM
alex, this could be an extreme request but, do you think its possible to combine pinkdotremover (from EOS M thread) with Dual ISO cr2hdr module? %1 just created Dual ISO fw for M, and works great, only issue is resolution is different than other cameras.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: daancalo2013 on August 31, 2013, 05:41:08 AM
Alex remarkable result in the latest update

http://www.flickr.com/photos/[email protected]/9631374125/
Title: Re: Dual ISO - massive dynamic range improvement
Post by: v8rrc on August 31, 2013, 06:41:31 AM
60D Dual ISO build https://skydrive.live.com/redir?resid=12771D90C09D1BCB!156&authkey=!AKIlr0pSWRB0uLs (https://skydrive.live.com/redir?resid=12771D90C09D1BCB!156&authkey=!AKIlr0pSWRB0uLs)
and Dual ISO.c https://skydrive.live.com/redir?resid=12771D90C09D1BCB!185&authkey=!ABf4aY3iIWdNzG8 (https://skydrive.live.com/redir?resid=12771D90C09D1BCB!185&authkey=!ABf4aY3iIWdNzG8)
Works on my 60D photo and video.  :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: telecastDaveRI on August 31, 2013, 08:00:35 AM
I saw this mentioned on this forum before and I just did a search for it, and cannot locate anything on it.
(It's possible that the posts were screenshots and that is why search cannot locate them).

I shot some raw video tonight using the dual ISO and then after creating seperate DNG files for each of them, and then dragging them into the newest release of Cr2hdr, I get a message that it "doesn't look like interlaced ISO" as it reviews each file.

Ideas? I thought I may have read somewhere it was because the scene could have been too dark.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 31, 2013, 08:44:04 AM
@daancalo2013: may I see that CR2?

@zuzukasuma: for video? try PinkDotRemover first, then cr2hdr.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 31, 2013, 09:14:33 AM
http://www.filedropper.com/dual0429_1

EOSM won't convert at all.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 31, 2013, 09:21:34 AM
That's not funny. There are 4 lines at one ISO, 4 at another...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: sletts02 on August 31, 2013, 09:35:58 AM
I tried processing a RAW video, 5D3, Dual ISO 1600/25600 and it doesn't seem to have worked using raw2dng

Here is a single DNG frame https://www.dropbox.com/s/bzxlo7zkkcdemcs/000000.dng

Have I done something wrong?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 31, 2013, 09:39:43 AM
See the first post.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on August 31, 2013, 09:51:01 AM
Experimental cr2hdr that fixes the DNGs from swinxx: cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe)

Can't run my pixel peeping tests right now, so... if you find files that are recognized by cr2hdr from first post, but not with this one, let me know (upload the CR2 or the interlaced DNG).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on August 31, 2013, 10:02:37 AM
Really nice to see improvements on videofiles. Any chance getting A.D,s very nice mac app gui working with video dng,s?
//D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 31, 2013, 10:47:00 AM
Quote
That's not funny. There are 4 lines at one ISO, 4 at another...

This is what it made. Maybe problems?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on August 31, 2013, 12:21:25 PM
60D Dual ISO build

Great, though a bitbucket pull request is preferred - here's at least a diff for easier merging:

Code: [Select]
--- dual_iso.c.old 2013-08-31 12:14:58.407095000 +0200
+++ dual_iso.c 2013-08-31 12:34:04.478646500 +0200
@@ -90,6 +90,7 @@
 static int is_5d2 = 0;
 static int is_50d = 0;
 static int is_6d = 0;
+static int is_60d = 0;
 static int is_500d = 0;
 static int is_550d = 0;
 static int is_600d = 0;
@@ -743,6 +744,30 @@
         CMOS_FLAG_BITS = 3;
         CMOS_EXPECTED_FLAG = 4;
     }
+    else if (streq(camera_model_short, "60D"))
+    { 
+        /*
+        100 - 0
+        200 - 0x024
+        400 - 0x048
+        800 - 0x06c
+        1600 -0x090
+        3200 -0x0b4
+        */
+        is_60d = 1;   
+
+        FRAME_CMOS_ISO_START = 0x407458fc; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)
+        FRAME_CMOS_ISO_COUNT =          6; // from ISO 100 to 25600
+        FRAME_CMOS_ISO_SIZE  =         30; // distance between ISO 100 and ISO 200 addresses, in bytes
+
+        PHOTO_CMOS_ISO_START = 0x4074464c; // CMOS register 0000 - for photo mode, ISO 100
+        PHOTO_CMOS_ISO_COUNT =          6; // from ISO 100 to 12800
+        PHOTO_CMOS_ISO_SIZE  =         18; // distance between ISO 100 and ISO 200 addresses, in bytes
+
+        CMOS_ISO_BITS = 3;
+        CMOS_FLAG_BITS = 2;
+        CMOS_EXPECTED_FLAG = 0;
+    }
     else if (streq(camera_model_short, "500D"))
     { 
         is_500d = 1;   
Title: Re: Dual ISO - massive dynamic range improvement
Post by: v8rrc on August 31, 2013, 12:44:27 PM
I'm getting stripes in some areas without much detail.
http://sdrv.ms/14g00M9 (http://sdrv.ms/14g00M9)



Great, though a bitbucket pull request is preferred - here's at least a diff fine for easier merging with new dual_iso.c files from trunk:

Code: [Select]
--- ml-trunk/modules/dual_iso/dual_iso.c 2013-08-31 12:14:58.407095000 +0200
+++ ml-trunk/modules/dual_iso/dual_iso.c.new 2013-08-31 12:10:41.227385100 +0200
@@ -743,6 +732,30 @@
         CMOS_FLAG_BITS = 3;
         CMOS_EXPECTED_FLAG = 4;
     }
+    else if (streq(camera_model_short, "60D"))
+    { 
+        /*
+        100 - 0
+        200 - 0x024
+        400 - 0x048
+        800 - 0x06c
+        1600 -0x090
+        3200 -0x0b4
+        */
+        is_60d = 1;   
+
+        FRAME_CMOS_ISO_START = 0x407458fc; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)
+        FRAME_CMOS_ISO_COUNT =          6; // from ISO 100 to 25600
+        FRAME_CMOS_ISO_SIZE  =         30; // distance between ISO 100 and ISO 200 addresses, in bytes
+
+        PHOTO_CMOS_ISO_START = 0x4074464c; // CMOS register 0000 - for photo mode, ISO 100
+        PHOTO_CMOS_ISO_COUNT =          6; // from ISO 100 to 12800
+        PHOTO_CMOS_ISO_SIZE  =         18; // distance between ISO 100 and ISO 200 addresses, in bytes
+
+        CMOS_ISO_BITS = 3;
+        CMOS_FLAG_BITS = 2;
+        CMOS_EXPECTED_FLAG = 0;
+    }
     else if (streq(camera_model_short, "500D"))
     { 
         is_500d = 1;   

Maybe Alex or someone should give it a quick check over to make sure I got it right.  :)

EDIT: Just submitted a pull request
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on August 31, 2013, 05:38:38 PM
I'm getting stripes in some areas without much detail.

With the test shots I just did I didn't see this, but I probably just was lucky and the assembly worked for me. It does produce artifacts though when merging a high iso shot due to the different noise patterns. In general the 60d works 100% with this module as far as I can tell - thanks!

1. Request: dual iso should really be combined with an ettr/ettl option to expose the first image to one side and then automatically figure out the required ev spacing for the 2nd shot, in essence minimizing the required dual_iso ev spacing. Currently you have to decide for a + or - ev variant and then guess the ev value, I don't see a reason why ml shouldn't be able to do this (as usual with an option about how many clipped shadows & blown highlights are acceptable).

2. Request: The "alternate shot" mode should be amended with an option not to just take the base image, but also as a 2 shot bracket the higher iso shot so the user can decide later if the ml composite shot is necessary or if a "real" hdr assembly is possible (i.e. static scene) or even necessary due to dual_iso artifacts.

3. Bug: Alternate frame doesn't work when holding the shutter pressed with continuous drive (yes, I know it probably won't be possible to intercept Canon here, but maybe a warning about this is a good idea).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a59192 on August 31, 2013, 06:53:01 PM
(http://farm4.staticflickr.com/3672/9635733579_2d6fbe13e7_h.jpg)

Thank ML team effort! Compared to the initial cr2hdr stripes, now restored cr2hdr perfect lot!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: frogcement on August 31, 2013, 09:26:42 PM
60D Dual ISO build https://skydrive.live.com/redir?resid=12771D90C09D1BCB!156&authkey=!AKIlr0pSWRB0uLs (https://skydrive.live.com/redir?resid=12771D90C09D1BCB!156&authkey=!AKIlr0pSWRB0uLs)
and Dual ISO.c https://skydrive.live.com/redir?resid=12771D90C09D1BCB!185&authkey=!ABf4aY3iIWdNzG8 (https://skydrive.live.com/redir?resid=12771D90C09D1BCB!185&authkey=!ABf4aY3iIWdNzG8)
Works on my 60D photo and video.  :)

After reading the dual iso pdf, I thought this module was only able to work on 5d3 && 7d because of the hardware, so how does this now work on the xxD cameras that only have the 4 channel readout vs. the 8 channel readout of the 5d3 && 7d? && does this work on xxxD as well? I'm soon to get another aps-c body and was going to get another 7d but if this will work on xxD && xxxD I may well get a rebel. TIA.
--
K.C.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Rewind on August 31, 2013, 09:29:56 PM
Quote
...but if this will work on xxD && xxxD I may well get a rebel.

For example, dual ISO works fine on t2i both in photo and video modes ;)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 31, 2013, 09:31:49 PM
I think its ported to almost every camera, some don't do it in LV tho.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: frogcement on August 31, 2013, 10:08:25 PM
I think its ported to almost every camera, some don't do it in LV tho.
That's quite amazing, considering only 2 maybe 3 cameras have the 8 channel sensor, the 5d3, 7d and the ?6d? I'm not sure about the 6d, I'm just wondering how its done on cams that only have the 4 channel sensor. Amazing amazing work !!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 31, 2013, 11:27:31 PM
The register has 2 isos in it.. maybe on a 4 channel it uses 2 and 2, on an 8, 4 + 4?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on August 31, 2013, 11:40:43 PM
The register has 2 isos in it.. maybe on a 4 channel it uses 2 and 2, on an 8, 4 + 4?

yeah, I remember sth like that on cr2hdr.exe built
Title: Re: Dual ISO - massive dynamic range improvement
Post by: frogcement on August 31, 2013, 11:41:26 PM
The register has 2 isos in it.. maybe on a 4 channel it uses 2 and 2, on an 8, 4 + 4?
I wonder if this means the 8 channel sensors may be able to do "Quad ISO" that would be astounding,
wonder if anyone's looked into this.... Very intriguing .... Thanks 1%.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Karmaschinken on August 31, 2013, 11:53:24 PM
Ummm, just can´t resist to ask because I am too curious. What are the reasons this feature can be implemented in XXD and XXXD- cams but not on the 5D2 yet? It all started with a 5D2, not? I hope my good old horse is not too outdated for the newest features coming up here?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on August 31, 2013, 11:53:55 PM
Probably no quad iso but some other regs do interesting things.

5DII does it, just not in LV.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Karmaschinken on August 31, 2013, 11:55:36 PM
Yes, I mean dual iso video. I suppose if not available in LV, it´s not available for video, or did I miss something?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on August 31, 2013, 11:55:56 PM
Ummm, just can´t resist to ask because I am too curious. What are the reasons this feature can be implemented in XXD and XXXD- cams but not on the 5D2 yet? It all started with a 5D2, not? I hope my good old horse is not too outdated for the newest features coming up here?

no, if you read the pdf a1ex prepared on the 1st page of this thread, it all started with 5D Mark III & 7D because of they have dual CPUs
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 01, 2013, 12:03:21 AM
Yes, I mean dual iso video. I suppose if not available in LV, it´s not available for video, or did I miss something?

its available on 550D but resolution is terrible already and my other cam, EOS M has dots all over the frames. its definitely worth to have more dynamic range than RAW on video
Title: Re: Dual ISO - massive dynamic range improvement
Post by: frogcement on September 01, 2013, 12:08:51 AM
no, if you read the pdf a1ex prepared on the 1st page of this thread, it all started with 5D Mark III & 7D because of they have dual CPUs
Not so, here's the quote from the PDF ...... Re-read it, it says....

Unfortunately, most other Canon cameras follow a different pattern for the CMOS #0 register, without any obviously duplicate fields. As we know from Canon specs, most cameras have a 4-channel sensor readout, except for 5D Mark III and 7D which have a 8-channel readout.

8channel readout, is the readout from the sensor, has nothing to do with how many cpu's said cam has, Afaik, the 5d3 doesn't have dual digic like the 7d
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 01, 2013, 12:17:10 AM
Not so, here's the quote from the PDF ...... Re-read it, it says....

Unfortunately, most other Canon cameras follow a different pattern for the CMOS #0 register, without any obviously duplicate fields. As we know from Canon specs, most cameras have a 4-channel sensor readout, except for 5D Mark III and 7D which have a 8-channel readout.

8channel readout, is the readout from the sensor, has nothing to do with how many cpu's said cam has, Afaik, the 5d3 doesn't have dual digic like the 7d

sorry, confused it with 1D X. obviously digic 5"+" processor can handle more threads than digic5 normal
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Karmaschinken on September 01, 2013, 12:19:27 AM
... it all started with 5D Mark III & 7D ...

First ML ran on 5D2, I have here ML 0.16 or so... So does it mean dual iso will definitely not be available on the 5D2?  : ???
Title: Re: Dual ISO - massive dynamic range improvement
Post by: frogcement on September 01, 2013, 12:26:54 AM
First ML ran on 5D2, I have here ML 0.16 or so... So does it mean dual iso will definitely not be available on the 5D2?  : ???

1%  said just a few posts ago, dual iso works on 5d2 just not in live view, I've personally never used live view for dual iso, I rarely use lv at all.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: frogcement on September 01, 2013, 12:28:37 AM
Probably no quad iso but some other regs do interesting things.

5DII does it, just not in LV.

1%, care to elaborate, pm is fine so we don't muck up the thread anymore idc either way. Ty
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Karmaschinken on September 01, 2013, 12:34:24 AM
1%  said just a few posts ago, dual iso works on 5d2 just not in live view, I've personally never used live view for dual iso, I rarely use lv at all.

I am asking for dual iso video! So if dual iso is not available in LV I was only wondering if dual iso video turned out to be impossible at all on the 5D2. I think I am not the only one being very keen on that feature.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 01, 2013, 12:42:45 AM
5DII/50D don't change the register while in LV so no video, probably ever. For photos it works. 550D/600D+ should do LV. 7D not sure, don't have it yet to check.

Title: Re: Dual ISO - massive dynamic range improvement
Post by: Karmaschinken on September 01, 2013, 01:25:34 AM
Booooom! That´s bad news. Ok, so I have to save money for a MK3. Thanks nevertheless for clearing this question.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: kaos42ze on September 01, 2013, 03:58:08 AM
how to get Dual ISO working int he EOS-M ?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: N/A on September 01, 2013, 08:14:16 AM
600D, its not perfect. I saw some artifacts.

http://www.filedropper.com/600ddualiso
This link is dead but I have the zip, mind if I re-up it so I can update the 600D raw thread?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 01, 2013, 05:08:37 PM
Yep, go ahead and repost it, I don't think I made a newer one yet.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: nanomad on September 01, 2013, 05:57:41 PM
Uhm, bad news for the 650D/700D/100D/EOSM owners I think...

The dual iso pattern is dBdB which probably means we'll loose some information due to how the bayer pattern is designed

@a1ex, can you confirm this is indeed bad?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 01, 2013, 06:18:11 PM
or ddddBBBBdddd

There is a hardcoded 5DIII color matrix in the converter? Why?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: N/A on September 01, 2013, 06:30:08 PM
Yep, go ahead and repost it, I don't think I made a newer one yet.
Cool, thanks.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 01, 2013, 06:34:34 PM
There is a hardcoded 5DIII color matrix in the converter? Why?

Color matrix is hardcoded from 5D3. I thought exiftool will take care of it, but this info is not in the CR2 (it's hardcoded in dcraw).

Don't know an easy way to fix it. Maybe IliasG?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 01, 2013, 06:35:57 PM
Can't we just use the correct color matrix based on camera name?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 01, 2013, 06:37:31 PM
Of course, but there's a lot to type :D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 01, 2013, 06:51:07 PM
Ok, so that is the only issue then.... we can now set line pattern on EOSM/700D/650D/etc
Title: Re: Dual ISO - massive dynamic range improvement
Post by: daancalo2013 on September 01, 2013, 07:01:27 PM
@daancalo2013: may I see that CR2?

http://we.tl/BdGglxq1sO
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 01, 2013, 07:24:20 PM
a1ex, we have a problem.

when you shoot sky (or a scene with really low contrast and no obvious details) using dual ISO (no point but I've tried to create dual iso timelapse), cr2hdr.exe doesn't recognize the pattern of the shots. I'm now on a low bandwidth, but you can try it out yourself.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 01, 2013, 07:36:54 PM
Also what about this: dual iso + shutter HDR flipping for raw video.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 01, 2013, 07:46:50 PM
Dual shutter might be possible (look in the ADTG thread), so really extreme HDR could become a piece of cake.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 01, 2013, 07:51:09 PM
I fixed the "advanced" hdr mode so while its not one exposure, you could do dual iso + switch the shutter every other frame. The workflow might be weird but i wonder what it would make at this point.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: engardeknave on September 01, 2013, 07:55:41 PM
.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: nanomad on September 01, 2013, 07:56:14 PM
For the record, the line skip fix on the 650D / EOSM and other cameras should be:

Code: [Select]
raw &= 0x1FF
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 01, 2013, 07:57:08 PM
Quote
I fixed the "advanced" hdr mode so while its not one exposure, you could do dual iso + switch the shutter every other frame. The workflow might be weird but i wonder what it would make at this point.
Yes, but you will get both motion artifacts and aliasing. You could have just one of them with shutter HDR only ;)

The workflow is tricky because I'm not sure how to get interframe to work on raw data.
Title: Nit: .exe should be .zip? [Re: Dual ISO - massive dynamic range improvement]
Post by: l_d_allan on September 02, 2013, 04:36:23 AM
Postprocessing
- For CR2 files: cr2hdr.c (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso/cr2hdr.c) (Windows:cr2hdr.exe (http://acoutts.com/a1ex/cr2hdr.zip))   [ [.zip??? ] ]. It requires dcraw and (optional) exiftool in your executable path (they are in the zip). Drag and drop the CR2 files over cr2hdr.exe.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mk11174 on September 03, 2013, 04:26:51 AM
Anyone have any movie mode issues with 600D not working with Dual_Iso?

Photo mode works great and shows preview after shot but before the new API 3.0 I was able to get a realtime preview in Movie Mode?

Still works on my 550D, it is just the 600D I am having issues with with this new API version?

I tried both Raw_REC modules from main repo and Tragic-Lantern both same results, just not working in Movie Mode now.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: vstrglv on September 03, 2013, 06:09:16 PM
There is a problem in 192 frame RAW video file. 5D3, iso 800/3200
I have converted raw to dng with raw2dng.exe on  ‎August ‎29, ‎2013, ‏‎8:49:50 PM, 254 KB (261,102 bytes). All dng files became interlaced.
After cr2hdr.exe on  ‎August ‎29, ‎2013, ‏‎1:17:52 PM, 284 KB (291,563 bytes) first 130 frames stay interlaced with error
Code: [Select]
W:\cr2hdr1\1>cr2hdr.exe 000131.DNG

Input file     : 000131.DNG
Full size      : 1920 x 1080
Active area    : 1920 x 1080
White level    : 12400
Black subtract didn't work
ISO pattern    : ddBd RGGB
Bright/dark detection error
ISO blending didn't work
Frames from 132 until 192 are good - not interlaced with
Code: [Select]
W:\cr2hdr1\1>cr2hdr.exe 000132.DNG

Input file     : 000132.DNG
Full size      : 1920 x 1080
Active area    : 1920 x 1080
White level    : 12400
Black subtract didn't work
ISO pattern    : dBBd RGGB
Noise levels   : 8.00 8.00 8.00 8.00 (14-bit)
Estimating ISO difference...
ISO difference : 1.00 EV (200)
Black delta    : 20.10
Interpolation  : mean23-chroma5x5-alias
Dynamic range  : 10.34 (+) 10.34 => 11.34 EV (in theory)
Matching brightness...
Looking for hot/cold pixels...
Hot pixels     : 9
Full-res reconstruction...
ISO overlap    : 6.3 EV (approx)
Half-res blending...
Chroma filtering...
Building alias map...
Filtering alias map...
Smoothing alias map...
Noise level    : 8.00 (16-bit)
Dynamic range  : 12.34 EV (cooked)
Output file    : 000132.DNG
    1 image files updated
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 03, 2013, 06:15:41 PM
Try cr2hdr_exp (scroll up) and upload a DNG.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 03, 2013, 06:49:04 PM
Yep, someone broke it... like AE/Value +state using the LV only value.

6D values were copied for LV... dunno why. Bad merge? And then I merged it from main:

Code: [Select]
               
        FRAME_CMOS_ISO_START = 0x406957C8; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)
        FRAME_CMOS_ISO_COUNT =          6; // from ISO 100 to 25600
        FRAME_CMOS_ISO_SIZE  =         30; // distance between ISO 100 and ISO 200 addresses, in bytes
Title: Re: Dual ISO - massive dynamic range improvement
Post by: vstrglv on September 03, 2013, 06:52:09 PM
Same result with cr2hdr_exp.
http://dfiles.ru/files/r3fdt4fv9  good file

http://dfiles.ru/files/oszztsuk1   bad file
Title: Re: Dual ISO - massive dynamic range improvement
Post by: doctormord on September 03, 2013, 06:52:24 PM
Dual shutter would be awesome.

What i got in mind some days ago is, "long expo at real time" which i meant to be like so:

Shutterspeed: 1/30 splitted to 1/n lines. I.e.

odd line: 1/15
even line 1/15

So you may record video at 30fps mit 1/15 shutterspeed by interlacing odd and even lines. Setting up a 1/4 (4 lines) would give a 1/7.5.. :D

Postproduction could be a wired thing then.

Just thougt..
Title: Re: Dual ISO - massive dynamic range improvement
Post by: vstrglv on September 03, 2013, 07:01:02 PM
Same issue with another file. 736 frames total - frames from 394frame until 606 frame are bad (stay interlaced)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 03, 2013, 07:12:58 PM
131 works for me with cr2hdr_exp.exe...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mk11174 on September 03, 2013, 07:32:42 PM
Yep, someone broke it... like AE/Value +state using the LV only value.

6D values were copied for LV... dunno why. Bad merge? And then I merged it from main:

Code: [Select]
               
        FRAME_CMOS_ISO_START = 0x406957C8; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)
        FRAME_CMOS_ISO_COUNT =          6; // from ISO 100 to 25600
        FRAME_CMOS_ISO_SIZE  =         30; // distance between ISO 100 and ISO 200 addresses, in bytes
Man, I should of checked the darn addresses myself, der! 8] Thanks 1%, just never thought the address would ever accidently get changed.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: vstrglv on September 03, 2013, 07:49:22 PM
Alex, please check this one http://dfiles.ru/files/3yiefe6rq
131 works for me too with cr2hdr_exp.exe. Maybe I was wrong
With cr2hdr_exp.exe. only files from 0 until 28 are bad
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 03, 2013, 08:23:03 PM
Solved and updated cr2hdr from first post (since all pixel peeping tests passed).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: vstrglv on September 03, 2013, 08:48:34 PM
It works. Thank you very much!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 04, 2013, 03:16:44 AM
How was banding on 5DIII solved? It looks like there is a pattern to this noise.

(http://i.imgur.com/UF1Sx8Ol.jpg)

Took some of it in crop where its the worst with lens cap on.

Zoom (Folder Link)
https://mega.co.nz/#F!w0ICgKxA!JhgB4SXMYBJI59KIJw5rOg
Title: Re: Dual ISO - massive dynamic range improvement
Post by: lomka on September 04, 2013, 04:12:10 AM
can somebody explain to me what a dual shutter is please?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 04, 2013, 08:07:37 AM
Can you upload one DNG? I've downloaded one of the raws, but it's just a dark frame...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: enliten on September 04, 2013, 10:13:15 AM
Yes, but you will get both motion artifacts and aliasing. You could have just one of them with shutter HDR only ;)

I asked this much earlier in the thread and it got buried. Again I'm not sure if I'm understanding correctly but:

are we talking about taking 2 frames with alternating ISO locations and combining them in post?

i.e. frame 1 has odd rows as iso 100, even as 1600
then frame 2 has odd rows as iso 1600, even as 100

Would this work? would we gain more detail from this method? or would you be better off just bracketing ISO?

Cheers,

Ben
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 04, 2013, 10:17:00 AM
This would work, but the math for matching the frames is an order of magnitude more complex than what I did here. So... good luck!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 04, 2013, 01:32:18 PM
I asked this much earlier in the thread and it got buried. Again I'm not sure if I'm understanding correctly but:

are we talking about taking 2 frames with alternating ISO locations and combining them in post?

i.e. frame 1 has odd rows as iso 100, even as 1600
then frame 2 has odd rows as iso 1600, even as 100

Would this work? would we gain more detail from this method? or would you be better off just bracketing ISO?

Cheers,

Ben


read this Ben, http://acoutts.com/a1ex/dual_iso.pdf
Title: Re: Dual ISO - massive dynamic range improvement
Post by: enliten on September 04, 2013, 01:54:04 PM
I have read that, I understand the basics of the dual ISO and used it in the few days after it was first released.

What I am asking is: Could the process of taking two separate image be automated, whereby one image has alternating rows of low ISO then high ISO, followed by the second image that has alternating rows of high ISO then low ISO.

Almost like a HDR, but with dual ISO and with two images rather than one. The benefit would be that some detail would be retained in the shadows and highlights as there would be enough information to fill the blank areas. The downside being that if there is any movement in the frame (like a standard HDR) you're likely to get movement / ghosting. The difference is that the ghosting would be interlaced, rather than "solid"

I don't know if I have explained this succinctly, please let me know if I need to clarify more.

-Ben
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 04, 2013, 02:01:38 PM
For video it could work.

For photo, it completely misses the point, since you will get better results with a simple 2-frame bracketing.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: enliten on September 04, 2013, 02:14:49 PM
For video it could work.

For photo, it completely misses the point, since you will get better results with a simple 2-frame bracketing.

So recording video at 60 fps and then merging frames to 30 in post?

Does the current HDR use the raw buffer? I'm just thinking that if you're capturing two consecutive frames, i.e. silent picture / raw video style, those images are going to be closer together in terms of time. Better for moving subjects (less ghosting). Excuse my ignorance on the topic.

-Ben
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 04, 2013, 02:18:59 PM
Advice: try playing with it first, watch the video from the user guide, read my previous posts...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: enliten on September 04, 2013, 02:22:57 PM
k, thanks for the quick responses :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: bkummel on September 04, 2013, 03:16:20 PM
Hi,

I have been playing around with dual ISO on my 7D. I was able to shoot some pics and convert them to DNG with cr2hdr. However, it seems the DNG-files created by cr2hdr aren't completely well-formed. The Preview app on my Mac, as well as iPhoto and Aperture should be able to open or import DNG-files. However, it turns out they can't read the files created by cr2hdr. From what I've read on the forum, I understand that Lightroom also isn't able to open the files, but Photoshop is. (But I don't have the Adobe apps, so I can't confirm that.) I've also tried GIMP, but that didn't work either.

I tried converting to JPG with ufraw-batch, but that gives very dark images with no details in the shadows. Besides, when converting to JPG, I loose the dynamic range, so that isn't an option anyway. I'd like to import the DNG's in Aperture, to be able to store them in Aperture's library along with all my other CR2's and edit them in my normal workflow.

Does anyone know a way to get Dual ISO images into Aperture without losing the RAW-data? Or is there any chance someone would be able to fix cr2hdr, so that it creates well-formed DNG's. (Note that you don't need Aperture or iPhoto to test. The Preview-app, which is part of OS X, uses the same RAW-reading technology as the other Apple-apps.)

PS.
I even tried to compile cr2hdr myself yesterday, with the latest sources from Bitbucket. I managed to compile it and it works, but gives the same unusable DNG-files.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 04, 2013, 03:17:00 PM
Here is the folder with images with content + dark frames for non crop mode.

In dual ISO the 100 will be clean and the 1600/3200/6400 will be like this.

nonZoom (Folder Link)
https://mega.co.nz/#F!UwgxkBwb!Y4BtsijIZU06FGE-XSRJJQ

Title: Re: Dual ISO - massive dynamic range improvement
Post by: 3rdwave on September 04, 2013, 03:26:18 PM
Hello everyone!
I seem to be running into an issue with the DNG's becoming blue or pink after processing through RawMagic.
(http://www.3rdwavenow.com/DNG_Sample.jpg)
Any idea what is causing this? Is this the way I'm processing or my capture?
I'm using a 60D with the most recent build.

Cheers! :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Kuvaldoff on September 04, 2013, 05:08:59 PM
...
I tried converting to JPG with ufraw-batch, but that gives very dark images with no details in the shadows. Besides, when converting to JPG, I loose the dynamic range, so that isn't an option anyway.
...
Try Darktable or Luminance HDR. IMHO, Luminance works with dynamic range better. I came to the conclusion that it is better to use converters, based on dcraw or libraw.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 04, 2013, 05:52:17 PM
Now it detects camera model:

https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/downloads/cr2hdr.exe
Title: Re: Dual ISO - massive dynamic range improvement
Post by: nanomad on September 04, 2013, 08:50:14 PM
It's still not polished up though. But feel free to use it and report any bugs you find. It will be merged in the main tree soon
Title: Re: Dual ISO - massive dynamic range improvement
Post by: martie on September 05, 2013, 12:41:30 PM
Is there any quick way to filter for dual ISO raw pictures in Lightroom? I tried the ISO meta data, but there seems to be only one ISO value for Dual ISO pictures, so this won't work. (Btw, I think would be nice feature if the two ISO values would be stored instead of just one.)

Also, I would like to know it is possible to give the output file name for cr2hdr as parameter. If a file is already converted to dng, it is overwritten with the output of cr2hdr, which probably is not always desired. I am using https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/downloads/cr2hdr.exe.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 05, 2013, 01:52:49 PM
Try the DUAL file prefix option.

If a file is DNG, you can rename it to CR2 and it will still work (dirty trick). There are no command line options for now.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 05, 2013, 04:01:03 PM
Try the DUAL file prefix option.

Since deflicker and dualiso are mutually exclusive, cr2hdr could also add an xmp file with a similar comment like deflicker and put the other iso value used into it - I think you don't even need to embed it in the dng, an external xmp will also do if there's no tag conflict with the in-dng data.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 05, 2013, 04:13:48 PM
cr2hdr already uses exiftool, so it can add a custom tag. No need for external files.

I don't use these tags, so it will have to be done by somebody who does ;)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 05, 2013, 08:16:19 PM
dual_iso module does not compile>
Opps: module compiles but cr2hdr does not.

Code: [Select]
../../src/chdk-dng-host.o: In function `pow_calc_2':
/home/magiclantern/magic-lantern/modules/dual_iso/../../src/chdk-dng.c:504: undefined reference to `pow'
/home/magiclantern/magic-lantern/modules/dual_iso/../../src/chdk-dng.c:504: undefined reference to `pow'
/home/magiclantern/magic-lantern/modules/dual_iso/../../src/chdk-dng.c:504: undefined reference to `pow'
collect2: error: ld returned 1 exit status
make: *** [cr2hdr] Error 1
Title: Re: Dual ISO - massive dynamic range improvement
Post by: nanomad on September 05, 2013, 09:43:45 PM
Fixed, for some reason it worked on my  laptop but not ony PC ... go figure
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 06, 2013, 12:40:17 AM
Now it detects camera model:

https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/downloads/cr2hdr.exe

Thanks for the update I have noticed higher resolution and less noise in the shadows and mids with this update.  I also had some problems with some cold pixels which have now been resolved..  I will be testing this further with some wildlife photography.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 06, 2013, 01:57:26 AM
I can't get cr2hdr to work on the linux box and no idea how to compile for windows.

To add a keyword tag for easy searching, the command is

Code: [Select]
exiftool "-xmp:subject=Dual-ISO" 1234567.dng
I think this will do it if anyone wants to try

Code: [Select]
# HG changeset patch
# User Audionut <[email protected]>
# Date 1378425319 0
# Branch unified
# Node ID 0f20e1936e12cff0cd623569753ffb76e707e5f3
# Parent  e0f48240e08b9aeace74228416ce6a128f5a2e6c
Added Dual-ISO keyword metadata for easy searching

diff --git a/modules/dual_iso/exiftool-bridge.c b/modules/dual_iso/exiftool-bridge.c
--- a/modules/dual_iso/exiftool-bridge.c
+++ b/modules/dual_iso/exiftool-bridge.c
@@ -8,7 +8,7 @@
 void copy_tags_from_source(const char* source, const char* dest)
 {
     char exif_cmd[1000];
-    snprintf(exif_cmd, sizeof(exif_cmd), "exiftool -tagsFromFile \"%s\" -all:all \"-UniqueCameraModel<Model\" \"%s\" -overwrite_original", source, dest);
+    snprintf(exif_cmd, sizeof(exif_cmd), "exiftool -tagsFromFile \"%s\" -all:all \"-xmp:subject=Dual-ISO\" \"-UniqueCameraModel<Model\" \"%s\" -overwrite_original", source, dest);
     int r = system(exif_cmd);
     if(r!=0)
     {

Also, looks like the only way to add the second ISO speed is via keyword/label/user comment tags.
The correct ISO tags only accept integer.
And I have no idea how to parse the ISO speeds from code for exiftool to use.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 06, 2013, 07:34:26 AM
Looks good to me.

The ISO estimation from cr2hdr is relative to the other ISO (can be either positive or negative, but there's no way to tell - other than a rough guess). If EXIF ISO is 100 and the estimated one is 3 EV, it's clear in which direction it is. If exif ISO is 800 and estimated 3 EV, it's not clear whether it's ISO 800/100 or 800/6400.

You could even use the noise levels and the usual Canon DR curve to figure out which ISO is likely to be.

Or maybe just add a comment that says ISO difference: 3.14 EV.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 06, 2013, 07:41:18 AM
EV difference would be fine imo. 

I would also suggest that it be added to the current tag above, and have the value rounded to an integer.

So you would end up with

Dual-ISO 1EV
Dual-ISO 2EV
Dual-ISO 3EV

etc, etc for searching.

Other ideas?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: kazeone on September 07, 2013, 02:10:11 AM
if anyone has any Android programming skills would be cool if we had the CR2hdr program work on Android devices so if we are on the go we dont need a laptop and what not, only an idea and not sure if its possible, Im not a programmer so yeah. ^^;;
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 07, 2013, 04:41:33 AM
The processing time is around 30 seconds on my overclocked 2600k.  I dread to think what it would be like on a current android device.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: kazeone on September 07, 2013, 04:57:07 AM
The processing time is around 30 seconds on my overclocked 2600k.  I dread to think what it would be like on a current android device.

well if it was designed to take advantage off all the cores I think it would do fine on my quad core Nexus 7 2013.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 07, 2013, 05:24:29 AM
The processing time is around 30 seconds on my overclocked 2600k.

It's 45 seconds on my 4-year old laptop (P8600). I thought the i7 must be a little faster than that, hehe.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 07, 2013, 09:45:23 AM
30% single threaded performance increase is decent enough.  Make it multi-threaded  ;D

well if it was designed to take advantage off all the cores I think it would do fine on my quad core Nexus 7 2013.

ARM is power consumption focused.  It's still going to be rather slow.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: kazeone on September 07, 2013, 10:01:52 AM
30% single threaded performance increase is decent enough.  Make it multi-threaded  ;D

ARM is power consumption focused.  It's still going to be rather slow.

I wouldnt be to quick to say such, in the past only high end CPUs could handle HD playback and now its mere childs play for ARM cpus and they have been shown to be fantastic at mathematical calculations which is whats taking place here when processing the image besides why are you so against it? where is the harm in having the option?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 07, 2013, 10:12:53 AM
Quote
where is the harm in having the option?

Somebody must spend time to maintain the arm port (which is likely to be quite a bit more than just a straightforward compilation of current sources).

If you are willing to do that, be my guest. I'm curious how fast it can go. Maybe put Ubuntu on the Nexus and run it in console?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: kazeone on September 07, 2013, 10:32:23 AM
Somebody must spend time to maintain the arm port (which is likely to be quite a bit more than just a straightforward compilation of current sources).

If you are willing to do that, be my guest. I'm curious how fast it can go. Maybe put Ubuntu on the Nexus and run it in console?

well I would be willing to buy it so it wouldnt have to be free, shoot Im sure others would like the option as well, maybe we can talk to chainfire, hes done a lot in terms of creating various apps that control Canon DSLRs, including the famed DSLR Controller that offered a lot neat features well before ML finally got around to getting ported.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 07, 2013, 10:59:07 AM
The ISO estimation from cr2hdr is relative to the other ISO (can be either positive or negative, but there's no way to tell - other than a rough guess).

Maybe in this case directly writing this information into an xmp file from ml is the way to go after all? You'd then have it next to the cr2 which some people might want to keep because better cr2hdr versions might be coming up, and cr2hdr could read the xmp and write the information into the dng. Or of course as it was suggested cr2hdr could try to figure out the + or - iso ev for itself - in any case, imho just loosing this information along the way would be a pitty.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 07, 2013, 04:53:38 PM
I wouldnt be to quick to say such, in the past only high end CPUs could handle HD playback and now its mere childs play for ARM cpus and they have been shown to be fantastic at mathematical calculations which is whats taking place here when processing the image besides why are you so against it? where is the harm in having the option?

I'm not against it.  But some facts.

ARM is power consumption focused.
The current processing is single threaded (bad news for slow processors that rely on multi threading), and I can't see how the processing could be easily multi threaded.  a1ex would know better.

Video processing is done via dedicated chip and/or dedicated processing routines, neither of which currently help this processing, nor are they likely to in the future.

I'm not saying it can't be done!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Francis on September 07, 2013, 06:24:15 PM
I'm curious as to why you would want a cr2hdr conversion tool on your tablet. The resulting DNG files still need a good deal of PP to get them to look decently exposed. Then you're going to need Lightroom or GIMP/dcraw or whatever running on your tablet also. You might as well just wait until you are back at your 'editing' computer.

Seems like a waste of time. If you are just wanting a preview, then image review does a pretty decent job of blending the exposures.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: kazeone on September 07, 2013, 08:30:21 PM
I'm curious as to why you would want a cr2hdr conversion tool on your tablet. The resulting DNG files still need a good deal of PP to get them to look decently exposed. Then you're going to need Lightroom or GIMP/dcraw or whatever running on your tablet also. You might as well just wait until you are back at your 'editing' computer.

Seems like a waste of time. If you are just wanting a preview, then image review does a pretty decent job of blending the exposures.

Sometimes Im on site for a week at a time or Im hiking so yes I could see it being handy to be able to work with a tablet and I already have a program like lightroom on my tablet that lets me work on Raws and DNG just like in lightroom.

As for a decent job blending the image in review mode on the camera you must be doing something different then I am because all I see is distinct lines across the whole image.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Francis on September 07, 2013, 09:58:50 PM
Don't zoom in to 100%.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 08, 2013, 12:36:44 AM
It's 45 seconds on my 4-year old laptop (P8600). I thought the i7 must be a little faster than that, hehe.

well, I've checked it around and seemed no luck running cr2hdr.exe even with multiple user accounts on same computer, so I've created 4 Windows 7 virtual systems, my ram is about 8gigs and able to run 4 VMs simultaneously. only bottleneck is I have to use my system SSD for file i/o, because both systems and cr2hdr.exe's temp file crashes the harddrives.

so far, I'm able to render 5 frames at the same time with i5 2450M, 8gigs ram, 5 x windows 7 and a single SSD, but I'd like to have a multithread support on cr2hdr.exe :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on September 08, 2013, 04:42:15 AM
Some help requested.

After having used ML for a few months, my favourite most feature is clearly the Dual_ISO capability by a mile. And more so on my EOS M ( even more than on my 5D Mark III, for practical reasons - even though the quality I get on my 5D is far superior ).

The only thing I am struggling with is recovering the shadows on skin. I get these really black areas concentrated in regions of shadows that recover but not quite. There are these black patches that refuse to go away no matter what version of cr2hdr or dcraw I adopt.

And, this is a problem, because most of my photography is done after sunset ( busy schedule at work ).

I am now thinking flash to recover shadows since the highlights are clearly brilliantly handled with the Dual_ISO feature. Fast lenses are an option but not always.

Any other ideas will help greatly.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 08, 2013, 06:09:18 AM
I'm having trouble with cr2hdr.exe when I shot sunset raw video at dual iso, anyone have same issue?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: telecastDaveRI on September 08, 2013, 06:38:32 AM
I'm having trouble with cr2hdr.exe when I shot sunset raw video at dual iso, anyone have same issue?

I dunno....whats the issue?  :D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 08, 2013, 07:02:37 AM
its gonna be double post but photo upload failed.

(http://img802.imageshack.us/img802/8141/w8gr.jpg) (http://imageshack.us/photo/my-images/802/w8gr.jpg/)

I'm having trouble with cr2hdr.exe when I shot sunset raw video at dual iso, anyone have same issue on bright areas?

Title: Re: Dual ISO - massive dynamic range improvement
Post by: kazeone on September 08, 2013, 09:06:29 AM
can you upload the original raw file? so I can try it on my end and see if I get the same results.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 08, 2013, 12:35:10 PM
can you upload the original raw file? so I can try it on my end and see if I get the same results.

here it is https://www.transferbigfiles.com/c58348c4-8282-4dcd-b58b-59e1dbb9a684/Uw7m4GTs2sg8-yDtjvYYjA2
Title: Re: Dual ISO - massive dynamic range improvement
Post by: dadinio13 on September 08, 2013, 05:10:07 PM
with cr2hdr for mac i can't process dng files. With wine + dos prompt and cr2hdr.exe i can't with this command : cr2hdr.exe MXX-XXXX-XX... I don't know what command i can use for select all dng in a folder... i can't do drag & drop onto cr2hdr.exe...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: fpena06 on September 08, 2013, 05:20:02 PM
On osx 10.8.4 and cr2hdr (2013-09-05). can't process dng files either.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 08, 2013, 05:35:45 PM
with cr2hdr for mac i can't process dng files. With wine + dos prompt and cr2hdr.exe i can't with this command : cr2hdr.exe MXX-XXXX-XX... I don't know what command i can use for select all dng in a folder... i can't do drag & drop onto cr2hdr.exe...

as a1ex said couple posts earlier, rename them to xxxxxx.cr2's, then cr2hdr.exe will process them
Title: Re: Dual ISO - massive dynamic range improvement
Post by: fpena06 on September 08, 2013, 05:53:14 PM
heres a workaround. extract cr2hdr from cr2hdr.app "cr2hdr.app/Contents/Resources/bin/cr2hdr"

will have to use command line. to process entire folder simple do this.

./cr2hdr /Volumes/EOS_DIGITAL/DCIM/100EOS7D/*

This worked for me. I was able to process a complete folder with DNG files.

./cr2hdr /Volumes/EOS_DIGITAL/DCIM/100EOS7D/*

Input file     : /Volumes/EOS_DIGITAL/DCIM/100EOS7D/IMG_0674.DNG
Canon EOS 7D detected
Full size      : 5360 x 3516
Active area    : 5202 x 3465
White level    : 12500
Black borders  : 158 left, 51 top
Black level    : 1953
ISO pattern    : dBBd GBRG
Noise levels   : 11.76 19.78 21.37 11.32 (14-bit)
Estimating ISO difference...
ISO difference : 1.04 EV (205)
Black delta    : -3.07
Interpolation  : mean23-chroma5x5-alias
Dynamic range  : 9.86 (+) 8.95 => 9.98 EV (in theory)
Matching brightness...
Looking for hot/cold pixels...
Hot pixels     : 20
Cold pixels    : 1236
Full-res reconstruction...
ISO overlap    : 5.8 EV (approx)
Half-res blending...
Chroma filtering...
Building alias map...
Filtering alias map...
Smoothing alias map...
Noise level    : 29.11 (16-bit)
Dynamic range  : 10.50 EV (cooked)
Black adjust   : -4
Output file    : /Volumes/EOS_DIGITAL/DCIM/100EOS7D/IMG_0674.DNG
    1 image files updated

Title: Re: Dual ISO - massive dynamic range improvement
Post by: dadinio13 on September 08, 2013, 07:33:50 PM
yep thank it worked fine for me but that is not the last version im wrong ?
and for the cr2hdr app how i can change the extension of dng's to cr2's in the folder at one time?,
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 08, 2013, 07:43:24 PM
yep thank it worked fine for me but that is not the last version im wrong ?
and for the cr2hdr app how i can change the extension of dng's to cr2's in the folder at one time?,
Get this one:  www.bulkrenameutility.co.uk
Title: Re: Dual ISO - massive dynamic range improvement
Post by: dadinio13 on September 08, 2013, 07:47:47 PM
thank you, sir :)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: escho on September 08, 2013, 07:48:45 PM
I played a bit with dual-iso in video-mode on my 600D. (dual-iso compiled from latest sources of TL2.0 on Linux). Just a little report:

Compilation on Linux:
No problem for an english system. But with my german language-settings, there is this pgm_with-error using cr2hdr. I made a workaround for me for this problem. I asked, whether this could be coded in the way, I told in this thread: http://www.magiclantern.fm/forum/index.php?topic=7693.0 . But i got no answer, so I guess, this way is not the best way and not wanted.

Colors:
Much cleaner colors on raw-vido than on mov-file. And dual-iso let a lot of noise disappear. Great!

Disadvantage:
The hot-pixels get much more visible  using dual_iso.
On the mov-file, canon map most of these pixels out. In the RAW-file, these hotpixels can be surpressed using rawtherapee, other opensource linux-software, I know, is not able to render them out effectively. But using dual-iso, there is a little problem:
In the dng-files from raw2dng (./raw2dng *.RAW ) these pixels cannot be surpressed anymore. and in the output-files of cr2hdr (./cr2hdr *.dng ) I also had no success in deleating the hot pixels.

For me this doesn´t matter, because I only make videos for my astronomical shootings. And after stacking 5000 frames of saturn there is no more noise in the picture and all the hotpixels have gone magically :)

This was my little report of using dual-iso with my 600D in video-mode

Edgar
Title: Re: Dual ISO - massive dynamic range improvement
Post by: laneo on September 09, 2013, 12:21:05 AM
Hi there.
I'm trying to get rid of the horizontal lines but i can't.
I'm on a mac. I'm using the last version of raw2dng *0.13).
When i drop the dng files to the last version of cr2hdr it says: name.dng File not supported.
I tried creating the dng files using the last version of RAWMagic (beta7b). CR2HDR doesnt work with those dng files.
Am I missing any steps or it should be working ?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: lourenco on September 09, 2013, 04:43:18 AM
I was taking photos at the local drag strip today. I accidentally left dual ISO on. Sometimes I am worried about certain custom paint designs not coming out correctly, so I turn it off when needed. I very happy to see this photo turned out great! This is not an extreme dual ISO example. The owner of the car bought multiple prints of this photo.

(https://sphotos-a-mia.xx.fbcdn.net/hphotos-prn2/9255_594194790623658_495677599_n.jpg)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: engardeknave on September 09, 2013, 08:38:51 AM
Wouldn't it be easy to multi-thread when batch processing? I made a quick script to do this. It's kind of stupid, though. It makes a new folder for each .cr2, copies the .cr2 and cr2hdr/dcraw/exiftool to that folder and then runs cr2hdr for each one. I do like 10-15 files at once this way on my i5 3570k. I haven't really timed it, but it doesn't take long. There's got to be a less dumb way of doing this though.

It would at least make scripting a lot easier if the temp files were named something different for each file being processed, so you don't have to put everything in separate folders to accomplish the above.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Shizuka on September 09, 2013, 09:08:05 AM
pretty much the cheap way to do this is to use a multithread capable dispatcher + modify cr2hdr to use temp files named from the CR2's checksum. it's not really possible to speed up cr2hdr if dcraw is not multithread-capable (amdahl's law etc)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mediavr on September 09, 2013, 09:30:55 AM
Dual ISO is really good for shooting interactive 360 panoramas -- where you constantly have to deal with high dynamic range subjects and these look really bad in 360 panoramas if there are large areas with blown out highlights or shadow areas without details. Here is one I shot on the weekend -- 5DMkIII, 100/1600
http://www.mediavr.com/botgardens/botgardens1.htm

To extract all the detail I could from the dngs I made 3 "exposures" jpgs from each dng with ACR -- for shadows, midtones and highlights and then loaded all image variants into the stitching program I use (PTGui). This can use Enfuse, which is an exposure blending plugin
http://wiki.panotools.org/Enfuse
also available apparently as a plugin for Lightroom
http://www.photographers-toolbox.com/products/lrenfuse.php

I think the combination of Enfuse plus extracted exposure variants can be a useful technique for dealing with the vast dynamic range available in Dual ISO dngs

Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 09, 2013, 11:26:58 AM
Wouldn't it be easy to multi-thread when batch processing? I made a quick script to do this. It's kind of stupid, though. It makes a new folder for each .cr2, copies the .cr2 and cr2hdr/dcraw/exiftool to that folder and then runs cr2hdr for each one. I do like 10-15 files at once this way on my i5 3570k. I haven't really timed it, but it doesn't take long. There's got to be a less dumb way of doing this though.

It would at least make scripting a lot easier if the temp files were named something different for each file being processed, so you don't have to put everything in separate folders to accomplish the above.

I'm using multiple virtual machines for this job, now 4 virtual 1 real windows 7 crunching numbers to process photos***.

(http://img89.imageshack.us/img89/8809/x82w.png) (http://imageshack.us/photo/my-images/89/x82w.png/)

***for those who still wants multithread processing, you must use industrial grade harddrives or ssds for this job. temp file and final dng file at multithread kills the drive. I'm using intel 520 series, which has best price/performance/security ratio on the market.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 09, 2013, 11:51:30 AM
Most of it is memory transfer/cpu processing.  Having 1 drive for each execution will certainly help though :)

Why are you using virtual machines with all of their overhead?  Make 5 (or whatever) copies of the executable in the base OS.

edit:  Any decent HDD connect via sata should be fine.
(http://i.imgur.com/ZAnX099.jpg)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 09, 2013, 11:54:40 AM
Most of it is memory transfer/cpu processing.  Having 1 drive for each execution will certainly help though :)

Why are you using virtual machines with all of their overhead?  Make 5 (or whatever) copies of the executable in the base OS.

it doesn't work, I've tried every possible combination. I have a network server has 5 drives but my pc has only 2 gigabit connections, still a bottleneck. I'm considering switching to E3 Xeon machine for better stability and faster processing. also I already have my virtual machines for other purposes now helping out crunching numbers of Dual ISO :)

*edit: virtual windows 7s have 1gigs ram only, not really overhead for the real system, theoretically I can use 7 virtual machines on 8gigs ram.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 09, 2013, 12:04:03 PM
it doesn't work,

Check my edit above.

4 copies of the executables in different folders.  Bunch of dual ISOs in each folder.  Drag and drop just like normal.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 09, 2013, 12:06:02 PM
Check my edit above.

4 copies of the executables in different folders.  Bunch of dual ISOs in each folder.  Drag and drop just like normal.

your version supports EOS M's crazy dual iso with multiple sequences?

*edit: found it. testing
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 09, 2013, 12:12:13 PM
Huh??

100 dual ISO cr2's.
4 folders on the 1 HDD.
In each folder, a copy of each of the executables (cr2hdr, dcraw, exiftool), and 25 of the cr2's
Open 1 folder and drag and drop the cr2's onto cr2hdr.
Open the next folder and do the same
Yada
Yada

Batch processing using poor mans multithreading just like you were doing above.  Difference being, you don't need SSDs, or virtual machine, or xeons, or server racks or any else.
A decent HDD connect via sata and enough cores in the processor are all that's needed.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 09, 2013, 12:16:33 PM
Huh??

100 dual ISO cr2's.
4 folders on the 1 HDD.
In each folder, a copy of each of the executables (cr2hdr, dcraw, exiftool), and 25 of the cr2's
Open 1 folder and drag and drop the cr2's onto cr2hdr.
Open the next folder and do the same
Yada
Yada

Batch processing using poor mans multithreading just like you were doing above.  Difference being, you don't need SSDs, or virtual machine, or xeons, or server racks or any else.
A decent HDD connect via sata and enough cores in the processor are all that's needed.

latest version sucked at processing

(http://img18.imageshack.us/img18/1725/io91.png) (http://imageshack.us/photo/my-images/18/io91.png/)

well you know the saying "the best way is you know" :)

*also, I've tried it before. unfortunately 5400rpm laptop harddrives doesn't help much with multiple instances.

**I'm switching it because of long-awaited "Intel® TSX-NI" support, today not so important but xeon for a decent price with greater performance than core i7, excellent ram support, dual gigabit ethernet and wait lower power usage? tech miracle.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 09, 2013, 12:23:46 PM
*also, I've tried it before. unfortunately 5400rpm laptop harddrives doesn't help much with multiple instances.

I wouldn't all a 5400rpm drive decent ;)  Especially if that's in a laptop.

Let me rephrase.  And decent 7200rpm HDD connect via sata to a desktop should be fine!  That's still well under the recommendation of SSD's, virtual machines and xeons though :P

latest version sucked at processing

Have you uploaded a CR2 for a1ex to look at? 
And just to clarify, latest version date stamped 3/9/13.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 09, 2013, 12:26:10 PM
I wouldn't all a 5400rpm drive decent ;)  Especially if that's in a laptop.

Let me rephrase.  And decent 7200rpm HDD connect via sata to a desktop should be fine!  That's still well under the recommendation of SSD's, virtual machines and xeons though :P

sorry, I have one laptop and a home server. used 7200rpm seagate 500gb drives for once and now them in enclosures, noisy and overheating drives with only %10 performance difference? no thanks.

edit: this is the exe : https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/downloads/cr2hdr.exe

edit 2: found another exe, date 2/9/13 can kick 6 thread same time

(http://img835.imageshack.us/img835/5774/xv95.png) (http://imageshack.us/photo/my-images/835/xv95.png/)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 09, 2013, 12:29:08 PM
sorry, I have one laptop and a home server. used 7200rpm seagate 500gb drives for once and now them in enclosures, noisy and overheating drives with only %10 performance difference? no thanks.

I agree.  But it's useful to point out that those drives are sufficient for this purpose, for those people without the funds for new toys.

edit: this is the exe : https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/downloads/cr2hdr.exe

This is the official one http://acoutts.com/a1ex/cr2hdr.zip
Bug 1% about that version you are using.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 09, 2013, 12:33:52 PM
...for those people without the funds for new toys.

currently I'm considering to upgrade my machines before getting new lenses or cameras. because I have t2i and EOS M, if I buy EF-M lens it'll be EOS M only, if I buy EF-S lens it'll be t2i and EOS M only. if/when I upgrade to 5D.III then all other equipment will stay dead for weeks.

edit: found the bug, my DUAL ISO files are older than normal. new version supports dBBd files, some of mine are dBdB.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 09, 2013, 12:46:25 PM
dual iso should really be combined with an ettr/ettl option to expose the first image to one side and then automatically figure out the required ev spacing for the 2nd shot, in essence minimizing the required dual_iso ev spacing.

Almost working: https://bitbucket.org/hudson/magic-lantern/commits/eb3cc38d1870

How it works:
- remember that ETTR has SNR limits for midtones and shadows (it will try really hard to meet or exceed that SNR).
- if dual ISO is not enabled, it may have to clip many highlights in order to meet that SNR. If you turn off SNR limits, it will underexpose, but keep all the highlights.
- if dual ISO is enabled, ETTR will adjust it to meet or exceed the SNR you have set, and keep all the highlights (except the specular ones, handled by "Highlight ignore")

Problem: ETTR is iterative, and it relies on the previous image (where it measures the white point, some SNR numbers and stuff like that). SNR seems to change quite a bit with dual ISO (even if it shouldn't); need to investigate why. So, at this point, ETTR with dual ISO may be a little undecided: it may say 100/800 or 100/1600.

How to use it:
- Make sure you have raw zebras and histogram enabled (so you know what's going on, what's clipped, what may be aliased...)
- Enable Auto ETTR (Always ON) and dual ISO.
- Point the camera at your scene and take a few test pics.
- If the exposure doesn't converge, try underexposing a bit. When the image is overexposed, it has to guess by how much (and it doesn't always work). When it's a little underexposed, it can compute exactly what settings are needed. When it's heavily underexposed, noise will affect the metering.

ETTR will try to:
- use a low ISOs and a low EV spacing
- meet or exceed the selected SNR minus 0.5 EV
- do not clip more highlights than you have selected for "Highlight ignore"
- do not exceed max auto ISO from Canon menu

So, if that SNR can be met without clipping highlights, the scene does not require dual ISO. If not... it does.

Enable debug info to show the math behind it.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 09, 2013, 01:00:24 PM
Problem: ETTR is iterative, and it relies on the previous image (where it measures the white point, some SNR numbers and stuff like that). SNR seems to change quite a bit with dual ISO (even if it shouldn't); need to investigate why. So, at this point, ETTR with dual ISO may be a little undecided: it may say 100/800 or 100/1600....

do you think its possible to save dual iso RAW + normal JPG at the same time? maybe ETTR can use info from the JPG?

edit: or highlight LOG?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 09, 2013, 01:03:16 PM
What?!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 09, 2013, 01:07:18 PM
What?!

create and use log of previous shot for ETTR correction.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 09, 2013, 01:16:51 PM
Quote
create and use log of previous shot for ETTR correction.

How's that different from what I'm doing now?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 09, 2013, 01:23:15 PM
How's that different from what I'm doing now?

I'm not sure if we're talking about same stuff. get some rest a1ex
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Shizuka on September 09, 2013, 01:51:12 PM
I'm using multiple virtual machines for this job, now 4 virtual 1 real windows 7 crunching numbers to process photos***.

(http://img89.imageshack.us/img89/8809/x82w.png) (http://imageshack.us/photo/my-images/89/x82w.png/)

***for those who still wants multithread processing, you must use industrial grade harddrives or ssds for this job. temp file and final dng file at multithread kills the drive. I'm using intel 520 series, which has best price/performance/security ratio on the market.

too much work
i wrote custom utilities for this sort of thing before
http://www.mediafire.com/download/5n5pjih98b80909/MT_cr2hdr.zip
note that cr2hdr is hacked to use unique tempfiles based off input file name.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 09, 2013, 02:01:47 PM
Can you share the temp file hack?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: zuzukasuma on September 09, 2013, 02:10:02 PM
too much work
i wrote custom utilities for this sort of thing before
http://www.mediafire.com/download/5n5pjih98b80909/MT_cr2hdr.zip
note that cr2hdr is hacked to use unique tempfiles based off input file name.

this is what I'm looking for! tell devs to import this to pack! it works like a charm (missing dcraw.exe but no prob). thank you!

edit: worked like a charm :) when you close the example.cmd cr2hdr.exe's running in the background till the all CR2s converted.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Shizuka on September 09, 2013, 02:14:38 PM
Can you share the temp file hack?
it's assembly patched - source would be the bindiff sorry

I would imagine the changes in source are here:
//line 145
        printf("\nInput file     : %s\n", filename);
        char tempname[4];
             tempname[0] = filename[4];
             tempname[1] = filename[5];
             tempname[2] = filename[6];
             tempname[3] = filename[7];
             char tempname_txt[8];
             char tempname_pgm[8];
             snprintf(tempname_txt,8,"%s.txt",tempname);
             snprintf(tempname_pgm,8,"%s.pgm",tempname);

        char dcraw_cmd[1000];
//line 148
        snprintf(dcraw_cmd, sizeof(dcraw_cmd), "dcraw -v -i -t 0 \"%s\" > %s", filename, tempname_txt);
        int exit_code = system(dcraw_cmd);
        CHECK(exit_code == 0, "%s", filename);
       
        unsigned int model = get_model_id(filename);
        exit_code = get_raw_info(model, &raw_info);

        CHECK(exit_code == 0, "RAW INFO INJECTION FAILED");
//line 157
        FILE* t = fopen(tempname_txt, "rb");
        CHECK(t, tempname_txt);

also replace "tmp.pgm" with tempname_pgm (line 184, 188, 189)

finally, replace the unlink() parameters at 283/284


a1ex: a suggestion: turn on sse floating point instructions - it'll give a 20-30% speedup for free on many speed critical routines. -msse -msse2 -mfpmath=sse are the compiler flags for gcc, and will produce executables that run on pentium4/amdK8 or higher
edit: maybe not. I only found two routines
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a59192 on September 09, 2013, 02:54:46 PM
Currently I'm still experiencing problems on dng converter, set up in the same shot, CR2 conversion to DNG is good and bad, and attach using 5DII CR2 shoot, ask if there is any way to stabilize conversion?

https://mega.co.nz/#!QIVmVCzS!RKZgVGp2-ntkv2OFPqZ3w3ZO2XYcUCnDRkYJmAbhuZQ
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 09, 2013, 03:56:05 PM
a1ex: a suggestion: turn on sse floating point instructions - it'll give a 20-30% speedup for free on many speed critical routines. -msse -msse2 -mfpmath=sse are the compiler flags for gcc, and will produce executables that run on pentium4/amdK8 or higher
edit: maybe not. I only found two routines

Fprofiling might help also.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 09, 2013, 04:02:17 PM
Experimental cr2hdr (http://acoutts.com/a1ex/cr2hdr_exp.exe):
- converts the CR2 for a59192 (but didn't check if it breaks any others)
- uses pipes instead of temporary files (should work better if you start 5 copies in the same directory)

I didn't try it on Windows, only compiled the exe. On Linux I've tried on 2 pics in parallel (got 57 seconds for both, compared with 45 for a single picture).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 09, 2013, 04:32:22 PM
It's complaining that everything doesn't look like interlaced ISO on my windows box.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 09, 2013, 04:42:24 PM
Full log?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 09, 2013, 04:43:42 PM
Code: [Select]
I:\1>cr2hdr_exp _UAL5976.CR2

Input file     : _UAL5976.CR2
Canon EOS 5D Mark III detected
Full size      : 5920 x 3950
Active area    : 5796 x 3870
Doesn't look like interlaced ISO

Title: Re: Dual ISO - massive dynamic range improvement
Post by: a59192 on September 09, 2013, 04:53:02 PM
Experimental cr2hdr (http://acoutts.com/a1ex/cr2hdr_exp.exe):
- converts the CR2 for a59192 (but didn't check if it breaks any others)
- uses pipes instead of temporary files (should work better if you start 5 copies in the same directory)

I didn't try it on Windows, only compiled the exe. On Linux I've tried on 2 pics in parallel (got 57 seconds for both, compared with 45 for a single picture).

Used cr2hdr_exp.exe with the current new construction cr2hdr, can not normally turn out, do not know which part wrong, ask how to use the uses pipes? windows can use it?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 09, 2013, 04:53:46 PM
Looks like you can't use pipes with large amounts of data under Windows.

http://marc.info/?l=mingw-users&m=134554372503503
http://lists.gnu.org/archive/html/bug-binutils/2012-09/msg00064.html
http://mingw.5.n7.nabble.com/popen-from-a-GUI-program-without-opening-a-cmd-exe-window-td20873.html
https://lists.fedoraproject.org/pipermail/mingw/2009-September/002087.html

I've uploaded one again, with _popen instead of popen, but I have no idea whether it will work or not.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 09, 2013, 04:56:12 PM
Same problem.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 09, 2013, 07:08:20 PM
Can you try again? Got it working under Wine.

I had to set binary mode on the pipe and stop using fseek.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 09, 2013, 07:19:15 PM
Yeah that got it working.  Can you add a command line option for the user to select how many instances to run at any one time.  And then when a bunch of CR2s are dropped on it, it's spawns all those instances?

I tried adding a heap of compiler options (-msse etc etc), and it made no difference to the processing time :(
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 10, 2013, 04:10:09 AM
Camera tested 5D3
I tried the latest commit (da752ca33b18) and the when using the AETTR + Dual ISO I get a problem. 
I am setting the Dual Iso to +3EV
The first picture I take shows as dual ISO possilbly at base iso + 3EV
The next picture the Dual ISO module has the base iso = recovery iso.  So no Dual Iso Picture is take.
Repeatable.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: telecastDaveRI on September 10, 2013, 05:43:50 AM
Yeah that got it working.  Can you add a command line option for the user to select how many instances to run at any one time.  And then when a bunch of CR2s are dropped on it, it's spawns all those instances?

I tried adding a heap of compiler options (-msse etc etc), and it made no difference to the processing time :(


If I can ask, how did you get it working? Did you just download a1ex's experimental upload (seen above) again? (which I assume has been updated)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 10, 2013, 05:57:49 AM
Did you just download a1ex's experimental upload (seen above) again? (which I assume has been updated)

That's correct.

Camera tested 5D3
I tried the latest commit (da752ca33b18) and the when using the AETTR + Dual ISO I get a problem. 
I am setting the Dual Iso to +3EV
The first picture I take shows as dual ISO possilbly at base iso + 3EV
The next picture the Dual ISO module has the base iso = recovery iso.  So no Dual Iso Picture is take.
Repeatable.

Are you sure it's not this?

ETTR will try to:
- use a low ISOs and a low EV spacing
- meet or exceed the selected SNR minus 0.5 EV
- do not clip more highlights than you have selected for "Highlight ignore"
- do not exceed max auto ISO from Canon menu

So, if that SNR can be met without clipping highlights, the scene does not require dual ISO. If not... it does.

----------------

Using autoexpo for the base exposure with AETTR enabled in autosnap with dual ISO.  All images in a test batch had the dual ISO file prefix.  Some images were not dual ISO as not needed.

If I just have dual ISO enabled with alternate frames, the file prefix is working as expected.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 10, 2013, 11:22:12 PM
When I take a photo with AETTR 02% h.i. and SNR OFF I get
DR = 10.9
Same scene with Dual Iso only with Recovery ISO = +3 EV I get
DR = 13.4

Now I turn on AETTR 0.2% h.i plus and SNR = OFF + Dual ISO with Recovery ISO = +3 EV I get
DR=10.9  and Base ISO = Recovery ISO  DUAL ISO is turned OFF automatically

Now I set DUAL ISO TO 800 ISO plus the AETTR as above and I get
DR=10.9 and Base Iso = Recovery ISO , my ISO 800 has been automatically replaced by base ISO

Conclusion:  Dual ISO + AETTR is not working properly.  If I roll back to Sept 07 build then the results are totally different and DUAL ISO works as expected!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 11, 2013, 12:44:16 AM
Enable some SNR limits.  This will give the code something to aim for :)

Both functions are now linked together when both are enabled.  With no SNR limits applied, you're telling it, I don't care how noisy the midtones/shadows are.
With SNR limits applied, previously AETTR would blow the highlights to attain the required SNR, now it will enable dual ISO instead to keep the highlights in check while producing the required SNR.

I guess it could be coded so that SNR limits off mean restore previous functionality.  But to be honest, I think that will cause to much confusion for users in the future.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 11, 2013, 01:25:30 AM
Using autoexpo for the base exposure with AETTR enabled in autosnap with dual ISO.  All images in a test batch had the dual ISO file prefix.  Some images were not dual ISO as not needed.

I tried adding 100ms delay to autosnap but it still wanted to use the same prefix as the previous shot regardless.  And 100ms was becoming noticeable.

a1ex can you send the output of cr2hdr to a logfile?  It would be nice to check the results of processed files without having to manually process each one.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 11, 2013, 04:12:11 AM
Enable some SNR limits.  This will give the code something to aim for :)
I want to keep all of the highlights, that is why I turned off SNR.  So my only option to conserve the highlights is Midtones SNR = 1 if I want to use Dual Iso with Recoveru ISO +3EV
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 11, 2013, 04:21:57 AM
Ok I tested with SNR and it works with Midtones and Shadows = 1.  So I dont have the option to keep the highlights by doing a standard AETTR at h.i.=0.2% + DUAL ISO.

In my opinion this is not good.  I will be using September 07 build until this gets resolved.
Thanks anyway.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 11, 2013, 04:23:08 AM
AETTR works just like previously.  You decide how many highlights to blow (or not).
Enabling SNR limits will try and keep those highlights, but will blow highlight detail to ensure the SNR limits are met.
Enabling SNR limits and dual ISO will keep those highlights and use dual ISO as necessary to ensure the SNR limits are met without blowing the highlights.

If the SNR limit can be met without the use of dual ISO, dual ISO will not be used.  Highlight detail is controlled by the settings in AETTR just as it's always been.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 11, 2013, 04:27:11 AM
Enable AETTR and set your required highlight settings.  Have dual ISO and SNR limits off.  Observe the results.
Now enable dual ISO and set the SNR limits (use the default settings for a start 6/3), observe the results.
Set the midtone SNR limit to 8EV.  Observe the results.  Highlight detail should not change, dual ISO settings should change (automatically) to boost the midtone luminance.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 11, 2013, 04:38:29 AM
Enabling SNR limits will try and keep those highlights, but will blow highlight detail to ensure the SNR limits are met.

Just a simple question.  I want to keep all of the highlight therefore I use AETTR with h.i. = 0

Now I want to boost the shadows and midtones without blowing the highlihgts using DUAL ISO.  What should I set the Dual ISO module to accomplish this?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 11, 2013, 04:42:01 AM
Nothing.  Dual ISO is controlled automatically.  Just turn it on.

Control the midtones/shadows with the SNR limits.  With dual ISO on, it won't blow your highlight detail like previously, it will adjust dual ISO settings as needed to boost the midtones/shadows.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 11, 2013, 06:19:53 AM
I guess it could be coded so that SNR limits off mean restore previous functionality.  But to be honest, I think that will cause to much confusion for users in the future.

Or maybe a setting in dual ISO, "link to AETTR".  Else dual ISO is always enabled at the settings specified.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 11, 2013, 02:38:36 PM
Nothing.  Dual ISO is controlled automatically.  Just turn it on.

Control the midtones/shadows with the SNR limits.  With dual ISO on, it won't blow your highlight detail like previously, it will adjust dual ISO settings as needed to boost the midtones/shadows.

Nothing is not a good answer because after the first picture the camera automatically sets the Base ISO = Recovery ISO and Dual ISO stops working.   So again, there appears to be a problem with Dual ISO module that should be resolved.

The problem is explained here:

Camera tested 5D3
I tried the latest commit (da752ca33b18) and the when using the AETTR + Dual ISO I get a problem. 
I am setting the Dual Iso to +3EV
The first picture I take shows as dual ISO possilbly at base iso + 3EV
The next picture the Dual ISO module has the base iso = recovery iso.  So no Dual Iso Picture is take.
Repeatable.

And here....

When I take a photo with AETTR 02% h.i. and SNR OFF I get
DR = 10.9
Same scene with Dual Iso only with Recovery ISO = +3 EV I get
DR = 13.4

Now I turn on AETTR 0.2% h.i plus and SNR = OFF + Dual ISO with Recovery ISO = +3 EV I get
DR=10.9  and Base ISO = Recovery ISO  DUAL ISO is turned OFF automatically

Now I set DUAL ISO TO 800 ISO plus the AETTR as above and I get
DR=10.9 and Base Iso = Recovery ISO , my ISO 800 has been automatically replaced by base ISO

Conclusion:  Dual ISO + AETTR is not working properly.  If I roll back to Sept 07 build then the results are totally different and DUAL ISO works as expected!

and here:

I want to keep all of the highlights, that is why I turned off SNR.  So my only option to conserve the highlights is Midtones SNR = 1 if I want to use Dual Iso with Recoveru ISO +3EV
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 11, 2013, 03:35:20 PM
Nothing is not a good answer because after the first picture the camera automatically sets the Base ISO = Recovery ISO and Dual ISO stops working.   So again, there appears to be a problem with Dual ISO module that should be resolved.

Can you post 2 cr2s please.  The first shot where you had your initial settings, and the second shot where dual ISO was reset to recovery ISO = base ISO.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 11, 2013, 03:43:41 PM
Can you post 2 cr2s please.  The first shot where you had your initial settings, and the second shot where dual ISO was reset to recovery ISO = base ISO.
Would love to do it but I dont have the bandwidth to do that.  What should I look for in the cr2.  maybe I can analyze it.  I know what dual iso is and there is no dual ISO on the second image because DUAL ISO says so, "Both ISO are identical, nothing to do"
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 11, 2013, 03:45:32 PM
If the first CR2 has a SNR higher or equal to what you have requested in menu, ETTR will disable dual ISO. This is normal and not a bug.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 11, 2013, 03:50:12 PM
If the first CR2 has a SNR higher or equal to what you have requested in menu, ETTR will disable dual ISO. This is normal and not a bug.
SNR=OFF
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 11, 2013, 03:51:30 PM
What highlights are or are not being blown in each of the photos.

Does setting SNR limits correlate with the description provided by a1ex 2 pages back. 
Are the highlights being blown with SNR limits enabled.
Set SNR limits to 8/3EV midtones/shadows.  Are the highlights being blown.


Always take enough photos after changing AETTR settings to ensure convergence has been met.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 11, 2013, 11:41:56 PM
Had a chance to play with it today. I've set it on Auto ETTR (SET button in LiveView), highlight ignore 0.1%, midtone SNR at 7 EV and shadow SNR disabled.

In most cases it worked quite well. Only had some minor problems:

1) when ISO reached 6400 (it was undecided between 400/6400 and plain 6400 - quite a big diference). The problem appears only when it's too dark and exposure limits are reached (extra DR needed is estimated incorrectly).

2) in "always on" mode, confirmation beep was not always right (overall exposure didn't change, but dual ISO range did; however, the "beep system" didn't notice that). So, after getting a single beep, I had to take one more picture.

LiveView + SET worked fine.

3) sometimes it chose 3200/6400, and there was probably no benefit in doing that. Of course, I could have limited it to 1600 or 3200, but I also wanted to watch the algorithm making the choices.

Besides these small quirks, I think ETTR is now doing near-optimal exposure for dual ISO.

I may post some pics if there'll be anything worth showing.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 12, 2013, 02:54:10 AM
There appears to be a problem with the noise calculation for ISO 6400 (for the 5D3 at least).
The reported noise level continues to drop for all dual ISOs until ISO 6400 recovery.

Quote
Input file     : _UAL6623.cr2
Canon EOS 5D Mark III detected
White level    : 12500
Black level    : 1963
Noise levels   : 10.65 6.10 6.26 10.49 (14-bit)
ISO difference : 3.97 EV (1571)
Black delta    : 2.39
Dynamic range  : 10.75 (+) 9.95 => 13.92 EV (in theory)
Hot pixels     : 64
Cold pixels    : 378
Noise level    : 2.17 (16-bit)
Dynamic range  : 14.25 EV (cooked)
Black adjust   : 1
Output file    : _UAL6623.DNG

Quote
Input file     : _UAL6624.cr2
Canon EOS 5D Mark III detected
White level    : 12500
Black level    : 1961
Noise levels   : 16.08 6.13 6.22 15.70 (14-bit)
ISO difference : 4.91 EV (3006)
Black delta    : 1.01
Dynamic range  : 10.75 (+) 9.36 => 14.27 EV (in theory)
Hot pixels     : 5
Cold pixels    : 419
Noise level    : 1.66 (16-bit)
Dynamic range  : 14.63 EV (cooked)
Black adjust   : 1
Output file    : _UAL6624.DNG

Quote
Input file     : _UAL6625.cr2
Canon EOS 5D Mark III detected
White level    : 12500
Black level    : 1954
Noise levels   : 26.39 6.21 6.34 26.81 (14-bit)
ISO difference : 5.87 EV (5828)
Black delta    : -0.31
Dynamic range  : 10.73 (+) 8.62 => 14.48 EV (in theory)
Hot pixels     : 4
Cold pixels    : 649
Noise level    : 3.22 (16-bit)
Dynamic range  : 13.68 EV (cooked)
Black adjust   : 1
Output file    : _UAL6625.DNG

This occurred in 3 different scenes with varying levels of DR.    DxO results (http://www.sensorgen.info/CanonEOS_5D_MkIII.html) suggest that the advantage of ISO 6400 over ISO 3200 should be neither here nor there, and my own testing showed a very minimal gain (Std Dev continued to drop for a selection of flat textures in the scenes).

https://www.dropbox.com/sh/di2lkp2knv5bqso/XPN9ptKf-Y

edit:  The results are for base ISO 100
@ base ISO 200, the noise level only increases slightly for recovery ISO 6400.
@ base ISO 400 and 800, the noise level continued to drop for recovery ISO 6400 and increase for recovery ISO 12800.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: N/A on September 12, 2013, 02:59:05 AM
Getting some damn great pictures on the 600d, amazing work gentlemen.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Khairilasny on September 12, 2013, 03:11:02 AM
Getting some damn great pictures on the 600d, amazing work gentlemen.

2nd that!!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: optik on September 12, 2013, 05:36:44 AM
Is it me or the app please help I'm using  cr2hdr // Beta 1.0 app to convert dual ISO I have success  in windows but preferred to use Mac but this is the error that I keep getting

cr2hdr // Beta 1.0


Input file : 6I0A5237.CR2
sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile    : 6I0A5237.txt

THE END

Keep up the incredible work that you guys do and if you're working too hard then definitely get rest...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: fpena06 on September 12, 2013, 11:53:54 AM
Your machine is missing dcraw.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 12, 2013, 06:14:53 PM
I have done some test to try to understand how the new link between AETTR and DUAL ISO is behaving.

As I stated before, the new system when using the SNR = OFF or low SNR numbers, always defaults to Base ISO = Recovery ISO.  My explanation to this behavior is that the modules is doing all the calculations and deciding automatically that DUAL ISO is not needed and therefore making Base ISO = Recovery ISO.

So now I will test this theory as follows:
1. Set AETTR to 0.2% h.i. and SNR = OFF and took some pictures until the exposure converges.
2. Turn AETTR OFF (to maintain its corrected exposure) and use DUAL ISO to manually test different iso combinations.
3. Compare these results with AETTR+DUAL ISO at different levels of SNR

First is the manual method:
(http://s20.postimg.org/3yi29gvql/AETTR_w_Manual_DUAL_ISO.jpg)
In these images it is obvious that the shadow noise is improved with the use of DUAL ISO

Now lets look at the new AETTR + DUAL ISO
(http://s20.postimg.org/m01779prh/DUAL_ISO_AETTR_WITH_SNR.jpg)
We see that the system only responds to some very high SNR numbers 8/3 to reduce the noise in the shadows, but with lower numbers that shadows are totally ignored and DUAL ISO is disabled automatically. 
Also note that the AETTR with SNR=8/3 activated the DUAL ISO at 100/400 and this results where similar  to the simple 100/400 DUAL ISO test done first.   At  maximum SNR = 8/4 I was able to get 100/1600 dual ISO but it took a few tries until I got most of the DUAL ISO options to show up with the new system.

Conclusion is that the new system will use the DUAL ISO technology when needed to maintain the h.i. setting (0.2% in this test) and reduce the SNR shadow and midtones.  A similar result can be obtained by first doing the AETTR, disabling AETTR, and enabling DUAL ISO with manual settings.  The main difference is that the new system will search automatically for the DUAL ISO needed but the SNR numbers have to be very high for these parameters to make effect in the DUAL ISO.  I had used AETTR and never used 8/4 to obtained good results, I typically used 4/2 but at these levels DUAL ISO is not enabled.  This was my main conflict in thinking that it was not working but I just had to raise the SNR to the limits to get it to work.

This analysis then led me to explore what SNR settings will allow me to exploit the DUAL ISO benefits.  I found the following combinations that will give me what I am looking for in DUAL ISO performance with full sun and shadow outside setting.

SNR = 8/4 = Dual ISO = 100/1600
SNR = 7/4 = Dual ISO = 100/800
SNR = 6/3 = Dual ISO = 100/400
SNR = 5/3 = Dual ISO = 100/200

Now  I switched to a full sun and lots of full shadow through a window ( inside my house):

SNR = 8/4 = Dual ISO = 100/3200
SNR = 6/2 = Dual ISO = 100/1600
SNR = 3/1 = Dual ISO = 100/800
SNR = 2/1 = Dual ISO = 100/400
SNR = 1/1 = Dual ISO = 100/200

Now this leads me believe that we basically have maximum six levels of use for DUAL ISO when the condition are adequate.  If we have lower light situations where the base ISO is lower we lose one lever for each base ISO increase.  For example if we have base ISO of  200 we have five levels l (200/400, 200/800, 200/1600, 200/3200, 200/6400), for base iso = 400 we have four levels of adjustment (400/800, 400/1600, 400/3200, 400/6400).

This makes me think that we could simplify this system and present the user with just the number of shadow recovery options based on the base iso instead of having all of these SNR options.  For example if base iso is 100 then the allows adjustments are from 1 to 6.  If the base ISO=200 then we have only 1 to 5, etc.

Right now I am presented with 32 possible adjustments ie 8/4, 8/3, .... 7/4, 7/3,  6/4, 6/3 and some of them will not produce any apparent results.  Now if this could be reduced from 32 to 6 options then ease the usability will be much greater.

Just a final note the highlights have been independently maintained without overexposure which lead me to believe that are independently controlled by highlight ignore.  Will probably need a separate test.

And finally I thank you Alex for this amazing technology and I do appreciate all the hard work that you do.

P.S.  I am learning  from doing these test so please correct me if I did something wrong!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on September 12, 2013, 06:38:13 PM
Nice test!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 12, 2013, 07:12:59 PM
@RenatoPhoto: that's way too complicated. Simply state:

a) how much noise you are OK with (let's say 6EV or 7EV)

b) how many highlights can be ignored (0.1% is a good overall choice, 0.5 for something like a fairly large window in a dark cave where not even 20 stops of DR are enough, and 0% if you don't have any specular highlights).

Then let ETTR do the rest. That's what I do.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 12, 2013, 07:17:42 PM
I've been exploring the idea of using dual ISO for quasi ISOless mode.  Base ISO is always ISO 100, with adjustments to recovery ISO only.  ETTR+dual ISO is fine for still shots, but for running and gunning under varying light conditions, it is much to slow.

In a quasi ISOless mode, I could set dual ISO 100/400 and cover an exposure value range of 6-16EV (http://www.fredparker.com/ultexp1.htm#Light%20Intensity%20Chart) in Av mode (1/125-F/1.4 with this lens).  If I didn't care about my highlights so much, I could use dual ISO 100/1600 and cover an exposure value range of 4-16EV and deal with the reduced resolution in post.  With dual ISO 100/400, this also allows 2 EV of underexposure leeway for the base ISO.  ie: not only do I not need to worry about ISO adjustments (as much), I don't have to worry about being extremely accurate with my exposure to ensure my base ISO is always ETTR.  The recovery ISO takes up the slack ;)


Some caveats with the current functionality.
The histogram always uses the lowest ISO.  Where recovery ISO becomes the dominant brighter exposure, the histogram becomes unreliable.  ETTR hint does not work, and the histogram will no longer display overexposure results.
It doesn't appear as if base ISO is boosted in anyway (in cr2hdr) to fill in the exposure gaps in the recovery ISO, where recovery ISO is brighter.  Suggestion, where recovery ISO is brighter then base ISO, determine white level of base ISO, boost the exposure of the base ISO and use that data for AA correction in the recovery ISO. It's better to have some accurate data, even if that data has a higher noise level, then to interpolate missing data.
This could also be handy with the current functionality, to fill in any resolution loss where base ISO is not ETTR.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 12, 2013, 07:26:29 PM
I don't get it; cr2hdr only cares about higher iso and lower iso (doesn't matter which is base and which is recovery - this one is mostly relevant when using LiveView). So 100/1600 and 1600/100 are processed identically.

Of course, these two might have slightly different noise characteristics because Canon sets some more parameters, not just amplifier gains (maybe NR or who knows what else). In practice I didn't notice much difference.

The histogram always uses the lowest ISO for ETTR reasons (so you can see how far you can push it to the right). If something is covered by at least the lowest ISO, it's not overexposed.

About ISO-less mode: I found that for disco lights (changing all the time, but overall pretty dark with some very bright areas), 1/30 (slower than that results in shake) and ISO 100/1600 covers pretty much everything (some sort of auto that actually gets it right).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 12, 2013, 07:58:15 PM
The test shot I was looking at had some overexposure that I did not determine during capture.  I then assumed that the resolution loss in the highlights was due to not using the darker exposure  to fill in the resolution loss of the brighter exposure.  I guess because I am always thinking of base ISO as the darker exposure.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 13, 2013, 12:31:58 AM
@RenatoPhoto: that's way too complicated. Simply state:

a) how much noise you are OK with (let's say 6EV or 7EV)

b) how many highlights can be ignored (0.1% is a good overall choice, 0.5 for something like a fairly large window in a dark cave where not even 20 stops of DR are enough, and 0% if you don't have any specular highlights).

Then let ETTR do the rest. That's what I do.

Thanks for your reply,

I did the first photos in manual mode to explore the area that the new AETTR + DUAL ISO does not allow me to explore. 
For what purpose?  So I can compare these iso combinations and results with the results of AETTR + DUAL ISO. 
Why do I want to explore this area?  Because three days ago I could do it by simply putting my Recovery ISO = +x EV.  Now I cant get these results because the new module just decided that this is not what should be done and sets BASE ISO = RECOVERY ISO based on the SNR settings.

Request: Can the DUAL ISO module be configured with an option to do the automatic ISO calculation and one option to do what it did before?  i.e.  Auto Dual Iso (ON - OFF)?  This would give this module the flexibility to take your optimization or not.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 13, 2013, 12:36:31 AM
Show me a scene that requires dual ISO, but auto ETTR said it doesn't.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 13, 2013, 12:47:22 AM
Show me a scene that requires dual ISO, but auto ETTR said it doesn't.
In the first row of images above, all of them some of them have lower noise than that obtained by AUTO ETTR + DUAL ISO.
Why?  Because I am using higher ISO combinations that the new module does not allow bases on the  calculations.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 13, 2013, 12:55:22 AM
Let me correct that, I get lower noise with higher ISOs that the new module does not allow for the scene.  For example I the 100-800 dual iso image is lower in noise than the 100-400 which was the maximum iso range that the module allowed for this scene.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 13, 2013, 03:06:33 AM
In the images you have AETTR SNR 8/3 @ ISO 400 for auto recovery.  Why did you not have SNR 8/4?
You then go on to show in other scenes (with no images), where setting SNR 8/4 resulted in much higher recovery ISOs.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 13, 2013, 03:22:57 AM
In the images you have AETTR SNR 8/3 @ ISO 400 for auto recovery.  Why did you not have SNR 8/4?
You then go on to show in other scenes (with no images), where setting SNR 8/4 resulted in much higher recovery ISOs.

Yes in this scene I din not think it was required to max out the SNR but later I realized that you could get higher ISO if you max out at 8/4.  Under similar conditions as the presented photos I was only able to get 100-800 ISO but no higher as I wished to test.

In similar tests the obtained result under similar ISO are similar but the point I am trying to make and seems difficult to understand is that a user may want to test higher ISOs which are now not available under automated scenarios of AETTR + DUAL ISO.  These scenarios where quite easy to use by allowing the ISO module to set the Recovery ISO as a relative number of +xEV.  I personally think this is good to leave in the module.  Currently the +xEV option is not working in Aettr.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 13, 2013, 04:49:00 AM
Can you describe a use for manually setting the recovery ISO value other then testing?

In real world shooting, SNR settings are perfectly fine for determining the required amount of recovery ISO.  The SNR settings are a fine balance of midtone/shadow SNR requirements vs dual ISO setting.  ie:  Rather then using recovery ISO 1600 just because, by setting the SNR limits based on actual noise level figures, you could shot the scene with a recovery ISO of 400.

Why is this useful?

SNR limits take into account actual noise figures which directly effect the result.
By allowing the automated code to use these values, your scene at recovery ISO 400 vs recovery ISO 1600 has 2 stops better highlight detail, while still retaining the desired amount of SNR in the midtones/shadows.

Let go of any misconstrued conceptions you have about the SNR setting.

Think of SNR limits as DR control (remember highlight control is directly controlled by highlight ignore and is no longer affected by SNR limits (with dual ISO enabled)).  Want more DR (less noise in the midtones/shadows), increase SNR value (which in turn will increase the recovery ISO as needed).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on September 13, 2013, 07:24:52 AM
@audionut. Question. How do I work with the test builds? I used to download Lourencos test builds but there seems to be more going on at this forum? For instance If I would like to try any testing with the dual iso module. Do I download the Audionut test build or do I simply add the dual iso module to my build from Lourenco build which is from august 21?
Thanks guys
//D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 13, 2013, 07:33:21 AM
Looks like the dual_ISO module has made it's way into the new nightly builds  :)

http://builds.magiclantern.fm/#/
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on September 13, 2013, 07:44:49 AM
Wow! Smooth platform :)
Thanks!
//D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 13, 2013, 08:09:19 AM
There appears to be a problem with the noise calculation for ISO 6400

Fixed; the noise in top-left black corner was not white, and the blending algorithm was interpreting it as detail (which uses data from both ISO - a lot noisier).

There might be a tiny difference in the output because of slightly different noise values (they are used as thresholds in a few places). Unlikely to be noticeable.

Now I also print the noise level from the high-ISO image (ideally, the noise in very dark areas should reach that value). If it's slightly bigger, it's not really a problem (the noise may sometimes be mistaken for detail); if it's a lot higher (say 1 stop difference or more), it may indicate a bug or a problem in the sensor output (e.g. lots of hot pixels).

Next step: I want to print the SNR values too, in order to check whether ETTR works properly.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 13, 2013, 09:38:09 AM
Can the spot meter sample different areas of the image simultaneously?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 13, 2013, 10:42:26 AM
What's the use case? You can move it around.

There was a demo script with many spotmeters, if you want to try to fix the scripting engine.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 13, 2013, 11:51:09 AM
Using it to print out SNR values for the midtone/shadows on LCD.  Overlayed on the midtone/shadow areas of the image?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 13, 2013, 01:27:38 PM
I repeated the test to check the snr 8/3 setting.  The maximum ISO ranged achieved was 100-400.

In DUAL ISO  you can see that there is some improvement in noise available above 100-400.  I expected to see less noise at hgher ISO like 100-1600 but did not get it.. I cannot explain this but I had seem lower noise at higher ISOs with previous builds.  But anyway there is room for a bit of noise reduction and the SNR algorithm does not allow.

It is like saying that cannon should only give you ISO 100-1600 instead of 100-25400 because technically there is no use.  But still some people may want to use it, form the usability point of view I am suggesting to expand the range a bit more.

I can imagine that after performing the math, there is not much difference in SNR between the 100-400 and the 100-800, but I see it, so the math should conform to what I see and not to what the numbers tell.  After all we always try to make our images look like what we perceive i.e. higher dynamic range for example.
(http://s20.postimg.org/sobkz3df1/New_Comparison_dual_iso.jpg)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 13, 2013, 03:33:51 PM
No SNR 8/4 test?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 13, 2013, 03:47:35 PM
Yes, my mistake but the snr 8/4 also had the same 100-400.  I will try again and see if I can get the higher ISO.  As I said the module limits the ISO range so it not always possible to get higher ISO. 
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 13, 2013, 04:01:30 PM
Yes, I was correct, I took some pictures and the same thing at 8/4 get iso of 100-400.  The image shows as 8/3 is in fact 8/4
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 13, 2013, 04:07:55 PM
Here is the scene, bright sun with deep shadows:

This one is DUAL ISO 100-3200

(http://s20.postimg.org/gu8vyklgt/Dual_ISO_scene_from_DNG_100_3200.jpg)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 13, 2013, 11:09:56 PM
Without the SNR 8/4 CR2 to analyze, it's going to be near impossible for a1ex to fix any issues.

My first best guess would be that the area you are showing in your cropped observations above, is below the 5% threshold.

For shadows, I've chosen the 5% percentile. So, in this context, the shadows having a SNR of 3 EV means 5% of the image pixels have a SNR lower than 3 EV, and the other pixels will be brighter than that.

Keep in mind that 5% is bigger than you may think, since it refers to image area, not linear size. The linear percentage is roughly 22% (1/4.5).

Secondly, I assume those crops are of the extremely dark shadow area in that wood shed?  Would you really tone map that to a brightness level where detail is visible?
Would you tone map that area to a brightness level where the detail difference between 100/400 and 100/800 is visible?

If the answer is yes to the second tone map question, have you observed the highlight detail trade offs?  It's not fair to only observe the shadow areas when highlight detail is also affected by the EV difference between the 2 ISOs.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 14, 2013, 12:20:11 AM
Without the SNR 8/4 CR2 to analyze, it's going to be near impossible for a1ex to fix any issues.
Should be able to get to some bandwidth sometime next week.  It seems that this type of condition could be found in many situations where the sun is bright.

My first best guess would be that the area you are showing in your cropped observations above, is below the 5% threshold.
I kind of feel that I am splitting hairs so the resolution to the issue would be better addressed by allowing the user to simply choose between this type of calculation or not.  I was quite happy with the previous version  and a simple ON-OFF will be great.

Secondly, I assume those crops are of the extremely dark shadow area in that wood shed?  Would you really tone map that to a brightness level where detail is visible?
Would you tone map that area to a brightness level where the detail difference between 100/400 and 100/800 is visible?

I have already brighten the shadows to expose the noise difference.  Although there is less noise at higher ISO combinations it is hardly noticeable.  In retrospect the level of noise given the super dark are is supper acceptable at 100-800 or 100-400.

Secondly, I assume those crops are of the extremely dark shadow area in that wood shed?  Would you really tone map that to a brightness level where detail is visible?
Would you tone map that area to a brightness level where the detail difference between 100/400 and 100/800 is visible?

If the answer is yes to the second tone map question, have you observed the highlight detail trade offs?  It's not fair to only observe the shadow areas when highlight detail is also affected by the EV difference between the 2 ISOs.
In my opinion there was very little issue with the highlights, the algorithm was working well and keeping overexposure to the 0.2% h.i, so there is no tradeoff.  This kind of result is achievable thanks to DUAL ISO.

-----------

Finally and most importantly

I shoot a lot of wildlife and the speed of congruence is very crucial; now when the modules are linked the calculations have become slower so more shots are required.  This also is another reason to leave AETTR do its thing and the add DUAL ISO just on top of AETTR as it was before with +-XEV option for Recovery ISO.  I had become very happy with setting my AETTR at 4/2 and DUAL ISO at +3EV and shooting without too many delays.  Now I cant do this, I just takes one or two or three more shots to get optimum results.

And finally dont take me wrong the images are great and the technology is great, but please put an ON-OFF switch for the more precise calculations.  Please!

Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 14, 2013, 12:55:46 AM
I have already brighten the shadows to expose the noise difference.  Although there is less noise at higher ISO combinations it is hardly noticeable.  In retrospect the level of noise given the super dark are is supper acceptable at 100-800 or 100-400.

So ideally, the fundamental code base is sound.  And you're just splitting hairs.


I shoot a lot of wildlife and the speed of congruence is very crucial; now when the modules are linked the calculations have become slower so more shots are required.  This also is another reason to leave AETTR do its thing and the add DUAL ISO just on top of AETTR as it was before with +-XEV option for Recovery ISO.  I had become very happy with setting my AETTR at 4/2 and DUAL ISO at +3EV and shooting without too many delays.  Now I cant do this, I just takes one or two or three more shots to get optimum results.


I have noticed that it's ability to converge quickly is directly related to the initial exposure settings.  Start off overexposed and expect problems (especially with the following).
Scene conditions and expectations play a large part.  In a scene with a very large DR where you have no, or very little highlight clipping, and you have the SNR settings @ 8/4, you're asking quite a bit.

Relax your expectations of what should be achievable, perhaps use SNR 7/3 instead, and convergence is met much faster.

Having said all that, I have mentioned,

Or maybe a setting in dual ISO, "link to AETTR".  Else dual ISO is always enabled at the settings specified.

I guess then a1ex would have to ignore any SNR settings applied (just consider highlight ignore and leave the user to make his own mistakes).  That sounds ugly and isn't something I would expect a1ex to be open to.
Or perhaps fully disable SNR settings when dual ISO is not linked to AETTR.

For the absolute fastest results, you should look at AutoExpo/Dual ISO combination.  There are no LV calculations.  Point and shoot at burst speed :)
You lose some highlight precision sure, but lets face it, 0.5EV below FWC and 1.0EV below FWC (with the recover ISO taking up the slack), is probably splitting hairs also.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 14, 2013, 02:41:31 AM
So ideally, the fundamental code base is sound.  And you're just splitting hairs.

These are my observations, I cannot analyze code.

I guess then a1ex would have to ignore any SNR settings applied (just consider highlight ignore and leave the user to make his own mistakes).  That sounds ugly and isn't something I would expect a1ex to be open to.
Or perhaps fully disable SNR settings when dual ISO is not linked to AETTR.

Just give the user a choice between slow converging exact process or a quick process that produces excellent results. 

We know that DUAL ISO should be only used in scenes where the dynamic range is greater then what the camera can capture.  So that means we are going to capture the highlights as well as the shadows.   We have also determined that h.i. controls the highlights and so to increase dyamic range we protect the highlights with h.i. and capture the shadows with DUAL ISO.  That is what I have been doing before.  The SNR numbers give me a scale of shadow recovery which is very complicated 32 different options while DUALISO gives me 6 options.  For this reason, when I was using AETTR+DUAL ISO I was leaving SNR OFF and then enabling the DUAL ISO at a reasonable number (ie +3EV) so I could capture the details in the shadows at acceptable noise level.

In the scenes where the dynamic range is not great, or in scenes where I am not concerns with the highlights, the I USE SNR 4/2 and 6/3 as I see fit based on experience.  This has always given me good results.  I even tested this in Timelapse with very good results see here:

Dual ISO vs Auto ETTR
http://www.magiclantern.fm/forum/index.php?topic=7835.0

This new system took away everything that I like and replaced it with a better, more accurate, splitting hairs system, which I may want to use from time to time when time allows and I can take a few shots at a static scene.

For the absolute fastest results, you should look at AutoExpo/Dual ISO combination.  There are no LV calculations.  Point and shoot at burst speed :)
You lose some highlight precision sure, but lets face it, 0.5EV below FWC and 1.0EV below FWC (with the recover ISO taking up the slack), is probably splitting hairs also.

Interesting, I will have to look into this option.  I tried it a couple of times but so many freaking curves that I could not make scense of what this thing is doing.  It needs a user manual for dummies like me.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 14, 2013, 08:54:59 AM
I can imagine that after performing the math, there is not much difference in SNR between the 100-400 and the 100-800, but I see it

I can barely see any difference between 100/400 and 100/6400. Maybe you can use a higher contrast for that area?

Maybe I should check the standard deviation of the noise in some out-of-focus area.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on September 14, 2013, 10:51:17 AM
@Audionut
"For the absolute fastest results, you should look at AutoExpo/Dual ISO combination.  There are no LV calculations.  Point and shoot at burst speed :)
You lose some highlight precision sure, but lets face it, 0.5EV below FWC and 1.0EV below FWC (with the recover ISO taking up the slack), is probably splitting hairs also."


Been fiddling with the Auto exposure settings and it seems to work very well with dual iso. My settings right now pointing out the windows seems to give consistent good results;
 
AV range, I used f5.6-f5.6 but free of choice of course
TV minimum 1/44, free of choice of course
EC range -1.5 EV- +3.0 EV try this one out a little
EC +3.0 EV probably the most important setting to change. Try +2.0 EV or til you get it as you want
ISO range 100-3200

The big plus here is you don,t need to open up liveview to get a reading as with Aettr. The low iso, though, has to be changed manually from within dual iso.
Gonna try some more...

*After developing I can only say wow. This autofeature is exactly what I,ve been looking for. If an automated feature were implemented to disable dual iso when in not so contrasty areas and maybe also have an auto setting for the lower iso setting I could finally shut of my brain and start taking pictures ;)
*So, to simplyfi. Set auto exposure to overexpose by two or three steps, then hit dual iso to get the blown highlights back. Am I right?

//D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 14, 2013, 11:36:48 AM
Set auto exposure to overexpose by two or three steps, then hit dual iso to get the blown highlights back. Am I right?

Only for your particular test scene  ;)

That's because auto expo meters for midtones and ignores highlights. Try on a blank wall, then try pointing it out the window. I just did on 5D3: on the blank wall it underexposed by 2-3 stops, on the window it overexposed by 3-4 stops. I've tried evaluative and center weighted, could not see any difference between them.

Quote
If an automated feature were implemented to disable dual iso when in not so contrasty areas...
That's exactly what auto ETTR does (and Renato is reporting it as bug).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on September 14, 2013, 11:41:17 AM
 
Only for your particular test scene  ;)
:P
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 14, 2013, 01:14:23 PM
Quote
Or maybe a setting in dual ISO, "link to AETTR".

What about a setting like this in ETTR?

Link to Dual ISO
- OFF (old way)
- SNR priority (as it's now)
- Highlight priority (similar to SNR, but if dual ISO reaches 100/max, ignore the SNR and capture all the highlights).

I'm still working on convergence issues; with ETTR in LiveView (on request via SET button), convergence is quite fast and you don't have to waste any test shots.

The biggest problem when metering outside LV is that current heuristics (simple and fast) sometimes get tricked because of black level difference. With a black difference of around 30 (typical for ISO 100/6400), the metering error can be up to 4 stops (!).

In LiveView, ETTR is metering on only one of the two exposures (no black difference), and estimates the other exposure from DxO data. That's why it works a lot better than metering outside LV.

Another cause of non-convergence is that 1600/3200/6400 are almost identical, so the algorithm may switch from one to another just because of round-off errors. This one should be easy to fix.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 14, 2013, 01:47:16 PM
That's exactly what auto ETTR does (and Renato is reporting it as bug).

Hi Alex,
I like the feature very much, I like the concept of using DUAL ISO only when the dynamic range requires of its use.  ;)  Please dont take it out, it has its uses.
I find the implementation a bit difficult to use with ease and speed. I find the final hair splitting a bit inconsistent and difficult to obtain.

I am suggesting in order of importance:

1. Implement an ON OFF switch for the more robust and automated calculations.
2. When the switch is in OFF position, and DUAL ISO is enabled set h.i.=0.2%, SNR=OFF, have AETTR to its thing, have DUAL ISO take the Base ISO and add the user selected Recovery ISO via +-XEV
3. If user has switch in OFF position, and DUAL ISO is enabled and the user wants to use other settings for SNR or h.i., then let him but warn him that these settings work best with the automated version.  But I suppose there is no sense in setting h.i=5% when using DUAL ISO, it defeats its use since you no longer require DUAL ISO to capture the whole dynamic range.  The SNR settings could be fiddle with as long as the highlights are not clipped.  If the user boosts SNR and the scene is overexposed then warn the user.
3. When the switch is ON your calculations take over, the SNR is optimized via combination of AETTR and DUAL ISO. 

I see you posted above while I was writing..

Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 14, 2013, 01:56:43 PM
But I suppose there is no sense in setting h.i=5% when using DUAL ISO, it defeats its use since you no longer require DUAL ISO to capture the whole dynamic range.

Sometimes you do. I had to do this when taking photos in a cellar - there was a bright window (something like this (http://www.festeggiata.com/included/images/cellar04_600x400.jpg) - not my picture, but a similar scene), and it was impossible to have detail there and see something in the cellar, so I had to ignore it. Of course, dual ISO maxed out.

Also street lights: they are usually so bright that dual ISO can't cover them, and you have to ignore them (otherwise you may get a lot of noise).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 14, 2013, 02:25:49 PM
What about a setting like this in ETTR?

Link to Dual ISO
- OFF (old way)
- SNR priority (as it's now)
- Highlight priority (similar to SNR, but if dual ISO reaches 100/max, ignore the SNR and capture all the highlights).

I am sure that your fingers will do the magic!  ;D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mikeortner on September 14, 2013, 04:39:38 PM
Can someone tell me what i am doing wrong, when the cr2hdr.app(OSX) is not processing the dng files of my raw video?
"Hmm… There are no more Dual ISO .CR2 left in this folder."

do i have to rename the dng to cr2?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 14, 2013, 05:15:35 PM
What about a setting like this in ETTR?

Link to Dual ISO
- OFF (old way)
- SNR priority (as it's now)
- Highlight priority (similar to SNR, but if dual ISO reaches 100/max, ignore the SNR and capture all the highlights).

They look good.

I like the highlight priority option.  I find it tends to use 100/max twice at the end and then gives up.  This would have to help convergence (I shot through the viewfinder), as instead of giving up after 4 shots, based on my testing it would often give up after 2 (save the highlights and be done with it).

The biggest problem when metering outside LV is that current heuristics (simple and fast) sometimes get tricked because of black level difference. With a black difference of around 30 (typical for ISO 100/6400), the metering error can be up to 4 stops (!).

In LiveView, ETTR is metering on only one of the two exposures (no black difference), and estimates the other exposure from DxO data. That's why it works a lot better than metering outside LV.

Can you make it sample just the darker exposure (outside of LV)?  Would reading the current setting of dual ISO help to determine the black level difference?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 14, 2013, 05:31:59 PM
Can someone tell me what i am doing wrong, when the cr2hdr.app(OSX) is not processing the dng files of my raw video?
"Hmm… There are no more Dual ISO .CR2 left in this folder."

do i have to rename the dng to cr2?

First look at the cr2 files at 100% or 200% magnification and you should see the interlaced dual iso.  If you do not see the horizontal lines then you do not have dual iso image.

I cannot help you with apple stuff..
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 14, 2013, 05:34:32 PM
Quote
Can you make it sample just the darker exposure (outside of LV)?

If the bright exposure has black level at say 2033 and the dark one has black level at 2063, in deep shadows, the current code will sample the bright exposure thinking it's actually the dark one (since it simply returns min or max from two neighbor pixels).

Ideally, bright exposure = dark exposure * gain (that is, bright EV = dark EV + ISO spacing in EV).

Say the midtones are at 3 EV above black in the dark exposure (that is, 2063+8). In the bright exposure, they will be at 2033+8*64 for ISO 6400. So, midtones will be metered at 4.5/8.95 instead of 3/9 (so, at 3 EV I will have a 1.5-stop error in metering).

To correct this, I could either:
- figure out which lines are bright and which lines are dark before sampling anything (requires rewriting a lot of things)
- figure out how big the error is (because I can find the black difference easily) and correct the metered values

Edit: actually the exposure inversion is not a very big problem (it only appears in really deep shadows); the problem is the magnitude of the black level difference (which alters metering results).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: khurra on September 14, 2013, 05:51:21 PM
does cr2hdr work on 650d RAW files?

i converted the 650d dual iso RAW file to DNG using raw2dng and then changed file names to .CR2 and then tried opening with the cr2hdr for mac, but got this error:

Input file : 000000.CR2
sh: exiftool: command not found sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile    : 000000.txt

Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 14, 2013, 06:32:53 PM
We know that DUAL ISO should be only used in scenes where the dynamic range is greater then what the camera can capture.

Not any longer, not with SNR limits.  It can also be used to tone map in camera.  Say I have a scene of 9EV of dynamic range, but in this scene the skin tones are at -5EV or lower, I can enable the required SNR limit, and boost the skin tones to -3EV (much less noise in critical areas).

I haven't increased DR, because the DR was scene limited.  I did lift the skin tones though.

With adjustments to both midtone/shadow detail, on scenes with large dynamic range, I can also do my best in camera to lift skin tones while leaving the shadows to fall where they may (minimize EV spacing).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: nanomad on September 14, 2013, 08:17:01 PM
does cr2hdr work on 650d RAW files?

i converted the 650d dual iso RAW file to DNG using raw2dng and then changed file names to .CR2 and then tried opening with the cr2hdr for mac, but got this error:

Input file : 000000.CR2
sh: exiftool: command not found sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile    : 000000.txt
They work but you are missing the dcraw executable
Title: Re: Dual ISO - massive dynamic range improvement
Post by: khurra on September 14, 2013, 09:11:25 PM
They work but you are missing the dcraw executable

so i need to follow these instructions:  http://bellavist.com/blog/?p=466

on my computer running OSX lion and then try running the mac gui verison of cr2hdr again?

thanks for your help nanomad!!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 14, 2013, 11:41:30 PM
ETTR seems to work a bit better now with dual ISO. There's a little quirk that you may want to be aware of:

- without dual ISO, it will try hard to meet or exceed the SNR limits (clipping as many highlights as needed)
- with dual ISO, SNR limits are respected until dual ISO gets maxed out (100 / max auto ISO). After that, it will not clip any more highlights than allowed by "highlight ignore" (but it will also get a lower SNR).

I recommend triggering it via SET (to meter in LiveView) whenever the lighting changes. This mode converged very quickly in my tests, even in the most difficult lighting (I've tried a light bulb in a very dark room).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 15, 2013, 12:05:42 AM
ETTR seems to work a bit better now with dual ISO. There's a little quirk that you may want to be aware of:

When i try to do hg update I get the following error and ettr.c does not update.  Any  ideas what I can do?

hg update
modules/bolt_rec/README.rst: untracked file differs
abort: untracked files in working directory differ from files in requested revision
[email protected]:~/magic-lantern$


Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 15, 2013, 12:08:59 AM
Try deleting it.

http://stackoverflow.com/questions/4822129/mercurial-untracked-file-in-working-directory-differs-from-file-in-requested-r
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 15, 2013, 12:37:04 AM
Try deleting it.

Thank you very much.  I deleted README.rst
and hg update...
Now I am testing.  It feels wonderful!
Thank you again your are a magician!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 15, 2013, 01:38:06 AM
I had a quick look at the images and everything has fallen in its place.
Nice images, good capture of dynamic range, proper automatic decisions on when to use DUAL ISO, very responsive to SNR settings.  The usability factor seem very good.
It seems PERFECT!  (I will probably find something else to complain about... :o)

If I could, I would nominate you for a Nobel Prize for ingenuity.  ;D ;D ;D

P.S I was forgetting to mention that now the camera can capture what I see and beyond!!!!

P.S. 2 Maybe they should add this features to imaging space telescopes and I bet we could see little green men on the sun!!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mikeortner on September 15, 2013, 02:13:41 AM
First look at the cr2 files at 100% or 200% magnification and you should see the interlaced dual iso.  If you do not see the horizontal lines then you do not have dual iso image.

I cannot help you with apple stuff..

Yes the lines are there, i already checked that before ... maybe i should try the PC version
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on September 15, 2013, 10:13:49 AM
The aettr together with dual iso works really well now :). THanks for that. Trying to understand when to use what settings in snr for shadow and midtone. 8/4 is the most extreme and 1/1 the least extreme sort of speak. What would be the best way to find rules for when to use what? I read comments about skintones etc. Anybody tried any settings? Is there a way to visually understand the optimal snr settings maybe with raw zebras, histogram?
Thanks again
//D
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 15, 2013, 10:17:35 AM
Audionut did some tests here and I chose the defaults according to them: http://www.magiclantern.fm/forum/index.php?topic=5200.msg70150#msg70150

Also see this: http://photography-on-the.net/forum/showthread.php?t=1326078&page=2&highlight=snr

I'm thinking to draw something to show where midtones and shadows are metered exactly. If you remember the old bulb ramping with percentiles, it would be something like that (since the math is the same).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on September 15, 2013, 10:37:22 AM
Audionut did some tests here and I chose the defaults according to them: http://www.magiclantern.fm/forum/index.php?topic=5200.msg70150#msg70150

Also see this: http://photography-on-the.net/forum/showthread.php?t=1326078&page=2&highlight=snr

I'm thinking to draw something to show where midtones and shadows are metered exactly. If you remember the old bulb ramping with percentiles, it would be something like that (since the math is the same).

Alex, I noticed something today. With Dual_ISO & AETTR on my 5DM3 on the numerical EV showing at the bottom right end of the screen under my light situations shows +0.3 EV when ML ISO is at 100 and Canon ISO at 3200; and +4.0 EV when ML ISO is at 3200 and Canon ISO at 100. The same for my EOS M.

Could there be a valid explanation for the cameras behaving this way?

Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 15, 2013, 10:39:33 AM
Canon metering always meters the image that you are previewing.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on September 15, 2013, 10:43:04 AM
Canon metering always meters the image that you are previewing.
Okay. That makes sense. Thanks as always. 
Title: Re: Dual ISO - massive dynamic range improvement
Post by: akumiszcza on September 15, 2013, 12:00:58 PM
I'm testing new build with auto ettr & dual iso link on my 50D. The idea is great, photos look nice. Some issues:
- Quite often auto ettr starts (LV turns on for a split second and turns off), but It does not do anything. It does not say "Expo out of limits" or "Whoops". Thinking it's on I shoot and get much underexposed photo. When I turn LV on and press SET, Auto ETTR works fine. Would it be possible to write "auto ettr ok" on image review after the shot if ettr worked and finished with no errors?
- auto snap and always on does not work on 50D still. It works with 1%'s tragic lantern builds – tl50d compiled by Andy600, but not nightly builds.

It looks like exposure limits for auto ettr is greater than before, too.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 15, 2013, 12:49:23 PM
Status messages solved. Also found some more bugs regarding convergence.

Exposure limits are Canon limits, nothing should change here. I didn't check them very well though.

For auto snap, one has to compare 1% repo with ML one, extract the relevant pieces of code (most likely ettr.c, dual_iso.c and the raw backend) and create a pull request. I prefer this to be done by a 50D user.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 15, 2013, 01:11:02 PM
@alex (sorry if this request has been issued before, you know, long threads and all...):

Could you please change the "dr gained" field into a setting so that the user can set a "min. dr gain" value? On high iso, doing dual iso doesn't make much of a difference dr-wise and isn't worth the postprocessing hassle, so this would only enable dual_iso on lower base iso.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on September 15, 2013, 02:12:00 PM
THanks for linking ALex. Seem 6ev is a good compromise getting about enough grain.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 15, 2013, 04:26:44 PM
I haven't done any tests yet, but judging from that last test, I would use 7EV or 8EV for the midtones with dual ISO, depending on your preference of EV separation.

6 EV could lift the midtones well while still retaining good highlight detail, back before it had the help of dual ISO.  7EV should now give you good midtone detail with little EV separation (100/400).  8EV would give you excellent midtone detail (metered correctly), with only 100/800.

Of course the recovery setting is dependent on scene luminance.

Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on September 15, 2013, 04:38:11 PM
Thanks for clarifying.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 16, 2013, 02:09:39 PM
Reporting a repeatable bug 5D3, compile ae68c60000f2

Tried to use AETTR+DUAL ISO for a night-to-day timelapse.

The module started ok at typical setting ie. 30 seconds at iso 6400.  As light increased the exposure never changed and remained at 30 seconds iso 6400.  I caught it early but if the sun hit my sensor it could have fried it.

I repeated the test under my desk in a very dark place and then pointed to my monitor with lots of light.  The exposure never changed from 30 sec at iso 64000.

Settings used:

Manual lens f/22, iso 6400, 30 second shutter, Canon in M mode, Image review 2 sec

AETTR settings: Trigger mode: Always on, Slowest shutter: 32", Exposure target: -0.5, H.I:02%, Allow clipping:Green channel, Midtone SNR limit: 7EV, Shadow SNR limit:2EV, Link to Canon:OFF, Link to DUAL:ON

DUAL ISO: ON

Intervalometer: Take pic every 1 m, Start after 3s
Post Deflicker: Adobe xmp, 50%, -4 EV
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 16, 2013, 03:46:14 PM
Good catch.

What's happening here is that sensor saturation point (at least on 5D3) goes lower than usual, so the zebra indicator and ETTR analysis don't even realize the image is overexposed (they think it's just perfect).

If you change the exposure target at -1 EV, it will start correcting the overexposure slowly (one stop at a time). It will continue to do that until the white level returns to normal.

At 8 seconds, the white level looks normal (from there it converged from half-screen blown out to perfect exposure in 2 steps). Didn't try in-between shutter speeds.

The issue is quite subtle and I'm not sure how to fix it in a portable way. Maybe with some guessing like in cr2hdr.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 16, 2013, 04:00:27 PM
What's happening here is that sensor saturation point (at least on 5D3) goes lower than usual, so the zebra indicator and ETTR analysis don't even realize the image is overexposed (they think it's just perfect).

Maybe this is a good time to add a check based on canon brightness value.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 17, 2013, 11:52:05 AM
The new metered areas is handy.

Currently it just continues to draw over itself.  Is there a way to determine if the scene has changed dramatically and clear the overlay for a fresh draw?  I don't know how to do this, so not sure of the best way to stop it continually updating where it doesn't need to.
edit:  Since it seems to just draw everything again (when updating), what about just clearing the old overlay first before drawing again?

What about linking it to zebras?  An option to show either the zebras as they are now, or the metered areas.  Not sure of the cleanest way to do this.

Can it be enabled for QR?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 17, 2013, 11:58:21 AM
Heh, I've only tested it in QR.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 17, 2013, 12:23:42 PM
I must need sleep then.  It works fine in LV, and works fine on press set, but nothing shows in the quick image review.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 17, 2013, 03:43:42 PM
Good catch.

What's happening here is that sensor saturation point (at least on 5D3) goes lower than usual, so the zebra indicator and ETTR analysis don't even realize the image is overexposed (they think it's just perfect).

If you change the exposure target at -1 EV, it will start correcting the overexposure slowly (one stop at a time). It will continue to do that until the white level returns to normal.

At 8 seconds, the white level looks normal (from there it converged from half-screen blown out to perfect exposure in 2 steps). Didn't try in-between shutter speeds.

The issue is quite subtle and I'm not sure how to fix it in a portable way. Maybe with some guessing like in cr2hdr.

Thanks Alex, I tested with Timelapse today and worked properly.  Now I have to figure out how to process some of the dual iso images in Timelapse.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: NateVolk on September 18, 2013, 12:30:40 AM
Did a shoot last night with the newest build.  Used auto ettr linked to dual iso.  Not sure I'm doing it right, but got good results with the standard settings.  After cr2hdr, the dng files seemed under exposed by about 1.5-2 stops but I was able to raise them without a problem.  Is this the way it is supposed to work, or am I missing something?  Thanks!

https://docs.google.com/file/d/0B_Id0y3JG-KEdU9qZGQ2WkFuQTQ
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Danne on September 18, 2013, 12:50:45 AM
I think you get the dark exopsure which is the way it was exposed keeping the highlights. As you say, to lift the shadows are not any problem
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 18, 2013, 03:56:38 AM
Reporting new test of TIMELAPSE with ETTR and DUAL ISO link:

5D3 with yestedays compile before memory back-end.  be77944f2b0d

I tested again night to day transition using ETTR+DUAL ISO, SNR 6/3.  The previous exposure problems are gone.

The DUAL ISO module did not become necessary until the dynamic range increased when daylight started to increase the brightness of the sky.  All cr2 files where able to read the the xmp in ACR.  All the xmp files had adjustment applied to cr2 and then exported to DNG. 

I was able to process the DUAL ISO (DNG) files with this workflow:

1. Opened all cr2 with ACR and exported as DNG, then processed all DNG with cr2hdr, then dng to tiff with ACR.

And the beginning of DUAL ISO enhancement, there were times when one image would be dual iso and then the next did not, after development and final review the images blended very well.  Except for a couple of times when a small flicker appeared.  By simple analysis I could tell which was dual iso and which was standard.

I did not let the system go into full daylight but I think that it is important that the dual iso system does not go beyond 100/800 iso to maintain good images.  For this reason I think the use of dual iso module should be limited to SNR settings no higher than 6/3.   A setting like 8/4 would raise the dual iso to 100/6400 which would deteriorate the image resolution and colors.

Hopefully will be able to run a new test with extended times for a more complete analysis.

The dickering with slide car files is amazing!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on September 18, 2013, 06:40:52 AM
Reporting new test of TIMELAPSE with ETTR and DUAL ISO link:

5D3 with yestedays compile before memory back-end.  be77944f2b0d


Hi RenatoPhoto, where can I get hold of the binary for the be77944f2b0d? Searched, the forums but could not find any refernce to this specific version.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 18, 2013, 07:53:14 AM
To limit the max ISO, change it in Canon menu (max auto ISO).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 18, 2013, 01:51:17 PM
Hi RenatoPhoto, where can I get hold of the binary for the be77944f2b0d? Searched, the forums but could not find any reference to this specific version.
You would have to compile it yourself.  I suppose today's build should be as good, I don't know if there are any issues.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 18, 2013, 01:52:57 PM
To limit the max ISO, change it in Canon menu (max auto ISO).
The problem is I need the base ISO of 6400 for the night shots but I dont want DUAL ISO to go very high.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on September 18, 2013, 05:16:08 PM
You would have to compile it yourself.  I suppose today's build should be as good, I don't know if there are any issues.

Don't know anything about building. Guess, will wait till I a complied version is available.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 18, 2013, 05:29:13 PM
Don't know anything about building. Guess, will wait till I a complied version is available.

http://builds.magiclantern.fm/#/
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 18, 2013, 08:40:37 PM
The problem is I need the base ISO of 6400 for the night shots but I dont want DUAL ISO to go very high.

That's why I recently proposed to turn the "dr gained" field into a setting - if you for example set it to at least 1.0 ev, dual_iso would be disabled on high iso settings
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Canon eos m on September 18, 2013, 09:04:04 PM
http://builds.magiclantern.fm/#/

Thanks.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 18, 2013, 10:35:12 PM
I used ETTR + DUAL ISO to do a timelapse from night to day.

The workflow is:

Open all cr2 images and save them as DNG.  This puts the deflicker adjustment (in xml) in the DNG file for further processing.
Process the files with cr2hdr.  This processed the DUAL-ISO dngs and saves them as a normal DNG.
At this point I can import the DNG into Vegas and make the video.  It is perfect at this point.
Now I want to generate tiffs which can be handled by other video editors, not all can read the dngs.
To do this I open the DNGs in ACR and notice that the deflicker exposure is in the exposure field.  If I export the tiffs with this exposure number then it generates flicker, it appears that this exposure has already changed the image itself so the exposure number is not needed for further processing.  So I select all the dngs in ACR and set the exposure to 0.  Problem fixed, now the tiffs can be imported to video editor and there is no flicker.

Finally I get another problems that I cannot resolve:
The Dual ISO dngs that were converted to tiffs get some really strange effects like if there was a crop!  After analyzing the faulty images in ACR I noticed that the DUAL ISO dngs have a CAMERA PROFILE set to embedded and cannot be changed, while the standard dngs have CAMERA PROFILE = Adobe Standard.  I cannot find anything else that is different from the two files so I wonder if this is causing this strange effect.

Something interesting to note is that the video made with the dngs coming out of cr2hdr is ok.  I assume that Vegas cannot read the much of the embedded information and is just importing the image.  When I import these dngs into ACR to make the tiff is when this problems arises.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 18, 2013, 10:48:36 PM
Do they both have the same resolution? If not, try developing the CR2 files with dcraw.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 18, 2013, 10:52:43 PM
Do they both have the same resolution? If not, try developing the CR2 files with dcraw.

They are cropped  The tiff from standard dng is 5796x3870  while the tiff form dual iso dng is 5760x3840. 
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 18, 2013, 10:57:44 PM
Swap the numbers ;)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 19, 2013, 04:12:07 AM
Swap the numbers ;)
Right: standard dng is 5760x3840 while the tiff form dual iso dng is 5796x3870.

Ok I was able to trim the dual iso tiffs using Photoshop action and batch.
The timelapse is good but I wll try it again with different settings.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 19, 2013, 09:16:12 PM
I got a problem with conversion from interlaced dual iso dng to normal dng. I shot fire in the dark with fps override to 65 fps. Brighter shots are converted correct, but darker aren't recognized as interlaced. Even in the same take one (brighter) frame is recognized well...

Code: [Select]
Input file     : E:\81_22-0544_00016
5.DNG
Canon EOS 5D Mark III detected
Full size      : 1920 x 508
Active area    : 1920 x 508
White level    : 12500
Black subtract didn't work
ISO pattern    : BddB RGGB
Noise levels   : 8.00 8.00 8.00 8.00 (14-bit)
Estimating ISO difference...
ISO difference : 4.61 EV (2450)
Black delta    : 67.67
Interpolation  : mean23-chroma5x5-alias
Dynamic range  : 10.35 (+) 10.35 => 14.97 EV (in theory)
Matching brightness...
Looking for hot/cold pixels...
Full-res reconstruction...
ISO overlap    : 3.0 EV (approx)
Half-res blending...
Chroma filtering...
Building alias map...
Filtering alias map...
Smoothing alias map...
Noise level    : 8.00 (16-bit), ideally 8.00
Dynamic range  : 12.35 EV (cooked)
Output file    : E:\81_22-0544_00016
5.DNG
Image/ExifTool/MinoltaRaw.pm did not return a true value at (eval 149) line 3.
Can't find table Image::ExifTool::MinoltaRaw::PRD
Image/ExifTool/MinoltaRaw.pm did not return a true value at (eval 151) line 3.
Can't find table Image::ExifTool::MinoltaRaw::PRD
Image/ExifTool/MinoltaRaw.pm did not return a true value at (eval 154) line 3.
Can't find table Image::ExifTool::MinoltaRaw::PRD
Image/ExifTool/MinoltaRaw.pm did not return a true value at (eval 156) line 3.
Can't find table Image::ExifTool::MinoltaRaw::PRD
    1 image files updated

...and next other not.

Code: [Select]
Input file     : E:\81_22-0544_00016
6.dng
Canon EOS 5D Mark III detected
Full size      : 1920 x 508
Active area    : 1920 x 508
White level    : 12500
Black subtract didn't work
ISO pattern    : BdBd RGGB
Interlacing method not supported
ISO blending didn't work

Input file     : E:\81_22-0544_00016
7.dng
Canon EOS 5D Mark III detected
Full size      : 1920 x 508
Active area    : 1920 x 508
White level    : 12500
Black subtract didn't work
ISO pattern    : dBBd RGGB
Noise levels   : 8.00 8.00 8.00 8.00 (14-bit)
Estimating ISO difference...
Doesn't look like interlaced ISO
ISO blending didn't work



Is there a way to force cr2hdr.exe to convert dngs or other method than cr2hdr.exe?
Please, help. This footage is extremely important to me.

Camera: 5DM3
tried different versions of cr2hdr.exe
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 19, 2013, 09:23:13 PM
Well, if you really want to remove the error checks, feel free to do so, but you'll get just garbage.

Hint: I can't troubleshoot anything from these logs (other than reaching the above conclusion).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 19, 2013, 10:41:44 PM
Well, if you really want to remove the error checks, feel free to do so, but you'll get just garbage.

Hint: I can't troubleshoot anything from these logs (other than reaching the above conclusion).

I attached these logs just to show where it stuck. I have no programming skills, I simply thought that maybe it's a matter of tolerance in ISO difference estimating process. Is it possible to do the conversion in photoshop? Any idea?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 19, 2013, 11:01:15 PM
I attached these logs just to show where it stuck. I have no programming skills, I simply thought that maybe it's a matter of tolerance in ISO difference estimating process. Is it possible to do the conversion in photoshop? Any idea?
maybe you can upload the two cr2 for some one to analyze.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 19, 2013, 11:50:52 PM
maybe you can upload the two cr2 for some one to analyze.

If somebody is able to convert second file, let me know, please.

http://www.mediafire.com/?kr1che73pz9tmdl (http://www.mediafire.com/?kr1che73pz9tmdl)
http://www.mediafire.com/?dxoaa17l87dfgec (http://www.mediafire.com/?dxoaa17l87dfgec)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: NateVolk on September 20, 2013, 08:11:45 AM
Shot today in DualIso, using stock ETTR settings, pushed Set in LiveView to meter.  Ended up with a bunch of files that can't be converted.  Here's the first one.  I'm using the last experimental update from a1ex a few pages back.  Thanks!

https://docs.google.com/file/d/0B_Id0y3JG-KEQldiSFNTSWpJbWs/edit?usp=sharing
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 20, 2013, 08:27:09 AM
That's almost completely overexposed in one ISO. Can you reproduce it?

A video of the camera screen when metering the scene should help. None of my ETTR pictures in full sunlight looked like this...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 20, 2013, 02:44:43 PM
Latest cr2hdr should handle these shots properly.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: SpcCb on September 20, 2013, 04:59:11 PM
Last build (2013-09-20, second run) tested on 5D mkII in both ways (high light and shadow recovery): works well. :)

I just noticed when base ISO is high (like 5000 ISO) hot pixels are not fully registered, compared with a regular single RAW shoot at the same ISO.
But with a good de-rawtizer soft it's easy to set base ISO at 1600 and push to 5000 eq-ISO.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 20, 2013, 05:09:53 PM
I don't get it, but if you shoot above 1600 on 5D2 you are just throwing away useful data.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: NateVolk on September 20, 2013, 07:19:34 PM
That's almost completely overexposed in one ISO. Can you reproduce it?

A video of the camera screen when metering the scene should help. None of my ETTR pictures in full sunlight looked like this...

I'll try again today with a video camera on the screen.  I re-downloaded the cr2hdr from the first post and it looks to be converting the missed files ok.  Thanks,
Title: Re: Dual ISO - massive dynamic range improvement
Post by: yousufarain on September 21, 2013, 06:02:35 AM
Hi,
I apologize in advance.  I am sure this has been asked ad nauseam but couldn't find it.  I am not sure what am I doing wrong here but I get an error when I drag the dng files to cr2hdr (mac user) saying files not supported.  I suspect I am not dialing in the right settings for the Dual ISO mode.  Can you please help. 

5D2 + Mac

- I the installed the latest build 4b3129d5a739
- Turned 'Disable all modules' to OFF
- Selected all the modules and rebooted
- Changed the Quality in Canon menu to JPEG (Does this need to be RAW?)
- Set RAW video ON (1880x1058)
- Set HDR to OFF
- Set Dual ISO to ON (100/1600)


I take the .RAW and convert to dng using RAW2DNG
I drag all the .dng files in a folder to CR2HDR but I get an error

Also the dng files look like regular RAW files and not the Dual ISO files like I expect to see them with interlaced ISO.

Completely new to this.  Thank you
Title: Re: Dual ISO - massive dynamic range improvement
Post by: yousufarain on September 21, 2013, 06:17:03 AM
Forgot to add that I am using the MAC GUI CR2HDR (Do I need something else too)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: akumiszcza on September 21, 2013, 05:25:35 PM
Today I tested some dual iso + aettr. Notifications after settling ettr is great. Two things I've noticed:
1. In Av mode I sometimes get "ettr settled at iso 0/1600". I think it was when I had auto iso turned on in Av mode.
2. I get some photos without "dual" prefix in filename that are really dual iso. These files even have "Dual-ISO" keyword.

I would love to have an option in Aettr and dual iso to turn it off while not in M mode. Now if I have to take a quick shot without aettr and dual iso I have to turn these off in menus. Using aettr in Av even gives additional problems — if I turn auto iso, the option gets removed by auto ettr in work.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: akumiszcza on September 21, 2013, 08:40:37 PM
Below is an example of a photo not possible without Dual-ISO. Not a very good one, but you can see both shadows (desks, closed doors), and very bright areas (in open doors). Canon EOS 50D, which is not good for low-light photos. Normally such shot require multiple exposure. In this case people would move and there would be quite a work in Photoshop. The problem is the fringing around people in the door. In my Photoshop CS5 there's no option to remove chromatic aberration in ACR (it appeared in CS6, process 2012), and I don't get good results in Lens Correction filter in Photoshop. I tried it in trial Photoshop CC with somewhat better results, but not great. I guess this is a "special" case of fringing, appearing only in Dual-ISO? Anyways, making near-HDR photos with one shot is a much greater advantage than some additional fringing :)

(http://i.imgur.com/OI50Pm8.jpg)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 22, 2013, 09:35:41 PM
I'm still trying to rescue my shot, since not all of the frames converted. Cr2hdr failed on low key shots. Alex, would it be possible to create a version of cr2hdr without automated recognizing interlace, which extract just one iso channel, at half res, just to get rid of interlacing on unconvertable images?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 22, 2013, 09:49:53 PM
It requires the automatic recognizing to know which fields are bright and which are dark. This varies from camera to camera and sometimes from frame to frame.

Can you upload another CR2?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 22, 2013, 11:27:58 PM
It requires the automatic recognizing to know which fields are bright and which are dark. This varies from camera to camera and sometimes from frame to frame.

Can you upload another CR2?
Thanks for explanation. In attached images fields are changing from frame to frame. Maybe a command of indication if the top field is bright or dark would rescue them?

http://www.mediafire.com/?20d3fr3hjrspb6n
http://www.mediafire.com/?j9lacm5cj4moohd
http://www.mediafire.com/?i4pmhpejwt61l5r
http://www.mediafire.com/?q8bkbh0vcq7n9he
http://www.mediafire.com/?exqi8cqvajlgkat
Title: Re: Dual ISO - massive dynamic range improvement
Post by: 1% on September 23, 2013, 12:01:19 AM
Hmm.. I looked at one of those and there is so little data its probably confusing the algo. I guess it would have to check from the right where the barrel? is.

Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 23, 2013, 08:03:26 AM
I think the confusion comes from the black level difference: in dark areas, the low ISO pixels are brighter than the high ISO ones. The difference is usually small (say under 50 units), but it accumulates.

This should be the key to solving the autodetection.

A command-line option may help if you are willing to process each shot manually and fiddle with the offset until you get it right (I'm not). There are 4 values that you have to try.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 23, 2013, 08:18:49 AM
A command-line option may help if you are willing to process each shot manually and fiddle with the offset until you get it right (I'm not). There are 4 values that you have to try.

It would be better than nothing - given the time cr2hdr takes to convert an image, batch-conversion with all 4 possible values wouldn't make much of a difference, at least to me if one of them has as little artifacts as possible.

If cr2hdr has any means to guess which of the 4 options is the correct one it would be nice to be able to append a "confidence" level or something like this to the file name so when sorting manually through them users would know which one is most likely to be the best.

Personally, I'm unlikely to use dual_iso all the time, so with the remaining images where I really needed it I'm prepared to spend a little time converting it.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 23, 2013, 08:22:06 AM
Only one of these 4 values will result in a valid image.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 23, 2013, 09:24:00 AM
Only one of these 4 values will result in a valid image.

Ok, I didn't understand that - but imho one more reason to make them available from the command line if the (rare?) case should occur that the dual_iso autodetection chooses the wrong one? It's not like it'd be a big coding effort :-p
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 23, 2013, 09:29:36 AM
Quote
It's not like it'd be a big coding effort :-p

In this case, feel free to submit a patch ;)
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 23, 2013, 09:41:51 AM
In this case, feel free to submit a patch ;)

Nonono, you got that wrong - it's not much of an effort for *you*, for *me* "Hello World" takes quite some time :-p

But to be earnest: I find submitting a full fledged pull request as it's desired around here to be too much overhead for trivial patches (long explanation cut, someone has to read this...) :-\ ... just posting a diff in a code box seems easier to me.

Also like in this case looking at some source code for the first time - and I'll never need to look at cr2hdr again - takes some time to get the bearings, just like working on the ml source takes some time if some months have passed. What I do in 1h you'll do in 10sec - I don't even remember how linux command lines are parsed and where the 4 options you mentioned might be located.

That's why I usually put my trivial or "open for discussion" ideas as a comment or request first, and only if push comes to shove change the source code myself. I know from dev's perspective this might look lazy, but just now I think for small matters this is a more economical way.

Speaking with Star Trek: I'm a doctor (no, actually I'm a sociologist) not a hardcore coder :-p
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 23, 2013, 09:56:32 AM
A command-line option may help if you are willing to process each shot manually and fiddle with the offset until you get it right (I'm not). There are 4 values that you have to try.
I'm begging for a patch. Everything would be better than cutting and pasting 2 pixel lines in photoshop on 50 frames :o. Please!
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 23, 2013, 10:14:31 AM
I'm begging for a patch. Everything would be better than cutting and pasting 2 pixel lines in photoshop on 50 frames

But it would teach you how to use photoshop actions :->
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 23, 2013, 11:29:36 AM
But it would teach you how to use photoshop actions :->
:D I know actions. Resulted images from photoshop are awful. Aliasing is much worse than from cr2hdr converted at half res. HDR merge doesn't work as it should too.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: robinlee on September 23, 2013, 07:44:02 PM
Hi a1ex, your new cr2hdr works a treat now with actual exiftool are working! (saves me doing them 1 by 1 :D)

I have to say many thanks to you and the members who helped in developing the cr2hdr  ;)

Cheers
Robin
Title: Re: Dual ISO - massive dynamic range improvement
Post by: pointbob1 on September 23, 2013, 07:44:21 PM
basic question here; aren't you anchored to use it at iso 100 - what what would you do trying to take pics of friends in a dimly lit pub? iso 100 forces a real slow shutter speed, yes? ie. iso 100/1600...is my thinking wrong?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 23, 2013, 09:44:05 PM
basic question here; aren't you anchored to use it at iso 100 - what what would you do trying to take pics of friends in a dimly lit pub? iso 100 forces a real slow shutter speed, yes? ie. iso 100/1600...is my thinking wrong?

You can manually set the iso on your camera and then the second iso in the module, either as a relative ev or absolute iso value. Just note that combining high iso values will gain only a small bit of dynamic range, see the dr curve for your camera here: http://www.sensorgen.info/
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 24, 2013, 01:38:28 PM
...so no developers interested in modification of cr2hdr to have manual control instead of automatic detection of fields order? Maybe easier than implementing command line would be 4 versions of cr2hdr, since there is 4 combination of fields order?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 24, 2013, 01:40:48 PM
...so no developers interested in modification of cr2hdr to have manual control instead of automatic detection of fields order?

Give them some time, will you? Unless someone says it *won't* be implemented, given the amount of features being worked on in parallel and this not being a major bug it's not on top of the list... look again after a week or so?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 24, 2013, 02:09:14 PM
Give them some time, will you? Unless someone says it *won't* be implemented, given the amount of features being worked on in parallel and this not being a major bug it's not on top of the list... look again after a week or so?
I know, I keep asking cause tomorrow I got an exam in filmschool and one shot in my exam movie is missed...
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 24, 2013, 02:27:29 PM
Try this: int is_bright[4] = {1,1,0,0}, then {0,1,1,0}, then {0,0,1,1} and {1,0,0,1}. These are the 4 combinations.

Right now I'm a bit busy with some other stuff, sorry.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 24, 2013, 03:04:28 PM
Thanks Alex. How exactly I can use this code? Cr2hdr closes immediately after double-click. Where should I write the code and filename? Sorry for asking banal questions.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: RenatoPhoto on September 24, 2013, 03:42:36 PM
Thanks Alex. How exactly I can use this code? Cr2hdr closes immediately after double-click. Where should I write the code and filename? Sorry for asking banal questions.
You have to have a deep understanding of how the cr2hdr is created.  In the dual-iso module there is the C code named cr2hdr.c which is where all of the details of this program lays.  If you have the knowledge then go into this code and make the necessary changes.  But of course you first have to set compiler. 
Of course if you are not a hardcore ML programmer this will not be possible.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 24, 2013, 04:16:53 PM
Try this: int is_bright[4] = {1,1,0,0}, then {0,1,1,0}, then {0,0,1,1} and {1,0,0,1}. These are the 4 combinations.

I've compiled 4 different version of cr2hdr with the combinations quoted.
https://www.dropbox.com/sh/qyh4agzisg3g0c5/16RZU1Z2Bg
Title: Re: Dual ISO - massive dynamic range improvement
Post by: jOt on September 24, 2013, 05:05:51 PM
I've compiled 4 different version of cr2hdr with the combinations quoted.
https://www.dropbox.com/sh/qyh4agzisg3g0c5/16RZU1Z2Bg
Many thanks, Audionut. Unfortunately when I drop dng on cr2hdr I got a message that version of this file is not compatible with my windows version. I got win7 64x. What version I shoud have?
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 24, 2013, 06:21:45 PM
Sorry, I had to setup a new build environment and never bothered to check the compiles before uploading.
I'll try and get it fixed tomorrow.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: whumber on September 25, 2013, 05:23:22 PM
Thanks Alex. How exactly I can use this code? Cr2hdr closes immediately after double-click. Where should I write the code and filename? Sorry for asking banal questions.

Just drag your dual ISO CR2 file(s) onto the CR2HDR executable file.  Alternatively you could write a batch file to point CR2HDR to the CR2 files but dragging is the easier method.  Double clicking doesn't work because CR2HDR is not an interactive program but a console program that relies on input arguments.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: orim on September 25, 2013, 11:13:01 PM
Hi, a1ex

thank you for DualISO on my 50D, it is awesome.
It however started to work for me as expected only since the last build of cr2hdr.exe
(till then i had only green whites + pink shadows as dng, do not know why...)

But, is there any chance to create cr2hdr with multicore CPUs support, please?
This single core version is really time consuming, which is time limiting for testing dualISO pictures...
It is a waste of resources we have. :-)

Anyway, thanks again.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: a1ex on September 25, 2013, 11:23:15 PM
You can already use a batch script that does this. There's one posted somewhere in this thread.

Also found something here: http://stackoverflow.com/questions/672719/parallel-execution-of-shell-processes
Title: Re: Dual ISO - massive dynamic range improvement
Post by: swinxx on September 25, 2013, 11:49:16 PM
hi,

today i shot a dual iso raw video. after converting the files with rawmagic to dng files they are unprocessed with both isos - perhaps rawmagic can implement those file types in the future.. ?

i have now noticed that when i use the mac gui 1.2 cr2hdr i get an error message: the file type is not supported..

here is the link to the dng file and a screenshot of the error message:
https://www.dropbox.com/sh/wzh6q2994ve7i6d/kKL5Whmuln

another question:
is it possible to implement an option to mark dual iso raw videos with a prefix DUAL like dualiso photos...
that would be really handy because i have shot so many small video files, some with dualiso, others not at the same day..

thank you. sw

Title: Re: Dual ISO - massive dynamic range improvement
Post by: orim on September 26, 2013, 09:44:13 AM
You can already use a batch script that does this. There's one posted somewhere in this thread.

Also found something here: http://stackoverflow.com/questions/672719/parallel-execution-of-shell-processes

Thanks, a1ex
but I have no idea how this could be helpful for me :-)
As far as I understand that script, it is meant for different king of task.

Drag&drop dualiso cr2s on cr2hdr.exe is nice and simple, I would not change it.
Plus, parallel processing of multiple cr2 files would perhaps overload HDD...
so multicore processing of one cr2 at a time still sounds better. At least for me. :-)

Have a nice day.


Title: Re: Dual ISO - massive dynamic range improvement
Post by: Audionut on September 26, 2013, 09:56:58 AM
As far as I see, the code has steps that rely on previous steps.  This can't be multi-threaded.

The HDD load is small.  It's only a 40mb (or so) read/save at any one time.  And the processing time far outweighs the file loading/saving.
Title: Re: Dual ISO - massive dynamic range improvement
Post by: Marsu42 on September 26, 2013, 10:15:05 AM
It's only a 40mb (or so) read/save at any one time.

By the way - you guys most likely know that the cr2hdr output isn't compressed, or not optimally so? Re-compressing the resulting dng in Lightroom (ACR) cuts the size about in half, of course as lossless conversion.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: RenatoPhoto on September 26, 2013, 06:43:36 PM
The successful vbs script by engardeknave has been re-namend moved to Scripts Repository:
Script for Automatic Multiple instances of cr2hdr
http://www.magiclantern.fm/forum/index.php?topic=8520.0
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on September 26, 2013, 10:45:33 PM
I'm getting some Dual-ISO files, that do not have "DUAL" prefix and some files with this prefix that are not Dual-ISO. I think this happens especially when I shoot series of photos. AETTR with linked Dual-ISO, Canon 50D, latest nightly (it happened in previous builds, too).
Title: Re: Dual ISO - massive dynamic range improvement
Post by: mk11174 on September 27, 2013, 12:10:02 AM
Thanks Alex. How exactly I can use this code? Cr2hdr closes immediately after double-click. Where should I write the code and filename? Sorry for asking banal questions.
I think I made the correct changes from what Alex said, here are the 4 versions plus the normal one. Like always make sure the file is in the same folder as the EXE file when you drag and drop to convert.
https://bitbucket.org/mk11174/magic-lantern/downloads/cr2hdr_combo.zip

Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 27, 2013, 07:52:13 AM
All the raw files recorded with the last build that I put in this software say:

/Volumes/RAID/GRILL/100EOS5D/M26-1348.RAW File not Supported
THE END

I tried with RAW2DNG and I get Pro Res video with lot of horizontal lines.

Please, help!!!
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on September 27, 2013, 09:12:40 AM
You need to convert the raw file to DNGs.  Then process the DNGs with cr2hdr.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 27, 2013, 09:47:56 AM
This is the message for the files:

1349_C0000/M26-1349_C0000_00587.dng File not Supported
/Volumes/RAID/LE GRILL CHILLA EVENTOS/converted/M26-1349_C0000/M26-1349_C0000_00588.dng File not Supported
/Volumes/RAID/LE GRILL CHILLA EVENTOS/converted/M26-1349_C0000/M26-1349_C0000_00589.dng File not Supported
/Volumes/RAID/LE GRILL CHILLA EVENTOS/converted/M26-1349_C0000/M26-1349_C0000_00590.dng File not Supported
THE END
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on September 27, 2013, 09:55:25 AM
You're using cr2hdr to process the DNGs right?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 27, 2013, 09:58:50 AM
1- RAW file to Raw2DNG
2- DNGs to ACR
3- DNGs files to CR2HDR (Mac version)

result: .dng File not Supported
THE END

Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on September 27, 2013, 10:01:01 AM
Ditch step 2!

Upload a DNG (straight from raw2dng) for someone to look at.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 27, 2013, 10:02:21 AM
Same Result

.DNG File not Supported
THE END
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 27, 2013, 10:04:22 AM
https://www.dropbox.com/s/zp1qb5rwrjri4ts/M26-1349_C0000_00160.dng
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on September 27, 2013, 10:07:16 AM
Works for me (with the Linux/Windows versions).

If you are using the MAC GUI, try renaming it to .CR2 (and tell a.d. to allow DNGs too).
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 27, 2013, 10:12:40 AM

Input file : M26-1349_C0000_00002.CR2
sh: dcraw: command not found sh: exiftool: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile    : M26-1349_C0000_00002.txt

THE END
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 27, 2013, 10:13:54 AM
TXT  file say:



Input file     : M26-1349_C0000_00002.CR2
Canon EOS 5D Mark III detected
Full size      : 0 x 0
Active area    : 0 x 0
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a.d. on September 27, 2013, 01:50:28 PM
Input file : M26-1349_C0000_00002.CR2
sh: dcraw: command not found sh: exiftool: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile    : M26-1349_C0000_00002.txt

THE END
You also need to download OSX_cr2hdr.zip. It's an installer for dcraw & exiftool.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on September 27, 2013, 05:21:54 PM
@a.d
Love your mac app. Any possibility to to be able to convert dng;s created from extracted dual iso raw like with Alex windows/linux build?
Thanks!
//D
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 27, 2013, 06:09:37 PM
Where I can download that software?

It's not explaining how this work in the topic? Can you explain the entire DUAL ISO process for Mac users please?
Thanks.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: RenatoPhoto on September 27, 2013, 06:14:58 PM
Opening Post. ie post number 1 on this thread.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 27, 2013, 08:11:13 PM
No RENATO

If you use the instruction in the beginning of the post only get this:

cr2hdr // Beta 0.4
/M26-1349_C0000_00160.dng File not Supported
THE END


I'm trying to know in differents mac why this is not working.

A1ex Recommend to rename DNG files as CR2 and add to the software the possibility to recognize DNG files directly.
This is not explained.

A.D. recommends use the file above and works only with CR2.

So, first:

- we have to open RAW file with RAW2DNG
- rename all the files as .CR2
- And after that put on CR2HDR 0.4 - not early

Except a easy way that I can't find, this is the process.

Can you update this?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: RenatoPhoto on September 27, 2013, 08:54:36 PM
No RENATO

I thought you were looking for Mac.  In the first post there is this link to Mac:

- Mac GUI for cr2hdr by a.d.

If you click on this on the first post, it takes you to the place where you can download the Mac version here:

https://bitbucket.org/a_d_/magic-lantern/downloads

These files are generated by a.d. who has provided some support for Mac users (I am a PC user)

The first one on the list is OSX_cr2hdr_app.zip

Try that one. 

I hope someone else can help you with the MAC stuff.

Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 27, 2013, 09:01:53 PM
Thanks RENATO

We are trying to create sth in AUTOMATOR - Mac - To rename DNG files easly.

A.D. explain me why the software only recognize CR2 files.

For future mac users of DUAL ISO Post Processing it´s important they know to rename the files and use all this app for the moment:

https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip

thks
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: RenatoPhoto on September 27, 2013, 09:20:13 PM
https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip

And from the post of a.d. you also have to dowload:

You also need to download OSX_cr2hdr.zip. It's an installer for dcraw & exiftool.

So you have to download both pieces of software:

https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
and
https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr.zip

For  Bulk rename i use www.bulkrenameutility.co.uk
but dont know if there is a Mac version.

Here is a link to google find of bulk rename for mac:
https://www.google.com.ec/?gws_rd=cr&ei=QLlFUt2BCIaE9QSqxYDYCw#psj=1&q=bulk+rename+for+mac
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on September 27, 2013, 10:31:06 PM
Dang, it works on my mac. I renamed to cr2 when it has to be capitals CR2. WHat was I thinking :) A.D your app rocks!

I rename my files with this tool http://namechanger.en.softonic.com/mac

Thanks everyone
//D
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 28, 2013, 04:58:32 AM
Good Danne, works too!
CR2 it´s case sensitive, only works in UpperCase!!!
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: N/A on September 28, 2013, 05:11:21 AM
Good Danne, works too!
CR2 it´s case sensitive, only works in UpperCase!!!
Also, once you've renamed them to .CR2, the newest CR2HDR allows you to drag and drop the entire folder to the icon. Pretty damn nifty and efficient. Now if CR2HDR saved the converted .dng's into a sub-folder, it would make for an even better workflow.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 28, 2013, 07:16:03 AM
Soon appears that we can drop DNG into the soft directly ;)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: jOt on September 28, 2013, 12:08:38 PM
I think I made the correct changes from what Alex said, here are the 4 versions plus the normal one. Like always make sure the file is in the same folder as the EXE file when you drag and drop to convert.
https://bitbucket.org/mk11174/magic-lantern/downloads/cr2hdr_combo.zip
mk11174, thanks a lot. I could convert successfully half of missing frames, but still on the rest there is some interlacing, I don't know why.
Before conversion:
http://www.mediafire.com/view/on7joso4ngljnsc/000189.dng
After:
http://www.mediafire.com/view/gqam4hatatuq3jd/000189_0110.DNG
(It's a still from the shot of throwing molotov cocktail)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Legushka on September 28, 2013, 02:16:51 PM
canon 60 not supported
this train?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on September 29, 2013, 08:59:21 AM
A.D. has working in CR2HDR and now support DNG!

https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on September 29, 2013, 11:09:27 AM
Thanks A.D!
Alex,s improvement skills on the converter is amazing. Time to try out some new dual iso filming
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Aquillum on September 29, 2013, 04:11:24 PM
Hi !

First try with dual iso today.

Canon 550D, today's ML build, dual_iso-Module enabled.
Internal ISO:100
Dual ISO 100/800 or 100/1600 enabled

Dropping the CR2 files on the EXE (Windows, 20.09.13) it says:
Code: [Select]
D:\Software\Canon\cr22hd>cr2hdr.exe IMG_3535.CR2

Input file     : IMG_3535.CR2
Canon EOS 550D detected
Full size      : 5344 x 3516
Active area    : 5202 x 3465
Doesn't look like interlaced ISO

I don't see any interlaced lines at 100% for reference.
Any ideas why no dual ISO files are created ?

Thanks ! Harry
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: RenatoPhoto on September 29, 2013, 04:55:24 PM
Canon 550D, today's ML build, dual_iso-Module enabled.
Internal ISO:100
Dual ISO 100/800 or 100/1600 enabled

Input file     : IMG_3535.CR2
Canon EOS 550D detected
Full size      : 5344 x 3516
Active area    : 5202 x 3465
Doesn't look like interlaced ISO

Maybe the nightly build does not yet support the DUAL ISO feature for this camera.

I think it has been enabled by another developer, check this thread:

550D raw video recording port official thread
http://www.magiclantern.fm/forum/index.php?topic=5582.0
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: RenatoPhoto on September 29, 2013, 04:57:25 PM
canon 60 not supported

Look in this thread:
60D RAW video - it's working !!!
http://www.magiclantern.fm/forum/index.php?topic=5653.0
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on September 29, 2013, 07:09:02 PM
Maybe the nightly build does not yet support the DUAL ISO feature for this camera.

In this case, he wouldn't be able to see dual ISO in the menu.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: RenatoPhoto on September 29, 2013, 07:24:02 PM
In this case, he wouldn't be able to see dual ISO in the menu.

@ Aquillum
In that case then you need to learn how to use it:

(Auto) ETTR (Exposure to the Right): -- History & Beginners Guide
http://www.magiclantern.fm/forum/index.php?topic=5693.0

AETTR + DUAL ISO: The Ultimate Automated Perfect Image Exposure-Beginners'Guide
http://www.magiclantern.fm/forum/index.php?topic=8322.0
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Aquillum on September 29, 2013, 07:41:28 PM
I used ETTR since May, even with a timelapse, thats not the problem.

The problem is with the latestbuild:
I am able to enable the modules, they are loaded after restart and I see ETTR and Dual Iso in Menu.

I enable Dual-Iso, set to 100/800, take the raw but the CR2 is NOT Dual Iso.

I also tried the ML_550D_Dual_ISO_Aug_25_2013_Update.zip build posted in that forum after cleaning ML folder completely (not to mix modules) but it also doesn't work.
Does anyone with a 550D shot a DualIso successfully ? I can't.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: RenatoPhoto on September 29, 2013, 07:50:08 PM
Did you know that you have to reboot the camera for the modules to work?  It is a new thing!!
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: N/A on September 29, 2013, 07:51:25 PM
A.D. has working in CR2HDR and now support DNG!

https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
What about the actual dual iso raw files? That would be epic.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Aquillum on September 29, 2013, 07:58:52 PM
As I wrote before I restarted, the modules loaded OK.
I use the standard settings in Dual Iso Menu (100/800).
I take the pic in Av or M, with or without LV, but the resulting CR2 doesnt have scanlines and cr22hdr also doesn't recognize it as an DualIso file.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on September 29, 2013, 08:00:49 PM
Just tried it on 550D, works fine.

Any warnings in menu?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Aquillum on September 29, 2013, 08:24:25 PM
No warnings.
As it works for you its definitly on my side - thanks for trying !
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on September 29, 2013, 08:52:26 PM
Can you post a video of the camera screen, while you are trying to use it?

It should show the dual ISO menu, Canon settings (main photo screen, with ISO, shutter and so on), then take a picture, go to playback mode and zoom to 100%. Maybe this gives some hints about what's going on.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Aquillum on September 29, 2013, 11:01:33 PM
Now it works.
Things done (from my memory): (DualIso was enabled)
- switched Canon Iso from 100 to 200
- switched to DUAL Prefix
- enabled ETTR - Switched from AutoSnap to SET
- switched LV - shot ETTR and realized the Image Scanlines in PLAY
- switched ETTR OFF
- DUAL ISO shot still works
- CR2 decoded fine

THX for your support (even if its just "Here it works", that was enough  for me)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: swinxx on September 29, 2013, 11:49:37 PM
@a.d.
nice now the conversion of dual iso dng files work.

i have one question a.d.:

when i see my magicraw converted dng files (from a raw file) in the folder, i can preview that file with the space bar.. (mac preview)
after the conversion of the dual iso dng with your tool (i can see that it renames to CR2 and then again to dng) there is no preview in the finder anymore..?
is this because of the conversion? why do i lose the preview?


thx and keep on developing great tools. sw
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a.d. on September 30, 2013, 10:06:33 PM
I need to rename, 'cause after cr2hdr save_dng run it called copy_tags_from_source, to copy EXIF from original to converted file. That means I need to save the original file to get the EXIF Data. Alternatively I could inject the Camera Model through chdk-dng, however it hardcoded. (only one Camera Model)
You are right, the preview is lost through conversion.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: lucabutera on September 30, 2013, 10:54:25 PM
I got dng files from raw video but I can not use cr2hdr!
please, help me!
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 01, 2013, 03:44:37 AM
I got dng files from raw video but I can not use cr2hdr!
please, help me!

Help us to help you by extending your bug report to include useful information.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 01, 2013, 12:15:47 PM
Bug: cr2hdr on 6d adds 2 black borders to shots, here's a random dng with this problem (no, this shot won't win any prizes :-p): https://bitbucket.org/Marsu42/ml-mod6d/downloads/DUAL2689.CR2

Btw see also these feature requests:
http://www.magiclantern.fm/forum/index.php?topic=8579.0
http://www.magiclantern.fm/forum/index.php?topic=8580.0

Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: artiswar on October 02, 2013, 05:49:34 AM
Any idea why my DNGs out of cr2hdr look like this for some shots?
(http://i44.tinypic.com/2cs7ybs.png)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 02, 2013, 06:56:34 AM
Quote
Bug: cr2hdr on 6d adds 2 black borders to shots, here's a random dng with this problem (no, this shot won't win any prizes :-p): https://bitbucket.org/Marsu42/ml-mod6d/downloads/DUAL2689.CR2

Upgrade your dcraw. No bars here.

But I do get some conversion artifacts on this shot:

(http://i.imgur.com/2ZpeLcc.jpg)

Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 02, 2013, 08:02:45 AM
Have a link to an updated dcraw that works on Windows?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 02, 2013, 10:08:53 AM
Have a link to an updated dcraw that works on Windows?

Here ye go, fresh compile just now: https://bitbucket.org/Marsu42/ml-mod6d/downloads/dcraw_9_19-m42.zip

Don't be too shy to install the mingw packages on your linux though, then just compile with "i686-pc-cygwin-gcc.exe -o dcraw -O4 dcraw.c -lm -DNODEPS" and get an .exe file.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 02, 2013, 10:24:32 AM
Thanks, will update the zip in the first page when I'll be able to access the ftp (it seems broken now).
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 02, 2013, 12:20:55 PM
Upgrade your dcraw. No bars here.

Yes, that did it - thanks! I forgot that cr2hdr is not the only component to keep up to date.

Btw, this again reminds me of how important it is too keep the old cr2 around (I recently suggested an option to embed them in the dng for safekeeping & easy updating with a newer cr2hdr). Is cr2hdr considered "production-ready" now, or are there outstanding important bugfixes or enhancements?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: artiswar on October 02, 2013, 03:45:13 PM
Any idea on the previous shot I posted? The log states something about bad black delta. Only on a few takes from a short this weekend that utilized a higher ISO spread.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 02, 2013, 04:02:22 PM
Is cr2hdr considered "production-ready" now, or are there outstanding important bugfixes or enhancements?

I've processed roughly 2000 pics with it, and for testing I have a directory with 50 difficult shots (most of which were taken from this thread).

Known issue: the night fire shots from jOt. I've outlined a robust formula for autodetection, which relies on comparing 4 histograms, only need some time to implement it and run the tests.

So, now it's up to you to try it and upload the CR2 files that fail to convert or have major issues (hint for @artiswar).
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: artiswar on October 02, 2013, 04:31:33 PM
As soon as I get home, I'll upload a CR2.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: artiswar on October 03, 2013, 12:07:49 AM
Here's a problematic CR2.

http://www.megafileupload.com/en/file/456879/M29-0538-C0000-00773-CR2.html
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 03, 2013, 12:23:05 AM
Quote
I recently suggested an option to embed them in the dng for safekeeping & easy updating with a newer cr2hdr

I keep them around in case there is a better CR2hdr released and I might want to reprocess the shots. The DNGs are already 2x the size of the cr2 files.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 03, 2013, 01:10:32 AM
I keep them around in case there is a better CR2hdr released and I might want to reprocess the shots.

Embedding the cr2 in the dng would allow a seamless in-place update with a newer cr2hdr, keeping the tags and file location: http://www.magiclantern.fm/forum/index.php?topic=8336.0

The DNGs are already 2x the size of the cr2 files.

No, they aren't, it's just that ml's dcraw process doesn't output a proper dng - if you (losslessly of course) recompress the dual_iso dng with ACR like in Lightroom, the size is cut by half.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: artiswar on October 03, 2013, 03:01:48 AM
@1%

Dying for a fix here. Very integral shots that need to be processed. Or if there's a way to deinterlace without processing ISOs that would be fine too. I just need the shots. Thanks so much.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 03, 2013, 03:42:38 AM
Well I was stumbling around looking for a newer build of cr2hdr.exe I tried all the 9/27 I found... your "cr2" looks fucked up... lines in the shadows.

http://www.filedropper.com/oldx-cr2hdr

My old self compiled from 9/5 did this:
(http://i.imgur.com/sQSM5ap.jpg)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: artiswar on October 03, 2013, 03:55:15 AM
@1%

You're a potential lifesaver. Anyway to run this in the Mac environment? Or fix my CR2?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 03, 2013, 03:57:10 AM
VM or wine or (parallels?) it was made on win7 x64 but should be ok on any windows environment..
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: artiswar on October 03, 2013, 05:08:18 AM
VM or wine or (parallels?) it was made on win7 x64 but should be ok on any windows environment..

Any reason why I'd get a console window in Windows when trying to convert? Sorry for all the questions.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 03, 2013, 06:11:09 AM
Its a command line only app
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: artiswar on October 03, 2013, 06:15:21 AM
It doesn't stay open and does no conversion.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 03, 2013, 06:22:11 AM
Open a command prompt manually and process the CR2.  This way you will see the error message before the prompt auto closes.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a.d. on October 03, 2013, 02:31:12 PM
Well I was stumbling around looking for a newer build of cr2hdr.exe I tried all the 9/27 I found... your "cr2" looks fucked up... lines in the shadows.
...
I notice the same thing ... It broke up between 13 and 20 September
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 03, 2013, 03:31:34 PM
Solved.

If you notice these things, post a CR2 that shows  the problem, I can't read your minds ;)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: delorossa on October 03, 2013, 05:13:52 PM
Where can i change in MLn dual Iso??? I cant see nothing?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 03, 2013, 08:38:22 PM
I've processed roughly 2000 pics with it, and for testing I have a directory with 50 difficult shots (most of which were taken from this thread).

Are there any specifics we should watch out for in our dual_iso shots? My shots look just fine, but then again I don't really know what type of problem(s) to look for @100% crop.

Question: The help says 100/1600 is too high, but it looks great on the 6D. is the +4ev spacing warning there because the higher iso has more noise (i.e. worse on crop than ff), or because a large spacing might confuse cr2hdr for reasons other than iso noise?

Edit: Also you might want to add some version tag to cr2hdr, or with so many binaries floating around it might get confusing sooner or later with what version a cr2 was converted?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 03, 2013, 08:59:54 PM
ISO 100/800 will most likely look the same in shadows, with less aliasing in highlights. Plus, in video mode, I don't have black correction info, and 100/1600 or higher will do a lot of nasty things to black level.

The latest problematic CR2 had exactly this issue, so... had artiswar followed the indications from the menu, he wouldn't have had this issue.

Look at raw zebras; if you get weak zebras over important detail, it's likely to have aliasing there.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 03, 2013, 09:06:29 PM
lus, in video mode, I don't have black correction info, and 100/1600 or higher will do a lot of nasty things to black level.

To clarify again: this is only a problem for video, photo mode is more likely to work ok with 100/1600, correct?

lus, in video mode, I Look at raw zebras; if you get weak zebras over important detail, it's likely to have aliasing there.

Thanks!
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 03, 2013, 09:14:33 PM
Yes, the warning in photo mode should appear at 3200 on most cameras. On 5D2/500D/50D it appears at 1600, because this is the max analog ISO there.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 03, 2013, 10:57:50 PM
I've downloaded cr2hdr.zip from OP today (I don't know when it was updated) and found out dcraw.exe included there requires cygwin1.dll installed, which might be an issue sometimes. Previously it shipped with a standalone version (200k bigger). Reverting back to previously included dcraw.exe works fine I think (I hope no new dcraw.exe features are required?)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 03, 2013, 11:56:58 PM
Depends on the camera.. .Ie 6D/5DIII/M old dcraw might cause issues.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Rai2121 on October 04, 2013, 05:37:38 AM
I just want to give a huge thanks to everyone working on Dual ISO. I decided to test it out in the video function and here is a quick sampling of it. There are definite circumstances where it really shines and looks quite good in video form. I think it looks pretty good in the first 3 shots. The last shot was really just to see how much dynamic range I could get. All were shot 100/3200, so that maybe the problem with the chroma artifacts in the last shot. I've learned that if a shadow is right next to a highlight, there can be issues. I am really excited to continue to see which scenarios it looks the best in.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 04, 2013, 07:14:55 AM
Next time, try following the advice from the menu (100/800). The noise improvement from 800 to 3200 is less than 1 stop, but the difference in aliased areas is 2 stops. The improvement from 1600 to 3200 is close to 0 (but for that you pay 1 full stop of aliased data).
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: swinxx on October 04, 2013, 07:54:33 AM
@alex
do you also prefer 100/800 when shooting a low light scenario?

i have read somewhere that 1. we should ettr the scene and then 2. apply the second iso for raising the shadows..
so this could also be 800/1600.

am i right? 
thx sw
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 04, 2013, 08:44:56 AM
so this could also be 800/1600.  am i right?

No, you're not, because the dynamic range gained from 100/800 (+3ev: 100, 200, 400, 800) is much larger than 800/1600 (+1ev) - ettr even shows this in the menu.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 04, 2013, 10:09:22 AM
800/1600 is fine, but it's only 0.5 stops of improvement. 800/3200 is 0.75 on 5D3. At this point you may as well just denoising the shadows a tiny bit (without getting any aliasing).

Here you can see the difference between ISO 1600/3200/6400 in shadows: http://www.guillermoluijk.com/article/isos5dmkii/index.htm

(hint: I can't see any)

But if that low light scenario also has some bright areas (think at street lighting), I'd go for 100/800, or 100/1600 if it's really dark, or try what ETTR suggests.

Here's an example:
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_7111-small.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_7111.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_7111-1600-small.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_7111-1600.jpg)

First is ISO 100/1600; second is how it would have looked at ISO 1600 (it's not a real 1600 shot, but reconstructed from the dual exposure). It doesn't look bad, and the clipped areas are pretty small, but I prefer the first one.

And here's a night shot that did not require dual ISO according to ETTR, even though I was tempted to use it. Why? Because at 1/10 ISO 1600 there was no important detail overexposed (only some tiny specular highlights), and I don't think I could have used a slower shutter speed handheld. But if I would have allowed ETTR use a slower shutter, or asked it not to ignore any highlights, I think it would have chosen dual ISO.

(http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_5808-small.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_5808.jpg)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: ouuzi on October 04, 2013, 10:28:48 AM
Alex Where are you from?Where are these pics from?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: swinxx on October 04, 2013, 11:32:11 AM
thx for your posting alex. i also cant see any difference on the iso shots 1600 3200 6400 from your link.  everything makes sense but i have to get used to it.. when you prefere 100/800 for a night scene with some highlight areas it means that you make no ettr without dual iso first, and then apply the second (lower iso) to get rid of the noise.. right?

and btw.. wouldn´t it be great to implement a "lock the 180degree shutter rule" for the raw video as an option for ettr, so that it only adapts the right iso for the scene. ?
when shooting video i always want my shutter to stay at e.g. 1/48 for 24fps..

thx sw
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 04, 2013, 11:51:58 AM
Simply choose that shutter speed in ETTR menu.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Onoxyne on October 04, 2013, 04:39:50 PM
I try the dual iso with a 60D but my computer work with Ubuntu 12.04 and the compilation of cr2hdr dont work (problem with Makefile).
Someone have the .bin of this software ? And other to use this great module with Linux ?
I have download cr2hdr-static.linux.x86.2013-10-03-47b5360e8f49.7z found in the web but after unzip, I dont know what I can do with the file : cr2hdr-static.linux.x86.2013-10-03-47b5360e8f49.
Thank's for your lights...
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on October 04, 2013, 09:29:41 PM
Was out doing real estate photography and I decided to film two short sequences to see how dual iso holds up with today,s conversion tool. Also filmed a raw clip for comparison. Even though aliasing still exists and some jagged edges, It,s still eyeblowing to be able to get an even higher dynamic range than raw. I,ll try to find some time to get a moving picture.

Frame from raw 1920x1080p
(http://s11.postimg.org/v87y4aqw3/000009.jpg)

Frame from dual iso raw 100/1600 (ISO difference : 3.98 EV (1577)
(http://s11.postimg.org/kjzlrprj7/000000.jpg)

Still image (just for fun), three pictures, enfused to one.
(http://s24.postimg.org/q7ob3e90l/IMG_8484_1.jpg)

Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 04, 2013, 10:02:44 PM
Frame from dual iso raw 100/1600(or 800) Can,t remember

Another +1 for adding a xmp with the iso settings used :-o ... for bugtracking or recreating a scene it's important to remember how it was done, and this information simply isn't available unless you have a pencil and a piece of paper ready all the time.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 04, 2013, 10:16:25 PM
Sure, just find a command for exiftool that would add the ISO gap in a way recognized by Adobe software.

The second ISO is a little hard to guess with current metadata (some cases are ambiguous). Example: main ISO 400 and a gap of 2 stops - is it 400/100 or 400/1600?

The noise levels may be a good hint for solving the ambiguity (but right now we only have them in photo mode).
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on October 04, 2013, 10:16:56 PM
Actually @Marsu, my test was to see how well the latest cr2hdr holds up. It does a pretty decent job don,t you think? Also want to get some more dual iso filming going since I can,t find any good examples out there yet. Thanks for watching.

Updated the correct dual iso information, ISO difference : 3.98 EV (1577) 100/1600 that is :)
Also added a still image (third image) 3 exposures enfused to one. Amazingly the dual iso has a better range giving me blue skies in the windows
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 05, 2013, 02:03:05 AM
Updated the correct dual iso information, ISO difference : 3.98 EV (1577) 100/1600 that is :)

Indeed, dual_iso is giving amazingly good results, and I have yet to see any 100/1600 shots from the 6d to show any noticeable problems - but I'll keep looking for them :-)

The second ISO is a little hard to guess with current metadata (some cases are ambiguous). Example: main ISO 400 and a gap of 2 stops - is it 400/100 or 400/1600?

* I remember you writing that, that's why I suggested writing a xmp in-camera when the accurate information is still available just like deflicker ... but if you think you can reconstruct both iso values from the cr2 I'll provide an exiftool cmd line.

* One "problem" cr2 from me: It's a 30sec long time test exposure with a 10x nd filter and shows sprinkles of various colors all over the place. Is this due to me using iso100/1600, or is this a cr2hdr problem which could be fixed with an anti-sprinkle algorithm? Unfortunately I didn't take a non-dual_iso shot to compare with this one. Here's the cr2 and a converted jpeg so you can have a quick look:

https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2#005512.jpg
https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2#005512.cr2
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: blahjovic on October 05, 2013, 07:11:48 PM
Does anyone know if medium or small raw files are supported with dual iso? I can't seem to find any info about it.
I tried a timelapse last night with small raw set on my 6D.  I get an error when converting using cr2hdr (latest from https://bitbucket.org/a_d_/magic-lantern/downloads):

Input FIle : DUAL6977.cr2
Canon Eos 6D detected
Full Size : 2736 x 1824
Active Area : 2736 x 1824
Error: dcraw output is not a valid PGM file

Thanks for any help and for all the work guys!
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 05, 2013, 07:40:40 PM
If the menu allowed you to enable dual ISO with sRAW, this is a bug. Post some screenshots.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 05, 2013, 07:57:19 PM
The bug seems to happen if Q == sraw/mraw and you take a still in video mode. Or I guess on 6D if you left raw_rec on in photo mode (6D DOES need this if you want to monitor raw via wifi)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 05, 2013, 08:15:22 PM
Just pushed a fix (but didn't try it yet).
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: painya on October 05, 2013, 08:35:13 PM
Sorry if this has been brought up, but I can't drag and drop because the cygwin1.dll is missing from my computer. Any thoughts? This is being shot RAW (.CR2) with a 6d.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 05, 2013, 09:52:11 PM
Sorry if this has been brought up, but I can't drag and drop because the cygwin1.dll is missing from my computer.

Your dcraw.exe, cr2hdr.exe or whatever.exe has been compiled with cygwin (unnecessarily, btw) and not mingw ... so now it needs this dll in the same directory or in the search path. Get the dll from http://www.cygwin.com/

Or use non-cygwin compiles, for example get a native windows dcraw here: http://www.fastpictureviewer.com/downloads/
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: painya on October 05, 2013, 10:28:15 PM
Your dcraw.exe, cr2hdr.exe or whatever.exe has been compiled with cygwin (unnecessarily, btw) and not mingw ... so now it needs this dll in the same directory or in the search path. Get the dll from http://www.cygwin.com/

Or use non-cygwin compiles, for example get a native windows dcraw here: http://www.fastpictureviewer.com/downloads/
The fast picture viewer will work for this HDR method? (with the add on of course)
Thanks.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 05, 2013, 10:57:53 PM
The fast picture viewer will work for this HDR method? (with the add on of course)

No, their site just happens to provide current x86 and x64 compiles of dcraw.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: painya on October 05, 2013, 11:00:30 PM
I just started a thread on a seemingly too easy workflow. Mind checking it out to make sure it works for you as well?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: painya on October 06, 2013, 01:16:10 AM
Your dcraw.exe, cr2hdr.exe or whatever.exe has been compiled with cygwin (unnecessarily, btw) and not mingw ... so now it needs this dll in the same directory or in the search path. Get the dll from http://www.cygwin.com/

Or use non-cygwin compiles, for example get a native windows dcraw here: http://www.fastpictureviewer.com/downloads/
I install cygwin, and it still says I am missing the .dll. Any thoughts? I also installed the fast picture viewer. (I checked and found the .dll in the bin for cygwin)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: psihodrill on October 06, 2013, 01:48:45 PM
just update cr2hdr
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 06, 2013, 02:03:00 PM
https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2#005512.jpg
https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2#005512.cr2

Broken links?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: scrax on October 06, 2013, 03:42:52 PM
Broken links?
this too?
Quote
For Mac download:
Mac GUI for cr2hdr: https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 06, 2013, 05:05:37 PM
Broken links?

This really seems to be broken, maybe a bitbucket error/downtime - it should be there on my bitbucket download page https://bitbucket.org/Marsu42/ml-mod6d/downloads

If it continues not to work I'll re-upload them, maybe it's also the # in the name that confuses the system.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 06, 2013, 05:24:16 PM
Doh, it was really the # in the filename.. here ye go:

https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2X005512.jpg
https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2X005512.cr2
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 06, 2013, 11:32:31 PM
After taking quite a big series of dual iso photos on my Canon 50D I've noticed that each and every one DNG created has white balance values off by about -500K, +30 tint (in ACR) compared to non-dual-iso photos taken in the same circumstances. As this value is quite constant, is it cr2hdr error or some ACR bug in reading "As shot" value from DNG?

Additionally, here's one conversion that gives strange results:
CR2: https://mega.co.nz/#!Yc9VFC6T!cidlsw8J2YOGGTVnsmTzlVjuPK8v8MtM4oAZTK_Sgrs
DNG: https://mega.co.nz/#!oQs0kYob!NLGjazsBAfPAuhBT4KuUN0vwUw8kbPlp2kIK3oqgS8Q
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 06, 2013, 11:48:31 PM
Is exiftool running?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 07, 2013, 12:14:14 AM
Is exiftool running?

I think so – cr2hdr gives no error while running, exiftool.exe is in the same dir and runs fine when called. It's in the path too.

Here's an example of what I'm talking about:
https://mega.co.nz/#!gYcwSYLJ!cQuj1zx4Q2k1IphjgMZIOBHjOoJpwEC4HUHsTPDM2jI
In the zip you get non-dual-iso CR2, dual-iso CR2 and generated DNG. I need to change WB of the DNG by about +500 K, -30 tint to have it similar to non-dual-iso CR2.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: engardeknave on October 07, 2013, 05:32:07 AM
I actually tested this today.

(http://i.imgur.com/J02ZXjC.png)

(http://i.imgur.com/Z4sWuT1.png)

The .dng seemed to be about right judging from both the photo and histogram at +26 tint (+34 change) and no change in temperature.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: budafilms on October 07, 2013, 05:48:37 AM
(I see greener .dng)

Do you know if DNG extracted from raw2dng and Son of the batch are the same file?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 07, 2013, 10:15:11 AM
As this value is quite constant, is it cr2hdr error or some ACR bug in reading "As shot" value from DNG?

I wouldn't say the tint/wb change is constant across the board, but maybe consistent enough to get some "standard" correction for different iso spacings and bv values so less needs to be done in post.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 07, 2013, 11:21:22 AM
cr2hdr does not touch WB coefficients and it's not meant to do so.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 07, 2013, 01:48:10 PM
cr2hdr does not touch WB coefficients and it's not meant to do so.

How come then that two photos — one normal, one dual iso — taken seconds apart in the same conditions have completely different WB? I don't have a single DNG file with proper WB set. Maybe it's ACR issue? See the example files: https://mega.co.nz/#!gYcwSYLJ!cQuj1zx4Q2k1IphjgMZIOBHjOoJpwEC4HUHsTPDM2jI
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 07, 2013, 02:00:47 PM
As I said earlier, the issue is in EXIF. So, just find the exiftool command that copies WB properly from the CR2 and I'll integrate it.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 07, 2013, 03:59:29 PM
A little big usability breakthrough:

https://bitbucket.org/hudson/magic-lantern/pull-request/251/dual-iso-preview/
related: http://www.magiclantern.fm/forum/index.php?topic=8667
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Onoxyne on October 07, 2013, 04:02:23 PM
Hello, thanks for your job a1ex !
I've got some problem with my cr2hdr in Ubuntu :

Quote
./cr2hdr _MG_5881.CR2

Input file     : _MG_5881.CR2
Canon EOS 60D detected
Full size      : 1336 x 3516
Active area    : 1336 x 3516
Unknown option "-E".
Error: dcraw output is not a valid PGM file

I've download the last Hudson repository this day.
Canon 60D.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 07, 2013, 04:06:35 PM
Your dcraw is too old.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Onoxyne on October 07, 2013, 04:46:13 PM
But I do the compilation in this day and in the source code, the dcraw_bridge date is 2013_09_08 ! What's wrong ?
How can I take the last build ? Thank you much.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 07, 2013, 04:49:47 PM
Type "dcraw" into google.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 07, 2013, 05:36:48 PM
A little big usability breakthrough:

Looks to be working good.
Extracted JPG previews.

Highlights
(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_UAL1501.jpg)

Lowlights
(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_UAL1500.jpg)

Sharplights
(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_UAL1502.jpg)

Right click | view image for full res.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: fotojohni on October 07, 2013, 06:01:18 PM
(http://i.imgur.com/FiDYiCj.jpg)

Not possible without dual iso.  This is a truly incredible development.  Now if only it didn't have to go line by line and instead could be a dual bayer pattern with 6 instead of 3. 

Also, any chance for an adaptive algorithm that measures scene contrast in live view then auto selects?   

Just putting out random ideas I bet you are already cooking up something even better.  can't wait.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 07, 2013, 06:08:39 PM
Also, any chance for an adaptive algorithm that measures scene contrast in live view then auto selects?   

http://www.magiclantern.fm/forum/index.php?topic=8322.0
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 07, 2013, 06:21:44 PM
Bug report: I just had a look at the exifdata cr2hdr and acr generate - and ml screws up the lens information, this is part of a diff between ml (wrong) and acr (correct).

Code: [Select]
-Circle Of Confusion             : 0.008 mm
-Field Of View                   : 30.0 deg
-Focal Length                    : 17.0 mm (35 mm equivalent: 67.1 mm)
-Hyperfocal Distance             : 2.37 m
-Lens                            : 17.0 - 40.0 mm (35 mm equivalent: 67.1 - 158.0 mm)
+Circle Of Confusion             : 0.030 mm
+Field Of View                   : 93.3 deg
+Focal Length                    : 17.0 mm (35 mm equivalent: 17.0 mm)
+Hyperfocal Distance             : 0.60 m
+Lens                            : 17.0 - 40.0 mm (35 mm equivalent: 17.0 - 40.0 mm)

In the cr2 the data is also correct, but cr2hdr adds a wrong crop factor:

Code: [Select]
-Scale Factor To 35 mm Equivalent: 1.0
+Scale Factor To 35 mm Equivalent: 3.9

Bug ticket here: https://bitbucket.org/hudson/magic-lantern/issue/1697/cr2hdr-messes-up-the-lens-meta-information
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 07, 2013, 06:22:48 PM
Extracted JPG previews.

What about enabling HTP? It should make the preview brighter by one stop, right?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: fotojohni on October 07, 2013, 06:28:46 PM
http://www.magiclantern.fm/forum/index.php?topic=8322.0

Okay this pretty much solves that.  I wonder how you get the metering data so that there is no need to go into live view for every shot.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 07, 2013, 06:43:57 PM
What about enabling HTP? It should make the preview brighter by one stop, right?

Yes it does. 
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 07, 2013, 06:45:56 PM
Okay this pretty much solves that.  I wonder how you get the metering data so that there is no need to go into live view for every shot.

Raw data is available in image quick review.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Onoxyne on October 07, 2013, 07:06:24 PM
Type "dcraw" into google.

OK, sorry for all the idiots questions !  ::)
cr2hdr is working ! That normal but my way to succed was very long.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 07, 2013, 07:07:49 PM
Type "dcraw" into google.

or, preferably, into some search engine that doesn't track your every movement like ixquick or duckduckgo :-o
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Onoxyne on October 07, 2013, 09:33:10 PM

Not possible without dual iso.  This is a truly incredible development.  Now if only it didn't have to go line by line and instead could be a dual bayer pattern with 6 instead of 3. 

Also, any chance for an adaptive algorithm that measures scene contrast in live view then auto selects?   

Just putting out random ideas I bet you are already cooking up something even better.  can't wait.

fotojohni : what are the photo parameters ? Which camera ?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: fotojohni on October 08, 2013, 04:20:29 AM
fotojohni : what are the photo parameters ? Which camera ?

Camera: 6D
ISO 100/1600
Lens: 50mm @ 1.4
VAF filter to reduce moire/aliasing
Shot in manual mode @ 1/45

For night shooting I switch recovery iso to 100 and shoot at 1600, then I can actually see what I am shooting in live view.  I didn't do that here though.

For grading I just went into ACR and bumped the exposure up 3.0 stops, boosted shadows etc.  I have a profile I like. 
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 08, 2013, 09:55:31 AM
As I said earlier, the issue is in EXIF. So, just find the exiftool command that copies WB properly from the CR2 and I'll integrate it.

This is what I found, maybe something will be useful in the future:

I renamed the files mentioned before to norm.cr2, dual.cr2 and dual.dng.

Adobe Camera Raw shows the following temperature/tint values ("As shot"):
norm.cr2: 5200/+14
dual.cr2: 5850/+38
dual.dng: 4850/+46

As you can see dual.cr2 has it wrong, too, but better than dual.dng anyway.

I tried using exiftool commands, but none worked for me:
exiftool -tagsfromfile norm.cr2 -Canon:WhiteBalance -WB_RGGBLevelsAsShot -WB_RGGBLevelsAuto -WB_RGGBLevelsMeasured dual.dng
(mentioned somewhere by the author of exiftool)
exiftool -WhiteBalance#=5200 dual.dng

Then I used hex editor to alter dng on my own.
After setting crs:Temperature="5200" and crs:Tint="+14" ACR shows the proper values of dual.dng file. This could be a good way of changing dng directly, but I don't know how to read temperature and tint value from a file. I've tried exiftool, exiv2, dcraw and all of them had other formats. And, as I found out just before posting it, it only works if ACR already exported xmp values into dng (standard cr2hdr output does not have it). But if it's possible to add some xmp to dngs, then crs:Temperature and crs:Tint works.

Second thing I've noticed: dcraw -v -w norm.cr2 shows the following: multipliers 2.102539 1 1.365234 1.

Now the command dcraw -T -r 2.102539 1 1.365234 1 dual.dng creates dual.tiff with correct WB, so it might be possible to use this method somewhere in cr2hdr.c.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1drey on October 08, 2013, 10:09:04 AM
Sorry for stupid question.

I've dragged CR2 created with Ml and dual ISO, the window got appeared and seemingly the conversion happened, but I can't find a resulting DNG
Where it should be?

PS. 've wasted a plenty of time trying to use 32 bit dcraw in 64 bit environment. Probably it is necessary to mention that users with 64x Windows need proper dcraw
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 08, 2013, 10:24:26 AM
PS. 've wasted a plenty of time trying to use 32 bit dcraw in 64 bit environment. Probably it is necessary to mention that users with 64x Windows need proper dcraw

It isn't mentioned because it isn't true, x86 apps work just fine on x64 Windows. If you are really determined to try yourself - Get both versions of the current dcraw here: http://www.fastpictureviewer.com/downloads/

The dng file you seek ends up in the same folder as the cr2 after drag n' drop.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: pest01 on October 08, 2013, 10:27:54 AM
Hi to everyone, i download the new Windows:cr2hdr.exe and it doesn't seem to be working on my windows i get a dcraw.exe - System error that says <<cant start cygwin1.dll is missing>> how do i fix it or do i stay with the previous one that worked? Are there any major changes in those two?

Thanks!!
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1drey on October 08, 2013, 10:33:50 AM
It is strange. I was getting " 'the application was unable to start correctly (0xc000007b)' " until 32 bit dcraw was replaced with x64 one.

And for some mysterious reason I don't see resulting DNG in the source folder. It was the first place where I expected it to appear.
Seems that *******.dng file was not created at all - I was searching it through the file search on my PCwith no avail.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 08, 2013, 10:37:13 AM
And for some mysterious reason I don't see resulting DNG in the source folder. It was the first place where I expected it to appear.
Seems that *******.dng file was not created at all - I was searching it through the file search on my PCwith no avail.

Put the .cr2 dual iso file in the same folder with cr2hdr.exe, dcraw.exe, exiftool.exe. Run cmd.exe there and execute "cr2hdr <yourfilename>". See if any errors appear.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1drey on October 08, 2013, 10:37:44 AM
Hi to everyone, i download the new Windows:cr2hdr.exe and it doesn't seem to be working on my windows i get a dcraw.exe - System error that says <<cant start cygwin1.dll is missing>> how do i fix it or do i stay with the previous one that worked? Are there any major changes in those two?

Thanks!!

I was fighting with the same issue.
Re-installed Cigwin, but started to receive a message  'the application was unable to start correctly (0xc000007b)'
Replaced dcraw with x64 version
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 08, 2013, 10:40:56 AM
Hi to everyone, i download the new Windows:cr2hdr.exe and it doesn't seem to be working on my windows i get a dcraw.exe - System error that says <<cant start cygwin1.dll is missing>> how do i fix it or do i stay with the previous one that worked? Are there any major changes in those two?

Just replace dcraw.exe with a version not requiring cygwin1.dll or install cygwin in your system.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 08, 2013, 10:47:48 AM
Do you have a link to a non-cygwin dcraw.exe that also works for 6D, so I can put that one in the zip?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1drey on October 08, 2013, 10:48:43 AM
Put the .cr2 dual iso file in the same folder with cr2hdr.exe, dcraw.exe, exiftool.exe. Run cmd.exe there and execute "cr2hdr <yourfilename>". See if any errors appear.
Got this
(http://pano.1drey.com/stills/cr2hdr.jpg)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 08, 2013, 11:00:36 AM
Do you have a link to a non-cygwin dcraw.exe that also works for 6D, so I can put that one in the zip?

I don't have 6D to check. I use some 9.17 version 576064 byte on my 50D. Some new builds of 9.19 are here: https://sites.google.com/site/manuelllorens/ – but I didn't test those.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 08, 2013, 11:05:23 AM
Got this
(http://pano.1drey.com/stills/cr2hdr.jpg)

Looks like you're in another folder…

Open the folder in explorer. Check if all your files are here. shift+rightclick and choose "open command prompt here" (it does not work on every windows version) and check again. Or run cmd.exe and cd <yourfolderpath> before running the command.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 08, 2013, 11:16:26 AM
Do you have a link to a non-cygwin dcraw.exe that also works for 6D, so I can put that one in the zip?

As I'm posting to the extend of feeling ignored :-o ... http://www.fastpictureviewer.com/downloads/ ... you gotta scroll down a bit or search the page for "dcraw"

Btw a multi-threaded dcraw is here, but as it's not the newest version it unfortunately will only work for older cameras (i.e. not the 6D) - but for the rest it should be faster: http://www.heliconsoft.com/heliconsoft-products/dcraw/
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1drey on October 08, 2013, 11:18:51 AM
Looks like you're in another folder…

Open the folder in explorer. Check if all your files are here. shift+rightclick and choose "open command prompt here" (it does not work on every windows version) and check again. Or run cmd.exe and cd <yourfolderpath> before running the command.

It kind of helped! :)

How I get this:
(http://pano.1drey.com/stills/ch2hdr2.jpg)

If I replace x64 dcraw with one from archive - I get an error message ('couldn't start correctly' etc.)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 08, 2013, 11:44:20 AM
Updated the zip on main page with dcraw 9.19 32-bit from the link from Marsu42. Does it work?

@1drey: make sure you update ML for EOS M. Early versions produced files that can't be converted.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 08, 2013, 11:45:14 AM
I'd like to share my quick and very dirty way to get dual iso files from CF card as fast as possible (…)
Topic moved here: http://www.magiclantern.fm/forum/index.php?topic=8738.msg81768#msg81768
Please post your comments there.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: painya on October 09, 2013, 03:50:11 AM
I'd like to share my quick and very dirty way to get dual iso files from CF card as fast as possible.
Normal steps are:
1. Get files from CF cards (it's a long process for me especially with USB 2.0 card reader and some slow 30MB/s cards).
2. Process them by cr2hdr (I use the script provided by engardeknave (http://www.magiclantern.fm/forum/index.php?topic=8520.0) without any prefix as I get quite many dual-iso files without prefix).
3. Rename them (I use exif date based system that moves filename.cr2 into something like a:\2013\10\07\20131007T121930filename.cr2).
4. Find dual-iso .cr2s in those created dirs and move them to dualiso subdir (I want to see only generated dngs while browsing, if something goes wrong I can alway go to dualiso subdir for original).

This takes a looong time.

Here is a zip file containing cr2hdr.exe, exiftool.exe, dcraw.exe (9.19, not requiring cygwin1.dll) together with dualiso.vbs (altered script by engardeknave), commands.cmd (this calls cr2hdr.exe and changes names), and runiffiles.cmd (a loop that calls dualiso.vbs if there's any file left to process).

Now, put all those in one directory, upload all .cr2 files in the same dir and run dualiso.vbs (you can change maxprocs to higher number for 4-core processors, but see KNOWN BUGS at the end). After some time you'll get all your files processed and put into different directories

Second method is even more dirty but gives you ability to run the script while getting files from the cards. Just start downloading from your card into the directory with the unzipped files and run runiffiles.cmd (after some bunch of .cr2 files are already there). The crazy script just checks constantly if there are any .cr2 files and calls dualiso.vbs then. So you can put all photos from several different cards and in the meantime the computer will process and rename them for you. You can leave the computer while moving the files and at the end everything will be done.

THESE SCRIPTS ARE PROVIDED AS IS AND I DON'T GIVE ANY WARRANTY IT WORKS AND DOES NOT DELETE YOUR PHOTOS SO PLEASE TEST IT ON SOME COPIES FIRST!!!!

KNOWN BUGS:
1. The script sometimes (especially at the beginning) calls more than maxprocs threads of cr2hdr.exe – that's probably because it tests for working cr2hdr.exe but calls commands.cmd, which gives some racing condition.
2. runiffiles.cmd sometimes never stops because some file already exists and exiftool won't move it, which makes runiffiles call dualiso.vbs again and it loops. I think it's because dualiso.vbs exits before all cr2hdr.exe processes are done. The easiest way to cope with this is to move the lonely files to other location, wait for runiffiles to stop and check if the files are already processed (should be).

ZIP FILE
http://bit.ly/dualisopack2

As I said, it's very dirty but WFM and saves me some time. Feel free to make it somewhat more user friendly and post it here.
I think this deserves its own thread in the post processing forum. Any one else?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: chris_overseas on October 09, 2013, 11:54:30 PM
I'm using a custom file prefix for my images, set via the standard Canon menus (5D mark 3). When using dual-ISO I would like to use the "DUAL" prefix instead, but enabling the "Custom file prefix" option in the dual-ISO settings doesn't seem to have any effect. All my dual-ISO images are still prefixed with my custom file prefix. Is that by design or is it a bug? I'm hoping it is a bug... :)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 10, 2013, 12:07:10 AM
He who gets there first, gets the prefix and keeps it. So if you have one from bracketing or one from dual ISO you won't be able to change it until whatever changed it releases it.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 10, 2013, 01:37:41 PM
Two badly processed Dual-ISO photos:

1. CR2 (https://mega.co.nz/#!Yc9VFC6T!cidlsw8J2YOGGTVnsmTzlVjuPK8v8MtM4oAZTK_Sgrs) DNG (https://mega.co.nz/#!oQs0kYob!NLGjazsBAfPAuhBT4KuUN0vwUw8kbPlp2kIK3oqgS8Q)
(http://i.imgur.com/kOHLRgq.png)

2. CR2 (https://mega.co.nz/#!cI1hzJhS!CqVmRQBoQv4Mu34tjy20LgGH83tQJSHtGwnFUoSwcU0) DNG (https://mega.co.nz/#!tIMjxZiR!bvz87l0yQiyXWGhAx4vxwh9mB0tRSwFN2poDXISqCnE)
(http://i.imgur.com/Kx4PDif.jpg)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 10, 2013, 01:50:40 PM
Sensor response was nonlinear... I may need to estimate ISO in log space.

First one solved.

Second one is harder and it seems to have issues with chroma smoothing.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Doyle4 on October 10, 2013, 02:19:41 PM
What camera did you use akumiszcza? i noticed i had an image like your bottom sample on my 600D, only the one though.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 10, 2013, 04:25:23 PM

What camera did you use akumiszcza? i noticed i had an image like your bottom sample on my 600D, only the one though.

It's 50d. The problem on the bottom sample appeared in some previous photos in dark scenes, but only on small fragments.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 10, 2013, 04:32:47 PM
Here's an experimental cr2hdr (http://acoutts.com/a1ex/cr2hdr_exp.exe) that solves the second picture.

It also fixes some dark spots that sometimes appeared in noisy areas.

Problems (this is why I didn't update it on the first page):

- you can see more aliasing on resolution charts, and in IMG_6706 from Danne (maybe others too).
- noise pattern is also different; visible on IMG_8485 from naturalsound, for example.

So, if you find a situation where you get noticeable shadow aliasing (more than with cr2hdr from first post), upload a CR2 and a side-by-side comparison.

Next time try exposing with ETTR. The second picture was really underexposed (and in this case it's better to simply shoot at ISO 1600).
Title: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 10, 2013, 04:50:01 PM
Both photos were made with AETTR+Dual-ISO, but on AutoSnap mode and both, I think, were not the final, settled, photos.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 10, 2013, 04:53:22 PM
Got it. If it's the one after a overexposed attempt, this explains it.

When ETTR looks at something underexposed, it knows exactly how much to adjust, but if it looks at something overexposed, it has no idea. It tries to use some heuristics from previous photos; sometimes works, sometimes not. If you know this, you should be able to predict what it's going to do.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: djronbxs on October 10, 2013, 06:02:18 PM
shoudl I use dual iso when taking a bracketed hdr ?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Walter Schulz on October 10, 2013, 06:43:46 PM
Dual-ISO is meant for situations where HDR is not possible. Dynamic scenes vs. static scenes (with a grain of salt). If you are able to use HDR in a given situation you won't need Dual-ISO.

Ciao
Walter
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 10, 2013, 07:32:11 PM
If you are able to use HDR in a given situation you won't need Dual-ISO.

... unless you haven't won in the lottery lately and would like to use 1 shutter cycle instead of 3, extending the camera's lifetime. Also using dual_iso will save you a lot of postprocessing time as it's "sooc", esp. the 6d benefits because unlike the 5d3 it doesn't do raw in-camera hdr.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 10, 2013, 07:42:26 PM
Only problem is the 1/2 res shadows.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 10, 2013, 07:46:31 PM
Do you have an example where this is noticeable?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 10, 2013, 08:24:04 PM
Somewhere, its not that bad. The only real issues I've had were when I did dual ISO 800/100 and the image comes out completely black and then is a bit noisy when its brought up. I'm not a fan of using it in reverse, is this even proper?

I think this one the ISO 100 was just too far under. I have to look for or make some reversed ones where similar things happen.

http://www.filedropper.com/dual0955
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: SpcCb on October 10, 2013, 10:41:55 PM
I use Dual ISO in reverse (ie. 800/100 || 400/100) most of time, without any problem [5D2].
It's very useful to recover hight lights and/or to get max dynamic.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 10, 2013, 10:52:55 PM
Okay, found the bug that was causing noisy shadows in the forest shot.

Before and after:
(http://acoutts.com/a1ex/forest00.jpg) (http://acoutts.com/a1ex/forest-0-0.jpg)

Also it fixes some black spots and banding, visible in the shot from 1% (which was also extremely underexposed; both test shots were developed at +6 EV in ufraw):
(http://acoutts.com/a1ex/DUAL09550.jpg) (http://acoutts.com/a1ex/DUAL0955.jpg)

This one passed all my pixel peeping tests, so you can download it from the first post. There's some small color fringing in the resolution charts, but I highly doubt you'll notice it in practice (all my other test shots look the same or better). If anything is rendered worse than with previous version, upload a CR2 and a side-by-side comparison.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 10, 2013, 11:32:59 PM
Looks good, now have to see if I have any more @ rerun what I did already.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 11, 2013, 07:34:52 AM
Thank you, a1ex – works great! As far as it's possible for these bad and unexposed photos ;)

BTW, I've found a quick way of checking if a given .cr2 is Dual-ISO, or not (I get quite many Dual-ISO photos without the prefix and some non-Dual-ISO with the prefix – I guess it's some kind of racing condition while having Auto Snap or Always On and shooting in quick series?). By turning small image previews in additional panel of Bridge (smaller than thumbnails for me at least) I see the stripes quite clearly – Moire effect. It's much faster than opening 100% previews at least. It should work with other software I guess, but you need to find the proper preview size.
(http://i.imgur.com/hr5kAdC.png)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 11, 2013, 08:50:16 AM
I simply run cr2hdr on all files (usually, more than half of them are dual ISO, and the ones that are not are skipped quickly). Then, my raw processing script picks the DNG if there's one, or the CR2 otherwise.

The file prefix is not quite synced in burst mode, so I'm not sure whether to keep it or not. I don't think I can make it reliable in the near future.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 11, 2013, 09:01:37 AM
I simply run cr2hdr on all files (usually, more than half of them are dual ISO, and the ones that are not are skipped quickly). Then, my raw processing script picks the DNG if there's one, or the CR2 otherwise.

I do the same way — see the scripts here: http://www.magiclantern.fm/forum/index.php?topic=8738.msg81768#msg81768
I've just noticed the moire in small previews and thought it might be useful for some people.

The file prefix is not quite synced in burst mode, so I'm not sure whether to keep it or not. I don't think I can make it reliable in the near future.

I would opt for removing it — it's unreliable and might leave some photos processed wrongly (it's not visible at first glance sometimes if photos are dual-iso and somebody might batch process them without using cr2hdr first, which can give unpredictable results).
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 11, 2013, 10:21:24 AM
The file prefix is very handy.  Sure it doesn't work in burst mode but that's what warnings are for.
Processing 2000+ wedding photos, it's much easier to pick out the dual ISO shots with the prefix then to wait hours batch processing the lot of them.

It's only a problem where you have alternate shooting enabled.  I'd settle for reduced capture rate where file prefix is not default and alternate capture is enabled.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on October 11, 2013, 02:39:01 PM
Dual iso prefix. How do I enable this? Tried to in dual iso menu but there was no difference in prefix? I, m not in burst mode.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 11, 2013, 03:23:58 PM
I just do the prefix + check the photos in FS viewer when moving them... has worked for like 400-500 pics.
Title: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 11, 2013, 06:28:19 PM
The file prefix is very handy.  Sure it doesn't work in burst mode but that's what warnings are for.
Processing 2000+ wedding photos, it's much easier to pick out the dual ISO shots with the prefix then to wait hours batch processing the lot of them.

It's only a problem where you have alternate shooting enabled.  I'd settle for reduced capture rate where file prefix is not default and alternate capture is enabled.

I've made a short benchmark for 20 non-dual-iso cr2 files using cr2hdr alone and scripts I have there: http://www.magiclantern.fm/forum/index.php?topic=8738.msg81768#msg81768
cr2hdr alone (drag&drop) = 1:46.52
No renaming of files, just many processes of cr2hdr:
timer cmd /c dualisonorename.vbs = 48.391 (maxprocs=4) 40.902 (maxprocs=8 ) 41.252 (maxprocs=16)
Many processes of cr2hdr + renaming using exiftools:
timer cmd /c dualiso.vbs = 59.851 (maxprocs=4) 62.404 (maxprocs=8 )

So for 2000 photos it would take 1 hour 8 minutes on my old computer (Core2 Q6600). Less than that, because the program takes some time to start and stop (less processes at once then). Of course, for Dual-ISO files the time will increase, but so as with your method.

And I think I have some photos with wrongly assinged prefix that were not made by burst but by Always On AETTR. Probably ML finished calculations of new exposure at the exact moment I took the photo?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 11, 2013, 07:21:33 PM
It's possible; when ETTR switches from dual to non-dual ISO, the change may not be applied right away; so if you catch the wrong timing, you are out of luck.

I've added a warning for now. Having a file list (like with intervalometer) may be easier to sync.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: djronbxs on October 11, 2013, 07:38:05 PM
what do we need to download now from your first post ? cr2hdr or the dual iso module ?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 11, 2013, 07:43:56 PM
Having a separate log of which files are dual ISO would make it easy for a post application.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 11, 2013, 09:59:26 PM
Small cr2hdr update for video guys: better guess of black level in the absence of optical black bars.

Before:
(http://acoutts.com/a1ex/000166-0.jpg)

After:
(http://acoutts.com/a1ex/000166.jpg)

There may be subtle differences in photo mode too, in shadows (since this guess is used during blending).



Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: tron on October 12, 2013, 04:03:53 AM
@alex: Can you please check:

https://www.dropbox.com/s/cvdz6nz2t0rvnxy/DUAL0650.CR2

I converted it with the Oct-10 version (latest I guess) and at 100% (and more) patterns are visible.
(Shot at ISO 100/1600. The resultant DNG was set as: Exposure +3.2 Highlights -100 Shadows +100 Temperature: 3200)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: tron on October 12, 2013, 04:18:55 AM
@alex again. I had a problem with the dcraw (continuous error messages about not being valid win32 application if I recall correctly) from the latest .zip (Oct 10) I have overridden it with the dcraw from Sep 20.

 
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: djronbxs on October 12, 2013, 05:12:04 AM
what do we need to download for the fix now from your first post ? cr2hdr or the dual iso module ?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 13, 2013, 09:43:23 PM
@alex: Can you please check:

https://www.dropbox.com/s/cvdz6nz2t0rvnxy/DUAL0650.CR2

I converted it with the Oct-10 version (latest I guess) and at 100% (and more) patterns are visible.

I must be blind, I can't see any patterns here (except for some noise in shadows, which is normal). Not even with crazy curves with extreme contrast.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: tron on October 13, 2013, 10:00:43 PM
I must be blind, I can't see any patterns here (except for some noise in shadows, which is normal). Not even with crazy curves with extreme contrast.
Alex I used Oct 10th cr2hdr.exe when I posted this. When I used Oct 11th version it seemed OK to me too!!!!
In fact I thought that maybe that's why you changed it!
What I still see is zig zag lines in a specific part where there is a round object. But if I recall correctly you had mentioned this yourself a few months ago as a characteristic of the procedure.

Sorry for not updating my observations with Oct 11th cr2hdr.exe

Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on October 13, 2013, 10:55:30 PM
Nice to see more improvements upon dual iso and movie mode.
A question. @tron or anybody. Eager to try the changed prefix for dual iso which I understand is unreliable. Still, selecting this in dual iso the filenames are still without the new prefix I see some of you get? I use a 5d mark 3. Am I missing something? I,m in single shooting mode, Taking one picture, waiting. Still no changes in prefix?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: swinxx on October 13, 2013, 11:48:56 PM
Hi alex,

here is a Dual Iso Shot where i shot a face of a person and a bright sky. when i use the latest cr2hdr 1.5 app for mac the dngs look strange.. (especially the hair of the guy - like there is less resolution)

here are converted dual iso dngs:
https://www.dropbox.com/sh/ukdu06xedivedog/FyiCLZcHkm
 
and here are the unconverted dngs out of rawmagic 1.0
https://www.dropbox.com/sh/guqv68nxfn8uvc1/UKS4lnnjA9

thx. sw
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 14, 2013, 03:13:29 AM
Nice to see more improvements upon dual iso and movie mode. A question. @tron or anybody. Eager to try the changed prefix for dual iso which I understand is unreliable. Still, selecting this in dual iso the filenames are still without the new prefix I see some of you get? I use a 5d mark 3. Am I missing something? I,m in single shooting mode, Taking one picture, waiting. Still no changes in prefix?

Do you have a custom prefix set via Canon menus?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on October 14, 2013, 05:49:03 AM
Nice @Audionut. Thanks. Had to change under menu "filename" to "preset code".
Tried to shoot in burst mode and it didn,t miss naming to "DUAL" once. Maybe isn,t that unreliable after all? PLease do not take this feature away @Alex. Gonna save me tooons of time :)
By the way. Had no idea I could change naming prefix in the camera, sweet.
Thanks a lot for yet another feature Alex
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 14, 2013, 06:10:50 AM
Tried to shoot in burst mode and it didn,t miss naming to "DUAL" once.

Set dual ISO to alternate frames only and then rapid fire ;)
Depending on the scene and the SNR limits in AETTR with linked dual ISO, it could miss the prefix there also when trigger happy.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on October 14, 2013, 06:20:25 AM
hehe, I see. Luckily I,m not that trigger happy atm but will try to see what happens. Still very useful :)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 14, 2013, 06:42:20 AM
I wouldn't use dual ISO on large shoots without the prefix.  PP is already a time consuming job.

So for 2000 photos it would take 1 hour 8 minutes

On top of the process time for the actual dual ISO shots.  So if I have 100 dual ISO shots, I've doubled that part of the processing time for no other reason then there is no other way to distinguish which shots need to be processed. 

Then there's the file management.  Prefixed shots get dumped into a separate folder and the processed DNG's copied into the shot folder.  This way I can keep the original CR2's separate for later processing (updated cr2hdr or whatever), without having to process all 2000 photos again, and, I don't have interlaced CR2's polluting the image viewer.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 14, 2013, 08:34:27 AM
here is a Dual Iso Shot where i shot a face of a person and a bright sky. when i use the latest cr2hdr 1.5 app for mac the dngs look strange.. (especially the hair of the guy - like there is less resolution)

Is this a regression? (was it rendered better with some older version?)

I'm not sure where to look, it seems OK to me.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 14, 2013, 08:47:32 AM
Alex I used Oct 10th cr2hdr.exe when I posted this. When I used Oct 11th version it seemed OK to me too!!!!
In fact I thought that maybe that's why you changed it!

I didn't expect any change in banding patterns (since the last changes were mostly for deep shadows). It may also be because I run all my test shots through ufraw, without any sharpening or denoising.

I have an idea for horizontal banding correction, so I'm looking for some test shots where this is clearly visible.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: swinxx on October 14, 2013, 09:58:24 AM
hi alex.

i don´t know if that shot would be better with older versions.
perhaps it is too much difference between the brightness and the darker areas in that special shot.?

thx. sw

Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 14, 2013, 01:08:55 PM
On top of the process time for the actual dual ISO shots.  So if I have 100 dual ISO shots, I've doubled that part of the processing time for no other reason then there is no other way to distinguish which shots need to be processed. 

If the prefix works for you, no reason then. But I get about 1 in 10 shots wrongly prefixed, both ways.

Then there's the file management.  Prefixed shots get dumped into a separate folder and the processed DNG's copied into the shot folder.  This way I can keep the original CR2's separate for later processing (updated cr2hdr or whatever), without having to process all 2000 photos again, and, I don't have interlaced CR2's polluting the image viewer.

That's what dualiso.vbs script does – it renames all files according to exif date and time, moves real Dual-ISO cr2s to separate folder than dngs and even makes a "_dualiso" suffix to Dual-ISO .cr2 and .dng to make them easier to distinguish. Anyway, I've already almost processed a Dual-ISO photo with some batch processing because I didn't notice it was Dual-ISO on quick preview (and it had no prefix). The result would have been strange…
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 14, 2013, 04:44:40 PM
If the prefix works for you, no reason then. But I get about 1 in 10 shots wrongly prefixed, both ways.

So disable it!
Title: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: akumiszcza on October 14, 2013, 04:46:45 PM
So disable it!

Yeah. That's the best remedy. Thanks…


BTW. Would it be possible to have a message during a shot review taken after SET mode in ETTR? 50D does not have ettr confirmation beeps and I don't know if ettr worked during LV after pressing SET, making the mode less useful than other. It would be great to have a message saying if ettr failed or not, especially since set mode is the fastest 
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: tron on October 14, 2013, 05:48:17 PM
Can the dcraw.exe fall back to sep 20 or before version? It does not work...
On the other hand Sept 20  version - or earlier, I never checked dcraw.exe creation date -anyway - works FINE (Even with newest cr2hdr.exe)
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 14, 2013, 05:56:16 PM
See http://www.magiclantern.fm/forum/index.php?topic=7139.msg81571#msg81571
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: tron on October 14, 2013, 06:19:30 PM
@alex: Thanks. I got dcraw.zip
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 15, 2013, 04:48:13 AM
BTW. Would it be possible to have a message during a shot review taken after SET mode in ETTR? 50D does not have ettr confirmation beeps and I don't know if ettr worked during LV after pressing SET, making the mode less useful than other. It would be great to have a message saying if ettr failed or not, especially since set mode is the fastest

Can't you judge the results from the histogram?
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 15, 2013, 02:04:01 PM
It's possible; when ETTR switches from dual to non-dual ISO, the change may not be applied right away; so if you catch the wrong timing, you are out of luck. I've added a warning for now. Having a file list (like with intervalometer) may be easier to sync.

An idea concerning the prefix problem: Would switching from "_MG_" Adobe RGB to "IMG_" sRGB work? Since dual_iso is about raw, it doesn't matter what color space the sidecar jpeg is in, but using a Canon rename function might fix the sync problem if switching the color space prop(?) has no lag.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 15, 2013, 03:43:41 PM
Small update in cr2hdr regarding horizontal banding. Basically, after matching the brightness (ISO + black difference), there's still some room for improvement, so this update does some fine-tuning for every dark line (so it matches the brightness of its neighbours).

The result is pretty subtle. I didn't find any example with singificant banding in final output, but it's clearly visible in the intermediate results. Here's a full resolution blending (the one that minimizes aliasing, but has pretty high noise):

Before:
(http://acoutts.com/a1ex/tron0.jpg)

After:
(http://acoutts.com/a1ex/tron.jpg)

So, the algorithm simply has cleaner data to work with, and it's less likely to pick up false detail. Also, the noise in full-resolution areas is less correlated (whiter). With this, I was able to push the algorithm even more towards full-res shadow detail without increasing the artifacts (so the forest shot is still rendered very well).

Example:

Before:
(http://acoutts.com/a1ex/IMG_6706-barrel0.jpg)

After:
(http://acoutts.com/a1ex/IMG_6706-barrel1.jpg)


For prefix, I think switching the color space has the same sync issue. Didn't try, but I don't see why it will be different.

There's no beep for ETTR with SET on any other camera; I look at raw zebras, histogram and ETTR hint.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: 1% on October 15, 2013, 05:05:25 PM
I just turn the beep off globally... too much beeping in general.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on October 15, 2013, 05:37:58 PM
Very nice improvement on the horizontal banding!
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 15, 2013, 06:30:08 PM
Very nice improvement on the horizontal banding!

I'm re-processing all my dual_iso cr2 now, these updates really keep my computer busy :-p ... fortunately with Lightroom you just need to replace the old dng & update the preview, all the metadata is in the LR database. Also afterwards I'm converting the ML dng to ACR dng which about halves the file size.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 15, 2013, 06:36:36 PM
Would be nice if you can find a before/after example where the difference is clearly visible.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: RenatoPhoto on October 15, 2013, 07:44:47 PM
I have improvements in shadows but not in highlights.  No banding examples.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Danne on October 15, 2013, 10:00:15 PM
I'm re-processing all my dual_iso cr2 now, these updates really keep my computer busy :-p ... fortunately with Lightroom you just need to replace the old dng & update the preview, all the metadata is in the LR database.

Smart workflow. I noticed that LR kept the metadatas when switching cr2 to dngs after applying xmp sidecar files. Didn,t try a fully new conversion yet though of my older cr2:s
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 16, 2013, 09:13:39 AM
Would be nice if you can find a before/after example where the difference is clearly visible.

The only problem shot I delivered is unchanged, it's still colorful confetti time :-\ ... https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2X005512-CR2HDR131015.jpg

Could you provide a way to produce *two* dng from cr2hdr, one bright and one dark (with duplicated scanlines)? That would make it easier to see where a problem was, i.e. how the composite shots were exposed, and if a problem was there even before cr2hdr assembly.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 16, 2013, 09:16:04 AM
Lookup "fullres.dng" in cr2hdr.c and mark that block with #if 1. You'll get a bunch of intermediate files.

You can do that everywhere you see calls to save_dng.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 16, 2013, 10:18:14 AM
Lookup "fullres.dng" in cr2hdr.c and mark that block with #if 1. You'll get a bunch of intermediate files.

Thanks, I'll do that - and I know you're not very keen to add any switches to cr2hdr, but imho this option would be useful to all people and not only to the few who compile ml themselves or read this thread. Also the intermediate files could be a last resort if cr2hdr fails on some shots, you can then blend the intermediate file with the good parts from cr2hdr.

Yesyesyes, I could do a pull request, but I feel sufficiently busy with the auto_iso module, I've got a couple of (imho :-)) very nice ideas for that.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 16, 2013, 11:25:02 AM
From your shot, I've cleaned up some of the dots with this trick:
Code: [Select]
                    int is_hot_small = (d > b && b < white_darkened);

but finding a general solution that doesn't break resolution charts is a bit harder. Also, locating hot pixels where the high ISO is overexposed is hard and I didn't try to solve it yet (a median filter with thresholding should help a bit).
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Marsu42 on October 16, 2013, 12:12:13 PM
but finding a general solution that doesn't break resolution charts is a bit harder. Also, locating hot pixels where the high ISO is overexposed is hard and I didn't try to solve it yet (a median filter with thresholding should help a bit).

Thanks, I'll make my own cr2hdr with your patch for the time being (these were just test shots anyway) - but imho having a public fix for this is important, dual_iso is just made for high-dr longtime exposures since you really wouldn't want to bracket in these situations.

Question is if you can find auto-detection algorithms for all possible contingencies without years of coding and wading through thousands of samples, if not you might have no other option than to fall back for command line parameters for local tuning like this hot pixel removal.
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: a1ex on October 16, 2013, 12:32:37 PM
I'll try a thresholded median filter; if I get good results, I'll post it.

Meanwhile, here's an interesting read: http://nikonhacker.com/viewtopic.php?t=85
Title: Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
Post by: Audionut on October 17, 2013