Time for the first real image quality improvement in ML history :)
Here's a trick that samples half of the sensor at ISO 100 and the other half at ISO 1600, for example. If you mix these two, you can get almost the entire dynamic range the sensor is capable of (around 14 stops). 5D3/7D only. Now it works on most cameras!
There are no motion artifacts; not even a difference in motion blur for the two exposures. So, it works well for fast moving subjects.
(http://acoutts.com/a1ex/diso/014.jpg) (http://www.magiclantern.fm/forum/index.php?topic=7402.0) (http://acoutts.com/a1ex/diso/dual_iso_test.jpg) (http://www.magiclantern.fm/forum/index.php?topic=7402.msg63104#msg63104) (http://acoutts.com/a1ex/diso/TFzi2Uk.jpg) (http://www.magiclantern.fm/forum/index.php?topic=7402.msg71370#msg71370) (http://acoutts.com/a1ex/diso/fireworks.jpg) (http://www.magiclantern.fm/forum/index.php?topic=7139.msg65307#msg65307) (http://acoutts.com/a1ex/diso/9255_594194790623658_495677599_n.jpg) (http://www.magiclantern.fm/forum/index.php?topic=7139.msg74445#msg74445)
Please post your samples here -> Thread photos dual ISO (http://www.magiclantern.fm/forum/index.php?topic=7402)
More samples on Flickr (tag: Dual-ISO) (http://www.flickr.com/search/?q=dualiso)
Tutorial - Guide to using Dual ISO Mode on your Canon 5D Mark III (http://rohidassanap.wordpress.com/2013/08/10/guide-to-using-dual-iso-mode-on-your-canon-5d-mark-iii/)
Open Source Brings High-End Canon 5D MK III Dynamic Range Closer to Nikon D800 (http://www.zdziarski.com/blog/?p=2792)
Dual ISO vs Exmor: on CanonRumors (http://www.canonrumors.com/forum/index.php?topic=22230.0), reddit (http://www.reddit.com/r/photography/comments/2d6qiw/exmor_vs_dualiso_full_writeup/), POTN (http://photography-on-the.net/forum/showthread.php?t=1393255) and FredMiranda (http://www.fredmiranda.com/forum/topic/1316361).
Code
- dual_iso module (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso)
- recent nightly builds (http://builds.magiclantern.fm/#/) should have this working on most cameras
Technical doc
- dual_iso.pdf (http://acoutts.com/a1ex/dual_iso.pdf) (in-depth description of how it works)
- See also: ADTG and CMOS discussion (http://www.magiclantern.fm/forum/index.php?topic=6751)
Warning
This code changes low-level sensor parameters. In the technical doc you can see how this method messes with the feedback loop for optical black, for example. Since the original implementation, many people have tried it (including timelapse), with very few side effects reported (e.g. more hot pixels than usual on long exposures). However, that's not a guarantee.
We don't pay for repairs. Use it at your own risk.
Usage
- Select one ISO from Canon menu, the other from Dual ISO submenu.
- Expose to the right for the lower ISO (usually ISO 100). Maybe darken 1 stop from there.
- Start at ISO 100/800.
- Try a larger gap for really extreme situations, or a smaller gap for better midtone detail (less aliasing).
Histograms, ETTR, playback tricks
- ETTR integrates with dual ISO (just enable both). It will use dual ISO only in high dynamic range scenes. Adjust the SNR limits to let the algorithm know what exactly is a "high dynamic range scene" and to fine-tune the balance between midtone/shadow noise and highlight aliasing.
- Raw zebras: weak markers appear where only one of the two ISOs is overexposed (likely to get aliasing), strong markers appear where both ISOs are overexposed (there's no more detail in these areas). See here for examples. (https://www.magiclantern.fm/forum/index.php?topic=12096.0#post_dualzebras)
- Raw histogram will use the darker exposure. Consider the brighter one as some sort of shadow recovery.
- JPEG zebras/histogram are totally unreliable.
- In photo LiveView you will see the ISO from Canon menu. In movie mode, you will see horizontal lines.
- Playback in 1x will show some sort of HDR preview. At certain zoom levels, you'll be able to see only one of the two exposures, at very low resolution.
- The dynamic range indicator from the histogram is not accurate. (solved!)
Postprocessing
- Reference converter: cr2hdr.c (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso/cr2hdr.c) (Windows: cr2hdr.exe (https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/)). It requires dcraw and (optional) exiftool in your executable path (they are in the zip). Drag and drop the CR2 files over cr2hdr.exe.
- For RAW video files, use the following commands in a terminal:
mlv_dump clip.mlv --dng --no-fixcp --no-stripes
cr2hdr --same-levels *.DNG
- GUI frontends:
- Lightroom plugin (http://www.magiclantern.fm/forum/index.php?topic=11056) (WIN/MAC)
- BarracudaGUI (http://www.magiclantern.fm/forum/index.php?topic=9102) (WIN)
- Script for multiple instances of cr2hdr (http://www.magiclantern.fm/forum/index.php?topic=8520.0) (WIN, VBScript)
- Mac GUI for cr2hdr: OSX_cr2hdr_app.zip (https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip) and OSX_cr2hdr.zip (https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr.zip)
- MLV to DNG Batch Converter (http://www.magiclantern.fm/forum/index.php?topic=10198.0) (WIN, video only)
- MLP (http://www.magiclantern.fm/forum/index.php?topic=13512) (MAC)
- To get a natural HDR look: try my automatic color grading script (http://www.magiclantern.fm/forum/index.php?topic=7022).
Price to pay
- Half resolution in highlights and shadows
- Aliasing and moire - of course, in highlights and shadows
- You can no longer check critical focus when zooming in
Samples
This is actually a video frame from 5D Mark III. Here's the DNG (http://acoutts.com/a1ex/000000.dng).
Bottom-left: ISO 100. Top-right: ISO 100/1600.
Credits: Luke Neumann.
(http://i.imgur.com/I0gYqqD.jpg)
The next one is a still photo from 5D Mark III. Here's the DNG (http://acoutts.com/a1ex/0R0A0640-fullres-soft.DNG) and the original CR2 (http://neumannfilms.mediafire.com/download/6tvihparmfwfbch/0R0A0640.CR2). ISO 100/1600.
(http://i.imgur.com/ezz8sAR.jpg) (http://acoutts.com/a1ex/0R0A0640-fullres-soft.jpg)
Questions? Read the pdf (https://bitbucket.org/hudson/magic-lantern/downloads/dual_iso.pdf) first. The mystery should clear up, I hope.
That's it. Looking forward to your samples. (http://www.magiclantern.fm/forum/index.php?topic=7402)
a1ex - this is amazing! Super impressed.
Excellent work a1ex.
:o Mother of God, this is amazing! You guys never fail to impress!
Wow... so clever! And so useful!! Thank you for your work on this and for the in depth technical docs.
It's all very interesting but it's a shame that you can only use the trick on the 5DIII and sort of on the 7D.
I guess that this research could easily lead into other interesting things, being able to control the sensor readout is a BIG thing.
Is there any possibility to port it to the 5DII
That's too bad it only works on the 5D Mark III. Oh well, loving the images out of my 50D (http://"https://vimeo.com/70206717").
This is too crazy. Who could ever imagine. The mother of all updates, with raw video to. E x c e l l e n t!
Amazing, thank you for more great features!
Please, for everyones sake people, don't come in here and ask "when can we expect this for <insert canon model here>"
The 5dmkiii is a highly capable camera and with it's hardware it makes sense that many of these new developments are coming forward on this platform. I, myself, moved on from my 550d to the 5dmkiii because of the future course of Magic Lantern developments.
This is an insane development to come in the midst of all the raw development. So much new stuff to go forward with, I'd imagine it's going to be quite some time before a stable release is on the horizon
Incredible work. The white paper was illuminating. Thank you A1ex!
Words fail me.... Amazing.
Look forward to testing this out
Nice work my friend!!!!
The way i understand it is that no further moire or aliasing is introduced in raw while recording vs stills.
This is amazing.
Awesome! Should buy myself some new cameras instead of lenses. Looks promising.
Quote from: anchoricex on July 16, 2013, 07:43:48 PM
Please, for everyones sake people, don't come in here and ask "when can we expect this for <insert canon model here>"
Amen, just reading the PDF should answer that question. Other cameras aren't sold with the same level of hardware as the higher end cameras.
Amazing, I had to read this 3 times so I can make sure what I was reading was what think it is. Holly crap. Is official, 5dIII is a mini Alexa.
BTW Alex, I can't access the PDF you posted in Dropbox. It gives me an error.
Error (509)
This account's public links are generating too much traffic and have been temporarily disabled!
This just means that we all love you A1ex!
sounds an amazing feature!!! so useful for HDR stills with motion in them .. and for video might be even more amazing
I can't however get any of the links to work? they say :
Error (509)
This account's public links are generating too much traffic and have been temporarily disabled!
Could the files be hosted elsewhere please?
James
i took the liberty of mirroring it:
http://linuxdingsda.de/~wintix/dual_iso.pdf
wow! wish i had a 5D3 and not a 5D2.
Awesome work; I don't have a 5DIII, so wont' be able to enjoy the cool stuff like this, but it's definitely a sign of progress that you guys are getting deeper into how the sensors are working and what not.
Random question that has to do with the sensor workings: On lower end models, would it be possible to reduce moire/anti-aliasing by modifying the sensor read outs or something similar? I know the 5DIII is typically moire/anti-aliasing free, but I'm not quite sure how or why that is. I know there's also the VAF filters available for lower end cameras, but still, with this type of manipulation I'm curious if there would be a way to reduce this common problem on lower end models.
Thanks for the awesome work you guys do and any time it takes to answer my question (which might be really foolish).
Magical !!!!!!
Although the bad side effects (low resolution moire aliasing) are serious defects.
Just thinking loudly ..
What if you try with alternating ISO line by line and use no interpolation just scale the not-burned hiISO pixels by low/hi (zero point should be the Black level). Then a good highlight recovery algo can make the job it knows best ...
This could decrease the DR expansion a bit (1 stop ??) but I think denoise algos can make a good job there also .. I am thinking of denoise at raw level like those at the "RAW" tab of RawTherapee
Line by line interlacing will give the opportunity to use hi ISO on the weak line depending on the WB (red for daylight, blue for tungsten) ..
Amazing Work! Thanks for your effort. :)
Brilliant work!
Interesting black-level drift analysis and correction.
Btw, I also put a mirror at http://dhilung.com/vault/ml/dual_iso.pdf (http://dhilung.com/vault/ml/dual_iso.pdf)
This would be very interesting to try out with my 7d. So far ML is amazing (even though it's not developed like 5Ds etc) and this would be so great to have. At least for fun. However, I'm not sure, since I'm not a developer of any sort, how to get this working on my 7D? Is it possible at this stage? If anyone can help, I'd be thankful.
Very cinematic images - can't wait to see the first video footage. A1ex you are a genius - probably you will soon be abducted by some spy agency to work for their super secret decoding missions...
Anxious to test this out, tried putting these files on my card but I know I am missing a step. I guess we need to wait for Lourenco to compile for 5diii users?
https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso
Quote from: jrumans on July 16, 2013, 09:46:13 PM
Anxious to test this out, tried putting these files on my card but I know I am missing a step. I guess we need to wait for Lourenco to compile for 5diii users?
https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso
You're missing autoexec.bin
Quote from: Stedda on July 16, 2013, 09:47:17 PM
You're missing autoexec.bin
Thanks for the heads up.
Would love to see a video of this! Link?
Hi,
Impressive work! One little question, I read in your pdf that the native ISOs are 100, 200 and so on, but some videos demonstrated that in video mode there is much less noise (aka native ISOs) on 160, 320, 640, ... at least on the 7D (growing noise order is 160, 320, 640, 100, 200, 400, 800, 1250, 125, 250, 500, 1000, 1600, 2500, 2000, 3200, 4000, 5000, 6400). Is there any logical explanation I'm missing here?
Regards, Michel
@ml images Were those videos talking about ML RAW or just plain old h264? I think A1ex mentions in his PDF that that does not apply here.
"As expected, intermediate ISOs like 160 or 250 do not cause any changes in ADTG/CMOS configuration.
These ISOs are obtained by applying some digital gain to the raw data acquired
at the nearest full-stop ISO, and this gain is configured from the DIGIC register 0xC0F08030
(SHAD GAIN). In LiveView, the gain is only applied to the YUV image (it does not affect the 14-bit
raw data at all), but in photo mode, the gain is burned into the raw data. Don't ask me why."
@Redrocks indeed I'm talking Canon H264 here, not ML files. So my understanding is that Canon does change ADTG/CMOS configuration for 160, 320, 640 and doesn't for 100, 200, 400 which are obtained in software post processing. Does this sound correct?
I'm curious.... How much does the 5D mk III sensor differ from the sensor of fx the Alexa?
Could one force the data handling and interpretation from the Alexa down on the 5D chip?
Quote from: ml-images on July 16, 2013, 10:41:58 PM
@Redrocks indeed I'm talking Canon H264 here, not ML files. So my understanding is that Canon does change ADTG/CMOS configuration for 160, 320, 640 and doesn't for 100, 200, 400 which are obtained in software post processing. Does this sound correct?
I believe native ISOs refer to the available analog ISO gains to the sensor, which are the 100,200,etc. (set in the CMOS #0 register)
The ADTG/CMOS configuration can adjust only in those native increments, and the third stop gains are applied digitally after the nearest native ISO settings have been set in the configuration.
If I recall correctly, some videos that demonstrated the lower noise in 320, 640, etc were due to the fact that these are negative digital gain applied to the closest ISO of 400, 800, etc... which on our canon dslrs result in better shadow noise due to the fact that it is effectively "exposing to the right" then stopping down. Even the PDF says that Canons are notorious for having bad shadow recovery. This is why 125, 250, 500 yield higher noise since they are taking ISO 100,200,400 and digitally gaining +1/3 EV
It's interesting that Canon implemented this capability in their sensor. I believe the technique is patented by Eric Fossum, the inventor of the active-pixel CMOS sensor. Here is one reference to the idea on dpreview; there are other references but I couldn't find them quickly: http://www.dpreview.com/forums/post/32010565
after installing this we can remove it and return to the previous build ?
or the sensor settings are modified "for ever" ?
Impressive. For real.
Most Online Today: 564. Most Online Ever: 564 (Today at 07:09:43 PM)
That's about twice the previous high and it looks like this has gotten everyone's attention. An old 4k thread got bumped the other day and I'd just been reading about how Arri are reluctant to move on from 2k, obviously to squeeze more money out of what they already produce, but also they feel that the current tech hasn't been fully exploited. Now this, The 5d3's shelf life just extended a few years.
Hmmmmm half rez in shadows and highlights plus some moire. That might be ruff on video. Would be less of and issue on stills though. This is really interesting though. Amazing work!!!! Didnt even think this kind of thing was possible. I thought the same about raw vid from a dslr tho.
Does it seem even remotely possible that this type of functionality could later be accessed for stills in APS-C lines that share a cut down version of the same sensor (if not the exact same sensor) as the 7D such as T2i and 60D?
Quote from: payne on July 17, 2013, 01:16:13 AM
Does it seem even remotely possible that this type of functionality could later be accessed for stills in APS-C lines that share a cut down version of the same sensor (if not the exact same sensor) as the 7D such as T2i and 60D?
Probably not; the 5D Mark III has an 8 channel readout along with the 7D, the others have a 4 channel readout from the sensor.
@a1ex,
would a 7D with VAF have minimal aliasing and moire?
You are a Genius.
I can't get the windows executable to work.
dcraw is working fine, but cr2hdr reports "no such file or directory".
cr2hdr D46A8015.CR2
Input file : D46A8015.CR2
'D46A8015.CR2': No such file or directory
Error: D46A8015.CR2Wonder if it's related to this?
Quotesnprintf(dcraw_cmd, sizeof(dcraw_cmd), "dcraw -v -i -t 0 '%s' > tmp.txt", filename);
:)ANy Links to this MAgic?? Would like to try this cool hack. 8) (Canon 7D)
I've got a 5D3 build here (https://www.dropbox.com/sh/xiksvwbehzmycq4/QbuV17WuE5).
Thanks Audionut, I wish I had the 5DMiii :(
Quote from: Audionut on July 17, 2013, 03:20:30 AM
I've got a 5D3 build here (https://www.dropbox.com/sh/xiksvwbehzmycq4/QbuV17WuE5).
Everything goes into the 'modules' folder, except for the autoexecbin which goes to the root folder of the card, right?
Also, when this newest build is on my card, there is a possibility to shoot normal video(without the double iso thing), right?
Quote from: aaphotog on July 17, 2013, 04:13:01 AM
Everything goes into the 'modules' folder, except for the autoexecbin which goes to the root folder of the card, right?
Also, when this newest build is on my card, there is a possibility to shoot normal video(without the double iso thing), right?
Sounds right for the file locations. I believe H.264 video shouldn't be changed and you should be able to disable the module, but there's only one way to find out. I would experiment with it tonight, but I don't dare try something this new the day before a big shoot.
Hopefully reg addresses are the same on 6D. I need a reliable way to figure those out.
Dear A1ex, your team work is effortless, generous and FREE. I wonder how come we see people here complaining a new funcionality does work on this or that model. I here to say, I deeply thankfull for the work you do even if my camera is not able to new funcionalities just because there are people like you and your team putting hours, knowledge and inteligence to the benefit of all of us. THANK YOU MAGIC LANTERN!!!!
Amazing stuff as always.
A1ex hacking sensor to its roots is really good. Can I expect RGB Raw output from sensor now?
I want three RAW files for one click. Red RAW, Blue RAW and Green RAW. Sensor recording for only one color per RAW file and producing 3 RAW files.
We have Foveon sensor technology from Sigma that has 3 layered sensor. Each layer is dedicated to just one color. Can we mimick it with ML now.
For those wanting to see a test of the Dual ISO feature in use, I put together a video comparing normal raw video and video shot using the new Dual ISO feature. More details and testing methods used can be found at the Vimeo link... http://vimeo.com/70459941 (http://vimeo.com/70459941).
Wow, this is quite exciting and impressive. How dysfunctional of canon to not exploit the fact that they'd basically made a camera which can capture a true HDR image in one shot.
It should be possible to do some quite sophisticated denoising of the iso1600 image guided by the data in the iso100 one.
Updated cr2hdr.exe, works for me under Wine. The dependencies (dcraw and exiftool) are included in the zip file.
Quote from: Audionut on July 17, 2013, 03:20:30 AM
I've got a 5D3 build here (https://www.dropbox.com/sh/xiksvwbehzmycq4/QbuV17WuE5).
Does this one work on firmware 1.2.1?
Dears, I'm sorry for maybe stupid question, but I really want to know..
Is it possible to read one shot twice? I mean - read sensor at iso 100, save it to buffer, and immediately read the same data at iso 1600? For CCD sensors it is definitely impossible, but for CMOS sensor it is not so obvious. So could you please help me?
Hi,
is possible postprocessing dual iso's files on mac with raw2dng?
Quote from: sebamax on July 17, 2013, 09:31:56 AMis possible postprocessing dual iso's files on mac with raw2dng?
Yes, has to be consumed with WINE.
http://www.magiclantern.fm/forum/index.php?topic=5404.msg34763#msg34763
Ciao, Walter
Quote from: Lichtgestalt on July 17, 2013, 09:27:26 AM
Dears, I'm sorry for maybe stupid question, but I really want to know..
Is it possible to read one shot twice? I mean - read sensor at iso 100, save it to buffer, and immediately read the same data at iso 1600? For CCD sensors it is definitely impossible, but for CMOS sensor it is not so obvious. So could you please help me?
What about 720p 48p? A1ex would that be a way to get around the lineskipping?
Quote from: Walter Schulz on July 17, 2013, 09:38:20 AM
Yes, has to be consumed with WINE.
http://www.magiclantern.fm/forum/index.php?topic=5404.msg34763#msg34763
Ciao, Walter
Yep wine for now until scrax has the time to update the mac raw2dng.
Grazie Walter,
think will be possibile compiling an update to raw2dng ?
I hope in scrax
isnt the whole point of raw to record the entire RAW image the sensor sees? and isnt iso on the canons just digital gain? So how if this gaining any DR over just shooting raw if iso has no effect on raw?
Ouch.
(https://dl.dropboxusercontent.com/u/34113196/ML/D46A8019.jpg)
Reducing the image size to around 33% all but eliminates the problems in the tree. But the power line is beyond repair.
Here, a different interpolation method may give different results. Try choosing another one (in cr2hdr.c, there are some defines at the top).
I guess plain averaging will handle this one a bit better than medians.
Quote from: lagunak47 on July 17, 2013, 09:58:27 AM
isnt the whole point of raw to record the entire RAW image the sensor sees? and isnt iso on the canons just digital gain? So how if this gaining any DR over just shooting raw if iso has no effect on raw?
The real point of RAW is to be working with uncompressed footage in post. My understanding is this technique is like hdr at the photosite level and of course ISO affects RAW - try using footage shot at 6400+
Quote from: squig on July 17, 2013, 09:43:44 AM
Yep wine for now until scrax has the time to update the mac raw2dng.
Oh man, Wine again :)
Quote from: squig on July 17, 2013, 10:46:24 AM
I think I'll wait it out.
Keeping my fingers crossed for Scrax to come through. This is beautiful work!!
Can you use this module with the 5D Mark II and where do I find it?
First post in this thread, third line. End of it.
Ciao, Walter
Good news for the 7D owners. Thank you Alex, we appreciate your efforts ;)
Quote from: 1% on July 17, 2013, 04:26:01 AM
Hopefully reg addresses are the same on 6D. I need a reliable way to figure those out.
Go get them 1% :D
Finally some hope for the 6d owners :)
With almost the same specs and iso performance as 5d3 maybe you could port this in some time. Fingers crossed.
No hope for poor people like me(550D)? Finger Crossed :(
Read the White Paper and cry. You're not the only one, that's for sure.
Also linked in the first post of this thread: http://acoutts.com/a1ex/dual_iso.pdf
See 3.1
Ciao, Walter
Help me, I need a little tutorial to install this update. thank you guys for helping me.
Thank you
B.
Quote from: Braam544 on July 17, 2013, 02:23:53 PM
Help me, I need a little tutorial to install this update. thank you guys for helping me.
Thank you
B.
Keep an eye out on the nightly builds and follow the instructions for installing the RAW video builds. If you have difficulty doing so, then it would be probably be best to wait for an official release.
Thats so great! Will this work with magicraw?
Thank you! Finally some good news for 7D owners! Is there any build to test out for 7D?
First of all, wow, wow, wow!
A few questions. I may have missed this in reading through the doc, so apologies if so. Is it correct that when in this mode, that highlights will always be half-res regardless of the DR of the scene? If this is so, it sounds like we need to be careful when shooting a flat scene (low DR) and exposing to the right. You could end up with a substantial portion of the frame in half-res.
And if that's true, is it possible to perform the dual-ISO trick only as much as the DR of the frame calls for? For lower-DR scenes it seems like maintaining highlight fidelity should be the first priority. As DR increases, I'd want to sacrifice shadow resolution first, and then highlight resolution last. Or maybe the other way. Or maybe symmetrically.
Also, is there something special about 4 stops? I mean how about the option to run 100/6400 for a full 16 stops, understanding that the extra 2 bits will also be at half-res?.
I can't wait to start shooting with this. The few videos I've seen posted so far are like miracles. Soon we'll all be lifting shadows bravely and chuckling over "noise reduction". Which reminds me, care to SHARPEN? Don't mind if I do.
Quote from: Dns on July 17, 2013, 11:44:45 AM
No hope for poor people like me(550D)? Finger Crossed :(
I'm by no means an expert on this, so take my words with a pinch of salt (or a tablespoon for that matter) :)
They need to program the ISO for each separate amplifier circuit for this trick to work. If the "poor men's cameras" don't have separate amplifier circuits, they have to find another way to implement this, like maybe somehow finding a way to hack a single amplifier circuit to alternate ISO every two lines, if it's even possible.
I don't know if it has a lot to do with the channel readouts though; all I've read about that was that it helped with higher readout speed and hence higher drive speed and better contrast detect AF.
It sounds like it was possible to use this feature also as an on sensor gradual nd filter without the resolution loss even.
Thank again for another brilliant addition to the firmware for the 5DM3. I did some extensive still-shooting with the mod this morning...the dynamic improvement is incredible -- well done!
Just a few comments using the cr2hdr program (These are not complaints/criticisms, just letting you know what I experienced.):
1. Moire is extensive with bird feathers.
2. I'm seeing color artifacts around small, over-exposed areas. (By small, I mean points that are maybe 10 x 10 pixels or less.)
If anyone wants to see samples, let me know.
AJ
Yes, some samples will be useful.
Alex, here is some feedback about the technique from theSuede:
http://www.fredmiranda.com/forum/topic/1227550/0#11686217
A1ex, I know you don't need any more ego stroking, but gotta say, you're a flippin genius man. Did some testing with this last night.
The extent to which you can push the RAW settings (shadows, highlights, exposure) around without noticeable noise is ASTOUNDING. For video, however, the aliasing is noticeable enough to push this feature into the "only in extreme emergencies" category, for me at least. And that's not meant to be a criticism in any way - when you skip lines, that's what will happen. I will be watching for future developments on this, if you get any brilliant ideas on how to eliminate the aliasing, this would be downright unfair to every other camera on the market
By the way, I was looking around for the "Donate" link - would like to toss more into the kitty. But couldn't find it anywhere.
Quote from: a1ex on July 17, 2013, 06:14:22 PM
Yes, some samples will be useful.
a1ex...Here are some 100% cropped images showing the artifacts. The first image shows extensive moire in the feather detail:
(http://iwishicouldfly.com/iwishicouldfly/images/moire1.jpg)
The second image shows the color halo effect. (Close-up of an orange.) This reminded me of the halo problem that Canon corrected when both the 5DM2 and 5DM3 were first introduced. It usually reared its ugly head with night-astro shots of stars or xmas lights at night:
(http://iwishicouldfly.com/iwishicouldfly/images/color_halo.jpg)
If you want the originals, let me know where I can upload them.
Thanks again for all your hard work. --AJ
@horshack,
I don't know who theSuede is (he seems to know some stuff though), but please ask him to read the PDF. I'm pretty sure he didn't.
In deep shadows I only use data from the higher ISO, therefore his averaging formula does not apply here.
@ajay,
Of course I need the original files. Use some file sharing service.
@all,
For aliasing, there's nothing you can do to remove it completely; but you can reduce the artifacts by using a better interpolation algorithm. Even the debayering method you use in your raw software matters, so try to fiddle with it.
Or, use less aggressive settings (e.g. ISO 100/400) to minimize the areas where aliasing may appear.
Here's an alternate cr2hdr/raw2dng, with the mean23 interpolation algorithm (see the source code for details). It has more jagged edges in the Batman shot and my other test shots, but should handle trees and power lines a bit better.
cr2hdr_mean23.exe (http://acoutts.com/a1ex/cr2hdr_mean23.exe)
raw2dng_mean23.exe (http://acoutts.com/a1ex/raw2dng_mean23.exe)
You can further reduce shadow aliasing by increasing the overlap amount (you'll get more noise). See the paper and the source code for details.
Quote from: stevefal on July 17, 2013, 04:08:30 PM
First of all, wow, wow, wow!
A few questions. I may have missed this in reading through the doc, so apologies if so. Is it correct that when in this mode, that highlights will always be half-res regardless of the DR of the scene? If this is so, it sounds like we need to be careful when shooting a flat scene (low DR) and exposing to the right. You could end up with a substantial portion of the frame in half-res.
And if that's true, is it possible to perform the dual-ISO trick only as much as the DR of the frame calls for? For lower-DR scenes it seems like maintaining highlight fidelity should be the first priority. As DR increases, I'd want to sacrifice shadow resolution first, and then highlight resolution last. Or maybe the other way. Or maybe symmetrically.
Also, is there something special about 4 stops? I mean how about the option to run 100/6400 for a full 16 stops, understanding that the extra 2 bits will also be at half-res?.
I can't wait to start shooting with this. The few videos I've seen posted so far are like miracles. Soon we'll all be lifting shadows bravely and chuckling over "noise reduction". Which reminds me, care to SHARPEN? Don't mind if I do.
I hope the below data will answer your questions. :) ;)
Based on DXO screen-DR measures
5DIII
screen per ISO Total Gain
ISO DR Gain from base ISO
0100 10.97 0.00 0.00
0200 10.87 0.90 0.90
0400 10.69 0.82 1.72
0800 10.41 0.72 2.44
1600 9.94 0.53 2.97
3200 9.23 0.29 3.26
6400 8.30 0.07 3.33
12600 7.48 0.18 3.51
7D
screen per ISO Total Gain
ISO DR Gain from base ISO
0100 11.12 0.00 0.00
0200 11.08 0.96 0.96
0400 10.76 0.68 1.64
0800 10.10 0.34 1.98
1600 9.02 -0.08 1.90
3200 8.26 0.24 2.14
6400 7.09 -0.17 1.97
12600 6.22 0.13 2.10
As we can see there is no reason to go over ISO 800 for 7D or ISO 3200 for 5DIII. Because we gain negligible DR while we loose a 1 stop range of the "perfect midtones" (where all pixels are used so no resolution loss no artifacts) to the problematic hi and low ranges where only half the pixels are in use.
IMHO for 5DIII I would not sacrifice the range of 1 stop of full sensor use for 1/3 stop DR increase at ISO 3200 not even for the 1/2 stop gain from ISO 1600.
Lets look at it from another side .. for example the 100-1600 case .. the calculated 3 stop gain is valid if we use all the pixels.
In theory, by using only half of the sensor at darks we should have 0.5stop more noise there than a full 1600 ISO shot, so the net gain should be 2.5 stops.
What we see in the samples is not only the better sensor behavior at ISO1600 but also a kind of simplistic denoise (by interpolation - averaging) and the excellent holistic approach of ML team regarding black point regulation, channel imbalance and FPN elimination.
It will be interesting to see the differences between using 100-800 vs 100-1600 ranges with 5DIII ... especially after using a mild denoise in ACR or Rawtherapee ..
Would it, if possible, increase DR if the 70D dual photo-diodes were used separately with different iso?
This is about as good as i can get it. I was working between ISO 400 and 1600.... Exposed heavily to the right (I tried about 10 exposures, this was best). Some pretty heavy colour moire and stair stepping in the highlights... But the mids and shadows are pretty good.
(http://s15.postimg.org/f4im1z8tj/400_1600_800_exp.jpg) (http://postimg.org/image/f4im1z8tj/)
This is great alex! I'm happy to help test with my 5d3, but I'm a mac guy and having some difficulties post processing the files. Read the pdf, downloaded the zips from page one, but then what? Sorry for the stupid ?'s...
I searched for a tutorial on using the cr2hdr.exe and the draw.exe but no luck. Thanks anyone for some direction!
Quote from: a1ex on July 17, 2013, 07:04:20 PM
@horshack,
I don't know who theSuede is (he seems to know some stuff though), but please ask him to read the PDF. I'm pretty sure he didn't.
In deep shadows I only use data from the higher ISO, therefore his averaging formula does not apply here.
Hi Alex, theSuede is a software developer specializing in high-end color workflow products I believe. He is very knowledgeable on most areas of imaging technology but especially on sensors and color. I'm not sure if he's read through your PDF.
I read through the PDF and I saw the section on deep shadows where you only use the ISO 1600 data, indicating "corresponding data from ISO 100 will be just noise". The ISO 100 lines will have more read noise and tossing them out will improve DR but at the same time will reduce photons collected by half, reducing the shot-noise SNR of those shadows from 22.93:1 to 16.21:1, based on some quick calculations assuming the deep shadows start at around 7EV below ISO 100 saturation. As you descend further into the shadows the read noise represents an increasing percentage of total noise. The cross-over point between read noise and shot noise is about -7EV (ISO 12,800 equiv), so even starting in the deep shadows the shot noise represents 50% of the total noise, so discarding the ISO 100 lines is still significant to the total noise.
Wonderful work, & great white paper!
The funny thing about all this is that this is all a workaround largely b/c of the ADCs being on a separate chip from the image sensor (as surmised by ChipWorks). Essentially: your technique is trying to recover the dynamic range of the CMOS sensor itself, not the sensor & the entire signal processing chain. This'd all be pretty unnecessary on a D600/D800, where there's not much difference between the two (although the higher ISO exposure could help for very very deep shadows that might otherwise suffer from quantization error at ISO 100). The actual CMOS sensors on Canons have a similar DR to that of the best Sony sensors, if we believe the numbers from DXO/Sensorgen.info (though, this doesn't take into account FPN; I'm not sure exactly where in the chain FPN is introduced).
Do you know what the difference between the top & left black bars is? I've seen this & often wondered.
Quote from: horshack on July 17, 2013, 10:33:11 PM
I read through the PDF and I saw the section on deep shadows where you only use the ISO 1600 data, indicating "corresponding data from ISO 100 will be just noise". The ISO 100 lines will have more read noise and tossing them out will improve DR but at the same time will reduce photons collected by half, reducing the shot-noise SNR of those shadows from 22.93:1 to 16.21:1
Hi horshack-- why would you 'reduce photons collected by half'? The exposure at the sensor plane is not changed; there is only one exposure. The ML hack is simply changing what you do with that exposure data, & so should not affect the SNR save for actually increasing it in shadows by reducing the effects of downstream noise (after ISO amplifier) in the ISO 1600 exposure.
The focal plane exposure (in terms of shutter speed/aperture) remains the same for both ISO 100 & ISO 1600 exposures; it's not like two separate exposure are being made (where the ISO 1600 exposure would have 4 stops less exposure which, yes, would increase the effects of shot noise).
Let me know if I'm misunderstanding you.
See here an additional comparison of Normal raw video "Denoised" vs Dual ISO raw video. https://vimeo.com/70511941 (https://vimeo.com/70511941)
Whoa, the moire on high contrast vertical edges is pretty bad.
Just found a bug in the interpolation code near clipped highlights. Updated converters in first post.
Before (half-res ISO 1600 interpolated):
(http://acoutts.com/a1ex/median-fix-before.jpg)
After (half-res ISO 1600 interpolated):
(http://acoutts.com/a1ex/median-fix-after.jpg)
Please remember that in video mode it's skipping 6 lines at a time, and in photo mode it's skipping only 2 at a time. So my suggestion is to try to post photo samples too; videos are not the strong point for this method.
The artifact you fixed looks a lot like the "serrated" edges in the video in the previous post. In light of your note regarding video versus photo use of the feature, will this fix improve the video case nonetheless?
Quote from: horshack on July 17, 2013, 10:33:11 PM
The ISO 100 lines will have more read noise and tossing them out will improve DR but at the same time will reduce photons collected by half, reducing the shot-noise SNR of those shadows from 22.93:1 to 16.21:1
Can you explain the relationship between resolution and SNR?
I just did a test using fine details (fur and grass) to demonstrate the line-skipping. This is of course a "worst-case" video, but it should help to show the interlacing.
Dual-ISO: http://youtu.be/w4PxA1KcUFA
Single-ISO: http://youtu.be/yiMdOVfCgJ0
As with a lot of recent new features I'm wondering "Why should I care, will this eventually available on the xyz (replace with your camera model)?"
It'd be great if this information would make it in news announcements w/o the need to flip through dozens of pages, because you never know if some features *require* a dual-digic or new camera (like dual iso 7d/5d3) or if these are just the first models that get it.
I'm sick and tired of these "wish it was on XYZ model" posts.
Quote from: Marsu42 on July 18, 2013, 02:56:50 AM
It'd be great if this information would make it in news announcements w/o the need to flip through dozens of pages, because you never know if some features *require* a dual-digic or new camera (like dual iso 7d/5d3) or if these are just the first models that get it.
The information is linked in the pdf in the first post.
Asking people to read is not an overly unreasonable expectation imo.
/* 00:00:02.660613 */ CMOS
/* 00:00:02.660658 */ 00 0593 404520BC
/* 00:00:02.660702 */ 01 0000 404520BE
/* 00:00:02.660742 */ 02 0225 404520C0
/* 00:00:02.660782 */ 03 0000 404520C2
/* 00:00:02.660820 */ 04 04F0 404520C4
/* 00:00:02.660859 */ 05 0433 404520C6
/* 00:00:02.660898 */ 06 0000 404520C8
/* 00:00:02.660937 */ 07 0000 404520CA
/* 00:00:02.660974 */ 08 000A 404520CC
CMOS regs for 6D I hope... maybe this will work for ADTG too.
It did! ADTG shutters for all!
Quote from: Audionut on July 18, 2013, 03:12:29 AM
The information is linked in the pdf in the first post. Asking people to read is not an overly unreasonable expectation imo.
Thanks for the information, given with the usual
finesse if I may be so bold to comment. I'd even assist you in your reading efforts: the pdf is linked with the description "in-depth description of how it works" which is exactly
not where I'd expect to find the very basic information "will this come to my camera" - that's why I've suggested to provide this in a more obvious way, i.e. neither buried in a pdf nor somewhere in the thread.
For everyone else: No, it's not coming to cameras except 7d/5d3 :-\
QuoteNo, it's not coming to cameras except 7d/5d3
Lol, don't be so sure.
Dual ISO artefacts with mean23 but much less than with standard current raw2dng.
Quote from: driftwood on July 18, 2013, 04:56:12 AM
Dual ISO artefacts with mean23 but much less than with standard current raw2dng.
Wow!
Did you get a 100/1600 shot?
Quote from: NateVolk on July 17, 2013, 09:08:53 PM
This is great alex! I'm happy to help test with my 5d3, but I'm a mac guy and having some difficulties post processing the files. Read the pdf, downloaded the zips from page one, but then what? Sorry for the stupid ?'s...
I searched for a tutorial on using the cr2hdr.exe and the draw.exe but no luck. Thanks anyone for some direction!
2nd that...also on Mac. Got some shots, just can't figure out how to make them look normal in AE.
@driftwood,
Can you upload 1 second (or less) of raw video (the RAW file, not the DNGs), where mean23 gives better results than the default median?
In all my tests, median was a lot better on edges (see my bugfix above - that's an interpolated image, 2 lines from the sensor, 2 interpolated). I did not shoot power lines, I don't like them :P
Quote from: a1ex on July 18, 2013, 08:54:01 AM
I did not shoot power lines, I don't like them :P
Who does?
From what I've seen so far 100/400 has a lot more DR, has Samuel run his DR test?
I hope this is helpful, sorry if it is not.
I downloaded the CR2 and DNG to have a play with and I found that by denoising the noisy rows in the shadow areas before demosiacing and then demosicing all rows some artefacts are reduced.
This is my version created from the cr2 (via dcraw then G'MIC) and then the DNG version (via UFRAW).
(https://dl.dropboxusercontent.com/u/10782279/ML.gif)
The denoising is a simple median filter on the noisy rows. It is applied to 3 consecutive pixels of the same colour EG R1,R2 and R3 and also on G1,G2 and G3 and so on.
R1 G1 R2 G2 R3 G3
I repeated the filter 3 times.
I used my own demosaicing method. I'm not a programmer, I just play with scripting in G'MIC , but I thought this might be useful.
Quote from: 1% on July 18, 2013, 03:54:08 AM
Lol, don't be so sure.
That would be great!
Anyway, what happened to iso-less? The increased DR is great for the guys tied to canon but the really interesting thing was that iso-less thing.
I'd like to test these for 7d but I'm still not sure how to implement / install it :(
You need to compile the code for yourself at this point... there are tutorials here on the site.
or you must wait for either a forum user to host their compiled code of the Devs to release an updated Alpha.
Regarding shadow aliasing: which of these 3 pics would you choose, and why?
http://acoutts.com/a1ex/shadow/a.jpg
http://acoutts.com/a1ex/shadow/b.jpg
http://acoutts.com/a1ex/shadow/c.jpg
Just for your reference, to get an idea of the noise levels:
ISO 100 half: http://acoutts.com/a1ex/shadow/iso100.jpg
ISO 1600 half: http://acoutts.com/a1ex/shadow/iso1600.jpg
I prefer 'b' because it has more texture detail than 'c', and tames the chroma noise that's objectionable in 'a'.
Quote from: sarangiman on July 17, 2013, 10:45:50 PM
Hi horshack-- why would you 'reduce photons collected by half'? The exposure at the sensor plane is not changed; there is only one exposure. The ML hack is simply changing what you do with that exposure data, & so should not affect the SNR save for actually increasing it in shadows by reducing the effects of downstream noise (after ISO amplifier) in the ISO 1600 exposure.
The focal plane exposure (in terms of shutter speed/aperture) remains the same for both ISO 100 & ISO 1600 exposures; it's not like two separate exposure are being made (where the ISO 1600 exposure would have 4 stops less exposure which, yes, would increase the effects of shot noise).
Let me know if I'm misunderstanding you.
For the shadows the ISO 100 lines are being discarded from the composite raw, which reduces the light-capturing surface area of the sensor in half.
I'd choose c. The first two have too much chroma noise in the shadows. I think texture detail between b and c are comparable.
a1ex: Not at home right now so can't send you original raws. Here's my latest screenshots shoot produced today showing the processing of the same RAW file fed thru each of the methods: mean23 v median6 v shadow100.
https://www.dropbox.com/sh/4rpnyayri6n1jrk/2iMOz1n5Nl/mean2rb3g%20450%20avg%20hot%20pixels%20per%20frame.jpg
https://www.dropbox.com/sh/4rpnyayri6n1jrk/osCxpEq1xz/median6-1500%20avg%20frame%20hot%20pixels.jpg
mean23 averaging 450 hot pixels per frame.
median6 1500 per frame.
Shadow100 averages 1000 hot pixels per frame on the same tested RAW file.
EDIT: UPDATED (now with shadow100) picture showing the major differences of each against each other:
https://www.dropbox.com/s/2h7x59girnbxdpu/mean23_median6_shadow100.png
As to your A,B,C, question the grain in A to C goes from courser to finer chroma noise. Most people are going to choose the aesthetically more pleasing C, but I'd probably hunch for B. There's a tad more shadow detail in A which could be processed in post.
Here's how I've got the 3 pictures:
a) should have no shadow aliasing (shadows were handled as if they were midones). To get similar results, use this raw2dng: http://acoutts.com/a1ex/raw2dng_shadow100.exe
b) is a 50% blend between a) and the high ISO image. Less noise, a little aliasing. http://acoutts.com/a1ex/raw2dng_shadow050.exe
c) is 25% a) and 75% high-iso. Therefore, even less noise, but a little more aliasing. http://acoutts.com/a1ex/raw2dng_shadow025.exe
By default, shadows used data from high ISO only. This gets minimal noise, but it seems to have serious problems with aliasing.
Of course, now the formula from Horshack's friend should be valid for case a).
@MA Visuals: can you repeat your test with these 3 raw2dng's? Just post-process the same video with them.
Latest cr2hdr uses 50% blend (link in first post).
@ajay and @audionut: can you develop your test shots (the same pictures that you posted earlier) with latest cr2hdr?
For the 3 images and choosing with a relaxed eye, and not scrutinizing/pixel peeping the image, C is more "pleasing" than the rest, its less noisy.
It might not be the same choice with a different subject matter and composition.
Read the PDF, maybe I missed this. Regarding 50% vertical rez loss in dark and highlights. So on either end we're getting line doubled 1920 X 540 instead of true 1920 X 1080? And whats the range of the lower rez? If we are getting 14 stops, is the low rez sections in the bottom 4 stops and upper 4 stops or what? You guys are the new gods of film.
Figure 8 shows where you get full resolution.
With my latest modifications, you also get almost full resolution in deep shadows, but with more noise (see my previous post).
https://www.dropbox.com/s/2h7x59girnbxdpu/mean23_median6_shadow100.png (https://www.dropbox.com/s/2h7x59girnbxdpu/mean23_median6_shadow100.png)
I'd pick picture C, but I could go B.
C obviously has less noise, it also doesn't seem to soften up the details.
Aliasing seems about the same, A a tiny bit better, but it's really noisy compared to the other two.
C because it has the least noise and artifacts. Although it has more aliasing around the top silver edge of the camera on the table and the table edge, and more color noise in the silver edges (of the camera). There is more perception of detail in the table in B, but it looks to be the luma noise.
A has by far the least aliasing.
Quote from: a1ex on July 18, 2013, 05:39:52 PM
@MA Visuals: can you repeat your test with these 3 raw2dng's? Just post-process the same video with them.
Rendering now... will upload the results shortly.
median6 v mean23 v shadow25 v shadow 50 v shadow 100
https://www.dropbox.com/s/noa57vx3h6g7r0y/shadow25_50_100test.png (https://www.dropbox.com/s/noa57vx3h6g7r0y/shadow25_50_100test.png)
I really like shadow 50 together with the less purple artifacting in mean23
Nice, which one do you like from Driftwood's test?
Another tip: in crop video mode, it will skip only 2 lines at a time (instead of 6). Therefore, you can expect a lot less aliasing.
You can also record at higher vertical resolution (1320) and then resize it to something lower. It should hide some artifacts.
my choice would be either 25 or 50.
50 has less aliasing.
25 has less noise
(https://fbcdn-sphotos-g-a.akamaihd.net/hphotos-ak-ash3/1005186_541996815857388_1702224196_n.jpg)
Quote from: a1ex on July 18, 2013, 05:39:52 PM
@ajay and @audionut: can you develop your test shots (the same pictures that you posted earlier) with latest cr2hdr?
A1ex ... I took the two images that I sampled before using the latest cr2hdr and the halo effect is completely gone from the orange. The second image still has moire caused by the feathers, but the false colors are reduced quite a bit. Definitely better with both images.
AJ
have tried out the dual_iso module with raw2dng at 7351be3d3c0c
sequences processed through resolve.
room was so dark i moved the Base ISO up to 800 and that may not have been enough.
did no camera preparation before this event - so i can't confirm that the comparison clips had significant Ev offsets applied.
dual iso mode seemed to work at all resolutions i tried - here's a 3586x1320 DNG
http://50.56.67.113/ml-17july13/000012.dng
http://50.56.67.113/ml-17july13/1927-3584x1320-000012.jpg - same thing as an RPP processed jpg at 1/2 scale
(http://50.56.67.113/ml-17july13/1927-3584x1320-000012.jpg)
3.5K did not seem to work as well as builds from last week or at least the usual small and memory hacks didn't work as well when i tried them - only got 1-2 seconds of 3.5K. 2.2K seemed continuous.
had one lock up where i needed to pull the battery
which repository has the code with median23 and other options? will have time to test this weekend.
To choose other options, just flip the conditional defines from the source code (INTERP_*). For shadow aliasing, adjust f_shadow.
Quote from: a1ex on July 18, 2013, 07:50:04 PM
To choose other options, just flip the conditional defines from the source code (INTERP_*). For shadow aliasing, adjust f_shadow.
great we can make our own custom blends.
modules/lv_rec/raw2dng.c
modules/dual_iso/cdr2hdr.c
if a clip is dual_iso will the current raw2dng always display the HDR info and the EV offset?
Ok... here's the new comparison using shadow100, shadow50, and shadow25. I enabled the download link on Vimeo for those wanting a more accurate representation of the footage.
Shadow 50 has my vote as a set it and forget it default.
Shadow 100 is too noisy... Shadow 25 has too much aliasing for me to use the footage... Shadow 50 has minor aliasing but much less noise than Shadow 100. On the other hand... Shadow 100 combined with a light pass of de-noising in post would work as well.
Vimeo link... https://vimeo.com/70574082 (https://vimeo.com/70574082)
Is the dual iso method similar to the canon video raw strategy?
http://blog.abelcine.com/2012/11/05/what-exactly-is-canon-raw/
I bet some of the C500 developers who are watching ML Raw development remember some of these steps very well.
A look at the artefacts of each of the test interp modes. Same dual ISO DNG opened 'As Shot - 5500 / Tint +17' (RawMagic and v0.13 OSX raw2dng no interp modes and opened with a tint of +66 then changed to +17).
https://www.dropbox.com/s/a3wd7s9sh5ud2sv/interp_artifact_test.png
Dual ISO test on 5D3 (ISO 100 & ISO 1600), post-processing using "raw2dng_mean23" and After Effects.
ok so I have a few thoughts here.
when using this in a video mode (60p as an example), could you alternate which exposure you use each row for each frame? that way you'd have the full resolution midtones output as it is now, as well as what is essentially a iso100 60i output and a iso1600 60i output. I would imagine losing some temporal resolution is probably better looking than losing half of your vertical resolution.
the other idea is, although I have much less hope for this to be feasible, is it required to choose the exposure per row or can it be adjusted on an individual photosite basis? Because if that's the case we could take a page out of fujifilm's X-Trans filter strategy and set up the sampling in a pseudo-random fashion. That would make the aliasing less obvious and could actually be kindof cool and filmic.
I can alternate the exposures, but the postprocessing will be a lot more complex. It should help with highlight aliasing (because the shadow one is pretty much solved).
In some videos, the exposure pattern is moving, and in others it's fixed. I don't know yet why this happens.
I don't know how to alternate ISO for every single line - if you look in the paper, all I do is changing a register to low-high or high-low. The ADTG chip does the rest.
a1ex: In the dual ISO raw2dng interpolation, what ways are you looking at improving the accuracy of returned values and reducing artefacts? Ah I see you're already answering this.
Quote from: horshack on July 18, 2013, 04:44:12 PM
For the shadows the ISO 100 lines are being discarded from the composite raw, which reduces the light-capturing surface area of the sensor in half.
Thanks Horshack-- funny how the obvious sometimes escapes me. Although, b/c they're interpolating between the two ISO exposures, you can set up the algorithm to only use the ISO 1600 file where read noise trumps shot noise contributions significantly. But I see what you're saying -- when you do that, you pay a cost due to shot noise contributions (vs. exposing the whole image at ISO 1600, for example).
But typical 'engineering' DR calculations using SNR of 1 as the lowest signal don't consider shot noise, correct? Therefore, one could still say that the engineering DR is extended to FWC/read noise @ISO 1600, which is ~14EV. If you're using a higher SNR cutoff of, say, 20, then the DR is much more modest (considering the increase in shot noise contributions you mention).
I updated my original test to use the optimized version of RawToDNG (raw2dng_shadow050.exe). Aliasing is noticeably reduced now. Thanks Alex.
I'm finding it difficult to properly set exposure, though I'm sure this will improve as code revisions progress. This RAW converted with the Shadow 50 executable is particularly special.
(http://s2.postimg.org/dr71uroix/Untitled.png)
Sorry if I missed it...but....Is there a noob guide on how to implement this hack on the 7d? My guess is that you'll need the original Magic Lantern firmware and just copy and paste to some bin files..??? I know, I know...we're not worthy..but please, point us in the right direction..Thank You. :)
I was able to download the full-res dng file, and open it with GIMP 2.8, and didn't find much artifacts, and, when shrinked to 1920x1280, the picture was beautiful, but I couldn't get that much DR as the whitepaper did.
Couldn't wait to see when this good thing, plus raw video, plus 10 bit downbit raw video, will come to 7D. And I had a dream last night shooting that Dual-raw video with 7D. This is serious stuff.
I personally appreciate Alex's great job and his daring thought, which I think quite fun, and the whitepaper is really well written(though a little beyond me). However, I do notice that this setting only works with iso100, if it can be extended to other ISOs will be a genuine plus.
Quote from: jkdjedi on July 19, 2013, 03:30:04 AM
Sorry if I missed it...but....Is there a noob guide on how to implement this hack on the 7d? My guess is that you'll need the original Magic Lantern firmware and just copy and paste to some bin files..??? I know, I know...we're not worthy..but please, point us in the right direction..Thank You. :)
You'll either have to compile yourself, or wait for someone to release the updated autoexec.bin for 7D. You'll need both the dual_iso modules and the autoexec.bin
Quote from: dubzeebass on July 19, 2013, 03:05:07 AM
I'm finding it difficult to properly set exposure,
Set your base ISO for the highlights and the recovery ISO for the shadows.
Thanks Brapodam.. I guess I'll have to wait then... :( :'(
@dubzeebass: can you upload a sample? (less than 1 second of video, if possible).
Did it work better with a previous version?
Why can't the 5D Mark II get Dual ISO function, what the difference in 7D that allow it to function?
Hi guys,
to work with raw2dng_shadow050.exe on wine in osx,
when the terminal tells me "usage:
Z:\Users\Seba\Desktop\Raw\raw2dng_shadow050.exe file.raw [prefix]
=> will create prefix000000.dng, prefix0000001.dng and so on."
What should I do?
Thanks
Copy your RAW video file to a location you like. For you it will do in Z:\Users\Seba\Desktop\Raw\ and drag and drop this file onto raw2dng_shadow050.exe. After this step you will get a bunch of files in this very folder/directory for further processing.
Ciao, Walter
Quote from: Walter Schulz on July 19, 2013, 09:53:00 AM
Copy your RAW video file to a location you like. For you it will do in Z:\Users\Seba\Desktop\Raw\ and drag and drop this file onto raw2dng_shadow050.exe. After this step you will get a bunch of files in this very folder/directory for further processing.
Ciao, Walter
That works fine in Windows, I'm not sure about whether you can just use it like that if you have to run it using wine.
Normally what I'd do would be to run
Quotewine ./raw2dng_shadow050.exe file.raw
Of course replace file.raw with whatever the filename is.
If you want a prefix to your dng files then put the prefix after the filename
Thanks for the correction!
Ciao, Walter
Thanks Guys,.....perfect
Can,t wait for macversion converter so installed virtualbox and used Alex raw2dng converter. Some serious iso improvements here. Question. I get this famous pinkish cast when converting, how can I avoid it? Filmed in 3x crop mode but it shows otherwise as well.
Thanks!
//D
(http://s12.postimg.org/65736pk09/image.jpg) (http://postimg.org/image/65736pk09/)
Upload a short sample. I don't see it in any of my test files.
a1ex, have you tried sampling more than 1 frame (i.e. two) during the interpolation and if so were there no averaging improvements? I know it would be more taxing but Id be interested to know what your findings were.
How about frame x {1 second in} (settled) - do the math - return to frame 0 / begin?
I have to align the frames first, and since they contain motion, it's not trivial.
Sent you a pm with a short clip Alex. Sometimes 3x crop mode works fine with no pink cast, others it don,t. Hot pixels numbers skyrocket in the converter when pinkish. Normal filmings seems to work fine.
*update, just tried a regular 1920x1080 clip and pinkish unfortunately. Seems inconsistent on my camera. Or with the conversion.
Updated raw2dng.exe in the first post, seems to be OK now.
wow, that,s fast. Gonna try it out immedietaly
Did you change the raw2dng.exe? I still get the skyrocket high hot pixel numbers and pinkish cast as before.
Thanks
/D
*update, worked with my latest moviefiles. Will try in 3x mode an also do a comparison. Will post it later tonight
Thanks
//D
Works here. Cache issues when downloading?
Uploaded the same thing with another name: http://acoutts.com/a1ex/raw2dng_whitefix.exe
md5: 3af31ca290c8e4cb3e3013cf134a16f6
Nice Alex! Worked perfectly :)
I,ll post a comparison tonight.
Sweet
Is raw2dng_whitefix the latest raw2dng that handles dual iso raw files? Will it handle normal raw files as well?
yes.
I did a short test
the lighting conditions it was hardly
https://vimeo.com/70633755
ISO100 looks the best there. What's the difference between external and internal?
ISO 100 is better but the DR is much lower than dual ISO. Using ETTR: external exposure the shutter was 800 (ISO 100-F2.8) and internal exposure the shutter was 50 (ISO 100 F2.8).
What shutter speed did you use for dual ISO?
Quote from: sarangiman on July 18, 2013, 11:19:38 PM
But typical 'engineering' DR calculations using SNR of 1 as the lowest signal don't consider shot noise, correct? Therefore, one could still say that the engineering DR is extended to FWC/read noise @ISO 1600, which is ~14EV. If you're using a lower SNR cutoff of, say, 20, then the DR is much more modest (considering the increase in shot noise contributions you mention).
Right, SNR goes down but if one uses a 1:1 SNR (like DxO does) for DR calculations then then SNR reduction doesn't affect the DR number.
A little testing with the dual iso feature. Real cool stuff going on. Feel free to comment.
//D
Quote from: sebamax on July 19, 2013, 07:31:19 PM
ISO 100 is better but the DR is much lower than dual ISO. Using ETTR: external exposure the shutter was 800 (ISO 100-F2.8) and internal exposure the shutter was 50 (ISO 100 F2.8).
I don,t quite get this. Clearly you get the best results here in your movie "iso 100 raw expo internal ettr raw2dng".
Could you explain what your settings were exactly.
Thanks!
//D
Pixel peepers: which of these 3 would you choose?
http://acoutts.com/a1ex/bright1.dng
http://acoutts.com/a1ex/bright2.dng
http://acoutts.com/a1ex/bright3.dng
Both are ISO 1600 half interpolated.
bright3 better edge. check the curvature of the lamp. bright1 inconsistent. bright2 good balance.
Yep for me bright3 detail wins (see blind edges too). bright1 slightly better in highlight detail - see blind chord. (right click images into new window to analyse). bright2 somewhere in the middle.
(http://www.driftwoodproductions.co.uk/img/testresult.png)
(http://www.driftwoodproductions.co.uk/img/testresult2.png)
I think bright 1 is nice. The blinds seem smooth and the camera is less aliased than the others.
For the window, bright 1. Smoothest edges, least artifacting.
For everything else, bright 2. Bright 1 is a no show, to much aliasing.
Bright 3 has more artifacts. Check the difference between 2 and 3 on the full res samples in the areas I have highlighted below.
(https://dl.dropboxusercontent.com/u/34113196/ML/bright3.jpg)
Hi,
Thank to Audionut for supplying the binary install for the dual iso module. Uploaded and now sailing to go. Facing a challenge though and need desperate help. The raw2dng that I downloaded cannot be activated since windows 8 sees it as a threat if some sort and is locking it on my PC. Any idea how to deactivate the security to get the program running?
Second, how do I record stills with the dual iso module? Cold someone post the workflow?
Thank you again and to A1ex too.
Quote from: Danne on July 19, 2013, 10:37:26 PM
I don,t quite get this. Clearly you get the best results here in your movie "iso 100 raw expo internal ettr raw2dng".
Could you explain what your settings were exactly.
Thanks!
//D
Hi,
I wanted to try an extreme situation of light.
In the first shoot with the raw module and ETTR module, I pointed out of the window and ETTR suggested me ISO 100-shutter 1.000 F2.8. In the second shoot I pointed inside the room: ISO 100 shutter 60- F2.8.
With dual ISO module (ISO 100-1.600) with ETTR I pointed out of the window and it suggested me ISO 100 shutter 800-F2.8. I processed all in ACR using Highlight -100 and shadow +100.
With dual ISO module it was a little bit difficult using viewfinder to focus.
Pixel peepers: which of these two?
http://acoutts.com/a1ex/highlight/a.jpg
http://acoutts.com/a1ex/highlight/b.jpg
"a" seems more clear in the curtains
I agree 'a' seems to be better (crispier).
A1ex: How do I take still with the dual_iso functionality?
To take stills, press the shutter button, then read the first post. Picture quality should be RAW.
So... you all say it's better to have sharper false detail and aliasing artifacts?
To me, it seems better to have less artifacts, even if the highlights are softer. I've updated the converters with the new algorithm, it should print "mean23-vsmooth3". Can you run some tests?
Quote from: a1ex on July 20, 2013, 12:23:02 PM
To me, it seems better to have less artifacts, even if the highlights are softer.
100% Agreed.
Softer images, i can work with. Noticeable artefacts are a no go.
My vote for the smoother picture with less artefacts definitely.
I vote less artifacts.
My first Dual ISO production. Really am proud for having worked it out!
Link: https://www.dropbox.com/s/i51u04c44yb3t9w/first%20%20dyau%20iso%20video.mov
Quote from: Canon eos m on July 20, 2013, 02:55:20 PM
My first Dual ISO production. Really am proud for having worked it out!
Link: https://www.dropbox.com/s/i51u04c44yb3t9w/first%20%20dyau%20iso%20video.mov
A still photo using the dual iso functionality
https://www.dropbox.com/s/botulropqxbsjk9/SANDALS%20DUAL%20ISO%20MAGIC%20LANTERN.tif
Starting to get close to a usable settings. This was shot ISO 200-800 .... Exposure was pretty centred, to avoid too much highlight aliasing. Click for 1080p
(http://s18.postimg.org/gmtabf4d1/200_800.jpg) (http://postimg.org/image/gmtabf4d1/)
Yes, that's the ideal use case. Main subject covered by both ISOs (full detail), highlights and shadows as out-of-focus backgrounds (no aliasing).
yep... I did a rack focus and once the highlights come into focus, it gets a bit messy, but not too bad.
For now, i think protecting the highlights as much as possible is the best way to go, avoiding artefacts.... Then pulling out the deep shadows. Probably not getting quite such a DR boost, but it gives a more usable image.
yes - good to know better use cases may not be the extremes and
to consider the value (thematically and exposure wise) of what's in field of focus.
is raw2dng in unified supposed to be able to process dual iso clips?
tried 5bad65ff8cb8 and it isn't converting a dual shot clip - instead it creates banded DNGs.
easy enough to download a different version.
Yes, raw2dng from here is from unified.
Can you upload a short clip?
HDR clips are recognized from the first frame, by computing the ISO difference - so if that frame is dark, it probably won't work.
very underexposed - so good to know and explains why i have clips which when running raw2dng do not show the EV HDR correction info - though this is the first time i'm seeing the banded DNG.
uploading now - if you want to see - try these in a couple of minutes when upload is complete
http://50.56.67.113//ml-20july13/M20-0958.RAW 32.7MB
http://50.56.67.113//ml-20july13/000000.dng 2.7MB
A1ex, for video could you alternate 100<>1600 each frame and then de-interlace the highlights and shadows in order to address the aliasing? It could turn the extreme spatial artifact into a slight temporal one.
Yes, feel free to write down the math for deinterlacing. Alternating ISO is trivial.
Rather than filling line-skipped highlight/shadow pixels with synthesized values, fill them with values from the previous frame.
Continue until you get a precise mathematical description of what you said.
Hint: it's 100x more complex than what I did so far.
Gotcha. Just wondering what tricks could be used if ISOs were alternated.
Btw I've noticed in the past that when lifting RAW video shadows to the extreme, the noise "crawls" through the frame. I think it was diagonally down and to the left. It reminds me of how the alias patterns are crawling here. I don't understand why there is a barber pole effect during a locked-down shot. That's what I would expect if ISOs were alternating (without de-interlacing).
Another video ... I used the Shadow 050 one again and I think it gives the best results.
Is there any way to implement this in such a way that each sensor records individually to the CF card? Obviously it'd be double the data rate (ergo half the write speed per channel on the CF bus) but it could be a start?
https://www.youtube.com/watch?v=nGBRDbLeIgk
Quote from: a1ex on July 19, 2013, 08:36:41 AM
@dubzeebass: can you upload a sample? (less than 1 second of video, if possible).
Did it work better with a previous version?
a1ex, it appears to be just the frames at the beginning of the video that do that interlaced pink highlight thing. It's akin to the judder that some people get, myself included, when viewing a rendered RAW file - the first few frames repeat back and forth and then it continues normally.
Hi! Need some help what to do with my developed cr2 dual iso file. I throw the file on dcraw.exe and then to cr2hdr.exe which creates somekind of.ppm file. This opens fine in photoshop and it contains the info from the darker iso. The lines are still visible. Am I not supposed to get a combined file of the two iso,s?
Thank you!
//D
@dubzeebass: that was a bug solved a while ago, turn on sync beep if you still use an old version.
@Danne: first cr2hdr, then dcraw.
I have some major updates to the interpolation algorithm. It's too late here to post nice comparisons and stuff, but you can do that for me.
Basically, this update gets the noise levels from the original samples in out-of-focus areas and minimizes aliasing in high-detail areas.
http://acoutts.com/a1ex/shadow/000000.jpg
compare with http://acoutts.com/a1ex/shadow/b.jpg (shadow50) for example
or with http://acoutts.com/a1ex/highlight/b.jpg (highlight blurring)
or with http://acoutts.com/a1ex/shadow/median6.jpg (the first algorithm)
There's no denoising; the algorithm is just selecting the cleaner signal source (ISO 1600) in low-detail areas. There's some slight blurring in aliased highlight areas, but it has no impact on noise (it's only used to reduce the artifacts).
Now I think the algorithm is suitable for some really nice shots (photos+videos), not just aliased keyboards in black caves ;)
Quote from: a1ex on July 21, 2013, 12:33:22 AM
@dubzeebass: that was a bug solved a while ago, turn on sync beep if you still use an old version.
I use the latest build for 5Dmk3 from Lourenco, will turn on Beep and see if that sorts it out.
Quote from: a1ex on July 21, 2013, 12:33:22 AM
Now I think the algorithm is suitable for some really nice shots (photos+videos), not just aliased keyboards in black caves ;)
The new one looks way better. Still that chroma aliasing ... I wonder is it possible to pull the colour from surrounding pixels on anything that's got a jaggie and has an RGB level near clip while other levels are relatively low for that area? Might not work if something is supposed to be jaggie ... but could help with the chroma aliasing.
nice Alex, with the blurring!
Still didn,t get the cr2 to combine. I drop the the file first on cr2hdr wich opens up, than on dcraw, creates somekind of ppm-file. When viewed in 100% the horisontal lines are showing. Should I get a dng?
Thank you for your patience.
//D
*update. Got everything to work eventually. Thanks
a1ex can you link the cr2hdr executable separately to save some bandwidth.
Quote from: Danne on July 21, 2013, 12:54:25 AM
Still didn,t get the cr2 to combine. I drop the the file first on cr2hdr wich opens up,
You should get the DNG after dropping the CR2 onto the cr2dng app.
Hi audionut. Thanks for helping. cr2dng app? Do you mean cr2hdr? Can,t find any cr2dng
Quote from: Danne on July 21, 2013, 01:06:14 AM
Hi audionut. Thanks for helping. cr2dng app? Do you mean cr2hdr? Can,t find any cr2dng
Yes that's what he meant, available in the original post.
Got it to work. Had to move the whole package to the virtual box desktop. rather than using the virtual drive.
Thanks for your patience. THis is gonna be fun :)
Quote from: Danne on July 21, 2013, 01:31:27 AM
Got it to work. Had to move the whole package to the virtual box desktop. rather than using the virtual drive.
Thanks for your patience. THis is gonna be fun :)
Nice one.. I'm just getting my VM environment up and running again after failing with WINE and WINEBottler to get the EXE to run. Post some samples when you get em!
Hi Alex. Playing around with the smooth3 version. Is that the updated version of the combining tools?? With cr2 it seems to be a real winner. Havn,t done anything special yet but its wonderful to have such clean shadows. Will try to post something tomorrow. I,m away and on a real slow bandwith for a couple of days but I,ll try to post a picture.
Did you by the way update the conversion tools for film yet? I got some vertical pink stripes but probably due to an old version? Is whitefix still the latest raw2dng.exe version?
Thanks man.
//D
Quote from: a1ex on July 21, 2013, 12:33:22 AM
I have some major updates to the interpolation algorithm. It's too late here to post nice comparisons and stuff, but you can do that for me.
Basically, this update gets the noise levels from the original samples in out-of-focus areas and minimizes aliasing in high-detail areas.
http://acoutts.com/a1ex/shadow/000000.jpg
There's some aliasing on the coffee table that isn't in the shadow 50 shot. Can you post up a 100/400 shadow 50 shot for comparison. Now that you have the FPN under control the noise doesn't bother me, with a bit of chroma noise reduction it will look like grain.
That shot is just a tricky shot from EOSHD - here's his original rendering: http://www.eoshd.com/uploads/5d3-hdrx.jpg
I can remove the shadow aliasing completely, but right now I'm only cleaning 50% of it, because the noise gets pretty bad. Still lots of room for tweaking (but you need to compile yourself, there are no user parameters yet).
Here's the spot you need to change:
if (b0 < white && d < white)
f = MAX(f, c * FIXP_ONE / 2);
change to f = MAX(f, c * FIXP_ONE); to remove all the shadow aliasing, and maybe tweak CONTRAST_MAX to fine-tune stuff.
Quote from: a1ex on July 21, 2013, 07:00:03 AM
I can remove the shadow aliasing completely, but right now I'm only cleaning 50% of it, because the noise gets pretty bad. Still lots of room for tweaking (but you need to compile yourself, there are no user parameters yet).
Here's the spot you need to change:
if (b0 < white && d < white)
f = MAX(f, c * FIXP_ONE / 2);
change to f = MAX(f, c * FIXP_ONE); to remove all the shadow aliasing, and maybe tweak CONTRAST_MAX to fine-tune stuff.
I think that as soon as FPN is suppressed (and cr2dng does a beautiful job on this) then the optimal mix depends on the workflow. I suppose that with ACR (and it's very effective denoise in use) a user would be better to have a DNG with less aliasing at the cost of more noise and then denoise in ACR.
The same with Rawtherapee which has some nice tools for demosaic, denoise, false color suppression ...
With Resolve with it's primitive demosaic and denoise the needs are different.
User parameters would be a nice option.
Quote from: a1ex on July 21, 2013, 07:00:03 AM
(but you need to compile yourself, there are no user parameters yet).
Hehe, yeah like that's ever gonna happen. :D
Yeah noise is easily dealt with in ACR without much of a detail penalty, and besides if you want anything to look like film you still have to light. Samuel mentioned that 100/400 still gives you 13.5 stops and only the top and bottom 2 stops have aliasing vs top and bottom 4 with 100/1600. With only a half a stop DR penalty to substantially reduce the aliasing 100/400 looks like the best option so far.
Is shadow 50 100/1600 4+4 stops aliased?
Quote from: squig on July 21, 2013, 09:53:17 AM
.... Samuel mentioned that 100/400 still gives you 13.5 stops and only the top and bottom 2 stops have aliasing vs top and bottom 4 with 100/1600. With only a half a stop DR penalty to substantially reduce the aliasing 100/400 looks like the best option so far.
What you describe is for 100/800 not 100/400 .. http://www.magiclantern.fm/forum/index.php?topic=7139.msg59971#msg59971
And I also think 100/800 is optimal.
At 100/400 the DR increase is 1.25 less than 100/1600.
Hi Audionut could you please help with the latest binary of the revised module code. Do not know how to do it myself. Running the dual ISO module that you updated last.
If I got off my arse and compiled a 7D copy from the repo, I'd host it. I have no time for extra projects though and this would fall into that category. Hosting isn't a problem when I've pushed through 1 TB in 3 weeks off my home connection and Verizon hasn't cared...
How do you find which bits are iso and which bits are flag bits?
50D:
16C - 3200
16C - 1600
1FC - 1000
1B4 - 500 -320
94 - 250
04 100
10 11 0 1 1 00
11 11 1 1 1 00
11 01 1 0 1 00
01 00 1 0 1 00
1 00
Also FRAME_CMOS_ISO_SIZE
6D reg0 doesn't move... have to check the stubs:
iso 100 - 0
800 - 33
1600 -44
3200 -55
6400 - 77
128k - 77
256K - 77
/* 00:00:05.898461 */ 03 0000 40450E08
/* 00:00:25.234033 */ 03 0033 40450E3E
/* 00:00:55.186093 */ 03 0044 40450E50
/* 00:01:23.931307 */ 03 0055 40450E62
/* 00:01:39.858203 */ 03 0077 40450E74
11 0011
100 0100
101 0101
111 0111
Hi,
I took still photo mostly and didn't use ML before as I thought ML was for movie mostly.
I believe this update is amazing and drove me to try ML finally. So far my test did show some improvements in DR but depending on situations sometimes the difference are not very apparent.
I would also like to report some suspected bugs when using the Dual_ISO module for photo.
1. Auto ISO doesn't work well with relative setting of Recovery ISO. It will give very badly under-exposed photo (e.g. I usually set at +4EV but even setting to negative the symptom is the same.)
2. The "alternative frames only" option will fail for continuous shots and sometimes it seems just completely stop taking @ dual_ISO at all after a while.
3. The converted DNG always shows a WB of 5500K and tint=17 when I imported them into Lightroom, independent of the original WB at shot.
Regards,
Peter
1. Try a fixed recovery ISO (not relative).
2. It looks at file number (IMG_1234) and burst is not supported. Once you know that, it's completely predictable.
3. Check if exiftool is working (post a conversion log).
Using cr2hdr from today (as of https://bitbucket.org/hudson/magic-lantern/commits/00ea74a66945b9626e0fe53807232af831d62269) compiled on OSX, I get a crash on one particular file:
Input file : IMGL9830.CR2
Full size : 5920 x 3950
Active area : 5796 x 3870
Black borders : 124 left, 80 top
ISO pattern : BddB RGGB
Interpolation : mean23-vsmooth3-contrast
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0xaa5e3408
0x00006967 in hdr_interpolate () at cr2hdr.c:1126
1126 dark_smooth[x + y*w] = ev2raw[(raw2ev[a] + raw2ev[ b] + raw2ev[ b] + raw2ev[c]) / 4];
(gdb) p a
$1 = -30
Other files I've tried so far convert fine so I put the file up at https://www.dropbox.com/s/ruv13bn6ibkzm08/IMGL9830.CR2
For some reason, dcraw was writing a pixel value of 65506 (come on, for 14-bit data, the range is 0-16383...). This also breaks my calculations (even the robust statistics part). The image is read correctly though.
I've converted the shot by hacking the source code a bit, but before providing a fix, I want to understand exactly what's going on.
http://acoutts.com/a1ex/IMGL9830.DNG
http://acoutts.com/a1ex/IMGL9830.jpg
Quote from: a1ex on July 21, 2013, 10:19:11 PM
For some reason, dcraw was writing a pixel value of 65506 (come on, for 14-bit data, the range is 0-16383...). This also breaks my calculations (even the robust statistics part). The image is read correctly though.
I've converted the shot by hacking the source code a bit, but before providing a fix, I want to understand exactly what's going on.
http://acoutts.com/a1ex/IMGL9830.DNG
http://acoutts.com/a1ex/IMGL9830.jpg
Excellent, thank you!
Quote from: 1% on July 21, 2013, 03:24:12 PM
/* 00:00:05.898461 */ 03 0000 40450E08
/* 00:00:25.234033 */ 03 0033 40450E3E
/* 00:00:55.186093 */ 03 0044 40450E50
/* 00:01:23.931307 */ 03 0055 40450E62
/* 00:01:39.858203 */ 03 0077 40450E74
This one looks really good.
?_CMOS_ISO_START = 0x40450E08;
?_CMOS_ISO_COUNT = at least 7; // from ISO 100 to 6400
?_CMOS_ISO_SIZE = 18;
CMOS_ISO_BITS = 4;
CMOS_FLAG_BITS = 0;
CMOS_EXPECTED_FLAG = 0;
If it works, the theory about 8-channel readout is invalidated (the 6D has 4 channels afaik).
I get error 1 in photo & LV, reg not 0? So 7D + 5d3 = 8 channels and everything else = 4?
Quote from: a1ex on July 21, 2013, 09:16:13 PM
1. Try a fixed recovery ISO (not relative).
2. It looks at file number (IMG_1234) and burst is not supported. Once you know that, it's completely predictable.
3. Check if exiftool is working (post a conversion log).
Thanks.
1. Yes, autoISO with fixed recovery ISO or fixed ISO with relative ISO both okay. Just to report the combo I find not working.
2. IC. Thx.
3. I am using Mac OS and I compile cr2hdr myself. I found that the my exiftool is a perl interface to Image::ExifTool. There is no error message from stdout of cr2hdr, e.g.
Input file : 5D3_6476.CR2
Full size : 5920 x 3950
Active area : 5796 x 3870
Black borders : 124 left, 80 top
ISO pattern : dBBd RGGB
Interpolation : mean23-vsmooth3-contrast
ISO difference : 2.97 EV (784)
Black delta : 2
ISO overlap : 6.0 EV (approx)
Hot pixels : 1055692
Output file : 5D3_6476.DNG
1 image files updated
For conversion log, do you mean those tmp.txt at conversion?
I will check if exiftool really work later.
Quote from: 1% on July 22, 2013, 01:34:56 AM
I get error 1 in photo & LV, reg not 0?
From the dumps, it's register 3, right? (I didn't use the original dump format, but some custom hooks)
Try overriding it manually (just change that memory address so it reads 0x3 instead of 0x33, for example).
Some comparison results with cr2 files. No question you get much cleaner shadows with less noise in dual iso. It,s pretty amazing to say the least. Done with "smooth" version. I think I used 100-1600 in dual iso. Can,t remember but I think I even went 100-3200 on another example and got nice results on this particular view.
(http://s7.postimg.org/hk7ssvzwr/IMG_5713dngorig.jpg)
dng original(dual iso)
(http://s18.postimg.org/ypkzg6dl5/IMG_5713dng.jpg)
dng dual iso processed in lightroom
(http://s17.postimg.org/82m70avr3/IMG_5711.jpg)
regular cr2 processed in lightroom (not dual iso)
(http://s21.postimg.org/kix56r993/IMG_5713_2.jpg)
dng pressed shadow (dual iso)
(http://s13.postimg.org/68x56wxxj/IMG_5711_3.jpg)
cr2 regular pressed shadow
Here,s an example where I went between I believe 100-3200 or even further. I,ll post a exif from the conversion app which show the ev settings. It,s like "turning on the lights"
(http://s18.postimg.org/5694vcqq1/IMG_5716_2.jpg)
dng original(dual iso)
(http://s8.postimg.org/4xbiiam1x/IMG_5716.jpg)
dng processed in lightroom(dual iso)
(http://s23.postimg.org/hx5z8h80r/IMG_5716_4.jpg)
Crop 1, not very sharp due to handshake but clean!(dual iso)
(http://s15.postimg.org/uofag6dkb/IMG_5716_3.jpg)
Crop 2(dual iso)
(http://s21.postimg.org/6kgioe0t3/Ska_rmavbild_2013_07_22_kl_01_51_47.png)
ev settings
Dual ISO works well, great new function. Problem I'm having, which may be normal, is that when I activate Dual ISO the display is really "wavy", almost like the scan lines from out of sync artificial lights, but worse.
Would like to know if this is to be expected or is there a problem with my set up specifically?
Read the first post. All info there
ISO 100/800
(https://dl.dropboxusercontent.com/u/34113196/ML/D46A8690.jpg)
Base ISO 100 shot with same post processing.
(https://dl.dropboxusercontent.com/u/34113196/ML/D46A8686.jpg)
@Danne: try processing the same shots with the latest cr2hdr. You should get much cleaner shadows (so I shouldn't have to read the text to know which is which ;) )
In particular, the black spots should no longer be there.
Hi Audionut could you please load the binary for the latest dual USO module. I am currently using the one you created last.
a1ex maintains an updated cr2hdr binary in the first post. (http://www.magiclantern.fm/forum/index.php?topic=7139.msg59687#msg59687)
There doesn't appear to be any updates to the module since my last build. Only the cr2hdr binary.
Two examples with crops in extreme dual iso, 100-6400. As you can see it works fairly well and it blows the native cr2 miles, miles away. Even tried a 100-12800 iso but had to much magenta issues in certain mid-whites. Settings processed the same in lightroom for both the crg and dual iso dng. Had to drag the exposure slider all the way to the right with the dual iso-file. Probably could have gone a little further but lightroom stopped me from trying ;).
(http://s15.postimg.org/6uyu70fu3/IMG_5760_64k_3.jpg)
dng 100-6400 original
(http://s12.postimg.org/ljv8h7xtp/IMG_5760_64k.jpg)
dng 100-6400 processed
(http://s9.postimg.org/7z0ytnp5r/IMG_5760_64k_2.jpg)
dng 100-6400 cropped
(http://s23.postimg.org/g1nu4jx3v/IMG_5763_3.jpg)
cr2 iso 100 original
(http://s18.postimg.org/p6fzbj4g9/IMG_5763.jpg)
cr2 iso 100 processed
(http://s18.postimg.org/hy8xzzh3d/IMG_5763_2.jpg)
cr2 iso 100 crop
@Alex. Appears to be a new hdr converter from Alex. I will reconvert the pics in time. I,m on the countryside and slooow bandwidth. Today uploads took me two hour to complete.
Amazing work Alex.
Quote from: Danne on July 22, 2013, 02:41:51 PM
Had to drag the exposure slider all the way to the right with the dual iso-file. Probably could have gone a little further but lightroom stopped me from trying ;).
Using the brush tool you can apply layered effects that would exceed 100%. You may already know this though. :)
It seems a great idea!
which FW I've to load for double_iso on my 5DMK3?
Highlight problem with last raw2dng.
(http://dcmakecreative.altervista.org/ML/DNG000000.jpg)
DNG file
http://dcmakecreative.altervista.org/ML/DNG000000.dng (http://dcmakecreative.altervista.org/ML/DNG000000.dng)
Quote from: a1ex on July 20, 2013, 04:39:47 PM
Yes, feel free to write down the math for deinterlacing. Alternating ISO is trivial.
I now realize that alternating ISOs would likely affect midtones and cause them to shimmer as well if not de-interlaced. I suppose that simplifies the problem because the whole frame would need to be de-interlaced. So without having an alternating ISO video to play with, I tried an experiment in Premiere to see what it looks like to temporally smear video one frame. I did it by copying a set of clips, layering them on top of the originals, shift by one frame, and set opacity to 50%. The goal was for the output be the average of every two input frames.
The result looks good. Although there are temporal artifacts reminiscent of ML HDR video, static areas are crisp and actually cleaner because noise gets averaged down.
So the question is, couldn't such averaging of alternating dual ISO frames, whether performed by RAW2DNG or manually in an editor, treat the aliasing issue?
I'd be happy to experiment manually if I could generate some alternating ISO samples.
@JackDaniel412: record 0.5 seconds of video showing the problem, and upload the RAW file. The DNG is processed and doesn't help.
@stevefal: this patch should alternate ISO exactly the way you suggested: http://acoutts.com/a1ex/alt_iso.patch
You can play with it, see if you get improvements, and I'll probably add it as an option in menu.
Quote from: Danne on July 22, 2013, 02:41:51 PM
Two examples with crops in extreme dual iso, 100-6400.
Very impressive! Could you provide the 100-6400 iso dng for download. I'd like to try for myself. I need to decide whether I buy a 5D3...
I,ll give it ago later. Will try to find a faster upload spot :)
Quote from: a1ex on July 22, 2013, 05:47:24 PM
this patch should alternate ISO exactly the way you suggested: http://acoutts.com/a1ex/alt_iso.patch
You can play with it, see if you get improvements, and I'll probably add it as an option in menu.
I'm not set up to build - could never get my Windows setup to work. Also, I'm still stumped by the crawling aliasing artifacts in dual-ISO video. It looks like they are on a three frame cycle. Do these have to crawl? Even if the artifacts were flipping due to alternating ISO, my two-frame blending wouldn't work as long as the beats are crawling that way in the first place.
I'm not yet sure when crawling occurs; it appeared in one test shot from Luke Neumann, but it didn't appear in all of my other test shots...
Maybe it has to do with FPS, PAL/NTSC or something like that.
I'm not sure if we're talking about the same thing. I haven't seen one dual-ISO video where the alias beats aren't moving - self-crawling moire.
Doesn't it have something to do with the BBdd, dBBd, ddBB cycle I see when converting? Both are three frame cycles.
Yes, you see it in the bbDD pattern. For me, only that one clip from Luke Neumann is crawling; everything recorded by me, and the Batman clip (also from Luke Neumann) are stationary. The clip from EOSHD is also stationary.
Ok, I see a few that don't crawl. But everything I shoot crawls like crazy, just like the one above.
I'm shooting 1920x818, 23.976, C1, no FPS override, non-zoom.
Quote from: Digital Corpus on July 21, 2013, 12:08:46 PM
If I got off my arse and compiled a 7D copy from the repo, I'd host it. I have no time for extra projects though and this would fall into that category. Hosting isn't a problem when I've pushed through 1 TB in 3 weeks off my home connection and Verizon hasn't cared...
Please, if you can make a compile version for the 7D. Want to trie :-\
Another with crawl. Switch to 1080p and look at the bottom of the fence towards the end:
Small converter update:
- fix crashes in cr2hdr with some unusual pics
- fix red "zebras"
- nicer highlight handling (aliasing artifacts should be blurred)
- minor tweaks
Quote from: a1ex on July 22, 2013, 11:33:28 PM
- nicer highlight handling (aliasing artifacts should be blurred)
Old vs New.
(https://dl.dropboxusercontent.com/u/34113196/ML/old.jpg)(https://dl.dropboxusercontent.com/u/34113196/ML/new.jpg)
A1ex - I understand that the dual functionality is in development stages and hence most of the stuff will be things only developers can play with or comprehend. I am not a techie but try to keep up.
Have the raw2dng figured out thanks to Mixer2 and 1% from their contributions on the EOS M forum. Mixer2 has us really spoilt with his video casts and detailed explainations and steps.
Have the dual iso working on my 5D Mark III. Loving it.
Struggling to understand a major step. If I want to record in 100 /800 iso which one should I set on camera and which on the ML interface?
I first tried 100 on the camera and 800 on ML. But get better results if I do it the other way around. Is that the way this is supposed to work? Doing it the 800/100 way gives me better histogram accuracy and highlight /shadow feedback while recording.
Quote from: stevefal on July 22, 2013, 10:38:10 PM
Another with crawl. Switch to 1080p and look at the bottom of the fence towards the end:
Hahaha, that's my video ... and my wife ... and my backyard! #winning
Quote from: a1ex on July 22, 2013, 11:33:28 PM
Small converter update:
- fix crashes in cr2hdr with some unusual pics
- fix red "zebras"
- nicer highlight handling (aliasing artifacts should be blurred)
- minor tweaks
Is the EXE posted in the OP kept up to date?
Quote
I first tried 100 on the camera and 800 on ML. But get better results if I do it the other way around. Is that the way this is supposed to work? Doing it the 800/100 way gives me better histogram accuracy and highlight /shadow feedback while recording.
Is there any difference in the end result? Can you post a comparison?
a1ex and stevefal... Are you guys talking about one frame, having line 1,3,5,7... at iso 100 and 2,4,6,8... at iso 1600, then on the next frame switching them?
if this is the case, couldn't you shoot at 60fps and and in post production get a 30 fps average of 2 frames, eliminating the moire?
let me know if i'm way off the mark here.
Also has anybody compiled the 7D binary yet?
-Ben
Quote from: a1ex on July 23, 2013, 06:28:46 AM
Is there any difference in the end result? Can you post a comparison?
To my eye there was a difference - both in output and the way the camera reacted. Unless, by the time I swapped the ISO allocation, I had become better at playing around with this stuff and therefore began to imagine there was a difference. Will post the comparison tonight / tomorrow since I will have to look for a suitably contrastry situation to drive home the point.
By the way, I pointed the camara directly at the sun today and was able to get some awesome DR using the dual iso interface. The camera without the dual iso module activate threw some lovely blue clouds but a very shadowy foreground.
Will post the results tonight.
Converted with the latest converter from Alex. Better sharpening algorithm which works best for most dualiso stuff. When converting the 100-6400 I get some artefacts pronunciation with the latest converter. Probably from the image itself.THere are som white minimal dots from the roompicture. You can look for yourself. Probably gonna use the older converter for bigger iso intervals and for most other shots using the new one.
Crap, I,m n the woods in the middle of the sun. Can,t get the pics right :). One last try
(http://s2.postimg.org/vcfws3t7d/IMG_5716.jpg)
original
(http://s2.postimg.org/xv1lssexl/IMG_5716new.jpg)
new converter
(http://s2.postimg.org/kgoiwr89l/IMG_5716old.jpg)
old converter
(http://s2.postimg.org/46yctuxll/IMG_5760b_3.jpg)
(http://s2.postimg.org/dc6pnq109/IMG_5760b_new.jpg)
new converter
(http://s2.postimg.org/6pk1ujjbt/IMG_5760b_old.jpg)
old converter
(http://s2.postimg.org/ig2kpo1i1/IMG_5760new.jpg)
new converter
(http://s2.postimg.org/62pqirbtl/IMG_5760old.jpg)
old converter
The old converter from here was different from the one used for the previous set (http://www.magiclantern.fm/forum/index.php?topic=7139.msg60936#msg60936) (the shadow artifacts are no longer there). Was it median6 (the very first version)?
I don't see any quality loss with the new converter. If anything, the sharpness was set too high when postprocessing the picture.
I'd also try these DNGs with this script (http://www.magiclantern.fm/forum/index.php?topic=7022). Would be nice if you could upload one, maybe when you come back.
Can you do now a comparison between ISO 100 and ISO 100/1600?
Hi Alex! I see what I can do. Used the latest converter from the zipfile in the first post. Soon getting back to civilisation to do some more comparison. Gettin on it asap, probably tomorrow
This update should be pretty close to the ideal auto ISO option - for my needs.
https://bitbucket.org/hudson/magic-lantern/commits/373d8b5b0e69
You expose for highlights, it takes care of the shadows. It's pretty much impossible to do the other way, because you can't figure out how much a picture is overexposed if you look at the histogram.
New conversion tool from Alex works very good. Check out the difference and also the possibilities that the dualiso module gives.
Happy peeping!
Feel free to download the dual-iso cr2 rawfiles here. Both of them in 100-1600iso
https://docs.google.com/file/d/0B4tCJMlOYfirVWtwbDNpSXBMM2c/edit?usp=sharing
(http://s23.postimg.org/4ms54criz/IMG_5714_iso800full.jpg)
full res
(http://s23.postimg.org/twmwrfyaj/IMG_5715_iso1600old.jpg)
old converter
(http://s23.postimg.org/wbesbvejf/IMG_5715_iso1600new.jpg)
new converter
(http://s22.postimg.org/cvzqpigdt/IMG_5760_iso6400full.jpg)
full res
(http://s22.postimg.org/z97hcbhbl/IMG_5762_1600old.jpg)
old converter
(http://s22.postimg.org/tmb4euesx/IMG_5762_iso1600new.jpg)
new converter
hello I have a 5d mark 3 and want to know how do I install dual iso, video tutorial please.
Quote from: daancalo2013 on July 23, 2013, 09:07:16 PM
hello I have a 5d mark 3 and want to know how do I install dual iso, video tutorial please.
- It is a good idea, help us. Por favor chicos ayudarnos ?(http://tomhanks.chez.com/film16.jpg)
If you are incapable of searching google on how to install ML, you clearly aren't ready for it.
Quote from: daancalo2013 on July 23, 2013, 09:07:16 PM
hello I have a 5d mark 3 and want to know how do I install dual iso, video tutorial please.
I'm not going to make a video tutorial because I don't feel it's needed, plus I don't have a 5D3.
Anyways (5D3 users please correct me if I'm wrong, I don't have a 5D3).
1. Follow the instructions in this thread to install ML http://www.magiclantern.fm/forum/index.php?topic=2602.0
2. After you're done doing all those stuff, download everything here: https://www.dropbox.com/sh/xiksvwbehzmycq4/QbuV17WuE5 (I just linked this from the first post in this thread)
3. Replace autoexec.bin on your SD card with the one you downloaded in step 2
4. Create a folder under the ML folder on the SD card, and call it "modules" (without the quotation marks)
5. Copy everything else you downloaded from step 2 into that folder
There are some video tutorials here too http://www.magiclantern.fm/forum/index.php?topic=6099.0
I've updated the dropbox folder with the latest build.
@a1ex, with auto shadow recovery enabled in photo LV, ML menus become laggy.
It is looking WAY BETTER a1ex! I am really impressed with the quality... however the vertical line problem is back. I've edited the picture below for clarity and if you look closely at the video in 1080p you'll see the lines in the upper-right corner. Image full-size is here: http://postimg.org/image/j93dgsinh/
(http://s22.postimg.org/j93dgsink/Converted000105.jpg)
I am just BLOWN AWAY by the extra Dynamic Range for still photography. Wow!
(http://s22.postimg.org/aavomq7ap/MG_3448.jpg)
Quote from: Audionut on July 24, 2013, 02:05:05 AM
@a1ex, with auto shadow recovery enabled in photo LV, ML menus become laggy.
Audionut I just downloaded the builds you posted and I can't find the auto shadow recovery under the Dual ISO menu. Any idea?
It only works in photo LV. In dual ISO, hit set on recovery ISO and scroll all the way to bottom.
One more ... this time comparing 100 to 6400 and then combined. This was purposefully a very taxing image to capture with two light sources, diagonal lines, and very dark and very light parts of the image.
(http://s24.postimg.org/p0icqwj1f/MG_3455_2.jpg)
It can't be stressed enough that this development is groundbreaking and frankly I think a1ex for prez!
So is there a difference in dual iso images for still photography vs DNGs?
@dubzeebass: for banding, you started the video with a very underexposed image. Banding correction is computed from the first frame only, so if you start from the sky, it should be a lot better.
You may try something really extreme now... maybe a photo from a cave outside? or maybe something like this (http://www.youtube.com/watch?v=fjtGenTSfX8) or this (http://vimeo.com/36863089)?
Quote from: a1ex on July 24, 2013, 08:14:40 AM
you started the video with a very underexposed image
What's the best method to ensure both ISOs are picking up the maximum signal without clipping? Histogram I guess? I was using the spot meter (%) but I am thinking that may not be good enough.
Set your base ISO for highlights (Zebras/Histogram/AutoETTR), and enable the needed recovery (Histogram/Zebras).
Quote from: Audionut on July 24, 2013, 04:00:20 PM
Set your base ISO for highlights (Zebras/Histogram/AutoETTR), and enable the needed recovery (Histogram/Zebras).
Cheers. I notice 3 colours in the zebras: white, green, and red. Red is obv overexposed, white is under ... what does green signify? Back on topic after I promise!
#a1exforpres
The still images are starting to looking really nice... a bit of edge fringing, but usable
http://postimg.org/image/5s5d2fkx5/full/
did you try to defringe chromatic abberations in lightroom or another application? I got rid of most of it. It,s a marvellous thing, dual-iso
Yea, that did a very good job of clearing it up. Resized down to 4K (and 2K), it looks pretty much perfect.
http://postimg.org/image/ot17d1ytd/full/
would anyone have an archive of the cr2hdr?
The one on first page isn't working with the test pictures I took yesterday. It complains it's not an interlaced file.
Upload a sample.
Quote from: fcpk on July 25, 2013, 12:00:01 AM
would anyone have an archive of the cr2hdr?
The one on first page isn't working with the test pictures I took yesterday. It complains it's not an interlaced file.
The "not interlaced" is related to cr2 files that,s not photographed with dual iso I believe. Try a dual-iso file, works splendid
I'm pretty sure it is a dual iso, there's clear interlacing in the raw when I preview it, with two different ISO visible
The dynamic range looks good, but the samples I have seen look pixelated and I wonder what-then is the point of using it at all...
Quote from: AnotherDave on July 25, 2013, 01:26:38 AM
The dynamic range looks good, but the samples I have seen look pixelated and I wonder what-then is the point of using it at all...
It's a work in progress ... and has already improved by leaps and bounds. Oh ye of little faith.
Quote from: dubzeebass on July 24, 2013, 06:00:43 PM
Cheers. I notice 3 colours in the zebras: white, green, and red. Red is obv overexposed, white is under ... what does green signify?
Dotted white is underexposed. Each other color represents over exposure in that channel. Black represents over exposure in all channels.
I've uploaded a 7D build.
See here: http://www.magiclantern.fm/forum/index.php?topic=7139.msg61920#msg61920
Hi guys, ive installed the dual iso distro on 7/19 and, for some clips i am getting this error in the raw2dng conversion:: "Bright/dark detection error"
if (is_bright[0] + is_bright[1] + is_bright[2] + is_bright[3] != 2)
{
printf("Bright/dark detection error\n");
return 0;
}
Looking at the code it seems that the algorithm finds clipped brights in both ISOs and returns with an error, but im just guessing.
Is there something i can do to save those videos since raw2dng is leaving the frames "interlaced"?
Thanks!
Quote from: Audionut on July 25, 2013, 03:00:39 AM
I've uploaded a 7D build here (https://www.dropbox.com/sh/yp6dl9i104i1j2g/9Ta_9074YP).
I have no idea if it works, or if it will make your camera explode. I merely built the source.
I don't even know which modules will work. Obviously Dual_ISO should.
No warranty is implied or otherwise. If your house burns down, you're on your own.
No warranty noted.
How do I install this? The latest 7D Alpha 2 has no modules.
Quote from: feureau on July 25, 2013, 04:50:22 AM
No warranty noted.
How do I install this? The latest 7D Alpha 2 has no modules.
Create a module folder inside the ML folder. Copy everything except the autoexec.bin file to the module folder.
There doesn't appear to be a devkit released for the 7D. I'm pretty sure you might need this first, and that's out of my hands. See here: http://www.magiclantern.fm/forum/index.php?topic=3974.msg61662#msg61662
Quote from: mucher on July 25, 2013, 04:58:45 AM
I guess that I am one of those "luna"s. I load Audionut's autoexec.bin with Alpha 2 into 7D, nothing happens. I mean really nothing, the ML didn't load into camera. I download everything in Audionut's 7D build directory and put them all in CF card's root directory. And start up the camera again. Nothing happens, again. I don't visually see that my camera is damaged neither.
@a1ex
@1%
@g3gg0
It's not possible to do nothing for we trie the Dual ISO on the 7D ?
Dual ISO - massive dynamic range improvement for 5D3 and 7D
I am having an awesome time doing after dark street photography with the new dual iso stuff.
Struggling a bit with video though since the files are heavy and rendering takes a lot of time. I have developed a sort of work around to get a better preview of the files before these are sent to raw2dng.
I bascially, first secure a preview on rawnaizar.exe. This tells me the files I want to work with first. The ones that I mark out, I copy to my harddisk and breakout in raw2dng_whitefix.exe. Then the files are directly sent to AE CS6 to render.
A few photo samples shot yesterday and processed exclusively on ACR 8.1:
http://www.flickr.com/photos/89977905@N05/9360108468/
http://www.flickr.com/photos/89977905@N05/9360245550/
http://www.flickr.com/photos/89977905@N05/9357757481/
http://www.flickr.com/photos/89977905@N05/9360352442/
http://www.flickr.com/photos/89977905@N05/9357878171/
Must have shot the Mapletree signage a million time since it is on my way back home from office. Nothing special ever came out of it till I met Mr. Dual Iso. Rest is history as they say.
I am really proud of the red cup picture. This was a transparent discarded plastic glass lying on a bed of fallen leaves. There was not much reflected light. Mr. Dual Iso came to my rescure. I then exposed the picture in ACR 8.1 and coloured the glass red so that it could standout. ACR 8.1 is now capable of much more with Mr. Dual Iso coming to its aid. They should now call this ACR 801.1 or something since the new functionality is a game changer. Thanks to all developers and a special bow of respect to A1ex.
Tell me what you folks think.
Could someone please teach me how to upload the picture directly on the magiclantern forum rather than pasting a link.
[img]http://farm6.staticflickr.com/5510/9360108468_2404c0c2e8_h.jpg[/img]
Ciao, Walter
You will need to link a thumbnail so that the embeded image size is less then 900 pixels as per the forum rules (http://www.magiclantern.fm/forum/index.php?topic=934.msg1141#msg1141).
Any further discussion about image linking should go in this thread (http://www.magiclantern.fm/forum/index.php?topic=6061.0) please.
Quote from: Audionut on July 25, 2013, 07:55:41 AM
You will need to link a thumbnail so that the embeded image size is less then 900 pixels as per the forum rules (http://www.magiclantern.fm/forum/index.php?topic=934.msg1141#msg1141).
Any further discussion about image linking should go in this thread (http://www.magiclantern.fm/forum/index.php?topic=6061.0) please.
Thanks everyone. Audionet noted.
This shot was taken keeping the bright sun in the frame. Post processed in ACR 8.1.
(http://www.flickr.com/photos/89977905@N05/9359793024/)
(https://www.dropbox.com/s/qe12wdg58plz2qv/saisha%20balcony%20west%20bay.mp4?m)
@Alex. Trying to understand dual iso and movie mode. WHen taking pictures I get noisefree cr2 files however when filming with the exact same settings I can,t get the same results after conversion. Am I doing something wrong? Providing a link to the movie clip and some comparisons.
Settings on camera was manual mode shutter 1/50 dual iso 200-6400. Magic lantern 22nd juli build
Link movie 0,5s
https://docs.google.com/file/d/0B4tCJMlOYfirQmtHenZNbTNkVU0/edit?usp=sharing
example still
(http://s10.postimg.org/ah2kydy55/IMG_5849.jpg)
(http://s10.postimg.org/i5zfa3ymx/test3still.png)
example movie
(http://s8.postimg.org/64wxusz5h/000000.jpg)
(http://s8.postimg.org/xqzpfhiid/test3movie.png)
@Alex. Found out what was going on sort of. By reversing the dual iso from 100-6400 to 6400-100 the conversion tool "gets it right" in moviemode. I believe it does some kind of inverted conversion right now. A bug? Tested a short clip with magic lantern from 16th juli, same problem.
example 200-6400 (regular)
(http://s8.postimg.org/rrshpavhx/000000.jpg)
example 6400-200 (reversed)
(http://s22.postimg.org/ejc8si169/000000.jpg)
By reversing, do you mean you set the camera's ISO to 6400 and the ML ISO to 200?
Or something in the raw2dng settings?
I set the camera to the higher iso and then proceed to the dual-iso settings. There I see to it that the higher iso is to the left and the lower to the right. One way or the other, If the other one does,nt work try reversed :)
raw2dng.exe stop extracting frames at frame 493 with all raw files i had tested.
anyones noticed that?
Quote from: Canon eos m on July 23, 2013, 03:42:13 AM
A1ex - I understand that the dual functionality is in development stages and hence most of the stuff will be things only developers can play with or comprehend. I am not a techie but try to keep up.
Have the raw2dng figured out thanks to Mixer2 and 1% from their contributions on the EOS M forum. Mixer2 has us really spoilt with his video casts and detailed explainations and steps.
Have the dual iso working on my 5D Mark III. Loving it.
Struggling to understand a major step. If I want to record in 100 /800 iso which one should I set on camera and which on the ML interface?
I first tried 100 on the camera and 800 on ML. But get better results if I do it the other way around. Is that the way this is supposed to work? Doing it the 800/100 way gives me better histogram accuracy and highlight /shadow feedback while recording.
I agree with Danne. Precisely my observation. Pointed this to A1ex. He asked for samples but could not produce any since I was travelling.
I try 3200/100 ISO with zebra raw on (raw2dg_whitefix)
I noticed:
24p http://www.youtube.com/watch?v=p2Q24us23w8
25p http://www.youtube.com/watch?v=_JG2RwWjTG4
What did you notice? Why didn,t you film in 1920x1080p?
Hi,
in 24p more flicker
in 25p no flicker
I filmed in 1920X1080
Ok, I see. Interesting.
How did you compile this? I'm using gcc and getting all sorts of errors?
Quote from: ap1hk on July 22, 2013, 03:28:31 AM
Thanks.
1. Yes, autoISO with fixed recovery ISO or fixed ISO with relative ISO both okay. Just to report the combo I find not working.
2. IC. Thx.
3. I am using Mac OS and I compile cr2hdr myself. I found that the my exiftool is a perl interface to Image::ExifTool. There is no error message from stdout of cr2hdr, e.g.
Input file : 5D3_6476.CR2
Full size : 5920 x 3950
Active area : 5796 x 3870
Black borders : 124 left, 80 top
ISO pattern : dBBd RGGB
Interpolation : mean23-vsmooth3-contrast
ISO difference : 2.97 EV (784)
Black delta : 2
ISO overlap : 6.0 EV (approx)
Hot pixels : 1055692
Output file : 5D3_6476.DNG
1 image files updated
For conversion log, do you mean those tmp.txt at conversion?
I will check if exiftool really work later.
One more comparison. This time between raw and dual-iso in movie-mode. two extracted dng,s from raw and dual-iso raw-moviefiles. Even if dual-iso benefits most from high contrast scenes there is still one or two steps better dynamic range with dual-iso even in more evenly lit situations. Pretty amazing since regular raw is already amazingly good.
(http://s16.postimg.org/arjkc4jd1/1600.jpg)
Original raw 1600 iso
(http://s16.postimg.org/9drxgtk3p/6400_1600.jpg)
Original dual-iso 6400-1600 iso
(http://s16.postimg.org/do6lcep6t/1600processed.jpg)
Original raw processed in lightroom
(http://s16.postimg.org/5eupxzvgl/6400_1600processed.jpg)
Original dual-iso processed in lightroom
(http://s16.postimg.org/87nxi0vt1/isoraw.jpg)
Cropped orignal raw flattened to show iso-grain
(http://s16.postimg.org/4a0nsm8zp/isodualiso.jpg)
Cropped original dual-iso to show iso-grain
@ Danne,
Can you upload the RAWs of this last comparison please.
I think something is badly wrong with the single-ISO sample.
Quote from: Danne on July 26, 2013, 12:50:59 AM
One more comparison. This time between raw and dual-iso in movie-mode. two extracted dng,s from raw and dual-iso raw-moviefiles. Even if dual-iso benefits most from high contrast scenes there is still one or two steps better dynamic range with dual-iso even in more evenly lit situations. Pretty amazing since regular raw is already amazingly good.
Danne, I cannot spot the difference between the single & dual iso versions except in the last 2 crops. There has to be something I am missing. Could you please Dropbox the unprocessed RAW's.
There is hardly any visible difference in the first pictures as you can see which is a good thing. It is when you start to push the shadows out that you start to see a subtle difference, This gets more visible when you shoot in contrasty situations but there is still a difference, slightly.
The movie raws are about 350mb each, to big to upload from where I sit atm. You can play around with two extracted dng-files if you like :). Link below.
https://docs.google.com/file/d/0B4tCJMlOYfird3RMV0l5c2c0RzQ/edit?usp=sharing
@Ilias G. I don,t see anything wrong with the single iso-file?
I've managed to hack the source to work with 7D.
edit: removed, to many problems.
Don't bug the devs about my build. I've created it for the sole reason of people with 7D's being able to test this feature. It comes with no warranty or support (but I'll help where I can).
Quote from: Danne on July 26, 2013, 06:46:52 AM
There is hardly any visible difference in the first pictures as you can see which is a good thing. It is when you start to push the shadows out that you start to see a subtle difference, This gets more visible when you shoot in contrasty situations but there is still a difference, slightly.
The movie raws are about 350mb each, to big to upload from where I sit atm. You can play around with two extracted dng-files if you like :). Link below.
https://docs.google.com/file/d/0B4tCJMlOYfirTTRETlpaQnZVMG8/edit?usp=sharing
@Ilias G. I don,t see anything wrong with the single iso-file?
Danne, it's amazing, thanks. Video is getting better and better (whitefix?)... Only some blur in highlights so it seems to begin to be pretty usable now. Is that interpolation becomes less noticiable with smaller interval (i.e 400-100)?
Have not done any serious filming yet but it could be nice in certain situations. Nighttime, citylights, maybe real estate filming getting the highligts from windows and so on. When taking stills it works really well.
Certainly is a huge step taking stills! On the other hand, i think the real revolution comes with (pretty) usable dual iso video function -which from what i see is taking giant steps in fews days-
Looking forward to see more dual iso clips (no that agressive interval ones?) 8)
Quote from: Danne on July 26, 2013, 10:19:48 AM
...maybe real estate filming getting the highligts from windows and so on. When taking stills it works really well.
I also see amazing serious cinema 14-stops DR beautifully rendered imagery... ;)
Quote from: Danne on July 22, 2013, 02:41:51 PM
When I made this comparison with 100-6400 and regular 100 iso one realize there has to be more shadows regained than 2 extra stops. By dividing in such a big interval i,d argue that you suddenly gained about 4-5 extra steps. I was able to lift the shadows to the extent I couldn,t go any further with the lightroom exposure slider. Could have used the brushes though but still.
These shots are even converted with the old converter.
(http://s15.postimg.org/6uyu70fu3/IMG_5760_64k_3.jpg)
dng 100-6400 original
(http://s12.postimg.org/ljv8h7xtp/IMG_5760_64k.jpg)
dng 100-6400 processed
(http://s9.postimg.org/7z0ytnp5r/IMG_5760_64k_2.jpg)
dng 100-6400 cropped
(http://s23.postimg.org/g1nu4jx3v/IMG_5763_3.jpg)
cr2 iso 100 original
(http://s18.postimg.org/p6fzbj4g9/IMG_5763.jpg)
cr2 iso 100 processed
(http://s18.postimg.org/hy8xzzh3d/IMG_5763_2.jpg)
cr2 iso 100 crop
There is a working build for the 7D in the 7D alpha2 topic.
The produced files (ISO 100/1600 and ISO 1600/100):
http://pel.hu/down/100_1600_7D.CR2
http://pel.hu/down/100_1600_7D.DNG
http://pel.hu/down/1600_100_7D.CR2
http://pel.hu/down/1600_100_7D.DNG
Last raw2dng.exe stops at frames 493 for all my footages with the error : mallocxels.
look at this :
(http://img442.imageshack.us/img442/4373/245y.png)
i have downloaded raw2dng.exe many times i was thinking it was a browser cache issue but is not...
any ideas ? please !
Quote from: Danne on July 26, 2013, 06:46:52 AM
The movie raws are about 350mb each, to big to upload from where I sit atm. You can play around with two extracted dng-files if you like :). Link below.
https://docs.google.com/file/d/0B4tCJMlOYfirTTRETlpaQnZVMG8/edit?usp=sharing
@Ilias G. I don,t see anything wrong with the single iso-file?
Thanks for the samples.
The problem I see in your 200% crop of the single ISO file is a magenta cast at the darks of the sculpture and some clipping. Which is much less present with my conversions using Rawtherapee.
What was the lighting during the shots ? I have a hard time finding a correct WB.
Was it consistent in color and intensity ?. Because if it was a candle we cannot come at a valid conclusion comparing only two single frames.
How can you whitebalance rawmovies. Rawmovie converted with latest raw2dng.exe. Dngs provided in the link. The test was aimed for noisecomparison. Thanks for watching :)
Hi everyone,
somewhere here in this thread I read that the 5D3 produces less noise in dark areas with a higher ISO, therefore the dark areas are recovered from high ISO while light areas are recovered from low ISO.
I did some tests today and filmed the same scene (without Dual ISO) in ISO steps from 100 to 3200 and adjusted the exposure accordingly.
After running the DNGs through ACR in AE I can't see this effect. The lowest ISO still has the least noise in recovered dark areas. Starting at ISO 800 there's a noticeable shift towards magenta in the shadows.
In other words: I don't understand the Dual ISO. What did I miss?
As I see it lowest ISO is best for highlights and for shadows.
In the sample images you see the background in dark shadow under a roof, front in sun light.
(https://dl.dropboxusercontent.com/u/5752966/Permanent/ML/Bildschirmfoto%202013-07-26%20um%2013.13.20.png)
ISO 100
(https://dl.dropboxusercontent.com/u/5752966/Permanent/ML/Bildschirmfoto%202013-07-26%20um%2013.13.35.png)
ISO 800
(https://dl.dropboxusercontent.com/u/5752966/Permanent/ML/Bildschirmfoto%202013-07-26%20um%2013.13.42.png)
ISO 3200
Hi Yanone, I think the point is missed here. In normal circumstances (read 10 bit and below situations) you should use the plain RAW mode for still and video. Extreme situations (read 11 bit and above) require extreme treatment. This is what Dual-ISO is all about. Faced with a high Dynamic Range situation (facing the sun, bright reflection), it is better to use this functionality to capture and recover most of the shadows and highlights that would otherwise be lost forever. It is just a special tool to be used in special situations.
Quote from: yanone on July 26, 2013, 01:24:56 PM
Hi everyone,
somewhere here in this thread I read that the 5D3 produces less noise in dark areas with a higher ISO, therefore the dark areas are recovered from high ISO while light areas are recovered from low ISO.
I did some tests today and filmed the same scene (without Dual ISO) in ISO steps from 100 to 3200 and adjusted the exposure accordingly.
In other words: I don't understand the Dual ISO. What did I miss?
As I see it lowest ISO is best for highlights and for shadows.
(Faulty step in bold).
You missed the fact that to take the effect of cleaner darks with hi ISO you have to keep the exposure constant between shots .. and exactly is what dual-ISO does ..
IliasG could you elaborate the concept in some more detail.
Quote from: Danne on July 26, 2013, 01:19:09 PM
How can you whitebalance rawmovies. Rawmovie converted with latest raw2dng.exe. Dngs provided in the link. The test was aimed for noisecomparison. Thanks for watching :)
To make fair noise comparisons we have to compare shots at the same exposure (light inconsistency can give faulty conclusions) and the compared images been at the same contrast, saturation, WB and detail levels ...
Your uploaded dual ISO DNG is named 6400-100 .. wasn't your shot at 6400-1600 ??
Can you declare what lighting you used ??.
And upload the first frames of each session if possible, I would like to have them because the Black Level data is calculated on the start of each session.
Wrote wrong. Dual iso was shot 6400-1600, regular shot 1600, same lighting, same settings on the camera same processing in lightroom. Could upload the first dual iso dng later today. The other dng is the first one I believe? I used a lightbulb
Quote from: Canon eos m on July 26, 2013, 02:13:36 PM
Hi Yanone, I think the point is missed here. In normal circumstances (read 10 bit and below situations) you should use the plain RAW mode for still and video. Extreme situations (read 11 bit and above) require extreme treatment. This is what Dual-ISO is all about. Faced with a high Dynamic Range situation (facing the sun, bright reflection), it is better to use this functionality to capture and recover most of the shadows and highlights that would otherwise be lost forever. It is just a special tool to be used in special situations.
Right. I do consider my sample scene an extreme lighting situation. Front in bright sunlight with reflections, background in dark shadow. The background used to be almost black in the undeveloped DNG, front used to be very bright.
Quote from: IliasG on July 26, 2013, 02:26:53 PM
You missed the fact that to take the effect of cleaner darks with hi ISO you have to keep the exposure constant between shots .. and exactly is what dual-ISO does ..
Hmm, I see. Now I'm curious to actually try the Dual ISO. If I get a noticable difference, I'll post an update.
Quote from: Canon eos m on July 26, 2013, 02:51:46 PM
IliasG could you elaborate the concept in some more detail.
Alex explains it in http://acoutts.com/a1ex/dual_iso.pdf page 1-2 and gives some links ..
http://www.guillermoluijk.com/article/nonoise/index_en.htm (it's a dead link for the moment, let's hope for recovery)
http://www.dpreview.com/forums/post/28749589
http://www.dpreview.com/forums/post/28750076
The explanation for the decreased of noise at the dark areas at hi ISOs is that there is electronic noise at the late stages (transfer lines, DAC) after ISO amplification which noise is constant. If we feed the DAC with low signal (ISO100) lets say the noise is equal to this signal (SNR 1.0). If we feed it with amplified signal (ISO 1600) then the signal is 16 times stronger while the noise remains constant so SNR is 16.0 .. This with an ideal sensor and first stage electronics with no noise. In practice there is first stage noise also (but lower than late stage) so the SNR becomes around 8.0 and we gain 3 stops DR instead of 4 ..
Here's the scene with Dual ISO 100/1600.
Indeed, the dark background looks fantastic. But it's about the only detail that looks nice in this picture. (Same workflow: raw2dng.exe (July 22nd) -> AE -> ACR) :-\
(https://dl.dropboxusercontent.com/u/5752966/Permanent/ML/Bildschirmfoto%202013-07-26%20um%2015.43.53.png)
Ouch! :(
@Ilias
updated my link now containing the first dngfile from both of the examples. base-iso 1600 for both files. 6400-1600 on the dual-iso. (reversed due to converting issue, still base-iso 1600 though). Lighting and camerasettings identical for both shots.
https://docs.google.com/file/d/0B4tCJMlOYfird3RMV0l5c2c0RzQ/edit?usp=sharing
//D
Quote from: Pelican on July 26, 2013, 11:52:58 AM
There is a working build for the 7D in the 7D alpha2 topic.
The produced files (ISO 100/1600 and ISO 1600/100):
http://pel.hu/down/100_1600_7D.CR2
http://pel.hu/down/100_1600_7D.DNG
http://pel.hu/down/1600_100_7D.CR2
http://pel.hu/down/1600_100_7D.DNG
THANKS Pelican !!! U ROCK :D
Quote from: Danne on July 26, 2013, 05:58:53 PM
@Ilias
updated my link now containing the first dngfile from both of the examples. base-iso 1600 for both files. 6400-1600 on the dual-iso. (reversed due to converting issue, still base-iso 1600 though). Lighting and camerasettings identical for both shots.
https://docs.google.com/file/d/0B4tCJMlOYfird3RMV0l5c2c0RzQ/edit?usp=sharing
//D
Do you even gain any sort of benefit doing dual iso with recovery ISOs above 1600 (APS-C, 5D2) / 3200 (5D3+) - also known as the point where the read noise no longer decreases?
The theory suggests that 100/[200,400,800,1600] only make sense, and 100/3200 only if you've got a 5D3.
check out my picture of a 100-6400 some posts up. Worked rather well.
Thanks for the samples. :)
Quote from: Danne on July 26, 2013, 07:19:02 PM
check out my picture of a 100-6400 some posts up. Worked rather well.
It did worked.
But the point is that the difference between 100-3200 vs 100-6400 is very small in noise quality (0.09 stops in favor of 100-6400) while the difference in artifacts and resolution - detail is significant in favor of 100-3200.
Keep in mind that the result you see using 100-6400 is not only due to the cleaner ISO 6400 but also due to better processing by cr2hdr (black point calibration, channel matching ..) and a kind of denoise ( the data averaging by median filters) which get activated even when the ISO range is small.
You will need tests 100-6400 vs 100-3200 vs 100-1600 vs 100-800 (or 1600-3200 vs 1600-6400) combined with the appropriate for each case denoise by your raw converter to have useful data for a correct strategy for dual-ISO shots.
Of course one can spend a whole week comparing. Actually tried 3200 1600 and 12800. 100-6400 worked best. Fully aware of the algorithms with the converter creating the hdr and its importance. My tests so far are a blend of pseudoscience combined with vacation and just having fun ;)
By the way Ilias, did you look at the noisetest dng,s?
One small issue I noticed with the converter is that some tags, noticeably GPS coordinates, don't get transferred from CR2 to DNG, but adding -all:all to the exiftool command line fixed that for me:
diff -r 728e571a1276 modules/dual_iso/cr2hdr.c
--- a/modules/dual_iso/cr2hdr.c Wed Jul 24 16:58:18 2013 +0200
+++ b/modules/dual_iso/cr2hdr.c Fri Jul 26 22:42:56 2013 +0300
@@ -247,7 +247,7 @@
raw_info.white_level /= 4;
char exif_cmd[100];
- snprintf(exif_cmd, sizeof(exif_cmd), "exiftool -tagsFromFile \"%s\" \"%s\" -overwrite_original", filename, out_filename);
+ snprintf(exif_cmd, sizeof(exif_cmd), "exiftool -tagsFromFile \"%s\" -all:all \"%s\" -overwrite_original", filename, out_filename);
int r = system(exif_cmd);
if (r != 0)
printf("Exiftool didn't work\n");
Quote from: Danne on July 26, 2013, 09:25:47 PM
..
By the way Ilias, did you look at the noisetest dng,s?
Yes, and in this comparison the single ISO shot matches better the Dual ISO one. The difference is smaller than with the first pair, no weird casts etc.
@ Alex
Do you consider instead of calculating "black data" in camera just save the side optically black pixels for better calculations later with raw2dng ?. For these black data 12bit are enough as it is now and if there is a possibility to subtract the black offset 8bit will be OK (for BL at 2048 subtract 2048-128= 1920 so 256 will cover the range +/- 128).
Hi A1ex!
Your warning about use this method is strong than others advices. I will start a big project - movie - And I need to know wich is the real problem about work with this - I did some shots and no problem yet. So, it's Temperature? Card? Nikon people watching Canon Dual Iso quality?
;)
@A1ex
I try the dual iso on the 7D but i take 1 or 2 pictures and show up the #70 error and didn't record any file to the card. Any ideas or solution for this?
Thanks for all, it bring a new breeze to the 7D owners.
Quote from: arrinkiiii on July 27, 2013, 11:06:04 PM
@A1ex
I try the dual iso on the 7D but i take 1 or 2 pictures and show up the #70 error and didn't record any file to the card. Any ideas or solution for this?
Quote from: Audionut on July 26, 2013, 08:56:18 AM
Don't bug the devs about my build. I've created it for the sole reason of people with 7D's being able to test this feature. It comes with no warranty or support (but I'll help where I can).
I don't know why some are getting errors. I would suggest resetting the camera settings for a start, only shooting raw, no liveview shooting and only using base ISO's (100, 200, 400, 800, etc).
Quote from: Audionut on July 28, 2013, 02:19:56 AM
I don't know why some are getting errors. I would suggest resetting the camera settings for a start, only shooting raw, no liveview shooting and only using base ISO's (100, 200, 400, 800, etc).
Using fresh installs from pelican's EOSCard, and no live view: my 7D crashed at ISO 800 and 400 and 100. The crash at ISO 100 is worst. The camera just died and removing and reinserting battery wouldn't revive it. Left it batteryless for a few minutes. Works again.
It would shoot a frame or two, then I could keep firing the shutter, but nothing gets written to card.
None of the crashes produces crash log though. :(
Quote from: Audionut on July 28, 2013, 02:19:56 AM
I don't know why some are getting errors. I would suggest resetting the camera settings for a start, only shooting raw, no liveview shooting and only using base ISO's (100, 200, 400, 800, etc).
Sorry Audionut, thanks for the alert and for the build :D
If my words don't belong were feel free to delete or to move to the post of the 7D (new ports).
In short words,
-My last picture that i take just freeze de camera, only the red light stay on (forever, evan if i turn of the camera) and in the upper scrren of the camera it's say "busy". I toke the battery off. Its possible to be a format card problem?
-Wend i manage to take a photo in dual iso, i review the photo in camera and i make a zoom to the maximum that the camera can and i see only in the screen/photo lines. Do i need to convert this photo? Or is some error?
Thanks one more time Audionut.
Just checkout the dynamic range of the photograph (must be more than 14bit after some post processing):
http://www.flickr.com/photos/89977905@N05/9379336159/lightbox/
However, the RAW@DNG_Whitefix is causing some issues in the initial frames in video especially when the iso ratio is high (especially 100-6400 and higher). The alternating horizontal lines between the low and high iso show up even after rendering. Any idea what I am doing wrong with the workflow.
An interesting article on the cost of going RAW for full lenght features (120 minutes of the final product):
http://www.redsharknews.com/post/item/885-preparing-to-shoot-raw?page=all
Quote from: kinematicdigit on July 25, 2013, 08:48:17 PM
How did you compile this? I'm using gcc and getting all sorts of errors?
I downloaded the entire ML source and then go to the modules directory and compile the cr2hdr alone by "make cr2hdr".
It seems to work but I am not 100% sure. I compared with the result of wine cr2hdr.exe and didn't see much differences in the end picture so I assume this is going to work. I don't like using wine unless really necessary.
@ap1hk. See you got something going on mac. Why not share it? A lot of guys out there in desperate need of a mac conversion tool :)
//D
Quote from: Danne on July 28, 2013, 11:53:49 AM
@ap1hk. See you got something going on mac. Why not share it? A lot of guys out there in desperate need of a mac conversion tool :)
//D
I am not sure if my compilation of cr2hdr is 100% okay. :-\
And cr2hdr is just a small issue but to get dcraw (and maybe exiftool as well) working on Mac, one has to install either Macport or Homebrew which actually could be very troublesome. However if one was using wine already then probably they have one of these unix ports installed.
Anyway, I don't mind sharing it if anyone wants it. :)
@Audionut
Can you upload again the files in the dropbox (its empy) for the 7D users that solve the problem of crashing ?
Thank you :D
I already manage to get yours last build and install but still crash with the 70error after a few photos and the pink/magenta still there wend use the cr2hdr.
What we can do?
(7D)
I think that pink highlights is a result of incorrect processing of clipped channels in some cases by cr2hdr.
"--force" option is needed because cr2hdr does not always detects that cr2 is interlaced. Option for reversed order of fields needed too.
One more comparison. No noise reduction (only "Remove dot noise" option), RPP. Aperture, shutter speed and processing settings are the same. Unbelievable clear shadows. (7D)
https://dl.dropboxusercontent.com/u/4624786/dual-iso_test2.1.jpg
Quote from: xmd5a on July 28, 2013, 09:16:16 PM
I think that pink highlights is a result of incorrect processing of clipped channels in some cases by cr2hdr.
"--force" option is needed because cr2hdr does not always detects that cr2 is interlaced. Option for reversed order of fields needed too.
One more comparison. No noise reduction (only "Remove dot noise" option), RPP. Aperture, shutter speed and processing settings are the same. Unbelievable clear shadows. (7D)
https://dl.dropboxusercontent.com/u/4624786/dual-iso_test2.1.jpg
Its possible, with me is the same problem. In camera and computer the preview of the cr2 of the photo seems to be ok, then wend use cr2hdr its just appear the pink/magenta in the highlights... In my case i think its not clipped channels, wend i toke my photos i have the preoccupation to not clip (overexpose) the photos but still appear the magenta/pink color in the highlights wed use the cr2hdr.
Alex - how about 5D mkII is any chance to get dual iso on that? :)
Some random examples of dual iso cr2 images processed in lightroom. Nothing to compare to but the original exposure.
(http://s21.postimg.org/7eggror3b/IMG_6003.jpg)
(http://s21.postimg.org/dqvm1iu5j/IMG_6003_2.jpg)
(http://s21.postimg.org/z4hn6d5bb/IMG_6010.jpg)
(http://s21.postimg.org/spe0vy97r/IMG_6010_2.jpg)
(http://s21.postimg.org/9o96g6pev/IMG_6105.jpg)
(http://s21.postimg.org/v9e93sm5j/IMG_6105_2.jpg)
(http://s21.postimg.org/jzlj8uh47/IMG_6106.jpg)
(http://s21.postimg.org/qfuhyxpnr/IMG_6106_2.jpg)
(http://s21.postimg.org/9e1nwuasn/IMG_6128.jpg)
(http://s21.postimg.org/8ebd14vmv/IMG_6128_2.jpg)
I don't suppose this will work in 'Auto' movie mode will it?
really amazing!
and I wonder if the imagine could be as sharp as the original or not..
Hey Danne, thanks for these; look awesome. Quick question: are they downscaled video frames or just full res pics? Also, which ISO interval on these? (artifacts are minimal)
thnx
Hi Alex. They,re fullres pics. Good stuff :)
Feature request: can we have any Dual-ISO pictures prefixed with some sort of indicator so that it's easier to sort? I took 400 pictures this weekend, 37 of them were Dual-ISO and it was a pain to go through and pull out the original CR2s. If they're prefixed and sorted by Name, it'd be easier cuz it'd go PREFIX-FILENAME.CR2, PREFIX-FILENAME.DNG, etc.
Just a thought.
Quote from: Danne on July 29, 2013, 02:05:46 PM
Hi Alex. They,re fullres pics. Good stuff :)
I see. Thanks ;)
Folks, any idea how to recover the exif data of a photo taken using dual_iso. Want to see what settings was the camera using when I took the shot.
You may have noticed the filmy grain look (as opposed to noise on a DSLR) when shooting with dual_iso. There are situations that don't need the dual functionality since the DR is not too high but I still cannot get myself to not using dial_iso. I just love the results. ACR 8.1 handles RAW so much better when shot with multiple iso.
Any update on if you can shoot 'auto' video exposure and dual iso raw?
Quote from: Danne on July 26, 2013, 07:19:02 PM
check out my picture of a 100-6400 some posts up. Worked rather well.
First post.. So please bear with me if I seem dumb.
But Danne I'm very impressed with your pictures! Would you like to share how managed to render them? I've been trying for a few days now almost full time to get some good pics out of this Dual ISO function, but always get the horrid pink magenta highlights from cr2hdr and even some green stuff when under exposed, don't know what I'm doing wrong.. Did you use cr2hdr? Or how did you process your images can't wait to get a convenient workflow for this amazing function. And can't thank everyone working with it enough!!
Yup, used cr2hdr. In the nightpictures there is some green and red moiré or something. Only minor though. In the daylight pictures almost none. When filming a lot more green/red issues appear. Mostly 100-1600 or 100-3200 used in the pics. Clean the pictures in lightroom to get rid of purple fringing
Quote from: Pelican on July 26, 2013, 11:52:58 AM
There is a working build for the 7D in the 7D alpha2 topic.
The produced files (ISO 100/1600 and ISO 1600/100):
http://pel.hu/down/100_1600_7D.CR2
http://pel.hu/down/100_1600_7D.DNG
http://pel.hu/down/1600_100_7D.CR2
http://pel.hu/down/1600_100_7D.DNG
In these files, the white level is close to 13000; in my 7D test sample it's 15382. Normally you get these values with intermediate ISOs (160, 320).
If you change the white level in cr2hdr.c to 13000 instead of 15000, these files will be converted without pink highlights. Now trying to figure out how to get this value from exif or what settings to use to get raw values higher than 15000.
Updated the converters (download from first post):
cr2hdr:
- fix memory leak, tested with 100 GB of pictures
- white level autodetection (should fix pink highlights)
- exiftool fix from jpaana, not tested but looks OK
raw2dng:
- fix memory leak, not tested (should be OK with large clips)
I did not address the black level issues yet.
If you are having issues with image quality, please upload the raw files (*.CR2 or *.RAW).
Some more examples of what dual iso can achieve. The shadows pop out like never before. Mostly use interval between 100-1600 and then use ETTR to find exposure settings. Works perfect most of the time. From a visit in Copenhagen.
Due to bandwith reasons and uploaded images being too big i decided to post a youtubelink containing the stills instead. Enjoy.
Quote from: Danne on July 30, 2013, 08:52:56 PM
Some more examples of what dual iso can achieve. The shadows pop out like never before. Mostly use interval between 100-1600 and then use ETTR to find exposure settings. Works perfect most of the time.
It's incredible!
Sorry if this is a noob question, but what is ETTR...? I was actually going to ask you how you know what exposure to do before.
@a1ex thanks a thousand for your hard work!!! How can I send you a cookie? :)
ETTR is a great function that require some testing out. It,s in the menu. You can search on the forum on how to work with it.
I just hit ETTR on and then proceed to dual iso where I set the interval lets say 100-1600 or 100-3200. WHen I hit the set button the mirror flips and the ETTR is making its magic (calculating exposure to the right). Don,t be surprised if the iso and shutter changes. I believe the intervals stays the same though. Not sure. Than take the picture.
You could always do it manually by selecting the first iso for the highlights and then the other iso for shadows. If there,s a lot of contrast usually between 100-1600 or bigger works nice. Even 100-6400 could work sometimes.
Yup, it,s kind of magical. Feels like another camera atm :)
@Magic 7D: Next time use the search function, please.
http://www.magiclantern.fm/forum/index.php?topic=5693.0
Ciao, Walter
Quote from: a1ex on July 30, 2013, 04:37:20 PM
In these files, the white level is close to 13000; in my 7D test sample it's 15382. Normally you get these values with intermediate ISOs (160, 320).
If you change the white level in cr2hdr.c to 13000 instead of 15000, these files will be converted without pink highlights. Now trying to figure out how to get this value from exif or what settings to use to get raw values higher than 15000.
Some Canon models (I think all APS-C with 18Mp) have lower white levels for ISO 100 than the rest ISOs. For 7D it's around 13500 for ISO 100 and 15300 for the rest "integer ISOs".
It would be very useful if you find a way to declare this from *.CR2 and it's exif :)
The sad thing is that some channels (B and G2 in my samples) have raw values higher that this white clipping point making the detection difficult even if we have burned areas ..
dcraw uses 13584 (hardcoded). And yes, since the white points changes with ISOs, it makes things difficult; for now, I just use the smallest value from burned areas, if it's above 10000. Probably a bit overkill, maybe 13000 should be a better starting point?
It would be fantastic to see some video examples of this. Perhaps indoors with windows to see how it can compensate for blown windows/doors.
Also, could someone try recording in auto exposure movie mode and report back it it works with dual ISO - I will be forever grateful!
Right now, auto exposure won't work in raw video, because the digital ISO gain is not applied to raw data.
But if you include the post deflicker algorithm in raw2dng, you may even get something usable. Worth trying IMO.
Quote from: a1ex on July 30, 2013, 09:36:33 PM
dcraw uses 13584 (hardcoded). And yes, since the white points changes with ISOs, it makes things difficult; for now, I just use the smallest value from burned areas, if it's above 10000. Probably a bit overkill, maybe 13000 should be a better starting point?
Dcraw is ineffective regarding WP, it uses a single value for every ISO and runs into the known problem, either pink highlights (ISO 100-160-320-...) or 1/3 stop burned highlights.
The best solution is to have a lookup table with the correct clipping point for every case. Keep in mind that we have data scaling (and higher WP) even when shooting at wide apertures (wider than f/2.8) with Canon glass ..
Adobe DNG converter uses 13000 as WP for 100-160-320 .. and 15000 for 200-400-800 .. For 250-500... I think it goes up to 16000.
But takes no care for wide apertures so in this case up to 1/3 stop highlights get lost ..
I do think that if you use a single WP then 13000 is a safe bet. Although you loose 1/3 stops of highlights for ISO 200-400-.. and maybe more for 250-500.. and when using wide apertures.
13584 is on the limits for ISOs 100-160-320-.. because when there is raw denoise (like "black frame subtraction") then we take not a single clipping point but a distribution extending -/+ 100 raw levels from the center value .. it's the same case as in Guillermo's study .. http://www.guillermoluijk.com/tutorial/satlevel/index.htm
Well, I'd use 13000 or higher. If there are no burned areas, the only side effect should be picture being a little brighter than normal, and maybe a few clipped pixels (and highlight recovery should catch them). If there are burned areas, it will use the true white level.
Quote from: a1ex on July 30, 2013, 05:35:34 PM
Updated the converters (download from first post):
cr2hdr:
- fix memory leak, tested with 100 GB of pictures
- white level autodetection (should fix pink highlights)
- exiftool fix from jpaana, not tested but looks OK
raw2dng:
- fix memory leak, not tested (should be OK with large clips)
I did not address the black level issues yet.
If you are having issues with image quality, please upload the raw files (*.CR2 or *.RAW).
The new cr2hdr works very good, no more pink highlights in the photos taking with the 7D. Tomorrow will take decent photos.
Thank you a1ex :D
Btw, is there a way to make a drag-and-drop a bunch of cr2 to convert to DNR using cr2hdr? (batch convert)
On Linux I run cr2hdr *.CR2
No idea if this works on Windows, but try it.
Tried a little quick and dirty test filming in 3x crop mode with a 14mm samyang lens. I used 6400-100 setting in a very dark kitchen. Surprisingly I can,t find any moiré or aliasing. The picture is totally clean! Check out the movielink. Genious work.
Also a link to my still pictures made with dual iso on youtube.
Quote from: feureau on July 31, 2013, 08:43:08 AM
Btw, is there a way to make a drag-and-drop a bunch of cr2 to convert to DNR using cr2hdr? (batch convert)
Drag and drop a bunch of them on the executable.
Quote from: Audionut on July 31, 2013, 11:34:20 AM
Drag and drop a bunch of them on the executable.
Thank yoU! :D
These are not extreme examples, but show practical use for me at the track. 100-800 ISO.
(http://trackshotsphotos.smugmug.com/photos/i-6wwwBjv/0/L/i-6wwwBjv-L.jpg)
(http://trackshotsphotos.smugmug.com/photos/i-kDp4MHm/0/L/i-kDp4MHm-L.jpg)
(http://trackshotsphotos.smugmug.com/photos/i-gvHxBdq/0/L/i-gvHxBdq-L.jpg)
(http://trackshotsphotos.smugmug.com/photos/i-k9BGZ4V/0/L/i-k9BGZ4V-L.jpg)
It was a big help for this shot.
(http://trackshotsphotos.smugmug.com/photos/i-m88f7Q9/0/L/i-m88f7Q9-L.jpg)
(http://trackshotsphotos.smugmug.com/photos/i-zVsdvWm/0/L/i-zVsdvWm-L.jpg)
Nice shots! Is it me or does it seem these "merged" shots seem kinda unrealistic? Or is it because we have rarely seen this. Looks like HDR. What's the difference?
Quote from: a1ex on August 01, 2013, 10:17:08 AM
Would be very nice if you can upload some DNG files (or CR2 if you still notice aliasing artifacts).
Aliasing seems under control these days. Mainly false colors.
(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/falsecolor.jpg)
There is a little aliasing on his left elbow (https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/D46A9993.CR2).
@Audionut. Is it green fringing? Could it be removed in lighroom?
Yeah LR does remove it. It requires fairly heavy defringing though which can cause artifacts.
I see. Nice work, Looking forward to more examples using this great function :)
@Alex. I uploaded three cr2 with smaller issues on them. I don,t expect anything, think it works good as is. Provided in the link three cr2-files and a word-document with issuedescription.
https://docs.google.com/file/d/0B4tCJMlOYfirc1NZbVhJR2tMNHM/edit?usp=sharing
Thanks Alex
//D
It's a bird? It's a aeroplane? It's a bulldozer? NO! Its A1ex and the rest of the ML gang ;D
BIG THANK YOU for fixing the 70error for the 7D.
-7D IS BREATHING AGAIN -
Quick test for sharpness: 100% middle crop of 7d with 17-55/2.8. 17mm @ 2.8
No dual iso (iso 100):
(http://img713.imageshack.us/img713/5523/w7ij.jpg) (http://imageshack.us/photo/my-images/713/w7ij.jpg/)
Uploaded with ImageShack.us (http://imageshack.us)
Dual iso 100/1600:
(http://img843.imageshack.us/img843/3016/ennf.jpg) (http://imageshack.us/photo/my-images/843/ennf.jpg/)
Uploaded with ImageShack.us (http://imageshack.us)
cr2 and raw files: https://www.dropbox.com/sh/tw3lo4kblurabkn/zjZGCfABR5 (https://www.dropbox.com/sh/tw3lo4kblurabkn/zjZGCfABR5)
That's why you should try to cover your main subject with both ISOs, and keep the extra range for highlight/shadow recovery (e.g. detail in clouds).
Quote from: a1ex on August 01, 2013, 03:15:36 PM
That's why you should try to cover your main subject with both ISOs, and keep the extra range for highlight/shadow recovery (e.g. detail in clouds).
I think in this situation this wasn't the case. Light was even on the desk with no over or under exposure.
I think the cr2hdr needs a little bit of tweaking to leave alone the mids, and work with only highlights and shadows.
Or maybe its exposure problem. What is av mode exposing for, 100 iso? Shouldn't I expose for around 640?
I wonder if the posts about how it looks too "HDR" are just because we're not used to this sort of latitude coming from the camera. I was watching Ray Donovan last night and also True Blood a couple of nights ago and I paused a couple of times where there was a lot of DR in the scene and it frankly looks like HDR.
Both the Alexa and film shoot at 14 stops and pausing such an image looks like the HDR pictures from the camera. Just my $0.02!
-Ben
Quote from: pit3k on August 01, 2013, 03:29:25 PM
I think the cr2hdr needs a little bit of tweaking to leave alone the mids, and work with only highlights and shadows.
Did you read my paper?
Quote from: a1ex on August 01, 2013, 05:35:08 PM
Did you read my paper?
Now I have, my bad mate. Your work is amazing.
It would be cool if the latest dynamic range features could be combined to provide a better and more convenient high-bit-depth still image. This maybe a blue-sky idea, but oh well.
First to clarify a term, "HDR" is casually used to describe two different things: 1) image data with high bit-depth (16+ bit) per channel, and 2) an image that was captured with high bit-depth and then compressed (curves, microcontrast etc), to reveal detail on low-contrast devices, sometimes with garish results - hence, "that looks unrealistic, like HDR".
Savvy photographers want the latitude of high bit-depth images in order to create the best possible result in post. So in-camera "HDR" features that generate an 8-bit JPEG from multiple exposures defeat the purpose.
Bracketing discreet still shots for the purpose of creating high bit-depth negatives in post is better, but the drawback is artifacts due to motion between shots.
So the question is, would it be possible to improve on the dual-exposure model by creating a new ML High-Range mode that does the following:
- shoot through viewfinder (ideally) a la still mode
- two iso-bracketed exposures during one shutter actuation - a la rawrec and miracle-happens-here
- two shots are combined in-camera - mids from first and highlights/shadows from second - kinda a la dual-iso modulo a1ex genius
- save 16-bit DNG that opens directly in ACR
This approach would have some drawbacks:
- lower still burst rate
- highlight/shadow artifacts due to motion
and advantages:
- pristine IQ for still scenes - no aliasing
- less severe motion artifacts than dual-shutter bracketing
- no workflow before ACR
versus current dual-iso advantages:
- higher burst rate
- no artifacts due to motion
and drawbacks
- always highlight/shadow artifacts due to interlace aliasing
- workflow steps before ACR
If the in-camera part was impossible or way too slow, combining could be done in workflow while keeping the IQ advantages.
Some tests
NORMAL
(http://imageshack.us/a/img837/1861/r3mz.jpg)
DUAL ISO
(http://imageshack.us/a/img407/5785/80qu.jpg)
This is wanderful and never show up the 70error :D
I see some things happen that i don't know if it's normal
1- Wend i back home to see the photos, i see in the structure of the CF card (in canon folders) that after i take a few photos he creates another folder and put the rest of the pictures there.
2-Wend i pick-up all the photos and drag to the cr2hdr, he fail always the first pictures. The rest of the pictures, he transcribe good and make the dng file. But almost in the end of each file, wend cr2hdr are doing is work, its show up a message saying that the exiftool didn't work, like the image below. Evan with this error the cr2hdr make is job and make the dng that i can open in photoshop for making the right tweek.
(http://imageshack.us/a/img441/9287/uod5.jpg)
3- A problem that i come across is that is hard for make the right exposure. Didn't, yet, get the best way to exposure.
4- Also noted that the photos make a strange border, but can be because of the jpg. I can crop wth no problem. Just for let you guys know about this.
Beside this DUAL ISO is one more great function in the ML arsenal =D.
THANK YOUU !!!
Quote from: arrinkiiii on August 01, 2013, 07:14:32 PM
2-Wend i pick-up all the photos and drag to the cr2hdr, he fail always the first pictures. The rest of the pictures, he transcribe good and make the dng file. But almost in the end of each file, wend cr2hdr are doing is work, its show up a message saying that the exiftool didn't work, like the image below. Evan with this error the cr2hdr make is job and make the dng that i can open in photoshop for making the right tweek.
Try running cr2hdr in a directory with no spaces just off the root directory. It works for me like that. You'll see the weird letters; for some reason it doesn't like spaces in the directory name (exifinfo.exe).
Quote from: dubzeebass on August 01, 2013, 09:41:11 PM
Try running cr2hdr in a directory with no spaces just off the root directory. It works for me like that. You'll see the weird letters; for some reason it doesn't like spaces in the directory name (exifinfo.exe).
Much appreciated dubzeebass, thank you.
Thanks to ALL for this wonderfull module - dual ISO! But how to work with resulted .CR2 on Mac OS? Can anyone help Mac users?
Somes tests again :
(https://lh3.googleusercontent.com/-Vole3VqBpEw/Ufruw-mKe6I/AAAAAAAAAD0/Q_9t2QkRf_0/w1336-h892-no/HL9A2300.jpg)
(https://lh3.googleusercontent.com/--hNgniMt3Og/Ufrrn_50yII/AAAAAAAAADs/a47J1f2LXsk/w1336-h892-no/HL9A2297.jpg)
(https://lh5.googleusercontent.com/-ZiL9q7Kss40/Ufrzjc7c7GI/AAAAAAAAAFg/qhWKVklxB-k/w1336-h892-no/HL9A2336.jpg)
:)
Here is a link to problematic cr2 (waves in the center of the picture painted in the colors of the rainbow)
https://dl.dropboxusercontent.com/u/4624786/_MG_8792.CR2
This should help you understand better how it works and how to expose the pictures:
http://acoutts.com/a1ex/split_MG_8792.dng
Quote from: xmd5a on August 02, 2013, 09:36:04 PM
Here is a link to problematic cr2 (waves in the center of the picture painted in the colors of the rainbow)
https://dl.dropboxusercontent.com/u/4624786/_MG_8792.CR2
I am new but eager to aprendre, here I leave another example
http://we.tl/BeTdsXJe4G
Here I'm afraid the only solution is to cover the palm trees with both ISOs. Remember that aliasing can't be removed in software (Nyquist theorem), only the artifacts can be somewhat masked.
Quote from: a1ex on August 02, 2013, 11:55:39 PM
Here I'm afraid the only solution is to cover the palm trees with both ISOs. Remember that aliasing can't be removed in software (Nyquist theorem), only the artifacts can be somewhat masked.
Quem sabe, sabe
who knows, knows
Any video examples yet? It'd be great to see some =)
Quote from: a1ex on August 02, 2013, 10:32:37 PM
This should help you understand better how it works and how to expose the pictures:
http://acoutts.com/a1ex/split_MG_8792.dng
Undoubtedly that the picture from bottom have the right exposure. But how you get at that settings? (7D)
Basically, what needs to have full detail or it's likely to get aliasing must be covered by both ISOs. In LiveView, check the exposure with both ISO values and zebras.
In these cases, less extreme settings like 100/400 should be very helpful.
Quote from: a1ex on August 03, 2013, 07:58:58 AM
Basically, what needs to have full detail or it's likely to get aliasing must be covered by both ISOs. In LiveView, check the exposure with both ISO values and zebras.
In these cases, less extreme settings like 100/400 should be very helpful.
Indeed, thank you, will try more pic's.
I think with the silent picture burst in the 7D (the beginning of raw=) it will be more easy... if we get the ETTR working, no?
ETTR should be already working. Be careful though - with dual ISO it exposes for the least amount of noise and ignores moire.
Awesome news.
Ok, for the dumbo here, how do I use the CR2toHDR.exe file?
Is there any documentation as I'm a photographer not a programmer.
Thanks
Quote from: hamishniven on August 03, 2013, 11:28:11 AM
Awesome news.
Ok, for the dumbo here, how do I use the CR2toHDR.exe file?
Is there any documentation as I'm a photographer not a programmer.
Thanks
Drag and drop cr2 files to the cr2tohdr.exe.
Quote from: a1ex on August 03, 2013, 11:09:33 AM
ETTR should be already working. Be careful though - with dual ISO it exposes for the least amount of noise and ignores moire.
Massive!!!
But do you know if the ETTR are in the new version of Pelican? For now we just got the silent burst pictures, almost 2 seconds of video.
Going post a video with that in the 7D treat.
If it's not, copy it from some other nightly. Modules are portable by design.
Quote from: a1ex on August 03, 2013, 12:23:28 PM
If it's not, copy it from some other nightly. Modules are portable by design.
Alright, super duper easy :D
@A1ex
I have download the 5DII ETTR module from https://bitbucket.org/a_d_/magic-lantern/downloads (the first zip file), this is correct?
Also have the autoexpo.mo, it will work in the 7D?
I'm experimenting with some algorithms that should handle aliasing a bit better. Here's an experimental cr2hdr: http://acoutts.com/a1ex/cr2hdr_exp.exe
Can you compare it with the one from first post? I'm interested especially in cases where the old converter gets better results.
Main difference: in aliased areas I use chroma blur (luma is no longer blurred), and the algorithm uses a stronger bias towards full-resolution mode (which is only possible in midtones, where the subject is covered by both ISOs).
Here are my before and after samples (CR2's from Danne). Click for full-res jpegs:
(http://acoutts.com/a1ex/IMG_6706_before_crop.jpg) (http://acoutts.com/a1ex/IMG_6706_before.jpg) (http://acoutts.com/a1ex/IMG_6706_after_crop.jpg) (http://acoutts.com/a1ex/IMG_6706_after.jpg)
(http://acoutts.com/a1ex/IMG_6701_before_crop.jpg) (http://acoutts.com/a1ex/IMG_6701_before.jpg) (http://acoutts.com/a1ex/IMG_6701_after_crop.jpg) (http://acoutts.com/a1ex/IMG_6701_after.jpg)
For 100% aliasing-free results, don't forget to cover your main subject with both ISOs. There's no other way.
Wow! Big difference in aliasing. I,ll try some comparisons conversions soon. On the road...
Found something surprisingly similar to this method:
http://image-sensors-world.blogspot.tw/2013/01/sony-explains-hdr-video-mode-in-its.html
Hi all, newbie here but I have done all the installations... on some test shots I've done, when viewing at 100% I can see some horizontal lines as opposed to normal exposed images. Is there any reason for this?
I tried cr2hdr.exe but seems that the converter does nothing just open and close in 1/2 secs... any thoughts?
Here is a sample, this is 100% crop, exposed to highlight to prevent pink/magenta at highlight area, dual ISO 100/1600...
(http://farm6.staticflickr.com/5519/9430444662_0a8836103f_o.jpg) (http://www.flickr.com/photos/robinsslee/9430444662/)
_38A6284 (http://www.flickr.com/photos/robinsslee/9430444662/) by Robin SS Lee (http://www.flickr.com/people/robinsslee/), on Flickr
Thanks
Robin
Quote from: robinlee on August 03, 2013, 06:11:36 PM
Hi all, newbie here but I have done all the installations... on some test shots I've done, when viewing at 100% I can see some horizontal lines as opposed to normal exposed images. Is there any reason for this?
I tried cr2hdr.exe but seems that the converter does nothing just open and close in 1/2 secs... any thoughts?
Thanks
Robin
The lines is the interlaced ISO. You need to "de-interlaced" using cr2hdr.exe
Put the cr2hdr.exe in the same folder that your pictures. Click and drag the pictures to the cr2hdr.exe and you are done. It will create the DNG files.
Read the first post... and the rest of it.
Keep the folder with the cr2 with no spaces, short name otherwise it won,t convert.
Thanks for your help arrinkiiii and Danne... :)
Shot some raw video today and when unpacking with the raw2dng.exe it gets about halfway and then:
"Error: mallocixels"
E:\RAW\5>raw2dng m03-2004.raw
Resolution : 1920 x 1080
Frames : 881
Frame size : 3629056 bytes
FPS : 23.976
Black level : 2079
White level : 15000
Processing frame 1 of 881...
Vertical stripes correction:
1.000 1.000 1.009 0.970 0.987 0.997 0.985 1.036
ISO pattern : ddBB
Interpolation : mean23-vsmooth3-contrast
ISO difference : 6.03 EV (6516)
Black delta : 89
ISO overlap : 3.0 EV (approx)
Processing frame 494 of 881... [HDR BBdd]Error: mallocixels]
Any ideas?
5d3, Aug 2 build.
Figured it out - was using "Bachelor" to unpack the raw files, and it had an older version of Dcraw.exe in it. Problem fixed.
Hi, I am ashamed to ask this as a software developer, but I couldn't manage to get the modules menu on 7D.. What did I do wrong?
1. I downloaded the ML from http://upload.g3gg0.de/pub_files/17248a00956f1e932457094756b2a3ba/magiclantern_7D_203_Alpha2.zip
2. I downloaded modifications from the post #1 of this thread. ANd put modules in ML/MODULES folder.
3. I managed to get the ML Menu but no Dual Iso module, or menu.
Sorry to interrupt :(
There is a thread specific for the 7D. Try the search function
http://www.magiclantern.fm/forum/index.php?topic=3974.975
Quote
For 100% aliasing-free results, don't forget to cover your main subject with both ISOs. There's no other way.
@A1ex: Downloaded the ch2hdr exp version. The results are much better. Don't understand what you mean by "don't forget to cover your main subject with both ISOs"
For full resolution and no moire, you should have detail with both ISOs. Where one of them gets clipped, you will get half-resolution. If the subject doesn't have fine detail, half-resolution will not be noticeable; but if it has, you will get moire.
In shadows, this is less of a problem; you still have detail in both ISOs, but one of them has too much noise. That detail is usually enough to reduce aliasing, and the algorithm takes care of that. In highlights, if it's clipped, there's zero detail, so you just can't bring back the missing lines.
So, you have to be careful what is covered by both ISOs (OK) and what is covered by only one (risk of moire).
Try zebras with the higher ISO in LiveView: if the main subject is not overexposed, should be OK. Background highlights can be overexposed; if they are not too sharp (e.g. clouds), you will not see the aliasing at all.
This should also help: https://bitbucket.org/hudson/magic-lantern/commits/84d446e1e06487fdf9a6cd1e3969ac717c84651c
any chance of the new algorithm working with RAW2DNG too?
There,s a nice app here that works for mac with dual iso.
http://www.magiclantern.fm/forum/index.php?topic=7266.msg62587#msg62587
DualIso works well on 5D3, thank you for this feature! With the first tests the detail loss of structures was a little bit to much, but with newer converters it is much better. But the shadow recovery works very very good.
But I have a little "other" request. I install the ML-version from the dropbox onto the CF-Card (not SD), and it's working good. Yesterday I want to format the CF, because of 300+ pics, and the format menu cleans the whole card, with ML. Grrr, I had to copy it again with the card reader.
With the ML-version of my 60D I know the "feature", which reads the whole ML-directory (and autoexec) into the RAM, formats the card, and rewrite ML back to the card. Is this feature broken or is it deactivated on purpose?
That part needs to be reverse engineered almost from scratch because of the double card slot. Same for the installer.
A little testing with Alex new aliasing changes in the experimental cr2hdr_.exe and the most recent one from the first post.
A little more jagged edges in the aliasing parts in some motifs but it also completely removes aliasing from other motifs in the experimental. Hard to tell what is best, I,m leaning towards the experimental, especially from the pic of the old woman and of course from the store. Here,s some pics. I strived to get the motif with both isos.
The originals can be downloaded from this link (119mb 4 pictures)
https://docs.google.com/file/d/0B4tCJMlOYfirRlFjZ3ljVEtDc0U/edit?usp=sharing
(http://s9.postimg.org/e9v70rpu7/1bnew_1_av_1.jpg)
experimental
(http://s9.postimg.org/6ndmy53m7/2bnew_1_av_1.jpg)
recent
(http://s9.postimg.org/nmmldeetr/2bold_1_av_1.jpg)
(http://s9.postimg.org/c2qycukjz/1new_1_av_1.jpg)
experimental
(http://s9.postimg.org/iqiylpeof/2new_1_av_1.jpg)
recent
(http://s9.postimg.org/48lrdpndb/2old_1_av_1.jpg)
experimental
(http://s9.postimg.org/5g4ajo2vj/4new_1_av_1.jpg)
recent
(http://s9.postimg.org/k4f6ur3bj/4old_1_av_1.jpg)
(http://s9.postimg.org/z6w0j6d8v/3cnew_1_av_1.jpg)
experimental
(http://s9.postimg.org/sy63buqa7/4cnew_1_av_1.jpg)
recent
(http://s9.postimg.org/4gdznynpr/4cold_1_av_1.jpg)
Magentaissue in some pics from the experimantal build, not seen in the converter from the first post.
I also only manage to convert 35 pictures, cr2 with the converter from the first post and the it quits. This don,t happen with the experimental build.
(http://s9.postimg.org/rmyc3duof/magenta_1_av_1.jpg)
If any of these are of interest as originals please tell me which and I,ll upload those.
*post updated with link to the originals(see above)
Yes, upload them. These are very good for my test shot collection.
@Alex
Post updated with originals
https://docs.google.com/file/d/0B4tCJMlOYfirRlFjZ3ljVEtDc0U/edit?usp=sharing
Thanks!
//D
@Danne - What causes the magenta issue in the highlights? I have a few pics I can't get that to go away on...
Download cr2hdr_exp.exe again and upload the CR2's if you still have trouble. It was hot pixels in overexposed areas tricking the white level autodetection code.
Thanks A1ex, That fixed the magenta highlight issues, but still major ailiasing. I probably exposed way wrong and didn't keep things covered by both iso's.
One thought I had that might make things easier shooting is to be able to reverse the iso's. Meaning shoot at iso 1600 and use iso 100 as recovery for the highlights. I did a real estate shoot where if I exposed to not clip the highlights at iso 100 and set the dual iso to 1600 to save the mid tones, the live view was basically black and I couldn't frame my shots. If I switched to 1600, then I could see the scene more properly exposed. I would then switch back to 100 to shoot...
Maybe your script is already that smart, I haven't tested it yet. I also have found that for video to come out correct, I have to reverse the settings as well...
Thanks for all your accomplishments, it really is mind boggling!
@NateVolk
You can shoot with the higher ISO and then select the highlight recovery ISO in the dual-ISO menu. You can use the +-EV settings also to either consistently use +whatever EV for recovery or consistently -whatever EV.
I'm having an issue I would like help with please, I don't think I've seen this before on the forum and I couldn't find it with a search.
A number of my pictures will begin processing only to say "file doesn't look like dual ISO"
The images are definitely dual ISO so I'm not really sure where to start troubleshooting, any suggestions? Thanks very much.
For the 1000th time: if you are having issues and you don't upload some CR2 files, there's nothing I can do to help you.
I'm sorry about that, that should have been obvious to me. Here's the .cr2 file. https://www.dropbox.com/s/s2tew00wftvlef4/7O4C8804.CR2
Uploaded a fix (download cr2hdr_exp.exe again). Can you check with your other pics?
a1ex can you add something to the filename to differentiate dual-ISO's?
I shot 1000 photos the other day with dual-ISO on alternate frames thinking that would be a great idea. 1 back-up with each dual-ISO shot. It worked great, and I love the results of +2EV shadow recovery, but processing them was a pita, simply because they take up space otherwise, if you don't delete the original CR2's.
Canon uses different filenames for AdobeRGB and sRGB color spaces. Perhaps you could hook into that?
Once everything gets closer to RC, I would like to suggest the option of deleting the original CR2's after being processed by cr2hdr.
My workflow is to drag all the files on cr2hdr: it will autodetect which shots were dual and which not. Sure, it takes a bit of time. My autoexposure script uses DNG by default, so it's quite straightforward: I just run both tools, leave it crunching numbers for a few hours, and I get nearly-finished pics.
For changing file names, I can add an option to shoot dual ISOs in adobe RGB, and normal pics in sRGB, for example. Or, I can change the first 4 letters (suggestion?)
I would not suggest deleting the CR2 files. The algorithm is improving, so if you reprocess the same thing a month later, you already get better results.
I was selecting everything over 26000kb to drop on the exe. Still takes time to process 000's of photos though.
The first 4 letters are a good idea. DUAL0001.CR2 :)
Small problem with this approach: they are no longer sorted (you need to rename them back in post so you can watch them in the same order). Plus, syncing the file names with the alternate option (semi-burst) is tricky (it's too easy to mis-label the pictures, because capturing and saving are done at different times).
Hmmmm. I would be happy if their was a cmd option to move processed files. Including non dual-ISO files put through the exe.
I could dump the CF onto an SSD and have all the processed files moved to storage. Not sure of the usefulness of this for others though.
The seperation of processed and original dual-ISO's is what I'm chasing. I can't think of any easier ways.
edit: And thanks. Just noticed the commit.
Hi Alex, can you post the download link for this on your first page please?
Thanks
Robin
Quote from: a1ex on August 05, 2013, 11:14:02 AM
Uploaded a fix (download cr2hdr_exp.exe again). Can you check with your other pics?
Small update for cr2hdr_exp.exe: better black level handling. This is only noticeable in extremely underexposed shots (e.g. when you need to push the shadows at +10 EV).
Before and after (CR2 from AdamTheBoy):
(http://acoutts.com/a1ex/7O4C8804_before.jpg) (http://acoutts.com/a1ex/7O4C8804_after.jpg)
For grading: I've developed the DNG at +4 EV ... +12 EV in 1 EV increments and ran the shots through enfuse. The image was quite underexposed, next time try using ETTR ;)
Just for fun: here's how it would have looked at ISO 100. Good luck denoising that.
(http://acoutts.com/a1ex/7O4C8804_dark.jpg)
For higher-res crops, download the CR2 and grade it yourself. This update was about color cast in shadows, not about fine detail.
I'll update it on the first page and in raw2dng when I'll solve the remaining issues in highlights.
@IliasG: you around? can you check this and see if you get better results?
Quote from: a1ex on August 05, 2013, 10:39:05 PM
Small update for cr2hdr_exp.exe: better black level handling. This is only noticeable in extremely underexposed shots (e.g. when you need to push the shadows at +10 EV).
Before and after (CR2 from AdamTheBoy):
http://acoutts.com/a1ex/7O4C8804_before.jpg http://acoutts.com/a1ex/7O4C8804_after.jpg
For grading: I've developed the DNG at +4 EV ... +12 EV in 1 EV increments and ran the shots through enfuse. The image was quite underexposed, next time try using ETTR ;)
Just for fun: here's how it would have looked at ISO 100. Good luck denoising that.
http://acoutts.com/a1ex/7O4C8804_dark.jpg
For higher-res crops, download the CR2 and grade it yourself. This update was about color cast in shadows, not about fine detail.
I'll update it on the first page and in raw2dng when I'll solve the remaining issues in highlights.
@IliasG: you around? can you check this and see if you get better results?
Yep, around and just checked. Much better about color cast at darks, but at a cost. The noise is harsh and it's not points (easy to denoise) but like candles. And I think that there is too much black clipping.
As I see here the Black Level in the CR2 file is around 2032-2034 for the low ISO lines and 2062-2066 for the hi ISO lines. How do you normalize for this ?. (In the meantime I have to read again your pdf I suppose ..).
This shows clearly that Canon tries to normalize at the desired average of 2048 as BL but how exactly ??. Is it just a data sliding up or down or a multiplication ?.
It happens this shot is with f/1.8 so there should be data scaling (due to wide aperture) and the White Level must be around 16000 ... the the DNG's exif says 49600 !! ?? even lower than 4*13000 ...
In the case of white level I still vote for a Lookup table.
@a1ex,
Took this dual ISO shot while i was testing burst mode. I know a res chart is not the best for testing dynamic range which is the benefit of using dual ISO but I figured the behavior of aliasing and moire can be better observed and tweaked in a repeatable test environment.
Just hoping it helps.
From dual iso shot 200 cam 1600 recovery. o=sharpening in acr
(http://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-ash3/1102430_354578578006266_222939689_o.jpg)
From normal cr2 same settings at 200 iso
(http://fbcdn-sphotos-d-a.akamaihd.net/hphotos-ak-prn2/1097275_354578574672933_1186918413_o.jpg)
If dual iso can be ported to raw video in the 7D, it sure would be much better in handling moire and aliasing. :)
Here's a chart from burst mode compared to dual iso in my previous post.
(http://fbcdn-sphotos-f-a.akamaihd.net/hphotos-ak-prn2/1149779_354583941339063_760796991_o.jpg)
@Ilias: I split the raw file in 4 sub-images (group by y mod 4) and bring the black level to 2048 for each one (scaling to keep the old white level and taking care of drift). Then I run the interpolation routines. At the end, I average the black level again (to compensate for what the interpolation might do to noise), and it still finds a few units (for this image, 3 units).
The output DNGs are 16 bit. Not sure if the extra 2 bits have anything useful in them or just rounding noise (but at least they open pretty fast in ufraw compared to CR2's).
@Ted: the result with resolution chart will vary greatly with exposure (so, for best results, you should be careful when exposing). Yes, the VAF will help a lot for video (and I'm sure it will work on 7D too).
In the mean time... do you have a VAF for 7D? if so, can you shoot the test chart in photo mode with dual ISO at some extreme setting? (where aliasing appears normally)
Quote from: a1ex on August 05, 2013, 10:39:05 PM
I'll update it on the first page and in raw2dng when I'll solve the remaining issues in highlights.
Hi Alex is the raw2dng now updated on the first page?
Wondering if it would be possible to do audio with dual ISO active during video taking on my 5D Mark 3.
Is there any reason for no video examples?
Not being rude, just wondering if there is a technical reason :)
Probably I'm doing it wrong, but I haven't gotten any good video samples yet.... I'll try again today :)
I can't tell the difference between this and HDR from the photos POV, not the technical aspect POV. Anyone mind telling me? :D
Quote from: ShootingStars on August 07, 2013, 04:39:49 AM
I can't tell the difference between this and HDR from the photos POV, not the technical aspect POV. Anyone mind telling me? :D
http://www.magiclantern.fm/forum/index.php?topic=7139.0
Read the pdf embedded towards the very end of the very first post by A1ex.
Dual_ISO is HDR on steroids - literally. While the results may appear similar since it is bonding of different EV's as is the case with HDR. But that is where the similarity ends.
Conceptually, both come from different planets. For a good HDR yeild, you would need more a few photos with the same output at different EV's (3 or 3+ is ideal). For Dual_ISO, all the action happens on the same picture but in alternating horizontal lines - each alternate bright horizontal line holds the highlights; and the underexposed ones hold the shadows (pulled up to a point where the overall picture is correctly exposed). To acheive this each line representing the highlights and shadows are drawn from two different ISO's (hence the name Dual_ISO).
The results are stunning. The camera easily handles RAW/ CR2 in Dual_ISO mode. The grain is superb and unlike the sensor noise you would get on a normal RAW frame process to pull up the shadows in post. The range you get is about DR (14-16bit) - basically a 3 EV stop improvement. This functionalilty works both on photo and video.
My take on this is simple. For normal situations shoot normal. For abnormal situations shoot Dual_ISO.
Plus, the grain is something to die for.
Just look out for the grain on this link: http://www.flickr.com/photos/89977905@N05/9449868520/
Thanks for the explanation! :) Will go test it out in abnormal situations.
I think it is better to cut 2 or 3 pixel lines/rows on top and left side.
(https://dl.dropboxusercontent.com/u/4624786/dual_iso_cut.jpg)
Is there a thread somewhere explaining how to do this through wine for us Mac users or maybe I am missing something im stuck ???
I am using August 2 build from lourenco. I've also read that Son of Batch for Mac now supports dual ISO is that for video only?
Also I will be shooting a wedding this weekend and would love to use this feature , that being said the only thing that I have besides the Mac Pro is a laptop that has XP on it and every time I try to drag a file on the app it opens then closes. :-\
Another update regarding aliased areas, for cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe). This time I've tried a very difficult shot - the CR2 from daancalo2013:
cr2hdr from first post:
(http://acoutts.com/a1ex/6X8A9415_before.jpg)
previous cr2hdr_exp:
(http://acoutts.com/a1ex/6X8A9415_after.jpg)
today's update:
(http://acoutts.com/a1ex/6X8A9415_after2.jpg)
Still not perfect, but I doubt it can get much better than this.
Pixel peeping is pretty hard, who'd like to run some tests to make sure it handles pretty much everything better than the older one?
Hello everyone,
I've been following all the recent developments with the ML team over the last few months, trying to read everything there is. It's a really exciting time for Canon DSLR owners and I want to give a huge thanks to all the developers for their hard work bring this to all of us common folk.
Here are some examples of a few Dual ISO shots that I've taken so far on my 7D. These are both 100/1600. I'm still trying to get the hang of it, but it is an amazing feature.
I'm using the cr2hdr.exe from A1ex's Reply #469
(http://i.imgur.com/HjYkh4q.jpg)
(http://i.imgur.com/BITsX1z.jpg)
(http://i.imgur.com/FqYRrkK.jpg)
(http://i.imgur.com/JDztUQI.jpg)
Itching itching itching to see video samples. My line of work is interior real estate video and if it means reliable high dynamic range to compensate for the interior and exterior then I am sold on getting a 5D mkiii ;D
Hi a1ex,
Just an idea...
would it be possible or practical to emulate a graduated ND filter in a new dual ISO mode? Let's say the top 1/3 of the image is in ISO 100, the bottom 2/3 is in ISO 800/1600, etc. You could move up and down the transition line with the joystick and the transition could be hard or soft (variable). You could emulate even a reverse ND for sunset shooting.
This would be a great advantage for landscape photographers who use filters to reach the proper results. And in this case we wouldn't have to deal with resolution loss and aliasing, etc. Only the transition area could be more problematic by using alternating lines of ISOs as today. So basically we would narrow down the usage of the current dual ISO mode with alternating lines to a small horizontal segment of the image, while the other parts of the image would be in 2 different ISOs.
I am curious about your thoughts on this.
Great work, anyway!
Quote from: weigertj on August 08, 2013, 01:15:02 AM
Hi a1ex,
Just an idea...
would it be possible or practical to emulate a graduated ND filter in a new dual ISO mode? Let's say the top 1/3 of the image is in ISO 100, the bottom 2/3 is in ISO 800/1600, etc. You could move up and down the transition line with the joystick and the transition could be hard or soft. You could emulate even a reverse ND for sunset shooting.
This would be a great advantage for landscape photographers who use filters to reach the proper results. And in this case we wouldn't have to deal with any resolution loss and aliasing, etc. Only the (soft) transition are could be more problematic with using alternating lines of ISOs. I am curious about your thoughts on this.
Great work, anyway!
That would be amazing!
On another note, I am tossing up between the 5D2 and 5D3 at the moment. Will the Dual ISO ever come to the 5D2?
Quote from: sletts02 on August 08, 2013, 01:26:28 AM
That would be amazing!
On another note, I am tossing up between the 5D2 and 5D3 at the moment. Will the Dual ISO ever come to the 5D2?
Unfortunately only the 5D3 and 7D have the hardver for it, it's impossible on other models. Go for the 5D3! ;)
Quote from: weigertj on August 08, 2013, 01:15:02 AM
Hi a1ex,
Just an idea...
would it be possible or practical to emulate a graduated ND filter in a new dual ISO mode? Let's say the top 1/3 of the image is in ISO 100, the bottom 2/3 is in ISO 800/1600, etc. You could move up and down the transition line with the joystick and the transition could be hard or soft (variable). You could emulate even a reverse ND for sunset shooting.
This would be a great advantage for landscape photographers who use filters to reach the proper results. And in this case we wouldn't have to deal with resolution loss and aliasing, etc. Only the transition area could be more problematic by using alternating lines of ISOs as today. So basically we would narrow down the usage of the current dual ISO mode with alternating lines to a small horizontal segment of the image, while the other parts of the image would be in 2 different ISOs.
I am curious about your thoughts on this.
Great work, anyway!
its been asked already.
http://www.magiclantern.fm/forum/index.php?topic=6335.msg50620#msg50620
Quote from: weigertj on August 08, 2013, 01:39:06 AM
Unfortunately only the 5D3 and 7D have the hardver for it, it's impossible on other models. Go for the 5D3! ;)
Well that settles that. There was a 5D2 with battery grip/batteries for $1100AUD. But I think the extra $2200AUD will be worth this feature if it's stable and workable.
Quote from: a1ex on August 07, 2013, 11:34:12 PM
Another update regarding aliased areas, for cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe)
Does the link above have the latest update?
@optik re: running on xp
I don't know why, but to run on xp, you have to right click and open with. Otherwise it doesn't work. Also, we can't batch by dragging onto, makes processing a bunch of images a giant pain...
Quote from: ted ramasola on August 08, 2013, 01:45:11 AM
its been asked already.
http://www.magiclantern.fm/forum/index.php?topic=6335.msg50620#msg50620
It's a different question.
Instead of scanning alternate lines, scan the top half (1/3rd whatever) at ISO whatever, scan the rest at another ISO. Make it graduated too ;)
a1ex mentioned that he is only tweaking a register (for dual-ISO). There isn't enough hardware control (currently) to adjust the scanning procedure.
@Canon eos m
You could try downloading it and looking at the timestamps!
Quote from: sletts02 on August 08, 2013, 12:04:18 AM
Itching itching itching to see video samples. My line of work is interior real estate video and if it means reliable high dynamic range to compensate for the interior and exterior then I am sold on getting a 5D mkiii ;D
Me too, if you find any good samples, post them up! I haven't gotten it worth selling. Yet :)
Quote from: NateVolk on August 08, 2013, 03:19:34 AM
Me too, if you find any good samples, post them up! I haven't gotten it worth selling. Yet :)
Here you go! http://bit.ly/11NFGnh (http://bit.ly/11NFGnh)
Quote from: NateVolk on August 08, 2013, 03:17:44 AM
@optik re: running on xp
I don't know why, but to run on xp, you have to right click and open with. Otherwise it doesn't work. Also, we can't batch by dragging onto, makes processing a bunch of images a giant pain...
I guess I do have the option to install and run boot camp Windows or should I just use something like parallels or VMware
Quote from: weigertj on August 08, 2013, 01:15:02 AM
Hi a1ex,
Just an idea...
would it be possible or practical to emulate a graduated ND filter in a new dual ISO mode? Let's say the top 1/3 of the image is in ISO 100, the bottom 2/3 is in ISO 800/1600, etc. You could move up and down the transition line with the joystick and the transition could be hard or soft (variable). You could emulate even a reverse ND for sunset shooting.
This would be a great advantage for landscape photographers who use filters to reach the proper results. And in this case we wouldn't have to deal with resolution loss and aliasing, etc. Only the transition area could be more problematic by using alternating lines of ISOs as today. So basically we would narrow down the usage of the current dual ISO mode with alternating lines to a small horizontal segment of the image, while the other parts of the image would be in 2 different ISOs.
I am curious about your thoughts on this.
Great work, anyway!
You can always emulate a GND from a "properly" (no highlights blown out) photograph in the computer; problem is that raising the shadows produces too much noise. But now that you can have files with a massive dynamic range, it is probably worthy trying again. At the end of the day, the results are going to be roughly the same, and much more versatile.
@Alex
Latest cr2hdr seems to be the best so far. I,m really impressed. Comparison as follows. These are crops to emphasize on defects and improvements.
Beta(first converter)
beta0.1(next one)
beta0.2(latest converter, also the best one)
beta
(http://s13.postimg.org/tsvftiz3r/beta.jpg)
beta01
(http://s13.postimg.org/xgb6zl7av/beta0_1.jpg)
beta0.2(compare the bottom of the sign)
(http://s13.postimg.org/cuwf7oppz/beta0_2.jpg)
beta
(http://s13.postimg.org/kjoo5o0tz/beta_b.jpg)
beta0.1
(http://s13.postimg.org/5qelynign/beta0_1b.jpg)
beta0.2(pay attention to the right arm)
(http://s13.postimg.org/omkaihkc7/beta0_2b.jpg)
Quote from: eduperez on August 08, 2013, 11:38:56 AM
You can always emulate a GND from a "properly" (no highlights blown out) photograph in the computer; problem is that raising the shadows produces too much noise. But now that you can have files with a massive dynamic range, it is probably worthy trying again. At the end of the day, the results are going to be roughly the same, and much more versatile.
I will definitely will try to shoot a sunset landscape in dual ISO. Expose for the sky in ISO 100 and bring up the shadows in post.
Gooooddd work Alex !!!!!!!!!!!!!!!!!!
I,m VERY impressed..
Wow A1ex! This is just getting better and better, so thankful for all your hard work. Noticed that the colours are a lot more vibrant with the latest version of cr2hdr too.
Now what I'm aobut to say is just dreaming.. :)
Imagine if we could controll every second line vertically too, wouldn't this make it even more powerfull, or maybe better yet in a bayer pattern with 3 different ISO values like 100, 800, and 1600 just imagining it would bring even more DR to the final image. Now as I said it's just imagination on my side and I don't think it's possible. And maybe it wouldn't even help.. but then again who thought dual ISO would be possible before A1ex revealed it to us. :)
Quote from: a1ex on August 07, 2013, 11:34:12 PM
Another update regarding aliased areas, for cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe).
Hi Alex,
Could you please advise what is wrong with my cr2hdr_exp.exe post processing.
The original RAW Dual_ISO DNG shot with my 5DM3 is just fine.
https://www.dropbox.com/s/m86dyhnrbtajhi5/_G1A7637.CR2
Then when I processed it on the latest cr2hdr_exp.exe, I see big huge white patches all over the picture.
https://www.dropbox.com/s/rf44z0tqy4sl4ub/_G1A7637.DNG
Ah, those tiny hot pixels.
It seems better with my latest copy (download the exe again), but there are still a bunch of them. I'll see what I can do, never seen that in my test samples.
Grrreat!
Where can I get the latest version of ch2hdr.
Second, I downloaded enfuse after reading you used it for HDR type work. Any idea if enfuse will process CR2/
DNG files or will this require conversion first to TIFF/ JPEG?
Quote from: a1ex on August 07, 2013, 11:34:12 PM
Pixel peeping is pretty hard, who'd like to run some tests to make sure it handles pretty much everything better than the older one?
I just realized that the new version works great removing the reddish shadows. Unfortunately there are some artifacts near highlights which may be caused by the anti-aliasing? Each highlight is surrounded by a ring of horizontal lines which are about three to four pixels long.
(http://www.naturalsound.de/MagicLantern/IMG_8485_exp.jpg)
I hosted the .cr2 at: http://www.naturalsound.de/MagicLantern/IMG_8485_ML.CR2 (http://www.naturalsound.de/MagicLantern/IMG_8485_ML.CR2)
(I may have to remove the link if traffic gets huge).
Update: This CR2 is of an EOS 7D
The last experimental version did not show the lines and rendered the highligts clearly. Of course there was the reddish shadow cast.
I really appreciate your work!
@Nikon eos m: this was ISO 12800/400? Here, the hot pixels are grouped, and my algorithm assumes they are isolated. Try not going higher than 3200.
To download, lookup my earlier posts (it's the same link). To merge with enfuse, I use this: www.magiclantern.fm/forum/index.php?topic=7022
naturalsound: here, the ISO response seems highly nonlinear, so the algorithm has trouble matching the brightness. Probably caused by aliasing, not yet sure.
Hi a1ex,
I forgot to mention the fact that this is an EOS 7D picture, but I guess you already noticed from the EXIF.
I have some additional RAW files in case you need another one.
Thank you for your fast response. I think it is a minor issue and will probably only happen in these very high DR situations. Maybe some pixels bleed because of the very high photon count and create the artifacts?
Alex I call myself Canon eos m and not Nikon eos m - not that it matters, it is just that this a Canon forum and it is embarrassing as hell to have some one refer to me as a Nikon fellow :-)
@a1ex
using aug 3 cr2hdr_exp
(https://fbcdn-sphotos-h-a.akamaihd.net/hphotos-ak-prn2/1093880_355572661240191_1837277124_o.jpg)
using aug 5 cr2hdr
(https://fbcdn-sphotos-b-a.akamaihd.net/hphotos-ak-frc3/1090950_355572857906838_595474699_o.jpg)
using aug 7 cr2hdr
(https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-ash3/1091175_355572994573491_396787713_o.jpg)
great improvement with color aliasing.
edit: used 7D
QuoteAlex I call myself Canon eos m and not Nikon eos m
When using dual ISO it's like having a Nikon sensor ;)
@ted: can you upload the CR2 for this shot?
@a1ex,
compressed it and sent it to your email.
Quote from: a1ex on August 08, 2013, 06:05:53 PM
@Nikon eos m: this was ISO 12800/400? Here, the hot pixels are grouped, and my algorithm assumes they are isolated. Try not going higher than 3200.
To download, lookup my earlier posts (it's the same link). To merge with enfuse, I use this: www.magiclantern.fm/forum/index.php?topic=7022
naturalsound: here, the ISO response seems highly nonlinear, so the algorithm has trouble matching the brightness. Probably caused by aliasing, not yet sure.
Alex got myself the latest cr2hdr.exe from page 1 of this thread. Like you said, magically, almost all hot pixels gone (at least the big and bothersome ones).
Same picture processed with the latest cr2hdr.exe
https://www.dropbox.com/s/vys1cux6yi7iege/kevin%27s%20workshop.JPG
That's the old converter, which does not check whether the hot pixels are isolated or not. So it just corrects them blindly.
The problem is that it's too easy to mistake the hot pixels for aliasing. So, for now, I don't have a fix that solves both problems.
Quote from: a1ex on August 08, 2013, 07:19:10 PM
That's the old converter, which does not check whether the hot pixels are isolated or not. So it just corrects them blindly.
Do you mean the one on page 1 is the 'old convertor'? If it is, where do I get the latest one?
Quote from: a1ex on August 07, 2013, 11:34:12 PM
Another update regarding aliased areas, for cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe)
Today's update keeps freezing as soon as it reaches the "looking for hot pixels" point. Am I the only one with this issue? It seems to happen regardless of which image I try. Thanks again A1ex for all the awesome work, it's exciting to see it get better and better.
Can you upload a CR2? Shouldn't happen.
Quote from: a1ex on August 08, 2013, 07:24:30 PM
That is the one I used the first time and ended with a lot of hot pixels. Will try to deploy it once again for another few pictures and see what happens! It is already 2:00 am here in Singapore. Good night!
BTW saw this awesome video today: http://nofilmschool.com/2013/07/magic-lantern-firmware-canon-linuxtag/
You folks rock!
Quote from: a1ex on August 08, 2013, 07:24:30 PM
Alex, re-checked. It is only on this the photo I posted and only if I use the ch2hdr_exp.exe and not when I am using the ch2hdr.exe taken from the first page of this thread.
Sure thing, here's one that I tried https://www.dropbox.com/s/4r0pfxn9kaepl87/7O4C8813.CR2
I'm a bit confused. It seems that the last (7Aug) A1ex build has a huge step forward regarding antialiasing (which is really awesome BTW) but... how does this translates into video performance? Does it has huge advantage as well? Video samples anyone?
Thnx.
A1ex: i see, thank you. Looking so forward for it.
It's not yet updated for video.
Updated the experimental cr2hdr (http://acoutts.com/a1ex/cr2hdr_exp.exe) again, this time fixed the brightness matching on 7D (test shot from naturalsound - the sensor response is linear, it was just my algorithm too sensitive to noise) and also fixed some buffer overflow errors.
I'm also on the way to fixing the black banding from 7D at ISO 100 (like this (http://www.magiclantern.fm/forum/index.php?topic=7402.msg64607#msg64607)). How many of you did you notice it? Do you have more samples? Did you notice similar problems on other cameras?
Great work - Thank you!
I would not even have dreamed my 7D could deliver that clean shadows!
(http://www.naturalsound.de/MagicLantern/fireworks.jpg)
Quote from: a1ex on August 08, 2013, 10:30:53 PM
I'm also on the way to fixing the black banding from 7D at ISO 100 (like this (http://www.magiclantern.fm/forum/index.php?topic=7402.msg64607#msg64607)). How many of you did you notice it? Do you have more samples? Did you notice similar problems on other cameras?
I hope this helps a bit .. http://www.dpreview.com/forums/post/51945370
Maybe not the best test when it comes to dual iso but a little comparison between 3x cropmode and regular raw using a corrected 8mm peleng and a 20-35mm wideangle shot with dual iso.
Quote from: Danne on August 09, 2013, 12:12:30 AM
Maybe not the best test when it comes to dual iso but a little comparison between 3x cropmode and regular raw using a corrected 8mm peleng and a 20-35mm wideangle shot with dual iso.
Looks promising! Unfortunate that you didn't have a contrasty blue sky to get a good feel for the range. If possible, would you be able to do a quick test where you shoot the interior of a room with windows so we can see how it handles the typically blown highlights from windows? That would be ultra helpful please and thank you :-*
EDIT: Just what I was after - Thanks heaps! I'm buying the 5D3 immediately now haha. It's just a shame that shipping for CF cards will take a week or two from the states.
Nice balcony and neighborhood :)
5D III:
5796 x 3870 = 22,4MPx
3870 / 2 = 1935
2892 x 1935 = 5,6MPx Ultra High Quality ;D
Well, it's not that easy.
In many practical cases you won't notice the detail loss (this happens when the aliased areas are out of focus backgrounds, or when the missing data is guessed correctly).
In some extreme cases (brick walls maybe?) aliasing might be so bad that it will be visible at 720x480 too. But the same is true about 1080p video with line skipping.
Anyway. The latest updates have a heuristic that I believe it puts a limit of how bad the aliasing can get (it was the key to solving the palm tree shot):
/* if the brighter copy is overexposed, the guessed pixel for sure has higher brightness */
fullres[x + y*w] = f < white_darkened ? f : MAX(f, dark[x + y*w]);
Anyone with good math skills can come up with a proof?
Hi,
I was playing a bit around Dual ISO with my 7D and took few shots.
Made DNG with cr2HDR. But somehow dont know the workflow ahead as if I open the dng in ACR it shows very dark like 100 iso shots.
Sharing my CR2 as well as DNG folder link here if someone can guide how to achieve the finest dual iso process.
https://www.dropbox.com/sh/xz77sv2bfow98s4/dZZmUIMnkZ
@A1ex.
Since you have the cr2 of the test I did, you can check yourself what I noticed on the latest experimental you posted.
I compared it with the last 3 prior versions and I noticed that maybe your "hot pixel fix" might be getting too aggressive already as
I am now beginning to see dark spots coming out in the shadow areas which I havent seen in the previous cr2hdr builds. Check in the area I circled.
This pic I'm posting is already compressed so maybe you can verify with the cr2 I sent.
(https://fbcdn-sphotos-g-a.akamaihd.net/hphotos-ak-ash3/1149377_355793761218081_1129444886_o.jpg)
These are cold pixels and they are no longer corrected. I'll take a look.
I think I've got it working on 5D2: https://bitbucket.org/hudson/magic-lantern/commits/b3999097eed7
Also, since the experimental converter handles almost all cases better than the old one, I've updated the converter from the first post (so cr2hdr.exe now has the latest algorithms).
So does the 5D2 have a 8-channel readout after all a1ex ?
It doesn't need an 8-channel readout, but two separate analog amplifiers. The paper says:
Quote
Unfortunately, most other Canon cameras follow a different pattern for the CMOS #0 register,
without any obviously duplicate fields.
...
We can now speculate that cameras with 8-channel readout have two separate amplifier circuits,
and you can program the ISO for each one separately.
Translation: at that time I knew how to program the cameras with 8-channel readout (because it was obvious) and I had no idea how to program the others, or whether they will work or not. Now I've discovered the missing bit from 5D2 and it works.
I still have no idea about the other cameras. I'd say the 6D has 99% chances to work, since 1% almost got it working, but I did not check the new trick on any other camera. Follow the ADTG thread (http://www.magiclantern.fm/forum/index.php?topic=6751) and give it a try.
Understood now ;D. Congrats on the find.
A1ex,
is there a compiled 5d2 dual iso module I can test?
Quote from: a1ex on August 09, 2013, 05:34:10 PM
It doesn't need an 8-channel readout, but two separate analog amplifiers. The paper says:
Translation: at that time I knew how to program the cameras with 8-channel readout (because it was obvious) and I had no idea how to program the others, or whether they will work or not. Now I've discovered the missing bit from 5D2 and it works.
I still have no idea about the other cameras. I'd say the 6D has 99% chances to work, since 1% almost got it working, but I did not check the new trick on any other camera. Follow the ADTG thread (http://www.magiclantern.fm/forum/index.php?topic=6751) and give it a try.
50D has got 2, from my pictures.
Right?
http://chdk.setepontos.com/index.php?topic=6161.180
50D, photo mode:
/* 00:00:04.078841 */ CMOS
/* 00:00:04.078911 */ 00 0004 404B548E
/* 00:00:14.214297 */ CMOS
/* 00:00:14.214376 */ 00 0094 404B549C
/* 00:00:26.551046 */ CMOS
/* 00:00:26.551116 */ 00 01B4 404B54AA
/* 00:00:47.349123 */ CMOS
/* 00:00:47.349194 */ 00 016C 404B54C6
PHOTO_CMOS_ISO_START = 0x404B548E; // CMOS register 0000 - for photo mode, ISO 100
PHOTO_CMOS_ISO_COUNT = 5; // from ISO 100 to 12800
PHOTO_CMOS_ISO_SIZE = 14; // distance between ISO 100 and ISO 200 addresses, in bytes
CMOS_ISO_BITS = 3;
CMOS_FLAG_BITS = 2;
CMOS_EXPECTED_FLAG = 0;
is_5d2 = 1;
This produces Iso1 != iso2 right after its set. Using size of 12 causes err reg not 0. Setting cmos expected flag like 5DII errors on the flag.
*HiNT. Seems the addresses when cmos updates all regs together are not what we're looking for... its when reg 0 is updated by itself.
For 50D, these are the values for ISO 200/400/800/1600?
First we need to know what bits do what. You can set the values manually, e.g. *(uint16_t*)0x404B548E = whatever. Only after you are sure about the meaning of each bit (at least the two ISO fields), you can configure the constants in dual_iso and see the errors.
For experimenting, just change the memory values manually.
On 50D they change on the middle ISOs.. like 160, 320, etc. I think somewhere at the begining of the thread I did what went with what. 04 is 100-160 tho. I went 160, 320, 640, 1250. I think I didn't do the last one.
4 I think is only 100
94 is 100 10 100
1B4 110 110 100
16C 101 101 100
So which ones are likely to be ISO.
could it be
004 000 000 100
094 010 010 100
1B4 110 110 100
16C 101 101 100
so is it iso iso xxx
Is it ok to report tests and bugs on dual iso for 5d2 here? or should I start another thread for 5d2 dual iso?
did some tests on the dual ISO and heres some quirks.
1. after activating dual iso, i can take 1 shot with it then the following shots are normal cr2. I did not turn on alternate.
I sometimes take the card out and back before I can take a dual iso shot. Sometimes reboot cam will work sometimes not.
or I will download the cr2 via usb cable. after downloading i will take another shot, they will not be in dual iso. looking at ML menu dual iso is still at ON.
2. changing the recovery ISO has a bug, after entering submenu i can change it to 3200 for example. exiting sub menu I can see the the dual iso reads 100/1600 still.
I have no problems using dual iso with my 7D.
6D photos, all ISOs starting with 100-128k.. there is no 66?
03 0000 40450E08
03 0011 40450E1A
03 0022 40450E2C
03 0033 40450E3E
03 0044 40450E50
03 0055 40450E62
03 0077 40450E74
100 0 1
100 0 10
110 0 11
100 0 100
101 0 101
111 0 111
Seems like ISO 0 ISO
50D seems ISO ISO 100
so how to set up flag bits, etc for this.. also on 50D the reg address never changes in LV.
Quote from: a1ex on August 09, 2013, 05:08:12 PM
Also, since the experimental converter handles almost all cases better than the old one, I've updated the converter from the first post (so cr2hdr.exe now has the latest algorithms).
The cr2hdr in the first post is timestamped 30/07/13?
Hi @Alex! Tried some dualiso and some birds. The converter seems to have trouble converting certain lighting conitions such bright backgrounds with smaller subjects lika flying duck for instance? The first converter managed to convert this but not the latest
https://docs.google.com/file/d/0B4tCJMlOYfirWmVDd1pZVlYwU2M/edit?usp=sharing
Is the conversion dependent on motif? Have some pictures of an eagle , rather tiny in the original but could be cropped after conversion if converted. Seem the next latest of the converters could do some of the files the other two converters can,t handle but not all of them.
*update. Added the exiftool, didn,t think it was necessary but seems to convert ok now, still not able to convert linked example though.
Another strange thing is that the converter creates an extra dng_original file when exiftool included? Exact same size as the converted dng-file.
*update2. The converter gives random results. Sometimes it doesn,t recognize the cr2,s as interlace iso, but retrying, suddenly it works?
(http://s15.postimg.org/ni9ccddsr/extra.png)
Thanks
//D
Is there crop mode functionality on the 5DM III shooting video?
Whats the difference to just use the RAW function in 50fps mode in combination with the older HDR function that records two different frames. I mean...wouldn't it help the antialiasing a bit? The motionblur now may be the new old problem...
i got a 5d mark III.
hello ml team!
is it possible to name the dual iso shots automatically (e.g. DI_99A6430.CR2 .. because in the mac finder it is not possible to see which cr files are dual isos and which are not.. ?
a really satisfied ml user.. :)
Wow, not a single sample from 5D2?!
See you in roughly 2 weeks (maybe earlier if I get net access).
6D actually looks doable for LV and photo, maybe need to try to set dual ISO manually. I'm just not sure if its
ISO1 0 ISO2 in ISO1 reg address
or
ISO2 0 ISO1 in ISO2 address, etc.
I guess I have to experiment.
What happens if you set *(uint16_t*)40450E08 = 0x50 or 0x05?
I'd say ISO_BITS=4, FLAG_BITS=0.
I just took off some of the checks and got some interesting raw shots. It looks striped. Looking at the CR2s and then will just try to set it like that.
I kept getting error 1 and reg = 3... then got error 5 that prev iso was less than 10.
Here is one from 6D.
http://www.qfpost.com/file/d?g=KSwEmqWs8 (http://www.qfpost.com/file/d?g=KSwEmqWs8)
Quote from: Danne on August 10, 2013, 03:53:21 PM
Hi,
When I do the conversion the exiftool always say didn't work for some reason ???
Ok, after conversion the pic looks normal. So now I'll try setting the 0x50, 0x05, etc. It *looks* like its working but I'd need some scene to obviously test on. Both in LV and photo it does the same thing. Opening original CR2 is filled with black horizontal lines... is this how it should be?
0x50 turns everything solid black.... bummer, i should try 0x70 but from filming in lv a second ago it does appear to be working. The shadows lit up when I turned on dual iso... of course have the undulating black lines in the preview. The way it could be not working if its just set iso 1600 and the black really is just black.
Yep, it's working.
My dcraw doesn't know the black border sizes for 6D; maybe fix it and send a patch to Dave Coffin? or maybe the latest one already fixed it.
Some checks may not be needed; they are mostly to make sure the address doesn't change over time (and if it does, to prevent unwanted side effects).
I think the latest latest dcraw it was fixed. First version of it and raw-therapee I could see the black borders and before the meta data method counted them like that.
So bitbucket took the upload and never listed it, now upload box is gone again... sdlfjasdljfklasjdfklasdjflasjdfl
http://www.filedropper.com/6ddualiso-fullpack
Quote2. changing the recovery ISO has a bug, after entering submenu i can change it to 3200 for example. exiting sub menu I can see the the dual iso reads 100/1600 still.
This is not a bug, max analog ISO in 5D2 seems to be 1600. Maybe I should remove the higher ISOs from menu.
QuoteThe converter seems to have trouble converting certain lighting conitions such bright backgrounds with smaller subjects lika flying duck for instance?
I'm afraid it needs robust linear regression in estimate_iso (plain least squares is too sensitive to outliers). Can you find a simple method for doing that? Exif info won't do, I need precise fitting.
So for 50D how to handle the binary 100 at the end? Is that the CMOS expected flag? I think 100 is 4dec
Yeah, the post from Elmeri makes sense. 3 bits for flag and 3 for ISO?
It doesn't do ISO FLAG ISO FLAG that way? It will work just 3, 3 and expected flag of 4?
The current algorithm only knows ISO ISO FLAG, and the 50D seems to follow this pattern. The flag is not really used (it's just preserved and used for the internal checks).
The exact meaning of the ISO field doesn't really matter (it's copied from the higher ISO to the lower ISO). So far it had ascending values. As long as it repeats, it should work.
Will the 5D2 be able to do dual-iso raw video like the mk3 or just too old hardware?
If I read right that the mk3 can do that :)
Working on the 5D2 fine for me. I'm trying to figure out how to put it to the test. I'll post some samples in the coming days.
Quote from: Francis on August 11, 2013, 10:09:28 PM
Working on the 5D2 fine for me. I'm trying to figure out how to put it to the test. I'll post some samples in the coming days.
Will test any builds posted for 5D2. I suppose this thread's opening post should be updated, along with the title?
5DII has multiple regs in LV? 50D probably won't work in LV as its just one REG, I can try to set the spacing to 0.
It doesn't do anything in LV so far but photo works.
cr2hdr doesn't take wildcards in windows but on linux its working, odd.
Hi, just installed dual iso andy600 build on my 50D, but can not find any CR2HDR tool for Mac OSX. Is there any CR2HDR or alternative tool for mac?
Hey Alex, when is the new raw2dng with new algorithm for dual isos coming? just shot around 140 gb pure dual iso raw video crazy stuff, haha.
Building the cr2hdr tool in a mac environment is super easy. Just follow the instructions for downloading the source repository and a prebuilt tool-chain in the 'getting started with development' thread. Then make in the dual_iso modules folder. It builds both the module and cr2hdr.
Here is the binary built as of Aug 11th. cr2hdr for Mac (http://avlmotofoto.com/cr2hdr)
This is a command line version. I would suggest making a symbolic link to where ever you save it in /usr/bin so you can run it in whatever folder you save your images. You also are going to need to install dcraw. I installed it using MacPorts.
@a1ex
did a test similar to the 7D this one with the 5D2.
100_1600
(https://fbcdn-sphotos-d-a.akamaihd.net/hphotos-ak-ash3/1150633_357070077757116_1735920732_o.jpg)
100 iso only no adjustments.
(https://fbcdn-sphotos-h-a.akamaihd.net/hphotos-ak-prn2/1102615_357072681090189_124992235_o.jpg)
tell me if you need the cr2 files.
Hi Alex look what happens in many of my photos
Raw http://we.tl/bcZHyu0dW1
http://www.flickr.com/photos/99061000@N08/9490490087/lightbox
Quote from: 1% on August 11, 2013, 10:47:19 PM
5DII has multiple regs in LV? 50D probably won't work in LV as its just one REG, I can try to set the spacing to 0.
It doesn't do anything in LV so far but photo works.
cr2hdr doesn't take wildcards in windows but on linux its working, odd.
Many compliments for this astounding achievement 1% ! For the LV problem, if it can be of help, I can dig inside rom to find more ADTG regs, if this route has not been squeezed dry yet...
Quote from: Francis on August 12, 2013, 07:09:58 AM
Building the cr2hdr tool in a mac environment is super easy. Just follow the instructions for downloading the source repository and a prebuilt tool-chain in the 'getting started with development' thread. Then make in the dual_iso modules folder. It builds both the module and cr2hdr.
Here is the binary built as of Aug 11th. cr2hdr for Mac (http://avlmotofoto.com/cr2hdr)
This is a command line version. I would suggest making a symbolic link to where ever you save it in /usr/bin so you can run it in whatever folder you save your images. You also are going to need to install dcraw. I installed it using MacPorts.
Thanks a lot, but I get error:
studios-mac-pro:canon studio$ ./cr2hdr IMG_0155.CR2
Input file : IMG_0155.CR2
sh: dcraw: command not found
got Xcode and gcc tools, I will try to compile on local
WEeeeeeeeeee!
Works on the 6D
*happy.
Thank you all.
Quote from: jgerstel on August 12, 2013, 12:48:35 PM
Thanks a lot, but I get error:
studios-mac-pro:canon studio$ ./cr2hdr IMG_0155.CR2
Input file : IMG_0155.CR2
sh: dcraw: command not found
got Xcode and gcc tools, I will try to compile on local
You need dcraw. As I said, I used MacPorts to install it. Macports (http://www.macports.org/) is a command line based package installation system for OSX that is similar to Linux package systems like APT.
Any chance of dual ISO for the 50D?
Hi
i just installed the latest build from Andy600 tragic lantern for dual ISO on canon 50d
and this is what i got
is this image is as it supposed to be?
the horizontal zebra lines?
or am i doing something wrong?
thank you anybody for quick answer!
(http://i024.radikal.ru/1308/46/ac18eeefa2de.jpg)
You haven't read how the Dual ISO feature works and how to process it, right?
oh yeah sorry i have read the first post
i downloaded that 3 files tools
and i try to drag and drop CR2 file on the cr2hdr.exe icon
but the command dial says "it seems like it is not interlaced ISO"
and shuts down...
here what i got
(http://s019.radikal.ru/i612/1308/e9/407824467458.jpg)
Conversion works for me with my 6D files but i am not sure about the en result.
@A1ex:does the cr2hdr need to be changed with data from other cameras?
I would love to upload the dual iso file now but it's not possible at the moment.
modified
ah no it works fine! it just doesnt pick up underexposed pictures.
(http://s003.radikal.ru/i204/1308/0e/2e24f283cdad.jpg)
Quote from: Silkway on August 12, 2013, 05:01:47 PM
oh yeah sorry i have read the first post
i downloaded that 3 files tools
and i try to drag and drop CR2 file on the cr2hdr.exe icon
but the command dial says "it seems like it is not interlaced ISO"
and shuts down...
[spoiler]here what i got
(http://s019.radikal.ru/i612/1308/e9/407824467458.jpg)[/spoiler]
Did you shoot in live view? If so, it doesn't work. Use photo mode only.
Quote from: Andy600 on August 12, 2013, 05:51:25 PM
Did you shoot in live view? If so, it doesn't work. Use photo mode only.
no dear sir Andy600
i shot via viewfinder of canon 50d
but it seems like there is not enough info for processing on too dark images. thats a reason it didnt convert. but other images who shot on bright areas with good highlights it is just perfect working!
Don,t shoot in too extreme situations. I got the same problems when shooting birds in very bright areas such as skies. Otherwise it,s fine
Can I please request that the EXE for CR2HDR be updated in the OP? Seems like it's an earlier build.
I've noticed green cast in shadows and highlights after converting dual iso cr2 into dgn on Canon 50D.
Normal case:
(http://i.imgur.com/4LAE0J5.jpg)
without Dual ISO:
(http://i.imgur.com/WiFTupz.jpg)
Extreme case:
cr2hdr.exe:
(http://i.imgur.com/htMTg7g.jpg)
cr2hdr_exp.exe:
(http://i.imgur.com/8MSQ4yg.jpg)
Is there some trick to remove green cast in ACR (in normal, not "extreme" cases at least)? Split toning won't do the trick. Setting WB sets it either for highlights and shadows or for middle parts.
Yea, I had some problem with green cast on 50D. I think some of the problem might be that 50D uses iso 100, 320, 640, etc NOT 100, 200, 400, 800, etc. Have to check it out and port to 600D and *maybe* eosM if the module loads with CMOS logging enabled.
Hi all.
Really new to this and I have a "dumb" question. I'm on Mac (Mountain Lion) I have the source code and pre-built toolchain. Could someone point me where to go from there. Not quite sure what to do with it to use my 5D2 .cr2 files and get cr2hdr working.
Quote from: REDcineX on August 13, 2013, 01:30:30 AM
Hi all.
Really new to this and I have a "dumb" question. I'm on Mac (Mountain Lion) I have the source code and pre-built toolchain. Could someone point me where to go from there. Not quite sure what to do with it to use my 5D2 .cr2 files and get cr2hdr working.
I just wrote up something http://www.magiclantern.fm/forum/index.php?topic=7645
Error: Port hg not found
oh yeah indeed!
there is a green stuff going on on canon 50d dual iso...
and i dont know if it is suitable hmm...
in some way its not so bad of course
but would be nice to see the solution...
Quote from: REDcineX on August 13, 2013, 03:12:20 AM
Error: Port hg not found
My mistake.
port install mercurial
Working great on my 6D - well done all!
If you look closely in extreme areas you can see some fringing/jaggies/moire - see my 100% crop here -> http://img19.imageshack.us/img19/3097/tbje.jpg
BUT it is a small price to pay for the results.
Which ISO did you set in camera and which in ML menu?
Aww..
"-bash: cr2hdr: command not found"
To back track a little further. When I do this line and of course the same line for raw2dng:
"Type 'make' then 'sudo ln -s cr2hdr /usr/bin/cr2hdr'."
Terminal says: "make: *** No targets specified and no makefile found. Stop."
Download/build it. and run ./cr2hdr
600D photo mode is working. 600D has real iso 3200. LV too.
I can make inside the raw_rec and dual_iso module folder and modules + binaries come out.
The only other thing I might have done wrong was download the toolchain (mac version), renamed it gcc, and placed it inside the magic-lantern folder in Downloads. I used all your commands, Francis, after that to keep it consistent.
Sounds like you weren't in the right folder to build cr2hdr. You have to be in the directory containing the dual ISO code. It is in magic-lantern/modules/dual_iso
When I type "./magic-lantern/modules/dual_iso"
Terminal replies "-bash: ./magic-lantern/modules/dual_iso: is a directory"
I type "make" after cd to /dual_iso
Terminal replies "[ GCC ] cr2hdr
[ GCC ] cr2hdr
[ GCC ] cr2hdr
~/Downloads/magic-lantern/gcc/bin/arm-none-eabi-gcc-4.7.3 -Os -mthumb-interwork -march=armv5te -D__ARM__ -Wp,-MMD,./dual_iso.d -Wp,-MT,dual_iso.o -nostdlib -fomit-frame-pointer -fno-strict-aliasing -I../../include -Wall -W -Wno-unused-parameter -Wno-unused-function -Wno-implicit-function-declaration -Wno-missing-field-initializers -Wno-format -Wdouble-promotion -ffast-math -fno-trapping-math -fsingle-precision-constant -fshort-double -std=gnu99 -Winline -I../../platform/all -I../../platform/all/include -I../../src -I../../picoc -DMODULE -DMODULE_NAME=dual_iso -c -o dual_iso.o dual_iso.c
/bin/sh: /Users/Admin/Downloads/magic-lantern/gcc/bin/arm-none-eabi-gcc-4.7.3: No such file or directory
I enter "sudo ln -s cr2hdr /usr/bin/cr2hdr"
Terminal responds "ln: /usr/bin/cr2hdr: File exists"
I know previously we said that 'auto' video mode would not be possible with dual iso as iso changes were needed.
Is it possible to have dual iso with 'auto video' exposure mode, but just changing shutter-speed/aperture? This would be a massive, massive help for my line of work!
In other news, my 5D3 arrived today! Just waiting for Transcend 1000x to arrive on Monday now.
Quote from: REDcineX on August 13, 2013, 07:29:56 AM
/bin/sh: /Users/Admin/Downloads/magic-lantern/gcc/bin/arm-none-eabi-gcc-4.7.3: No such file or directory
You need to make sure you have the arm toolchain https://launchpad.net/gcc-arm-embedded
And edit your makefile to point to the location where you store it.
Take further discussion to a developing help thread please. Here are some threads that should help.
http://www.magiclantern.fm/forum/index.php?topic=991.0
http://www.magiclantern.fm/forum/index.php?topic=6783.0
http://www.magiclantern.fm/forum/index.php?topic=6425.0
600D, its not perfect. I saw some artifacts.
http://www.filedropper.com/600ddualiso
Quote from: 1% on August 13, 2013, 05:59:48 PM
600D, its not perfect. I saw some artifacts.
http://www.filedropper.com/600ddualiso
I just tried it on my 600D and it works fine! Just amazing!
It's so impressive to see how much progress has been made in the last time. Such insane stuff.
First 14-Bit RAW video and now almost 3 steps more dynamic range! What's next? :o
This really is MAGIC!
Thank you very much for posting this build. It is freaking awesome that Magic Lantern gets developed since 2009 and still it looks like the biggest treasures inside the cameras aren't even found yet. Who would have thought that any DSLR can handle RAW video? Even 2K RAW video! Or that a Canon camera could get the dynamic range of a Nikon sensor?
I am really curious to see how this firmware add-on will change in the future. It is by far the coolest thing you can download from the internet.
Quote from: 1% on August 13, 2013, 05:59:48 PM
600D, its not perfect. I saw some artifacts.
http://www.filedropper.com/600ddualiso
If this works on the 600D doesnt it mean the 550D should have the ability as well?
Quote from: CFP on August 13, 2013, 07:58:35 PM
First 14-Bit RAW video and now almost 3 steps more dynamic range! What's next? :o
Rumor has it they have cracked 3D Video, should be out in a week or two.
I understand, that dual iso is for RAW recording only.
Is there a chance it will become available for H264 encoded recordings later, or is this not possible?
The raw video workflow is really time consuming, and if dual iso was possible with H264 directly in camera, this would be a fantastic time saver..
Quote from: mk11174 on August 13, 2013, 08:19:25 PM
If this works on the 600D doesnt it mean the 550D should have the ability as well?
Read the last 1% quote on our old 550D raw video thread: http://www.magiclantern.fm/forum/index.php?topic=5582.new#new
It seems possible... we must investigate asap ! ;D
Hi,
For me it is also new, but got dual iso tool working. Please use next guide for general setup:
Also install XCode and command line tools:
Xcode menu > Preferences > Downloads > choose "Command line tools" > Click "Install" button
Then ML stuff using next guide:
http://magiclantern.wikia.com/wiki/Magic_Lantern_Development_on_Mac
Then you also need to download dcraw and exiftool:
http://www.sno.phy.queensu.ca/~phil/exiftool/
http://lightzoneproject.org/content/mac-version-new-dcraw-available
Then download sources from Bitbucket, I use Andy's.
Goto dual iso dir and type make. It will give the cr2hdr tool.
If you only need cr2hdr and don't want arm tooling (it is complaining about arm then), you can remove dual_iso.c. Make will give some error, but the cr2hdr will be created anyway and works :-)
root# cr2hdr IMG_0155.CR2
Input file : IMG_0155.CR2
Full size : 4832 x 3228
Active area : 4770 x 3177
White level : 13327
Black borders : 62 left, 51 top
Black level : 995
ISO pattern : dBBd GBRG
Estimating ISO difference...
ISO difference : 4.03 EV (1628)
Black delta : 2.30
Interpolation : mean23-chroma5x5-contrast
Matching brightness...
Looking for hot pixels...
Hot pixels : 18316
Full-res reconstruction...
Alias filtering...
Dark chroma filtering...
Building contrast map...
Dilating contrast map...
Smoothing contrast map...
ISO overlap : 5.0 EV (approx)
Black adjust : 1
Output file : IMG_0155.DNG
1 image files updated
Cheers
Jorgen
Hi
Tested the 50D "green" issue. Looks like Windows version has the " green " issue for 50d image, but custom build on Mac shows normal image. Please see comparison of same image
green Windows build:
http://www.gerstel.org/images/canon/dual_iso/IMG_0155win.jpg
(http://www.gerstel.org/images/canon/dual_iso/IMG_0155win.jpg)
OK Mac 50D build:
http://www.gerstel.org/images/canon/dual_iso/IMG_0155mac.jpg
(http://www.gerstel.org/images/canon/dual_iso/IMG_0155mac.jpg)
50D Mac cr2hdr Tool can also be downloaded:
http://www.gerstel.org/images/canon/dual_iso/cr2hdr
Then you also need to download dcraw and exiftool:
http://www.sno.phy.queensu.ca/~phil/exiftool/
http://lightzoneproject.org/content/mac-version-new-dcraw-available
Hi A1ex, 1%, Audionut,
I get this horrible black border and overall tone (especially on the skin) when I convert RAW to DNG using cr2hdr_exp:
Sample: http://www.flickr.com/photos/89977905@N05/9503180650/
Picture processed on ACR 8.1.
Any chance of this on the 60D - can I help investigate?
Quote from: a1ex on August 07, 2013, 11:34:12 PM
Another update regarding aliased areas, for cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe). This time I've tried a very difficult shot - the CR2 from daancalo2013:
Hi A1ex,
Is this is the latest version of cr2hdr or the one on the OP?
Quote from: 1% on August 13, 2013, 05:59:48 PM
600D, its not perfect. I saw some artifacts.
http://www.filedropper.com/600ddualiso
This one works pretty well. Very nice work!
Quote from: Canon eos m on August 14, 2013, 05:01:22 PM
Hi A1ex,
Is this is the latest version of cr2hdr or the one on the OP?
I too would like to know ;)
Try both and see. :)
I haven't been following this from the beginning, but just tried it and the latest version seems pretty good.
However, I guess because my brain works strangely, intuitively I would use it the other way around - set 100 ISO in the Dual ISO menu and shoot the shot at ISO1600. So I tried this and although the results were reasonable, they were quite different and not as good as shooting 1600/100 with exactly the same aperture and SS.
Is this to be expected? Simplistically I would have though it would be identical since it's just the ordering that is different.
Anyone compared dualISO with a regular 2 expo HDR picture ?
Cause dualISO is fine, you don't need a tripod (!) but it costs in aliasing and resolution.
As read in posts, dualISO enhance pic's DR.
Anyone knows how DR can be estimated from a HDR (ISO shift, not aperture shift) of 2/3/5 pics ?
On the video side, would be interesting to test DualISO with Mosaic Engineering filter...
About aliasing mainly.
Update: Spent a few hours in Photomatix Pro, mixing 2 or 3 expo "classical" HDRs..... (ISO shift)
i never get even close to the result of DualIso....
How are mixed the 2 expo in cr2hdr ???
It's a fusion, of course, not a tone map but i sould miss something !
Someone knows ?
Quote from: teo770 on August 15, 2013, 12:35:27 AM
On the video side, would be interesting to test DualISO with Mosaic Engineering filter...
About aliasing mainly.
Dial ISO sample I posted here is on 5d2 WITH mosaic filter.
http://www.magiclantern.fm/forum/index.php?topic=7139.msg66196#msg66196
Quote from: teo770 on August 15, 2013, 12:35:27 AM
Anyone compared dualISO with a regular 2 expo HDR picture ?
Cause dualISO is fine, you don't need a tripod (!) but it costs in aliasing and resolution.
I would also say for me it looks like not going to be good enough for a high quality landscape shot, but perhaps fine for many other applications. Now if I could get two full res raw files, one at ISO100 and one at ISO1600 from a single exposure, that would be a real game-changer for times when multiple exposures won't work. Not sure what you are doing in Photomatix, but the DualISO will reduce noise
I will likely use it when I would normally blow out highlights rather than when the highlights are my primary concern.
Like many of these techniques you need to figure out when it's going to work for you.
Photomatix is something else, if you want true extended dynamic range you should use Enfuse(as a plugin for LR) to add up the exposures.
Quote from: ed.jenner on August 15, 2013, 03:13:44 AM
I would also say for me it looks like not going to be good enough for a high quality landscape shot,
OK, quoting myself, but just trying this on the 7D it's really good. Seems better than on the 5DII with whatever workflow I used yesterday. Need to test more on the 5DII, it might be something I'd use for 'serious' shooting where I might want to make a 20x30 print.
Quote from: ed.jenner on August 15, 2013, 03:13:44 AM
Not sure what you are doing in Photomatix, but the DualISO will reduce noise
Just to test, I used the HDR software I prefer and the two shot option is better (without any tone mapping) as expected.
I'm having a little problem with getting the colors to come out right, but it's not a showstopper, just a little extra PP.
The photo side progress has been good. I'm still getting mixed results, the same as @ed.jenner. Switching the iso's weirds things up. I always end up with an image that is about 3 stops underexposed. The shadows come back pretty nice most of the time, and the highlights are great.
Here's the best I can get with video. Mind you ,I'm on a mac, and not smart enough to run the command line stuff, so I converted this with son of batch app for osx. I toyed with the exposure a bit, the first one has no clipped highlights in the live view preview, the second one has some in the clouds. I boosted both the shadows to 100 and held the highlights to -100 just to see how far I can push it. Now if someone can help make the upgrades that a1ex made to the cr2hdr to the video end, we would have something amazing!!!
Quote from: NateVolk on August 15, 2013, 06:23:15 AM
Here's the best I can get with video. Mind you ,I'm on a mac, and not smart enough to run the command line stuff, so I converted this with son of batch app for osx. I toyed with the exposure a bit, the first one has no clipped highlights in the live view preview, the second one has some in the clouds. I boosted both the shadows to 100 and held the highlights to -100 just to see how far I can push it. Now if someone can help make the upgrades that a1ex made to the cr2hdr to the video end, we would have something amazing!!!
i did about 140 gb of dual iso video material, i haven't processed it yet because of the outdated raw2dng. waiting for the upodate too.
First off, my apologies if this is covered elsewhere. I've dug and dug, and can't find anywhere this is explicitly answered: is DualISO working on the 5DII or 50D for video?
I've gotten it to work well with stills (though I have the same issue another poster reported that no matter what I set the recovery ISO to, it will not go above 1600 on my 5DII or 50D) but I see no difference between RAW video shot with DualISO on and off for either the 5DII or 50D. Since I'm using a mac, exclusively, it could also be that Son of Batch just doesn't deal with DualISO raw video -- Son of Batch is the only RAW -> DNG converter for mac I've seen that supposedly supports DualISO.
That's all for now. I greatly *greatly* appreciate your time, and my hat is off to all the folks working on ML for the amazing work to date on this project. ML RAW now gives me a way to process and grade that makes sense to my photo-centric brain. ;D
Quote from: packgrad2k1 on August 15, 2013, 06:39:12 AM
First off, my apologies if this is covered elsewhere. I've dug and dug, and can't find anywhere this is explicitly answered: is DualISO working on the 5DII or 50D for video?
I've gotten it to work well with stills (though I have the same issue another poster reported that no matter what I set the recovery ISO to, it will not go above 1600 on my 5DII or 50D) but I see no difference between RAW video shot with DualISO on and off for either the 5DII or 50D. Since I'm using a mac, exclusively, it could also be that Son of Batch just doesn't deal with DualISO raw video -- Son of Batch is the only RAW -> DNG converter for mac I've seen that supposedly supports DualISO.
That's all for now. I greatly *greatly* appreciate your time, and my hat is off to all the folks working on ML for the amazing work to date on this project. ML RAW now gives me a way to process and grade that makes sense to my photo-centric brain. ;D
Then the answer should be pretty straightforward. No Dual ISO in video mode on the 5D Mark II, which has been said many times before now ;)
Quote from: NateVolk on August 15, 2013, 06:23:15 AM
The photo side progress has been good. I'm still getting mixed results, the same as @ed.jenner. Switching the iso's weirds things up. I always end up with an image that is about 3 stops underexposed. The shadows come back pretty nice most of the time, and the highlights are great.
Here's the best I can get with video. Mind you ,I'm on a mac, and not smart enough to run the command line stuff, so I converted this with son of batch app for osx. I toyed with the exposure a bit, the first one has no clipped highlights in the live view preview, the second one has some in the clouds. I boosted both the shadows to 100 and held the highlights to -100 just to see how far I can push it. Now if someone can help make the upgrades that a1ex made to the cr2hdr to the video end, we would have something amazing!!!
Very nice image. I can see some moire and maybe some fringe in the top of the cabane...
Quote from: manniac on August 13, 2013, 11:25:40 PM
I understand, that dual iso is for RAW recording only.
Is there a chance it will become available for H264 encoded recordings later, or is this not possible?
The raw video workflow is really time consuming, and if dual iso was possible with H264 directly in camera, this would be a fantastic time saver..
I am believe that the compression would kill the fine detail needed to reconstruct the HDR image. Also there is a current HDR solution for h264 shooting, with temporal errors instead of spatial errors.
However if the interlacing could be conserved through compression I'm sure you could comb the fields out for processing.
I thought of shooting XXiso/100 too. In the day time yea, 100/1600 but at night you have the opposite problem and want to expose backwards... ie you still get lights, etc. But I think the converter isn't set up for it.
Quote from: arrinkiiii on August 15, 2013, 09:38:09 AM
Very nice image. I can see some moire and maybe some fringe in the top of the cabane...
Yeah, on the computer it looks pretty bad. Straight up RAW is still the best option. You could process the dual iso better to remove the fringing, but the amount of work added is huge.
The stills converter works if you flip the iso's, but it produces very different results from the 100/1600 at the same settings. I'm still having a real hard time understanding how to properly expose the image. Any tips?
7D test shots on dual ISOs.
Here are four shots, unlighted room with res chart and open window noontime. Processed the shots to get a "good" image indoors and try to recover as much info from the window. I was trying see the noise patterns and also if i reverse the recovery if there is a difference.
Took 4 shots;
1. 100 iso
2. 1600 iso
3. 100-1600 dual iso
4. 1600-100 dual iso
shot 1 - 100 iso
https://fbcdn-sphotos-c-a.akamaihd.net/hphotos-ak-prn2/1150512_358773894253401_433820467_o.jpg
shot 2 - 1600 iso
https://fbcdn-sphotos-e-a.akamaihd.net/hphotos-ak-frc3/1146262_358773880920069_1154231185_o.jpg
Shot 3 - 100_1600 iso
https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-ash3/1172292_358773887586735_1710098679_o.jpg
shot 4 - 1600_100 iso
https://fbcdn-sphotos-g-a.akamaihd.net/hphotos-ak-prn1/1172556_358774064253384_781772650_o.jpg
Great test Ted, can you upload the CR2 files for it? (or at least the 100/1600 one).
Also @Canon eos m: need the CR2 from here: http://www.magiclantern.fm/forum/index.php?topic=7139.msg66734#msg66734
I've recompiled cr2hdr.exe and re-uploaded it, just in case. No code changes. Do you still have issues?
Quote
Update: Spent a few hours in Photomatix Pro, mixing 2 or 3 expo "classical" HDRs..... (ISO shift)
i never get even close to the result of DualIso....
Try Zero Noise: http://www.guillermoluijk.com/software/zeronoise/index.htm
It should use a similar blending method (though I didn't try it, I've only read their articles and borrowed the technique).
Quote from: a1ex on August 16, 2013, 09:05:33 AM
I've recompiled cr2hdr.exe and re-uploaded it, just in case. No code changes. Do you still have issues?
That fixed it. Ta.
So which algo was the best? From the repo I get v-23-contrast, from the EXE in the first post I had v22-smooth or something like that.
Quote from: a1ex on August 16, 2013, 09:05:33 AM
Great test Ted, can you upload the CR2 files for it? (or at least the 100/1600 one).
Also @Canon eos m: need the CR2 from here: http://www.magiclantern.fm/forum/index.php?topic=7139.msg66734#msg66734
I've recompiled cr2hdr.exe and re-uploaded it, just in case. No code changes. Do you still have issues?
Try Zero Noise: http://www.guillermoluijk.com/software/zeronoise/index.htm
It should use a similar blending method (though I didn't try it, I've only read their articles and borrowed the technique).
Looking for it. Will send you soon as I track it down or will send another CR2 with similar blackening ... have a lot with this same issue.
Quote from: a1ex on August 16, 2013, 09:05:33 AM
I've recompiled cr2hdr.exe and re-uploaded it, just in case. No code changes. Do you still have issues?
Some issue with ch2hdr.exe download from the OP - the black command prompt screen flashes for a second and then dies out. The ch2hdr_exp.exe works fine.
Zero Noise also has problems is asking for a missing MSSTDFMT.DLL file.
Whilst batch converting mac im getting this:
Input file : /Users/doyle/Desktop/DCIM/100EOS5D/IMG_9937.CR2
Full size : 5792 x 3804
Active area : 5634 x 3753
White level : 15713
Black borders : 158 left, 51 top
Black level : 973
ISO pattern : BddB GBRG
Estimating ISO difference...
ISO difference : 4.22 EV (1860)
Black delta : -8.79
Interpolation : mean23-chroma5x5-contrast
Matching brightness...
Looking for hot pixels...
Hot pixels : 2232
Full-res reconstruction...
Alias filtering...
Dark chroma filtering...
Building contrast map...
Dilating contrast map...
Smoothing contrast map...
ISO overlap : 4.8 EV (approx)
Black adjust : -5
Output file : /Users/doyle/Desktop/DCIM/100EOS5D/IMG_9937.DNG
sh: -c: line 0: unexpected EOF while looking for matching `"'
sh: -c: line 1: syntax error: unexpected end of file
Exiftool didn't work
But dng's have been created and seem to be ok as a file that was updated? noticed it does the same on windows too when batch converting,
Cheers guys, and great work again!
Quote from: a1ex on August 16, 2013, 09:05:33 AM
Great test Ted, can you upload the CR2 files for it? (or at least the 100/1600 one).
@a1ex
emailed cr2
ted
Does Dual Iso still only work for Still images on Eos 7D?
Quote from: Ozgur on August 16, 2013, 08:00:42 PM
Does Dual Iso still only work for Still images on Eos 7D?
yes.
Quote from: a1ex on August 16, 2013, 09:05:33 AM
I've recompiled cr2hdr.exe and re-uploaded it, just in case. No code changes.
Just wondering if/when there will be a build that takes command line switches? Or does it already do this and I am missing it?
Cheers....
--
K.C.
5D mkII test shots on dual ISOs.
The latest cr2hdr works in removing greenish cast from shadow areas.
Unlighted room with res chart and open window noontime. Processed the shots to get a "good" image indoors and try to recover as much info from the window. Compared the noise patterns and also tested with VAF filter to control color aialing and moire.
3 shots;
1. 100 iso
2. 100-1600 dual iso
3. 100-1600 dual iso with VAF
1.100 iso
https://fbcdn-sphotos-e-a.akamaihd.net/hphotos-ak-ash3/1150526_359389720858485_1133039032_o.jpg
100-1600 dual iso
https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-prn2/1146938_359389874191803_1768168050_o.jpg
3. 100-1600 dual iso with VAF
https://fbcdn-sphotos-c-a.akamaihd.net/hphotos-ak-frc3/1093863_359389800858477_834816052_o.jpg
Could someone point me to the link to the latest cr2hdr.exe update?
Quote from: Canon eos m on August 17, 2013, 03:59:08 AM
Could someone point me to the link to the latest cr2hdr.exe update?
first post.
Ted did that already but it doesn't work for me - whereas the previous on ch2hdr_exp.exe works just fine.
Quote from: ted ramasola on August 17, 2013, 04:20:10 AM
first post.
The file does not convert instead cr2hdr.exe generates this tmp file:
Filename: C:\Users\USer\Downloads\cr2hdr August 16 2013 download\cr2hdr\_G1A8405.CR2
Timestamp: Wed Aug 14 18:42:17 2013
Camera: Canon EOS 5D Mark III
ISO speed: 12800
Shutter: 1/98.7 sec
Aperture: f/2.5
Focal length: 35.0 mm
Embedded ICC profile: no
Number of raw images: 1
Thumb size: 5760 x 3840
Full size: 5920 x 3950
Image size: 5796 x 3870
Output size: 5796 x 3870
Raw colors: 3
Filter pattern: RGGBRGGBRGGBRGGB
Daylight multipliers: 2.125175 0.943985 1.338680
Camera multipliers: 1259.000000 1024.000000 3666.000000 1024.000000
Quote from: Canon eos m on August 17, 2013, 01:50:08 PM
The file does not convert instead cr2hdr.exe generates this tmp file:
Filename: C:\Users\USer\Downloads\cr2hdr August 16 2013 download\cr2hdr\_G1A8405.CR2
Timestamp: Wed Aug 14 18:42:17 2013
Camera: Canon EOS 5D Mark III
ISO speed: 12800
Shutter: 1/98.7 sec
Aperture: f/2.5
Focal length: 35.0 mm
Embedded ICC profile: no
Number of raw images: 1
Thumb size: 5760 x 3840
Full size: 5920 x 3950
Image size: 5796 x 3870
Output size: 5796 x 3870
Raw colors: 3
Filter pattern: RGGBRGGBRGGBRGGB
Daylight multipliers: 2.125175 0.943985 1.338680
Camera multipliers: 1259.000000 1024.000000 3666.000000 1024.000000
make sure the folder where the cr2hdr.exe and the cr2 files have a simple file name, no dashes and special characters also try to move the said folder to the root of the drive.
Quote from: ted ramasola on August 17, 2013, 07:06:30 PM
make sure the folder where the cr2hdr.exe and the cr2 files have a simple file name, no dashes and special characters also try to move the said folder to the root of the drive.
Worked!
has there been any simple tutorial or video help, for this? I'm getting single pics with lines, new to this, not to ML though, HELP!!!
my tmp file is empty, as well
Quote from: sdesign on August 18, 2013, 04:36:06 AM
has there been any simple tutorial or video help, for this? I'm getting single pics with lines, new to this, not to ML though, HELP!!!
The single pics (cr2) with lines that your are getting is the dual iso (interlaced iso), you need to click and drag to cr2hdr.exe and he will give you dng's, already with the dual iso merged.
The dng´s must be in the same folder than the cr2hdr.exe
Attended a wedding and tested the duel iso for my 50D. Some beautiful dynamic range.
(https://dl.dropboxusercontent.com/u/9746153/Site%20Images%20%26%20Files/duels.jpg) (https://dl.dropboxusercontent.com/u/9746153/Site%20Images%20%26%20Files/duell.jpg)
When nighttime hit, I tried to see if I could do something like 800/12800 but even though it was selected, it still defaulted back to 800/1600. Maybe a bug? (I know it would be incredi-noise, but I just wanted to see how bad it would be).
Dual iso works with "analog" ISO only. 12800 is definitely digital
Hi everyone and thx for this feature.
I use ML from many month and i take recently a new feature for Dual ISo.
But i have a problem.
When i can put my CR2 on cr2hdr.exe
Many of them don't work , the windows open just 1 second and shutdown.
Why a lot of my Cr2 work but not everything of this ?
( sorry for my english .... )
Quote from: Asufeld_ins on August 18, 2013, 11:17:05 AM
Hi everyone and thx for this feature.
I use ML from many month and i take recently a new feature for Dual ISo.
But i have a problem.
When i can put my CR2 on cr2hdr.exe
Many of them don't work , the windows open just 1 second and shutdown.
Why a lot of my Cr2 work but not everything of this ?
( sorry for my english .... )
Please look for post #633, #644 and #645 for help.
Best is to save the file in the root director. Name the directory cr2hdr. Download the zip in the directory and unzip. Save the cr2 files from the camera in the cr2hdr unzip directory where the cr2hdr.exe files have been extracted. It should work without a problem. :)
Thx for answers
I read it before you post for my.
I have extract on c:/ch2hdr
and i taking ch2hdr_exp.exe too.
I clear my TMP file
And is the same problem.
My DSLR is a 50D. Have a feature ( ch2hdr ) Just for it or is the same for all ?
Quote from: Asufeld_ins on August 18, 2013, 11:40:48 AM
Thx for answers
I read it before you post for my.
I have extract on c:/ch2hdr
and i taking ch2hdr_exp.exe too.
I clear my TMP file
And is the same problem.
My DSLR is a 50D. Have a feature ( ch2hdr ) Just for it or is the same for all ?
Same for all.
Hi Guys,
New firmware is great, but...
After I install it (booting from CF card) my 7D eat energy very fast. Batteries (two in battery pack) are on half of capacity after taking around 150 photos :(
I have ML firmware on one CF card and I using it when I need Dual ISO. If the "Dual ISO" card is in my 7D the back red LED is blinking once a 2 second period. If I need normal shooting I place into my 7D another card (no red LED blinking). In both cases the consumption of energy is very fast. One more think, when I turn camera on there is about 2 second delays then camera is ready to work...
For me energy and readiness is priority number one. Can I remove booting from my 7D? Can you help me, please?
Thank you for your response
PS Sorry for my bad English
Fanous
Quote from: fanous on August 18, 2013, 12:46:19 PMIf the "Dual ISO" card is in my 7D the back red LED is blinking once a 2 second period.
Described in the 7D installation procedure thread: Prefs -> Warning for bad setting -> Q -> Turn all 3 warning items OFF
Quote from: fanous on August 18, 2013, 12:46:19 PMCan I remove booting from my 7D? Can you help me, please?
- Download http://pel.hu/ML/bootoff.bsdiff
Follow the steps creating 7Dbooton.fir but create 7Dnoboot.fir
See http://www.magiclantern.fm/forum/index.php?topic=3974.msg64342#msg64342 for detailed instructions.
Ciao
Walter
Thank you Walter
Quote from: 1% on August 13, 2013, 05:59:48 PM
600D, its not perfect. I saw some artifacts.
http://www.filedropper.com/600ddualiso
Thanks a lot 1%, trying right now, the results looks incredible! Thanks for keep integrating the updates into the 600D.
Quote from: D.L. Watson on August 18, 2013, 10:55:15 AM
Attended a wedding and tested the duel iso for my 50D. Some beautiful dynamic range.
(https://dl.dropboxusercontent.com/u/9746153/Site%20Images%20%26%20Files/duels.jpg) (https://dl.dropboxusercontent.com/u/9746153/Site%20Images%20%26%20Files/duell.jpg)
When nighttime hit, I tried to see if I could do something like 800/12800 but even though it was selected, it still defaulted back to 800/1600. Maybe a bug? (I know it would be incredi-noise, but I just wanted to see how bad it would be).
Hi. What ISOs and converters did you use for this shot? I've just checked with latest tragic lanter build and latest cr2hdr (for windows) from first post here and still get greenish shadows.
I agree about greenish tint, using latest cr2hdr with my 5D2 many images are greenish and no color correction can get rid of it, any attempt
just adds some other color on the greens.
getting very frustrated, shooting pictures with my 50d, newest tragic lantern, aug17th, keep getting nogo saying something about interlaced something rather. Simple files names and all, still weird nogo's, HELP!!!!
Quote from: sdesign on August 18, 2013, 07:28:32 PM
getting very frustrated, shooting pictures with my 50d, newest tragic lantern, aug17th, keep getting nogo saying something about interlaced something rather. Simple files names and all, still weird nogo's, HELP!!!!
Are you using the latest cr2hdr.exe? http://www.magiclantern.fm/forum/index.php?topic=5586.msg67977#msg67977 (http://www.magiclantern.fm/forum/index.php?topic=5586.msg67977#msg67977)
Are you sure you actually have Dual ISO enabled?
Quote from: akumiszcza on August 18, 2013, 03:14:10 PM
Hi. What ISOs and converters did you use for this shot? I've just checked with latest tragic lanter build and latest cr2hdr (for windows) from first post here and still get greenish shadows.
I used ISO 100 and 1600 with the latest cr2hdr for Windows. They converted beautifully for me.
Quote from: nanomad on August 18, 2013, 11:03:49 AM
Dual iso works with "analog" ISO only. 12800 is definitely digital
Then why is it even an option in the Duel ISO settings?
Quote from: D.L. Watson on August 18, 2013, 09:13:09 PM
I used ISO 100 and 1600 with the latest cr2hdr for Windows. They converted beautifully for me.
I've checked again with latest build and latest cr2hdr compiled by Andy600. It looks like "extreme" cases (with blown highlights) are still converted wrong by it (green), but otherwise it's a huge improvement (most photos are nice).
Example of wrong processing:
https://mega.co.nz/#!lMUQ1Syb!bUPbAzlUBwnqWgjGER5eyGBbZ3JptHNwzZKRlXJsgkE (CR2)
https://mega.co.nz/#!IFtzGYqB!fEk-PEU91rsOyq6bGxSwYA9_IOJUFLRaTpbMKgl7lnY (DNG)
Different cameras have different isos... 128K I don't think anything has native, maybe 5d3 or 1dc/x.
Quote from: 1% on August 18, 2013, 09:39:38 PM
Different cameras have different isos... 128K I don't think anything has native, maybe 5d3 or 1dc/x.
That makes sense. Thanks for the clarification.
Yes i'm actually using it and its giving me a quick error that i cannot read fast enough and closes
For those of you with a window just flashing when you drag-drop onto the .exe, try putting the files you want to convert right in the same folder, right off the root of your drive, ie. c:\tmp. There was a problem with the path when I tried dragging directly from the CF card at first. (Failing that, maybe try running cmd.exe and run the program so you can see the error.)
Does anyone know the best way to expose with dual ISO in the daytime? Specifically I mean: under or over? The photo seems way brighter after taking the shot than in LV. Am I going to get better highlight or shadow recovery? I'm going to do a shoot tomorrow and I won't have time to read/experiment and figure this out beforehand.
Quote from: engardeknave on August 19, 2013, 08:46:51 AM
Does anyone know the best way to expose with dual ISO in the daytime? Specifically I mean: under or over? The photo seems way brighter after taking the shot than in LV. Am I going to get better highlight or shadow recovery? I'm going to do a shoot tomorrow and I won't have time to read/experiment and figure this out beforehand.
Select ISO 100 and set ISO 100/1600 in Dual ISO settings. Expose to get highlights properly (not cut). Best would be exposing to the right. 1600 ISO will take care of shadows.
Most often setting EV-2 in Av while taking a shot works fine.
I did not check 1600/100 method though, but I think proper exposure will be trickier (you won't be sure if highlights are ok). Also 200/800 might get much less noise if the dynamic range is not too great.
As the Highlights and Shadows are half resolution,
could there be an option in the cr2hdr app to make the other iso image half resolution to match the resolution of the shadow and highlights? people may wonder why i would like this but i noticed Dual Iso only works on RAW setting no sRAW1 or sRAW2, resize is an option in photoshop or other editing software but i would be halving the resolution again in the highlights and shadows.
cr2hdr could produce two images with this option enabled, one is the final image that the program does now and a 2nd image has been scaled to match the resolution of the halved highlights and shadows.
Cheers :)
The question is: Is it broken?
I'm not so sure about this feature (if it blocks ML developers "resources"). I learned mid tones have big influence on sharpness reception (and more). Look what LR/ACR ruler Clarity does and this is done (AFAIK) mostly by midtone manipulation. Therefore I don't have a big problem with lacking resolution in shadows/highlights.
At the moment this all is theoretical and has to be verified/falsified in the field with some valid examples.
Maybe it is just the other way round and diminished highlight/shadow resolution will stick out like a sore thumb when viewed by Mr. and Mrs. Common.
Ciao, Walter
im really sorry to bother againg, but the root trick amongst every other trick, doesnt work. I've changed filenames to be simpler, downloaded the latest and greatest cr2hdr and all it does is still flash and complain about some not interlaced or something like that, i cant read it so fast, as it comes up.
Old admin trick: Make a video at highest possible frame rate! ;-)
Ciao, Walter
Or open a cmd line and process the cr2 manually. The cmd line will remain open when it's finished processing and you will be able to see the error message.
Path_to_cr2hdr\cr2hdr.exe D46A0001.CR2
Quote from: Walter Schulz on August 19, 2013, 01:30:23 PM
The question is: Is it broken?
I'm not so sure about this feature (if it blocks ML developers "resources"). I learned mid tones have big influence on sharpness reception (and more). Look what LR/ACR ruler Clarity does and this is done (AFAIK) mostly by midtone manipulation. Therefore I don't have a big problem with lacking resolution in shadows/highlights.
At the moment this all is theoretical and has to be verified/falsified in the field with some valid examples.
Maybe it is just the other way round and diminished highlight/shadow resolution will stick out like a sore thumb when viewed by Mr. and Mrs. Common.
Ciao, Walter
I see what your saying :)
Unfortunately the mkii cant do Dual Iso Video, but i know what you mean about mr n mrs common not been able to notice, cheers for replying.
Is it a limitation or what causes the mk2 not to be able to do video with this?
Final verdict or just atm? =)
Quote from: Andy600 on August 18, 2013, 07:34:19 PM
Are you using the latest cr2hdr.exe? http://www.magiclantern.fm/forum/index.php?topic=5586.msg67977#msg67977 (http://www.magiclantern.fm/forum/index.php?topic=5586.msg67977#msg67977)
Are you sure you actually have Dual ISO enabled?
Oh yeah thank you.
the old CR2HDR.exe don't work.
But your Link is OK !!
Good job and Thank you ^^
Edit: New feature on your download page from this day.
Tragic.Lantern-2.0-Andy600.Build.2013Aug19.50D109
What's is new ?
( sorry for my english ... )
Quote from: oddname on August 19, 2013, 04:54:12 PM
Is it a limitation or what causes the mk2 not to be able to do video with this?
Final verdict or just atm? =)
not 100% sure my self, i know the mkiii and 600D do DI video.
Quote from: akumiszcza on August 19, 2013, 10:33:42 AM
Select ISO 100 and set ISO 100/1600 in Dual ISO settings. Expose to get highlights properly (not cut). Best would be exposing to the right. 1600 ISO will take care of shadows.
Most often setting EV-2 in Av while taking a shot works fine.
I did not check 1600/100 method though, but I think proper exposure will be trickier (you won't be sure if highlights are ok). Also 200/800 might get much less noise if the dynamic range is not too great.
Thanks, I used this and I think it worked out well. May or may not post examples later after I'm done processing everything.
I did a shoot just now alternating between advanced bracket for HDR and dual ISO. I did this by toggling each in junkie mode. When I got home I found that all the file names were "DUALXXXX.CR2", even the files that were part of a bracket with dualiso disabled from the menu. The files are definitely non-dualiso shots, but they are named incorrectly.
Quote from: engardeknave on August 19, 2013, 11:57:03 PM
I did a shoot just now alternating between advanced bracket for HDR and dual ISO. I did this by toggling each in junkie mode. When I got home I found that all the file names were "DUALXXXX.CR2", even the files that were part of a bracket with dualiso disabled from the menu. The files are definitely non-dualiso shots, but they are named incorrectly.
Exactly.
I've tried latest ML build + dual ISO module.
Started doing shots in pairs: 1st with dualISO off, 2nd with dualISO on.
I got one file named IMG_xxx1.cr2 and then all the files named DUALxxx2.cr2 and so on.
This is why I registered there, but you posted it first. :)
Quote from: TLN on August 20, 2013, 09:57:33 AM
I got one file named IMG_xxx1.cr2 and then all the files named DUALxxx2.cr2 and so on.
I've pointed it out to a1ex. Hopefully he can debug it.
https://bitbucket.org/hudson/magic-lantern/commits/8bd01d4dbb46386e50d4f97b71b7148a29571764
Im getting same problem, DualIso name stays, iv also noticed when you have the option to shoot one normal one dual sometimes it will take either two normal and two dual iso missing the normal/dual iso and replacing it with vice versa, anyone else had that?
I shot some tests with Dual ISO on my 5D2 under real world conditions and in a challenging szenario. But what can say ... just a great addition and so powerful. :-)
This really isn't a scientific test, but I just wanted to try how good it is for my purposes. A sunrise seems quite challenging. As I understand, the best way to shoot Dual ISO is to expose for the highlights and to get them almost clipping. But not that much as I normally shoot. And then I hoped for the power on the dark side. ;-)
The unprocessed version:
http://www.flickr.com/photos/dark-ness/9552218087/
Just some slight adjustments:
http://www.flickr.com/photos/dark-ness/9552216865/
And a quick processed version:
http://www.flickr.com/photos/dark-ness/9552220807/
Does anybody know what those white dots are (there are just a few)? Seems to be some processing error. Are those in some way dead pixel that show up much more through cr2hdr?
And I also have just a few other shots that have strong magenta highlights. No correction possible through white balance.
X-RAY how did you expose? ETTR-like or just normal exposure for the highlights?
AND for the ones that know, will video support be available for the 5D2 or is just a model limitation that says "Never"? =)
Hate to sound like a broken record, but fun to know if one is thinking about a mk3 like I am, dealbreaker if the 5D2 will get the support too eventually.
Quote from: Doyle4 on August 20, 2013, 12:05:56 PM
Im getting same problem, DualIso name stays, iv also noticed when you have the option to shoot one normal one dual sometimes it will take either two normal and two dual iso missing the normal/dual iso and replacing it with vice versa, anyone else had that?
I'm having same problems with DUAL (_UAL for AdobeRGB) file names. As for one normal/one dual shots, I've noticed it does not work for continuous shots. Probably you took double photos, then stopped, then double again. Each of those doubles will be the same type, so in this case it will be 2 normals and two duals for instance. And probably all named dual...
How to use this function? I have the Canon 6D. I downloaded the latest version of the firmware build. Found a download module features DUAL ISO / But I do not understand how it works. In the photo just strip. What am I doing wrong?
Quote from: Mirazimov on August 20, 2013, 07:46:27 PM
How to use this function? I have the Canon 6D. I downloaded the latest version of the firmware build. Found a download module features DUAL ISO / But I do not understand how it works. In the photo just strip. What am I doing wrong?
Read the first post of this thread Mirazimov.
BTW, after some experiments with DualISO and ETTR, I found that I'm missing some options in ETTR module.
There's exposure settings, and it have an option like -0.5EV, -1EV, -2, -3 and -4. With dual ISO i'd like to have something like 0EV, +1EV and may be +2EV as well. This will make a brighter shots, and can be useful with DualISO. Without it, i shoud pull my blacks alot to see the difference.
@ oddname
I didn't activate ETTR in ML but my approach was quite the same.
Using dual ison on my 5D2 and this is what I get when done with the postprocessing.
http://img823.imageshack.us/img823/2647/nqrs.jpg
Is it really supposed to be this greenish?
This seems to be just an incorrect whitebalance setting. Are the EXIF information in the DNG file correct (like aperture, shutter speed...)?
I am using one of the cr2hdr_exp.exe and it always give different exposure results even though my images were exposed manually and in the same light setting.
Another issue is sometimes it has blue overcasts on shadow too.
Plus it doesn't always convert the dualiso raw file and sometimes it need few attempts before it detects the dualiso raw files.
A1ex, any chance for you to look into this.
Thanks
Robin
White balance has been completely erratic with dual ISO. I just tried copying all the exif tags from the .cr2 to the .dng with exiftool (presumably what cr2hdr already does) and the white balance was still off.
Not to suggest that it really matters.
Hi engardeknaye,
Do you mind to PM me on how to copy exif tags from *.cr2 to *.dng with exiftool please?
Many thanks.
Robin
Quote from: engardeknave on August 21, 2013, 10:18:50 PM
White balance has been completely erratic with dual ISO. I just tried copying all the exif tags from the .cr2 to the .dng with exiftool (presumably what cr2hdr already does) and the white balance was still off.
Not to suggest that it really matters.
Iv noticed white balance can be massively out also on my MKii, to the point even auto white balance on PS can get confused and even with custom is difficult to fix, i use ML auto white balance now, haven't tested since then though.
Quote from: robinlee on August 21, 2013, 10:37:53 PM
Hi engardeknaye,
Do you mind to PM me on how to copy exif tags from *.cr2 to *.dng with exiftool please?
Many thanks.
Robin
I just did one: exiftool -tagsfromfile img_1234.cr2 img_1234.dng
But I think cr2hdr already does this.
See http://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html#copying_examples (http://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html#copying_examples)
I'm going to commit blasphemy and say I think Dual ISO is a dead end for video. Some of the samples out there have amazing dynamic range, but the aliasing is just terrible.
Having HDR with no time-based artifacts may be worth it for stills, but even there, bracketing is an option with even more range, and it works fine in many cases.
I say this because I really wish that RAW video was perfected and truly field-ready. RAW video was already drop-dead awesome before dual-ISO came along, and yet there's still plenty of work before it's really usable. Solving audio, external displays, cropping overlays, status indication UI, file format, time-code, exposure/focus aids, workflow and, of course, stabilizing the system are all worthwhile if not critical. Finishing them is a job.
I think dual-ISO video is intriguing, but a huge "feature-creep" on RAW video, and it would be best to ship the first before becoming consumed by the second. ML RAW video could mean a revolution in film-making, but it's essentially still a prototype.
How about getting RAW video done? Think of the films, the press, the accolades! It's that good.
I'm happy to help any way I can.
Dual ISO for video is a very usefull tool. And like every tool it has specific task to do. It is not usable in every situation though, for me it is great in cooperation with HDR function.
If I want to have high dynamic range with lots of detail - i use HDR. It gives motion artifacts so i use it when there is no fast camera movement. If I want high dynamic range in motion - i use dual ISO. Aliasing and moire can be less visible with motion blur and of course when background is out of focus, i tested it also with some chroma blur and it gives quite good results in decreasing artefacts.
It is great to have such awesome tools, they are still work in progress and i don't feel it is a dead end.
Even RED's HDRx has some quirks with motion so seems there are no way to have an ideal solution.
Quote from: X-RAY on August 21, 2013, 09:46:20 PM
This seems to be just an incorrect whitebalance setting. Are the EXIF information in the DNG file correct (like aperture, shutter speed...)?
My result is after "correcting" White balance.
Will check the dng when possible.
Quote from: oddname on August 22, 2013, 11:49:52 AM
My result is after "correcting" White balance.
Will check the dng when possible.
I looked into your uploaded file and I got it perfectly right with just the white balance eyedropper in ACR (I think on the curtain in the front should give the result).
Or do you mean those agressive greens in the tree? These may be chromatic aberrations.
Quote from: X-RAY on August 22, 2013, 03:44:25 PM
Or do you mean those agressive greens in the tree? These may be chromatic aberrations.
Fringe artifacts from the aliasing. Defringe it in ACR/LR.
Quote from: kgv5 on August 22, 2013, 11:27:43 AM
Aliasing and moire can be less visible with motion blur and of course when background is out of focus, i tested it also with some chroma blur and it gives quite good results in decreasing artefacts.
Even if dual-ISO video has value in those cases, I still wish that RAW video could be finished and shipped first. Shooting RAW provides massive value compared to in-camera video, and without the huge caveat and complication of aliasing.
Quote from: stevefal on August 22, 2013, 04:50:37 PM
Even if dual-ISO video has value in those cases, I still wish that RAW video could be finished and shipped first. Shooting RAW provides massive value compared to in-camera video, and without the huge caveat and complication of aliasing.
In your opinion!
Besides, it's not like work has stalled on raw recording, g3gg0 is doing some awesome work on it here (http://www.magiclantern.fm/forum/index.php?topic=7122.0). That's a massive job going on there, coding an entire container format. How long does it take the MPEG to deliver a new format. Proposals, drafts, deciding who gets the biggest percentage of the patent pool. Years!
That includes a ton of stuff you mentioned in your previous post.
P.s. Take further discussion into another thread please.
Of course my opinion! Also agree the file format etc. work is fantastic!
Thanks but still have no joy with this, cr2hdr always show exiftool don't work... :(
Quote from: engardeknave on August 22, 2013, 06:18:56 AM
I just did one: exiftool -tagsfromfile img_1234.cr2 img_1234.dng
But I think cr2hdr already does this.
See http://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html#copying_examples (http://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html#copying_examples)
hello,
I downloaded the latest version for 5D2 (31302bbc6461.zip) and the corresponding cr2 to dng tool.
I see repeatable (non-random) noise patterns at dark areas (that were amplified with the use of Shadows slider (Camera Raw 8.1) in the resultant DNG file.
I have a screenshot at 200% but in "Attachments and other options" I do not see an option to upload a photo. I'll try dropbox (if I still have an account...)
https://www.dropbox.com/s/rvfwb2an3kxukb0/5D2_DUAL_ISO_100_1600.JPG
Quote from: tron on August 22, 2013, 07:10:48 PM
I have a screenshot at 200%.
Upload the CR2 also.
As a start can you read the uploaded screenshot ?
Quote from: tron on August 22, 2013, 07:34:00 PM
As a start can you read the uploaded screenshot ?
a1ex can't bugfix screenshots ;)
Quote from: Audionut on August 22, 2013, 07:35:44 PM
a1ex can't bugfix screenshots ;)
Very funny! ;D I asked however if you can read the dropbox link (as a start) because ... I just reactivated and I hadn't use it for sharing links ever!
CR2 will follow in 5 mins...
Yeah it's fine. :D
Quote from: Audionut on August 22, 2013, 07:40:19 PM
Yeah it's fine. :D
https://www.dropbox.com/s/rjjpbcloe0dbfvl/IMG_0200test.CR2
i spotted couple of them on my images, sometimes blue or red
Guys listen, I think all RAW video development should be put on hold for now because I personally don't need it. I know this might be a controversial point of view, but if you'd stop making video so much, I'm sure you'd see things my way.
More seriously, dual ISO is awesome. It's perfect for shooting stills outside where one exposure sometimes doesn't have enough range but stuff is blowing around too much for an HDR bracket. I've already started using it during every shoot. This feature is infinitely more important to me than anything that involves video.
Can someone list a good post workflow for Dual Iso video? is it the same as RAW Video?
Quote from: engardeknave on August 22, 2013, 10:41:13 PM
Guys listen, I think all RAW video development should be put on hold for now because I personally don't need it. I know this might be a controversial point of view, but if you'd stop making video so much, I'm sure you'd see things my way.
More seriously, dual ISO is awesome. It's perfect for shooting stills outside where one exposure sometimes doesn't have enough range but stuff is blowing around too much for an HDR bracket. I've already started using it during every shoot. This feature is infinitely more important to me than anything that involves video.
Fair point, but continued on another thread: http://www.magiclantern.fm/forum/index.php?topic=7839 (awaiting moderator approval)
DUAL ISO concept is awesome indeed. It was THE reason I loaded the latest ML in my camera! However, I was taken aback by the pattern problem I reported (Reply #714). But that is why this is (a great) work in progress. I am also not interested in video but at the same time I respect others who want video too.
Quote from: v8rrc on August 14, 2013, 08:34:02 AM
Any chance of this on the 60D - can I help investigate?
*Bump* - I'd also really like to use this on my 60d, any tutorial for finding the required camera specific values, ideally w/o the risk of bricking it? Or is anyone else already on it with more experience like the people who also managed to put raw video on the 60d?
I just wanted to include my experience using Dual ISO somewhere for others that might be having the same struggle with the conversion to dng.
I am running Windows 8 64 bit which might be my problem :P.
I downloaded the cr2hdr zip folder from this post. I unzipped into the same folder as my dual ISO cr2 image file. I dragged the cr2 image onto cr2hdr and it launched, but it seemed to stop only part way through processing the file. It produced 2 files:
tmp.pgmCamera
tmp.txt
I searched for references to pgmcamera in an attempt to find a solution but found none, at first I thought I had the wrong copy of the .exe. In the end I had to run cr2hdr from the command prompt in order to produce a dng file I could use. After navigating to the correct folder just type:
cr2hdr.exe image.cr2
Where image.cr2 should be replaced with the name of the image you want to process.
Folder issue? Keep the name short without spacing. Drag and drop produces dng-files.
Do you have dcraw.exe on the same folder as well?
R
Quote from: n8ben on August 24, 2013, 03:16:56 PM
I just wanted to include my experience using Dual ISO somewhere for others that might be having the same struggle with the conversion to dng.
I am running Windows 8 64 bit which might be my problem :P.
I downloaded the cr2hdr zip folder from this post. I unzipped into the same folder as my dual ISO cr2 image file. I dragged the cr2 image onto cr2hdr and it launched, but it seemed to stop only part way through processing the file. It produced 2 files:
tmp.pgmCamera
tmp.txt
I searched for references to pgmcamera in an attempt to find a solution but found none, at first I thought I had the wrong copy of the .exe. In the end I had to run cr2hdr from the command prompt in order to produce a dng file I could use. After navigating to the correct folder just type:
cr2hdr.exe image.cr2
Where image.cr2 should be replaced with the name of the image you want to process.
I've been using dual iso on 50D on win 8 x64 for about a week, have not seen the green tint you guys showed in your samples. I use ML wb and correct in post
I've noticed the need to use some slight defringing.
When dragging multiple cr2s over cr2hdr.exe it only processes 30 or so images. Here's what happens afterwards: (http://i.imgur.com/uXHL3tO.png) (30 more of them get processed when re-attempted)
here are some sample dngs with the latest tragiclantern build (aug 19?) and cr2hdr (with no exif data, exiftool errors out for me): https://www.dropbox.com/sh/wwmu3uuz6l1e7wd/4qazwI0KH1
I'm thrilled with dual iso, did not expect it to work so well on my aged 50D.
some edited results:
http://31.media.tumblr.com/2dae2151ca0235b14ede9500e55fc888/tumblr_mrtf6cWRtY1qculbmo1_1280.jpg
http://24.media.tumblr.com/899d4bf349e0f74d40cf4d28c7f71b40/tumblr_ms0xaypHSO1qculbmo1_r1_1280.jpg
Quote from: tron on August 22, 2013, 07:43:22 PM
https://www.dropbox.com/s/rjjpbcloe0dbfvl/IMG_0200test.CR2
hello, on the generated dng (from the above CR2) I also noticed that the repeatable pattern I reported can be observed not only in shadows but in mid tones as well. Generally it can be seen everywhere, it is just easier to see it in shadows. You may have to use 300% or 400% to see it but it is there :(
I've seen the same thing on all my files. Still gives nicer images in some situations than not using it though.
It's caused by the bias towards more shadow details and less aliasing. The first cr2hdr was minimizing the noise, but people pointed out the aliasing issues very quickly. Instead of nice pictures in extreme lighting conditions, first samples were power lines, trees and dark keyboards...
Here's the lowest shadow noise that you can get from this shot: IMG_0200test_bright.dng (http://a1ex.magiclantern.fm/bleeding-edge/isoless/IMG_0200test_bright.dng)
Could the cr2hdr.exe obtain an optional parameter switch that will allow the caller to choose the bias?
I'd like something that chooses near-optimal settings every time. But I think I'll add something that outputs the components before blending (e.g. the ISO 100 half, the ISO 1600 half, the full-res image and so on).
In contrast to the dng I sent I cannot open the bright dng either with CS4 or CS6.
Re-uploaded, should be OK now.
The new .dng was a disaster. Pink or Magenta instead of white everywhere! Maybe a WB problem but I am not sure that it can be fixed (in contrast to the original .dng which was easily fixable)
However, I do not think I see these patterns now at the shadows!!!
But it is unusable. Even with patterns the previous was better :(
In shadows?!
Quote from: a1ex on August 26, 2013, 10:30:08 PM
In shadows?!
The new .dng was a disaster. Pink or Magenta instead of white everywhere! Maybe a WB problem but I am not sure that it can be fixed (in contrast to the original .dng which was easily fixable)
However, I do not think I see these patterns now at the shadows!!!
But it is unusable. Even with patterns the previous was better :(
... I've only showed you the theoretical limit for shadow noise and you complain about all sorts of stuff?!
Quote from: a1ex on August 26, 2013, 10:38:38 PM
... I've only showed you the theoretical limit for shadow noise and you complain about all sorts of stuff?!
OK! I was afraid that you had changed the cr2exe !!!!
But still very IMPRESSIVE. If only the changes you demonstrated could be applied to the shadows with no other effects Nikon would file for bankruptcy!
LOGAN57 on the 7d thread uploaded a dual iso cr2 mac converter app. The thread can be found here http://www.magiclantern.fm/forum/index.php?topic=7752.msg68051;topicseen#msg68051
I put it up on google drive for anyone interested.
cr2hdr_mac
https://docs.google.com/file/d/0B4tCJMlOYfirUW1GRGw5bk02Q0E/edit?usp=sharing
*Actually the compiled stuff was from A.D here https://bitbucket.org/a_d_/magic-lantern/downloads/
500D test :
ISO 100
(http://imageshack.us/a/img27/8718/l9i1.jpg)
DUAL ISO 100-1600
(http://imageshack.us/a/img836/9485/jr96.jpg)
Quote from: Greg on August 27, 2013, 02:16:31 PM
500D test :
ISO 100
DUAL ISO 100-1600
Will the 500D how to use Dual ISO?
Updated cr2hdr (first post):
- better handling of shadow aliasing (thanks Ted for this resolution chart (http://www.magiclantern.fm/forum/index.php?topic=7139.msg67164#msg67164))
- robust line fitting when estimating ISO (median is doing wonders, fixes the duck from Danne (http://www.magiclantern.fm/forum/index.php?topic=7139.msg65662#msg65662) and some banding issues)
- fix some tricky overexposed shots (like the CR2 from akumiszcza (http://www.magiclantern.fm/forum/index.php?topic=7139.msg68042#msg68042))
- better handling of hot/cold pixels in some extreme situations (e.g. the CR2 from canon eos m (http://www.magiclantern.fm/forum/index.php?topic=7139.msg65236#msg65236)) - but still has some subtle artifacts
- reduced some shadow noise patterns (e.g. the CR2 from tron (http://www.magiclantern.fm/forum/index.php?topic=7139.msg69260#msg69260))
Also I've added some dynamic range analysis (estimated from how much noise you have in the optical black areas). The formula is just log2(full well capacity / noise stdev) and ignores the autocorrelation of the noise (which I don't know how to handle), so it's probably grossly inflated.
Sample before/after:
(http://acoutts.com/a1ex/100_1600-1750-2400_before.jpg) (http://acoutts.com/a1ex/100_1600-1750-2400_after.jpg)
If you still have problems with image quality, upload the CR2.
That's a massive improvement dude! Cool 8)
Alex thanks and congratulations! I will try to make some tests during the weekend.
A1ex,
Seeing how much better color aliasing and moire is handled by tweaking the cr2hdr code, and also in my tests on the 5dmkII WITHOUT the VAF filter,
that dual iso image is better with aliasing compared to a Raw video and h264 video without vaf,
Would it be possible that you can FIX once and for all the issues with aliasing and moire that plagued these cameras by implementing this algorithms even without dual iso on the raw video image?
hello, I run it against my CR2 (the one you have access to) and it crashed :(
It initiated the Visual Studio debugger so I let it run and then captured the debugger screenshot that shows the offending instruction.
https://www.dropbox.com/s/2qtnuft59et5yfi/ch2hdr-27-8-crash.JPG
The last message printed was Looking for hot/cold pixels...
I can only fix aliasing where there's some information about the missing pixels (and that's what I'm doing here).
For example: let's say the higher ISO is completely overexposed. This means the missing pixels are for sure higher than white level minus ISO difference (for 100/1600, that's white minus 4 stops). This info is essential for guessing what's there (it was the key for solving the palm tree shot).
In 5D2, there's no info about what might be in the skipped lines, so, according to Nyquist theorem, this is something that can't be fixed in post.
I love Dual ISO, it really can produce some stunning imagery! I'm excited to test the newest processing method on raw video but I don't know if it's been adapted for video yet, is that further down the line? Still, this is really awesome.
Quote from: a1ex on August 27, 2013, 05:48:14 PM
I can only fix aliasing where there's some information about the missing pixels (and that's what I'm doing here).
For example: let's say the higher ISO is completely overexposed. This means the missing pixels are for sure higher than white level minus ISO difference (for 100/1600, that's white minus 4 stops). This info is essential for guessing what's there (it was the key for solving the palm tree shot).
In 5D2, there's no info about what might be in the skipped lines, so, according to Nyquist theorem, this is something that can't be fixed in post.
Is the pattern for skipping lines constant? I mean do we have the information on which lines from the sensor is being skipped? Is this not constant so you can apply an algorithm for resolving that missing line?
Maybe just maybe if you apply your dual iso algorithm tweaked a bit for non dual iso shot that compensates not for an overexposure but the similar then maybe it would be better in guessing it than how the current debayer is handling it.
I will email to you a dng of full resolution chart properly framed without the vaf.
@AdamTheBoy: not yet. Dirty trick: you can try to process the video with a tool that doesn't know about dual ISO, then drag the DNG files over cr2hdr. This may not work with the latest one, because it relies on the black bars (not present in raw video files), but with the older one (or cr2hdr_exp, look it up in the thread) should work just fine.
@ted: it reads one line, skips 2, and I don't know how to change it. I think it can be shifted somehow with ADTG registers, but this area is still a mistery.
To hide color aliasing where I can't recover it, I do some median filtering, but I believe all serious converters do it (dcraw does, I took the algorithm from there).
@tron: solved, download again.
Solved indeed :) Many thanks.
https://www.dropbox.com/s/d3n7qd18i1b653b/M232154J.000378.JPG
https://www.dropbox.com/s/20mhvdfwbnfmfp1/M232154J.000378.dng
https://www.dropbox.com/s/j60135pvsx09bb8/M232154J.000380.dng
Here's a raw video still processed using the latest cr2hdr! I'd say it's looking pretty damn good, I wanted to try a particularly alias prone shot to see how well the new techniques were working. I can't wait to see where this goes.
I also put up an unprocessed and processed DNG to check out if you'd like.
can someone please post a link to a working conversion version for raw video, cause i can not make it work.. with that one i have.. and i can only find the newest one in the first post.
thx you so much. greets. sw
@a1ex,
Heres a test comparing aug 16 vs aug 27 crshdr.
(https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-prn2/1291846_364137963716994_324677966_o.jpg)
numbers correspond to numbered circles in attached image.
1. aug 27 better in handling fine frequency lines, better in handling aliasing and moire.
2. bright pixels popping out not shown in previous version
3. dark pixels/noise? popping out not present in previous.
Emailed to you cr2 of this image
Yeah, these dark pixels are easily mistaken for aliasing (and viceversa). Not that easy to find a good balance between the two.
Here is the same set up but I used a VAF filter.
Used aug 16 build.
Obviously the VAF plays a key physical component in this approach, but those without it can go for either versions?
this version offers control over hot pixels, + vaf cuts down moire and aliasing.
Is there a middle ground?
(https://fbcdn-sphotos-h-a.akamaihd.net/hphotos-ak-ash4/1266495_364143190383138_129722111_o.jpg)
For the troublesome image, here are the components that are being blended:
fullres (http://acoutts.com/a1ex/ted_fullres.dng) - best detail, least aliasing, but noisy in shadows
fullres_smooth (http://acoutts.com/a1ex/ted_fullres_smooth.dng) - the above, but with chroma filtered (used in highlights, where one ISO is overexposed)
halfres (http://acoutts.com/a1ex/ted_halfres.dng) - least noise, but aliased; not used directly
halfres_smooth (http://acoutts.com/a1ex/ted_halfres_smooth.dng) - the above, with chroma filtered; used in deep shadows if the algorithm thinks it's unlikely to have aliasing
The likelyhood of getting aliasing is computed from the difference between fullres and halfres images (and filtered a little).
In this image, the resolution chart detail is quite dark (close to the ISO 100 noise floor), so the algorithm chooses the less noisy version. Also, there's strong banding in the fullres shot (didn't see it that bad in the other shots). Also, the ISO 100 noise is a bit higher than normal (sensor got hot?)
At a closer look, the two exposures are not properly matched (ISO 100 shows a strong green cast in shadows), so the root of the problem is when matching the ISOs. The sensor response is probably nonlinear, and my algorithm assumes it's perfectly linear.
Quote from: a59192 on August 27, 2013, 04:22:23 PM
Will the 500D how to use Dual ISO?
You need to compile the module from the repository.
Canon dual iso without moire? ;)
http://image-sensors-world.blogspot.se/2013/08/canon-files-for-dual-range-column.html
If it has 20 stops of DR, I'm sold :D
I've tried to shoot a moonrise timelapse, and dual ISO struggled a bit. Didn't develop the shots yet, but I had to clip the moon in order to see the other stuff on the screen (not just pure black and street lamps).
@ted: I'm afraid the 5D2 has a nasty behavior in the optical black area. Here's the ISO 100 exposure after black correction:
(http://acoutts.com/a1ex/ted_5d2_dark)
The ISO 1600 exposure is fine, but it couldn't be matched with the ISO 100 one because of this.
I think I prefer the half res smooth.
I think I've solved it with some aggressive nonlinear correction. The two exposures are now adjusted at each bright raw level; before, I've used a gain (ISO) and an offset (black correction).
I've uploaded it to cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe), since it suffers from overfitting in some cases (e.g. Danne's duck).
BTW, even if it looked like a hot pixel problem, I didn't touch that code at all.
BOOM!!
You nailed it A1ex!
Clean shadow areas, very minimal aliasing and moire!
If that can be achieved in raw video, I'd throw away the VAF filter!!
(https://fbcdn-sphotos-b-a.akamaihd.net/hphotos-ak-frc3/1262462_364215747042549_1577653041_o.jpg)
Gotta try this out asap on some old files. Awesome!
Danne's duck working again (a simple outlier check did the trick); all my pixel peeping tests passed (I have a script that runs it through 40 difficult test shots collected from the forum).
Also solved the problem from aseek (tested with 500 pics and still going strong).
=> updated cr2hdr from first post.
____URGENT____
Can someone PLEASE link me to where I can find the newest working download of CR2DNG? I was foolishly experimenting with this and then forgot to switch it off before a paid shoot. I have to process these photos tonight, Please help!
This will be for 5D3 CR2 files. I'm not sure if that makes a difference in which conversion version I will use.
Thanks for your help.
EDIT:
I've tried using the link to the CR2HDR.exe on the first page of this thread, but each time I drop the .CR2 files on the executable file command prompt opens very briefly and then nothing happens. What am I doing wrong?
folder name, keep it without spaces
cr2hdr is running great and my shots look even better with this new version. However, exiftool doesn't seem to be working. It's in the same folder as cr2hdr, do I need to do something different?
Here's how it looks (Windows 7 64bit running in VMware on OS X):
(http://i.imgur.com/h0h4OGq.png)
I have named my folder with the copied CR2 files "processing" without any spaces. It's created a blank text file (tmp.txt) but I can't get these to process correctly. Does CR2HDR create DNG copies of the CR2 files automatically?
I was using the July 22nd build for the firmware just fyi.
Shutterlab, are you dragging your cr2's over from another folder? You have to have cr2hdr, dcraw and exiftool in the same folder as the cr2 files you're dragging onto it for it to work.
Ahhh. I was hoping it was something simple. Thanks for the tip!
Also on win7 make sure to unblock all of these programs, cr2hdr can't run exif tool if its restricted.
hello, I have noticed that in the specific CR2 I had sent although dark shadows do not show these repeatable patterns, I can see them in mid tones (well actually less deep shadows). I have tested using the cr2hdr.exe from the first post and the _exp variance too.
(I must remind myself how the shadows would look with the Shadows slider close to maximum though in a normal Canon raw file though to keep things in perspective)
https://www.dropbox.com/s/7ilrw9bouph1t5u/cr2hdr_27aug_0708pm.JPG
https://www.dropbox.com/s/gvwfedjsujpx1rk/cr2hdr_28aug_1222am.JPG
https://www.dropbox.com/s/01osc2bj3ud8rka/cr2hdr_exp_28aug_0332am.JPG
I have uploaded the above screenshots (you have the .CR2) to see what I mean and for comparison...
Quote from: 1% on August 28, 2013, 01:34:20 AM
Also on win7 make sure to unblock all of these programs, cr2hdr can't run exif tool if its restricted.
Not sure if I'm missing a step - I went into properties for exiftool just now and set "Full Control" for every account option I could find. I ran it as Administrator as well. Looking at that screenshot though it looks like it just can't find the file because something weird is happening with the name.
Quote from: Oedipax on August 28, 2013, 03:41:40 AM
Not sure if I'm missing a step - I went into properties for exiftool just now and set "Full Control" for every account option I could find. I ran it as Administrator as well. Looking at that screenshot though it looks like it just can't find the file because something weird is happening with the name.
try to move the folder containing all the cr2 and the cr2hdr files is in the root of your drive not in another folder.
@tron: this time you are oversharpening the picture. I don't see anything bad in my jpeg.
Can someone please add the 550D diff code to the commit?
diff -r ee7c743dfb14 modules/dual_iso/dual_iso.c
--- a/modules/dual_iso/dual_iso.c Thu Aug 08 23:03:02 2013 +0200
+++ b/modules/dual_iso/dual_iso.c Wed Aug 28 01:44:59 2013 -0400
@@ -91,6 +91,7 @@
static int is_50d = 0;
static int is_6d = 0;
static int is_500d = 0;
+static int is_550d = 0;
static int is_600d = 0;
static int is_700d = 0;
@@ -742,6 +743,29 @@
CMOS_FLAG_BITS = 3;
CMOS_EXPECTED_FLAG = 0;
}
+ else if ( streq(camera_model_short, "550D") )
+ {
+
+ FRAME_CMOS_ISO_START = 0x40695494; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)
+ FRAME_CMOS_ISO_COUNT = 6; // from ISO 100 to 3200
+ FRAME_CMOS_ISO_SIZE = 30; // distance between ISO 100 and ISO 200 addresses, in bytes
+
+// 00 0000 406941E4 = 100
+// 00 0024 406941F6 = 200
+// 00 0048 40694208 = 400
+// 00 006C 4069421A = 800
+// 00 0090 4069422C = 1600
+// 00 00B4 4069423E = 3200
+
+ PHOTO_CMOS_ISO_START = 0x406941E4; // CMOS register 0000 - for photo mode, ISO 100
+ PHOTO_CMOS_ISO_COUNT = 6; // from ISO 100 to 3200
+ PHOTO_CMOS_ISO_SIZE = 18; // distance between ISO 100 and ISO 200 addresses, in bytes
+
+ CMOS_ISO_BITS = 3;
+ CMOS_FLAG_BITS = 2;
+ CMOS_EXPECTED_FLAG = 0;
+ is_550d = 1;
+ }
else if (streq(camera_model_short, "600D"))
{
/*
Quote from: mk11174 on August 28, 2013, 07:49:20 AM
Can someone please add the 550D diff code to the commit?
Pull request?
Quote from: Audionut on August 28, 2013, 07:52:09 AM
Pull request?
Yes please, I dont know how to do that, sorry. :-[
Quote from: mk11174 on August 28, 2013, 08:20:08 AM
Yes please, I dont know how to do that, sorry. :-[
It's very easy for small lines of code like this. Give me a few mins and I'll do a quick tut.
It can all be done via web browser :D
@Oedipax
you may want to try my OSX release with gui? (https://bitbucket.org/a_d_/magic-lantern/downloads)
Quote from: Audionut on August 28, 2013, 08:34:22 AM
It's very easy for small lines of code like this. Give me a few mins and I'll do a quick tut.
It can all be done via web browser :D
cool, sounds good :)
Quote from: a.d. on August 28, 2013, 08:43:31 AM
@Oedipax
you may want to try my OSX release with gui? (https://bitbucket.org/a_d_/magic-lantern/downloads)
Thank you! Didn't know anyone was working on this.
Quote from: mk11174 on August 28, 2013, 08:43:55 AM
cool, sounds good :)
Here you go :)
http://www.magiclantern.fm/forum/index.php?topic=7940.msg70958#msg70958
a1ex has already merged your code that I used to create the tutorial.
https://bitbucket.org/hudson/magic-lantern/commits/2a3faad36479e6bb576cd03ba10ec2a6649fe803
Quote from: a1ex on August 28, 2013, 07:29:47 AM
@tron: this time you are oversharpening the picture. I don't see anything bad in my jpeg.
I hadn't changed the sharpening in any case. I had just restored WB to auto (instead of as shot) to make the image realistic (to match the wall color).
Quote from: Audionut on August 28, 2013, 09:27:16 AM
Here you go :)
http://www.magiclantern.fm/forum/index.php?topic=7940.msg70958#msg70958
a1ex has already merged your code that I used to create the tutorial.
https://bitbucket.org/hudson/magic-lantern/commits/2a3faad36479e6bb576cd03ba10ec2a6649fe803
Ok, but still, thanks for the tut, at least it will help in future. :)
Maybe the hot dot remove algorithm is a bit aggressive and do harm to the shadow detail，can this be improved？
Is dual ISO already supported in raw2dng GUI app for Mac?
During conversion, why the exiftool always say didn't work when I have them in every folder for conversion?
???
Quote from: robinlee on August 28, 2013, 02:09:59 PM
During conversion, why the exiftool always say didn't work when I have them in every folder for conversion?
???
In my case (Win7-64bit) the exiftool worked after doing the convertion in a rootfolder ("C:\DualIso" for example instead of "C:\Pictures\2013\2013-08-28").
Wow, the new CR2HDR is very nice for me.
I had one problematic file with magenta patterns in the shadows, but with the new version they're totally gone.
Not just the magenta but also those patterns in the dark. At this point it should be obvious, that we have to save the original CR2-Files for the future.
For demonstration.
The CR2-file:
https://dl.dropboxusercontent.com/u/35226956/DualIso/DUAL4390.CR2
This was processed with the old version (Shadows +100)
https://dl.dropboxusercontent.com/u/35226956/DualIso/DUAL4390_CR2HDR_OLD.jpg
This with the new version (same Settings)
https://dl.dropboxusercontent.com/u/35226956/DualIso/DUAL4390_CR2HDR_08_28.jpg
@ Alex
But there is a small issue. Is there a difference in the embedded Camera Profile? With the old CR2HDR it was "Adobe Standard" and with the new one "Embedded". Did you include something there?
The thing is...do you see this strange looking sun-corona? On the Adobe Standard-Profile this looked perfectly normal and on the embedded profile thats a little ugly. ;-)
But thats just a minor issue.
Nice work!
I don't touch the exif info, except from copying it from the CR2. Try adding (or deleting) exiftool, to see if it makes any difference.
I also don't see anything strange in the sun, other than a tiny brightness change...
The image is quite oversharpened for my taste.
@alex: it is not oversharpened.
https://www.dropbox.com/s/cp954uq8f3reg0r/dual_iso_28-8-0810AM.JPG
I have used Adobe's defaults. You can see them in the screenshot.
I have used the latest cr2hdr (28-8-0810AM) but all the latest look alike.
Updated cr2hdr again:
- small differences in shadow aliasing and hot pixel handling
(7D test chart perfectly clean, 5D2 chart - which was more underexposed - almost there)
- should fix exiftool in long folders
- a little less aggressive with nonlinear corrections (not visible on my test shots, but should be less likely to get artifacts in tricky cases)
Before/after, 7D chart:
(http://acoutts.com/a1ex/100_1600-1750-2400_after.jpg) (http://acoutts.com/a1ex/100_1600-1750-2400_after2.jpg)
Before/after, 5D2 chart:
(http://acoutts.com/a1ex/100_1600_dots-2100-2500_before.jpg) (http://acoutts.com/a1ex/100_1600_dots-2100-2500_after.jpg)
(all compared with the first update from yesterday)
I chose not to fully correct the 5D2 chart (even if it's possible), because in very deep shadows, noise is easily mistaken for detail (and when it happens, it looks very ugly).
@tron: Adobe defaults have too much sharpening, look it up in the raw video section.
@a1ex:
https://www.dropbox.com/s/e9yfrvk1odyen41/dual_iso_28-8-0810AM-no-sharpening.JPG
https://www.dropbox.com/s/2ss9i46tb9touf5/dual_iso_28-8-722PM-no-sharpening.JPG
The above are with sharpening set to zero. You can see from the name the version of cr2hdr that was used.
Although Sharpening cannot always be as low as that I'd like your opinion on the above (regarding artifacts)
P.S Is that Canikon instead of the real camera name necessary ? Or you are using it temporarily to distinguish between versions of the cr2hdr?
Here's how it looks in ufraw. I don't see anything bad, a little noise is normal.
http://acoutts.com/a1ex/img0020test.png
Here, the camera name for your shot is Canon EOS 5D Mark II. Try changing it from exiftool and see if it makes a difference.
Quote from: a1ex on August 28, 2013, 07:07:25 PM
Here's how it looks in ufraw. I don't see anything bad, a little noise is normal.
http://acoutts.com/a1ex/img0020test.png
Here, the camera name for your shot is Canon EOS 5D Mark II. Try changing it from exiftool and see if it makes a difference.
It's the "Unique Camera Model" that is added to dng that has value "Canikon". It is displayed by Adobe Camera Raw as you can see in the screenshot that corresponds to the latest cr2hdr.exe
OK, change it and report.
exiftool -UniqueCameraModel="Nikanon" foo.dng
Quote from: a1ex on August 28, 2013, 07:31:57 PM
OK, change it and report.
exiftool -UniqueCameraModel="Nikanon" foo.dng
Yes it was changed in ACR 8.1 It says: Camera Raw 8.1 - Nikanon
Change it to the real thing, see if it makes any difference in output.
Quote from: a1ex on August 28, 2013, 07:57:37 PM
Change it to the real thing, see if it makes any difference in output.
You mean to set it to Canon Eos 5D Mark II
?? Actually it seems it has been added. So I was thinking to delete it!
But if you need another experiment I can set the value that is displayed by ACR with the original .CR2
EDIT: Done it displays the new value.
I delete it and it displayed: Digital Negative
I have saved the .dng however (no problem)
Why is this happening?
To delete it: exiftool -UniqueCameraModel= foo.dng
And ... the specific tag was added back...
The previous cr2hdr exe did not set this tag to Canikon though...
Quote from: a1ex on August 28, 2013, 08:04:09 PM
To delete it: exiftool -UniqueCameraModel= foo.dng
Yes, thanks I have deleted and it displayed: Digital Negative
100% sure it did.
The question is: what value is best? There was a guy who said there are differences in output caused by this tag.
cr2hdr from this morning still showed hot pixel in dark black area, but the latest cr2hdr works great. :D
@ Alex
I tried to process my file through the old and the new cr2hdr on the exact same path directly under the root directory. Also with and without the exiftool.
But nothing changes ... in the old DNG are all camera profiles listed and in the new DNG only the "Embedded".
And the sun looks more pleasant on the "Adobe Standard" camera profile. Of course I know that a camera profile is just a "look" but I don't know if that was intended.
https://dl.dropboxusercontent.com/u/35226956/DualIso/DUAL4390_OLD.DNG
https://dl.dropboxusercontent.com/u/35226956/DualIso/DUAL4390_NEW.DNG
Quote from: a1ex on August 28, 2013, 08:12:09 PM
100% sure it did.
The question is: what value is best? There was a guy who said there are differences in output caused by this tag.
I wonder: in the previous version had you just copied the camera model in this tag?
Do Adobe created DNGs have this tag or not?
Having the same value as the camera model however seems to display the same that ACR's displays for .CR2 files
Maybe it's this? http://www.magiclantern.fm/forum/index.php?topic=7139.msg62068#msg62068
@X-RAY: exiftool is not working for you.
Updated cr2hdr to set UniqueCameraModel to the real thing. Make sure exiftool is working.
It works fine regarding UniqueCameraModel :)
Quote from: a1ex on August 28, 2013, 08:56:19 PM
Updated cr2hdr to set UniqueCameraModel to the real thing. Make sure exiftool is working.
(http://imageshack.us/a/img834/3085/cmfi.jpg)
IMHO, the best way to open and convert DNG, is Luminance HDR - "Open HDR image".
Quote from: Greg on August 27, 2013, 08:29:53 PM
You need to compile the module from the repository.
Canon dual iso without moire? ;)
http://image-sensors-world.blogspot.se/2013/08/canon-files-for-dual-range-column.html
Not quite understand ~? Refers (dual_iso.c) do? I can not find 500D available module (dual_iso.mo)? Consult advanced ...
@A1ex:
I took again the same picture DUAL_ISO 100/1600. I used the Aug 28 10:07 PM cr2hdr.exe version.
I quite liked this time the result NO ARTIFACTS :) (even with default sharpening!)
DUAL ISO was not necessary for that specific image but that was what I could shoot late at night at home at 5:00 AM but this is only a test!!
Unfortunately, I noticed white and black pixels that did not exist in original CR2.
I have sent you the raw, dng and screenshot files.
@alex: Any chance adding back black/white pixel fixing without changing anything else? (especially the latest aliasing tweaks which proved superb) :)
In addition, if there are interconnected issues for example if fixing something could damage something else (I am speaking generally not necessarily for this feature) maybe cr2hdr could be made to accept switches for the users to choose.
Yep, solved most of them.
A few of these white pixels are in the high-ISO images (and I only correct the ones from low ISO). With a lower setting (100/800 or 100/400) it shouldn't happen.
Also, the latest cr2hdr handles much better the files with missing or incorrect black correction info (e.g. raw video frames and those shots that were displaying a high value for "Black delta").
Here's a video frame from 6D, from nandoide (http://www.magiclantern.fm/forum/index.php?topic=7945.msg71110#msg71110):
Before / after solving black delta / after solving hot pixels, compared to cr2hdr from last week (since yesterday's one failed at black correction):
(http://acoutts.com/a1ex/M24-1127-600-50_before.jpg) (http://acoutts.com/a1ex/M24-1127-600-50_afterblack.jpg) (http://acoutts.com/a1ex/M24-1127-600-50_after.jpg)
@a1ex:
I have checked the latest cr2hdr (29-aug 1-17PM) and I have found sets of black and white pixels some of which look more complex (include color pixels):
https://www.dropbox.com/s/vsh84lmuihmqq1t/cr2hdr-29aug-117PM-1.JPG
https://www.dropbox.com/s/2kocv2kc6vup2p4/cr2hdr-29aug-117PM-2.JPG
EDIT: In the same DUAL0364.CR2 you already have
I'm afraid I can't solve these (they are not hot pixels, but large white patches). They are located where the high ISO is overexposed, so there's no hint whether they may be hot pixels or specular highlights.
That's what your sensor is outputting. Use less aggressive settings.
What do you mean aggressive settings?
You can compare with IMG_0363.CR2
Nothing aggressive...
Something like this:
(http://i482.photobucket.com/albums/rr186/GinaDevon/AP-Cooling-System-03.jpg)
ISO 100/1600 is the absolute maximum on 5D2, how's that non-aggressive?! especially at 10 seconds of exposure...
Ever tried to do a long exposure at ISO 1600? how does it look?
You imply hot pixels?
I was looking at the previous .dng when I realized that I had shot at less than 1 sec.
I wonder how it will behave if I force long exposure noise reduction...
What phot is that??? Aggressive !!! Me like it !!! It's water cooling ?!?!
Yeah, from here: http://stargazerslounge.com/topic/146756-1100d-cold-finger-sensor-cooling-with-tec-and-water-cooling/
Long expo NR is worth trying, since it subtracts a dark frame.
Exiftool says 10 seconds for both 363 and 364.
Something else: Temperature changed from 3100 (tint +3) to 4800 (tint -9) after conversion
I will do the same experiment tonight with long exposure NR and I will let you know...
Color matrix is hardcoded from 5D3. I thought exiftool will take care of it, but this info is not in the CR2 (it's hardcoded in dcraw).
Don't know an easy way to fix it. Maybe IliasG?
***
Can anything be done for bad pixels like this?
Already post converted by raw2dng... originals are raw files like 500Mb-1GB
http://www.filedropper.com/m28-2304-000333
http://www.filedropper.com/m28-1958-000178
<3
@a1ex
Impressive modding :D
QuoteCan anything be done for bad pixels like this?
Try extracting the interlaced DNGs with an older raw2dng (or delete the HDR code from it). Then, decompose them by enabling the debug flags in cr2hdr (those files with "dark.dng", "bright.dng", "fullres.dng" and so on).
If the hot pixels are not isolated, they are not corrected because they may be mistaken for aliasing. So, a hot pixel is only corrected if it's clearly brighter than its neighbours.
edit: I've just removed the old processing code from raw2dng, because:
- raw format is near end of life
- quality is much better with cr2hdr
- the cr2hdr workflow isn't that hard (extra step: select all, drag and drop)
- the algorithm is still improving, so it's easier to maintain a single way for doing things
The hot pixels show up as purple instead of red. It never corrected them (dual or not), esp the ones from FPS override. not sure why, they don't move and are very obvious. Also the color noise/banding in the shadows.
I'll have to try to get all of the files separated and see what is produced.
There's no temporal analysis in the hot pixel algorithm, so they may not be that obvious mathematically.
I'll have to find the worst offenders and break them up/post them. The noise is easier to get rid of than the hot pixels afterwards.
I've been building Linux and Windows binaries for cr2hdr, as the code seems to be changing pretty often. They're available here, if anyone wants them, tagged with the date and Mercurial revision:
https://bitbucket.org/rufustfirefly/magic-lantern/downloads
Quote from: rufustfirefly on August 29, 2013, 07:43:26 PM
I've been building Linux and Windows binaries for cr2hdr, as the code seems to be changing pretty often. They're available here, if anyone wants them, tagged with the date and Mercurial revision:
https://bitbucket.org/rufustfirefly/magic-lantern/downloads
Thanks! rufustfirefly sharing, very helpful to me. :D
tried to convert some video dngs (son of a batch) in vmware windows 7 with the latest cr2hrd but got an error..
so what can i do now?
black subtrackt didn´t work
bright dark detection didnt work
iso blending didnt work..
greets. sw
You can read this: http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
Quote from: swinxx on August 29, 2013, 11:24:48 PM
tried to convert some video dngs (son of a batch) in vmware windows 7 with the latest cr2hrd but got an error..
so what can i do now?
black subtrackt didn´t work
bright dark detection didnt work
iso blending didnt work..
greets. sw
Sounds like you are lacking dcraw.
That would print dcraw didn't work or something like this.
i´m sorry alex.
i just recorded a dual iso enabled video raw file with the latest audionaut 5dmk3 build , then converted it with son of a batch onto my mac 10.8.4
then i had a directory with all the dng files onto my main hdd. with a vmware windows 7 machine i copied all the files into my desktop into a cr2 directory and then i selected those files (10 of them) and dragged them onto the latest cr2hrd.exe file..
with the cr2 photos it works..
thx. and sorry for my bad description, sw
Did you read my link? I know it's long, but... I have no idea what's going on. At least upload one DNG ;)
dear alex.
thx, i wil upload when i am at home.. (in a few hours.)
greetz.
@a1ex: Night experiment with same theme, exposure, lighting, shutter and aperture with Long Exposure NR set to ON.
No big changes. Strange patches continued to exist. Same image (no dual iso) with NR set to OFF at ISO 1600 at 10sec has very few and unobtrusive hot (of various colors) pixels.
Can we have a choice whether to activate white and/or black pixel fixing?
P.S I can send you a CR2 of course but it is VERY similar.
here is the link to the dng sequence, and to the raw file.
i had no success to convert it to a normal dng seuquence because of the error (seen in the screenshot. ) thx
https://www.dropbox.com/sh/wzh6q2994ve7i6d/kKL5Whmuln
Okay, will fix.
Meanwhile, try a more aggressive setting (not 100/200).
Same thing here, can´t cr2hdr the dngs extracted from raw vid. Thanks a lot for fixing, Alex! :-*
Sorry, I suppose this is my mistake, obviously the DUAL ISO is not yet implemented for 5D2, right? Can´t wait to have it on my cam soon.
Quote from: Karmaschinken on August 30, 2013, 09:27:14 PM
Sorry, I suppose this is my mistake, obviously the DUAL ISO is not yet implemented for 5D2, right?
Dual ISO should work on the 5d2 for photos, what we're really waiting for is for it to reach the 60d :-p
Look in the ADTG thread ;)
(I actually took the first dual ISO photo today on my 60D with the ADTG GUI module)
Quote from: a1ex on August 30, 2013, 09:51:35 PM
I actually took the first dual ISO photo today on my 60D with the ADTG GUI module
I didn't know this thread, but: Yippeeee!!! :-))) Thanks, it's really frustrating to see such a good feature ported everywhere else :-p
:'(
I bought a faster CF card some hours ago only to be able to record dual iso raw vids. I really love the style, I like the intro from Lars von Triers film "Melancholia", I would be very happy to be able to record footage that comes close to that.
Well, I suppose I will have to wait... ::)
alex, this could be an extreme request but, do you think its possible to combine pinkdotremover (from EOS M thread) with Dual ISO cr2hdr module? %1 just created Dual ISO fw for M, and works great, only issue is resolution is different than other cameras.
Alex remarkable result in the latest update
http://www.flickr.com/photos/68142375@N06/9631374125/
60D Dual ISO build https://skydrive.live.com/redir?resid=12771D90C09D1BCB!156&authkey=!AKIlr0pSWRB0uLs (https://skydrive.live.com/redir?resid=12771D90C09D1BCB!156&authkey=!AKIlr0pSWRB0uLs)
and Dual ISO.c https://skydrive.live.com/redir?resid=12771D90C09D1BCB!185&authkey=!ABf4aY3iIWdNzG8 (https://skydrive.live.com/redir?resid=12771D90C09D1BCB!185&authkey=!ABf4aY3iIWdNzG8)
Works on my 60D photo and video. :)
I saw this mentioned on this forum before and I just did a search for it, and cannot locate anything on it.
(It's possible that the posts were screenshots and that is why search cannot locate them).
I shot some raw video tonight using the dual ISO and then after creating seperate DNG files for each of them, and then dragging them into the newest release of Cr2hdr, I get a message that it "doesn't look like interlaced ISO" as it reviews each file.
Ideas? I thought I may have read somewhere it was because the scene could have been too dark.
@daancalo2013: may I see that CR2?
@zuzukasuma: for video? try PinkDotRemover first, then cr2hdr.
http://www.filedropper.com/dual0429_1
EOSM won't convert at all.
That's not funny. There are 4 lines at one ISO, 4 at another...
I tried processing a RAW video, 5D3, Dual ISO 1600/25600 and it doesn't seem to have worked using raw2dng
Here is a single DNG frame https://www.dropbox.com/s/bzxlo7zkkcdemcs/000000.dng
Have I done something wrong?
See the first post.
Experimental cr2hdr that fixes the DNGs from swinxx: cr2hdr_exp.exe (http://acoutts.com/a1ex/cr2hdr_exp.exe)
Can't run my pixel peeping tests right now, so... if you find files that are recognized by cr2hdr from first post, but not with this one, let me know (upload the CR2 or the interlaced DNG).
Really nice to see improvements on videofiles. Any chance getting A.D,s very nice mac app gui working with video dng,s?
//D
QuoteThat's not funny. There are 4 lines at one ISO, 4 at another...
This is what it made. Maybe problems?
Quote from: v8rrc on August 31, 2013, 06:41:31 AM
60D Dual ISO build
Great, though a bitbucket pull request is preferred - here's at least a diff for easier merging:
--- dual_iso.c.old 2013-08-31 12:14:58.407095000 +0200
+++ dual_iso.c 2013-08-31 12:34:04.478646500 +0200
@@ -90,6 +90,7 @@
static int is_5d2 = 0;
static int is_50d = 0;
static int is_6d = 0;
+static int is_60d = 0;
static int is_500d = 0;
static int is_550d = 0;
static int is_600d = 0;
@@ -743,6 +744,30 @@
CMOS_FLAG_BITS = 3;
CMOS_EXPECTED_FLAG = 4;
}
+ else if (streq(camera_model_short, "60D"))
+ {
+ /*
+ 100 - 0
+ 200 - 0x024
+ 400 - 0x048
+ 800 - 0x06c
+ 1600 -0x090
+ 3200 -0x0b4
+ */
+ is_60d = 1;
+
+ FRAME_CMOS_ISO_START = 0x407458fc; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)
+ FRAME_CMOS_ISO_COUNT = 6; // from ISO 100 to 25600
+ FRAME_CMOS_ISO_SIZE = 30; // distance between ISO 100 and ISO 200 addresses, in bytes
+
+ PHOTO_CMOS_ISO_START = 0x4074464c; // CMOS register 0000 - for photo mode, ISO 100
+ PHOTO_CMOS_ISO_COUNT = 6; // from ISO 100 to 12800
+ PHOTO_CMOS_ISO_SIZE = 18; // distance between ISO 100 and ISO 200 addresses, in bytes
+
+ CMOS_ISO_BITS = 3;
+ CMOS_FLAG_BITS = 2;
+ CMOS_EXPECTED_FLAG = 0;
+ }
else if (streq(camera_model_short, "500D"))
{
is_500d = 1;
I'm getting stripes in some areas without much detail.
http://sdrv.ms/14g00M9 (http://sdrv.ms/14g00M9)
Quote from: Marsu42 on August 31, 2013, 12:21:25 PM
Great, though a bitbucket pull request is preferred - here's at least a diff fine for easier merging with new dual_iso.c files from trunk:
--- ml-trunk/modules/dual_iso/dual_iso.c 2013-08-31 12:14:58.407095000 +0200
+++ ml-trunk/modules/dual_iso/dual_iso.c.new 2013-08-31 12:10:41.227385100 +0200
@@ -743,6 +732,30 @@
CMOS_FLAG_BITS = 3;
CMOS_EXPECTED_FLAG = 4;
}
+ else if (streq(camera_model_short, "60D"))
+ {
+ /*
+ 100 - 0
+ 200 - 0x024
+ 400 - 0x048
+ 800 - 0x06c
+ 1600 -0x090
+ 3200 -0x0b4
+ */
+ is_60d = 1;
+
+ FRAME_CMOS_ISO_START = 0x407458fc; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)
+ FRAME_CMOS_ISO_COUNT = 6; // from ISO 100 to 25600
+ FRAME_CMOS_ISO_SIZE = 30; // distance between ISO 100 and ISO 200 addresses, in bytes
+
+ PHOTO_CMOS_ISO_START = 0x4074464c; // CMOS register 0000 - for photo mode, ISO 100
+ PHOTO_CMOS_ISO_COUNT = 6; // from ISO 100 to 12800
+ PHOTO_CMOS_ISO_SIZE = 18; // distance between ISO 100 and ISO 200 addresses, in bytes
+
+ CMOS_ISO_BITS = 3;
+ CMOS_FLAG_BITS = 2;
+ CMOS_EXPECTED_FLAG = 0;
+ }
else if (streq(camera_model_short, "500D"))
{
is_500d = 1;
Maybe Alex or someone should give it a quick check over to make sure I got it right. :)
EDIT: Just submitted a pull request
Quote from: v8rrc on August 31, 2013, 12:44:27 PM
I'm getting stripes in some areas without much detail.
With the test shots I just did I didn't see this, but I probably just was lucky and the assembly worked for me. It does produce artifacts though when merging a high iso shot due to the different noise patterns. In general the 60d works 100% with this module as far as I can tell - thanks!
1. Request: dual iso should really be combined with an ettr/ettl option to expose the first image to one side and then automatically figure out the required ev spacing for the 2nd shot, in essence minimizing the required dual_iso ev spacing. Currently you have to decide for a + or - ev variant and then guess the ev value, I don't see a reason why ml shouldn't be able to do this (as usual with an option about how many clipped shadows & blown highlights are acceptable).
2. Request: The "alternate shot" mode should be amended with an option not to just take the base image, but also as a 2 shot bracket the higher iso shot so the user can decide later if the ml composite shot is necessary or if a "real" hdr assembly is possible (i.e. static scene) or even necessary due to dual_iso artifacts.
3. Bug: Alternate frame doesn't work when holding the shutter pressed with continuous drive (yes, I know it probably won't be possible to intercept Canon here, but maybe a warning about this is a good idea).
(http://farm4.staticflickr.com/3672/9635733579_2d6fbe13e7_h.jpg)
Thank ML team effort! Compared to the initial cr2hdr stripes, now restored cr2hdr perfect lot!
Quote from: v8rrc on August 31, 2013, 06:41:31 AM
60D Dual ISO build https://skydrive.live.com/redir?resid=12771D90C09D1BCB!156&authkey=!AKIlr0pSWRB0uLs (https://skydrive.live.com/redir?resid=12771D90C09D1BCB!156&authkey=!AKIlr0pSWRB0uLs)
and Dual ISO.c https://skydrive.live.com/redir?resid=12771D90C09D1BCB!185&authkey=!ABf4aY3iIWdNzG8 (https://skydrive.live.com/redir?resid=12771D90C09D1BCB!185&authkey=!ABf4aY3iIWdNzG8)
Works on my 60D photo and video. :)
After reading the dual iso pdf, I thought this module was only able to work on 5d3 && 7d because of the hardware, so how does this now work on the xxD cameras that only have the 4 channel readout vs. the 8 channel readout of the 5d3 && 7d? && does this work on xxxD as well? I'm soon to get another aps-c body and was going to get another 7d but if this will work on xxD && xxxD I may well get a rebel. TIA.
--
K.C.
Quote
...but if this will work on xxD && xxxD I may well get a rebel.
For example, dual ISO works fine on t2i both in photo and video modes ;)
I think its ported to almost every camera, some don't do it in LV tho.
Quote from: 1% on August 31, 2013, 09:31:49 PM
I think its ported to almost every camera, some don't do it in LV tho.
That's quite amazing, considering only 2 maybe 3 cameras have the 8 channel sensor, the 5d3, 7d and the ?6d? I'm not sure about the 6d, I'm just wondering how its done on cams that only have the 4 channel sensor. Amazing amazing work !!
The register has 2 isos in it.. maybe on a 4 channel it uses 2 and 2, on an 8, 4 + 4?
Quote from: 1% on August 31, 2013, 11:27:31 PM
The register has 2 isos in it.. maybe on a 4 channel it uses 2 and 2, on an 8, 4 + 4?
yeah, I remember sth like that on cr2hdr.exe built
Quote from: 1% on August 31, 2013, 11:27:31 PM
The register has 2 isos in it.. maybe on a 4 channel it uses 2 and 2, on an 8, 4 + 4?
I wonder if this means the 8 channel sensors may be able to do "Quad ISO" that would be astounding,
wonder if anyone's looked into this.... Very intriguing .... Thanks 1%.
Ummm, just can´t resist to ask because I am too curious. What are the reasons this feature can be implemented in XXD and XXXD- cams but not on the 5D2 yet? It all started with a 5D2, not? I hope my good old horse is not too outdated for the newest features coming up here?
Probably no quad iso but some other regs do interesting things.
5DII does it, just not in LV.
Yes, I mean dual iso video. I suppose if not available in LV, it´s not available for video, or did I miss something?
Quote from: Karmaschinken on August 31, 2013, 11:53:24 PM
Ummm, just can´t resist to ask because I am too curious. What are the reasons this feature can be implemented in XXD and XXXD- cams but not on the 5D2 yet? It all started with a 5D2, not? I hope my good old horse is not too outdated for the newest features coming up here?
no, if you read the pdf a1ex prepared on the 1st page of this thread, it all started with 5D Mark III & 7D because of they have dual CPUs
Quote from: Karmaschinken on August 31, 2013, 11:55:36 PM
Yes, I mean dual iso video. I suppose if not available in LV, it´s not available for video, or did I miss something?
its available on 550D but resolution is terrible already and my other cam, EOS M has dots all over the frames. its definitely worth to have more dynamic range than RAW on video
Quote from: zuzukasuma on August 31, 2013, 11:55:56 PM
no, if you read the pdf a1ex prepared on the 1st page of this thread, it all started with 5D Mark III & 7D because of they have dual CPUs
Not so, here's the quote from the PDF ...... Re-read it, it says....
Unfortunately, most other Canon cameras follow a different pattern for the CMOS #0 register, without any obviously duplicate fields. As we know from Canon specs, most cameras have a 4-channel sensor readout, except for 5D Mark III and 7D which have a 8-channel readout.
8channel readout, is the readout from the sensor, has nothing to do with how many cpu's said cam has, Afaik, the 5d3 doesn't have dual digic like the 7d
Quote from: frogcement on September 01, 2013, 12:08:51 AM
Not so, here's the quote from the PDF ...... Re-read it, it says....
Unfortunately, most other Canon cameras follow a different pattern for the CMOS #0 register, without any obviously duplicate fields. As we know from Canon specs, most cameras have a 4-channel sensor readout, except for 5D Mark III and 7D which have a 8-channel readout.
8channel readout, is the readout from the sensor, has nothing to do with how many cpu's said cam has, Afaik, the 5d3 doesn't have dual digic like the 7d
sorry, confused it with 1D X. obviously digic 5"+" processor can handle more threads than digic5 normal
Quote from: zuzukasuma on August 31, 2013, 11:55:56 PM... it all started with 5D Mark III & 7D ...
First ML ran on 5D2, I have here ML 0.16 or so... So does it mean dual iso will definitely not be available on the 5D2? : ???
Quote from: Karmaschinken on September 01, 2013, 12:19:27 AM
First ML ran on 5D2, I have here ML 0.16 or so... So does it mean dual iso will definitely not be available on the 5D2? : ???
1% said just a few posts ago, dual iso works on 5d2 just not in live view, I've personally never used live view for dual iso, I rarely use lv at all.
Quote from: 1% on August 31, 2013, 11:53:55 PM
Probably no quad iso but some other regs do interesting things.
5DII does it, just not in LV.
1%, care to elaborate, pm is fine so we don't muck up the thread anymore idc either way. Ty
Quote from: frogcement on September 01, 2013, 12:26:54 AM1% said just a few posts ago, dual iso works on 5d2 just not in live view, I've personally never used live view for dual iso, I rarely use lv at all.
I am asking for dual iso video! So if dual iso is not available in LV I was only wondering if dual iso
video turned out to be impossible at all on the 5D2. I think I am not the only one being very keen on that feature.
5DII/50D don't change the register while in LV so no video, probably ever. For photos it works. 550D/600D+ should do LV. 7D not sure, don't have it yet to check.
Booooom! That´s bad news. Ok, so I have to save money for a MK3. Thanks nevertheless for clearing this question.
how to get Dual ISO working int he EOS-M ?
Quote from: 1% on August 13, 2013, 05:59:48 PM
600D, its not perfect. I saw some artifacts.
http://www.filedropper.com/600ddualiso
This link is dead but I have the zip, mind if I re-up it so I can update the 600D raw thread?
Yep, go ahead and repost it, I don't think I made a newer one yet.
Uhm, bad news for the 650D/700D/100D/EOSM owners I think...
The dual iso pattern is dBdB which probably means we'll loose some information due to how the bayer pattern is designed
@a1ex, can you confirm this is indeed bad?
or ddddBBBBdddd
There is a hardcoded 5DIII color matrix in the converter? Why?
Quote from: 1% on September 01, 2013, 05:08:37 PM
Yep, go ahead and repost it, I don't think I made a newer one yet.
Cool, thanks.
Quote from: 1% on September 01, 2013, 06:18:11 PM
There is a hardcoded 5DIII color matrix in the converter? Why?
Quote from: a1ex on August 29, 2013, 06:05:53 PM
Color matrix is hardcoded from 5D3. I thought exiftool will take care of it, but this info is not in the CR2 (it's hardcoded in dcraw).
Don't know an easy way to fix it. Maybe IliasG?
Can't we just use the correct color matrix based on camera name?
Of course, but there's a lot to type :D
Ok, so that is the only issue then.... we can now set line pattern on EOSM/700D/650D/etc
Quote from: a1ex on August 31, 2013, 08:44:04 AM
@daancalo2013: may I see that CR2?
http://we.tl/BdGglxq1sO
a1ex, we have a problem.
when you shoot sky (or a scene with really low contrast and no obvious details) using dual ISO (no point but I've tried to create dual iso timelapse), cr2hdr.exe doesn't recognize the pattern of the shots. I'm now on a low bandwidth, but you can try it out yourself.
Also what about this: dual iso + shutter HDR flipping for raw video.
Dual shutter might be possible (look in the ADTG thread), so really extreme HDR could become a piece of cake.
I fixed the "advanced" hdr mode so while its not one exposure, you could do dual iso + switch the shutter every other frame. The workflow might be weird but i wonder what it would make at this point.
.
For the record, the line skip fix on the 650D / EOSM and other cameras should be:
raw &= 0x1FF
Quote
I fixed the "advanced" hdr mode so while its not one exposure, you could do dual iso + switch the shutter every other frame. The workflow might be weird but i wonder what it would make at this point.
Yes, but you will get both motion artifacts and aliasing. You could have just one of them with shutter HDR only ;)
The workflow is tricky because I'm not sure how to get interframe to work on raw data.
Quote from: a1ex on July 16, 2013, 06:33:50 PM
Postprocessing
- For CR2 files: cr2hdr.c (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso/cr2hdr.c) (Windows:cr2hdr.exe (http://acoutts.com/a1ex/cr2hdr.zip)) [ [.zip??? ] ]. It requires dcraw and (optional) exiftool in your executable path (they are in the zip). Drag and drop the CR2 files over cr2hdr.exe.
Anyone have any movie mode issues with 600D not working with Dual_Iso?
Photo mode works great and shows preview after shot but before the new API 3.0 I was able to get a realtime preview in Movie Mode?
Still works on my 550D, it is just the 600D I am having issues with with this new API version?
I tried both Raw_REC modules from main repo and Tragic-Lantern both same results, just not working in Movie Mode now.
There is a problem in 192 frame RAW video file. 5D3, iso 800/3200
I have converted raw to dng with raw2dng.exe on August 29, 2013, 8:49:50 PM, 254 KB (261,102 bytes). All dng files became interlaced.
After cr2hdr.exe on August 29, 2013, 1:17:52 PM, 284 KB (291,563 bytes) first 130 frames stay interlaced with error W:\cr2hdr1\1>cr2hdr.exe 000131.DNG
Input file : 000131.DNG
Full size : 1920 x 1080
Active area : 1920 x 1080
White level : 12400
Black subtract didn't work
ISO pattern : ddBd RGGB
Bright/dark detection error
ISO blending didn't work
Frames from 132 until 192 are good - not interlaced with W:\cr2hdr1\1>cr2hdr.exe 000132.DNG
Input file : 000132.DNG
Full size : 1920 x 1080
Active area : 1920 x 1080
White level : 12400
Black subtract didn't work
ISO pattern : dBBd RGGB
Noise levels : 8.00 8.00 8.00 8.00 (14-bit)
Estimating ISO difference...
ISO difference : 1.00 EV (200)
Black delta : 20.10
Interpolation : mean23-chroma5x5-alias
Dynamic range : 10.34 (+) 10.34 => 11.34 EV (in theory)
Matching brightness...
Looking for hot/cold pixels...
Hot pixels : 9
Full-res reconstruction...
ISO overlap : 6.3 EV (approx)
Half-res blending...
Chroma filtering...
Building alias map...
Filtering alias map...
Smoothing alias map...
Noise level : 8.00 (16-bit)
Dynamic range : 12.34 EV (cooked)
Output file : 000132.DNG
1 image files updated
Try cr2hdr_exp (scroll up) and upload a DNG.
Yep, someone broke it... like AE/Value +state using the LV only value.
6D values were copied for LV... dunno why. Bad merge? And then I merged it from main:
FRAME_CMOS_ISO_START = 0x406957C8; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)
FRAME_CMOS_ISO_COUNT = 6; // from ISO 100 to 25600
FRAME_CMOS_ISO_SIZE = 30; // distance between ISO 100 and ISO 200 addresses, in bytes
Same result with cr2hdr_exp.
http://dfiles.ru/files/r3fdt4fv9 good file
http://dfiles.ru/files/oszztsuk1 bad file
Dual shutter would be awesome.
What i got in mind some days ago is, "long expo at real time" which i meant to be like so:
Shutterspeed: 1/30 splitted to 1/n lines. I.e.
odd line: 1/15
even line 1/15
So you may record video at 30fps mit 1/15 shutterspeed by interlacing odd and even lines. Setting up a 1/4 (4 lines) would give a 1/7.5.. :D
Postproduction could be a wired thing then.
Just thougt..
Same issue with another file. 736 frames total - frames from 394frame until 606 frame are bad (stay interlaced)
131 works for me with cr2hdr_exp.exe...
Quote from: 1% on September 03, 2013, 06:49:04 PM
Yep, someone broke it... like AE/Value +state using the LV only value.
6D values were copied for LV... dunno why. Bad merge? And then I merged it from main:
FRAME_CMOS_ISO_START = 0x406957C8; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)
FRAME_CMOS_ISO_COUNT = 6; // from ISO 100 to 25600
FRAME_CMOS_ISO_SIZE = 30; // distance between ISO 100 and ISO 200 addresses, in bytes
Man, I should of checked the darn addresses myself, der! 8] Thanks 1%, just never thought the address would ever accidently get changed.
Alex, please check this one http://dfiles.ru/files/3yiefe6rq
131 works for me too with cr2hdr_exp.exe. Maybe I was wrong
With cr2hdr_exp.exe. only files from 0 until 28 are bad
Solved and updated cr2hdr from first post (since all pixel peeping tests passed).
It works. Thank you very much!
How was banding on 5DIII solved? It looks like there is a pattern to this noise.
(http://i.imgur.com/UF1Sx8Ol.jpg)
Took some of it in crop where its the worst with lens cap on.
Zoom (Folder Link)
https://mega.co.nz/#F!w0ICgKxA!JhgB4SXMYBJI59KIJw5rOg
can somebody explain to me what a dual shutter is please?
Can you upload one DNG? I've downloaded one of the raws, but it's just a dark frame...
Quote from: a1ex on September 01, 2013, 07:57:08 PM
Yes, but you will get both motion artifacts and aliasing. You could have just one of them with shutter HDR only ;)
I asked this much earlier in the thread and it got buried. Again I'm not sure if I'm understanding correctly but:
are we talking about taking 2 frames with alternating ISO locations and combining them in post?
i.e. frame 1 has odd rows as iso 100, even as 1600
then frame 2 has odd rows as iso 1600, even as 100
Would this work? would we gain more detail from this method? or would you be better off just bracketing ISO?
Cheers,
Ben
This would work, but the math for matching the frames is an order of magnitude more complex than what I did here. So... good luck!
Quote from: enliten on September 04, 2013, 10:13:15 AM
I asked this much earlier in the thread and it got buried. Again I'm not sure if I'm understanding correctly but:
are we talking about taking 2 frames with alternating ISO locations and combining them in post?
i.e. frame 1 has odd rows as iso 100, even as 1600
then frame 2 has odd rows as iso 1600, even as 100
Would this work? would we gain more detail from this method? or would you be better off just bracketing ISO?
Cheers,
Ben
read this Ben, http://acoutts.com/a1ex/dual_iso.pdf
I have read that, I understand the basics of the dual ISO and used it in the few days after it was first released.
What I am asking is: Could the process of taking two separate image be automated, whereby one image has alternating rows of low ISO then high ISO, followed by the second image that has alternating rows of high ISO then low ISO.
Almost like a HDR, but with dual ISO and with two images rather than one. The benefit would be that some detail would be retained in the shadows and highlights as there would be enough information to fill the blank areas. The downside being that if there is any movement in the frame (like a standard HDR) you're likely to get movement / ghosting. The difference is that the ghosting would be interlaced, rather than "solid"
I don't know if I have explained this succinctly, please let me know if I need to clarify more.
-Ben
For video it could work.
For photo, it completely misses the point, since you will get better results with a simple 2-frame bracketing.
Quote from: a1ex on September 04, 2013, 02:01:38 PM
For video it could work.
For photo, it completely misses the point, since you will get better results with a simple 2-frame bracketing.
So recording video at 60 fps and then merging frames to 30 in post?
Does the current HDR use the raw buffer? I'm just thinking that if you're capturing two consecutive frames, i.e. silent picture / raw video style, those images are going to be closer together in terms of time. Better for moving subjects (less ghosting). Excuse my ignorance on the topic.
-Ben
Advice: try playing with it first, watch the video from the user guide, read my previous posts...
k, thanks for the quick responses :)
Hi,
I have been playing around with dual ISO on my 7D. I was able to shoot some pics and convert them to DNG with cr2hdr. However, it seems the DNG-files created by cr2hdr aren't completely well-formed. The Preview app on my Mac, as well as iPhoto and Aperture should be able to open or import DNG-files. However, it turns out they can't read the files created by cr2hdr. From what I've read on the forum, I understand that Lightroom also isn't able to open the files, but Photoshop is. (But I don't have the Adobe apps, so I can't confirm that.) I've also tried GIMP, but that didn't work either.
I tried converting to JPG with ufraw-batch, but that gives very dark images with no details in the shadows. Besides, when converting to JPG, I loose the dynamic range, so that isn't an option anyway. I'd like to import the DNG's in Aperture, to be able to store them in Aperture's library along with all my other CR2's and edit them in my normal workflow.
Does anyone know a way to get Dual ISO images into Aperture without losing the RAW-data? Or is there any chance someone would be able to fix cr2hdr, so that it creates well-formed DNG's. (Note that you don't need Aperture or iPhoto to test. The Preview-app, which is part of OS X, uses the same RAW-reading technology as the other Apple-apps.)
PS.
I even tried to compile cr2hdr myself yesterday, with the latest sources from Bitbucket. I managed to compile it and it works, but gives the same unusable DNG-files.
Here is the folder with images with content + dark frames for non crop mode.
In dual ISO the 100 will be clean and the 1600/3200/6400 will be like this.
nonZoom (Folder Link)
https://mega.co.nz/#F!UwgxkBwb!Y4BtsijIZU06FGE-XSRJJQ
Hello everyone!
I seem to be running into an issue with the DNG's becoming blue or pink after processing through RawMagic.
(http://www.3rdwavenow.com/DNG_Sample.jpg)
Any idea what is causing this? Is this the way I'm processing or my capture?
I'm using a 60D with the most recent build.
Cheers! :)
Quote from: bkummel on September 04, 2013, 03:16:20 PM
...
I tried converting to JPG with ufraw-batch, but that gives very dark images with no details in the shadows. Besides, when converting to JPG, I loose the dynamic range, so that isn't an option anyway.
...
Try Darktable or Luminance HDR. IMHO, Luminance works with dynamic range better. I came to the conclusion that it is better to use converters, based on dcraw or libraw.
Now it detects camera model:
https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/downloads/cr2hdr.exe
It's still not polished up though. But feel free to use it and report any bugs you find. It will be merged in the main tree soon
Is there any quick way to filter for dual ISO raw pictures in Lightroom? I tried the ISO meta data, but there seems to be only one ISO value for Dual ISO pictures, so this won't work. (Btw, I think would be nice feature if the two ISO values would be stored instead of just one.)
Also, I would like to know it is possible to give the output file name for cr2hdr as parameter. If a file is already converted to dng, it is overwritten with the output of cr2hdr, which probably is not always desired. I am using https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/downloads/cr2hdr.exe.
Try the DUAL file prefix option.
If a file is DNG, you can rename it to CR2 and it will still work (dirty trick). There are no command line options for now.
Quote from: a1ex on September 05, 2013, 01:52:49 PM
Try the DUAL file prefix option.
Since deflicker and dualiso are mutually exclusive, cr2hdr could also add an xmp file with a similar comment like deflicker and put the other iso value used into it - I think you don't even need to embed it in the dng, an external xmp will also do if there's no tag conflict with the in-dng data.
cr2hdr already uses exiftool, so it can add a custom tag. No need for external files.
I don't use these tags, so it will have to be done by somebody who does ;)
dual_iso module does not compile>
Opps: module compiles but cr2hdr does not.
../../src/chdk-dng-host.o: In function `pow_calc_2':
/home/magiclantern/magic-lantern/modules/dual_iso/../../src/chdk-dng.c:504: undefined reference to `pow'
/home/magiclantern/magic-lantern/modules/dual_iso/../../src/chdk-dng.c:504: undefined reference to `pow'
/home/magiclantern/magic-lantern/modules/dual_iso/../../src/chdk-dng.c:504: undefined reference to `pow'
collect2: error: ld returned 1 exit status
make: *** [cr2hdr] Error 1
Fixed, for some reason it worked on my laptop but not ony PC ... go figure
Quote from: 1% on September 04, 2013, 05:52:17 PM
Now it detects camera model:
https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/downloads/cr2hdr.exe
Thanks for the update I have noticed higher resolution and less noise in the shadows and mids with this update. I also had some problems with some cold pixels which have now been resolved.. I will be testing this further with some wildlife photography.
I can't get cr2hdr to work on the linux box and no idea how to compile for windows.
To add a keyword tag for easy searching, the command is
exiftool "-xmp:subject=Dual-ISO" 1234567.dngI think this will do it if anyone wants to try
# HG changeset patch
# User Audionut <[email protected]>
# Date 1378425319 0
# Branch unified
# Node ID 0f20e1936e12cff0cd623569753ffb76e707e5f3
# Parent e0f48240e08b9aeace74228416ce6a128f5a2e6c
Added Dual-ISO keyword metadata for easy searching
diff --git a/modules/dual_iso/exiftool-bridge.c b/modules/dual_iso/exiftool-bridge.c
--- a/modules/dual_iso/exiftool-bridge.c
+++ b/modules/dual_iso/exiftool-bridge.c
@@ -8,7 +8,7 @@
void copy_tags_from_source(const char* source, const char* dest)
{
char exif_cmd[1000];
- snprintf(exif_cmd, sizeof(exif_cmd), "exiftool -tagsFromFile \"%s\" -all:all \"-UniqueCameraModel<Model\" \"%s\" -overwrite_original", source, dest);
+ snprintf(exif_cmd, sizeof(exif_cmd), "exiftool -tagsFromFile \"%s\" -all:all \"-xmp:subject=Dual-ISO\" \"-UniqueCameraModel<Model\" \"%s\" -overwrite_original", source, dest);
int r = system(exif_cmd);
if(r!=0)
{Also, looks like the only way to add the second ISO speed is via keyword/label/user comment tags.
The correct ISO tags only accept integer.
And I have no idea how to parse the ISO speeds from code for exiftool to use.
Looks good to me.
The ISO estimation from cr2hdr is relative to the other ISO (can be either positive or negative, but there's no way to tell - other than a rough guess). If EXIF ISO is 100 and the estimated one is 3 EV, it's clear in which direction it is. If exif ISO is 800 and estimated 3 EV, it's not clear whether it's ISO 800/100 or 800/6400.
You could even use the noise levels and the usual Canon DR curve to figure out which ISO is likely to be.
Or maybe just add a comment that says ISO difference: 3.14 EV.
EV difference would be fine imo.
I would also suggest that it be added to the current tag above, and have the value rounded to an integer.
So you would end up with
Dual-ISO 1EV
Dual-ISO 2EV
Dual-ISO 3EV
etc, etc for searching.
Other ideas?
if anyone has any Android programming skills would be cool if we had the CR2hdr program work on Android devices so if we are on the go we dont need a laptop and what not, only an idea and not sure if its possible, Im not a programmer so yeah. ^^;;
The processing time is around 30 seconds on my overclocked 2600k. I dread to think what it would be like on a current android device.
Quote from: Audionut on September 07, 2013, 04:41:33 AM
The processing time is around 30 seconds on my overclocked 2600k. I dread to think what it would be like on a current android device.
well if it was designed to take advantage off all the cores I think it would do fine on my quad core Nexus 7 2013.
Quote from: Audionut on September 07, 2013, 04:41:33 AM
The processing time is around 30 seconds on my overclocked 2600k.
It's 45 seconds on my 4-year old laptop (P8600). I thought the i7 must be a little faster than that, hehe.
30% single threaded performance increase is decent enough. Make it multi-threaded ;D
Quote from: kazeone on September 07, 2013, 04:57:07 AM
well if it was designed to take advantage off all the cores I think it would do fine on my quad core Nexus 7 2013.
ARM is power consumption focused. It's still going to be rather slow.
Quote from: Audionut on September 07, 2013, 09:45:23 AM
30% single threaded performance increase is decent enough. Make it multi-threaded ;D
ARM is power consumption focused. It's still going to be rather slow.
I wouldnt be to quick to say such, in the past only high end CPUs could handle HD playback and now its mere childs play for ARM cpus and they have been shown to be fantastic at mathematical calculations which is whats taking place here when processing the image besides why are you so against it? where is the harm in having the option?
Quotewhere is the harm in having the option?
Somebody must spend time to maintain the arm port (which is likely to be quite a bit more than just a straightforward compilation of current sources).
If you are willing to do that, be my guest. I'm curious how fast it can go. Maybe put Ubuntu on the Nexus and run it in console?
Quote from: a1ex on September 07, 2013, 10:12:53 AM
Somebody must spend time to maintain the arm port (which is likely to be quite a bit more than just a straightforward compilation of current sources).
If you are willing to do that, be my guest. I'm curious how fast it can go. Maybe put Ubuntu on the Nexus and run it in console?
well I would be willing to buy it so it wouldnt have to be free, shoot Im sure others would like the option as well, maybe we can talk to chainfire, hes done a lot in terms of creating various apps that control Canon DSLRs, including the famed DSLR Controller that offered a lot neat features well before ML finally got around to getting ported.
Quote from: a1ex on September 06, 2013, 07:34:26 AM
The ISO estimation from cr2hdr is relative to the other ISO (can be either positive or negative, but there's no way to tell - other than a rough guess).
Maybe in this case directly writing this information into an xmp file from ml is the way to go after all? You'd then have it next to the cr2 which some people might want to keep because better cr2hdr versions might be coming up, and cr2hdr could read the xmp and write the information into the dng. Or of course as it was suggested cr2hdr could try to figure out the + or - iso ev for itself - in any case, imho just loosing this information along the way would be a pitty.
Quote from: kazeone on September 07, 2013, 10:01:52 AM
I wouldnt be to quick to say such, in the past only high end CPUs could handle HD playback and now its mere childs play for ARM cpus and they have been shown to be fantastic at mathematical calculations which is whats taking place here when processing the image besides why are you so against it? where is the harm in having the option?
I'm not against it. But some facts.
ARM is power consumption focused.
The current processing is single threaded (bad news for slow processors that rely on multi threading), and I can't see how the processing could be easily multi threaded. a1ex would know better.
Video processing is done via dedicated chip and/or dedicated processing routines, neither of which currently help this processing, nor are they likely to in the future.
I'm not saying it can't be done!
I'm curious as to why you would want a cr2hdr conversion tool on your tablet. The resulting DNG files still need a good deal of PP to get them to look decently exposed. Then you're going to need Lightroom or GIMP/dcraw or whatever running on your tablet also. You might as well just wait until you are back at your 'editing' computer.
Seems like a waste of time. If you are just wanting a preview, then image review does a pretty decent job of blending the exposures.
Quote from: Francis on September 07, 2013, 06:24:15 PM
I'm curious as to why you would want a cr2hdr conversion tool on your tablet. The resulting DNG files still need a good deal of PP to get them to look decently exposed. Then you're going to need Lightroom or GIMP/dcraw or whatever running on your tablet also. You might as well just wait until you are back at your 'editing' computer.
Seems like a waste of time. If you are just wanting a preview, then image review does a pretty decent job of blending the exposures.
Sometimes Im on site for a week at a time or Im hiking so yes I could see it being handy to be able to work with a tablet and I already have a program like lightroom on my tablet that lets me work on Raws and DNG just like in lightroom.
As for a decent job blending the image in review mode on the camera you must be doing something different then I am because all I see is distinct lines across the whole image.
Don't zoom in to 100%.
Quote from: a1ex on September 07, 2013, 05:24:29 AM
It's 45 seconds on my 4-year old laptop (P8600). I thought the i7 must be a little faster than that, hehe.
well, I've checked it around and seemed no luck running cr2hdr.exe even with multiple user accounts on same computer, so I've created 4 Windows 7 virtual systems, my ram is about 8gigs and able to run 4 VMs simultaneously. only bottleneck is I have to use my system SSD for file i/o, because both systems and cr2hdr.exe's temp file crashes the harddrives.
so far, I'm able to render 5 frames at the same time with i5 2450M, 8gigs ram, 5 x windows 7 and a single SSD, but I'd like to have a multithread support on cr2hdr.exe :)
Some help requested.
After having used ML for a few months, my favourite most feature is clearly the Dual_ISO capability by a mile. And more so on my EOS M ( even more than on my 5D Mark III, for practical reasons - even though the quality I get on my 5D is far superior ).
The only thing I am struggling with is recovering the shadows on skin. I get these really black areas concentrated in regions of shadows that recover but not quite. There are these black patches that refuse to go away no matter what version of cr2hdr or dcraw I adopt.
And, this is a problem, because most of my photography is done after sunset ( busy schedule at work ).
I am now thinking flash to recover shadows since the highlights are clearly brilliantly handled with the Dual_ISO feature. Fast lenses are an option but not always.
Any other ideas will help greatly.
I'm having trouble with cr2hdr.exe when I shot sunset raw video at dual iso, anyone have same issue?
Quote from: zuzukasuma on September 08, 2013, 06:09:18 AM
I'm having trouble with cr2hdr.exe when I shot sunset raw video at dual iso, anyone have same issue?
I dunno....whats the issue? :D
its gonna be double post but photo upload failed.
(http://img802.imageshack.us/img802/8141/w8gr.jpg) (http://imageshack.us/photo/my-images/802/w8gr.jpg/)
I'm having trouble with cr2hdr.exe when I shot sunset raw video at dual iso, anyone have same issue on bright areas?
can you upload the original raw file? so I can try it on my end and see if I get the same results.
Quote from: kazeone on September 08, 2013, 09:06:29 AM
can you upload the original raw file? so I can try it on my end and see if I get the same results.
here it is https://www.transferbigfiles.com/c58348c4-8282-4dcd-b58b-59e1dbb9a684/Uw7m4GTs2sg8-yDtjvYYjA2
with cr2hdr for mac i can't process dng files. With wine + dos prompt and cr2hdr.exe i can't with this command : cr2hdr.exe MXX-XXXX-XX... I don't know what command i can use for select all dng in a folder... i can't do drag & drop onto cr2hdr.exe...
On osx 10.8.4 and cr2hdr (2013-09-05). can't process dng files either.
Quote from: dadinio13 on September 08, 2013, 05:10:07 PM
with cr2hdr for mac i can't process dng files. With wine + dos prompt and cr2hdr.exe i can't with this command : cr2hdr.exe MXX-XXXX-XX... I don't know what command i can use for select all dng in a folder... i can't do drag & drop onto cr2hdr.exe...
as a1ex said couple posts earlier, rename them to xxxxxx.cr2's, then cr2hdr.exe will process them
heres a workaround. extract cr2hdr from cr2hdr.app "cr2hdr.app/Contents/Resources/bin/cr2hdr"
will have to use command line. to process entire folder simple do this.
./cr2hdr /Volumes/EOS_DIGITAL/DCIM/100EOS7D/*
This worked for me. I was able to process a complete folder with DNG files.
./cr2hdr /Volumes/EOS_DIGITAL/DCIM/100EOS7D/*
Input file : /Volumes/EOS_DIGITAL/DCIM/100EOS7D/IMG_0674.DNG
Canon EOS 7D detected
Full size : 5360 x 3516
Active area : 5202 x 3465
White level : 12500
Black borders : 158 left, 51 top
Black level : 1953
ISO pattern : dBBd GBRG
Noise levels : 11.76 19.78 21.37 11.32 (14-bit)
Estimating ISO difference...
ISO difference : 1.04 EV (205)
Black delta : -3.07
Interpolation : mean23-chroma5x5-alias
Dynamic range : 9.86 (+) 8.95 => 9.98 EV (in theory)
Matching brightness...
Looking for hot/cold pixels...
Hot pixels : 20
Cold pixels : 1236
Full-res reconstruction...
ISO overlap : 5.8 EV (approx)
Half-res blending...
Chroma filtering...
Building alias map...
Filtering alias map...
Smoothing alias map...
Noise level : 29.11 (16-bit)
Dynamic range : 10.50 EV (cooked)
Black adjust : -4
Output file : /Volumes/EOS_DIGITAL/DCIM/100EOS7D/IMG_0674.DNG
1 image files updated
yep thank it worked fine for me but that is not the last version im wrong ?
and for the cr2hdr app how i can change the extension of dng's to cr2's in the folder at one time?,
Quote from: dadinio13 on September 08, 2013, 07:33:50 PM
yep thank it worked fine for me but that is not the last version im wrong ?
and for the cr2hdr app how i can change the extension of dng's to cr2's in the folder at one time?,
Get this one: www.bulkrenameutility.co.uk
thank you, sir :)
I played a bit with dual-iso in video-mode on my 600D. (dual-iso compiled from latest sources of TL2.0 on Linux). Just a little report:
Compilation on Linux:
No problem for an english system. But with my german language-settings, there is this pgm_with-error using cr2hdr. I made a workaround for me for this problem. I asked, whether this could be coded in the way, I told in this thread: http://www.magiclantern.fm/forum/index.php?topic=7693.0 . But i got no answer, so I guess, this way is not the best way and not wanted.
Colors:
Much cleaner colors on raw-vido than on mov-file. And dual-iso let a lot of noise disappear. Great!
Disadvantage:
The hot-pixels get much more visible using dual_iso.
On the mov-file, canon map most of these pixels out. In the RAW-file, these hotpixels can be surpressed using rawtherapee, other opensource linux-software, I know, is not able to render them out effectively. But using dual-iso, there is a little problem:
In the dng-files from raw2dng (./raw2dng *.RAW ) these pixels cannot be surpressed anymore. and in the output-files of cr2hdr (./cr2hdr *.dng ) I also had no success in deleating the hot pixels.
For me this doesn´t matter, because I only make videos for my astronomical shootings. And after stacking 5000 frames of saturn there is no more noise in the picture and all the hotpixels have gone magically :)
This was my little report of using dual-iso with my 600D in video-mode
Edgar
Hi there.
I'm trying to get rid of the horizontal lines but i can't.
I'm on a mac. I'm using the last version of raw2dng *0.13).
When i drop the dng files to the last version of cr2hdr it says: name.dng File not supported.
I tried creating the dng files using the last version of RAWMagic (beta7b). CR2HDR doesnt work with those dng files.
Am I missing any steps or it should be working ?
I was taking photos at the local drag strip today. I accidentally left dual ISO on. Sometimes I am worried about certain custom paint designs not coming out correctly, so I turn it off when needed. I very happy to see this photo turned out great! This is not an extreme dual ISO example. The owner of the car bought multiple prints of this photo.
(https://sphotos-a-mia.xx.fbcdn.net/hphotos-prn2/9255_594194790623658_495677599_n.jpg)
Wouldn't it be easy to multi-thread when batch processing? I made a quick script to do this. It's kind of stupid, though. It makes a new folder for each .cr2, copies the .cr2 and cr2hdr/dcraw/exiftool to that folder and then runs cr2hdr for each one. I do like 10-15 files at once this way on my i5 3570k. I haven't really timed it, but it doesn't take long. There's got to be a less dumb way of doing this though.
It would at least make scripting a lot easier if the temp files were named something different for each file being processed, so you don't have to put everything in separate folders to accomplish the above.
pretty much the cheap way to do this is to use a multithread capable dispatcher + modify cr2hdr to use temp files named from the CR2's checksum. it's not really possible to speed up cr2hdr if dcraw is not multithread-capable (amdahl's law etc)
Dual ISO is really good for shooting interactive 360 panoramas -- where you constantly have to deal with high dynamic range subjects and these look really bad in 360 panoramas if there are large areas with blown out highlights or shadow areas without details. Here is one I shot on the weekend -- 5DMkIII, 100/1600
http://www.mediavr.com/botgardens/botgardens1.htm
To extract all the detail I could from the dngs I made 3 "exposures" jpgs from each dng with ACR -- for shadows, midtones and highlights and then loaded all image variants into the stitching program I use (PTGui). This can use Enfuse, which is an exposure blending plugin
http://wiki.panotools.org/Enfuse
also available apparently as a plugin for Lightroom
http://www.photographers-toolbox.com/products/lrenfuse.php
I think the combination of Enfuse plus extracted exposure variants can be a useful technique for dealing with the vast dynamic range available in Dual ISO dngs
Quote from: engardeknave on September 09, 2013, 08:38:51 AM
Wouldn't it be easy to multi-thread when batch processing? I made a quick script to do this. It's kind of stupid, though. It makes a new folder for each .cr2, copies the .cr2 and cr2hdr/dcraw/exiftool to that folder and then runs cr2hdr for each one. I do like 10-15 files at once this way on my i5 3570k. I haven't really timed it, but it doesn't take long. There's got to be a less dumb way of doing this though.
It would at least make scripting a lot easier if the temp files were named something different for each file being processed, so you don't have to put everything in separate folders to accomplish the above.
I'm using multiple virtual machines for this job, now 4 virtual 1 real windows 7 crunching numbers to process photos***.
(http://img89.imageshack.us/img89/8809/x82w.png) (http://imageshack.us/photo/my-images/89/x82w.png/)
***for those who still wants multithread processing, you must use industrial grade harddrives or ssds for this job. temp file and final dng file at multithread kills the drive. I'm using intel 520 series, which has best price/performance/security ratio on the market.
Most of it is memory transfer/cpu processing. Having 1 drive for each execution will certainly help though :)
Why are you using virtual machines with all of their overhead? Make 5 (or whatever) copies of the executable in the base OS.
edit: Any decent HDD connect via sata should be fine.
(http://i.imgur.com/ZAnX099.jpg)
Quote from: Audionut on September 09, 2013, 11:51:30 AM
Most of it is memory transfer/cpu processing. Having 1 drive for each execution will certainly help though :)
Why are you using virtual machines with all of their overhead? Make 5 (or whatever) copies of the executable in the base OS.
it doesn't work, I've tried every possible combination. I have a network server has 5 drives but my pc has only 2 gigabit connections, still a bottleneck. I'm considering switching to E3 Xeon machine for better stability and faster processing. also I already have my virtual machines for other purposes now helping out crunching numbers of Dual ISO :)
*edit: virtual windows 7s have 1gigs ram only, not really overhead for the real system, theoretically I can use 7 virtual machines on 8gigs ram.
Quote from: zuzukasuma on September 09, 2013, 11:54:40 AM
it doesn't work,
Check my edit above.
4 copies of the executables in different folders. Bunch of dual ISOs in each folder. Drag and drop just like normal.
Quote from: Audionut on September 09, 2013, 12:04:03 PM
Check my edit above.
4 copies of the executables in different folders. Bunch of dual ISOs in each folder. Drag and drop just like normal.
your version supports EOS M's crazy dual iso with multiple sequences?
*edit: found it. testing
Huh??
100 dual ISO cr2's.
4 folders on the 1 HDD.
In each folder, a copy of each of the executables (cr2hdr, dcraw, exiftool), and 25 of the cr2's
Open 1 folder and drag and drop the cr2's onto cr2hdr.
Open the next folder and do the same
Yada
Yada
Batch processing using poor mans multithreading just like you were doing above. Difference being, you don't need SSDs, or virtual machine, or xeons, or server racks or any else.
A decent HDD connect via sata and enough cores in the processor are all that's needed.
Quote from: Audionut on September 09, 2013, 12:12:13 PM
Huh??
100 dual ISO cr2's.
4 folders on the 1 HDD.
In each folder, a copy of each of the executables (cr2hdr, dcraw, exiftool), and 25 of the cr2's
Open 1 folder and drag and drop the cr2's onto cr2hdr.
Open the next folder and do the same
Yada
Yada
Batch processing using poor mans multithreading just like you were doing above. Difference being, you don't need SSDs, or virtual machine, or xeons, or server racks or any else.
A decent HDD connect via sata and enough cores in the processor are all that's needed.
latest version sucked at processing
(http://img18.imageshack.us/img18/1725/io91.png) (http://imageshack.us/photo/my-images/18/io91.png/)
well you know the saying "the best way is you know" :)
*also, I've tried it before. unfortunately 5400rpm laptop harddrives doesn't help much with multiple instances.
**I'm switching it because of long-awaited "Intel® TSX-NI" support, today not so important but xeon for a decent price with greater performance than core i7, excellent ram support, dual gigabit ethernet and wait lower power usage? tech miracle.
Quote from: zuzukasuma on September 09, 2013, 12:16:33 PM
*also, I've tried it before. unfortunately 5400rpm laptop harddrives doesn't help much with multiple instances.
I wouldn't all a 5400rpm drive decent ;) Especially if that's in a laptop.
Let me rephrase. And decent 7200rpm HDD connect via sata to a desktop should be fine! That's still well under the recommendation of SSD's, virtual machines and xeons though :P
Quote from: zuzukasuma on September 09, 2013, 12:16:33 PM
latest version sucked at processing
Have you uploaded a CR2 for a1ex to look at?
And just to clarify, latest version date stamped 3/9/13.
Quote from: Audionut on September 09, 2013, 12:23:46 PM
I wouldn't all a 5400rpm drive decent ;) Especially if that's in a laptop.
Let me rephrase. And decent 7200rpm HDD connect via sata to a desktop should be fine! That's still well under the recommendation of SSD's, virtual machines and xeons though :P
sorry, I have one laptop and a home server. used 7200rpm seagate 500gb drives for once and now them in enclosures, noisy and overheating drives with only %10 performance difference? no thanks.
edit: this is the exe : https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/downloads/cr2hdr.exe
edit 2: found another exe, date 2/9/13 can kick 6 thread same time
(http://img835.imageshack.us/img835/5774/xv95.png) (http://imageshack.us/photo/my-images/835/xv95.png/)
Quote from: zuzukasuma on September 09, 2013, 12:26:10 PM
sorry, I have one laptop and a home server. used 7200rpm seagate 500gb drives for once and now them in enclosures, noisy and overheating drives with only %10 performance difference? no thanks.
I agree. But it's useful to point out that those drives are sufficient for this purpose, for those people without the funds for new toys.
Quote from: zuzukasuma on September 09, 2013, 12:26:10 PM
edit: this is the exe : https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/downloads/cr2hdr.exe
This is the official one http://acoutts.com/a1ex/cr2hdr.zip
Bug 1% about that version you are using.
Quote from: Audionut on September 09, 2013, 12:29:08 PM
...for those people without the funds for new toys.
currently I'm considering to upgrade my machines before getting new lenses or cameras. because I have t2i and EOS M, if I buy EF-M lens it'll be EOS M only, if I buy EF-S lens it'll be t2i and EOS M only. if/when I upgrade to 5D.III then all other equipment will stay dead for weeks.
edit: found the bug, my DUAL ISO files are older than normal. new version supports dBBd files, some of mine are dBdB.
Quote from: Marsu42 on August 31, 2013, 05:38:38 PM
dual iso should really be combined with an ettr/ettl option to expose the first image to one side and then automatically figure out the required ev spacing for the 2nd shot, in essence minimizing the required dual_iso ev spacing.
Almost working: https://bitbucket.org/hudson/magic-lantern/commits/eb3cc38d1870
How it works:
- remember that ETTR has SNR limits for midtones and shadows (it will try really hard to meet or exceed that SNR).
- if dual ISO is not enabled, it may have to clip many highlights in order to meet that SNR. If you turn off SNR limits, it will underexpose, but keep all the highlights.
- if dual ISO is enabled, ETTR will adjust it to meet or exceed the SNR you have set, and keep all the highlights (except the specular ones, handled by "Highlight ignore")
Problem: ETTR is iterative, and it relies on the previous image (where it measures the white point, some SNR numbers and stuff like that). SNR seems to change quite a bit with dual ISO (even if it shouldn't); need to investigate why. So, at this point, ETTR with dual ISO may be a little undecided: it may say 100/800 or 100/1600.
How to use it:
- Make sure you have raw zebras and histogram enabled (so you know what's going on, what's clipped, what may be aliased...)
- Enable Auto ETTR (Always ON) and dual ISO.
- Point the camera at your scene and take a few test pics.
- If the exposure doesn't converge, try underexposing a bit. When the image is overexposed, it has to guess by how much (and it doesn't always work). When it's a little underexposed, it can compute exactly what settings are needed. When it's heavily underexposed, noise will affect the metering.
ETTR will try to:
- use a low ISOs and a low EV spacing
- meet or exceed the selected SNR minus 0.5 EV
- do not clip more highlights than you have selected for "Highlight ignore"
- do not exceed max auto ISO from Canon menu
So, if that SNR can be met without clipping highlights, the scene does not require dual ISO. If not... it does.
Enable debug info to show the math behind it.
Quote from: a1ex on September 09, 2013, 12:46:25 PM
Problem: ETTR is iterative, and it relies on the previous image (where it measures the white point, some SNR numbers and stuff like that). SNR seems to change quite a bit with dual ISO (even if it shouldn't); need to investigate why. So, at this point, ETTR with dual ISO may be a little undecided: it may say 100/800 or 100/1600....
do you think its possible to save dual iso RAW + normal JPG at the same time? maybe ETTR can use info from the JPG?
edit: or highlight LOG?
What?!
Quote from: a1ex on September 09, 2013, 01:03:16 PM
What?!
create and use log of previous shot for ETTR correction.
Quotecreate and use log of previous shot for ETTR correction.
How's that different from what I'm doing now?
Quote from: a1ex on September 09, 2013, 01:16:51 PM
How's that different from what I'm doing now?
I'm not sure if we're talking about same stuff. get some rest a1ex
Quote from: zuzukasuma on September 09, 2013, 11:26:58 AM
I'm using multiple virtual machines for this job, now 4 virtual 1 real windows 7 crunching numbers to process photos***.
(http://img89.imageshack.us/img89/8809/x82w.png) (http://imageshack.us/photo/my-images/89/x82w.png/)
***for those who still wants multithread processing, you must use industrial grade harddrives or ssds for this job. temp file and final dng file at multithread kills the drive. I'm using intel 520 series, which has best price/performance/security ratio on the market.
too much work
i wrote custom utilities for this sort of thing before
http://www.mediafire.com/download/5n5pjih98b80909/MT_cr2hdr.zip
note that cr2hdr is hacked to use unique tempfiles based off input file name.
Can you share the temp file hack?
Quote from: Shizuka on September 09, 2013, 01:51:12 PM
too much work
i wrote custom utilities for this sort of thing before
http://www.mediafire.com/download/5n5pjih98b80909/MT_cr2hdr.zip
note that cr2hdr is hacked to use unique tempfiles based off input file name.
this is what I'm looking for! tell devs to import this to pack! it works like a charm (missing dcraw.exe but no prob). thank you!
edit: worked like a charm :) when you close the example.cmd cr2hdr.exe's running in the background till the all CR2s converted.
Quote from: a1ex on September 09, 2013, 02:01:47 PM
Can you share the temp file hack?
it's assembly patched - source would be the bindiff sorryI would imagine the changes in source are here:
//line 145
printf("\nInput file : %s\n", filename);
char tempname[4];
tempname[0] = filename[4];
tempname[1] = filename[5];
tempname[2] = filename[6];
tempname[3] = filename[7];
char tempname_txt[8];
char tempname_pgm[8];
snprintf(tempname_txt,8,"%s.txt",tempname);
snprintf(tempname_pgm,8,"%s.pgm",tempname);
char dcraw_cmd[1000];
//line 148
snprintf(dcraw_cmd, sizeof(dcraw_cmd), "dcraw -v -i -t 0 \"%s\" >
%s", filename, tempname_txt); int exit_code = system(dcraw_cmd);
CHECK(exit_code == 0, "%s", filename);
unsigned int model = get_model_id(filename);
exit_code = get_raw_info(model, &raw_info);
CHECK(exit_code == 0, "RAW INFO INJECTION FAILED");
//line 157
FILE* t = fopen(tempname_txt, "rb");
CHECK(t, tempname_txt);
also replace "tmp.pgm" with tempname_pgm (line 184, 188, 189)
finally, replace the unlink() parameters at 283/284
a1ex: a suggestion: turn on sse floating point instructions - it'll give a 20-30% speedup for free on many speed critical routines. -msse -msse2 -mfpmath=sse are the compiler flags for gcc, and will produce executables that run on pentium4/amdK8 or higher
edit: maybe not. I only found two routines
Currently I'm still experiencing problems on dng converter, set up in the same shot, CR2 conversion to DNG is good and bad, and attach using 5DII CR2 shoot, ask if there is any way to stabilize conversion?
https://mega.co.nz/#!QIVmVCzS!RKZgVGp2-ntkv2OFPqZ3w3ZO2XYcUCnDRkYJmAbhuZQ
Quote from: Shizuka on September 09, 2013, 02:14:38 PM
a1ex: a suggestion: turn on sse floating point instructions - it'll give a 20-30% speedup for free on many speed critical routines. -msse -msse2 -mfpmath=sse are the compiler flags for gcc, and will produce executables that run on pentium4/amdK8 or higher
edit: maybe not. I only found two routines
Fprofiling might help also.
Experimental cr2hdr (http://acoutts.com/a1ex/cr2hdr_exp.exe):
- converts the CR2 for a59192 (but didn't check if it breaks any others)
- uses pipes instead of temporary files (should work better if you start 5 copies in the same directory)
I didn't try it on Windows, only compiled the exe. On Linux I've tried on 2 pics in parallel (got 57 seconds for both, compared with 45 for a single picture).
It's complaining that everything doesn't look like interlaced ISO on my windows box.
Full log?
I:\1>cr2hdr_exp _UAL5976.CR2
Input file : _UAL5976.CR2
Canon EOS 5D Mark III detected
Full size : 5920 x 3950
Active area : 5796 x 3870
Doesn't look like interlaced ISO
Quote from: a1ex on September 09, 2013, 04:02:17 PM
Experimental cr2hdr (http://acoutts.com/a1ex/cr2hdr_exp.exe):
- converts the CR2 for a59192 (but didn't check if it breaks any others)
- uses pipes instead of temporary files (should work better if you start 5 copies in the same directory)
I didn't try it on Windows, only compiled the exe. On Linux I've tried on 2 pics in parallel (got 57 seconds for both, compared with 45 for a single picture).
Used cr2hdr_exp.exe with the current new construction cr2hdr, can not normally turn out, do not know which part wrong, ask how to use the uses pipes? windows can use it?
Looks like you can't use pipes with large amounts of data under Windows.
http://marc.info/?l=mingw-users&m=134554372503503
http://lists.gnu.org/archive/html/bug-binutils/2012-09/msg00064.html
http://mingw.5.n7.nabble.com/popen-from-a-GUI-program-without-opening-a-cmd-exe-window-td20873.html
https://lists.fedoraproject.org/pipermail/mingw/2009-September/002087.html
I've uploaded one again, with _popen instead of popen, but I have no idea whether it will work or not.
Same problem.
Can you try again? Got it working under Wine.
I had to set binary mode on the pipe and stop using fseek.
Yeah that got it working. Can you add a command line option for the user to select how many instances to run at any one time. And then when a bunch of CR2s are dropped on it, it's spawns all those instances?
I tried adding a heap of compiler options (-msse etc etc), and it made no difference to the processing time :(
Camera tested 5D3
I tried the latest commit (da752ca33b18) and the when using the AETTR + Dual ISO I get a problem.
I am setting the Dual Iso to +3EV
The first picture I take shows as dual ISO possilbly at base iso + 3EV
The next picture the Dual ISO module has the base iso = recovery iso. So no Dual Iso Picture is take.
Repeatable.
Quote from: Audionut on September 09, 2013, 07:19:15 PM
Yeah that got it working. Can you add a command line option for the user to select how many instances to run at any one time. And then when a bunch of CR2s are dropped on it, it's spawns all those instances?
I tried adding a heap of compiler options (-msse etc etc), and it made no difference to the processing time :(
If I can ask, how did you get it working? Did you just download a1ex's experimental upload (seen above) again? (which I assume has been updated)
Quote from: telecastDaveRI on September 10, 2013, 05:43:50 AM
Did you just download a1ex's experimental upload (seen above) again? (which I assume has been updated)
That's correct.
Quote from: RenatoPhoto on September 10, 2013, 04:10:09 AM
Camera tested 5D3
I tried the latest commit (da752ca33b18) and the when using the AETTR + Dual ISO I get a problem.
I am setting the Dual Iso to +3EV
The first picture I take shows as dual ISO possilbly at base iso + 3EV
The next picture the Dual ISO module has the base iso = recovery iso. So no Dual Iso Picture is take.
Repeatable.
Are you sure it's not this?
Quote from: a1ex on September 09, 2013, 12:46:25 PM
ETTR will try to:
- use a low ISOs and a low EV spacing
- meet or exceed the selected SNR minus 0.5 EV
- do not clip more highlights than you have selected for "Highlight ignore"
- do not exceed max auto ISO from Canon menu
So, if that SNR can be met without clipping highlights, the scene does not require dual ISO. If not... it does.
----------------
Using autoexpo for the base exposure with AETTR enabled in autosnap with dual ISO. All images in a test batch had the dual ISO file prefix. Some images were not dual ISO as not needed.
If I just have dual ISO enabled with alternate frames, the file prefix is working as expected.
When I take a photo with AETTR 02% h.i. and SNR OFF I get
DR = 10.9
Same scene with Dual Iso only with Recovery ISO = +3 EV I get
DR = 13.4
Now I turn on AETTR 0.2% h.i plus and SNR = OFF + Dual ISO with Recovery ISO = +3 EV I get
DR=10.9 and Base ISO = Recovery ISO DUAL ISO is turned OFF automatically
Now I set DUAL ISO TO 800 ISO plus the AETTR as above and I get
DR=10.9 and Base Iso = Recovery ISO , my ISO 800 has been automatically replaced by base ISO
Conclusion: Dual ISO + AETTR is not working properly. If I roll back to Sept 07 build then the results are totally different and DUAL ISO works as expected!
Enable some SNR limits. This will give the code something to aim for :)
Both functions are now linked together when both are enabled. With no SNR limits applied, you're telling it, I don't care how noisy the midtones/shadows are.
With SNR limits applied, previously AETTR would blow the highlights to attain the required SNR, now it will enable dual ISO instead to keep the highlights in check while producing the required SNR.
I guess it could be coded so that SNR limits off mean restore previous functionality. But to be honest, I think that will cause to much confusion for users in the future.
Quote from: Audionut on September 10, 2013, 05:57:49 AM
Using autoexpo for the base exposure with AETTR enabled in autosnap with dual ISO. All images in a test batch had the dual ISO file prefix. Some images were not dual ISO as not needed.
I tried adding 100ms delay to autosnap but it still wanted to use the same prefix as the previous shot regardless. And 100ms was becoming noticeable.
a1ex can you send the output of cr2hdr to a logfile? It would be nice to check the results of processed files without having to manually process each one.
Quote from: Audionut on September 11, 2013, 12:44:16 AM
Enable some SNR limits. This will give the code something to aim for :)
I want to keep all of the highlights, that is why I turned off SNR. So my only option to conserve the highlights is Midtones SNR = 1 if I want to use Dual Iso with Recoveru ISO +3EV
Ok I tested with SNR and it works with Midtones and Shadows = 1. So I dont have the option to keep the highlights by doing a standard AETTR at h.i.=0.2% + DUAL ISO.
In my opinion this is not good. I will be using September 07 build until this gets resolved.
Thanks anyway.
AETTR works just like previously. You decide how many highlights to blow (or not).
Enabling SNR limits will try and keep those highlights, but will blow highlight detail to ensure the SNR limits are met.
Enabling SNR limits and dual ISO will keep those highlights and use dual ISO as necessary to ensure the SNR limits are met without blowing the highlights.
If the SNR limit can be met without the use of dual ISO, dual ISO will not be used. Highlight detail is controlled by the settings in AETTR just as it's always been.
Enable AETTR and set your required highlight settings. Have dual ISO and SNR limits off. Observe the results.
Now enable dual ISO and set the SNR limits (use the default settings for a start 6/3), observe the results.
Set the midtone SNR limit to 8EV. Observe the results. Highlight detail should not change, dual ISO settings should change (automatically) to boost the midtone luminance.
Quote from: Audionut on September 11, 2013, 04:23:08 AM
Enabling SNR limits will try and keep those highlights, but will blow highlight detail to ensure the SNR limits are met.
Just a simple question. I want to keep all of the highlight therefore I use AETTR with h.i. = 0
Now I want to boost the shadows and midtones without blowing the highlihgts using DUAL ISO. What should I set the Dual ISO module to accomplish this?
Nothing. Dual ISO is controlled automatically. Just turn it on.
Control the midtones/shadows with the SNR limits. With dual ISO on, it won't blow your highlight detail like previously, it will adjust dual ISO settings as needed to boost the midtones/shadows.
Quote from: Audionut on September 11, 2013, 12:44:16 AM
I guess it could be coded so that SNR limits off mean restore previous functionality. But to be honest, I think that will cause to much confusion for users in the future.
Or maybe a setting in dual ISO, "link to AETTR". Else dual ISO is always enabled at the settings specified.
Quote from: Audionut on September 11, 2013, 04:42:01 AM
Nothing. Dual ISO is controlled automatically. Just turn it on.
Control the midtones/shadows with the SNR limits. With dual ISO on, it won't blow your highlight detail like previously, it will adjust dual ISO settings as needed to boost the midtones/shadows.
Nothing is not a good answer because after the first picture the camera automatically sets the Base ISO = Recovery ISO and Dual ISO stops working. So again, there appears to be a problem with Dual ISO module that should be resolved.
The problem is explained here:
Quote from: RenatoPhoto on September 10, 2013, 04:10:09 AM
Camera tested 5D3
I tried the latest commit (da752ca33b18) and the when using the AETTR + Dual ISO I get a problem.
I am setting the Dual Iso to +3EV
The first picture I take shows as dual ISO possilbly at base iso + 3EV
The next picture the Dual ISO module has the base iso = recovery iso. So no Dual Iso Picture is take.
Repeatable.
And here....
Quote from: RenatoPhoto on September 10, 2013, 11:22:12 PM
When I take a photo with AETTR 02% h.i. and SNR OFF I get
DR = 10.9
Same scene with Dual Iso only with Recovery ISO = +3 EV I get
DR = 13.4
Now I turn on AETTR 0.2% h.i plus and SNR = OFF + Dual ISO with Recovery ISO = +3 EV I get
DR=10.9 and Base ISO = Recovery ISO DUAL ISO is turned OFF automatically
Now I set DUAL ISO TO 800 ISO plus the AETTR as above and I get
DR=10.9 and Base Iso = Recovery ISO , my ISO 800 has been automatically replaced by base ISO
Conclusion: Dual ISO + AETTR is not working properly. If I roll back to Sept 07 build then the results are totally different and DUAL ISO works as expected!
and here:
Quote from: RenatoPhoto on September 11, 2013, 04:12:11 AM
I want to keep all of the highlights, that is why I turned off SNR. So my only option to conserve the highlights is Midtones SNR = 1 if I want to use Dual Iso with Recoveru ISO +3EV
Quote from: RenatoPhoto on September 11, 2013, 02:38:36 PM
Nothing is not a good answer because after the first picture the camera automatically sets the Base ISO = Recovery ISO and Dual ISO stops working. So again, there appears to be a problem with Dual ISO module that should be resolved.
Can you post 2 cr2s please. The first shot where you had your initial settings, and the second shot where dual ISO was reset to recovery ISO = base ISO.
Quote from: Audionut on September 11, 2013, 03:35:20 PM
Can you post 2 cr2s please. The first shot where you had your initial settings, and the second shot where dual ISO was reset to recovery ISO = base ISO.
Would love to do it but I dont have the bandwidth to do that. What should I look for in the cr2. maybe I can analyze it. I know what dual iso is and there is no dual ISO on the second image because DUAL ISO says so, "Both ISO are identical, nothing to do"
If the first CR2 has a SNR higher or equal to what you have requested in menu, ETTR will disable dual ISO. This is normal and not a bug.
Quote from: a1ex on September 11, 2013, 03:45:32 PM
If the first CR2 has a SNR higher or equal to what you have requested in menu, ETTR will disable dual ISO. This is normal and not a bug.
SNR=OFF
What highlights are or are not being blown in each of the photos.
Does setting SNR limits correlate with the description provided by a1ex 2 pages back.
Are the highlights being blown with SNR limits enabled.
Set SNR limits to 8/3EV midtones/shadows. Are the highlights being blown.
Always take enough photos after changing AETTR settings to ensure convergence has been met.
Had a chance to play with it today. I've set it on Auto ETTR (SET button in LiveView), highlight ignore 0.1%, midtone SNR at 7 EV and shadow SNR disabled.
In most cases it worked quite well. Only had some minor problems:
1) when ISO reached 6400 (it was undecided between 400/6400 and plain 6400 - quite a big diference). The problem appears only when it's too dark and exposure limits are reached (extra DR needed is estimated incorrectly).
2) in "always on" mode, confirmation beep was not always right (overall exposure didn't change, but dual ISO range did; however, the "beep system" didn't notice that). So, after getting a single beep, I had to take one more picture.
LiveView + SET worked fine.
3) sometimes it chose 3200/6400, and there was probably no benefit in doing that. Of course, I could have limited it to 1600 or 3200, but I also wanted to watch the algorithm making the choices.
Besides these small quirks, I think ETTR is now doing near-optimal exposure for dual ISO.
I may post some pics if there'll be anything worth showing.
There appears to be a problem with the noise calculation for ISO 6400 (for the 5D3 at least).
The reported noise level continues to drop for all dual ISOs until ISO 6400 recovery.
QuoteInput file : _UAL6623.cr2
Canon EOS 5D Mark III detected
White level : 12500
Black level : 1963
Noise levels : 10.65 6.10 6.26 10.49 (14-bit)
ISO difference : 3.97 EV (1571)
Black delta : 2.39
Dynamic range : 10.75 (+) 9.95 => 13.92 EV (in theory)
Hot pixels : 64
Cold pixels : 378
Noise level : 2.17 (16-bit)
Dynamic range : 14.25 EV (cooked)
Black adjust : 1
Output file : _UAL6623.DNG
QuoteInput file : _UAL6624.cr2
Canon EOS 5D Mark III detected
White level : 12500
Black level : 1961
Noise levels : 16.08 6.13 6.22 15.70 (14-bit)
ISO difference : 4.91 EV (3006)
Black delta : 1.01
Dynamic range : 10.75 (+) 9.36 => 14.27 EV (in theory)
Hot pixels : 5
Cold pixels : 419
Noise level : 1.66 (16-bit)
Dynamic range : 14.63 EV (cooked)
Black adjust : 1
Output file : _UAL6624.DNG
QuoteInput file : _UAL6625.cr2
Canon EOS 5D Mark III detected
White level : 12500
Black level : 1954
Noise levels : 26.39 6.21 6.34 26.81 (14-bit)
ISO difference : 5.87 EV (5828)
Black delta : -0.31
Dynamic range : 10.73 (+) 8.62 => 14.48 EV (in theory)
Hot pixels : 4
Cold pixels : 649
Noise level : 3.22 (16-bit)
Dynamic range : 13.68 EV (cooked)
Black adjust : 1
Output file : _UAL6625.DNG
This occurred in 3 different scenes with varying levels of DR. DxO results (http://www.sensorgen.info/CanonEOS_5D_MkIII.html) suggest that the advantage of ISO 6400 over ISO 3200 should be neither here nor there, and my own testing showed a very minimal gain (Std Dev continued to drop for a selection of flat textures in the scenes).
https://www.dropbox.com/sh/di2lkp2knv5bqso/XPN9ptKf-Y
edit: The results are for base ISO 100
@ base ISO 200, the noise level only increases slightly for recovery ISO 6400.
@ base ISO 400 and 800, the noise level continued to drop for recovery ISO 6400 and increase for recovery ISO 12800.
Getting some damn great pictures on the 600d, amazing work gentlemen.
Quote from: N/A on September 12, 2013, 02:59:05 AM
Getting some damn great pictures on the 600d, amazing work gentlemen.
2nd that!!
Is it me or the app please help I'm using cr2hdr // Beta 1.0 app to convert dual ISO I have success in windows but preferred to use Mac but this is the error that I keep getting
cr2hdr // Beta 1.0
Input file : 6I0A5237.CR2
sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile : 6I0A5237.txt
THE END
Keep up the incredible work that you guys do and if you're working too hard then definitely get rest...
Your machine is missing dcraw.
I have done some test to try to understand how the new link between AETTR and DUAL ISO is behaving.
As I stated before, the new system when using the SNR = OFF or low SNR numbers, always defaults to Base ISO = Recovery ISO. My explanation to this behavior is that the modules is doing all the calculations and deciding automatically that DUAL ISO is not needed and therefore making Base ISO = Recovery ISO.
So now I will test this theory as follows:
1. Set AETTR to 0.2% h.i. and SNR = OFF and took some pictures until the exposure converges.
2. Turn AETTR OFF (to maintain its corrected exposure) and use DUAL ISO to manually test different iso combinations.
3. Compare these results with AETTR+DUAL ISO at different levels of SNR
First is the manual method:
(http://s20.postimg.org/3yi29gvql/AETTR_w_Manual_DUAL_ISO.jpg)
In these images it is obvious that the shadow noise is improved with the use of DUAL ISO
Now lets look at the new AETTR + DUAL ISO
(http://s20.postimg.org/m01779prh/DUAL_ISO_AETTR_WITH_SNR.jpg)
We see that the system only responds to some very high SNR numbers 8/3 to reduce the noise in the shadows, but with lower numbers that shadows are totally ignored and DUAL ISO is disabled automatically.
Also note that the AETTR with SNR=8/3 activated the DUAL ISO at 100/400 and this results where similar to the simple 100/400 DUAL ISO test done first. At maximum SNR = 8/4 I was able to get 100/1600 dual ISO but it took a few tries until I got most of the DUAL ISO options to show up with the new system.
Conclusion is that the new system will use the DUAL ISO technology when needed to maintain the h.i. setting (0.2% in this test) and reduce the SNR shadow and midtones. A similar result can be obtained by first doing the AETTR, disabling AETTR, and enabling DUAL ISO with manual settings. The main difference is that the new system will search automatically for the DUAL ISO needed but the SNR numbers have to be very high for these parameters to make effect in the DUAL ISO. I had used AETTR and never used 8/4 to obtained good results, I typically used 4/2 but at these levels DUAL ISO is not enabled. This was my main conflict in thinking that it was not working but I just had to raise the SNR to the limits to get it to work.
This analysis then led me to explore what SNR settings will allow me to exploit the DUAL ISO benefits. I found the following combinations that will give me what I am looking for in DUAL ISO performance with full sun and shadow outside setting.
SNR = 8/4 = Dual ISO = 100/1600
SNR = 7/4 = Dual ISO = 100/800
SNR = 6/3 = Dual ISO = 100/400
SNR = 5/3 = Dual ISO = 100/200
Now I switched to a full sun and lots of full shadow through a window ( inside my house):
SNR = 8/4 = Dual ISO = 100/3200
SNR = 6/2 = Dual ISO = 100/1600
SNR = 3/1 = Dual ISO = 100/800
SNR = 2/1 = Dual ISO = 100/400
SNR = 1/1 = Dual ISO = 100/200
Now this leads me believe that we basically have maximum six levels of use for DUAL ISO when the condition are adequate. If we have lower light situations where the base ISO is lower we lose one lever for each base ISO increase. For example if we have base ISO of 200 we have five levels l (200/400, 200/800, 200/1600, 200/3200, 200/6400), for base iso = 400 we have four levels of adjustment (400/800, 400/1600, 400/3200, 400/6400).
This makes me think that we could simplify this system and present the user with just the number of shadow recovery options based on the base iso instead of having all of these SNR options. For example if base iso is 100 then the allows adjustments are from 1 to 6. If the base ISO=200 then we have only 1 to 5, etc.
Right now I am presented with 32 possible adjustments ie 8/4, 8/3, .... 7/4, 7/3, 6/4, 6/3 and some of them will not produce any apparent results. Now if this could be reduced from 32 to 6 options then ease the usability will be much greater.
Just a final note the highlights have been independently maintained without overexposure which lead me to believe that are independently controlled by highlight ignore. Will probably need a separate test.
And finally I thank you Alex for this amazing technology and I do appreciate all the hard work that you do.
P.S. I am learning from doing these test so please correct me if I did something wrong!
Nice test!
@RenatoPhoto: that's way too complicated. Simply state:
a) how much noise you are OK with (let's say 6EV or 7EV)
b) how many highlights can be ignored (0.1% is a good overall choice, 0.5 for something like a fairly large window in a dark cave where not even 20 stops of DR are enough, and 0% if you don't have any specular highlights).
Then let ETTR do the rest. That's what I do.
I've been exploring the idea of using dual ISO for quasi ISOless mode. Base ISO is always ISO 100, with adjustments to recovery ISO only. ETTR+dual ISO is fine for still shots, but for running and gunning under varying light conditions, it is much to slow.
In a quasi ISOless mode, I could set dual ISO 100/400 and cover an exposure value range of 6-16EV (http://www.fredparker.com/ultexp1.htm#Light%20Intensity%20Chart) in Av mode (1/125-F/1.4 with this lens). If I didn't care about my highlights so much, I could use dual ISO 100/1600 and cover an exposure value range of 4-16EV and deal with the reduced resolution in post. With dual ISO 100/400, this also allows 2 EV of underexposure leeway for the base ISO. ie: not only do I not need to worry about ISO adjustments (as much), I don't have to worry about being extremely accurate with my exposure to ensure my base ISO is always ETTR. The recovery ISO takes up the slack ;)
Some caveats with the current functionality.
The histogram always uses the lowest ISO. Where recovery ISO becomes the dominant brighter exposure, the histogram becomes unreliable. ETTR hint does not work, and the histogram will no longer display overexposure results.
It doesn't appear as if base ISO is boosted in anyway (in cr2hdr) to fill in the exposure gaps in the recovery ISO, where recovery ISO is brighter. Suggestion, where recovery ISO is brighter then base ISO, determine white level of base ISO, boost the exposure of the base ISO and use that data for AA correction in the recovery ISO. It's better to have some accurate data, even if that data has a higher noise level, then to interpolate missing data.
This could also be handy with the current functionality, to fill in any resolution loss where base ISO is not ETTR.
I don't get it; cr2hdr only cares about higher iso and lower iso (doesn't matter which is base and which is recovery - this one is mostly relevant when using LiveView). So 100/1600 and 1600/100 are processed identically.
Of course, these two might have slightly different noise characteristics because Canon sets some more parameters, not just amplifier gains (maybe NR or who knows what else). In practice I didn't notice much difference.
The histogram always uses the lowest ISO for ETTR reasons (so you can see how far you can push it to the right). If something is covered by at least the lowest ISO, it's not overexposed.
About ISO-less mode: I found that for disco lights (changing all the time, but overall pretty dark with some very bright areas), 1/30 (slower than that results in shake) and ISO 100/1600 covers pretty much everything (some sort of auto that actually gets it right).
The test shot I was looking at had some overexposure that I did not determine during capture. I then assumed that the resolution loss in the highlights was due to not using the darker exposure to fill in the resolution loss of the brighter exposure. I guess because I am always thinking of base ISO as the darker exposure.
Quote from: a1ex on September 12, 2013, 07:12:59 PM
@RenatoPhoto: that's way too complicated. Simply state:
a) how much noise you are OK with (let's say 6EV or 7EV)
b) how many highlights can be ignored (0.1% is a good overall choice, 0.5 for something like a fairly large window in a dark cave where not even 20 stops of DR are enough, and 0% if you don't have any specular highlights).
Then let ETTR do the rest. That's what I do.
Thanks for your reply,
I did the first photos in manual mode to explore the area that the new AETTR + DUAL ISO does not allow me to explore.
For what purpose? So I can compare these iso combinations and results with the results of AETTR + DUAL ISO.
Why do I want to explore this area? Because three days ago I could do it by simply putting my Recovery ISO = +x EV. Now I cant get these results because the new module just decided that this is not what should be done and sets BASE ISO = RECOVERY ISO based on the SNR settings.
Request: Can the DUAL ISO module be configured with an option to do the automatic ISO calculation and one option to do what it did before? i.e.
Auto Dual Iso (ON - OFF)? This would give this module the flexibility to take your optimization or not.
Show me a scene that requires dual ISO, but auto ETTR said it doesn't.
Quote from: a1ex on September 13, 2013, 12:36:31 AM
Show me a scene that requires dual ISO, but auto ETTR said it doesn't.
In the first row of images above,
all of them some of them have lower noise than that obtained by AUTO ETTR + DUAL ISO.
Why? Because I am using higher ISO combinations that the new module does not allow bases on the calculations.
Let me correct that, I get lower noise with higher ISOs that the new module does not allow for the scene. For example I the 100-800 dual iso image is lower in noise than the 100-400 which was the maximum iso range that the module allowed for this scene.
In the images you have AETTR SNR 8/3 @ ISO 400 for auto recovery. Why did you not have SNR 8/4?
You then go on to show in other scenes (with no images), where setting SNR 8/4 resulted in much higher recovery ISOs.
Quote from: Audionut on September 13, 2013, 03:06:33 AM
In the images you have AETTR SNR 8/3 @ ISO 400 for auto recovery. Why did you not have SNR 8/4?
You then go on to show in other scenes (with no images), where setting SNR 8/4 resulted in much higher recovery ISOs.
Yes in this scene I din not think it was required to max out the SNR but later I realized that you could get higher ISO if you max out at 8/4. Under similar conditions as the presented photos I was only able to get 100-800 ISO but no higher as I wished to test.
In similar tests the obtained result under similar ISO are similar but the point I am trying to make and seems difficult to understand is that a user may want to test higher ISOs which are now not available under automated scenarios of AETTR + DUAL ISO. These scenarios where quite easy to use by allowing the ISO module to set the Recovery ISO as a relative number of +xEV. I personally think this is good to leave in the module. Currently the +xEV option is not working in Aettr.
Can you describe a use for manually setting the recovery ISO value other then testing?
In real world shooting, SNR settings are perfectly fine for determining the required amount of recovery ISO. The SNR settings are a fine balance of midtone/shadow SNR requirements vs dual ISO setting. ie: Rather then using recovery ISO 1600 just because, by setting the SNR limits based on actual noise level figures, you could shot the scene with a recovery ISO of 400.
Why is this useful?
SNR limits take into account actual noise figures which directly effect the result.
By allowing the automated code to use these values, your scene at recovery ISO 400 vs recovery ISO 1600 has 2 stops better highlight detail, while still retaining the desired amount of SNR in the midtones/shadows.
Let go of any misconstrued conceptions you have about the SNR setting.
Think of SNR limits as DR control (remember highlight control is directly controlled by highlight ignore and is no longer affected by SNR limits (with dual ISO enabled)). Want more DR (less noise in the midtones/shadows), increase SNR value (which in turn will increase the recovery ISO as needed).
@audionut. Question. How do I work with the test builds? I used to download Lourencos test builds but there seems to be more going on at this forum? For instance If I would like to try any testing with the dual iso module. Do I download the Audionut test build or do I simply add the dual iso module to my build from Lourenco build which is from august 21?
Thanks guys
//D
Looks like the dual_ISO module has made it's way into the new nightly builds :)
http://builds.magiclantern.fm/#/
Wow! Smooth platform :)
Thanks!
//D
Quote from: Audionut on September 12, 2013, 02:54:10 AM
There appears to be a problem with the noise calculation for ISO 6400
Fixed; the noise in top-left black corner was not white, and the blending algorithm was interpreting it as detail (which uses data from both ISO - a lot noisier).
There might be a tiny difference in the output because of slightly different noise values (they are used as thresholds in a few places). Unlikely to be noticeable.
Now I also print the noise level from the high-ISO image (ideally, the noise in very dark areas should reach that value). If it's slightly bigger, it's not really a problem (the noise may sometimes be mistaken for detail); if it's a lot higher (say 1 stop difference or more), it may indicate a bug or a problem in the sensor output (e.g. lots of hot pixels).
Next step: I want to print the SNR values too, in order to check whether ETTR works properly.
Can the spot meter sample different areas of the image simultaneously?
What's the use case? You can move it around.
There was a demo script with many spotmeters, if you want to try to fix the scripting engine.
Using it to print out SNR values for the midtone/shadows on LCD. Overlayed on the midtone/shadow areas of the image?
I repeated the test to check the snr 8/3 setting. The maximum ISO ranged achieved was 100-400.
In DUAL ISO you can see that there is some improvement in noise available above 100-400. I expected to see less noise at hgher ISO like 100-1600 but did not get it.. I cannot explain this but I had seem lower noise at higher ISOs with previous builds. But anyway there is room for a bit of noise reduction and the SNR algorithm does not allow.
It is like saying that cannon should only give you ISO 100-1600 instead of 100-25400 because technically there is no use. But still some people may want to use it, form the usability point of view I am suggesting to expand the range a bit more.
I can imagine that after performing the math, there is not much difference in SNR between the 100-400 and the 100-800, but I see it, so the math should conform to what I see and not to what the numbers tell. After all we always try to make our images look like what we perceive i.e. higher dynamic range for example.
(http://s20.postimg.org/sobkz3df1/New_Comparison_dual_iso.jpg)
No SNR 8/4 test?
Yes, my mistake but the snr 8/4 also had the same 100-400. I will try again and see if I can get the higher ISO. As I said the module limits the ISO range so it not always possible to get higher ISO.
Yes, I was correct, I took some pictures and the same thing at 8/4 get iso of 100-400. The image shows as 8/3 is in fact 8/4
Here is the scene, bright sun with deep shadows:
This one is DUAL ISO 100-3200
(http://s20.postimg.org/gu8vyklgt/Dual_ISO_scene_from_DNG_100_3200.jpg)
Without the SNR 8/4 CR2 to analyze, it's going to be near impossible for a1ex to fix any issues.
My first best guess would be that the area you are showing in your cropped observations above, is below the 5% threshold.
Quote from: a1ex on August 25, 2013, 09:11:48 PM
For shadows, I've chosen the 5% percentile. So, in this context, the shadows having a SNR of 3 EV means 5% of the image pixels have a SNR lower than 3 EV, and the other pixels will be brighter than that.
Keep in mind that 5% is bigger than you may think, since it refers to image area, not linear size. The linear percentage is roughly 22% (1/4.5).
Secondly, I assume those crops are of the extremely dark shadow area in that wood shed? Would you really tone map that to a brightness level where detail is visible?
Would you tone map that area to a brightness level where the detail difference between 100/400 and 100/800 is visible?
If the answer is yes to the second tone map question, have you observed the highlight detail trade offs? It's not fair to only observe the shadow areas when highlight detail is also affected by the EV difference between the 2 ISOs.
Quote from: Audionut on September 13, 2013, 11:09:56 PM
Without the SNR 8/4 CR2 to analyze, it's going to be near impossible for a1ex to fix any issues.
Should be able to get to some bandwidth sometime next week. It seems that this type of condition could be found in many situations where the sun is bright.
Quote from: Audionut on September 13, 2013, 11:09:56 PM
My first best guess would be that the area you are showing in your cropped observations above, is below the 5% threshold.
I kind of feel that I am splitting hairs so the resolution to the issue would be better addressed by allowing the user to simply choose between this type of calculation or not. I was quite happy with the previous version and a simple ON-OFF will be great.
Quote from: Audionut on September 13, 2013, 11:09:56 PM
Secondly, I assume those crops are of the extremely dark shadow area in that wood shed? Would you really tone map that to a brightness level where detail is visible?
Would you tone map that area to a brightness level where the detail difference between 100/400 and 100/800 is visible?
I have already brighten the shadows to expose the noise difference. Although there is less noise at higher ISO combinations it is hardly noticeable. In retrospect the level of noise given the super dark are is supper acceptable at 100-800 or 100-400.
Quote from: Audionut on September 13, 2013, 11:09:56 PM
Secondly, I assume those crops are of the extremely dark shadow area in that wood shed? Would you really tone map that to a brightness level where detail is visible?
Would you tone map that area to a brightness level where the detail difference between 100/400 and 100/800 is visible?
If the answer is yes to the second tone map question, have you observed the highlight detail trade offs? It's not fair to only observe the shadow areas when highlight detail is also affected by the EV difference between the 2 ISOs.
In my opinion there was very little issue with the highlights, the algorithm was working well and keeping overexposure to the 0.2% h.i, so there is no tradeoff. This kind of result is achievable thanks to DUAL ISO.
-----------
Finally and most importantlyI shoot a lot of wildlife and the speed of congruence is very crucial; now when the modules are linked the calculations have become slower so more shots are required. This also is another reason to leave AETTR do its thing and the add DUAL ISO just on top of AETTR as it was before with +-XEV option for Recovery ISO. I had become very happy with setting my AETTR at 4/2 and DUAL ISO at +3EV and shooting without too many delays. Now I cant do this, I just takes one or two or three more shots to get optimum results.
And finally dont take me wrong the images are great and the technology is great, but please put an ON-OFF switch for the more precise calculations. Please!
Quote from: RenatoPhoto on September 14, 2013, 12:20:11 AM
I have already brighten the shadows to expose the noise difference. Although there is less noise at higher ISO combinations it is hardly noticeable. In retrospect the level of noise given the super dark are is supper acceptable at 100-800 or 100-400.
So ideally, the fundamental code base is sound. And you're just splitting hairs.
Quote from: RenatoPhoto on September 14, 2013, 12:20:11 AM
I shoot a lot of wildlife and the speed of congruence is very crucial; now when the modules are linked the calculations have become slower so more shots are required. This also is another reason to leave AETTR do its thing and the add DUAL ISO just on top of AETTR as it was before with +-XEV option for Recovery ISO. I had become very happy with setting my AETTR at 4/2 and DUAL ISO at +3EV and shooting without too many delays. Now I cant do this, I just takes one or two or three more shots to get optimum results.
I have noticed that it's ability to converge quickly is directly related to the initial exposure settings. Start off overexposed and expect problems (especially with the following).
Scene conditions and expectations play a large part. In a scene with a very large DR where you have no, or very little highlight clipping, and you have the SNR settings @ 8/4, you're asking quite a bit.
Relax your expectations of what should be achievable, perhaps use SNR 7/3 instead, and convergence is met much faster.
Having said all that, I have mentioned,
Quote from: Audionut on September 11, 2013, 06:19:53 AM
Or maybe a setting in dual ISO, "link to AETTR". Else dual ISO is always enabled at the settings specified.
I guess then a1ex would have to ignore any SNR settings applied (just consider highlight ignore and leave the user to make his own mistakes). That sounds ugly and isn't something I would expect a1ex to be open to.
Or perhaps fully disable SNR settings when dual ISO is not linked to AETTR.
For the absolute fastest results, you should look at AutoExpo/Dual ISO combination. There are no LV calculations. Point and shoot at burst speed :)
You lose some highlight precision sure, but lets face it, 0.5EV below FWC and 1.0EV below FWC (with the recover ISO taking up the slack), is probably splitting hairs also.
Quote from: Audionut on September 14, 2013, 12:55:46 AM
So ideally, the fundamental code base is sound. And you're just splitting hairs.
These are my observations, I cannot analyze code.
Quote from: Audionut on September 14, 2013, 12:55:46 AM
I guess then a1ex would have to ignore any SNR settings applied (just consider highlight ignore and leave the user to make his own mistakes). That sounds ugly and isn't something I would expect a1ex to be open to.
Or perhaps fully disable SNR settings when dual ISO is not linked to AETTR.
Just give the user a choice between slow converging exact process or a quick process that produces excellent results.
We know that DUAL ISO should be only used in scenes where the dynamic range is greater then what the camera can capture. So that means we are going to capture the highlights as well as the shadows. We have also determined that h.i. controls the highlights and so to increase dyamic range we protect the highlights with h.i. and capture the shadows with DUAL ISO. That is what I have been doing before. The SNR numbers give me a scale of shadow recovery which is very complicated 32 different options while DUALISO gives me 6 options. For this reason, when I was using AETTR+DUAL ISO I was leaving SNR OFF and then enabling the DUAL ISO at a reasonable number (ie +3EV) so I could capture the details in the shadows at acceptable noise level.
In the scenes where the dynamic range is not great, or in scenes where I am not concerns with the highlights, the I USE SNR 4/2 and 6/3 as I see fit based on experience. This has always given me good results. I even tested this in Timelapse with very good results see here:
Dual ISO vs Auto ETTR
http://www.magiclantern.fm/forum/index.php?topic=7835.0
This new system took away everything that I like and replaced it with a better, more accurate, splitting hairs system, which I may want to use from time to time when time allows and I can take a few shots at a static scene.
Quote from: Audionut on September 14, 2013, 12:55:46 AM
For the absolute fastest results, you should look at AutoExpo/Dual ISO combination. There are no LV calculations. Point and shoot at burst speed :)
You lose some highlight precision sure, but lets face it, 0.5EV below FWC and 1.0EV below FWC (with the recover ISO taking up the slack), is probably splitting hairs also.
Interesting, I will have to look into this option. I tried it a couple of times but so many freaking curves that I could not make scense of what this thing is doing. It needs a user manual for dummies like me.
Quote from: RenatoPhoto on September 13, 2013, 01:27:38 PM
I can imagine that after performing the math, there is not much difference in SNR between the 100-400 and the 100-800, but I see it
I can barely see any difference between 100/400 and 100/6400. Maybe you can use a higher contrast for that area?
Maybe I should check the standard deviation of the noise in some out-of-focus area.
@Audionut
"For the absolute fastest results, you should look at AutoExpo/Dual ISO combination. There are no LV calculations. Point and shoot at burst speed :)
You lose some highlight precision sure, but lets face it, 0.5EV below FWC and 1.0EV below FWC (with the recover ISO taking up the slack), is probably splitting hairs also."
Been fiddling with the Auto exposure settings and it seems to work very well with dual iso. My settings right now pointing out the windows seems to give consistent good results;
AV range, I used f5.6-f5.6 but free of choice of course
TV minimum 1/44, free of choice of course
EC range -1.5 EV- +3.0 EV try this one out a little
EC +3.0 EV probably the most important setting to change. Try +2.0 EV or til you get it as you want
ISO range 100-3200
The big plus here is you don,t need to open up liveview to get a reading as with Aettr. The low iso, though, has to be changed manually from within dual iso.
Gonna try some more...
*After developing I can only say wow. This autofeature is exactly what I,ve been looking for. If an automated feature were implemented to disable dual iso when in not so contrasty areas and maybe also have an auto setting for the lower iso setting I could finally shut of my brain and start taking pictures ;)
*So, to simplyfi. Set auto exposure to overexpose by two or three steps, then hit dual iso to get the blown highlights back. Am I right?
//D
Quote from: Danne on September 14, 2013, 10:51:17 AM
Set auto exposure to overexpose by two or three steps, then hit dual iso to get the blown highlights back. Am I right?
Only for your particular test scene ;)
That's because auto expo meters for midtones and ignores highlights. Try on a blank wall, then try pointing it out the window. I just did on 5D3: on the blank wall it underexposed by 2-3 stops, on the window it overexposed by 3-4 stops. I've tried evaluative and center weighted, could not see any difference between them.
QuoteIf an automated feature were implemented to disable dual iso when in not so contrasty areas...
That's exactly what auto ETTR does (and Renato is reporting it as bug).
QuoteOr maybe a setting in dual ISO, "link to AETTR".
What about a setting like this in ETTR?
Link to Dual ISO
- OFF (old way)
- SNR priority (as it's now)
- Highlight priority (similar to SNR, but if dual ISO reaches 100/max, ignore the SNR and capture all the highlights).
I'm still working on convergence issues; with ETTR in LiveView (on request via SET button), convergence is quite fast and you don't have to waste any test shots.
The biggest problem when metering outside LV is that current heuristics (simple and fast) sometimes get tricked because of black level difference. With a black difference of around 30 (typical for ISO 100/6400), the metering error can be up to
4 stops (!).
In LiveView, ETTR is metering on only one of the two exposures (no black difference), and estimates the other exposure from DxO data. That's why it works a lot better than metering outside LV.
Another cause of non-convergence is that 1600/3200/6400 are almost identical, so the algorithm may switch from one to another just because of round-off errors. This one should be easy to fix.
Quote from: a1ex on September 14, 2013, 11:36:48 AM
That's exactly what auto ETTR does (and Renato is reporting it as bug).
Hi Alex,
I like the feature very much, I like the concept of using DUAL ISO only when the dynamic range requires of its use. ;) Please dont take it out, it has its uses.
I find the implementation a bit difficult to use with ease and speed. I find the final hair splitting a bit inconsistent and difficult to obtain.
I am suggesting in order of importance:
1. Implement an ON OFF switch for the more robust and automated calculations.
2. When the switch is in OFF position,
and DUAL ISO is enabled set h.i.=0.2%, SNR=OFF, have AETTR to its thing, have DUAL ISO take the Base ISO and add the user selected Recovery ISO via +-XEV
3. If user has switch in OFF position,
and DUAL ISO is enabled and the user wants to use other settings for SNR or h.i., then let him but warn him that these settings work best with the automated version. But I suppose there is no sense in setting h.i=5% when using DUAL ISO, it defeats its use since you no longer require DUAL ISO to capture the whole dynamic range. The SNR settings could be fiddle with as long as the highlights are not clipped. If the user boosts SNR and the scene is overexposed then warn the user.
3. When the switch is ON your calculations take over, the SNR is optimized via combination of AETTR and DUAL ISO.
I see you posted above while I was writing..
Quote from: RenatoPhoto on September 14, 2013, 01:47:16 PM
But I suppose there is no sense in setting h.i=5% when using DUAL ISO, it defeats its use since you no longer require DUAL ISO to capture the whole dynamic range.
Sometimes you do. I had to do this when taking photos in a cellar - there was a bright window (something like this (http://www.festeggiata.com/included/images/cellar04_600x400.jpg) - not my picture, but a similar scene), and it was impossible to have detail there and see something in the cellar, so I had to ignore it. Of course, dual ISO maxed out.
Also street lights: they are usually so bright that dual ISO can't cover them, and you have to ignore them (otherwise you may get a lot of noise).
Quote from: a1ex on September 14, 2013, 01:14:23 PM
What about a setting like this in ETTR?
Link to Dual ISO
- OFF (old way)
- SNR priority (as it's now)
- Highlight priority (similar to SNR, but if dual ISO reaches 100/max, ignore the SNR and capture all the highlights).
I am sure that your fingers will do the magic! ;D
Can someone tell me what i am doing wrong, when the cr2hdr.app(OSX) is not processing the dng files of my raw video?
"Hmm... There are no more Dual ISO .CR2 left in this folder."
do i have to rename the dng to cr2?
Quote from: a1ex on September 14, 2013, 01:14:23 PM
What about a setting like this in ETTR?
Link to Dual ISO
- OFF (old way)
- SNR priority (as it's now)
- Highlight priority (similar to SNR, but if dual ISO reaches 100/max, ignore the SNR and capture all the highlights).
They look good.
I like the highlight priority option. I find it tends to use 100/max twice at the end and then gives up. This would have to help convergence (I shot through the viewfinder), as instead of giving up after 4 shots, based on my testing it would often give up after 2 (save the highlights and be done with it).
Quote from: a1ex on September 14, 2013, 01:14:23 PM
The biggest problem when metering outside LV is that current heuristics (simple and fast) sometimes get tricked because of black level difference. With a black difference of around 30 (typical for ISO 100/6400), the metering error can be up to 4 stops (!).
In LiveView, ETTR is metering on only one of the two exposures (no black difference), and estimates the other exposure from DxO data. That's why it works a lot better than metering outside LV.
Can you make it sample just the darker exposure (outside of LV)? Would reading the current setting of dual ISO help to determine the black level difference?
Quote from: mikeortner on September 14, 2013, 04:39:38 PM
Can someone tell me what i am doing wrong, when the cr2hdr.app(OSX) is not processing the dng files of my raw video?
"Hmm... There are no more Dual ISO .CR2 left in this folder."
do i have to rename the dng to cr2?
First look at the cr2 files at 100% or 200% magnification and you should see the interlaced dual iso. If you do not see the horizontal lines then you do not have dual iso image.
I cannot help you with apple stuff..
QuoteCan you make it sample just the darker exposure (outside of LV)?
If the bright exposure has black level at say 2033 and the dark one has black level at 2063, in deep shadows, the current code will sample the bright exposure thinking it's actually the dark one (since it simply returns min or max from two neighbor pixels).
Ideally, bright exposure = dark exposure * gain (that is, bright EV = dark EV + ISO spacing in EV).
Say the midtones are at 3 EV above black in the dark exposure (that is, 2063+8). In the bright exposure, they will be at 2033+8*64 for ISO 6400. So, midtones will be metered at 4.5/8.95 instead of 3/9 (so, at 3 EV I will have a 1.5-stop error in metering).
To correct this, I could either:
- figure out which lines are bright and which lines are dark before sampling anything (requires rewriting a lot of things)
- figure out how big the error is (because I can find the black difference easily) and correct the metered values
Edit: actually the exposure inversion is not a very big problem (it only appears in really deep shadows); the problem is the magnitude of the black level difference (which alters metering results).
does cr2hdr work on 650d RAW files?
i converted the 650d dual iso RAW file to DNG using raw2dng and then changed file names to .CR2 and then tried opening with the cr2hdr for mac, but got this error:
Input file : 000000.CR2
sh: exiftool: command not found sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile : 000000.txt
Quote from: RenatoPhoto on September 14, 2013, 02:41:31 AM
We know that DUAL ISO should be only used in scenes where the dynamic range is greater then what the camera can capture.
Not any longer, not with SNR limits. It can also be used to tone map in camera. Say I have a scene of 9EV of dynamic range, but in this scene the skin tones are at -5EV or lower, I can enable the required SNR limit, and boost the skin tones to -3EV (much less noise in critical areas).
I haven't increased DR, because the DR was scene limited. I did lift the skin tones though.
With adjustments to both midtone/shadow detail, on scenes with large dynamic range, I can also do my best in camera to lift skin tones while leaving the shadows to fall where they may (minimize EV spacing).
Quote from: khurra on September 14, 2013, 05:51:21 PM
does cr2hdr work on 650d RAW files?
i converted the 650d dual iso RAW file to DNG using raw2dng and then changed file names to .CR2 and then tried opening with the cr2hdr for mac, but got this error:
Input file : 000000.CR2
sh: exiftool: command not found sh: dcraw: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile : 000000.txt
They work but you are missing the dcraw executable
Quote from: nanomad on September 14, 2013, 08:17:01 PM
They work but you are missing the dcraw executable
so i need to follow these instructions: http://bellavist.com/blog/?p=466
on my computer running OSX lion and then try running the mac gui verison of cr2hdr again?
thanks for your help nanomad!!
ETTR seems to work a bit better now with dual ISO. There's a little quirk that you may want to be aware of:
- without dual ISO, it will try hard to meet or exceed the SNR limits (clipping as many highlights as needed)
- with dual ISO, SNR limits are respected until dual ISO gets maxed out (100 / max auto ISO). After that, it will not clip any more highlights than allowed by "highlight ignore" (but it will also get a lower SNR).
I recommend triggering it via SET (to meter in LiveView) whenever the lighting changes. This mode converged very quickly in my tests, even in the most difficult lighting (I've tried a light bulb in a very dark room).
Quote from: a1ex on September 14, 2013, 11:41:30 PM
ETTR seems to work a bit better now with dual ISO. There's a little quirk that you may want to be aware of:
When i try to do hg update I get the following error and ettr.c does not update. Any ideas what I can do?
hg update
modules/bolt_rec/README.rst: untracked file differs
abort: untracked files in working directory differ from files in requested revision
magiclantern@magiclantern-VirtualBox:~/magic-lantern$
Try deleting it.
http://stackoverflow.com/questions/4822129/mercurial-untracked-file-in-working-directory-differs-from-file-in-requested-r
Quote from: a1ex on September 15, 2013, 12:08:59 AM
Try deleting it.
Thank you very much. I deleted README.rst
and hg update...
Now I am testing. It feels wonderful!
Thank you again your are a magician!
I had a quick look at the images and everything has fallen in its place.
Nice images, good capture of dynamic range, proper automatic decisions on when to use DUAL ISO, very responsive to SNR settings. The usability factor seem very good.
It seems PERFECT! (I will probably find something else to complain about... :o)
If I could, I would nominate you for a Nobel Prize for ingenuity. ;D ;D ;D
P.S I was forgetting to mention that now the camera can capture what I see and beyond!!!!
P.S. 2 Maybe they should add this features to imaging space telescopes and I bet we could see little green men on the sun!!
Quote from: RenatoPhoto on September 14, 2013, 05:31:59 PM
First look at the cr2 files at 100% or 200% magnification and you should see the interlaced dual iso. If you do not see the horizontal lines then you do not have dual iso image.
I cannot help you with apple stuff..
Yes the lines are there, i already checked that before ... maybe i should try the PC version
The aettr together with dual iso works really well now :). THanks for that. Trying to understand when to use what settings in snr for shadow and midtone. 8/4 is the most extreme and 1/1 the least extreme sort of speak. What would be the best way to find rules for when to use what? I read comments about skintones etc. Anybody tried any settings? Is there a way to visually understand the optimal snr settings maybe with raw zebras, histogram?
Thanks again
//D
Audionut did some tests here and I chose the defaults according to them: http://www.magiclantern.fm/forum/index.php?topic=5200.msg70150#msg70150
Also see this: http://photography-on-the.net/forum/showthread.php?t=1326078&page=2&highlight=snr
I'm thinking to draw something to show where midtones and shadows are metered exactly. If you remember the old bulb ramping with percentiles, it would be something like that (since the math is the same).
Quote from: a1ex on September 15, 2013, 10:17:35 AM
Audionut did some tests here and I chose the defaults according to them: http://www.magiclantern.fm/forum/index.php?topic=5200.msg70150#msg70150
Also see this: http://photography-on-the.net/forum/showthread.php?t=1326078&page=2&highlight=snr
I'm thinking to draw something to show where midtones and shadows are metered exactly. If you remember the old bulb ramping with percentiles, it would be something like that (since the math is the same).
Alex, I noticed something today. With Dual_ISO & AETTR on my 5DM3 on the numerical EV showing at the bottom right end of the screen under my light situations shows +0.3 EV when ML ISO is at 100 and Canon ISO at 3200; and +4.0 EV when ML ISO is at 3200 and Canon ISO at 100. The same for my EOS M.
Could there be a valid explanation for the cameras behaving this way?
Canon metering always meters the image that you are previewing.
Quote from: a1ex on September 15, 2013, 10:39:33 AM
Canon metering always meters the image that you are previewing.
Okay. That makes sense. Thanks as always.
I'm testing new build with auto ettr & dual iso link on my 50D. The idea is great, photos look nice. Some issues:
- Quite often auto ettr starts (LV turns on for a split second and turns off), but It does not do anything. It does not say "Expo out of limits" or "Whoops". Thinking it's on I shoot and get much underexposed photo. When I turn LV on and press SET, Auto ETTR works fine. Would it be possible to write "auto ettr ok" on image review after the shot if ettr worked and finished with no errors?
- auto snap and always on does not work on 50D still. It works with 1%'s tragic lantern builds – tl50d compiled by Andy600, but not nightly builds.
It looks like exposure limits for auto ettr is greater than before, too.
Status messages solved. Also found some more bugs regarding convergence.
Exposure limits are Canon limits, nothing should change here. I didn't check them very well though.
For auto snap, one has to compare 1% repo with ML one, extract the relevant pieces of code (most likely ettr.c, dual_iso.c and the raw backend) and create a pull request. I prefer this to be done by a 50D user.
@alex (sorry if this request has been issued before, you know, long threads and all...):
Could you please change the "dr gained" field into a setting so that the user can set a "min. dr gain" value? On high iso, doing dual iso doesn't make much of a difference dr-wise and isn't worth the postprocessing hassle, so this would only enable dual_iso on lower base iso.
THanks for linking ALex. Seem 6ev is a good compromise getting about enough grain.
I haven't done any tests yet, but judging from that last test, I would use 7EV or 8EV for the midtones with dual ISO, depending on your preference of EV separation.
6 EV could lift the midtones well while still retaining good highlight detail, back before it had the help of dual ISO. 7EV should now give you good midtone detail with little EV separation (100/400). 8EV would give you excellent midtone detail (metered correctly), with only 100/800.
Of course the recovery setting is dependent on scene luminance.
Thanks for clarifying.
Reporting a repeatable bug 5D3, compile ae68c60000f2
Tried to use AETTR+DUAL ISO for a night-to-day timelapse.
The module started ok at typical setting ie. 30 seconds at iso 6400. As light increased the exposure never changed and remained at 30 seconds iso 6400. I caught it early but if the sun hit my sensor it could have fried it.
I repeated the test under my desk in a very dark place and then pointed to my monitor with lots of light. The exposure never changed from 30 sec at iso 64000.
Settings used:
Manual lens f/22, iso 6400, 30 second shutter, Canon in M mode, Image review 2 sec
AETTR settings: Trigger mode: Always on, Slowest shutter: 32", Exposure target: -0.5, H.I:02%, Allow clipping:Green channel, Midtone SNR limit: 7EV, Shadow SNR limit:2EV, Link to Canon:OFF, Link to DUAL:ON
DUAL ISO: ON
Intervalometer: Take pic every 1 m, Start after 3s
Post Deflicker: Adobe xmp, 50%, -4 EV
Good catch.
What's happening here is that sensor saturation point (at least on 5D3) goes lower than usual, so the zebra indicator and ETTR analysis don't even realize the image is overexposed (they think it's just perfect).
If you change the exposure target at -1 EV, it will start correcting the overexposure slowly (one stop at a time). It will continue to do that until the white level returns to normal.
At 8 seconds, the white level looks normal (from there it converged from half-screen blown out to perfect exposure in 2 steps). Didn't try in-between shutter speeds.
The issue is quite subtle and I'm not sure how to fix it in a portable way. Maybe with some guessing like in cr2hdr.
Quote from: a1ex on September 16, 2013, 03:46:14 PM
What's happening here is that sensor saturation point (at least on 5D3) goes lower than usual, so the zebra indicator and ETTR analysis don't even realize the image is overexposed (they think it's just perfect).
Maybe this is a good time to add a check based on canon brightness value.
The new metered areas is handy.
Currently it just continues to draw over itself. Is there a way to determine if the scene has changed dramatically and clear the overlay for a fresh draw? I don't know how to do this, so not sure of the best way to stop it continually updating where it doesn't need to.
edit: Since it seems to just draw everything again (when updating), what about just clearing the old overlay first before drawing again?
What about linking it to zebras? An option to show either the zebras as they are now, or the metered areas. Not sure of the cleanest way to do this.
Can it be enabled for QR?
Heh, I've only tested it in QR.
I must need sleep then. It works fine in LV, and works fine on press set, but nothing shows in the quick image review.
Quote from: a1ex on September 16, 2013, 03:46:14 PM
Good catch.
What's happening here is that sensor saturation point (at least on 5D3) goes lower than usual, so the zebra indicator and ETTR analysis don't even realize the image is overexposed (they think it's just perfect).
If you change the exposure target at -1 EV, it will start correcting the overexposure slowly (one stop at a time). It will continue to do that until the white level returns to normal.
At 8 seconds, the white level looks normal (from there it converged from half-screen blown out to perfect exposure in 2 steps). Didn't try in-between shutter speeds.
The issue is quite subtle and I'm not sure how to fix it in a portable way. Maybe with some guessing like in cr2hdr.
Thanks Alex, I tested with Timelapse today and worked properly. Now I have to figure out how to process some of the dual iso images in Timelapse.
Did a shoot last night with the newest build. Used auto ettr linked to dual iso. Not sure I'm doing it right, but got good results with the standard settings. After cr2hdr, the dng files seemed under exposed by about 1.5-2 stops but I was able to raise them without a problem. Is this the way it is supposed to work, or am I missing something? Thanks!
https://docs.google.com/file/d/0B_Id0y3JG-KEdU9qZGQ2WkFuQTQ
I think you get the dark exopsure which is the way it was exposed keeping the highlights. As you say, to lift the shadows are not any problem
Reporting new test of TIMELAPSE with ETTR and DUAL ISO link:
5D3 with yestedays compile before memory back-end. be77944f2b0d
I tested again night to day transition using ETTR+DUAL ISO, SNR 6/3. The previous exposure problems are gone.
The DUAL ISO module did not become necessary until the dynamic range increased when daylight started to increase the brightness of the sky. All cr2 files where able to read the the xmp in ACR. All the xmp files had adjustment applied to cr2 and then exported to DNG.
I was able to process the DUAL ISO (DNG) files with this workflow:
1. Opened all cr2 with ACR and exported as DNG, then processed all DNG with cr2hdr, then dng to tiff with ACR.
And the beginning of DUAL ISO enhancement, there were times when one image would be dual iso and then the next did not, after development and final review the images blended very well. Except for a couple of times when a small flicker appeared. By simple analysis I could tell which was dual iso and which was standard.
I did not let the system go into full daylight but I think that it is important that the dual iso system does not go beyond 100/800 iso to maintain good images. For this reason I think the use of dual iso module should be limited to SNR settings no higher than 6/3. A setting like 8/4 would raise the dual iso to 100/6400 which would deteriorate the image resolution and colors.
Hopefully will be able to run a new test with extended times for a more complete analysis.
The dickering with slide car files is amazing!
Quote from: RenatoPhoto on September 18, 2013, 03:56:38 AM
Reporting new test of TIMELAPSE with ETTR and DUAL ISO link:
5D3 with yestedays compile before memory back-end. be77944f2b0d
Hi RenatoPhoto, where can I get hold of the binary for the be77944f2b0d? Searched, the forums but could not find any refernce to this specific version.
To limit the max ISO, change it in Canon menu (max auto ISO).
Quote from: Canon eos m on September 18, 2013, 06:40:52 AM
Hi RenatoPhoto, where can I get hold of the binary for the be77944f2b0d? Searched, the forums but could not find any reference to this specific version.
You would have to compile it yourself. I suppose today's build should be as good, I don't know if there are any issues.
Quote from: a1ex on September 18, 2013, 07:53:14 AM
To limit the max ISO, change it in Canon menu (max auto ISO).
The problem is I need the base ISO of 6400 for the night shots but I dont want DUAL ISO to go very high.
Quote from: RenatoPhoto on September 18, 2013, 01:51:17 PM
You would have to compile it yourself. I suppose today's build should be as good, I don't know if there are any issues.
Don't know anything about building. Guess, will wait till I a complied version is available.
Quote from: Canon eos m on September 18, 2013, 05:16:08 PM
Don't know anything about building. Guess, will wait till I a complied version is available.
http://builds.magiclantern.fm/#/
Quote from: RenatoPhoto on September 18, 2013, 01:52:57 PM
The problem is I need the base ISO of 6400 for the night shots but I dont want DUAL ISO to go very high.
That's why I recently proposed to turn the "dr gained" field into a setting - if you for example set it to at least 1.0 ev, dual_iso would be disabled on high iso settings
I used ETTR + DUAL ISO to do a timelapse from night to day.
The workflow is:
Open all cr2 images and save them as DNG. This puts the deflicker adjustment (in xml) in the DNG file for further processing.
Process the files with cr2hdr. This processed the DUAL-ISO dngs and saves them as a normal DNG.
At this point I can import the DNG into Vegas and make the video. It is perfect at this point.
Now I want to generate tiffs which can be handled by other video editors, not all can read the dngs.
To do this I open the DNGs in ACR and notice that the deflicker exposure is in the exposure field. If I export the tiffs with this exposure number then it generates flicker, it appears that this exposure has already changed the image itself so the exposure number is not needed for further processing. So I select all the dngs in ACR and set the exposure to 0. Problem fixed, now the tiffs can be imported to video editor and there is no flicker.
Finally I get another problems that I cannot resolve:
The Dual ISO dngs that were converted to tiffs get some really strange effects like if there was a crop! After analyzing the faulty images in ACR I noticed that the DUAL ISO dngs have a CAMERA PROFILE set to embedded and cannot be changed, while the standard dngs have CAMERA PROFILE = Adobe Standard. I cannot find anything else that is different from the two files so I wonder if this is causing this strange effect.
Something interesting to note is that the video made with the dngs coming out of cr2hdr is ok. I assume that Vegas cannot read the much of the embedded information and is just importing the image. When I import these dngs into ACR to make the tiff is when this problems arises.
Do they both have the same resolution? If not, try developing the CR2 files with dcraw.
Quote from: a1ex on September 18, 2013, 10:48:36 PM
Do they both have the same resolution? If not, try developing the CR2 files with dcraw.
They are cropped The tiff from standard dng is 5796x3870 while the tiff form dual iso dng is 5760x3840.
Swap the numbers ;)
Quote from: a1ex on September 18, 2013, 10:57:44 PM
Swap the numbers ;)
Right: standard dng is 5760x3840 while the tiff form dual iso dng is 5796x3870.
Ok I was able to trim the dual iso tiffs using Photoshop action and batch.
The timelapse is good but I wll try it again with different settings.
I got a problem with conversion from interlaced dual iso dng to normal dng. I shot fire in the dark with fps override to 65 fps. Brighter shots are converted correct, but darker aren't recognized as interlaced. Even in the same take one (brighter) frame is recognized well...
Input file : E:\81_22-0544_00016
5.DNG
Canon EOS 5D Mark III detected
Full size : 1920 x 508
Active area : 1920 x 508
White level : 12500
Black subtract didn't work
ISO pattern : BddB RGGB
Noise levels : 8.00 8.00 8.00 8.00 (14-bit)
Estimating ISO difference...
ISO difference : 4.61 EV (2450)
Black delta : 67.67
Interpolation : mean23-chroma5x5-alias
Dynamic range : 10.35 (+) 10.35 => 14.97 EV (in theory)
Matching brightness...
Looking for hot/cold pixels...
Full-res reconstruction...
ISO overlap : 3.0 EV (approx)
Half-res blending...
Chroma filtering...
Building alias map...
Filtering alias map...
Smoothing alias map...
Noise level : 8.00 (16-bit), ideally 8.00
Dynamic range : 12.35 EV (cooked)
Output file : E:\81_22-0544_00016
5.DNG
Image/ExifTool/MinoltaRaw.pm did not return a true value at (eval 149) line 3.
Can't find table Image::ExifTool::MinoltaRaw::PRD
Image/ExifTool/MinoltaRaw.pm did not return a true value at (eval 151) line 3.
Can't find table Image::ExifTool::MinoltaRaw::PRD
Image/ExifTool/MinoltaRaw.pm did not return a true value at (eval 154) line 3.
Can't find table Image::ExifTool::MinoltaRaw::PRD
Image/ExifTool/MinoltaRaw.pm did not return a true value at (eval 156) line 3.
Can't find table Image::ExifTool::MinoltaRaw::PRD
1 image files updated
...and next other not.
Input file : E:\81_22-0544_00016
6.dng
Canon EOS 5D Mark III detected
Full size : 1920 x 508
Active area : 1920 x 508
White level : 12500
Black subtract didn't work
ISO pattern : BdBd RGGB
Interlacing method not supported
ISO blending didn't work
Input file : E:\81_22-0544_00016
7.dng
Canon EOS 5D Mark III detected
Full size : 1920 x 508
Active area : 1920 x 508
White level : 12500
Black subtract didn't work
ISO pattern : dBBd RGGB
Noise levels : 8.00 8.00 8.00 8.00 (14-bit)
Estimating ISO difference...
Doesn't look like interlaced ISO
ISO blending didn't work
Is there a way to force cr2hdr.exe to convert dngs or other method than cr2hdr.exe?
Please, help. This footage is extremely important to me.
Camera: 5DM3
tried different versions of cr2hdr.exe
Well, if you really want to remove the error checks, feel free to do so, but you'll get just garbage.
Hint: I can't troubleshoot anything from these logs (other than reaching the above conclusion).
Quote from: a1ex on September 19, 2013, 09:23:13 PM
Well, if you really want to remove the error checks, feel free to do so, but you'll get just garbage.
Hint: I can't troubleshoot anything from these logs (other than reaching the above conclusion).
I attached these logs just to show where it stuck. I have no programming skills, I simply thought that maybe it's a matter of tolerance in ISO difference estimating process. Is it possible to do the conversion in photoshop? Any idea?
Quote from: jOt on September 19, 2013, 10:41:44 PM
I attached these logs just to show where it stuck. I have no programming skills, I simply thought that maybe it's a matter of tolerance in ISO difference estimating process. Is it possible to do the conversion in photoshop? Any idea?
maybe you can upload the two cr2 for some one to analyze.
Quote from: RenatoPhoto on September 19, 2013, 11:01:15 PM
maybe you can upload the two cr2 for some one to analyze.
If somebody is able to convert second file, let me know, please.
http://www.mediafire.com/?kr1che73pz9tmdl (http://www.mediafire.com/?kr1che73pz9tmdl)
http://www.mediafire.com/?dxoaa17l87dfgec (http://www.mediafire.com/?dxoaa17l87dfgec)
Shot today in DualIso, using stock ETTR settings, pushed Set in LiveView to meter. Ended up with a bunch of files that can't be converted. Here's the first one. I'm using the last experimental update from a1ex a few pages back. Thanks!
https://docs.google.com/file/d/0B_Id0y3JG-KEQldiSFNTSWpJbWs/edit?usp=sharing
That's almost completely overexposed in one ISO. Can you reproduce it?
A video of the camera screen when metering the scene should help. None of my ETTR pictures in full sunlight looked like this...
Latest cr2hdr should handle these shots properly.
Last build (2013-09-20, second run) tested on 5D mkII in both ways (high light and shadow recovery): works well. :)
I just noticed when base ISO is high (like 5000 ISO) hot pixels are not fully registered, compared with a regular single RAW shoot at the same ISO.
But with a good de-rawtizer soft it's easy to set base ISO at 1600 and push to 5000 eq-ISO.
I don't get it, but if you shoot above 1600 on 5D2 you are just throwing away useful data.
Quote from: a1ex on September 20, 2013, 08:27:09 AM
That's almost completely overexposed in one ISO. Can you reproduce it?
A video of the camera screen when metering the scene should help. None of my ETTR pictures in full sunlight looked like this...
I'll try again today with a video camera on the screen. I re-downloaded the cr2hdr from the first post and it looks to be converting the missed files ok. Thanks,
Hi,
I apologize in advance. I am sure this has been asked ad nauseam but couldn't find it. I am not sure what am I doing wrong here but I get an error when I drag the dng files to cr2hdr (mac user) saying files not supported. I suspect I am not dialing in the right settings for the Dual ISO mode. Can you please help.
5D2 + Mac
- I the installed the latest build 4b3129d5a739
- Turned 'Disable all modules' to OFF
- Selected all the modules and rebooted
- Changed the Quality in Canon menu to JPEG (Does this need to be RAW?)
- Set RAW video ON (1880x1058)
- Set HDR to OFF
- Set Dual ISO to ON (100/1600)
I take the .RAW and convert to dng using RAW2DNG
I drag all the .dng files in a folder to CR2HDR but I get an error
Also the dng files look like regular RAW files and not the Dual ISO files like I expect to see them with interlaced ISO.
Completely new to this. Thank you
Forgot to add that I am using the MAC GUI CR2HDR (Do I need something else too)
Today I tested some dual iso + aettr. Notifications after settling ettr is great. Two things I've noticed:
1. In Av mode I sometimes get "ettr settled at iso 0/1600". I think it was when I had auto iso turned on in Av mode.
2. I get some photos without "dual" prefix in filename that are really dual iso. These files even have "Dual-ISO" keyword.
I would love to have an option in Aettr and dual iso to turn it off while not in M mode. Now if I have to take a quick shot without aettr and dual iso I have to turn these off in menus. Using aettr in Av even gives additional problems — if I turn auto iso, the option gets removed by auto ettr in work.
Below is an example of a photo not possible without Dual-ISO. Not a very good one, but you can see both shadows (desks, closed doors), and very bright areas (in open doors). Canon EOS 50D, which is not good for low-light photos. Normally such shot require multiple exposure. In this case people would move and there would be quite a work in Photoshop. The problem is the fringing around people in the door. In my Photoshop CS5 there's no option to remove chromatic aberration in ACR (it appeared in CS6, process 2012), and I don't get good results in Lens Correction filter in Photoshop. I tried it in trial Photoshop CC with somewhat better results, but not great. I guess this is a "special" case of fringing, appearing only in Dual-ISO? Anyways, making near-HDR photos with one shot is a much greater advantage than some additional fringing :)
(http://i.imgur.com/OI50Pm8.jpg)
I'm still trying to rescue my shot, since not all of the frames converted. Cr2hdr failed on low key shots. Alex, would it be possible to create a version of cr2hdr without automated recognizing interlace, which extract just one iso channel, at half res, just to get rid of interlacing on unconvertable images?
It requires the automatic recognizing to know which fields are bright and which are dark. This varies from camera to camera and sometimes from frame to frame.
Can you upload another CR2?
Quote from: a1ex on September 22, 2013, 09:49:53 PM
It requires the automatic recognizing to know which fields are bright and which are dark. This varies from camera to camera and sometimes from frame to frame.
Can you upload another CR2?
Thanks for explanation. In attached images fields are changing from frame to frame. Maybe a command of indication if the top field is bright or dark would rescue them?
http://www.mediafire.com/?20d3fr3hjrspb6n
http://www.mediafire.com/?j9lacm5cj4moohd
http://www.mediafire.com/?i4pmhpejwt61l5r
http://www.mediafire.com/?q8bkbh0vcq7n9he
http://www.mediafire.com/?exqi8cqvajlgkat
Hmm.. I looked at one of those and there is so little data its probably confusing the algo. I guess it would have to check from the right where the barrel? is.
I think the confusion comes from the black level difference: in dark areas, the low ISO pixels are brighter than the high ISO ones. The difference is usually small (say under 50 units), but it accumulates.
This should be the key to solving the autodetection.
A command-line option may help if you are willing to process each shot manually and fiddle with the offset until you get it right (I'm not). There are 4 values that you have to try.
Quote from: a1ex on September 23, 2013, 08:03:26 AM
A command-line option may help if you are willing to process each shot manually and fiddle with the offset until you get it right (I'm not). There are 4 values that you have to try.
It would be better than nothing - given the time cr2hdr takes to convert an image, batch-conversion with all 4 possible values wouldn't make much of a difference, at least to me if one of them has as little artifacts as possible.
If cr2hdr has any means to guess which of the 4 options is the correct one it would be nice to be able to append a "confidence" level or something like this to the file name so when sorting manually through them users would know which one is most likely to be the best.
Personally, I'm unlikely to use dual_iso all the time, so with the remaining images where I really needed it I'm prepared to spend a little time converting it.
Only one of these 4 values will result in a valid image.
Quote from: a1ex on September 23, 2013, 08:22:06 AM
Only one of these 4 values will result in a valid image.
Ok, I didn't understand that - but imho one more reason to make them available from the command line if the (rare?) case should occur that the dual_iso autodetection chooses the wrong one? It's not like it'd be a big coding effort :-p
QuoteIt's not like it'd be a big coding effort :-p
In this case, feel free to submit a patch ;)
Quote from: a1ex on September 23, 2013, 09:29:36 AM
In this case, feel free to submit a patch ;)
Nonono, you got that wrong - it's not much of an effort for *you*, for *me* "Hello World" takes quite some time :-p
But to be earnest: I find submitting a full fledged pull request as it's desired around here to be too much overhead for trivial patches (long explanation cut, someone has to read this...) :-\ ... just posting a diff in a code box seems easier to me.
Also like in this case looking at some source code for the first time - and I'll never need to look at cr2hdr again - takes some time to get the bearings, just like working on the ml source takes some time if some months have passed. What I do in 1h you'll do in 10sec - I don't even remember how linux command lines are parsed and where the 4 options you mentioned might be located.
That's why I usually put my trivial or "open for discussion" ideas as a comment or request first, and only if push comes to shove change the source code myself. I know from dev's perspective this might look lazy, but just now I think for small matters this is a more economical way.
Speaking with Star Trek: I'm a doctor (no, actually I'm a sociologist) not a hardcore coder :-p
Quote from: a1ex on September 23, 2013, 08:03:26 AM
A command-line option may help if you are willing to process each shot manually and fiddle with the offset until you get it right (I'm not). There are 4 values that you have to try.
I'm begging for a patch. Everything would be better than cutting and pasting 2 pixel lines in photoshop on 50 frames :o. Please!
Quote from: jOt on September 23, 2013, 09:56:32 AM
I'm begging for a patch. Everything would be better than cutting and pasting 2 pixel lines in photoshop on 50 frames
But it would teach you how to use photoshop actions :->
Quote from: Marsu42 on September 23, 2013, 10:14:31 AM
But it would teach you how to use photoshop actions :->
:D I know actions. Resulted images from photoshop are awful. Aliasing is much worse than from cr2hdr converted at half res. HDR merge doesn't work as it should too.
Hi a1ex, your new cr2hdr works a treat now with actual exiftool are working! (saves me doing them 1 by 1 :D)
I have to say many thanks to you and the members who helped in developing the cr2hdr ;)
Cheers
Robin
basic question here; aren't you anchored to use it at iso 100 - what what would you do trying to take pics of friends in a dimly lit pub? iso 100 forces a real slow shutter speed, yes? ie. iso 100/1600...is my thinking wrong?
Quote from: pointbob1 on September 23, 2013, 07:44:21 PM
basic question here; aren't you anchored to use it at iso 100 - what what would you do trying to take pics of friends in a dimly lit pub? iso 100 forces a real slow shutter speed, yes? ie. iso 100/1600...is my thinking wrong?
You can manually set the iso on your camera and then the second iso in the module, either as a relative ev or absolute iso value. Just note that combining high iso values will gain only a small bit of dynamic range, see the dr curve for your camera here: http://www.sensorgen.info/
...so no developers interested in modification of cr2hdr to have manual control instead of automatic detection of fields order? Maybe easier than implementing command line would be 4 versions of cr2hdr, since there is 4 combination of fields order?
Quote from: jOt on September 24, 2013, 01:38:28 PM
...so no developers interested in modification of cr2hdr to have manual control instead of automatic detection of fields order?
Give them some time, will you? Unless someone says it *won't* be implemented, given the amount of features being worked on in parallel and this not being a major bug it's not on top of the list... look again after a week or so?
Quote from: Marsu42 on September 24, 2013, 01:40:48 PM
Give them some time, will you? Unless someone says it *won't* be implemented, given the amount of features being worked on in parallel and this not being a major bug it's not on top of the list... look again after a week or so?
I know, I keep asking cause tomorrow I got an exam in filmschool and one shot in my exam movie is missed...
Try this: int is_bright[4] = {1,1,0,0}, then {0,1,1,0}, then {0,0,1,1} and {1,0,0,1}. These are the 4 combinations.
Right now I'm a bit busy with some other stuff, sorry.
Thanks Alex. How exactly I can use this code? Cr2hdr closes immediately after double-click. Where should I write the code and filename? Sorry for asking banal questions.
Quote from: jOt on September 24, 2013, 03:04:28 PM
Thanks Alex. How exactly I can use this code? Cr2hdr closes immediately after double-click. Where should I write the code and filename? Sorry for asking banal questions.
You have to have a deep understanding of how the cr2hdr is created. In the dual-iso module there is the C code named cr2hdr.c which is where all of the details of this program lays. If you have the knowledge then go into this code and make the necessary changes. But of course you first have to set compiler.
Of course if you are not a hardcore ML programmer this will not be possible.
Quote from: a1ex on September 24, 2013, 02:27:29 PM
Try this: int is_bright[4] = {1,1,0,0}, then {0,1,1,0}, then {0,0,1,1} and {1,0,0,1}. These are the 4 combinations.
I've compiled 4 different version of cr2hdr with the combinations quoted.
https://www.dropbox.com/sh/qyh4agzisg3g0c5/16RZU1Z2Bg
Quote from: Audionut on September 24, 2013, 04:16:53 PM
I've compiled 4 different version of cr2hdr with the combinations quoted.
https://www.dropbox.com/sh/qyh4agzisg3g0c5/16RZU1Z2Bg
Many thanks, Audionut. Unfortunately when I drop dng on cr2hdr I got a message that version of this file is not compatible with my windows version. I got win7 64x. What version I shoud have?
Sorry, I had to setup a new build environment and never bothered to check the compiles before uploading.
I'll try and get it fixed tomorrow.
Quote from: jOt on September 24, 2013, 03:04:28 PM
Thanks Alex. How exactly I can use this code? Cr2hdr closes immediately after double-click. Where should I write the code and filename? Sorry for asking banal questions.
Just drag your dual ISO CR2 file(s) onto the CR2HDR executable file. Alternatively you could write a batch file to point CR2HDR to the CR2 files but dragging is the easier method. Double clicking doesn't work because CR2HDR is not an interactive program but a console program that relies on input arguments.
Hi, a1ex
thank you for DualISO on my 50D, it is awesome.
It however started to work for me as expected only since the last build of cr2hdr.exe
(till then i had only green whites + pink shadows as dng, do not know why...)
But, is there any chance to create cr2hdr with multicore CPUs support, please?
This single core version is really time consuming, which is time limiting for testing dualISO pictures...
It is a waste of resources we have. :-)
Anyway, thanks again.
You can already use a batch script that does this. There's one posted somewhere in this thread.
Also found something here: http://stackoverflow.com/questions/672719/parallel-execution-of-shell-processes
hi,
today i shot a dual iso raw video. after converting the files with rawmagic to dng files they are unprocessed with both isos - perhaps rawmagic can implement those file types in the future.. ?
i have now noticed that when i use the mac gui 1.2 cr2hdr i get an error message: the file type is not supported..
here is the link to the dng file and a screenshot of the error message:
https://www.dropbox.com/sh/wzh6q2994ve7i6d/kKL5Whmuln
another question:
is it possible to implement an option to mark dual iso raw videos with a prefix DUAL like dualiso photos...
that would be really handy because i have shot so many small video files, some with dualiso, others not at the same day..
thank you. sw
Quote from: a1ex on September 25, 2013, 11:23:15 PM
You can already use a batch script that does this. There's one posted somewhere in this thread.
Also found something here: http://stackoverflow.com/questions/672719/parallel-execution-of-shell-processes
Thanks, a1ex
but I have no idea how this could be helpful for me :-)
As far as I understand that script, it is meant for different king of task.
Drag&drop dualiso cr2s on cr2hdr.exe is nice and simple, I would not change it.
Plus, parallel processing of multiple cr2 files would perhaps overload HDD...
so
multicore processing of one cr2 at a time still sounds better. At least for me. :-)
Have a nice day.
As far as I see, the code has steps that rely on previous steps. This can't be multi-threaded.
The HDD load is small. It's only a 40mb (or so) read/save at any one time. And the processing time far outweighs the file loading/saving.
Quote from: Audionut on September 26, 2013, 09:56:58 AM
It's only a 40mb (or so) read/save at any one time.
By the way - you guys most likely know that the cr2hdr output isn't compressed, or not optimally so? Re-compressing the resulting dng in Lightroom (ACR) cuts the size about in half, of course as lossless conversion.
The successful vbs script by engardeknave has been re-namend moved to Scripts Repository:
Script for Automatic Multiple instances of cr2hdr
http://www.magiclantern.fm/forum/index.php?topic=8520.0
I'm getting some Dual-ISO files, that do not have "DUAL" prefix and some files with this prefix that are not Dual-ISO. I think this happens especially when I shoot series of photos. AETTR with linked Dual-ISO, Canon 50D, latest nightly (it happened in previous builds, too).
Quote from: jOt on September 24, 2013, 03:04:28 PM
Thanks Alex. How exactly I can use this code? Cr2hdr closes immediately after double-click. Where should I write the code and filename? Sorry for asking banal questions.
I think I made the correct changes from what Alex said, here are the 4 versions plus the normal one. Like always make sure the file is in the same folder as the EXE file when you drag and drop to convert.
https://bitbucket.org/mk11174/magic-lantern/downloads/cr2hdr_combo.zip
All the raw files recorded with the last build that I put in this software say:
/Volumes/RAID/GRILL/100EOS5D/M26-1348.RAW File not Supported
THE END
I tried with RAW2DNG and I get Pro Res video with lot of horizontal lines.
Please, help!!!
You need to convert the raw file to DNGs. Then process the DNGs with cr2hdr.
This is the message for the files:
1349_C0000/M26-1349_C0000_00587.dng File not Supported
/Volumes/RAID/LE GRILL CHILLA EVENTOS/converted/M26-1349_C0000/M26-1349_C0000_00588.dng File not Supported
/Volumes/RAID/LE GRILL CHILLA EVENTOS/converted/M26-1349_C0000/M26-1349_C0000_00589.dng File not Supported
/Volumes/RAID/LE GRILL CHILLA EVENTOS/converted/M26-1349_C0000/M26-1349_C0000_00590.dng File not Supported
THE END
You're using cr2hdr to process the DNGs right?
1- RAW file to Raw2DNG
2- DNGs to ACR
3- DNGs files to CR2HDR (Mac version)
result: .dng File not Supported
THE END
Ditch step 2!
Upload a DNG (straight from raw2dng) for someone to look at.
Same Result
.DNG File not Supported
THE END
https://www.dropbox.com/s/zp1qb5rwrjri4ts/M26-1349_C0000_00160.dng
Works for me (with the Linux/Windows versions).
If you are using the MAC GUI, try renaming it to .CR2 (and tell a.d. to allow DNGs too).
Input file : M26-1349_C0000_00002.CR2
sh: dcraw: command not found sh: exiftool: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile : M26-1349_C0000_00002.txt
THE END
TXT file say:
Input file : M26-1349_C0000_00002.CR2
Canon EOS 5D Mark III detected
Full size : 0 x 0
Active area : 0 x 0
Quote from: budafilms on September 27, 2013, 10:12:40 AM
Input file : M26-1349_C0000_00002.CR2
sh: dcraw: command not found sh: exiftool: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile : M26-1349_C0000_00002.txt
THE END
You also need to download OSX_cr2hdr.zip. It's an installer for dcraw & exiftool.
@a.d
Love your mac app. Any possibility to to be able to convert dng;s created from extracted dual iso raw like with Alex windows/linux build?
Thanks!
//D
Where I can download that software?
It's not explaining how this work in the topic? Can you explain the entire DUAL ISO process for Mac users please?
Thanks.
Opening Post. ie post number 1 on this thread.
No RENATO
If you use the instruction in the beginning of the post only get this:
cr2hdr // Beta 0.4
/M26-1349_C0000_00160.dng File not Supported
THE END
I'm trying to know in differents mac why this is not working.
A1ex Recommend to rename DNG files as CR2 and add to the software the possibility to recognize DNG files directly.
This is not explained.
A.D. recommends use the file above and works only with CR2.
So, first:
- we have to open RAW file with RAW2DNG
- rename all the files as .CR2
- And after that put on CR2HDR 0.4 - not early
Except a easy way that I can't find, this is the process.
Can you update this?
Quote from: budafilms on September 27, 2013, 08:11:13 PM
No RENATO
I thought you were looking for Mac. In the first post there is this link to Mac:
- Mac GUI for cr2hdr by a.d. If you click on this on the first post, it takes you to the place where you can download the Mac version here:
https://bitbucket.org/a_d_/magic-lantern/downloads
These files are generated by a.d. who has provided some support for Mac users (I am a PC user)
The first one on the list is OSX_cr2hdr_app.zip
Try that one.
I hope someone else can help you with the MAC stuff.
Thanks RENATO
We are trying to create sth in AUTOMATOR - Mac - To rename DNG files easly.
A.D. explain me why the software only recognize CR2 files.
For future mac users of DUAL ISO Post Processing it´s important they know to rename the files and use all this app for the moment:
https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
thks
Quote from: budafilms on September 27, 2013, 09:01:53 PM
https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
And from the post of a.d. you also have to dowload:
Quote from: a.d. on September 27, 2013, 01:50:28 PM
You also need to download OSX_cr2hdr.zip. It's an installer for dcraw & exiftool.
So you have to download both pieces of software:
https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
and
https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr.zip
For Bulk rename i use www.bulkrenameutility.co.uk
but dont know if there is a Mac version.
Here is a link to google find of bulk rename for mac:
https://www.google.com.ec/?gws_rd=cr&ei=QLlFUt2BCIaE9QSqxYDYCw#psj=1&q=bulk+rename+for+mac
Dang, it works on my mac. I renamed to cr2 when it has to be capitals CR2. WHat was I thinking :) A.D your app rocks!
I rename my files with this tool http://namechanger.en.softonic.com/mac
Thanks everyone
//D
Good Danne, works too!
CR2 it´s case sensitive, only works in UpperCase!!!
Quote from: budafilms on September 28, 2013, 04:58:32 AM
Good Danne, works too!
CR2 it´s case sensitive, only works in UpperCase!!!
Also, once you've renamed them to .CR2, the newest CR2HDR allows you to drag and drop the entire folder to the icon. Pretty damn nifty and efficient. Now if CR2HDR saved the converted .dng's into a sub-folder, it would make for an even better workflow.
Soon appears that we can drop DNG into the soft directly ;)
Quote from: mk11174 on September 27, 2013, 12:10:02 AM
I think I made the correct changes from what Alex said, here are the 4 versions plus the normal one. Like always make sure the file is in the same folder as the EXE file when you drag and drop to convert.
https://bitbucket.org/mk11174/magic-lantern/downloads/cr2hdr_combo.zip
mk11174, thanks a lot. I could convert successfully half of missing frames, but still on the rest there is some interlacing, I don't know why.
Before conversion:
http://www.mediafire.com/view/on7joso4ngljnsc/000189.dng
After:
http://www.mediafire.com/view/gqam4hatatuq3jd/000189_0110.DNG
(It's a still from the shot of throwing molotov cocktail)
canon 60 not supported
this train?
A.D. has working in CR2HDR and now support DNG!
https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
Thanks A.D!
Alex,s improvement skills on the converter is amazing. Time to try out some new dual iso filming
Hi !
First try with dual iso today.
Canon 550D, today's ML build, dual_iso-Module enabled.
Internal ISO:100
Dual ISO 100/800 or 100/1600 enabled
Dropping the CR2 files on the EXE (Windows, 20.09.13) it says:
D:\Software\Canon\cr22hd>cr2hdr.exe IMG_3535.CR2
Input file : IMG_3535.CR2
Canon EOS 550D detected
Full size : 5344 x 3516
Active area : 5202 x 3465
Doesn't look like interlaced ISO
I don't see any interlaced lines at 100% for reference.
Any ideas why no dual ISO files are created ?
Thanks ! Harry
Quote from: Aquillum on September 29, 2013, 04:11:24 PM
Canon 550D, today's ML build, dual_iso-Module enabled.
Internal ISO:100
Dual ISO 100/800 or 100/1600 enabled
Input file : IMG_3535.CR2
Canon EOS 550D detected
Full size : 5344 x 3516
Active area : 5202 x 3465
Doesn't look like interlaced ISO
Maybe the nightly build does not yet support the DUAL ISO feature for this camera.
I think it has been enabled by another developer, check this thread:
550D raw video recording port official thread
http://www.magiclantern.fm/forum/index.php?topic=5582.0
Quote from: Legushka on September 28, 2013, 02:16:51 PM
canon 60 not supported
Look in this thread:
60D RAW video - it's working !!!
http://www.magiclantern.fm/forum/index.php?topic=5653.0
Quote from: RenatoPhoto on September 29, 2013, 04:55:24 PM
Maybe the nightly build does not yet support the DUAL ISO feature for this camera.
In this case, he wouldn't be able to see dual ISO in the menu.
Quote from: a1ex on September 29, 2013, 07:09:02 PM
In this case, he wouldn't be able to see dual ISO in the menu.
@ Aquillum
In that case then you need to learn how to use it:
(Auto) ETTR (Exposure to the Right): -- History & Beginners Guide
http://www.magiclantern.fm/forum/index.php?topic=5693.0
AETTR + DUAL ISO: The Ultimate Automated Perfect Image Exposure-Beginners'Guide
http://www.magiclantern.fm/forum/index.php?topic=8322.0
I used ETTR since May, even with a timelapse, thats not the problem.
The problem is with the latestbuild:
I am able to enable the modules, they are loaded after restart and I see ETTR and Dual Iso in Menu.
I enable Dual-Iso, set to 100/800, take the raw but the CR2 is NOT Dual Iso.
I also tried the ML_550D_Dual_ISO_Aug_25_2013_Update.zip build posted in that forum after cleaning ML folder completely (not to mix modules) but it also doesn't work.
Does anyone with a 550D shot a DualIso successfully ? I can't.
Did you know that you have to reboot the camera for the modules to work? It is a new thing!!
Quote from: budafilms on September 29, 2013, 08:59:21 AM
A.D. has working in CR2HDR and now support DNG!
https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
What about the actual dual iso raw files? That would be epic.
As I wrote before I restarted, the modules loaded OK.
I use the standard settings in Dual Iso Menu (100/800).
I take the pic in Av or M, with or without LV, but the resulting CR2 doesnt have scanlines and cr22hdr also doesn't recognize it as an DualIso file.
Just tried it on 550D, works fine.
Any warnings in menu?
No warnings.
As it works for you its definitly on my side - thanks for trying !
Can you post a video of the camera screen, while you are trying to use it?
It should show the dual ISO menu, Canon settings (main photo screen, with ISO, shutter and so on), then take a picture, go to playback mode and zoom to 100%. Maybe this gives some hints about what's going on.
Now it works.
Things done (from my memory): (DualIso was enabled)
- switched Canon Iso from 100 to 200
- switched to DUAL Prefix
- enabled ETTR - Switched from AutoSnap to SET
- switched LV - shot ETTR and realized the Image Scanlines in PLAY
- switched ETTR OFF
- DUAL ISO shot still works
- CR2 decoded fine
THX for your support (even if its just "Here it works", that was enough for me)
@a.d.
nice now the conversion of dual iso dng files work.
i have one question a.d.:
when i see my magicraw converted dng files (from a raw file) in the folder, i can preview that file with the space bar.. (mac preview)
after the conversion of the dual iso dng with your tool (i can see that it renames to CR2 and then again to dng) there is no preview in the finder anymore..?
is this because of the conversion? why do i lose the preview?
thx and keep on developing great tools. sw
I need to rename, 'cause after cr2hdr save_dng run it called copy_tags_from_source, to copy EXIF from original to converted file. That means I need to save the original file to get the EXIF Data. Alternatively I could inject the Camera Model through chdk-dng, however it hardcoded. (only one Camera Model)
You are right, the preview is lost through conversion.
I got dng files from raw video but I can not use cr2hdr!
please, help me!
Quote from: lucabutera on September 30, 2013, 10:54:25 PM
I got dng files from raw video but I can not use cr2hdr!
please, help me!
Help us to help you by extending your bug report to include useful information.
Bug: cr2hdr on 6d adds 2 black borders to shots, here's a random dng with this problem (no, this shot won't win any prizes :-p): https://bitbucket.org/Marsu42/ml-mod6d/downloads/DUAL2689.CR2
Btw see also these feature requests:
http://www.magiclantern.fm/forum/index.php?topic=8579.0
http://www.magiclantern.fm/forum/index.php?topic=8580.0
Any idea why my DNGs out of cr2hdr look like this for some shots?
(http://i44.tinypic.com/2cs7ybs.png)
QuoteBug: cr2hdr on 6d adds 2 black borders to shots, here's a random dng with this problem (no, this shot won't win any prizes :-p): https://bitbucket.org/Marsu42/ml-mod6d/downloads/DUAL2689.CR2
Upgrade your dcraw. No bars here.
But I do get some conversion artifacts on this shot:
(http://i.imgur.com/2ZpeLcc.jpg)
Have a link to an updated dcraw that works on Windows?
Quote from: a1ex on October 02, 2013, 08:02:45 AM
Have a link to an updated dcraw that works on Windows?
Here ye go, fresh compile just now: https://bitbucket.org/Marsu42/ml-mod6d/downloads/dcraw_9_19-m42.zip
Don't be too shy to install the mingw packages on your linux though, then just compile with "i686-pc-cygwin-gcc.exe -o dcraw -O4 dcraw.c -lm -DNODEPS" and get an .exe file.
Thanks, will update the zip in the first page when I'll be able to access the ftp (it seems broken now).
Quote from: 1% on October 02, 2013, 06:56:34 AM
Upgrade your dcraw. No bars here.
Yes, that did it - thanks! I forgot that cr2hdr is not the only component to keep up to date.
Btw, this again reminds me of how important it is too keep the old cr2 around (I recently suggested an option to embed them in the dng for safekeeping & easy updating with a newer cr2hdr). Is cr2hdr considered "production-ready" now, or are there outstanding important bugfixes or enhancements?
Any idea on the previous shot I posted? The log states something about bad black delta. Only on a few takes from a short this weekend that utilized a higher ISO spread.
Quote from: Marsu42 on October 02, 2013, 12:20:55 PM
Is cr2hdr considered "production-ready" now, or are there outstanding important bugfixes or enhancements?
I've processed roughly 2000 pics with it, and for testing I have a directory with 50 difficult shots (most of which were taken from this thread).
Known issue: the night fire shots from jOt. I've outlined a robust formula for autodetection, which relies on comparing 4 histograms, only need some time to implement it and run the tests.
So, now it's up to you to try it and upload the CR2 files that fail to convert or have major issues (hint for @artiswar).
As soon as I get home, I'll upload a CR2.
Here's a problematic CR2.
http://www.megafileupload.com/en/file/456879/M29-0538-C0000-00773-CR2.html
QuoteI recently suggested an option to embed them in the dng for safekeeping & easy updating with a newer cr2hdr
I keep them around in case there is a better CR2hdr released and I might want to reprocess the shots. The DNGs are already 2x the size of the cr2 files.
Quote from: 1% on October 03, 2013, 12:23:05 AM
I keep them around in case there is a better CR2hdr released and I might want to reprocess the shots.
Embedding the cr2 in the dng would allow a seamless in-place update with a newer cr2hdr, keeping the tags and file location: http://www.magiclantern.fm/forum/index.php?topic=8336.0
Quote from: 1% on October 03, 2013, 12:23:05 AM
The DNGs are already 2x the size of the cr2 files.
No, they aren't, it's just that ml's dcraw process doesn't output a proper dng - if you (losslessly of course) recompress the dual_iso dng with ACR like in Lightroom, the size is cut by half.
@1%
Dying for a fix here. Very integral shots that need to be processed. Or if there's a way to deinterlace without processing ISOs that would be fine too. I just need the shots. Thanks so much.
Well I was stumbling around looking for a newer build of cr2hdr.exe I tried all the 9/27 I found... your "cr2" looks fucked up... lines in the shadows.
http://www.filedropper.com/oldx-cr2hdr
My old self compiled from 9/5 did this:
(http://i.imgur.com/sQSM5ap.jpg)
@1%
You're a potential lifesaver. Anyway to run this in the Mac environment? Or fix my CR2?
VM or wine or (parallels?) it was made on win7 x64 but should be ok on any windows environment..
Quote from: 1% on October 03, 2013, 03:57:10 AM
VM or wine or (parallels?) it was made on win7 x64 but should be ok on any windows environment..
Any reason why I'd get a console window in Windows when trying to convert? Sorry for all the questions.
Its a command line only app
It doesn't stay open and does no conversion.
Open a command prompt manually and process the CR2. This way you will see the error message before the prompt auto closes.
Quote from: 1% on October 03, 2013, 03:42:38 AM
Well I was stumbling around looking for a newer build of cr2hdr.exe I tried all the 9/27 I found... your "cr2" looks fucked up... lines in the shadows.
...
I notice the same thing ... It broke up between 13 and 20 September
Solved.
If you notice these things, post a CR2 that shows the problem, I can't read your minds ;)
Where can i change in MLn dual Iso??? I cant see nothing?
Quote from: a1ex on October 02, 2013, 04:02:22 PM
I've processed roughly 2000 pics with it, and for testing I have a directory with 50 difficult shots (most of which were taken from this thread).
Are there any specifics we should watch out for in our dual_iso shots? My shots look just fine, but then again I don't really know what type of problem(s) to look for @100% crop.
Question: The help says 100/1600 is too high, but it looks great on the 6D. is the +4ev spacing warning there because the higher iso has more noise (i.e. worse on crop than ff), or because a large spacing might confuse cr2hdr for reasons other than iso noise?
Edit: Also you might want to add some version tag to cr2hdr, or with so many binaries floating around it might get confusing sooner or later with what version a cr2 was converted?
ISO 100/800 will most likely look the same in shadows, with less aliasing in highlights. Plus, in video mode, I don't have black correction info, and 100/1600 or higher will do a lot of nasty things to black level.
The latest problematic CR2 had exactly this issue, so... had artiswar followed the indications from the menu, he wouldn't have had this issue.
Look at raw zebras; if you get weak zebras over important detail, it's likely to have aliasing there.
Quote from: a1ex on October 03, 2013, 08:59:54 PMlus, in video mode, I don't have black correction info, and 100/1600 or higher will do a lot of nasty things to black level.
To clarify again: this is only a problem for video, photo mode is more likely to work ok with 100/1600, correct?
Quote from: a1ex on October 03, 2013, 08:59:54 PMlus, in video mode, I Look at raw zebras; if you get weak zebras over important detail, it's likely to have aliasing there.
Thanks!
Yes, the warning in photo mode should appear at 3200 on most cameras. On 5D2/500D/50D it appears at 1600, because this is the max analog ISO there.
I've downloaded cr2hdr.zip from OP today (I don't know when it was updated) and found out dcraw.exe included there requires cygwin1.dll installed, which might be an issue sometimes. Previously it shipped with a standalone version (200k bigger). Reverting back to previously included dcraw.exe works fine I think (I hope no new dcraw.exe features are required?)
Depends on the camera.. .Ie 6D/5DIII/M old dcraw might cause issues.
I just want to give a huge thanks to everyone working on Dual ISO. I decided to test it out in the video function and here is a quick sampling of it. There are definite circumstances where it really shines and looks quite good in video form. I think it looks pretty good in the first 3 shots. The last shot was really just to see how much dynamic range I could get. All were shot 100/3200, so that maybe the problem with the chroma artifacts in the last shot. I've learned that if a shadow is right next to a highlight, there can be issues. I am really excited to continue to see which scenarios it looks the best in.
Next time, try following the advice from the menu (100/800). The noise improvement from 800 to 3200 is less than 1 stop, but the difference in aliased areas is 2 stops. The improvement from 1600 to 3200 is close to 0 (but for that you pay 1 full stop of aliased data).
@alex
do you also prefer 100/800 when shooting a low light scenario?
i have read somewhere that 1. we should ettr the scene and then 2. apply the second iso for raising the shadows..
so this could also be 800/1600.
am i right?
thx sw
Quote from: swinxx on October 04, 2013, 07:54:33 AM
so this could also be 800/1600. am i right?
No, you're not, because the dynamic range gained from 100/800 (+3ev: 100, 200, 400, 800) is much larger than 800/1600 (+1ev) - ettr even shows this in the menu.
800/1600 is fine, but it's only 0.5 stops of improvement. 800/3200 is 0.75 on 5D3. At this point you may as well just denoising the shadows a tiny bit (without getting any aliasing).
Here you can see the difference between ISO 1600/3200/6400 in shadows: http://www.guillermoluijk.com/article/isos5dmkii/index.htm
(hint: I can't see any)
But if that low light scenario also has some bright areas (think at street lighting), I'd go for 100/800, or 100/1600 if it's really dark, or try what ETTR suggests.
Here's an example:
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_7111-small.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_7111.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_7111-1600-small.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_7111-1600.jpg)
First is ISO 100/1600; second is how it would have looked at ISO 1600 (it's not a real 1600 shot, but reconstructed from the dual exposure). It doesn't look bad, and the clipped areas are pretty small, but I prefer the first one.
And here's a night shot that did not require dual ISO according to ETTR, even though I was tempted to use it. Why? Because at 1/10 ISO 1600 there was no important detail overexposed (only some tiny specular highlights), and I don't think I could have used a slower shutter speed handheld. But if I would have allowed ETTR use a slower shutter, or asked it not to ignore any highlights, I think it would have chosen dual ISO.
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_5808-small.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/test/IMG_5808.jpg)
Alex Where are you from?Where are these pics from?
thx for your posting alex. i also cant see any difference on the iso shots 1600 3200 6400 from your link. everything makes sense but i have to get used to it.. when you prefere 100/800 for a night scene with some highlight areas it means that you make no ettr without dual iso first, and then apply the second (lower iso) to get rid of the noise.. right?
and btw.. wouldn´t it be great to implement a "lock the 180degree shutter rule" for the raw video as an option for ettr, so that it only adapts the right iso for the scene. ?
when shooting video i always want my shutter to stay at e.g. 1/48 for 24fps..
thx sw
Simply choose that shutter speed in ETTR menu.
I try the dual iso with a 60D but my computer work with Ubuntu 12.04 and the compilation of cr2hdr dont work (problem with Makefile).
Someone have the .bin of this software ? And other to use this great module with Linux ?
I have download cr2hdr-static.linux.x86.2013-10-03-47b5360e8f49.7z found in the web but after unzip, I dont know what I can do with the file : cr2hdr-static.linux.x86.2013-10-03-47b5360e8f49.
Thank's for your lights...
Was out doing real estate photography and I decided to film two short sequences to see how dual iso holds up with today,s conversion tool. Also filmed a raw clip for comparison. Even though aliasing still exists and some jagged edges, It,s still eyeblowing to be able to get an even higher dynamic range than raw. I,ll try to find some time to get a moving picture.
Frame from raw 1920x1080p
(http://s11.postimg.org/v87y4aqw3/000009.jpg)
Frame from dual iso raw 100/1600 (ISO difference : 3.98 EV (1577)
(http://s11.postimg.org/kjzlrprj7/000000.jpg)
Still image (just for fun), three pictures, enfused to one.
(http://s24.postimg.org/q7ob3e90l/IMG_8484_1.jpg)
Quote from: Danne on October 04, 2013, 09:29:41 PMFrame from dual iso raw 100/1600(or 800) Can,t remember
Another +1 for adding a xmp with the iso settings used :-o ... for bugtracking or recreating a scene it's important to remember how it was done, and this information simply isn't available unless you have a pencil and a piece of paper ready all the time.
Sure, just find a command for exiftool that would add the ISO gap in a way recognized by Adobe software.
The second ISO is a little hard to guess with current metadata (some cases are ambiguous). Example: main ISO 400 and a gap of 2 stops - is it 400/100 or 400/1600?
The noise levels may be a good hint for solving the ambiguity (but right now we only have them in photo mode).
Actually @Marsu, my test was to see how well the latest cr2hdr holds up. It does a pretty decent job don,t you think? Also want to get some more dual iso filming going since I can,t find any good examples out there yet. Thanks for watching.
Updated the correct dual iso information, ISO difference : 3.98 EV (1577) 100/1600 that is :)
Also added a still image (third image) 3 exposures enfused to one. Amazingly the dual iso has a better range giving me blue skies in the windows
Quote from: Danne on October 04, 2013, 10:16:56 PM
Updated the correct dual iso information, ISO difference : 3.98 EV (1577) 100/1600 that is :)
Indeed, dual_iso is giving amazingly good results, and I have yet to see any 100/1600 shots from the 6d to show any noticeable problems - but I'll keep looking for them :-)
Quote from: a1ex on October 04, 2013, 10:16:25 PM
The second ISO is a little hard to guess with current metadata (some cases are ambiguous). Example: main ISO 400 and a gap of 2 stops - is it 400/100 or 400/1600?
* I remember you writing that, that's why I suggested writing a xmp in-camera when the accurate information is still available just like deflicker ... but if you think you can reconstruct both iso values from the cr2 I'll provide an exiftool cmd line.
* One "problem" cr2 from me: It's a 30sec long time test exposure with a 10x nd filter and shows sprinkles of various colors all over the place. Is this due to me using iso100/1600, or is this a cr2hdr problem which could be fixed with an anti-sprinkle algorithm? Unfortunately I didn't take a non-dual_iso shot to compare with this one. Here's the cr2 and a converted jpeg so you can have a quick look:
https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2#005512.jpg
https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2#005512.cr2
Does anyone know if medium or small raw files are supported with dual iso? I can't seem to find any info about it.
I tried a timelapse last night with small raw set on my 6D. I get an error when converting using cr2hdr (latest from https://bitbucket.org/a_d_/magic-lantern/downloads):
Input FIle : DUAL6977.cr2
Canon Eos 6D detected
Full Size : 2736 x 1824
Active Area : 2736 x 1824
Error: dcraw output is not a valid PGM file
Thanks for any help and for all the work guys!
If the menu allowed you to enable dual ISO with sRAW, this is a bug. Post some screenshots.
The bug seems to happen if Q == sraw/mraw and you take a still in video mode. Or I guess on 6D if you left raw_rec on in photo mode (6D DOES need this if you want to monitor raw via wifi)
Just pushed a fix (but didn't try it yet).
Sorry if this has been brought up, but I can't drag and drop because the cygwin1.dll is missing from my computer. Any thoughts? This is being shot RAW (.CR2) with a 6d.
Quote from: painya on October 05, 2013, 08:35:13 PM
Sorry if this has been brought up, but I can't drag and drop because the cygwin1.dll is missing from my computer.
Your dcraw.exe, cr2hdr.exe or whatever.exe has been compiled with cygwin (unnecessarily, btw) and not mingw ... so now it needs this dll in the same directory or in the search path. Get the dll from http://www.cygwin.com/
Or use non-cygwin compiles, for example get a native windows dcraw here: http://www.fastpictureviewer.com/downloads/
Quote from: Marsu42 on October 05, 2013, 09:52:11 PM
Your dcraw.exe, cr2hdr.exe or whatever.exe has been compiled with cygwin (unnecessarily, btw) and not mingw ... so now it needs this dll in the same directory or in the search path. Get the dll from http://www.cygwin.com/
Or use non-cygwin compiles, for example get a native windows dcraw here: http://www.fastpictureviewer.com/downloads/
The fast picture viewer will work for this HDR method? (with the add on of course)
Thanks.
Quote from: painya on October 05, 2013, 10:28:15 PM
The fast picture viewer will work for this HDR method? (with the add on of course)
No, their site just happens to provide current x86 and x64 compiles of dcraw.
I just started a thread on a seemingly too easy workflow. Mind checking it out to make sure it works for you as well?
Quote from: Marsu42 on October 05, 2013, 09:52:11 PM
Your dcraw.exe, cr2hdr.exe or whatever.exe has been compiled with cygwin (unnecessarily, btw) and not mingw ... so now it needs this dll in the same directory or in the search path. Get the dll from http://www.cygwin.com/
Or use non-cygwin compiles, for example get a native windows dcraw here: http://www.fastpictureviewer.com/downloads/
I install cygwin, and it still says I am missing the .dll. Any thoughts? I also installed the fast picture viewer. (I checked and found the .dll in the bin for cygwin)
just update cr2hdr
Quote from: Marsu42 on October 05, 2013, 02:03:05 AM
https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2#005512.jpg
https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2#005512.cr2
Broken links?
Quote from: a1ex on October 06, 2013, 02:03:00 PM
Broken links?
this too?
QuoteFor Mac download:
Mac GUI for cr2hdr: https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
Quote from: a1ex on October 06, 2013, 02:03:00 PM
Broken links?
This really seems to be broken, maybe a bitbucket error/downtime - it should be there on my bitbucket download page https://bitbucket.org/Marsu42/ml-mod6d/downloads
If it continues not to work I'll re-upload them, maybe it's also the # in the name that confuses the system.
Doh, it was really the # in the filename.. here ye go:
https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2X005512.jpg
https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2X005512.cr2
After taking quite a big series of dual iso photos on my Canon 50D I've noticed that each and every one DNG created has white balance values off by about -500K, +30 tint (in ACR) compared to non-dual-iso photos taken in the same circumstances. As this value is quite constant, is it cr2hdr error or some ACR bug in reading "As shot" value from DNG?
Additionally, here's one conversion that gives strange results:
CR2: https://mega.co.nz/#!Yc9VFC6T!cidlsw8J2YOGGTVnsmTzlVjuPK8v8MtM4oAZTK_Sgrs
DNG: https://mega.co.nz/#!oQs0kYob!NLGjazsBAfPAuhBT4KuUN0vwUw8kbPlp2kIK3oqgS8Q
Is exiftool running?
Quote from: a1ex on October 06, 2013, 11:48:31 PM
Is exiftool running?
I think so – cr2hdr gives no error while running, exiftool.exe is in the same dir and runs fine when called. It's in the path too.
Here's an example of what I'm talking about:
https://mega.co.nz/#!gYcwSYLJ!cQuj1zx4Q2k1IphjgMZIOBHjOoJpwEC4HUHsTPDM2jI
In the zip you get non-dual-iso CR2, dual-iso CR2 and generated DNG. I need to change WB of the DNG by about +500 K, -30 tint to have it similar to non-dual-iso CR2.
I actually tested this today.
(http://i.imgur.com/J02ZXjC.png)
(http://i.imgur.com/Z4sWuT1.png)
The .dng seemed to be about right judging from both the photo and histogram at +26 tint (+34 change) and no change in temperature.
(I see greener .dng)
Do you know if DNG extracted from raw2dng and Son of the batch are the same file?
Quote from: akumiszcza on October 06, 2013, 11:32:31 PM
As this value is quite constant, is it cr2hdr error or some ACR bug in reading "As shot" value from DNG?
I wouldn't say the tint/wb change is constant across the board, but maybe consistent enough to get some "standard" correction for different iso spacings and bv values so less needs to be done in post.
cr2hdr does not touch WB coefficients and it's not meant to do so.
Quote from: a1ex on October 07, 2013, 11:21:22 AM
cr2hdr does not touch WB coefficients and it's not meant to do so.
How come then that two photos — one normal, one dual iso — taken seconds apart in the same conditions have completely different WB? I don't have a single DNG file with proper WB set. Maybe it's ACR issue? See the example files: https://mega.co.nz/#!gYcwSYLJ!cQuj1zx4Q2k1IphjgMZIOBHjOoJpwEC4HUHsTPDM2jI
As I said earlier, the issue is in EXIF. So, just find the exiftool command that copies WB properly from the CR2 and I'll integrate it.
A little big usability breakthrough:
https://bitbucket.org/hudson/magic-lantern/pull-request/251/dual-iso-preview/
related: http://www.magiclantern.fm/forum/index.php?topic=8667
Hello, thanks for your job a1ex !
I've got some problem with my cr2hdr in Ubuntu :
Quote./cr2hdr _MG_5881.CR2
Input file : _MG_5881.CR2
Canon EOS 60D detected
Full size : 1336 x 3516
Active area : 1336 x 3516
Unknown option "-E".
Error: dcraw output is not a valid PGM file
I've download the last Hudson repository this day.
Canon 60D.
Your dcraw is too old.
But I do the compilation in this day and in the source code, the dcraw_bridge date is 2013_09_08 ! What's wrong ?
How can I take the last build ? Thank you much.
Type "dcraw" into google.
Quote from: a1ex on October 07, 2013, 03:59:29 PM
A little big usability breakthrough:
Looks to be working good.
Extracted JPG previews.
Highlights(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_UAL1501.jpg)
Lowlights(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_UAL1500.jpg)
Sharplights(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_UAL1502.jpg)
Right click | view image for full res.
(http://i.imgur.com/FiDYiCj.jpg)
Not possible without dual iso. This is a truly incredible development. Now if only it didn't have to go line by line and instead could be a dual bayer pattern with 6 instead of 3.
Also, any chance for an adaptive algorithm that measures scene contrast in live view then auto selects?
Just putting out random ideas I bet you are already cooking up something even better. can't wait.
Quote from: fotojohni on October 07, 2013, 06:01:18 PM
Also, any chance for an adaptive algorithm that measures scene contrast in live view then auto selects?
http://www.magiclantern.fm/forum/index.php?topic=8322.0
Bug report: I just had a look at the exifdata cr2hdr and acr generate - and ml screws up the lens information, this is part of a diff between ml (wrong) and acr (correct).
-Circle Of Confusion : 0.008 mm
-Field Of View : 30.0 deg
-Focal Length : 17.0 mm (35 mm equivalent: 67.1 mm)
-Hyperfocal Distance : 2.37 m
-Lens : 17.0 - 40.0 mm (35 mm equivalent: 67.1 - 158.0 mm)
+Circle Of Confusion : 0.030 mm
+Field Of View : 93.3 deg
+Focal Length : 17.0 mm (35 mm equivalent: 17.0 mm)
+Hyperfocal Distance : 0.60 m
+Lens : 17.0 - 40.0 mm (35 mm equivalent: 17.0 - 40.0 mm)
In the cr2 the data is also correct, but cr2hdr adds a wrong crop factor:
-Scale Factor To 35 mm Equivalent: 1.0
+Scale Factor To 35 mm Equivalent: 3.9
Bug ticket here: https://bitbucket.org/hudson/magic-lantern/issue/1697/cr2hdr-messes-up-the-lens-meta-information
Quote from: Audionut on October 07, 2013, 05:36:48 PM
Extracted JPG previews.
What about enabling HTP? It should make the preview brighter by one stop, right?
Quote from: Audionut on October 07, 2013, 06:08:39 PM
http://www.magiclantern.fm/forum/index.php?topic=8322.0
Okay this pretty much solves that. I wonder how you get the metering data so that there is no need to go into live view for every shot.
Quote from: a1ex on October 07, 2013, 06:22:48 PM
What about enabling HTP? It should make the preview brighter by one stop, right?
Yes it does.
Quote from: fotojohni on October 07, 2013, 06:28:46 PM
Okay this pretty much solves that. I wonder how you get the metering data so that there is no need to go into live view for every shot.
Raw data is available in image quick review.
Quote from: a1ex on October 07, 2013, 04:49:47 PM
Type "dcraw" into google.
OK, sorry for all the idiots questions ! ::)
cr2hdr is working ! That normal but my way to succed was very long.
Quote from: a1ex on October 07, 2013, 04:49:47 PM
Type "dcraw" into google.
or, preferably, into some search engine that doesn't track your every movement like ixquick or duckduckgo :-o
Quote from: fotojohni on October 07, 2013, 06:01:18 PM
Not possible without dual iso. This is a truly incredible development. Now if only it didn't have to go line by line and instead could be a dual bayer pattern with 6 instead of 3.
Also, any chance for an adaptive algorithm that measures scene contrast in live view then auto selects?
Just putting out random ideas I bet you are already cooking up something even better. can't wait.
fotojohni : what are the photo parameters ? Which camera ?
Quote from: Onoxyne on October 07, 2013, 09:33:10 PM
fotojohni : what are the photo parameters ? Which camera ?
Camera: 6D
ISO 100/1600
Lens: 50mm @ 1.4
VAF filter to reduce moire/aliasing
Shot in manual mode @ 1/45
For night shooting I switch recovery iso to 100 and shoot at 1600, then I can actually see what I am shooting in live view. I didn't do that here though.
For grading I just went into ACR and bumped the exposure up 3.0 stops, boosted shadows etc. I have a profile I like.
Quote from: a1ex on October 07, 2013, 02:00:47 PM
As I said earlier, the issue is in EXIF. So, just find the exiftool command that copies WB properly from the CR2 and I'll integrate it.
This is what I found, maybe something will be useful in the future:
I renamed the files mentioned before to norm.cr2, dual.cr2 and dual.dng.
Adobe Camera Raw shows the following temperature/tint values ("As shot"):
norm.cr2: 5200/+14
dual.cr2: 5850/+38
dual.dng: 4850/+46
As you can see dual.cr2 has it wrong, too, but better than dual.dng anyway.
I tried using exiftool commands, but none worked for me:
exiftool -tagsfromfile norm.cr2 -Canon:WhiteBalance -WB_RGGBLevelsAsShot -WB_RGGBLevelsAuto -WB_RGGBLevelsMeasured dual.dng
(mentioned somewhere by the author of exiftool)
exiftool -WhiteBalance#=5200 dual.dngThen I used hex editor to alter dng on my own.
After setting
crs:Temperature="5200" and
crs:Tint="+14" ACR shows the proper values of dual.dng file. This could be a good way of changing dng directly, but I don't know how to read temperature and tint value from a file. I've tried exiftool, exiv2, dcraw and all of them had other formats. And, as I found out just before posting it, it only works if ACR already exported xmp values into dng (standard cr2hdr output does not have it). But if it's possible to add some xmp to dngs, then crs:Temperature and crs:Tint works.
Second thing I've noticed:
dcraw -v -w norm.cr2 shows the following:
multipliers 2.102539 1 1.365234 1.
Now the command
dcraw -T -r 2.102539 1 1.365234 1 dual.dng creates dual.tiff with correct WB, so it might be possible to use this method somewhere in cr2hdr.c.
Sorry for stupid question.
I've dragged CR2 created with Ml and dual ISO, the window got appeared and seemingly the conversion happened, but I can't find a resulting DNG
Where it should be?
PS. 've wasted a plenty of time trying to use 32 bit dcraw in 64 bit environment. Probably it is necessary to mention that users with 64x Windows need proper dcraw
Quote from: 1drey on October 08, 2013, 10:09:04 AM
PS. 've wasted a plenty of time trying to use 32 bit dcraw in 64 bit environment. Probably it is necessary to mention that users with 64x Windows need proper dcraw
It isn't mentioned because it isn't true, x86 apps work just fine on x64 Windows. If you are really determined to try yourself - Get both versions of the current dcraw here: http://www.fastpictureviewer.com/downloads/
The dng file you seek ends up in the same folder as the cr2 after drag n' drop.
Hi to everyone, i download the new Windows:cr2hdr.exe and it doesn't seem to be working on my windows i get a dcraw.exe - System error that says <<cant start cygwin1.dll is missing>> how do i fix it or do i stay with the previous one that worked? Are there any major changes in those two?
Thanks!!
It is strange. I was getting " 'the application was unable to start correctly (0xc000007b)' " until 32 bit dcraw was replaced with x64 one.
And for some mysterious reason I don't see resulting DNG in the source folder. It was the first place where I expected it to appear.
Seems that *******.dng file was not created at all - I was searching it through the file search on my PCwith no avail.
Quote from: 1drey on October 08, 2013, 10:33:50 AM
And for some mysterious reason I don't see resulting DNG in the source folder. It was the first place where I expected it to appear.
Seems that *******.dng file was not created at all - I was searching it through the file search on my PCwith no avail.
Put the .cr2 dual iso file in the same folder with cr2hdr.exe, dcraw.exe, exiftool.exe. Run cmd.exe there and execute "cr2hdr <yourfilename>". See if any errors appear.
Quote from: pest01 on October 08, 2013, 10:27:54 AM
Hi to everyone, i download the new Windows:cr2hdr.exe and it doesn't seem to be working on my windows i get a dcraw.exe - System error that says <<cant start cygwin1.dll is missing>> how do i fix it or do i stay with the previous one that worked? Are there any major changes in those two?
Thanks!!
I was fighting with the same issue.
Re-installed Cigwin, but started to receive a message 'the application was unable to start correctly (0xc000007b)'
Replaced dcraw with x64 version
Quote from: pest01 on October 08, 2013, 10:27:54 AM
Hi to everyone, i download the new Windows:cr2hdr.exe and it doesn't seem to be working on my windows i get a dcraw.exe - System error that says <<cant start cygwin1.dll is missing>> how do i fix it or do i stay with the previous one that worked? Are there any major changes in those two?
Just replace dcraw.exe with a version not requiring cygwin1.dll or install cygwin in your system.
Do you have a link to a non-cygwin dcraw.exe that also works for 6D, so I can put that one in the zip?
Quote from: akumiszcza on October 08, 2013, 10:37:13 AM
Put the .cr2 dual iso file in the same folder with cr2hdr.exe, dcraw.exe, exiftool.exe. Run cmd.exe there and execute "cr2hdr <yourfilename>". See if any errors appear.
Got this
(http://pano.1drey.com/stills/cr2hdr.jpg)
Quote from: a1ex on October 08, 2013, 10:47:48 AM
Do you have a link to a non-cygwin dcraw.exe that also works for 6D, so I can put that one in the zip?
I don't have 6D to check. I use some 9.17 version 576064 byte on my 50D. Some new builds of 9.19 are here: https://sites.google.com/site/manuelllorens/ – but I didn't test those.
Quote from: 1drey on October 08, 2013, 10:48:43 AM
Got this
(http://pano.1drey.com/stills/cr2hdr.jpg)
Looks like you're in another folder...
Open the folder in explorer. Check if all your files are here. shift+rightclick and choose "open command prompt here" (it does not work on every windows version) and check again. Or run cmd.exe and cd <yourfolderpath> before running the command.
Quote from: a1ex on October 08, 2013, 10:47:48 AM
Do you have a link to a non-cygwin dcraw.exe that also works for 6D, so I can put that one in the zip?
As I'm posting to the extend of feeling ignored :-o ... http://www.fastpictureviewer.com/downloads/ ... you gotta scroll down a bit or search the page for "dcraw"
Btw a multi-threaded dcraw is here, but as it's not the newest version it unfortunately will only work for older cameras (i.e. not the 6D) - but for the rest it should be faster: http://www.heliconsoft.com/heliconsoft-products/dcraw/
Quote from: akumiszcza on October 08, 2013, 11:05:23 AM
Looks like you're in another folder...
Open the folder in explorer. Check if all your files are here. shift+rightclick and choose "open command prompt here" (it does not work on every windows version) and check again. Or run cmd.exe and cd <yourfolderpath> before running the command.
It kind of helped! :)
How I get this:
(http://pano.1drey.com/stills/ch2hdr2.jpg)
If I replace x64 dcraw with one from archive - I get an error message ('couldn't start correctly' etc.)
Updated the zip on main page with dcraw 9.19 32-bit from the link from Marsu42. Does it work?
@1drey: make sure you update ML for EOS M. Early versions produced files that can't be converted.
I'd like to share my quick and very dirty way to get dual iso files from CF card as fast as possible (...)
Topic moved here: http://www.magiclantern.fm/forum/index.php?topic=8738.msg81768#msg81768
Please post your comments there.
Quote from: akumiszcza on October 08, 2013, 11:45:14 AM
I'd like to share my quick and very dirty way to get dual iso files from CF card as fast as possible.
Normal steps are:
1. Get files from CF cards (it's a long process for me especially with USB 2.0 card reader and some slow 30MB/s cards).
2. Process them by cr2hdr (I use the script provided by engardeknave (http://www.magiclantern.fm/forum/index.php?topic=8520.0) without any prefix as I get quite many dual-iso files without prefix).
3. Rename them (I use exif date based system that moves filename.cr2 into something like a:\2013\10\07\20131007T121930filename.cr2).
4. Find dual-iso .cr2s in those created dirs and move them to dualiso subdir (I want to see only generated dngs while browsing, if something goes wrong I can alway go to dualiso subdir for original).
This takes a looong time.
Here is a zip file containing cr2hdr.exe, exiftool.exe, dcraw.exe (9.19, not requiring cygwin1.dll) together with dualiso.vbs (altered script by engardeknave), commands.cmd (this calls cr2hdr.exe and changes names), and runiffiles.cmd (a loop that calls dualiso.vbs if there's any file left to process).
Now, put all those in one directory, upload all .cr2 files in the same dir and run dualiso.vbs (you can change maxprocs to higher number for 4-core processors, but see KNOWN BUGS at the end). After some time you'll get all your files processed and put into different directories
Second method is even more dirty but gives you ability to run the script while getting files from the cards. Just start downloading from your card into the directory with the unzipped files and run runiffiles.cmd (after some bunch of .cr2 files are already there). The crazy script just checks constantly if there are any .cr2 files and calls dualiso.vbs then. So you can put all photos from several different cards and in the meantime the computer will process and rename them for you. You can leave the computer while moving the files and at the end everything will be done.
THESE SCRIPTS ARE PROVIDED AS IS AND I DON'T GIVE ANY WARRANTY IT WORKS AND DOES NOT DELETE YOUR PHOTOS SO PLEASE TEST IT ON SOME COPIES FIRST!!!!
KNOWN BUGS:
1. The script sometimes (especially at the beginning) calls more than maxprocs threads of cr2hdr.exe – that's probably because it tests for working cr2hdr.exe but calls commands.cmd, which gives some racing condition.
2. runiffiles.cmd sometimes never stops because some file already exists and exiftool won't move it, which makes runiffiles call dualiso.vbs again and it loops. I think it's because dualiso.vbs exits before all cr2hdr.exe processes are done. The easiest way to cope with this is to move the lonely files to other location, wait for runiffiles to stop and check if the files are already processed (should be).
ZIP FILE
http://bit.ly/dualisopack2
As I said, it's very dirty but WFM and saves me some time. Feel free to make it somewhat more user friendly and post it here.
I think this deserves its own thread in the post processing forum. Any one else?
I'm using a custom file prefix for my images, set via the standard Canon menus (5D mark 3). When using dual-ISO I would like to use the "DUAL" prefix instead, but enabling the "Custom file prefix" option in the dual-ISO settings doesn't seem to have any effect. All my dual-ISO images are still prefixed with my custom file prefix. Is that by design or is it a bug? I'm hoping it is a bug... :)
He who gets there first, gets the prefix and keeps it. So if you have one from bracketing or one from dual ISO you won't be able to change it until whatever changed it releases it.
Two badly processed Dual-ISO photos:
1. CR2 (https://mega.co.nz/#!Yc9VFC6T!cidlsw8J2YOGGTVnsmTzlVjuPK8v8MtM4oAZTK_Sgrs) DNG (https://mega.co.nz/#!oQs0kYob!NLGjazsBAfPAuhBT4KuUN0vwUw8kbPlp2kIK3oqgS8Q)
(http://i.imgur.com/kOHLRgq.png)
2. CR2 (https://mega.co.nz/#!cI1hzJhS!CqVmRQBoQv4Mu34tjy20LgGH83tQJSHtGwnFUoSwcU0) DNG (https://mega.co.nz/#!tIMjxZiR!bvz87l0yQiyXWGhAx4vxwh9mB0tRSwFN2poDXISqCnE)
(http://i.imgur.com/Kx4PDif.jpg)
Sensor response was nonlinear... I may need to estimate ISO in log space.
First one solved.
Second one is harder and it seems to have issues with chroma smoothing.
What camera did you use akumiszcza? i noticed i had an image like your bottom sample on my 600D, only the one though.
Quote from: Doyle4 on October 10, 2013, 02:19:41 PM
What camera did you use akumiszcza? i noticed i had an image like your bottom sample on my 600D, only the one though.
It's 50d. The problem on the bottom sample appeared in some previous photos in dark scenes, but only on small fragments.
Here's an experimental cr2hdr (http://acoutts.com/a1ex/cr2hdr_exp.exe) that solves the second picture.
It also fixes some dark spots that sometimes appeared in noisy areas.
Problems (this is why I didn't update it on the first page):
- you can see more aliasing on resolution charts, and in IMG_6706 from Danne (maybe others too).
- noise pattern is also different; visible on IMG_8485 from naturalsound, for example.
So, if you find a situation where you get noticeable shadow aliasing (more than with cr2hdr from first post), upload a CR2 and a side-by-side comparison.
Next time try exposing with ETTR. The second picture was really underexposed (and in this case it's better to simply shoot at ISO 1600).
Both photos were made with AETTR+Dual-ISO, but on AutoSnap mode and both, I think, were not the final, settled, photos.
Got it. If it's the one after a overexposed attempt, this explains it.
When ETTR looks at something underexposed, it knows exactly how much to adjust, but if it looks at something overexposed, it has no idea. It tries to use some heuristics from previous photos; sometimes works, sometimes not. If you know this, you should be able to predict what it's going to do.
shoudl I use dual iso when taking a bracketed hdr ?
Dual-ISO is meant for situations where HDR is not possible. Dynamic scenes vs. static scenes (with a grain of salt). If you are able to use HDR in a given situation you won't need Dual-ISO.
Ciao
Walter
Quote from: Walter Schulz on October 10, 2013, 06:43:46 PMIf you are able to use HDR in a given situation you won't need Dual-ISO.
... unless you haven't won in the lottery lately and would like to use 1 shutter cycle instead of 3, extending the camera's lifetime. Also using dual_iso will save you a lot of postprocessing time as it's "sooc", esp. the 6d benefits because unlike the 5d3 it doesn't do raw in-camera hdr.
Only problem is the 1/2 res shadows.
Do you have an example where this is noticeable?
Somewhere, its not that bad. The only real issues I've had were when I did dual ISO 800/100 and the image comes out completely black and then is a bit noisy when its brought up. I'm not a fan of using it in reverse, is this even proper?
I think this one the ISO 100 was just too far under. I have to look for or make some reversed ones where similar things happen.
http://www.filedropper.com/dual0955
I use Dual ISO in reverse (ie. 800/100 || 400/100) most of time, without any problem [5D2].
It's very useful to recover hight lights and/or to get max dynamic.
Okay, found the bug that was causing noisy shadows in the forest shot.
Before and after:
(http://acoutts.com/a1ex/forest00.jpg) (http://acoutts.com/a1ex/forest-0-0.jpg)
Also it fixes some black spots and banding, visible in the shot from 1% (which was also extremely underexposed; both test shots were developed at +6 EV in ufraw):
(http://acoutts.com/a1ex/DUAL09550.jpg) (http://acoutts.com/a1ex/DUAL0955.jpg)
This one passed all my pixel peeping tests, so you can download it from the first post. There's some small color fringing in the resolution charts, but I highly doubt you'll notice it in practice (all my other test shots look the same or better). If anything is rendered worse than with previous version, upload a CR2 and a side-by-side comparison.
Looks good, now have to see if I have any more @ rerun what I did already.
Thank you, a1ex – works great! As far as it's possible for these bad and unexposed photos ;)
BTW, I've found a quick way of checking if a given .cr2 is Dual-ISO, or not (I get quite many Dual-ISO photos without the prefix and some non-Dual-ISO with the prefix – I guess it's some kind of racing condition while having Auto Snap or Always On and shooting in quick series?). By turning small image previews in additional panel of Bridge (smaller than thumbnails for me at least) I see the stripes quite clearly – Moire effect. It's much faster than opening 100% previews at least. It should work with other software I guess, but you need to find the proper preview size.
(http://i.imgur.com/hr5kAdC.png)
I simply run cr2hdr on all files (usually, more than half of them are dual ISO, and the ones that are not are skipped quickly). Then, my raw processing script picks the DNG if there's one, or the CR2 otherwise.
The file prefix is not quite synced in burst mode, so I'm not sure whether to keep it or not. I don't think I can make it reliable in the near future.
Quote from: a1ex on October 11, 2013, 08:50:16 AM
I simply run cr2hdr on all files (usually, more than half of them are dual ISO, and the ones that are not are skipped quickly). Then, my raw processing script picks the DNG if there's one, or the CR2 otherwise.
I do the same way — see the scripts here: http://www.magiclantern.fm/forum/index.php?topic=8738.msg81768#msg81768
I've just noticed the moire in small previews and thought it might be useful for some people.
Quote from: a1ex on October 11, 2013, 08:50:16 AM
The file prefix is not quite synced in burst mode, so I'm not sure whether to keep it or not. I don't think I can make it reliable in the near future.
I would opt for removing it — it's unreliable and might leave some photos processed wrongly (it's not visible at first glance sometimes if photos are dual-iso and somebody might batch process them without using cr2hdr first, which can give unpredictable results).
The file prefix is very handy. Sure it doesn't work in burst mode but that's what warnings are for.
Processing 2000+ wedding photos, it's much easier to pick out the dual ISO shots with the prefix then to wait hours batch processing the lot of them.
It's only a problem where you have alternate shooting enabled. I'd settle for reduced capture rate where file prefix is not default and alternate capture is enabled.
Dual iso prefix. How do I enable this? Tried to in dual iso menu but there was no difference in prefix? I, m not in burst mode.
I just do the prefix + check the photos in FS viewer when moving them... has worked for like 400-500 pics.
Quote from: Audionut on October 11, 2013, 10:21:24 AM
The file prefix is very handy. Sure it doesn't work in burst mode but that's what warnings are for.
Processing 2000+ wedding photos, it's much easier to pick out the dual ISO shots with the prefix then to wait hours batch processing the lot of them.
It's only a problem where you have alternate shooting enabled. I'd settle for reduced capture rate where file prefix is not default and alternate capture is enabled.
I've made a short benchmark for 20 non-dual-iso cr2 files using cr2hdr alone and scripts I have there: http://www.magiclantern.fm/forum/index.php?topic=8738.msg81768#msg81768
cr2hdr alone (drag&drop) = 1:46.52
No renaming of files, just many processes of cr2hdr:
timer cmd /c dualisonorename.vbs = 48.391 (maxprocs=4)
40.902 (maxprocs=8 ) 41.252 (maxprocs=16)
Many processes of cr2hdr + renaming using exiftools:
timer cmd /c dualiso.vbs = 59.851 (maxprocs=4) 62.404 (maxprocs=8 )
So for 2000 photos it would take 1 hour 8 minutes on my old computer (Core2 Q6600). Less than that, because the program takes some time to start and stop (less processes at once then). Of course, for Dual-ISO files the time will increase, but so as with your method.
And I think I have some photos with wrongly assinged prefix that were not made by burst but by Always On AETTR. Probably ML finished calculations of new exposure at the exact moment I took the photo?
It's possible; when ETTR switches from dual to non-dual ISO, the change may not be applied right away; so if you catch the wrong timing, you are out of luck.
I've added a warning for now. Having a file list (like with intervalometer) may be easier to sync.
what do we need to download now from your first post ? cr2hdr or the dual iso module ?
Having a separate log of which files are dual ISO would make it easy for a post application.
Small cr2hdr update for video guys: better guess of black level in the absence of optical black bars.
Before:
(http://acoutts.com/a1ex/000166-0.jpg)
After:
(http://acoutts.com/a1ex/000166.jpg)
There may be subtle differences in photo mode too, in shadows (since this guess is used during blending).
@alex: Can you please check:
https://www.dropbox.com/s/cvdz6nz2t0rvnxy/DUAL0650.CR2
I converted it with the Oct-10 version (latest I guess) and at 100% (and more) patterns are visible.
(Shot at ISO 100/1600. The resultant DNG was set as: Exposure +3.2 Highlights -100 Shadows +100 Temperature: 3200)
@alex again. I had a problem with the dcraw (continuous error messages about not being valid win32 application if I recall correctly) from the latest .zip (Oct 10) I have overridden it with the dcraw from Sep 20.
what do we need to download for the fix now from your first post ? cr2hdr or the dual iso module ?
Quote from: tron on October 12, 2013, 04:03:53 AM
@alex: Can you please check:
https://www.dropbox.com/s/cvdz6nz2t0rvnxy/DUAL0650.CR2
I converted it with the Oct-10 version (latest I guess) and at 100% (and more) patterns are visible.
I must be blind, I can't see any patterns here (except for some noise in shadows, which is normal). Not even with crazy curves with extreme contrast.
Quote from: a1ex on October 13, 2013, 09:43:23 PM
I must be blind, I can't see any patterns here (except for some noise in shadows, which is normal). Not even with crazy curves with extreme contrast.
Alex I used Oct 10th cr2hdr.exe when I posted this. When I used Oct 11th version it seemed OK to me too!!!!
In fact I thought that maybe that's why you changed it!
What I still see is zig zag lines in a specific part where there is a round object. But if I recall correctly you had mentioned this yourself a few months ago as a characteristic of the procedure.
Sorry for not updating my observations with Oct 11th cr2hdr.exe
Nice to see more improvements upon dual iso and movie mode.
A question. @tron or anybody. Eager to try the changed prefix for dual iso which I understand is unreliable. Still, selecting this in dual iso the filenames are still without the new prefix I see some of you get? I use a 5d mark 3. Am I missing something? I,m in single shooting mode, Taking one picture, waiting. Still no changes in prefix?
Hi alex,
here is a Dual Iso Shot where i shot a face of a person and a bright sky. when i use the latest cr2hdr 1.5 app for mac the dngs look strange.. (especially the hair of the guy - like there is less resolution)
here are converted dual iso dngs:
https://www.dropbox.com/sh/ukdu06xedivedog/FyiCLZcHkm
and here are the unconverted dngs out of rawmagic 1.0
https://www.dropbox.com/sh/guqv68nxfn8uvc1/UKS4lnnjA9
thx. sw
Quote from: Danne on October 13, 2013, 10:55:30 PM
Nice to see more improvements upon dual iso and movie mode. A question. @tron or anybody. Eager to try the changed prefix for dual iso which I understand is unreliable. Still, selecting this in dual iso the filenames are still without the new prefix I see some of you get? I use a 5d mark 3. Am I missing something? I,m in single shooting mode, Taking one picture, waiting. Still no changes in prefix?
Do you have a custom prefix set via Canon menus?
Nice @Audionut. Thanks. Had to change under menu "filename" to "preset code".
Tried to shoot in burst mode and it didn,t miss naming to "DUAL" once. Maybe isn,t that unreliable after all? PLease do not take this feature away @Alex. Gonna save me tooons of time :)
By the way. Had no idea I could change naming prefix in the camera, sweet.
Thanks a lot for yet another feature Alex
Quote from: Danne on October 14, 2013, 05:49:03 AM
Tried to shoot in burst mode and it didn,t miss naming to "DUAL" once.
Set dual ISO to alternate frames only and then rapid fire ;)
Depending on the scene and the SNR limits in AETTR with linked dual ISO, it could miss the prefix there also when trigger happy.
hehe, I see. Luckily I,m not that trigger happy atm but will try to see what happens. Still very useful :)
I wouldn't use dual ISO on large shoots without the prefix. PP is already a time consuming job.
Quote from: akumiszcza on October 11, 2013, 06:28:19 PM
So for 2000 photos it would take 1 hour 8 minutes
On top of the process time for the actual dual ISO shots. So if I have 100 dual ISO shots, I've doubled that part of the processing time for no other reason then there is no other way to distinguish which shots need to be processed.
Then there's the file management. Prefixed shots get dumped into a separate folder and the processed DNG's copied into the shot folder. This way I can keep the original CR2's separate for later processing (updated cr2hdr or whatever), without having to process all 2000 photos again, and, I don't have interlaced CR2's polluting the image viewer.
Quote from: swinxx on October 13, 2013, 11:48:56 PM
here is a Dual Iso Shot where i shot a face of a person and a bright sky. when i use the latest cr2hdr 1.5 app for mac the dngs look strange.. (especially the hair of the guy - like there is less resolution)
Is this a regression? (was it rendered better with some older version?)
I'm not sure where to look, it seems OK to me.
Quote from: tron on October 13, 2013, 10:00:43 PM
Alex I used Oct 10th cr2hdr.exe when I posted this. When I used Oct 11th version it seemed OK to me too!!!!
In fact I thought that maybe that's why you changed it!
I didn't expect any change in banding patterns (since the last changes were mostly for deep shadows). It may also be because I run all my test shots through ufraw, without any sharpening or denoising.
I have an idea for horizontal banding correction, so I'm looking for some test shots where this is clearly visible.
hi alex.
i don´t know if that shot would be better with older versions.
perhaps it is too much difference between the brightness and the darker areas in that special shot.?
thx. sw
Quote from: Audionut on October 14, 2013, 06:42:20 AM
On top of the process time for the actual dual ISO shots. So if I have 100 dual ISO shots, I've doubled that part of the processing time for no other reason then there is no other way to distinguish which shots need to be processed.
If the prefix works for you, no reason then. But I get about 1 in 10 shots wrongly prefixed, both ways.
Quote from: Audionut on October 14, 2013, 06:42:20 AM
Then there's the file management. Prefixed shots get dumped into a separate folder and the processed DNG's copied into the shot folder. This way I can keep the original CR2's separate for later processing (updated cr2hdr or whatever), without having to process all 2000 photos again, and, I don't have interlaced CR2's polluting the image viewer.
That's what dualiso.vbs script does – it renames all files according to exif date and time, moves real Dual-ISO cr2s to separate folder than dngs and even makes a "_dualiso" suffix to Dual-ISO .cr2 and .dng to make them easier to distinguish. Anyway, I've already almost processed a Dual-ISO photo with some batch processing because I didn't notice it was Dual-ISO on quick preview (and it had no prefix). The result would have been strange...
Quote from: akumiszcza on October 14, 2013, 01:08:55 PM
If the prefix works for you, no reason then. But I get about 1 in 10 shots wrongly prefixed, both ways.
So disable it!
Quote from: Audionut on October 14, 2013, 04:44:40 PM
So disable it!
Yeah. That's the best remedy. Thanks...
BTW. Would it be possible to have a message during a shot review taken after SET mode in ETTR? 50D does not have ettr confirmation beeps and I don't know if ettr worked during LV after pressing SET, making the mode less useful than other. It would be great to have a message saying if ettr failed or not, especially since set mode is the fastest
Can the dcraw.exe fall back to sep 20 or before version? It does not work...
On the other hand Sept 20 version - or earlier, I never checked dcraw.exe creation date -anyway - works FINE (Even with newest cr2hdr.exe)
See http://www.magiclantern.fm/forum/index.php?topic=7139.msg81571#msg81571
@alex: Thanks. I got dcraw.zip
Quote from: akumiszcza on October 14, 2013, 04:46:45 PM
BTW. Would it be possible to have a message during a shot review taken after SET mode in ETTR? 50D does not have ettr confirmation beeps and I don't know if ettr worked during LV after pressing SET, making the mode less useful than other. It would be great to have a message saying if ettr failed or not, especially since set mode is the fastest
Can't you judge the results from the histogram?
Quote from: a1ex on October 11, 2013, 07:21:33 PM
It's possible; when ETTR switches from dual to non-dual ISO, the change may not be applied right away; so if you catch the wrong timing, you are out of luck. I've added a warning for now. Having a file list (like with intervalometer) may be easier to sync.
An idea concerning the prefix problem: Would switching from "_MG_" Adobe RGB to "IMG_" sRGB work? Since dual_iso is about raw, it doesn't matter what color space the sidecar jpeg is in, but using a Canon rename function might fix the sync problem if switching the color space prop(?) has no lag.
Small update in cr2hdr regarding horizontal banding. Basically, after matching the brightness (ISO + black difference), there's still some room for improvement, so this update does some fine-tuning for every dark line (so it matches the brightness of its neighbours).
The result is pretty subtle. I didn't find any example with singificant banding in final output, but it's clearly visible in the intermediate results. Here's a full resolution blending (the one that minimizes aliasing, but has pretty high noise):
Before:
(http://acoutts.com/a1ex/tron0.jpg)
After:
(http://acoutts.com/a1ex/tron.jpg)
So, the algorithm simply has cleaner data to work with, and it's less likely to pick up false detail. Also, the noise in full-resolution areas is less correlated (whiter). With this, I was able to push the algorithm even more towards full-res shadow detail without increasing the artifacts (so the forest shot is still rendered very well).
Example:
Before:
(http://acoutts.com/a1ex/IMG_6706-barrel0.jpg)
After:
(http://acoutts.com/a1ex/IMG_6706-barrel1.jpg)
For prefix, I think switching the color space has the same sync issue. Didn't try, but I don't see why it will be different.
There's no beep for ETTR with SET on any other camera; I look at raw zebras, histogram and ETTR hint.
I just turn the beep off globally... too much beeping in general.
Very nice improvement on the horizontal banding!
Quote from: Danne on October 15, 2013, 05:37:58 PM
Very nice improvement on the horizontal banding!
I'm re-processing all my dual_iso cr2 now, these updates really keep my computer busy :-p ... fortunately with Lightroom you just need to replace the old dng & update the preview, all the metadata is in the LR database. Also afterwards I'm converting the ML dng to ACR dng which about halves the file size.
Would be nice if you can find a before/after example where the difference is clearly visible.
I have improvements in shadows but not in highlights. No banding examples.
Quote from: Marsu42 on October 15, 2013, 06:30:08 PM
I'm re-processing all my dual_iso cr2 now, these updates really keep my computer busy :-p ... fortunately with Lightroom you just need to replace the old dng & update the preview, all the metadata is in the LR database.
Smart workflow. I noticed that LR kept the metadatas when switching cr2 to dngs after applying xmp sidecar files. Didn,t try a fully new conversion yet though of my older cr2:s
Quote from: a1ex on October 15, 2013, 06:36:36 PM
Would be nice if you can find a before/after example where the difference is clearly visible.
The only problem shot I delivered is unchanged, it's still colorful confetti time :-\ ... https://bitbucket.org/Marsu42/ml-mod6d/downloads/131003151753_D2X005512-CR2HDR131015.jpg
Could you provide a way to produce *two* dng from cr2hdr, one bright and one dark (with duplicated scanlines)? That would make it easier to see where a problem was, i.e. how the composite shots were exposed, and if a problem was there even before cr2hdr assembly.
Lookup "fullres.dng" in cr2hdr.c and mark that block with #if 1. You'll get a bunch of intermediate files.
You can do that everywhere you see calls to save_dng.
Quote from: a1ex on October 16, 2013, 09:16:04 AM
Lookup "fullres.dng" in cr2hdr.c and mark that block with #if 1. You'll get a bunch of intermediate files.
Thanks, I'll do that - and I know you're not very keen to add any switches to cr2hdr, but imho this option would be useful to all people and not only to the few who compile ml themselves or read this thread. Also the intermediate files could be a last resort if cr2hdr fails on some shots, you can then blend the intermediate file with the good parts from cr2hdr.
Yesyesyes, I could do a pull request, but I feel sufficiently busy with the auto_iso module, I've got a couple of (imho :-)) very nice ideas for that.
From your shot, I've cleaned up some of the dots with this trick:
int is_hot_small = (d > b && b < white_darkened);
but finding a general solution that doesn't break resolution charts is a bit harder. Also, locating hot pixels where the high ISO is overexposed is hard and I didn't try to solve it yet (a median filter with thresholding should help a bit).
Quote from: a1ex on October 16, 2013, 11:25:02 AM
but finding a general solution that doesn't break resolution charts is a bit harder. Also, locating hot pixels where the high ISO is overexposed is hard and I didn't try to solve it yet (a median filter with thresholding should help a bit).
Thanks, I'll make my own cr2hdr with your patch for the time being (these were just test shots anyway) - but imho having a public fix for this is important, dual_iso is just made for high-dr longtime exposures since you really wouldn't want to bracket in these situations.
Question is if you can find auto-detection algorithms for all possible contingencies without years of coding and wading through thousands of samples, if not you might have no other option than to fall back for command line parameters for local tuning like this hot pixel removal.
I'll try a thresholded median filter; if I get good results, I'll post it.
Meanwhile, here's an interesting read: http://nikonhacker.com/viewtopic.php?t=85
Doesn't Canon apply some form of noise reduction even when it's turned off, even for raw?
The noise seems pretty much white to me, so I don't think there is any noise reduction applied.
ISO 100 crop from a dark frame:
(http://a1ex.magiclantern.fm/bleeding-edge/crop-iso100.png) (http://a1ex.magiclantern.fm/bleeding-edge/corr-iso100.png)
Octave code for plotting the autocorrelation:
system("dcraw -v -d -r 1 1 1 1 -g 1 1 -k 2000 -S 2096 100.CR2")
im = double(imread('100.pgm'));
im = im(501:600, 501:600);
imshow(im, []);
set(gca,'position',[0,0,1,1])
print -dpng "-S100,100" crop.png
x = xcorr2(im - mean(im(:)));
imshow(x, []);
set(gca,'position',[0,0,1,1])
print -dpng "-S199,199" corr.png
I've been out of the Dual ISO game for a while now and was hoping somebody could enlighten me on some topics.
Video Processing - is this the process?
1) RAW file on to PC
2) RAW2DNG
3) DNG's on to CR2HDR?
Does "RAWanizer" automatically do this process? It produced some non-banded DNGs that have less aliasing and comparable range so I am unsure. When the files were put straight in to RAW2DNG they came out alternated banding.
Are these the expected results?
Before cr2hdr: https://www.dropbox.com/s/mhgqe2eqkva67kq/beforecr2hdr.dng
After cr2hdr: https://www.dropbox.com/s/j22onsqq70yoq1e/aftercr2hdr.DNG
Should there be more shadow recovery in the processed version? Or am I doing something wrong?
Here is the resulting processed JPG, am I meant to raise the shadows and blacks 100%?
(http://i.imgur.com/egJ7F6E.jpg)
Hi,
Long time ML follower first time poster here.. First of all, AWESOME job on this dual ISO, can't wait for it to be fully operational and integrated! This method is the realization of a thought I had some time ago. But with me being completely useles with programming that was just a wish or a dream. So I would like to thank you for making this reality!
I do know that I shouldnt ask this question.. But after reading the update in the first post about this function being available for most models I am wondering if this function is available for the 1100D/T3? I downloaded the most recent nightly build but I cant find it anywhere in the menu system. Where should it be located exactly?
THANK YOU once again a1ex!
According to http://nanomad.magiclantern.fm/jenkins/features.html it doesn't work yet, but I'm pretty sure it's possible.
Here are the findings from Panzani, so you can start researching from here (I guess you just have to find the repeated pattern, i.e. which bits are for ISO):
/*
100 - 0 0x407444B2
200 - 0x120 0x407444C6
400 - 0x240 0x407444DA
800 - 0x360 0x407444EE
1600 -0x480 0x40744502
3200 -0x5A0 0x40744516
*/
is_1100d = 1;
FRAME_CMOS_ISO_START = 0x40745446; // CMOS register 00 00 - for LiveView, ISO 100 (check in movie mode, not photo!)
FRAME_CMOS_ISO_COUNT = 6; // from ISO 100 to 3200
FRAME_CMOS_ISO_SIZE = 28; // distance between ISO 100 and ISO 200 addresses, in bytes
PHOTO_CMOS_ISO_START = 0x407444B2; // CMOS register 00 00 - for photo mode, ISO
PHOTO_CMOS_ISO_COUNT = 6; // from ISO 100 to 3200
PHOTO_CMOS_ISO_SIZE = 20; // distance between ISO 100 and ISO 200 addresses, in bytes
Pixel peepers: which one looks best?
(http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-nearest.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-mean23.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-amaze-nearest.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-amaze-mean2.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-amaze-mean2w.jpg)
(http://a1ex.magiclantern.fm/bleeding-edge/dark-nearest.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/dark-mean23.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/dark-amaze-nearest.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/dark-amaze-mean2.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/dark-amaze-mean2w.jpg)
This is the reconstruction of only one exposure (the dark one), without any prior info from the bright one (so I'm just evaluating the interpolation methods). The interpolation is done before debayering. You may choose any of them, or any linear combination of them.
For reference: here's an interpolation done after debayering, with GIMP, Lanczos sinc filter. Don't choose this one, because it's too hard to implement :P
(http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-squeeze-ufraw-gimp-resize-sinc.jpg)
Debayering was done in all cases with ufraw, AHD, chroma smoothing, no denoise.
:) Nice test!
I think I decide for the last one. First I liked the 2nd exemple but now I,m leaning more towards the last two. Looking at the hair I think the last one, looking att the green tree background I really like the smooth clean in number one, looking at the shirt, either green or magenta, hmm ,Probably the last.
Quote from: a1ex on October 18, 2013, 04:47:50 PM
Pixel peepers: which one looks best?
For me: The second from the right, is looks more natural, closer to your "ideal" GIMP version and less over-sharpened than the very right one. It's an interpolation after all, and it shouldn't try too hard to reconstruct data that isn't there, I can do sharpening myself in postprocessing and double sharpening (in dual_iso and post) might even result in artifacts.
It's not about sharpening (I didn't apply any), but about minimizing the aliasing artifacts.
The texture looks like this:
(http://a1ex.magiclantern.fm/bleeding-edge/danne-fullres.jpg)
(not sure if the purple thing is lens fringing, because this crop is from an edge; maybe Danne can tell us)
Some more crops, this time from the final output:
(http://a1ex.magiclantern.fm/bleeding-edge/danne-mean23-nocs.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-mean23-cs5x5.jpg)
(http://a1ex.magiclantern.fm/bleeding-edge/danne-amaze-mean2w-nocs.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-amaze-mean2w-cs5x5.jpg)
Top: mean23
Bottom: AMaZE interpolation
Left: no chroma smoothing
Right: 5x5 chroma smoothing
(so, top right is pretty much what you get now with the current version)
I think it's a contest between the top right and bottom left. I'd lean toward the chroma smoothing because it seems like there's a lot of color bleeding with the AMaZE interpolation.
Quote from: a1ex on October 18, 2013, 09:29:35 PM
It's not about sharpening (I didn't apply any), but about minimizing the aliasing artifacts.
The texture looks like this:
(http://a1ex.magiclantern.fm/bleeding-edge/danne-fullres.jpg)
(not sure if the purple thing is lens fringing, because this crop is from an edge; maybe Danne can tell us)
It,s my good old canon 20-35mm f2.8 if I remember correct, and there,s surely some purple fringing going on from the lens :)
Quote from: a1ex on October 18, 2013, 09:52:40 PM
Some more crops, this time from the final output:
(http://a1ex.magiclantern.fm/bleeding-edge/danne-mean23-nocs.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-mean23-cs5x5.jpg)
(http://a1ex.magiclantern.fm/bleeding-edge/danne-amaze-mean2w-nocs.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-amaze-mean2w-cs5x5.jpg)
Top: mean23
Bottom: AMaZE interpolation
Left: no chroma smoothing
Right: 5x5 chroma smoothing
(so, top right is pretty much what you get now with the current version)
Top right looks really good. Hard to beat.
Nocs has color artifacts in the shirt, amaze version is better but the artifacts are still there. Also in the highlights in the hair above, and above left of the hair clip. However they don't suffer the aliasing in the hair on the left side of the head near the eye socket. Nocs also has sharper fine detail very noticeable in fine strands of hair on the back of the head.
My pick would be amaze-mean2w-nocs. Sharper finer detail without the aliasing and less color artifacts then mean23-nocs.
A lot of the color artifacts seem to be on specular highlights. Can you desaturate these areas?
I prefer the one produced by AMaZE interpolation for it has more fine detail and less color artifacts.
Quote from: Audionut on October 19, 2013, 05:04:37 AM
A lot of the color artifacts seem to be on specular highlights. Can you desaturate these areas?
That's where I'm applying chroma smoothing. To desaturate, I think I need to know the white balance first.
Quote from: Marsu42 on October 18, 2013, 09:21:33 PM
It's an interpolation after all, and it shouldn't try too hard to reconstruct data that isn't there
I'm only trying to reconstruct data that
is there. Current method does not fully exploit the inter-channel correlation, because I'm interpolating from the nearest Bayer pixels of the same color. With a temporary demosaic I have color info at every pixel, so I can find neighbouring pixels at a smaller distance => the Nyquist frequency should increase a little. Sure, the demosaiced pixels are guessed, not sampled, so I need a really good guess (for sure, one that considers the influence of say a red pixel to guess a green pixel, otherwise there will be no effect on Nyquist frequency. I'm pretty sure all modern demosaic algorithms do that - you have the proof in the pink dot remover thread).
Quote from: a1ex on October 19, 2013, 08:13:43 AM
That's where I'm applying chroma smoothing. To desaturate, I think I need to know the white balance first.
Um, sorry for being daft again, but the wb value is supposed to be in the raw file, isn't it? If the problem is calculating awb, the camera could shoot dual_iso with a fixed/known wb as it doesn't matter in raw and you could try fixing the color artifacts - I also noticed them in some of my shots on tree leaves in front of the sun.
Quote from: a1ex on October 18, 2013, 09:52:40 PM
Top: mean23
Bottom: AMaZE interpolation
Left: no chroma smoothing
Right: 5x5 chroma smoothing
Before chroma smoothing, AMaZE has more aliasing in the shirt pattern, but afterwards there are less color artifacts left - that's why I prefer bottom left (AMaZE + smoothing).
The WB value is what you set in post-processing, after running cr2hdr. For desaturation, it does matter, because if you use some other WB, you will get hue shift.
Quote from: a1ex on October 19, 2013, 09:09:38 AM
The WB value is what you set in post-processing, after running cr2hdr.
But.. but ... the camera has a wb setting (either awb, situation (cloudy, ...) or explicit kelvin) that is written to the cr2, in 90% of the cases I don't change that in postprocessing. Or is the in-camera wb different from the postprocessing wb (tint, ...)? Thanks a lot for explaining anyone, I really feel a bit dumb I cannot tell the difference :-o
Yes, but I don't want to rely on that, because I change WB in post pretty often.
Hi, a1ex,
I found today this black leaking into the picture:
https://www.dropbox.com/s/1llw7tzxzvrw79h/2013-10-19_3286.jpg (https://www.dropbox.com/s/1llw7tzxzvrw79h/2013-10-19_3286.jpg)
here is original cr2:
https://www.dropbox.com/s/go0cnmjv3hwdcyx/2013-10-19_3286.CR2
I processed it with the last cr2hdr from 15.10.
Could this be cured?
I'm not sure what to look for, can you be a little more specific?
Quote from: a1ex on October 15, 2013, 06:36:36 PM
Would be nice if you can find a before/after example where the difference is clearly visible.
This is all I have for banding. Badly underexposed image there is improvement in the banding but not removed. I see also resolution improved.
(http://s20.postimg.org/ipde1u5rf/Banding_in_underesposed.jpg)
a1ex,
There is black halo around the person, black color from his clothes is spreading/leaking
into surrounding area.
boosted effect: https://www.dropbox.com/s/d4q4n08ukggjoy8/2013-10-19_3286%20more%20visible.jpg (https://www.dropbox.com/s/d4q4n08ukggjoy8/2013-10-19_3286%20more%20visible.jpg)
LR: exposure 0, highl. -100, shadows -100, clarity +100
Can you see it now?
Ah, that one is from your postprocessing software, not from cr2hdr.
Here's how I would render it (with my HDR script (http://www.magiclantern.fm/forum/index.php?topic=7022), highlight_level = 10000, midtone_level = 10000, shadow_level = 5000, overall_bias = 1, ufraw_options = "--temperature=5500 --green=1 --saturation=1.3 ")
(http://a1ex.magiclantern.fm/bleeding-edge/2013-10-19_3286crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/2013-10-19_3286.jpg)
(click for full image)
Thanks, a1ex,
I do not know what to think, I have never had such a result from Lightroom,
so I asked you about dualISO.
But I see your result, it is really OK...
Hm. ... Thanks anyway, I will look further into it. Perhaps. :-)
Another interpolation method (added at the end). It's from the same data set (a half-resolution image).
(http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-nearest.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-mean23.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-amaze-nearest.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-amaze-mean2.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-amaze-mean2w.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/danne-dark-amaze-edge.jpg)
(http://a1ex.magiclantern.fm/bleeding-edge/dark-nearest.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/dark-mean23.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/dark-amaze-nearest.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/dark-amaze-mean2.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/dark-amaze-mean2w.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/dark-amaze-edge.jpg)
Another crop (this one is from the final output). DNG from Africashot. (http://www.magiclantern.fm/forum/index.php?topic=8035.msg74606#msg74606) First image is from the current version (a7492e4, 2013-10-15).
(http://a1ex.magiclantern.fm/bleeding-edge/africashot-a7492e4.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/africashot-amaze-mean2w-cs5x5.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/africashot-amaze-edge-cs2x2.jpg)
If I'm reinventing the wheel with all these interpolation methods, please point me in the right direction.
are those really the same pictures.? the last one looks better in every aspect. great.
Yes, they are the same pictures.
whow.. great alex.
can i try some of my pics? is there a cr2hdr version?
Here, just uploaded. I didn't try it on my test collection yet, so you may have surprises.
cr2hdr-amaze-edge.exe (http://acoutts.com/a1ex/cr2hdr-amaze-edge.exe)
First side effect seems a good one: it removes some of the hot pixels ("star eater").
great thank you. will try that out.
The last ones look amazingly good. Wonder how this will apply to dual-iso movies
Hi, a1ex,
I tried your new cr2hdr-amaze-edge.exe - it is really much better. Real improvement. Thank you.
However, I have a cr2 which crashes that new exe. (at least for me)
Previous version of cr2hdr did process it without problem.
Here is that cr2: https://www.dropbox.com/s/egdau7pki8620t1/2013-10-17_3125.CR2 (https://www.dropbox.com/s/egdau7pki8620t1/2013-10-17_3125.CR2)
Quote from: sletts02 on October 18, 2013, 12:25:58 AM
I've been out of the Dual ISO game for a while now and was hoping somebody could enlighten me on some topics.
Video Processing - is this the process?
1) RAW file on to PC
2) RAW2DNG
3) DNG's on to CR2HDR?
Does "RAWanizer" automatically do this process? It produced some non-banded DNGs that have less aliasing and comparable range so I am unsure. When the files were put straight in to RAW2DNG they came out alternated banding.
Are these the expected results?
Before cr2hdr: https://www.dropbox.com/s/mhgqe2eqkva67kq/beforecr2hdr.dng
After cr2hdr: https://www.dropbox.com/s/j22onsqq70yoq1e/aftercr2hdr.DNG
Should there be more shadow recovery in the processed version? Or am I doing something wrong?
Here is the resulting processed JPG, am I meant to raise the shadows and blacks 100%?
(http://i.imgur.com/egJ7F6E.jpg)
amidoingitright?
I don't quite understand the question, but you are free to color grade it as you wish.
Quote from: a1ex on October 20, 2013, 10:46:41 PM
Here, just uploaded. I didn't try it on my test collection yet, so you may have surprises.
The surprise over there is that this cr2hdr-amaze-edge.exe crashes in the "Interpolation" step on nearly every cr2 I test it on - from the 6d, that is. The current/old cr2hdr trunk version works fine. I really hope it's not again only me this time :-p
Here's a random cr2 to crash it: https://bitbucket.org/Marsu42/ml-aiso/downloads/DUAL8021.CR2
Here's one of the few cr2 that happen to work, maybe it helps you to find the bug: https://bitbucket.org/Marsu42/ml-aiso/downloads/DUAL7537.CR2
Yea, some are crashing it. At edge detection interpolation. The amze part is fine.
Pretty please can it come to raw2dng when the cause of the crash is found. What it processed looks very clean.
I'm even thinking to try this algorithm for upsampling 720p videos (since they are full-res horizontally, but have skipped lines, so it should be pretty similar). Though, here I'm exploiting the pattern of 2 lines sampled and 2 missing (so I have pretty good confirmation when edges are diagonal), but on 720p videos it's something like one line sampled and 5 missing.
There are a couple more good algos in raw therapee.. I think IGV, don't have a dng on here to test with.
Cool. I need to upgrade my ubuntu box to try the latest RT.
I guess any of them should work, since I copied AMaZE code from RT and did some minor changes (mostly C++ to C conversion).
ubuntu's RT is really old, I had to add a PPA or get it straight from them.
Here's another snapshot. Should fix the crash, and also has some different tuning parameters.
cr2hdr-amaze-edge2.exe (http://acoutts.com/a1ex/cr2hdr-amaze-edge2.exe)
If you find anything that looks better with the previous one, upload a DNG.
My previews broke on 6D :(... b/d errors on most frames and then very small 1/2 height preview on very large iso ranges, eg 100/6400
EOSM too, no B/D errors but 1/2 height.
(http://i.imgur.com/h40q7qAl.jpg)
The cr2hdr is now crash free... its cranking away on a folder. AVG 6 - 7 seconds to complete the debayering.
Yeah, that preview looks interrupted from half-shutter.
Try shooting in single mode (not continuous), since it doesn't check half-shutter there.
All in single mode.
Can you check whether is_continuous_drive() works properly? (should return 0 in single mode).
Or, undo the cache tricks (revert 283c1f2) to see if it solves anything.
Looking at auto ETTR, continous silent is detected properly but must be returning 0 for normal continuous and everything else since I don't see the warning.
Then, it's probably the cache or lack of vsync. On 5D3, if I comment out the msleep from the undo hook, I get two windows of preview (one at the top and the other at the bottom); in the middle, I get interlaced data (already undone). So there's some processing going in parallel, I guess.
Quote from: a1ex on October 22, 2013, 08:53:40 PM
If you find anything that looks better with the previous one, upload a DNG.
Ok, it's not crashing anymore - and I've got a sample right away, I'll pm you the link as it 's for debugging only.
Look at the white spot, the amaze interpolation leaves less artifacts and is better at reconstructing the fur. This is great news for me since I already had the problem with line/dot artifacts in areas before that are clipped in one of the dual_iso exposures, I can send you another sample of this artifact type if you want it.
What it does to the noise is phenomenal. Much cleaner output overall.
EOSM and 6D preview are identical though and both of these are using the same hook now. EOSM doesn't have the b/d detection problem I think.
CR2 is uploading.
Quote from: 1% on October 22, 2013, 11:27:42 PMWhat it does to the noise is phenomenal. Much cleaner output overall.
I have to say I'm also AMaZEd how good this works by now concerning sharpness & noise. When I first read about it I thought "well, yeah, there have to be a lot of downsides to it" - but there aren't! Except for processing time & the need to keep the original cr2 around, meaning 2x the data storage, 3x if you have backups - but that can't be helped.
Could we somehow use this method (i mean this new quality) for dual iso raw video?
I think that would already work with the bin provided.
Hey guys, anyone using cr2hdr.exe for video via 5d3 ever get the "The filename or extension is too long" error? It worked fine for two of my DNG folders, but now I keep getting this error. Is it because there are too many dng files being dropped on it?
Thanks in advance
***EDIT -- please ignore above... I ended up dragging and dropping half of dng's at a time in the folder. Working fine now... silly me.
need help, who can come across.
two computers. is winXP. both are not open DNG format.
here the original https://dl.dropboxusercontent.com/u/13183292/dualISO/2013/10/2121/IMG_2690.CR2 (https://dl.dropboxusercontent.com/u/13183292/dualISO/2013/10/2121/IMG_2690.CR2)
Here the resulting DNG https://dl.dropboxusercontent.com/u/13183292/dualISO/2013/10/2121/IMG_2690.DNG (https://dl.dropboxusercontent.com/u/13183292/dualISO/2013/10/2121/IMG_2690.DNG)
that's how I see DNG (print screen). https://dl.dropboxusercontent.com/u/13183292/dualISO/2013/10/2121/printscreen.JPG (https://dl.dropboxusercontent.com/u/13183292/dualISO/2013/10/2121/printscreen.JPG)
Known bug. Ignore the jpeg preview.
i am having problems with newest cr2hdr, I have downloaded cr2hdr-amaze-edge2.exe but drag and drop doesnt work. I have downloaded also dcraw, put it in the same folder, nothing. When i use command line and type all by hand it tells that DNG doesnt look like interlaced - but it surely is.
Normally I am using rawanizer with some new raw2dng, it works and deinterlaces dual iso footage but i wanted to try those newest solutions. Thanks
See here: http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
I got the same problem as kgv5.
Material was filmed with iso 100 and iso 1600. .RAW was converted to DNG with RAWMagic 1.0 beta 7b. The cr2hdr package was installed and after that I tried to convert the dng-dual-isofiles to cr2, but it didn't succeed. For me it looks like cr2hdr is temporary creating a cr2-file but then deleting it, because right before it finishes there is a .CR2 which is automatically deleted.
I'm on a Mac 10.8.5
cr2hdr is version 1.4
Any suggestions?
Before: (http://666kb.com/i/cimgg09lj76jaumdo.jpg)
After: (http://666kb.com/i/cimggndwgm7crkrf0.gif)
Are you sure you have used dual ISO?
Yes, I'm pretty sure. Both modules (raw and dual iso) are on in the modules-menu. I got them both turned on in the movie-menu. It shows Iso 100/800 on the lcd and it's also changing from 100 to 800 while recording and as you can see in the screenshot of the folder there is one dark frame and one bright frame. The only thing I am a bit curious about is that there are two bright frames at the beginning 00000 and 00001.
From your description and screenshot, I'm pretty sure you didn't.
You used HDR video, which is a different beast.
Quote from: a1ex on October 23, 2013, 12:23:44 PM
From your description and screenshot, I'm pretty sure you didn't.
You used HDR video, which is a different beast.
Sorry, my fault! That was a stupid one, thought HDR-video was dual iso...
I'm really AMaZEd!
Below is a shot i nearly lost hope on, but somehow i knew someday a1ex gonna fix it! :D
The 2nd one from left was developed with ufraw at default. Most right was tonemapped with Photomatix to gain some contrast on the helicopter.
With those on the left i fiddled for hours on any raw developer i could get hands on just to see if there is at least some details in the sky.
I expected there would be just a near white canvas behind a black spot.
(http://img262.imagevenue.com/loc378/th_529286951_DUAL6183_cr2hdr_122_378lo.jpg) (http://img262.imagevenue.com/img.php?image=529286951_DUAL6183_cr2hdr_122_378lo.jpg) (http://img293.imagevenue.com/loc510/th_252900541_DUAL61_20_09_cr2hdr_122_510lo.jpg) (http://img293.imagevenue.com/img.php?image=252900541_DUAL61_20_09_cr2hdr_122_510lo.jpg) (http://img178.imagevenue.com/loc135/th_529112214_DUAL61_amaze_edge2_122_135lo.jpg) (http://img178.imagevenue.com/img.php?image=529112214_DUAL61_amaze_edge2_122_135lo.jpg) (http://img234.imagevenue.com/loc518/th_529102645_DUAL61_amaze_tonemapped_PM_122_518lo.jpg) (http://img234.imagevenue.com/img.php?image=529102645_DUAL61_amaze_tonemapped_PM_122_518lo.jpg)
Link to CR2: https://dl.dropboxusercontent.com/u/90483500/DUAL-ISO/DUAL6183.CR2 (https://dl.dropboxusercontent.com/u/90483500/DUAL-ISO/DUAL6183.CR2)
This one is really hard because there's little or no overlap between the two exposures. What ISO did you use?
I've got a good DNG (http://a1ex.magiclantern.fm/bleeding-edge/DUAL6183.dng) by forcing ISO 100/3200, but the autodetection doesn't work. I'll keep it as a test case, since I want to try to autodetect ISO and bright/dark fields from histogram analysis.
I can't remember the Dual-Iso settings.
All i remember is wondering why my liveview is so bright while shooting a RAW-video of another helicopter.....
...till i found out that by activating modules to shoot RAW i also enabled DUAL-ISO (Of course with some high ISO d'oh).
It was the first Dual-Iso build for 550D by mk1174. And if memory serves me right going to ISO 6400 wasn't possible.
So i guess ISO 100/3200 is right, at least your DNG looks as good as it could get.
Another update for the amaze experiment, mostly on the speed side.
Basically, this algorithm is very powerful in semi-overexposed areas and (coupled with the alias map) in deep shadows. In midtones, you already have full detail from both ISOs, so there's no point in doing expensive interpolations that will be discarded anyway. So, this update will be a little faster for well-exposed images, and a little slower for tricky ones.
cr2hdr-amaze-edge3.exe (http://acoutts.com/a1ex/cr2hdr-amaze-edge3.exe)
Here's a crop that was quite difficult to solve, because all this detail was covered only by one ISO:
(http://acoutts.com/a1ex/D46A8019-2500-2000-before.jpg) (http://acoutts.com/a1ex/D46A8019-2500-2000-after.jpg)
(obviously, power lines can't be guessed, but most diagonal lines should be fine).
If there are no obvious regressions, I'll update the first post.
@alex: There is indeed significant improvement. A slight negative is that the lens correction part of ACR cannot remove CA as it could.
https://www.dropbox.com/s/5d77c4rkirfn3tj/oct-11-version.JPG
https://www.dropbox.com/s/4yby8ffj21r3658/oct-23-am-edge3-version.JPG
Wow! This looks amazing.
Quote from: a1ex on October 23, 2013, 06:25:32 PM
Another update for the amaze experiment, mostly on the speed side.
Basically, this algorithm is very powerful in semi-overexposed areas and (coupled with the alias map) in deep shadows. In midtones, you already have full detail from both ISOs, so there's no point in doing expensive interpolations that will be discarded anyway. So, this update will be a little faster for well-exposed images, and a little slower for tricky ones.
cr2hdr-amaze-edge3.exe (http://acoutts.com/a1ex/cr2hdr-amaze-edge3.exe)
Here's a crop that was quite difficult to solve, because all this detail was covered only by one ISO:
(http://acoutts.com/a1ex/D46A8019-2500-2000-before.jpg) (http://acoutts.com/a1ex/D46A8019-2500-2000-after.jpg)
(obviously, power lines can't be guessed, but most diagonal lines should be fine).
If there are no obvious regressions, I'll update the first post.
Quote from: tron on October 23, 2013, 11:09:00 PM
A slight negative is that the lens correction part of ACR cannot remove CA as it could.
That was probably chroma smoothing. You can apply it as a separate step (I don't know how it's called in PS, but in GIMP it's this: http://registry.gimp.org/node/10306 )
Dear ML team and members,
I am new to ML, I am using 5d Mark III. how can i install and use this feature?
how can we remove this add-on after installation?
have any one lost his camera because of this?
Regards,
Naveed
Quote from: naveed on October 24, 2013, 09:05:00 AM
I am new to ML, I am using 5d Mark III. how can i install and use this feature? how can we remove this add-on after installation? have any one lost his camera because of this?
Feel free to have a look at the tutorials section ... but here you go: Get a "nightly" release & enable the dual_iso in the module tab - you can disable it again anytime you want. However, if you are concerned about bricking your camera and safety you shouldn't use ml on your 5d3 at all right now as it's in the unstable alpha stage.
Quote from: tron on October 23, 2013, 11:09:00 PM
https://www.dropbox.com/s/4yby8ffj21r3658/oct-23-am-edge3-version.JPG
@tron: can you post the same crop with this version?
cr2hdr-amaze-edge4.exe (http://acoutts.com/a1ex/cr2hdr-amaze-edge4.exe)
I'm trying to find out why do you get jagged edges in highlights; I can't get them in either ufraw or rawtherapee, no matter how aggressive highlight recovery, sharpening or tone mapping I try. My crop looks like this:
(http://a1ex.magiclantern.fm/bleeding-edge/DUAL0650.jpg)
The only difference from the previous one (edge3) is in highlights above white level.
Hi folks,
first of all: thanks Alex for unleashing the dual_iso beast on the 5d2! (and other cams) (and overall for generally unleashing that beast at all...) THANKS!
Here are a couple of images, for the fun of it... (based on ...amaze_edge2.exe)
I know, I pretty much overdid the HDR effect, but heck, thats whats possible with dual_iso :-)
Some of those images were strongly underexposed, even to a degree where only the high-iso lines captured detail at all. Postprocessing all done in Lightroom 4...
Have fun,
Bass
(http://www.broome.de/pics/IMG_9162_klein.jpg) (http://www.broome.de/pics/IMG_9171_klein.jpg) (http://www.broome.de/pics/IMG_9174_klein.jpg) (http://www.broome.de/pics/IMG_9176_klein.jpg) (http://www.broome.de/pics/IMG_9252_klein.jpg)
edit: smaller images
Hi! did you use noise reduction as well on the pictures? Lacks detail?
yes, very strong noise reduction actually. Getting brightness up on underexposed images drastically turns up noise, so, yep, I had to hit that button hard...
Btw, how do I setup nicely clickable thumbnail images within the forum editor? Sorry for cluttering the thread with those bulky full-res images.... :-/
Bass
Edit: THANKS admin for changing those images into small tumbnails. How do I do that? (I initially used the "insert image" feature...)
I use something like [img width=200].
If you still have noise, you probably need something like 20 stops of dynamic range (or clip some more highlights).
Thanks, I just checked the corrected post. Will use the "img height = 100" tag in the future!
Yeah, well, black dog against direct sun is probably something at the very edge of whats possible dynamic range wise. I keep on working to find the right ETTR / dual_iso range balance...
A couple of days ago when I first started using dual_iso I noticed regular patterns in the resulting DNGs.
I reconverted one of the original CR2 with todays "...amaze_edge4.exe". Pattern is unchanged, see the 300% crop:
(http://www.broome.de/pics/dualISO_pattern.jpg)
If I remember correctly dual_iso was set to ISO 800/100. Any idea?
Can you upload a CR2?
yep, please check your inbox.
Diagnosed: there's gibberish in the top optical black bar, and cr2hdr interprets it as black level.
Thanks for checking!
I don't know if I fully understand. What kind of gibberish is there? Is there anything I can do to avoid that?
Look with dcraw -E IMG_1234.CR2. The top bar wasn't pitch black, as I've assummed until now.
Solved: cr2hdr-amaze-edge5.exe (http://acoutts.com/a1ex/cr2hdr-amaze-edge5.exe)
Also, the embedded previews should be OK now.
:-\ I'm forced to do some of my processing at work on a Mac OS X 10.5.8, I've downloaded the Mac apps (all three) from the first post in this thread and when I try to process a .CR2 file the cr2HDR crashes with this reported:
cr2hdr // Beta 1.4
Input file : IMG_1062.CR2
dyld: unknown required load command 0x80000022 /Users/rollinbergman/Desktop/Dual ISO ap/OSX_cr2hdr/cr2hdr.app/Contents/Resources/script: line 17: 94254 Trace/BPT trap "$CR2HDR" "$FILE_NAME".CR2 > "$FILE_NAME".txt Logfile : IMG_1062.txt
THE END
I've been searching the forums for a few hours looking for a solution but haven't been able to find the answers.
What am I missing here...when I'm processing at home on a PC the process goes off without a hitch.
Yep, great stuff there, Alex. Thx!
Couple of examples, also from other shots:
(alternating edge4, edge5, same lightroom settings per pair)
(http://www.broome.de/pics/di1_edge4.jpg)(http://www.broome.de/pics/di1_edge5.jpg) (http://www.broome.de/pics/di2_edge4.jpg)(http://www.broome.de/pics/di2_edge5.jpg)
Also the color rendition is much more natural now!
Quote from: a1ex on October 24, 2013, 05:57:07 PM
Solved: cr2hdr-amaze-edge5.exe (http://acoutts.com/a1ex/cr2hdr-amaze-edge5.exe)
This crashes for me after the "Black borders" step, same with the current version from the repo commit 0327295...
Got it, 6D has thin black bars => division by 0.
Solved and reuploaded.
Quote from: a1ex on October 24, 2013, 08:53:47 PM
Got it, 6D has thin black bars => division by 0.
Meaning what (and why is it only on 6d?) - that the active size on dual_iso is less than in normal mode?
Quote from: a1ex on October 24, 2013, 08:53:47 PM
Solved and reuploaded.
Yes, the crash is gone, thanks for working for the 6d :-)
See your previous post (http://www.magiclantern.fm/forum/index.php?topic=7139.msg79700#msg79700). These bars are important for noise and black level calibration.
@alex: I have used the latest amaze_edge_5 version (downloaded a few minutes ago so it is 100% the latest):
https://www.dropbox.com/s/lqey54l0kfhsmdb/dual650_amaze-edge-5.JPG
@Ivycreek
I suppose you are still on PowerPC based Mac. Sorry, no support!
I already said in the readme: "Lion and Mountain Lion".
Quote from: a.d. on October 25, 2013, 12:30:00 AM
@Ivycreek
I suppose you are still on PowerPC based Mac. Sorry, no support!
I already said in the readme: "Lion and Mountain Lion".
LOL figures, I don't know much about Macs, it's the only place I have ever used one. This just gives me an excuse to work a little bit from home!
I bet all this can be re-compiled for powerpc.
Hi A.D. Did you update the cr2hr? Says there was something going on at bitbucket 8 hours ago but it,s still the 1.5 oct 11th version :)?
Thanks
//D
update:
Oh, I think it's a bug in bitbucket cache server setting. I already deleted the old file. Somehow it still provides the old version instead the new one! Therefore I ned to rename the file.
Million thanks A.D! and Alex of course...
Quote from: a1ex on October 16, 2013, 11:25:02 AM
From your shot, I've cleaned up some of the dots with this trick, but finding a general solution that doesn't break resolution charts is a bit harder. Also, locating hot pixels where the high ISO is overexposed is hard and I didn't try to solve it yet (a median filter with thresholding should help a bit).
I tried the "confetti" long exposure shot with the new interpolation method, and the dots are definitely better/smaller than before, but still there so I'd still consider this shot "broken" with the latest cr2hdr.
If this is indeed so hard to solve, could you give a guideline what to shots to *avoid* that result in these colorful dots so we can avoid doing dual_iso on these? The sample I provided was a 30sec daylight longtime exposure with a 10x nd filter.
So far I've only noticed these dots on long exposures. Not sure if ISO 100/400 avoids them, but you can try.
Another experiment: I've replaced the ISO estimation step with a simple histogram matching.
cr2hdr-histmatch.exe (http://acoutts.com/a1ex/cr2hdr-histmatch.exe)
Advantages:
- much faster (of course, only the ISO estimation step)
- no interpolation needed, so ISO is estimated from real pixels only
- got rid of nonlinear exposure corrections, which were probably just estimation errors from the previous method
- solves the helicopter shot
- a major step towards solving the fire shots from jOt.
There's still room for improvement in very dark shots, where the noise influences the histogram in significant ways. If you can find some theory about histogram matching for noisy images, it will be very helpful.
And, finally solved the 4 dark shots from jOt!
Now I'm confident the autodetection is really robust and should handle pretty much anything you throw at it. If it doesn't, upload a CR2.
cr2hdr-histmatch2.exe (http://acoutts.com/a1ex/cr2hdr-histmatch2.exe)
Hi @a1ex
So many updates, but i have a question, should i use just the last one (now the cr2hdr-histmach2) assuming this one have alll the abilities of the previous versions or all are diferente and for diferent problems ?
Thanks for the excelent work
Ideally, the latest should result in the best overall output. But since it's only half an hour old, you may want to compare it to previous versions, and if you find any regressions, report them (with a CR2 attached).
Hi Alex,
thanks for the new cr2hdr version.
I seem to have run into a different visual artefact, that consistently occurs no matter edge5 or histmatch2 exe:
(http://www.broome.de/pics/di_lines.jpg)
(image contrast enhanced for demonstration)
The artefacts occur on a 100/1600 file, a similar 100/800 file (out of a series 100/100 to 100/1600 for shadow/noise comparison) was ok.
Please check your mailbox for the corresponding CR2 file.
Hi Alex,
latest cr2hdr-histmatch2.exe gives me interpolation lines in dark areas, while prev cr2hdr-histmatch.exe did not.
I've altered luma curve to push the effect: histmatch | histmatch2
(http://i.fotto.ru/hexh.jpeg)
Alex, do you think you can make cr2hdr compress the dng like acr does (I don't know the algorithm though)?
I reckon that re-converting the dual_iso cr2 might become a regular habit, and it would be nice to be able to skip also re-compressing the ml dng in Lightroom - the acr version is about half the size of the ml one. It would also be nice for people who don't use acr at all and take the ml dng as it is.
@Bassaidai: should be OK now, only had to do some minor adjustments.
cr2hdr-histmatch3.exe (http://acoutts.com/a1ex/cr2hdr-histmatch3.exe)
@Yaros525: I can't diagnose it from a JPEG. Try with this one, and if you still have trouble, send me the CR2.
@Marsu42: a script that calls cr2hdr and then adobe DNG should do the trick. I'm not aware of any open source software that can do this.
some more tests:
both hismatch2 and edge5:
(http://www.broome.de/pics/di_artefact2.jpg)
(nasty overall garbage. 100/400)
Left hismatch2, right edge5:
(http://www.broome.de/pics/di_artefact1_hismatch2.jpg) (http://www.broome.de/pics/di_artefact1_edge5.jpg)
(ISO-line pattern, greenish color cast. 400/1600)
Both CR2s are provided, see PM.
Cheers
Bass
Alex,
cr2hdr-histmatch3.exe is Perfectly O.K. and all-around amazing! )
If there's a command line utility to compress dng my script would be well suited to do that after cr2hdr.
Adobe DNG Converter.exe -dng14 IMG_1234.DNG
a1ex,
I can't seem to get the program to work with my RAW videos. I drag the DNGs over the program, and it briefly flashes open for a second before closing (I hate that command line programs don't stay open to alert you of an error)
I have both dcraw and exiftool in the same folder, but it just doesn't want to work. I extract the DNG's using your raw2dng_cs3x3_ahdlike_noise.exe (because Java can't install on my system) so they should work together, correct?
Thanks for the help.
Quote from: a1ex on October 27, 2013, 12:14:31 AM
Adobe DNG Converter.exe -dng14 IMG_1234.DNG
Done.
http://www.magiclantern.fm/forum/index.php?topic=8520.0
Quote from: papkee on October 27, 2013, 03:01:14 AM
I have both dcraw and exiftool in the same folder, but it just doesn't want to work. I extract the DNG's using your raw2dng_cs3x3_ahdlike_noise.exe (because Java can't install on my system) so they should work together, correct?
I didn't try both of them, but I don't think it can work in this way. Try with the plain raw2dng.
Hi, A1ex,
I may found a bug:
new cr2hdr-histmatch3.exe is creating DNGs from non DualISO files -
please, see text output in comparison with cr2hdr-amaze-edge5.exe result:
C:\dualISO>cr2hdr-amaze-edge5.exe 2013-10-27_3626.CR2
cr2hdr: a post processing tool for Dual ISO images
Last update: af9a60f on 2013-10-24 18:44:01 UTC by alex:
cr2hdr: always use last 8 lines from top optical black bar (should fi...
Input file : 2013-10-27_3626.CR2
Canon EOS 50D detected
Full size : 4832 x 3228
Active area : 4770 x 3177
White level : 12500
Black borders : 62 left, 51 top
Black level : 994
ISO pattern : BdBd GBRG
Interlacing method not supported
ISO blending didn't work
C:\dualISO>cr2hdr-histmatch3.exe 2013-10-27_3626.CR2
cr2hdr: a post processing tool for Dual ISO images
Last update: 6982824 on 2013-10-26 16:54:22 UTC by alex:
cr2hdr: fine-tuned horizontal stripe correction
Input file : 2013-10-27_3626.CR2
Canon EOS 50D detected
Full size : 4832 x 3228
Active area : 4770 x 3177
White level : 12500
Black borders : 62 left, 51 top
Black level : 994
ISO pattern : dBBd GBRG
Noise levels : 5.24 5.24 5.24 5.31 (14-bit)
Doesn't look like interlaced ISO
ISO difference : 0.00 EV (100)
Black delta : 1.01
Dynamic range : 11.10 (+) 11.08 => 11.08 EV (in theory)
Interpolation : amaze-edge-chroma2x2-alias
AMaZE interpolation ...
Amaze took 3.57 s
Edge-directed interpolation...
Semi-overexpo'd: 0.00%
Deep shadows : 2.83%
Horizontal stripe fix...
Looking for hot/cold pixels...
Cold pixels : 359
Full-res reconstruction...
ISO overlap : 8.1 EV (approx)
Half-res blending...
Chroma filtering...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 21.28 (16-bit), ideally 18.14
Dynamic range : 11.08 EV (cooked)
Black adjust : -7
Output file : 2013-10-27_3626.DNG
1 image files updated
It (cr2hdr-histmatch3) however detects "Doesn't look like interlaced ISO",
but dng is created anyway. It happened with a few of my cr2 files from today´s shooting.
orim
Quote from: a1ex on October 27, 2013, 02:05:01 PM
I didn't try both of them, but I don't think it can work in this way. Try with the plain raw2dng.
Actually, it DOES work, but only by using the command line. The problem appears to be with dragging and dropping files onto cr2hdr.exe. Whenever I try with DNGs, either from raw2dng or your special one for the 650d it just quickly closes. Any thoughts? I don't want to have to command line convert every one of my 433 frame individually.
What happens if you drag a small number of files? There may be a limit on command-line length.
QuoteI don't want to have to command line convert every one of my 433 frame individually.
http://www.magiclantern.fm/forum/index.php?topic=8520.0
QuoteI don't want to have to command line convert every one of my 433 frame individually.
You can also drag and drop the CR2 files by packs of tens or twentys. This will fire up multiple instances of the converter and hence drastically speed up the overall conversion progress (on multicore PCs at least...)
I'm experimenting with two fast versions of cr2hdr:
cr2hdr-fast-halfres.exe (http://acoutts.com/a1ex/cr2hdr-fast-halfres.exe): roughly 5x as fast, but it skips all the expensive postprocessing steps used to recover details in highlights and shadows. You will get lots of aliasing with it.
cr2hdr-fast-fullres.exe (http://acoutts.com/a1ex/cr2hdr-fast-fullres.exe): roughly 3x as fast; this one has full-resolution detail in midtones and shadows, but also lots of noise. It doesn't seem to recover more than 11 stops of DR.
Neither of these two versions are meant for the final output, but just for getting some quick previews. May be helpful if you shoot say 2000 pictures from which you want to select 50 good ones. They may also be useful for troubleshooting (because the first one minimizes noise and the second one minimizes aliasing).
Both of them are bad in highlights (of course, when compared with edge-directed interpolation).
If a picture comes out clean with the fast-fullres version, it probably doesn't need dual ISO.
Quote from: a1ex on October 28, 2013, 08:38:39 AM
What happens if you drag a small number of files? There may be a limit on command-line length.
Even dragging one .dng doesn't work. The program flashes for a second before closing, and the file is unchanged. Could it be something with the way windows 8 interacts with command line programs? I read somewhere that sometimes Windows puts "" around the file path.
Quote from: papkee on October 28, 2013, 04:04:30 PM
Even dragging one .dng doesn't work. The program flashes for a second before closing, and the file is unchanged. Could it be something with the way windows 8 interacts with command line programs? I read somewhere that sometimes Windows puts "" around the file path.
for windows: install total comander, open folder witch cr2hdr, press ctrl+enter on cr2hd.exe, press crrl+enter on Raw image, press enter.
Hello everyone. I've been following this particular forum thread quite a long time now but I have never read anything related to the usage of ML on a Canon 40d, which is the camera I currently use. I had a canon 600d but it was stolen :( , so now I only have my 4 years old 40d which is a great camera for stills, has a nice build quiality, fair resolution, fast, etc... I would be pleased to hear that is possible to install ML on a 40d, I would love to use the Dual Iso feature specially because of the poor dynamic range of the 40d that has the same 11 stops of DR of the 600d. thank you very much.
Andrés Garciá Hurtado.
Please select the proper forum next time ...
And use the search function ...
http://www.magiclantern.fm/forum/index.php?topic=1452.msg78621#msg78621
And read F.A.Q.
Quote from: a1ex on October 28, 2013, 03:39:47 PM
If a picture comes out clean with the fast-fullres version, it probably doesn't need dual ISO.
A human readable clear text message (maybe in %) from cr2hdr would be nice to let the user know how much a shot profited from being dual_iso... this could also be used as an error code to automatically trash dual_iso shots that were also taken in normal mode. Just a suggestion and if it's very easy to add of course.
Good idea, will think about it. Not sure how to quantify this though, since "clean" is subjective (so this is a test where you can judge with your own eyes).
Quote from: a1ex on October 28, 2013, 07:36:48 PM
Good idea, will think about it. Not sure how to quantify this though, since "clean" is subjective (so this is a test where you can judge with your own eyes).
Indeed, if I'd have a way to quantify this I'd have written it :-) ... when in doubt use your judgement, you have looked at more dual_iso shots than anybody else. One certain point for the dual_iso shot though is if there are not only cleaner shadows above a certain ev threshold, but blown highlights in one exposure that can be reconstructed with the other (i.e. not both slices are underexposed).
It already displays the percentage of reconstructed highlights and deep shadows (these are the areas where it applies the advanced processing). I can't say yet clearly what values are good and what values are bad though.
If the highlight percentage is near zero, the shot was probably underexposed (so consider it shooting at high ISO only). If it's close to 100%, consider shooting it at the low ISO only.
If you enable the intermediate DNGs, you can see the two exposures individually (bright.dng and dark.dng), so you can see exactly where you had recovered highlights and where you had noisy shadows. Maybe I should add a command-line flag for that.
Quote from: a1ex on October 28, 2013, 09:02:10 PMIf you enable the intermediate DNGs, you can see the two exposures individually (bright.dng and dark.dng), so you can see exactly where you had recovered highlights and where you had noisy shadows. Maybe I should add a command-line flag for that.
I'm all in favor for some command line options rather than patching & recompiling the source, getting both exposures apart to see what was what would certainly help to understand what dual_iso can do and what not, i.e. how to expose correctly in the first place.
I think command line flags would make customization and testing a lot easier. Perhaps you could add a destination directory option too.
"Cr2hdr.exe /?" or "Cr2hdr.exe / help" option does not work.
want to know all the parameters ( Cr2hdr.exe /... ) with which you can run cr2hdr.
Here you go:
cr2hdr-cmd.exe (http://acoutts.com/a1ex/cr2hdr-cmd.exe)
Command-line usage: cr2hdr [OPTIONS] [FILES]
Shortcuts:
--fast : disable most postprocessing steps (fast, but low quality)
(--mean23, --no-cs, --no-fullres, --no-alias-map, --no-stripe-fix)
Interpolation methods:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--mean23 : average the nearest 2 or 3 pixels of the same color from the Bayer grid (faster)
Chroma smoothing:
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--cs3x3 : apply 3x3 chroma smoothing in noisy and aliased areas
--cs5x5 : apply 5x5 chroma smoothing in noisy and aliased areas
--no-cs : disable chroma smoothing
Bad pixel handling:
--really-bad-pix: aggressive bad pixel fix, at the expense of detail and aliasing
--no-bad-pix : disable bad pixel fixing (try it if you shoot stars)
--black-bad-pix : mark all bad pixels as black (for troubleshooting)
Other postprocessing steps:
--no-fullres : disable full-resolution blending
--no-alias-map : disable alias map, used to fix aliasing in deep shadows
--no-stripe-fix : disable horizontal stripe fix
Troubleshooting options:
--debug-blend : save intermediate images used for blending:
dark.dng the low-ISO exposure, interpolated
bright.dng the high-ISO exposure, interpolated and darkened
halfres.dng half-resolution blending (low noise, high aliasing)
fullres.dng full-resolution blending (minimal aliasing, high noise)
*_smooth.dng images after chroma smoothing
--debug-black : save intermediate images used for black level subtraction
--debug-amaze : save AMaZE input and output
--debug-edge : save debug info from edge-directed interpolation
--debug-alias : save debug info about the alias map
--iso-curve : plot the curve fitting results for ISO and black offset (requires octave)
--mix-curve : plot the curve used for half-res blending (requires octave)
--fullres-curve : plot the curve used for full-res blending (requires octave)
I didn't play much with it, so there may be surprises. Most of these things were already available as compile-time options (except for bad pixel handling, where I've fine-tuned the algorithms and I've added a new option).
Quote from: a1ex on October 29, 2013, 04:30:18 PM
Here you go
Thanks! ... but what would "octave" be :-o ? Also, I guess a higher cs setting is there for reducing color bleeding on edges, or is there anything else it's good for?
Quote from: Marsu42 on October 29, 2013, 04:36:06 PM
Thanks! ... but what would "octave" be :-o ? Also, I guess a higher cs setting is there for reducing color bleeding on edges, or is there anything else it's good for?
Octave is presumably this: http://www.gnu.org/software/octave/
Quote from: chris_overseas on October 29, 2013, 05:10:25 PM
Octave is presumably this: http://www.gnu.org/software/octave/
Oh my, thanks - I'm embarrassed to say this even shows up as the #1 google result, I just didn't expect it to be something like this :-o
I'm still periodically building win32 and linux 32bit binaries for cr2hdr, available here:
https://bitbucket.org/rufustfirefly/magic-lantern/downloads (https://bitbucket.org/rufustfirefly/magic-lantern/downloads)
in case anyone is interested. I try to build a new version whenever I see a Mercurial check-in related to the cr2hdr code, usually once a day, at most.
make a separate thread for cr2hdr in bitbucket.org
that it was possible to see the changes by version-)
Thank you very much for the work done.
https://bitbucket.org/hudson/magic-lantern/commits/all?search=cr2hdr
I'm experimenting with some noise reduction, with an algorithm along these lines:
http://www.computersdontsee.net/index.php/post/2013/02/09/Introducing-NL-means
http://bengal.missouri.edu/~kes25c/nl2.pdf
http://scholar.harvard.edu/files/stanleychan/files/chan_zickler_lu_2013_0.pdf
http://www.cs.tut.fi/~foi/3D-DFT/ - watch the animation
(http://acoutts.com/a1ex/denoice/7O4C8804-orig.jpg) (http://acoutts.com/a1ex/denoice/6X8A0381-3200-1700_before.jpg) (http://acoutts.com/a1ex/denoice/IMG_5762-1600-2100_before.jpg)
(http://acoutts.com/a1ex/denoice/7O4C8804-denoised.jpg) (http://acoutts.com/a1ex/denoice/6X8A0381-3200-1700_after.jpg) (http://acoutts.com/a1ex/denoice/IMG_5762-1600-2100_after.jpg)
According to the logs, this squeezes roughly one more stop of dynamic range, and it seems to whiten the noise too.
Download: cr2hdr-denoice.exe (http://acoutts.com/a1ex/denoice/cr2hdr-denoice.exe)
The denoising step is not enabled by default; you need to specify --denoice in the command line.
Now the big question: how does this compare with state-of-art denoising software? (Lightroom, Neat Image, Noise Ninja, whatever you use). I don't have license for any of these, so I don't know if I'm reinventing the wheel or not.
For comparison, I suggest processing the CR2 with default options, then with --denoice, then postprocessing both DNGs in your favorite editor and denoise them so they have similar noise levels, and see which one holds more detail. Your quest is to find out whether my denoising step helps or hurts, and how much.
Original CR2 files for these examples: 7O4C8804.CR2 (http://acoutts.com/a1ex/denoice/7O4C8804.CR2), 6X8A0381.CR2 (http://acoutts.com/a1ex/denoice/6X8A0381.CR2) and IMG_5762.CR2 (http://acoutts.com/a1ex/denoice/IMG_5762.CR2) (credits: AdamTheBoy, daancalo2013 and Danne). To get the DNGs from the top row, process them with default settingss, and to get the DNGs from the bottom row, process them with --denoice.
Please note that each test run will create a different DNG (the denoising algorithm uses monte carlo sampling). It may be interesting to see what happens if you average the output from a few test runs (didn't try). Also it may be worth trying --denoice --cs5x5.
jpeg (-x) + jpeg (0) + jpeg (+ x) -> hdr, where x-expo
cr2 (iso100/1600) -> dng
whether it is possible to do so in cr2hdr: cr2 (iso100/1600) -> jpeg (-x) +{ jpeg (0) +} jpeg (+ x)?
2'Alex
Quote from: a1ex on October 30, 2013, 09:51:23 PM
I'm experimenting with some noise reduction, with an algorithm along these lines...
(https://dl.dropboxusercontent.com/u/13183292/dualISO/2013/10/31/error.JPG)
uses windowsXP 32
does not work works
First impression.
Does a better job of maintaining detail in textures (probably due to less aggressive denoise), but muddles the noise in flat textures.
LR - Luma 50 - Chroma 25
(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/test.jpg)
Default denoise cr2hdr
(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_46A2224-3.jpg)
edit: No noise reduction
(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/Nonoise.jpg)
@Legushka: can you upload the CR2? I have no errors or warnings in valgrind.
The CR2 and processed JPG from LR if you want target practice.
https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/_46A2224.CR2
https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/Lightroom.jpg
edit: Good test shot for the severe color artifact on edges also.
I remember using NL means way back in '06 on video. IIRC, it did a good job in the temporal domain, of course back then it was dog slow.
Yes, the algorithm is old and simple, but very slow. The papers talk about computing each pixel as a linear combination of *all* other image pixels (which is really crazy in practice, but useful to present the mathematical foundation behind it, theoretical limits and other stuff like that).
Here's a PhD thesis on this subject: http://www.mi.parisdescartes.fr/~buades/publicacions/tesi.pdf
(see chapter 5 for pixel peeping)
Don,t know how to change command line but made denoising, color fix and some sharpening in lightroom of my old original you uploaded :) Looks like this
Lightroom denoised
(http://s22.postimg.org/ncjhlllzh/image.jpg) (http://postimg.org/image/ncjhlllzh/)
(http://s18.postimg.org/cfs9wsio5/image.png) (http://postimg.org/image/cfs9wsio5/)
Original
(http://s22.postimg.org/f8bdguzkd/image.jpg) (http://postimg.org/image/f8bdguzkd/)
(http://s18.postimg.org/3m1dfovph/image.png) (http://postimg.org/image/3m1dfovph/)
Denoiced monte carlo style, 2 alternatives
(http://s22.postimg.org/9h55wkn7x/IMG_5762_0_1b.jpg) (http://postimg.org/image/9h55wkn7x/)
(http://s22.postimg.org/a7xw2cpl9/IMG_5762_0_2b.jpg) (http://postimg.org/image/a7xw2cpl9/)
(http://s22.postimg.org/7bav22jrh/image.png) (http://postimg.org/image/7bav22jrh/)
In a terminal, type this:
cr2hdr-denoice --denoice IMG_1234.CR2
or, on Mac/Linux:
wine cr2hdr-denoice.exe --denoice IMG_1234.CR2
and you get a denoised DNG.
Here's a tough situation that Dual Iso handled pretty exquisitely.
(http://s14.postimg.org/b1k6z3q0t/Screen_Shot_2013_10_30_at_6_34_59_PM.jpg) (http://postimg.org/image/b1k6z3q0t/)
Updated my comparisons with two examples of denoiced pictures. I say lightroom is a little bit better here, what do you think?
see Reply #1530
One has chroma noise and the other doesn't. Can you adjust them so they have similar noise levels?
Like this? It actually cleans up nicely with denoiced cr2hdr. Could maybe work
denoiced
(http://s16.postimg.org/5t93mra8x/IMG_5762_0_2b.jpg) (http://postimg.org/image/5t93mra8x/)
not denoiced
(http://s16.postimg.org/rgy1x7an5/IMG_5762_01.jpg) (http://postimg.org/image/rgy1x7an5/)
settings for both
(http://s16.postimg.org/iw4q610gx/Ska_rmavbild_2013_10_31_kl_00_36_46.png) (http://postimg.org/image/iw4q610gx/)
Sort of. Chroma noise seems similar now, but luma noise is much higher in the second one.
If you can measure the standard deviation of the noise, for example in the triangle between the table top and legs, it may help. To compare them, I think you need to have equal noise levels in both images, so you can judge texture details. Or, have the same amount of details, but that's difficult to quantify. If you have both of them different, it's difficult to draw a conclusion.
Trying this
Denoiced
(http://s23.postimg.org/bkw4h8n8n/IMG_5762_0_2b.jpg) (http://postimg.org/image/bkw4h8n8n/)
Not denoised
(http://s23.postimg.org/vtji2ymjr/IMG_5762_01.jpg) (http://postimg.org/image/vtji2ymjr/)
Here I tried to get them look similar by sharpen the denoiced to look the same as the other
denoiced
(http://s24.postimg.org/lybeni0z5/IMG_5762_0_2b_2.jpg) (http://postimg.org/image/lybeni0z5/)
No denoice
(http://s24.postimg.org/cp98dds35/IMG_5762_01_2.jpg) (http://postimg.org/image/cp98dds35/)
If you have PS, make a selection of a flat texture and expand the histogram to advanced. This will show the std deviation.
I'll post some more examples later tonight.
Quote from: a1ex on October 30, 2013, 09:51:23 PMNow the big question: how does this compare with state-of-art denoising software? (Lightroom, Neat Image, Noise Ninja, whatever you use). I don't have license for any of these, so I don't know if I'm reinventing the wheel or not.
Btw: For the best current nr algorithm try DxO Pro Optic's v9's PRIME noise reduction - slow as hell, but for noisy images at high iso one step in front of Adobe ACR. The advantage of other 3rd party nr plugins like Noise Ninja afaik is not that they are overall better at nr, but just that they are"smarter" and try to figure out which parts need more nr and which don't.
From their description (http://www.dxo.com/intl/photography/dxo-optics-pro/features/denoising), it sounds very similar to my approach. So yeah, I'm reinveting the wheel.
4 stops sounds impressive; for that, for every output pixel they need to average 256 other pixels if they don't do any weighting, or more if they use weights for averaging.
Quote from: a1ex on October 31, 2013, 06:10:31 PM
From their description (http://www.dxo.com/intl/photography/dxo-optics-pro/features/denoising), it sounds very similar to my approach. So yeah, I'm reinveting the wheel.
But as oss :-) ... here's the relevant thread from CR with some samples: http://www.canonrumors.com/forum/index.php?topic=17701.msg328534#msg328534
Beware: DxO's PRIME only works with raw cr2 pictures, so if anyone wants to use it don't concert your cr2 to raw dng...
Quote from: a1ex on October 31, 2013, 06:10:31 PM
4 stops sounds impressive; for that, for every output pixel they need to average 256 other pixels if they don't do any weighting, or more if they use weights for averaging.
This is probably the reason why their algorithm takes ages to process the picture - but it really doesn't matter because for your very best shots you'd be happy to wait (or process overnight) as long as the result is an improvement.
First off: All contributors to magiclantern thanks alot.
Question:
Is there a way to use cr2hdr-histmatch3 in combination with raw video?
If I use raw2dng It already process and interpolate the dual iso raw file.
Thanks in advance
Quote from: a1ex on October 31, 2013, 06:10:31 PM
From their description (http://www.dxo.com/intl/photography/dxo-optics-pro/features/denoising), it sounds very similar to my approach. So yeah, I'm reinveting the wheel.
Looks like a lot of marketing BS to me.
For instance, they talk about ISO 51,200 but show an example at ISO 25,600. Also, the example image seems to contain mostly color noise only. Color noise is easy to deal with, it's the luma noise that creates the problems (if you respect fine details), and in my tests, ISO 25600 creates a ton of luma noise.
As for reinventing the wheel. Well your solution is free.
after cr2 -> DNG lost the white balance settings in the DNG
Quote from: Legushka on November 01, 2013, 10:23:40 AM
after cr2 -> DNG lost the white balance settings in the DNG
Known problem, see some posts above - currently you have to manually correct the wb in dual_iso files, at least that's what I (have to) do with ACR.
Edit: Of course this can be tricky - either shoot a non dual_iso frame to get the wb or use a gray or combined gray/color card next to tehe subject for calibration, the latter is a good idea anyway in difficult lighting.
Speaking of which: Is it necessary to shoot a frame to get the awb -
or can/could ml get the current awb from live view (feature request) and display it in the info bar or somewhere else?
again, please (post #1499):
Am I really the only one, who gets DNG files from not dualISO CR2 files?
Didn't think I should make a bug report out of this as it is not directly ML related. Does anyone else have issues with processing too many photos with cr2-hdr-histmatch3? My limit is 30. Last page someone mentioned having issues, but I wasn't sure if it was with this. Also quick question, does this version have denoice (denoising)? Maybe this is the wrong thread, if so point me to the right one.
PS. It would be nice if on the front page all of the awesome new versions were being put up.
Thanks,
Painya
I'll try to update it these days. Sorry, I'll be busy with non-ML stuff till the end of the year.
Quote from: a1ex on November 03, 2013, 07:16:24 AM
I'll try to update it these days. Sorry, I'll be busy with non-ML stuff till the end of the year.
No need to apologize! You are already doing something incredible, and changing the world of photography forever. Thank you.
Quote from: a1ex on November 03, 2013, 07:16:24 AM
Sorry, I'll be busy with non-ML stuff till the end of the year.
Oh my, then let me take the opportunity to thank you again for all your terrific work, it's incredible how you were available nearly 24/7 during the last time - if your near absence will show one thing it's probably how important you are for ML and how helpful you are for all other devs ... so don't get hit by a bus when crossing the street, will you :-p ?
Thanks Alex
The good news is that I expect to make heavy use of ML and dual ISO, so I'll try to share some pics.
Quote from: a1ex on November 03, 2013, 11:19:30 AM
The good news is that I expect to make heavy use of ML and dual ISO, so I'll try to share some pics.
Great, and you using ML not only for test doorknob shots means that you have a good incentive to keep improving cr2hdr :-> and maybe even come up with a fix to set auto white balance and eliminate all confetti pixels from longer time exposures :-) ...
... but you found a good time for a vacation, dual_iso is working fine just like it is now, save some usability improvements in the front-end (namely auto-disable dual_iso if ev gain is too low). Btw be careful to use flash+ettl metering with dual_iso, next to needing massive -fec this seems to be somewhat erratic, but I haven't pinpointed the problem enough to submit a detailed report/proposal yet - I'll wait until 2014 then :-)
a1ex, spasibo!
Hi Guys!
I'm following the progress here and I think it all very exciting!
I'm producing full_raw dual iso with 600D now [for professional photography], and am finding amazing what gives me in dynamic range. I would just like to ask about the exif, if there is any method that even when unaware of this, because I would like to use dcp VisionLog curves for the grade in Lightroom, however he does not identify.
I'm currently using RawTherapee for the entire workflow, can I apply the curve dcp without exif info's, however, would be of great value can make direct changes in Lightroom to process less time-consuming and less tiring.
I thank you and I can help with respect to the development, testing and ideas, I am available (although I'm not a programmer). Hug!
Quote from: a1ex on November 03, 2013, 11:19:30 AM
The good news is that I expect to make heavy use of ML and dual ISO, so I'll try to share some pics.
Have FUN! :)
Just a quick question please: The cr2_hdr.exe in the first page is Oct 15. There is the denoise edition and the amazing edge edition. Is there a latest version that combines all? (with the mentioned switches).
In
https://bitbucket.org/a_d_/magic-lantern/downloads
the date is Oct 28. But change log in nightly builds suggests ... Nov 4th.
Of course I can wait for periodic updates, just asking for the place to look for the very latest cr2hdr.exe builds...
Thanks
Here's a version that should be pretty good for everyday use.
cr2hdr-hot.exe (http://acoutts.com/a1ex/cr2hdr-hot.exe)
What's new:
- better hot pixel handling (solves the shot from Marsu42 with --really-bad-pix, also fireworks are no longer mistaken for bad pixels)
- fixed a bug that caused massive shadow aliasing in the version with many command-line switches
- minor fixes
What's not:
- I've removed the denoising option for now, because DxO puts it to shame. I'll add it back when I'll get comparable results in terms of recovered detail, without the plastic look.
Have a quick play with it; if there are no major bugs (like stopping after 30 pictures, crashing, or regressions in image quality) I'll update the first post tonight. I've tested it on 700 pics in a single batch with --fast, 100 pics with default settings, and 2 pictures in valgrind, so I believe it should be pretty solid now.
Update: after running my usual tests, I've noticed the new hot pixel algorithm (in aggressive mode) has little or no effect on resolution charts. Another milestone completed!
Alex eres lo maximo, muy bien
Quote from: a1ex on October 30, 2013, 09:51:23 PM
I'm experimenting with some noise reduction, with an algorithm along these lines:
http://www.computersdontsee.net/index.php/post/2013/02/09/Introducing-NL-means
http://bengal.missouri.edu/~kes25c/nl2.pdf
http://scholar.harvard.edu/files/stanleychan/files/chan_zickler_lu_2013_0.pdf
http://www.cs.tut.fi/~foi/3D-DFT/ - watch the animation
(http://acoutts.com/a1ex/denoice/7O4C8804-orig.jpg) (http://acoutts.com/a1ex/denoice/6X8A0381-3200-1700_before.jpg) (http://acoutts.com/a1ex/denoice/IMG_5762-1600-2100_before.jpg)
(http://acoutts.com/a1ex/denoice/7O4C8804-denoised.jpg) (http://acoutts.com/a1ex/denoice/6X8A0381-3200-1700_after.jpg) (http://acoutts.com/a1ex/denoice/IMG_5762-1600-2100_after.jpg)
According to the logs, this squeezes roughly one more stop of dynamic range, and it seems to whiten the noise too.
Download: cr2hdr-denoice.exe (http://acoutts.com/a1ex/denoice/cr2hdr-denoice.exe)
The denoising step is not enabled by default; you need to specify --denoice in the command line.
Now the big question: how does this compare with state-of-art denoising software? (Lightroom, Neat Image, Noise Ninja, whatever you use). I don't have license for any of these, so I don't know if I'm reinventing the wheel or not.
For comparison, I suggest processing the CR2 with default options, then with --denoice, then postprocessing both DNGs in your favorite editor and denoise them so they have similar noise levels, and see which one holds more detail. Your quest is to find out whether my denoising step helps or hurts, and how much.
Original CR2 files for these examples: 7O4C8804.CR2 (http://acoutts.com/a1ex/denoice/7O4C8804.CR2), 6X8A0381.CR2 (http://acoutts.com/a1ex/denoice/6X8A0381.CR2) and IMG_5762.CR2 (http://acoutts.com/a1ex/denoice/IMG_5762.CR2) (credits: AdamTheBoy, daancalo2013 and Danne). To get the DNGs from the top row, process them with default settingss, and to get the DNGs from the bottom row, process them with --denoice.
Please note that each test run will create a different DNG (the denoising algorithm uses monte carlo sampling). It may be interesting to see what happens if you average the output from a few test runs (didn't try). Also it may be worth trying --denoice --cs5x5.
@alex: Don't be hard on yourself regarding denoise. You are doing SUPERB WORK!
Plus, I read that DXO works on .CR2 files not DNG. So if that is the case denoise option is helpful ANYWAY!
NOW, imagine if only cr2hdr could produce a .CR2 as output! That could be processed with DXO PRIME.
OK that is a crazy scenario ;D
Quote from: tron on November 05, 2013, 03:56:53 AM
NOW, imagine if only cr2hdr could produce a .CR2 as output! That could be processed with DXO PRIME.
OK that is a crazy scenario ;D
Why would a .CR2 output be preferable over the .DNG?
Quote from: a1ex on November 04, 2013, 12:00:55 PM
What's not:
- I've removed the denoising option for now, because DxO puts it to shame. ...
Quote from: tron on November 05, 2013, 03:56:53 AM
@alex: Don't be hard on yourself regarding denoise. You are doing SUPERB WORK!
Plus, I read that DXO works on .CR2 files not DNG. So if that is the case denoise option is helpful ANYWAY!
NOW, imagine if only cr2hdr could produce a .CR2 as output! That could be processed with DXO PRIME.
OK that is a crazy scenario ;D
Wondering the same... I just downloade a trial version of DxO Optics 9 and it can't read Dual-ISO or RAW DNGs.
Are you doing the denoising on tiff/jpg in DxO?
I tried on a CR2, but I'm sure you can find workarounds to open a DNG. IIRC, DXO opens the DNGs from cameras that use this format natively, so you probably need to fake the EXIF somehow.
To mangle the data from a CR2, a clumsy hack would be to modify the code from dual ISO preview (since it can burn the preview in the CR2 if you don't call the undo hook). It's nothing more than a proof of concept, not a usable solution. But it proves that it can be done.
Quote from: a1ex on November 04, 2013, 12:00:55 PM
- I've removed the denoising option for now, because DxO puts it to shame. I'll add it back when I'll get comparable results in terms of recovered detail, without the plastic look.
Indeed, it wasn't even hard before your attempt to get a shot with zero noise but plastic look :-> ... my guess is that the DxO algorithm is rather clever and tuned/tested on a lot of sample shots by a horde of beta testers - to me this seems to be beyond simple denoising and what you can achieve with one "catch all" cr2hdr algorithm... but fee free to prove me wrong :-)
Quote from: a1ex on November 05, 2013, 09:31:18 AM
I tried on a CR2, but I'm sure you can find workarounds to open a DNG. IIRC, DXO opens the DNGs from cameras that use this format natively, so you probably need to fake the EXIF somehow.
I'll include you in my will if you manage to make DxO read raw dng or write a dng->cr2 converter (no matter the wb or metadata, just the mosaic pixels) ... I've converted all my cr2 to dng for various reasons and now cannot use the new PRIME denoising on my older shots.
Here are the duds from 7D:
http://www.filedropper.com/mg4149
http://www.filedropper.com/ual4148
I used AutoETTR + Dual + auto expo, dunno if turning on auto expo causes this.
Also sometimes the preview is jacked like on 6D:
http://www.filedropper.com/ual3926
I shot single shooting always.
Did you make any more changes to get the FS preview/etc that aren't in that repo?
Quote from: painya on November 05, 2013, 04:32:40 AM
Why would a .CR2 output be preferable over the .DNG?
Because DXO PRIME does not work with .DNG.
So imagine a DUAL ISO converted file to have DXO PRIME noise reduction applied afterwards!
Hello everybody!
I'm a proud owner of a 5D3 and I use everyday ML with the latest nightly build, with many many satisfaction! Specially the dual iso feature, is AWESOME!!!
I have a question: when the changelog of a nightly says a change in cr2hdr (as the last 8813: cr2hdr: moved hot pixel correction before interpolation and fine-tuned the algorithm), where exactly is possible to download it from?
Maybe mine is a stupid question, but I follow this thread and I never find a corresponding link to the updated version (as mentioned in the changelog), and sometimes I try to download the cr2hdr.zip (from http://acoutts.com/a1ex/cr2hdr.zip), but it is the same from october 15...
Thanks guys!
Lorenzo
There is a newer download link from alex in:
http://www.magiclantern.fm/forum/index.php?topic=7139.msg86767#msg86767
There is also this
https://bitbucket.org/a_d_/magic-lantern/downloads/WIN_cr2hdr_exe.zip
(It contains just the cr2hdr.exe (28/10) so you have to use the other supporting .exes from a previous download like Oct 15)
Quote from: tron on November 08, 2013, 02:42:27 PM
There is also this
https://bitbucket.org/a_d_/magic-lantern/downloads/WIN_cr2hdr_exe.zip
(It contains just the cr2hdr.exe (28/10) so you have to use the other supporting .exes from a previous download like Oct 15)
Thank you, but the change
8813: cr2hdr: moved hot pixel correction before interpolation and fine-tuned the algorithm is dated 5th of november! I'm just wondering if it means that there is a newer cr2hdr version somewere...
Quote from: lureb74 on November 08, 2013, 03:32:09 PM
Thank you, but the change 8813: cr2hdr: moved hot pixel correction before interpolation and fine-tuned the algorithm is dated 5th of november! I'm just wondering if it means that there is a newer cr2hdr version somewere...
That's just a change to the source code, it doesn't necessarily mean a new compiled binary has been published. You can download the source code and compile it yourself if you want, or you can wait until the various folks that publish binaries update them.
@alex: your denoising work may prove even more useful than you think even if you reinvent the wheel: Apart from the fact that it removes noise from .DNG files it runs in Windows XP. I tried to setup a DXO Pro 9 demo and I got the message that Windows XP are not supported >:(
Have you tried just copying all the tags and color profile with exiftool?
Hi there everyone!
I have a thought/request/idea.. sometimes when out shooting I'm shooting without dual ISO and then I might get into a situation where I need it "quickly" but going into the menu to turn it on often times is not quick enough, just had a thought that it maight be possible to set for example the info button as a shortcut key for turning dual ISO on or off when not shooting in live view. Don't really know if this is the right place to put this, but just thought it would be very handy. And I have no knowledge whatsoever in coding etc. so I don't know if it's an easy thing to acomplish or not.
Keep the good work up! :)
Don't know if this is the right thread where post it...
Shooted from a dark side of a room to outside to the trees and sky with same expo and iso values.
EF 28-135mm IS @ 100mm f/5.6 1/50sec on a 5D III with nov 04 ML nighty build.
5x crop, 2230 x 950px RAW video @ 25 fps on a Lexar 1000x 32gb CF card, Dual ISO (100-1600) processed with cr2hdr "denoice" version. Graded all frames at same settings, stabilized in post and resized to full HD.
As the 7th frame 1/2stop overexposed (corrected in post), I don't know if it flickers because of a raw video recording error or a cr2hdr conversion error.
What do you think about the result?
Nice one. I tried in 5x the other day and I must say that the cr2hdr really kicks ass now. Alex work on this is remarkable. Fully usable with very minor cases of aliasing and artefacts.
About flickering. Could it be postproduction in acr causing this?
Think it,s time to start a thread with dual iso movies :)
Quote from: lureb74 on November 10, 2013, 04:33:08 PM
processed with cr2hdr "denoice" version.
really great result, did you download cr2hdr "denoice" from the first page of this thread?
Quote from: Danne on November 10, 2013, 04:44:28 PM
About flickering. Could it be postproduction in acr causing this?
No, some frames (as always the 7th frame) are REALLY overexposed! Used ACR to correct them, but still flickering is visible somewhere ...
Quote from: kgv5 on November 10, 2013, 05:44:05 PM
really great result, did you download cr2hdr "denoice" from the first page of this thread?
Agree! Exposed at 1600 ISO with AETTR (without dual iso link), dual iso set on -4 value (1600-100 iso)with the intent to recover as highlights as possible... and it did it well I think ;-)
Downloaded the A1ex's http://acoutts.com/a1ex/denoice/cr2hdr-denoice.exe
from his Reply #1523 on: October 30, 2013, 09:51:23 PM
http://www.magiclantern.fm/forum/index.php?topic=7139.msg85889#msg85889
Pay attention, the denoice funcion is available but not abilitated by default! I'm not a coder, so I created a shortcut of "cr2hdr-denoice.exe", then right click and in target link properties i added "--denoice" for activate it (must appear like "x:\...path...\cr2hdr-denoice.exe --denoice"). So I can process many files together!
I wonder why is not possible to move more than x files over the cr2hdr-denoice.exe for processing (where x > 495) or you get a windows "The filename or extension is too long" error... someone have a solution?
Is denoice really needed though? I tried converting with a.d; s converter version 1.6 worked good. Got some border lineproblems in the latest 1.8 version when converting 3x mode files. Im on a mac
Quote from: Danne on November 10, 2013, 06:34:56 PM
Is denoice really needed though? I tried converting with a.d; s converter version 1.6 worked good. Got some border lineproblems in the latest 1.8 version when converting 3x mode files. Im on a mac
Don't know what is "a.d; s converter version 1.6 or 1.8", think you mean some app for mac. I'm on windows...
Simply, with denoiced DNGs you have less noise on dark areas and less aa artifacts on hi contrast areas.
Quote from: Danne on November 10, 2013, 06:34:56 PM
Is denoice really needed though?
Only if the end result is too noisy without it. I have a bunch of really underexposed shots from a timelapse, because I've misconfigured ETTR (I've disabled highlight ignore completely while trying to shoot a moonrise).
A1ex, I still can't get cr2hdr to work with my .dng files. They convert fine if I execute them individually from the command line, but dragging them over the .exe doesn't work. It flashes onscreen for a second and then quits without changing the file.
I have a ton of Dual ISO clips from a fire juggling show that I'd love to see. Would appreciate some help here greatly.
Thanks.
Open command line, "browse" to folder where Dual-ISO files and CR2HDR (+ dcraw + exiftool) are and use
for /R %a in (*.CR2) do cr2hdr "%a"
Ciao
Walter
Quote from: Walter Schulz on November 11, 2013, 03:53:43 AM
Open command line, "browse" to folder where Dual-ISO files and CR2HDR (+ dcraw + exiftool) are and use
for /R %a in (*.CR2) do cr2hdr "%a"
Ciao
Walter
Well, it turns out I didn't have to do that. I wasn't aware that both the dngs and the executables had to be in the same folder in order to work. I suppose that was my problem the whole time. I just copied them in to the folder and now it appears to be working great.
Thanks for the indirect help Walter!
I hate to keep causing problems here, but I've encountered another issue with Dual Iso videos.
It seems that, at random times, the exposure will flicker from light to dark, almost like cr2hdr thought the certain frame should be exposed differently than all the rest.
Has anyone else encountered this issue?
DxO specific discussion moved here: http://www.magiclantern.fm/forum/index.php?topic=9298.0
Quote from: Magic 7D on November 09, 2013, 02:17:36 PM
Hi there everyone!
I have a thought/request/idea..
Sometimes when out shooting I'm shooting without dual ISO and then I might get into a situation where I need it "quickly" but going into the menu to turn it on often times is not quick enough, just had a thought that it might be possible to set for example the info button as a shortcut key for turning dual ISO on or off when not shooting in live view. Don't really know if this is the right place to put this, but just thought it would be very handy. And I have no knowledge whatsoever in coding etc. so I don't know if it's an easy thing to acomplish or not.
Keep the good work up! :)
Think this got lost in the heat of discussing DxO ;) .. or maybe I'm the only one thinking it would be useful :)
(Hope I'm not breakin the rules by quoting myself)
Alas, it appears there is not a very good way for 650D dual iso footage to be processed.
I've concluded that the combination of having to remove the pink dots from the RAW file and having to interpolate the dual iso lines is just a bad combination for any method I've tried.
With raw2dng_cs2x2_ahdlike_noise.exe and cr2hdr.exe, random and intermittent flickering is experienced with the footage.
With PinkDotRemover.jar and cr2hdr.exe, the pink dots are not removed fully.
a1ex, any thoughts on this? I have a large pile of very interesting Dual ISO fire juggling that I'd love to use, but I can't seem to get usable footage. :(
...good evening from Greece...
I just started with the last edition of ML for my 7D and I want to ask you a favor...
is there any thread about post processing the dual iso pictures? I rode the article by alex1 form the beginning but I'm lost...
I'm dragging the images over the cr2hdr.exe but nothing happens.....what is this? can you please give me an advice?
There's information on it in the first post. I just copy all files that come in the zip and paste them in the folder where the dual iso pictures are and drop the images on the exe. If you only have cr2hdr.exe and do not have dcraw or the other associated files, then the process will not work.
thank you crazyrunner33, I did that in the same folder and a DNG file has been created....so easy!
thank's for your help....sometimes, you read a post...continue to read...and after 30min your mind is away!!!
Hi,
I wonder if there is any way an CR2 image file using Dual_ISO can be differentiated from a non Dual_ISO file for example in the EXIF, I had a look but could not find anything obvious.
Reason i need it is to programatically differentiate Dual_ISO so i can script execution of cr2hdr.exe when new dual_iso files are added to the folder.
All the best,
Traste
Quote from: Traste on November 17, 2013, 08:10:20 PM
I wonder if there is any way an CR2 image file using Dual_ISO can be differentiated from a non Dual_ISO file for example in the EXIF, I had a look but could not find anything obvious.
No, there isn't - the camera doesn't detect dual_iso so the cr2 is just like a non dual_iso one, and cr2hdr just copies the exif/makernotes straight over to the dng. This is a bit awkward, since the "dual" prefix doesn't always work.
I had already suggested to save a xmp file just like with deflicker that contains the iso absolute values used (like 100/800) because that's currently also impossible to determine afterwards, cr2hdr just uses the relative difference and even this doesn't end up in any exif data.
Quoteprogramatically differentiate Dual_ISO so i can script execution of cr2hdr.exe when new dual_iso files are added to the folder
Cr2hdr makes this determination and skips regular .cr2 files (unless something goes wrong). So for each .dng created, the similarly named .cr2 is dual iso.
Quote from: engardeknave on November 17, 2013, 10:14:06 PM
Cr2hdr makes this determination and skips regular .cr2 files (unless something goes wrong). So for each .dng created, the similarly named .cr2 is dual iso.
Thank you for enlightening me! That makes things a whole lot easier, now i just need to check if a DNG already exists for each CR2 file and ... Robert is my uncle!
Just shot some dual iso on my 5dii, and can't get the thing to work - always get a "filename or extension too long" error signal when I drag the dngs. I have cr2hdr with the dcraw and exiftool in the same folder with the dngs.
Quote from: dickson on November 20, 2013, 10:46:58 PM
Just shot some dual iso on my 5dii, and can't get the thing to work - always get a "filename or extension too long" error signal when I drag the dngs. I have cr2hdr with the dcraw and exiftool in the same folder with the dngs.
This error is literally what the error message says, the file extension is too long. Try throwing the folder with your dual iso and cr2hdr tools onto the root of the C drive, it should work from there.
Thanks, seem to have it going now. But now what's happening is I drag the dngs, and they're processed - and nothing's changed. In Adobe bridge I can still see the alternating frames at different iso's. When it's processing there's a little note for each frame as it flashes by, something like "doesn't look like interlaced iso's." So I guess it's not recognizing that the frames are different. I lowered the difference between the two iso levels, could that be doing it?
I see now that this has been dealt with before - but I have the latest version of cr2hdr.
Today I finished my first tests with dual iso raw video with 600D T3i. I learned the workflow and I got good images.
Problem is: magenta dots everywhere.
(http://www.apefos.com/dualiso/md.jpg)
Reading the forum I found many people have this issue, so I have a few questions:
Is there one or more Canon DLSR cameras which can do dual iso raw video without any dots? (i am interested only in video)
Is there a good working solution for the dots in the 600D T3i? (and/or for the other cameras)
Thanks!
the cr2hdr-hot.exe work pretty good but if you drag more than 40 pics in one single click it will crash somewhere around picture 40; hope you can fix that so i can leave mi computer processing tons of photos (+1000) while i'm not home.
I've never got the confetti before. I wonder if my camera is just special and doesn't have any hot pixels.
Quote from: arturochu on November 21, 2013, 06:41:46 AM
the cr2hdr-hot.exe work pretty good but if you drag more than 40 pics in one single click it will crash somewhere around picture 40; hope you can fix that so i can leave mi computer processing tons of photos (+1000) while i'm not home.
http://www.magiclantern.fm/forum/index.php?topic=8520.0
Long time ago I found a plugin for dealing with hotpixels, it is free, no longer available, but I will allow a link for downloading it because maybe the developers can get inspired on it to find a solution for the issue:
http://www.apefos.com/dualiso/svpr.rar (http://www.apefos.com/dualiso/svpr.rar)
I did a test in the cr2hdr-hot.exe and ir deduces the size of the hot pixels, but do not remove them completely, see the difference between the results from cr2hdr and cr2hdr-hot:
(http://apefos.com/dualiso/semhot.jpg)
(http://apefos.com/dualiso/comhot.jpg)
Quote from: arturochu on November 21, 2013, 06:41:46 AM
the cr2hdr-hot.exe work pretty good but if you drag more than 40 pics in one single click it will crash somewhere around picture 40
Any hints about how to reproduce it?
Quote from: a1ex on November 04, 2013, 12:00:55 PM
I've tested it on 700 pics in a single batch with --fast, 100 pics with default settings, and 2 pictures in valgrind
Quote from: dickson on November 21, 2013, 01:14:32 AM
In Adobe bridge I can still see the alternating frames at different iso's.
Zoom into the picture and should see the dual iso LINES. There are no alternating frames in dual iso, it sounds like you are working with hdr video with frames of alternate isos??
http://www.magiclantern.fm/forum/index.php?topic=2595.msg10605#msg10605
Oops, my bad, you're right. But dual iso is greyed out on my camera. I can manipulate the settings, but I'm not getting anything.
Here it is a RAW dual iso video recorded by the 600D T3i for download. I uploaded it because it gives me lots of magenta dots and can be used for reproduce the problem and improve the CR2HDR-HOT (drag and drop over the cr2hdr-hot to reproduce the dots in dng sequence)
http://www.apefos.com/dualiso/di600d.rar (http://www.apefos.com/dualiso/di600d.rar)
These DNGs were already processed, what should we do with them?
I believe I uploaded the DNG extracted from the RAW video using RAW2DNG.exe, not processed by the CR2HDR yet, so the idea is to use these DNGs to test the hot pixel remover in the CR2HDR-HOT.exe
Drag them over the CR2DNG-HOT.exe to see if the hot pixel removing works, you can use them to test the CR2HDR-HOT.exe until find a version which can corrects the hot pixels.
I know you can do this using your RAW and DNG files, I just uploaded these because was shoot with the 600D, and maybe the 600D hotpixels can be different from the other cameras like 5D3.
This is the link for download the original dual iso .RAW file from 600D which gives me magenta dots (hot pixels). With the .RAW file the tests can be better than using the DNGs I uploaded before.
https://www.dropbox.com/s/i4gs4qvzsbrbhwr/600d_raw_video.rar (https://www.dropbox.com/s/i4gs4qvzsbrbhwr/600d_raw_video.rar)
Servus
I can confirm, that remooving bad pixel from 600D raw-files is a problem. dcraw (ufraw) fails, darktable too. The only prog here on Linux is RawTherapee, that succeed, But this cannot be used for dual-iso pics, because the romooval has to be done before interpolation, I guess.
I looked at the code of cr2hdr and saw, that the badpix-remooval funktion only works on the dark lines of the pic. That´s not enough for 600D. So I changed the code for to work with the bright side of life :) too and compiled the module new,
With this test-change all badpix are remooved! Only on the border of the image some new ones are created, what is no problem for cutting out in postprocessing.
In my eyes, this function should be extended, to work with the whole image. And it would be fine to append this function to raw2dng and mlv_dump.
Edgar
Can you upload a CR2 or interlaced DNG where this is visible? I didn't notice it in the ones from apefos (these shots have cold pixels in bright areas, but didn't see any hot pixels).
Alex, did you have a look at the raw-video, apefos did upload last? cr2hdr tells me:
Looking for hot/cold pixels...
Hot pixels : 116
Cold pixels : 107
I for myself have to search for a file with hotpix. I only have cold-ones at the moment available.
Edgar
Got it, that means the cold pixels can be anywhere. Since I didn't see any hot pixels on the bright lines, that check is for speedup and to avoid false positives.
I'll fix it when I'll get back.
Great :)
Edgar
I having a problem trying to do my first Dual-Iso.
I configure the camera in RAW mode, load the module on ML, reboot and then enable Dual-ISO. But when i load the file with cr2hdr, i get "Doesn´t Look like Interlaced ISO" error.
I try to restore defaults of ML and then do it all again, but i get the same error.
Any idea?. Im using the last stable ML version (22Nov) on a 7D. I asume im making some wrong becouse is not a common error.
Tks you.
Quote from: CaronteGF on November 26, 2013, 01:59:38 PM
I configure the camera in RAW mode, load the module on ML, reboot and then enable Dual-ISO. But when i load the file with cr2hdr, i get "Doesn´t Look like Interlaced ISO" error.
Check if dual_iso was really enabled and working or cr2hdr is the problem. The effect of dual_iso isn't hard to see, if you zoom to 100% either in camera or on the PC (for example using XnView) you can clearly see the interlaced lines, also the white balance is usually completely off and the shot has green and magenta areas.
Quote from: Marsu42 on November 26, 2013, 05:54:20 PM
Check if dual_iso was really enabled and working or cr2hdr is the problem. The effect of dual_iso isn't hard to see, if you zoom to 100% either in camera or on the PC (for example using XnView) you can clearly see the interlaced lines, also the white balance is usually completely off and the shot has green and magenta areas.
Tks for the tip. Dual Iso was really enable (Green Light), but the effect was not applied. I try with differents ML Nightly build, and i found the problem.
The 22 an 17 Nov just doesn´t work for me, But 25Nov works really great!. Problem Solved!
Small update to cr2hdr:
- fixed cold pixel handling
- experimental DNG compression (you need to install Adobe DNG Converter; should be autodetected)
DNG compression (requires Adobe DNG Converter):
--compress : Lossless DNG compression
--compress-lossy: Lossy DNG compression (be careful, may destroy shadow detail)
Download it from first post. It works pretty good for me (currently running it on a few thousands of pictures), but I'm still waiting for a detailed bug report for post #1602 (I could not reproduce it).
Also, auto ETTR should do slightly better choices when linked to dual ISO:
- dual ISO is only enabled by ETTR if the DR improvement is at least 1 stop (so it should no longer choose 100/200, for example)
- if the calculations result in something like 200/800, it will switch to 100/800 (better highlight protection at very small cost)
modules/dual_iso/adobedng-bridge.c, line 39
FILE* f = popen("winepath \"C:\\Program Files\\Adobe\\Adobe DNG Converter.exe\"", "r");
In Linux 64Bit the "Program Files" are called "Program Files (x86)".
Edgar
Mine is 64-bit, but under Wine I only have Program Files.
Should be OK now (all Windows search paths are now also tried via Wine).
QuoteAlso, auto ETTR should do slightly better choices when linked to dual ISO:
From testing today it seems to. I've had a few photos that are all black out of CR2HDR and I have to raise expo +5 in lightroom, almost the whole pic is half res.
Quote from: 1% on November 30, 2013, 09:32:20 PM
I've had a few photos that are all black out of CR2HDR and I have to raise expo +5 in lightroom, almost the whole pic is half res.
But that's bound to happen if you use dual_iso to catch highlights, isn't it? Actually I'm not really sure if I'm using dual_iso correctly, but if I have a valuable very bright part that I want to expose correctly, I underexpose the dual_iso shot so much that the bright part has some definition, accepting that the dark part will loose definition as you described - mostly I'm using about +3ev in LR.
I'm using auto ettr always on so it picks when dual ISO happens. Maybe if I combined with the auto expo module it wouldn't do this but then the 2 tend to fight.
wat!
(http://i.imgur.com/4eqpAl3l.jpg) (http://imgur.com/4eqpAl3)
Pre-built from 1st post? Latest commits touch this?
Fixed, long live Microsoft.
Quote from: a1ex on November 30, 2013, 03:16:29 PM
Small update to cr2hdr:
- fixed cold pixel handling
- experimental DNG compression (you need to install Adobe DNG Converter; should be autodetected)
DNG compression (requires Adobe DNG Converter):
--compress : Lossless DNG compression
--compress-lossy: Lossy DNG compression (be careful, may destroy shadow detail)
Download it from first post. It works pretty good for me (currently running it on a few thousands of pictures), but I'm still waiting for a detailed bug report for post #1602 (I could not reproduce it).
Also, auto ETTR should do slightly better choices when linked to dual ISO:
- dual ISO is only enabled by ETTR if the DR improvement is at least 1 stop (so it should no longer choose 100/200, for example)
- if the calculations result in something like 200/800, it will switch to 100/800 (better highlight protection at very small cost)
Hello A1ex -- hope you enjoyed your Thanksgiving weekend. I wanted to follow up regarding your recent update to your cr2hdr and I clicked on the Mac version link that is on the first post (according to your statement) and apparently the link is dead as of now.
Perhaps is the reason because of the recent bug regarding the black area losing resolution by half.
Should I stick with beta version .13?
Thanks!
I don't maintain the Mac version, only the source code and the windows exe. But you can run it under Wine.
An updated mac version would be really great. Thx
Heh, for some reason it doesn't work in batch with barracuda + compress. works by itself tho
I'm having some trouble here with the cr2hdr.exe on Mac ...I have recorded a video with the hdr function and in Raw mode, then I take the raw file and use RAWMagic to transform it in .dng folders. When i drag the dng folder to the cr2hdr.exe the program starts to erase the dng files... and it appears a message saying "Doesn't look like interlaced ISO" (but in the preview of the computer i can see that the files are with diferent ISOs) ... and when I rename the .dng to .CR2 and than drag to the cr2hdr.exe it appears an .txt file saying the same thing... what I'm doing wrong?
Quote from: diogo.fagundes on December 04, 2013, 07:03:30 PM
...I have recorded a video with the hdr function and in Raw mode
Sounds like maybe you are confusing the HDR mode with the dual ISO mode, they are different things.
yea.. for sure... only now i understood what is the dual iso.. but, so where is the dual iso submenu? ... i'm not finding it, and i have already turned the dual_iso mode on, and restarted the camera...
Edited: ok, I already found it.... it is on the topic expo of the ml menu... then dual iso...
My current main issue with the otherwise great dual_iso shots is the white balance, or to be exact the lack of.
It's not so much that cr2hdr doesn't copy the Canon (a)wb from the cr2 to the dng, but that after manually setting the *same* wb on the same shot, one normal, one dual_iso, the result isn't the same which esp. seems to result in a tint (cyan/magenta) cast.
I haven't done enough comparison shots of the same scene and will buy a color calibrator for my monitor to exclude this problem source - but does anyone experience the same problem? I guess I have to shoot a grey/color card more often with dual_iso for calibration...
Quote from: Marsu42 on December 05, 2013, 10:27:23 PM
My current main issue with the otherwise great dual_iso shots is the white balance, or to be exact the lack of.
It's not so much that cr2hdr doesn't copy the Canon (a)wb from the cr2 to the dng, but that after manually setting the *same* wb on the same shot, one normal, one dual_iso, the result isn't the same which esp. seems to result in a tint (cyan/magenta) cast.
I haven't done enough comparison shots of the same scene and will buy a color calibrator for my monitor to exclude this problem source - but does anyone experience the same problem? I guess I have to shoot a grey/color card more often with dual_iso for calibration...
The trick would be finding out which ISO is best to white balance off for the dual ISO.
The low iso, the high iso, or an iso directly between?
I think it's an exif issue. Can you try if this helps?
- convert the CR2 to DNG with Adobe DNG (lossless)
- process the DNG with cr2hdr
I always set WB manually, so I can say I don't have this problem :D
If it's daylight, it's easy (something around 5500 in sunlight, 6500 in shadow). A little magenta boost when shooting grass with older lenses. Indoors, I found balancing on something white a bit too harsh (I prefer to go a little warmer from there). Sometimes I get much better results if I do the white balance directly on skintones.
I have a small modification to ufraw that lets me do spot WB on skintones; if anyone is interested I can share it.
Share it!
Alright, see here: http://www.magiclantern.fm/forum/index.php?topic=9469
Quote from: ItsMeLenny on December 05, 2013, 11:08:26 PM
The trick would be finding out which ISO is best to white balance off for the dual ISO.
The low iso, the high iso, or an iso directly between?
Yes, you're correct, that might be it - I always copied the wb setting from the lower iso setting (usually 100) and haven't tried how much it changes when going to the upper (800 or 1600) value.
I hope that it's not that the wb of the dual_iso shots is completely screwed up because cr2hdr doesn't consistently merge the shots, resulting in different wb areas and requiring local correction in post?
cr2hdr only does linear processing, so it shouldn't alter colors; it should look exactly like a ISO 100 image, except for noise.
Does WB stay there if you take a regular CR2 and convert it to DNG with Adobe converter?
Quote from: a1ex on December 06, 2013, 09:16:45 AM
Does WB stay there if you take a regular CR2 and convert it to DNG with Adobe converter?
No, I didn't try yet, will do and report back in the evening :-) ... that's supposed to copy the camera (a)wb correctly?
Quote from: a1ex on December 06, 2013, 04:31:54 AM
I think it's an exif issue. Can you try if this helps?
- convert the CR2 to DNG with Adobe DNG (lossless)
- process the DNG with cr2hdr
Not to be misunderstood: My problem also occurs after *manually* setting the wb, cannot seem to find the correct setting as there seems to be a color cost somewhere at any wb value :-\
I really hope I'm not making this up and causing trouble... the problems I think I'm experiencing are slight mind you, but there seems to be a magenta/cyan cast in pastel autumn light beige/grey/green autumn colors. I really have to get that monitor calibration and take more dual_iso and normal shots to isolate the problem, but it's such a hassle to quickly switch because with the current implementation I have to set another ec and fec to get from dual_iso to normal and back.
If you do some tests, I think it's best to stay in M. Fix some exposure settings, set ISO 100 from Canon menu, set some recovery ISO from ML (800, 1600, whatever), enable "alternate frames only" and fire away. All shots should look identical in colors and exposure if you use the same parameters for developing.
Here's a broken dual_iso shot, the current cr2hdr produces a yellow overlay on the left side:
https://bitbucket.org/Marsu42/ml-mod60d/downloads/DUAL6363.CR2
I hope it's a cr2hdr problem and not dual_iso not working on the 6d properly or even my hardware :-o though I don't think the latter is the reason, this is the first time I experience this problem. Alex not currently committing code doesn't mean it's impossible to *code* an updated cr2hdr version without *committing* the code, correct :-> ?
This is caused by blooming (strong overexposure near the edge => the bright sun overflowed onto the optical black area). You can see the effect with dcraw -E DUAL6363.CR2.
Workaround for this shot: comment out the processing of the top black bar (around line 729).
Quote from: a1ex on December 18, 2013, 10:26:52 AM
This is caused by blooming (strong overexposure near the edge => the bright sun overflowed onto the optical black area). You can see the effect with dcraw -E DUAL6363.CR2.
Ok, thanks for the diagnosis, I would never have though of this... if this is what can happen and since dual_iso is bound to be used in high contrast (i.e. sunrise/sunset) shots - would this justify adding an command line switch to fix these shots?
On the left bar I simply went for ignoring some border pixels. But you are right, disabling black correction (either completely or just on one of the bars) can be helpful for troubleshooting.
Quote from: a1ex on November 04, 2013, 12:00:55 PM
Here's a version that should be pretty good for everyday use.
cr2hdr-hot.exe (http://acoutts.com/a1ex/cr2hdr-hot.exe)
What's new:
- better hot pixel handling (solves the shot from Marsu42 with --really-bad-pix, also fireworks are no longer mistaken for bad pixels)
- fixed a bug that caused massive shadow aliasing in the version with many command-line switches
- minor fixes
What's not:
- I've removed the denoising option for now, because DxO puts it to shame. I'll add it back when I'll get comparable results in terms of recovered detail, without the plastic look.
Have a quick play with it; if there are no major bugs (like stopping after 30 pictures, crashing, or regressions in image quality) I'll update the first post tonight. I've tested it on 700 pics in a single batch with --fast, 100 pics with default settings, and 2 pictures in valgrind, so I believe it should be pretty solid now.
Update: after running my usual tests, I've noticed the new hot pixel algorithm (in aggressive mode) has little or no effect on resolution charts. Another milestone completed!
Since DxO won't read DNG, is it possible to either
1. Put the denoising option back
2. Have a version of CR2HDR that can read DNG for its inputs so that, using DxO, we can denoise the CR2 first, and export it as DNG for HDR conversion?
I'll take a closer look at the denoising algorithm somewhen; noticed that darktable uses the same algorithm, so I'll have a look at how they have fine-tuned it. Nice read: http://www.darktable.org/2012/12/profiling-sensor-and-photon-noise/
The current version already reads DNG files, but I'm not sure what you want to do with this.
Oooh! Thanks!
I'll play with the DNG input.
Btw, I played with the cr2hdr-hot and cr2hdr-denoice to batch convert about 100 files, and it keeps crashing after converting about 40 files give or take a few.
Same with the one from the first post?
Quote from: a1ex on December 19, 2013, 09:34:30 PM
I'll take a closer look at the denoising algorithm somewhen; noticed that darktable uses the same algorithm, so I'll have a look at how they have fine-tuned it. Nice read: http://www.darktable.org/2012/12/profiling-sensor-and-photon-noise/
The profiled denoising in darktable is a good thing. The fine-tuning is done with user-measured curves, I think. But this kind of denoising does not fit to all kind of pictures. For my moon pics for example I cannot use it.Too much details go and too much artifacts come in. But for most of the other pics, it´s a great thing as I said before.
Edgar
Can you upload one so I can use it as a test shot?
Quote from: a1ex on December 19, 2013, 09:57:00 PM
Same with the one from the first post?
Using the one I got from this one:
Quote from: a1ex on November 04, 2013, 12:00:55 PM
Here's a version that should be pretty good for everyday use.
cr2hdr-hot.exe (http://acoutts.com/a1ex/cr2hdr-hot.exe)
What's new:
- better hot pixel handling (solves the shot from Marsu42 with --really-bad-pix, also fireworks are no longer mistaken for bad pixels)
- fixed a bug that caused massive shadow aliasing in the version with many command-line switches
- minor fixes
What's not:
- I've removed the denoising option for now, because DxO puts it to shame. I'll add it back when I'll get comparable results in terms of recovered detail, without the plastic look.
Have a quick play with it; if there are no major bugs (like stopping after 30 pictures, crashing, or regressions in image quality) I'll update the first post tonight. I've tested it on 700 pics in a single batch with --fast, 100 pics with default settings, and 2 pictures in valgrind, so I believe it should be pretty solid now.
Update: after running my usual tests, I've noticed the new hot pixel algorithm (in aggressive mode) has little or no effect on resolution charts. Another milestone completed!
Also, I tried putting in the DNG to cr2hdr but it won't spit out any result. I was just wondering what it'd look like if I use DxO for prime denoising then exporting as DNG and putting that DNG into cr2hdr.
Oh well... it must be 4 a.m. where you live... (I've asked something, you answered something else).
For DNG, I convert CR2 files with Adobe DNG Converter, and cr2hdr still accepts them.
Quote from: a1ex on December 19, 2013, 10:09:51 PM
Oh well... it must be 4 a.m. where you live... (I've asked something, you answered something else).
For DNG, I convert CR2 files with Adobe DNG Converter, and cr2hdr still accepts them.
*checks the clock*
Dammit. I should get some sleep now. Spot on on the time zone. Let me brain up when I get up this afternoon.
Let me try Adobe-ing that dxo DNGs.
Thanks.
SCNR!
Quote from: feureau on December 19, 2013, 10:44:22 PMLet me try Adobe-ing that dxo DNGs.
Made my day!
Ciao
Walter
Note that:
1) dxo dng's are linear (demosaiced); cr2hdr only works on bayer data
2) denoising must be applied after "deinterlacing", not before (try it with pen and paper on a 3x3 kernel if you don't believe)
Quote from: a1ex on December 19, 2013, 09:57:00 PM
Same with the one from the first post?
Quote from: a1ex on December 19, 2013, 10:09:51 PM
Oh well... it must be 4 a.m. where you live... (I've asked something, you answered something else).
For DNG, I convert CR2 files with Adobe DNG Converter, and cr2hdr still accepts them.
Okay, I brained up. Not sure which first post you're referring to though, but I'm not running the pre-built from first post. I'm using the ones you posted: the cr2hdr-denoice.exe, and cr2hdr-hot.exe Drag and dropping about 166 cr2 files on it. It will run like normal but after about 20-40 files, it will just stop. On the cr2hdr-denoice, it will look like it's doing something, but it won't progress, and if I open the task manager, the CPU is at 0. On the cr2hdr-hot, it will just crash and give out that window where windows ask if you want to wait or just close the program.
Before this, I've been running the cr2hdr-exp and it runs with no problem at all. (all this on win 7 x64)
I've asked you to try it and tell me if it has the same problem. Also check post #1620.
Quote from: arturochu on November 21, 2013, 06:41:46 AM
the cr2hdr-hot.exe work pretty good but if you drag more than 40 pics in one single click it will crash somewhere around picture 40;
Sorry to be slow participating in this thread.
Win-7-64
I've had intermittent cr2hdr.exe crashes when dragging lots of .cr2's onto cr2hdr.exe. It wasn't repeatable enough to report .. (sorry?). I didn't notice whether it had anything to do with 40 or so files. There was info from APPCRASH I could watch for, and send screen-shots of nitty-gritty details, if that would help.
I haven't seen this happen when I launch from the command line, such as:
for %c in (*.cr2) do cr2hdr %c
or
for %n in (1895 1896 1897 1898 1899 1900) do start for %c in (*%n*.cr2) do cr2hdr %c
My speculation is that using a command line, there might be more information showing up that didn't disappear when the APPCRASH dialog window was dismissed. Or not?
Make sure you use the latest binary from the first post. The ones posted in the thread are just experiments to get some quick feedback, but the one from the first post is more or less stable (and it's also the latest one). The only thing missing is denoice.
Quote from: a1ex on December 19, 2013, 10:02:25 PM
Can you upload one so I can use it as a test shot?
Is a 16bit png ok or do you need a cr2 (don´t know, where to upload cr2)?
Edgar
16-bit pgm is OK only if you got it by running dcraw -4 -E. You can upload on Dropbox or whatever other file sharing service.
Find a solution for CR2: IMG_6550.CR2 (http://www.file-upload.net/download-8424861/IMG_6550.CR2.html)
That`s one on my not so good moon pics, captured with ISO1600 (don´t know why ???) With darktable and the profiled denoising too much details get lost and the picture looks not natural.
Edgar
Got it, there's very little noise on the moon (only on the dark areas). Do you also have the JPG as a reference image of what not to do?
Here you are:
profiled denoising: img_0001_01.jpg (http://www.file-upload.net/download-8425024/img_0001_01.jpg.html) not natural
bilinear filter: img_0001.jpg (http://www.file-upload.net/download-8425009/img_0001.jpg.html) some little noise, but more natural
Both versions are made with darktable filters "dead pixel" and "chromatic abberation". I made the black a little more black to make the space free of noise.
Edgar
Quote from: a1ex on December 20, 2013, 02:29:54 PM
I've asked you to try it and tell me if it has the same problem. Also check post #1620.
Welp. I downloaded the one from the first post yesterday, and I've been cr2hdr-ing hundreds of photos from two different cameras, and so far, no crashes at all. Thanks for this.
Didn't try the DNG compression tag thingy though. How do I use that without having to command line each files?
On Linux I use cr2hdr --compress *.CR2
It doesn't work on Windows on plain command prompt (it might work on cygwin or msys shell, didn't try).
I read through quite a bit of this thread which is 67 pages as of my writing. Is the version of cr2hdr linked from the first post the most current/recommended version to use? Please could you (a1ex) update the first post if so or if not or add links to the other useful versions? Thanks.
Perhaps a stupid question... Will it be possible at some point to do this post processing in the camera? Or is that just not technically feasible?
I use lightroom. Is there an easy way to incorporate this cr2hdr step into a workflow without having to go to the command line?
Thanks!
Quote from: mgrant on December 27, 2013, 12:28:02 AM
Is the version of cr2hdr linked from the first post the most current/recommended version to use?
Here's the link I use:
https://bitbucket.org/a_d_/magic-lantern/downloads
There is a date-stamp to indicate whether the cr2hdr file for Mac or Windows has been refreshed since the last time you downloaded it.
Note that .zip file won't necessarily have the other needed files: dcraw.exe, exiftool.exe, readme.txt
The other randomly named files may be for modules. Or not?
I'm having trouble where the processed images is actually darker. These were taken at 100/3200:
interlaced image:
(https://dl.dropboxusercontent.com/u/1700402/DUAL3071.jpg)
cr2hdr processed image:
(https://dl.dropboxusercontent.com/u/1700402/DUAL3071-2.jpg)
C:\Users\Michael\Pictures\2013\2013-12-27>cr2hdr DUAL3071.CR2
cr2hdr: a post processing tool for Dual ISO images
Last update: 2f4e2df on 2013-12-02 17:45:36 UTC by alex:
cr2hdr: fix DNG compression under Windows
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default
)
Input file : DUAL3071.CR2
Camera : Canon EOS 7D
Full size : 5360 x 3516
Active area : 5202 x 3465
White level : 12500
Black borders : 158 left, 51 top
Black level : 1955
ISO pattern : dBBd GBRG
Noise levels : 5.47 33.98 39.40 5.28 (14-bit)
ISO difference : 5.09 EV (3402)
Black delta : 9.28
Dynamic range : 10.96 (+) 8.06 => 13.15 EV (in theory)
Looking for hot/cold pixels...
Hot pixels : 7
Cold pixels : 27592
AMaZE interpolation ...
Amaze took 5.49 s
Edge-directed interpolation...
Semi-overexposed: 6.46%
Deep shadows : 90.22%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap : 3.0 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 3.72 (16-bit), ideally 3.68
Dynamic range : 13.47 EV (cooked)
Black adjust : 0
Output file : DUAL3071.DNG
1 image files updated
The raw file is here: https://dl.dropboxusercontent.com/u/1700402/DUAL3071.CR2 (https://dl.dropboxusercontent.com/u/1700402/DUAL3071.CR2)
What are you all doing to get these amazing images? I can brighten up the shadow in lightroom but it gets really grainy. I feel I am doing something wrong.
First, I increase exposure while paying attention not to clip highlights. Then I increase shadows. But 7D is noisier than the 5D2 I experiment with...
Quote from: mgrant on December 27, 2013, 05:02:50 PM
taken at 100/3200
This is too extreme settings. The actual maximum ISO your camera is 1600, do not pass it, otherwise will have images with a lot of noise. I would recommend you to use a maximum of 100/800. Above that, it is not worth using dual_iso.
Quote from: mgrant on December 27, 2013, 05:02:50 PM
What are you all doing to get these amazing images? I can brighten up the shadow in lightroom but it gets really grainy. I feel I am doing something wrong.
This darkness in the image is caused by the linearization of gamma. Try increasing the exposure and then down the highlights... will leave with much noise, but will give to correct. I recommend you also use the profile VisionColor, the VisionLog, can recover enough without giving too much noise.
To remove the noise, an alternative is the PRIME noise reduction, of DxO (is really awesome, try). To sharpen, not use unsharp, use deconvolution.
Quote from: a1ex on July 16, 2013, 06:33:50 PM
Time for the first real image quality improvement in ML history :)
Postprocessing
- For CR2 files: cr2hdr.c (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso/cr2hdr.c) (Windows:cr2hdr.exe (http://acoutts.com/a1ex/cr2hdr.zip)). It requires dcraw and (optional) exiftool in your executable path (they are in the zip). Drag and drop the CR2 files over cr2hdr.exe.
- For RAW video files: latest raw2dng.exe (http://acoutts.com/a1ex/raw2dng.exe). extract the DNGs with any method you like (e.g. raw2dng), then drag the DNG files over cr2hdr.exe
- To get a natural HDR look: try my automatic color grading script (http://www.magiclantern.fm/forum/index.php?topic=7022).
It works as long the group of DNG files to drag on cr2hdr.exe, is less than 2300.
In small groups of DNG it works. But if I drag the whole 2300 DNGs files, it tells me that
"File name or extention is too long"
why when i process the photos with cr2hdr im loosing the icc profile information ?
thanks
ron
Problem and Possible Solution with Pink highlights in Video
I'm using dualiso a lot in video, and want to share some fails/successes with pink highlights. Maybe we can tweak some settings to get the best results:
(https://dl.dropboxusercontent.com/u/43462755/000001.jpg)
Workflow
I'm running all .RAW files against raw2dng, then running the DNGs against cr2hdr, then finally a converter to ProRes or another proxy.
Problem
When I use the latest cr2hdr.exe, if the shot has really tough lighting I get nasty pink lines in my highlights that are not present in the interlaced DNG I get from raw2dng. So the culprit is likely to be cr2hdr, right?
Possible Solutions
I've tried 5 different versions of cr2hdr (thanks A1ex for all the hard work), and have found varying degrees of success:
You can view DNGs, exes, logs, etc:
https://www.dropbox.com/sh/78nozdsmyu4waei/-Xu6-IjAOs
I'll keep posting results
cr2hdr_histmatch -- overall best; eliminated most pink interpolation highlights and green pixelation; does produce some flickering in highlights an a few pink highlights
(https://dl.dropboxusercontent.com/u/43462755/000900-cr2hdr-histmatch.jpg)
cr2hdr_hot -- eliminated interpolation, eliminated most pinks, eliminates some small green pixelation, but still has some pink highlight in a few frames
(https://dl.dropboxusercontent.com/u/43462755/cr2hdr_hot-pinks.jpg)
cr2hdr_histmatch2 -- eliminated interpolation, eliminated most pinks, but still has some pink highlight and green pixelation in a few frames
(https://dl.dropboxusercontent.com/u/43462755/000900-cr2hdr-histmatch2.jpg)
cr2hdr_histmatch3 -- eliminated interpolation, eliminated most pinks, but still has some pink highlight and green pixelation in a few frames
(https://dl.dropboxusercontent.com/u/43462755/000900-cr2hdr-histmatch3.jpg)
Video produced using cr2hdr-histmatch. Note the highlight flashing and pink highlights near the end of the video.
Still has some highlight issues:
(https://dl.dropboxusercontent.com/u/43462755/cr2hdr-histmatch-issue.jpg)
Have you tried the cr2hdr from the first post of this thread? It was updated about a month ago iirc.
Thank you! This feature works flawlessly minus the resolution loss.
(http://s28.postimg.org/4g6rob049/1_V2_A3471.jpg) (http://postimg.org/image/4g6rob049/)
(http://s28.postimg.org/7lrdeiiqh/1_V2_A3493.jpg) (http://postimg.org/image/7lrdeiiqh/)
(http://s28.postimg.org/mx16f4g2h/1_V2_A3571.jpg) (http://postimg.org/image/mx16f4g2h/)
(http://s28.postimg.org/8f3z74ord/1_V2_A3682.jpg) (http://postimg.org/image/8f3z74ord/)
Thanks Audionut.
Quote from: Audionut on January 01, 2014, 11:20:47 PM
Have you tried the cr2hdr from the first post of this thread? It was updated about a month ago iirc.
Yes. It yields the worst results - pink interpolation and green pixelation in highlights.
Currently histmatch rev1 produces the best images, though it still has issues - flickering highlights, some pink highlights.
I'm assuming I need to adjust the highlight parameter to something lower than 15000.
Any suggestions?
Checking the white level, it shifts from 12500 to 15886 - big shift that causes my highlights (sky in this case) to render as either blue with clouds or blown-out white.
Given that these changes might happen from frame to frame, that would explain the flickering highlights and pinks.
(https://dl.dropboxusercontent.com/u/43462755/cr2hdr_ss.jpg)
I tried tweaking the saturation in level in dcraw and got the following:
at 12500
(https://dl.dropboxusercontent.com/u/43462755/001864_at_12500sl.jpg)
at 15886
(https://dl.dropboxusercontent.com/u/43462755/001864_at_15886sl.jpg)
Any thoughts on how best to implement a solution?
Hi all! :)
Can I have access to some raw images (Dropbox or anything else)? I would like to do some tests with a HDR software that I designed.
If I can provide Dual Iso support, that would be great! :D
MarS: there are many links to raw files in this thread (one is in the first post), and I believe most of them are still online. Look for posts showing problems with some particular raw file. Can you share some details about your software?
@erik: it's indeed a white level problem. I should provide a way to override it in the command line, because right now it's autodetected. It's important to have it correct - or a little underestimated - during the conversion; I've changed it in white_detect_brute_force to 10000 and the output seems fine.
For flicker-free video, you have to equalize the exposure somehow. Either by playing with the white point, or maybe also with this deflickering script: http://www.magiclantern.fm/forum/index.php?topic=8850.0
Thank you a1ex and congratulations for your brilliant idea!!! :)
I've already downloaded the first link (Orange Juice Bottle). Unfortunately many links are dead. I'll still take the time to check all the pages for cr2 files.
I'm working on a software called MarScaper HDR for Mac. For more details, you can have a look at here: MarScaper HDR web site (http://www.marscaper.com).
I hope I'll be able to add Dual Iso support to my roadmap. I have already started studying your PDF to understand all the machinery. Very smart approach. If I find an idea to fight against artifacts, I'll share it with pleasure.
The current algorithms are far more advanced than what's in the paper, but the background info still applies; these days, even in highlights where you only have half-res data, the interpolation is clean in many practical cases (except for resolution charts and power lines). But of course, any improvement is welcome.
100 bracketed pictures? Come on... take a look at my experiments with CeroNoice (http://www.magiclantern.fm/forum/index.php?topic=9581) ;)
Quote from: a1ex on January 13, 2014, 04:00:05 PM
100 bracketed pictures? Come on... take a look at my experiments with CeroNoice (http://www.magiclantern.fm/forum/index.php?topic=9581) ;)
In normal "human" conditions it has no interest of course. It's more a marketing tool. ;D
But for more specifics conditions (High Iso, very low signal)...
(http://www.poisson-lune.com/astro20d/galerie_photos/galaxies/m51/files/page13_1.jpg) (http://www.poisson-lune.com/astro20d/galerie_photos/galaxies/m51/assets/M51_50_pourcent.jpg)
EOS 20D - 160x30s - 1600 Iso
And of course, some astronomical subjects have also a very high dynamic range: Andromeda Galaxy, Orion nebula, etc.
Quote from: MarS on January 13, 2014, 06:35:20 PM
In normal "human" conditions it has no interest of course. It's more a marketing tool. ;D
But for more specifics conditions (High Iso, very low signal)...
(...)
And of course, some astronomical subjects have also a very high dynamic range: Andromeda Galaxy, Orion nebula, etc.
Maybe it could help:
According to
Barton, H. and Byrne, K. Introduction to Human Vision, Visual Defects & Eye Tests (March 2007), a exercised human vision is capable of a DR about ~60db (~20 stops) in an instant field of view.
So, on a base of ~10 stops per frame (basic DSLR capability) and with a overlap range, no needs to take 100 frames to get a HDR image closed to the human vision.
About astronomical imaging, the number of frames is mainly to increase the SNR, not the DR.
For most objects _even those mentioned here_ a dozen of frames is good enough to render the full dynamic, especially with modern multiscale HDR processings.
I think 60dB is about 10 stops.
http://en.wikipedia.org/wiki/Human_eye
QuoteDynamic range
The retina has a static contrast ratio of around 100:1 (about 6.5 f-stops). As soon as the eye moves (saccades) it re-adjusts its exposure both chemically and geometrically by adjusting the iris which regulates the size of the pupil. Initial dark adaptation takes place in approximately four seconds of profound, uninterrupted darkness; full adaptation through adjustments in retinal chemistry (the Purkinje effect) is mostly complete in thirty minutes. Hence, a dynamic contrast ratio of about 1,000,000:1 (about 20 f-stops) is possible.
http://en.wikipedia.org/wiki/Dynamic_range
Factor (power) 1,000,000 = Decibels 60 = Stops 19.9
Quote from: SpcCb on January 15, 2014, 02:57:23 AM
About astronomical imaging, the number of frames is mainly to increase the SNR, not the DR.
DR is computed as log2(full well) - log2(noise in complete darkness). So, if you increase SNR by stacking identical exposures, you will also increase DR.
For example, if you stack 64 identical exposures, you get 8x less noise (stdev), so your DR increases with 3 stops.
Quote from: stevefal on January 15, 2014, 06:52:03 AM
I think 60dB is about 10 stops.
You're using the amplitude definition of decibels, not the power or intensity definition which is 10 * log10(ratio). If you have field amplitudes then the equation becomes 10 * log10 ( ratio^2 ) = 20 * log10(ratio), but that is incorrect to use here, we're measuring the intensity of the light, not the amplitude of the field.
20ev = 2 ^ 20 = 1048576
10 * log10(1048576) = 60.2059991328 dB
Quote from: a1ex on January 15, 2014, 08:27:31 AM
For example, if you stack 64 identical exposures
But ... with astrophotography, would those really be identical exposures, as everything is moving. Or would you need to be on a rotating, motor-driven tripod like telescopes use? (or is that assumed for this discussion?)
BTW ... impressive/intimidating math ...
Some more:
http://www.statlect.com/normal_distribution_linear_combinations.htm
http://robocup.mi.fu-berlin.de/buch/kalman.pdf
(I should actually apply these formulas in CeroNoice)
Quote from: a1ex on January 15, 2014, 08:27:31 AM
DR is computed as log2(full well) - log2(noise in complete darkness). So, if you increase SNR by stacking identical exposures, you will also increase DR.
For example, if you stack 64 identical exposures, you get 8x less noise (stdev), so your DR increases with 3 stops.
Of course you can increase DR by increasing SNR.
But the main goal of stacking is about SNR gain, not DR.
I can't confirm your reasoning about the link SNR/DR (huge offtopic in perspective :D), but in most of cases it's not relevant to run 64 frames acquisition just to gain 3 DR stops in astrophotography.
Quote from: l_d_allan on January 15, 2014, 02:29:25 PM
But ... with astrophotography, would those really be identical exposures, as everything is moving. Or would you need to be on a rotating, motor-driven tripod like telescopes use? (or is that assumed for this discussion?)
(...)
Your question is very interesting: I did not see anybody ask 'and what about dithering frames?' or other calculated miss-alignment pre-processing to drastically reduce noises (please not the 's' ending noise).
However, it's a bit offtopic...
Not sure if I should put this on the EOS-M thread or here, but here goes: Has anyone been getting these dots? http://i.imgur.com/g2wRO2H.png
It appears to be the EOS-M phase detection pixels. And it only appears on some photos. Any ideas what's going on/how to fix it?
CR2 and converted DNG:
https://www.dropbox.com/s/joajfhtsrtpbxad/DualISOPinkDot.zip
Running latest TL and CR2HDR
Old issue, look for pink dot remover. Also this is apparently a problem on 650D/700D too.
A chroma blur (median filter on R-G and B-G) on the input image should fix it. Of course, the code must be modified to only average pixels from the same channel *and* ISO.
So much to say about astronomy! Enough to spend many nights but my english is not good enough I think! :)
To stay on topic, my first test is pretty conclusive even if I have to improve many things...
http://www.youtube.com/watch?v=I8oiPVUCkss
My approach is may be a little bit different: I extract the two frames and inject them in my HDR fusion.
Tried that with enfuse, but resolution was not exactly the best. Check the PDF at recovering full-res detail; the general idea still holds, but the exposure matching was improved a lot.
I've updated the cr2hdr binary from the first post to consider two white levels (one for each exposure), and also lowered the safety margin. You will need it if you use Dual ISO with the new ADTG trick, and it might also squeeze a tiny bit of more highlight detail from plain dual ISO shots too.
As a side effect, some of your shots might become a little darker (because of the extra highlight detail) or brighter (if you have really underexposed them). Just drag the exposure slider when postprocessing.
Don't expect miracles though - most of my test shots did not change at all.
Source code on request, or after 3 weeks, as promised.
Quote from: a1ex on January 23, 2014, 12:11:53 PMYou will need it if you use Dual ISO with the new ADTG trick, and it might also squeeze a tiny bit of more highlight detail from plain dual ISO shots too.
Ugh - why only a tiny bit of more dr - or didn't you adjust the dual_iso module for mini_iso yet? Shouldn't the effect add up, i.e. 2x0.5ev improvement = ~1ev of more added gain for dual_iso :-o ... at least that's why I though, I hope I didn't get it completely wrong?
Btw: *Request* source code link, thanks.
The only real change was a more robust white level detection. The old one simply failed to detect the white level properly on my test shot and defaulted to 12500 instead of over 15000. On other test shots it might just work (I only tried one).
dual_iso.mo works out of the box; the only update needed is on the UI side (to print ISO 77/1230 instead of 100/1600).
# HG changeset patch
# User alex@thinkpad
# Date 1390472019 -7200
# Branch unified
# Node ID d0ac7698f856d247ee874ffd61b7f77c23758406
# Parent b41e4e47a8b0f6c2271216285807d44634ac25fd
cr2hdr: experimental white level routine with two separate levels for each exposure
diff -r b41e4e47a8b0 -r d0ac7698f856 modules/dual_iso/cr2hdr.c
--- a/modules/dual_iso/cr2hdr.c Wed Jan 22 11:17:25 2014 +0200
+++ b/modules/dual_iso/cr2hdr.c Thu Jan 23 12:13:39 2014 +0200
@@ -293,7 +293,7 @@
static int hdr_interpolate();
static int black_subtract(int left_margin, int top_margin);
static int black_subtract_simple(int left_margin, int top_margin);
-static int white_detect();
+static void white_detect(int* white_dark, int* white_bright);
static inline int raw_get_pixel16(int x, int y) {
unsigned short * buf = raw_info.buffer;
@@ -493,7 +493,6 @@
if (hdr_check())
{
- white_detect();
if (!black_subtract(left_margin, top_margin))
printf("Black subtract didn't work\n");
@@ -544,7 +543,7 @@
return 0;
}
-static int white_detect_brute_force()
+static void white_detect(int* white_dark, int* white_bright)
{
/* sometimes the white level is much lower than 15000; this would cause pink highlights */
/* workaround: consider the white level as a little under the maximum pixel value from the raw file */
@@ -553,10 +552,11 @@
/* ignore hot pixels when finding white level (at least 50 pixels should confirm it) */
- int white = raw_info.white_level * 5 / 6;
- int whites[8] = {white+500, white+500, white+500, white+500, white+500, white+500, white+500, white+500};
- int maxies[8] = {white, white, white, white, white, white, white, white};
- int counts[8] = {0, 0, 0, 0, 0, 0, 0, 0};
+ int white = 10000;
+ int whites[2] = {0, 0};
+ int maxies[2] = {white, white};
+ int counts[2] = {0, 0};
+ int safety_margins[2] = {100, 2000}; /* use a higher safety margin for the higher ISO */
int x,y;
for (y = raw_info.active_area.y1; y < raw_info.active_area.y2; y ++)
@@ -564,7 +564,7 @@
for (x = raw_info.active_area.x1; x < raw_info.active_area.x2; x ++)
{
int pix = raw_get_pixel16(x, y);
- #define BIN_IDX ((y%4) + (x%2)*4)
+ #define BIN_IDX is_bright[y%4]
if (pix > maxies[BIN_IDX])
{
maxies[BIN_IDX] = pix;
@@ -575,30 +575,26 @@
counts[BIN_IDX]++;
if (counts[BIN_IDX] > 50)
{
- whites[BIN_IDX] = maxies[BIN_IDX];
+ whites[BIN_IDX] = maxies[BIN_IDX] - safety_margins[BIN_IDX];
}
}
#undef BIN_IDX
}
}
- //~ printf("%8d %8d %8d %8d %8d %8d %8d %8d\n", whites[0], whites[1], whites[2], whites[3], whites[4], whites[5], whites[6], whites[7]);
- //~ printf("%8d %8d %8d %8d %8d %8d %8d %8d\n", counts[0], counts[1], counts[2], counts[3], counts[4], counts[5], counts[6], counts[7]);
+ /* no confirmed max? use unconfirmed ones */
+ if (whites[0] == 0) whites[0] = maxies[0] - safety_margins[0];
+ if (whites[1] == 0) whites[1] = maxies[1] - safety_margins[1];
+
+ //~ printf("%8d %8d\n", whites[0], whites[1]);
+ //~ printf("%8d %8d\n", counts[0], counts[1]);
- int white1 = MIN(MIN(whites[0], whites[1]), MIN(whites[2], whites[3]));
- int white2 = MIN(MIN(whites[4], whites[5]), MIN(whites[6], whites[7]));
- white = MIN(white1, white2);
- raw_info.white_level = white - 500;
- printf("White level : %d\n", raw_info.white_level);
- return 1;
+ *white_dark = whites[0];
+ *white_bright = whites[1];
+
+ printf("White levels : %d %d\n", *white_dark, *white_bright);
}
-static int white_detect()
-{
- return white_detect_brute_force();
-}
-
-
static int black_subtract(int left_margin, int top_margin)
{
if (debug_black)
@@ -1028,7 +1024,7 @@
{
/* guess ISO - find the factor and the offset for matching the bright and dark images */
int black = raw_info.black_level;
- int white = raw_info.white_level;
+ int white = MIN(raw_info.white_level, *white_darkened);
int w = raw_info.width;
int h = raw_info.height;
@@ -1492,7 +1488,10 @@
if (!identify_bright_and_dark_fields(rggb))
return 0;
-
+
+ int white_bright = white;
+ white_detect(&white, &white_bright);
+
double noise_std[4];
double noise_avg;
for (y = 0; y < 4; y++)
@@ -1514,6 +1513,9 @@
bright_noise *= 4;
dark_noise_ev += 2;
bright_noise_ev += 2;
+ white *= 4;
+ white_bright *= 4;
+ raw_info.white_level = white;
/* dark and bright exposures, interpolated */
unsigned short* dark = malloc(w * h * sizeof(unsigned short));
@@ -1578,14 +1580,14 @@
//~ printf("Histogram matching...\n");
/* estimate ISO difference between bright and dark exposures */
double corr_ev = 0;
- int white_darkened = white;
+ int white_darkened = white_bright;
int ok = match_histograms(&corr_ev, &white_darkened);
if (!ok) goto err;
/* estimate dynamic range */
double lowiso_dr = log2(white - black) - dark_noise_ev;
- double highiso_dr = log2(white - black) - bright_noise_ev;
+ double highiso_dr = log2(white_bright - black) - bright_noise_ev;
printf("Dynamic range : %.02f (+) %.02f => %.02f EV (in theory)\n", lowiso_dr, highiso_dr, highiso_dr + corr_ev);
/* correction factor for the bright exposure, which was just darkened */
Quote from: a1ex on January 23, 2014, 12:35:47 PM
On other test shots it might just work (I only tried one).
Any dual_iso shots of mine in the ADTG thread, were with mini_iso.
I believe the only side effect in the old one was - in some cases - white level too low in the output DNG, and wrong numbers printed in the log. If the raw processing software is respecting the EXIF white level religiously (like ufraw does), this bug could have resulted in highlight clipping too (negating the advantage of mini_iso).
Hi, It's there has any way to use the ETTR with DUAL_ISO when Recovery ISO is - EV?
because some time I shot on the street in afternoon,
have to use ISO like 1600 or higher to get the shuter speed.
but it's seem not work if the recovery ISO at - EV.
English is not my native language, Plz Forgive me if there have some bad grammar or typo. :D
again, Very thanks to Dev team for the amazing job!
Last time I've used this, it was working. See http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
Keep in mind that properly exposed (unclipped) ISO 1600 is going to be cleaner than ISO 1600/100. Dual ISO works best when you expose to the right for ISO 100, not for a higher ISO.
Sorry for the incoherent,
But after read the info. "Take a test picture.Next pic will be ETTR"
Maybe I need to check my Eyes now.... lol
Everything works perfectly now.
With dual ISO, I'd use ETTR in LiveView (on SET). With "always on" it may require many test pictures until it settles (and in LV it has more freedom to try various settings).
;D Thanks, The advise are very useful to me.
Quote from: a1ex on January 23, 2014, 12:11:53 PM
if you use Dual ISO with the new ADTG trick, and it might also squeeze a tiny bit of more highlight detail from plain dual ISO shots too
I did a search on "ADTG" and found some threads that were way over my non-dev head. I tried searching for "ADTG trick" and didn't come across anything that explained what the trick was. I infer it has something to do with Dual-ISO, and perhaps some parameter in cr2hdr.exe.
Is this something that if you have to ask what it stands for, you have no business trying it? Or re-read the .pdf?
Quote from: l_d_allan on January 24, 2014, 10:55:29 PM
I did a search on "ADTG" and found some threads that were way over my non-dev head. I tried searching for "ADTG trick" and didn't come across anything that explained what the trick was. I infer it has something to do with Dual-ISO, and perhaps some parameter in cr2hdr.exe.
Is this something that if you have to ask what it stands for, you have no business trying it? Or re-read the .pdf?
http://www.magiclantern.fm/forum/index.php?topic=10111.0
http://magiclantern.wikia.com/wiki/ADTG
EDIT: Found it, nevermind. Its in the "EXPO" tab. Sorry for a newbie question.
"Usage
- 5D3: works for both raw photo and raw video. 7D: photo only.
- Select one ISO from Canon menu, the other from Dual ISO submenu.
- Expose to the right for the lower ISO (usually ISO 100). Maybe darken 1 stop from there.
- Start at ISO 100/800.
- Try a larger gap for really extreme situations, or a smaller gap for better midtone detail (less aliasing)."
I can't find the submenu for Dual ISO :/ I have a Canon 7d with this firmware installed and activated dual_iso in the modules menu and restarted my camera, but where is the dual iso submenu? Tried clicking the Q button but there is only some information about the author.
Quote from: carpediem86 on January 25, 2014, 04:41:21 PM
Tried clicking the Q button but there is only some information about the author.
Try cycling through the top icon bar with the wheel :-> and find the expo tab ... ML is loaded with features which makes it confusing for newbies, but you can hide all options you don't need with the "customize menus" option (you'll stumble upon it sooner or later in one of the menus :-)).
Quote from: ted ramasola on January 25, 2014, 12:35:57 AM
http://www.magiclantern.fm/forum/index.php?topic=10111.0
Thanks ... impressive, but complicated stuff. Looks promising.
Quotehttp://magiclantern.wikia.com/wiki/ADTG
Thanks again. I neglected to look there ... my bad.
Quote from: a1ex on January 23, 2014, 12:35:47 PMdual_iso.mo works out of the box; the only update needed is on the UI side (to print ISO 77/1230 instead of 100/1600).
I've got 3 dual_iso + mini_iso files that refuse to convert with current/patched cr2hdr, two show the same lines while one results in a checkerboard look I've never seen before. I'll pm you the link to the cr2 files & I'm posting the information here in case I'm not the only one.
Patch updated; binary... not yet.
Quote from: a1ex on January 26, 2014, 10:07:55 PM
Patch updated; binary... not yet.
Thanks, this fixes it - binary until you upload the official one: https://bitbucket.org/Marsu42/ml-aiso/downloads/cr2hdr.exe
Concerning the cr2hdr chroma smoothing options: Is there some rule of thumb when to use what (yes, I know the default is probably the default for a reason)?
But might it be also sound to start with --no-cs and then add more --cs when aliasing (how does it look exactly?) occurs, or is the gain of --no-cs too small to be worth it, or is the --no-cs option even only there for speed?
Hi everyone, I noticed if use the dual_iso with the -EV (e.g. 800/100), will get the images like this (see the bottom) :
(https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-ash3/1601035_642540619114902_1859984542_n.jpg)
but no problem with the +EV(100/800)
(https://fbcdn-sphotos-g-a.akamaihd.net/hphotos-ak-prn1/t1/11498_642540622448235_44620911_n.jpg)
Is it normal or just get the wrong setting?
Quote from: joeexyz on January 29, 2014, 05:06:25 PM
Is it normal or just get the wrong setting?
Looks like a bug, the best course of action is to submit a bug ticket to to the ml tracker with camera model, ml version, what you did yadayadayada. Probably no one noticed yet because the overwhelming majority shoots with +ev settings...
I think that the reason is that you set the exposure according to ISO 800 but cr2hdr doesn't distinguish between 800/100 and 100/800, so to it the photo looked like an underexposed 100/800. Using -ev only makes sense if you overexpose to around ETTR + 1 stop. Whether you use +ev or -ev the higher ISO has to be for the shadows and therefore has to blow out the highlights.
@a_d_:
i have just read that some new dual iso conversion code is available..
is there an update for the mac tool planned in the near future? would be really great.
thx.
Is there a way to have dual iso enabled in photo mode and not in video mode? Im always switching between both and that would be a time saver.
Quote from: arturochu on January 31, 2014, 01:51:55 AM
Is there a way to have dual iso enabled in photo mode and not in video mode?
Nope, but you could add a feature request so the idea can be discussed on its own and doesn't get lost along the way. It's easy enough to implement, but would add yamlmo (yet another ml menu option).
First time poster here.
I would like to start by saying a super huge THANK YOU to the Magic Lantern team, i've been hesitating to make the jump, but a spit of inspiration hit me last night and I installed ML and i'm VERY impressed!!
On the Dual ISO topic, I have a question.
Maybe i'm not doing something right, but....
In the EXPO menu, where the option for Dual ISO lay, if the camera's iso is at (lets say) 400, I want to do 3 stops up (3200) to get a good amount of low light coverage. I enable this, but when I exit out of that menu, it still reads 400/1600. Then, if I goto the main ML menu, and go all the way to the right, it shows me everything thats modified (which is only DUAL iso) and in this menu it reads DUAL ISO 400/1600 (recovery ISO 3200) under that.
Also, I've noticed that the "Dynamic Range Gained" never goes up from 1.1EV when I change the recovery ISO from 1600 and up.
So I apologize for the noob question, but..
If i'm setting my Dual ISO at 3200 Recovery ISO,
Is it really stuck at 1600 ISO, or is it actually doing 400/3200?
Quote from: awesnap on January 31, 2014, 05:28:47 PM
Is it really stuck at 1600 ISO, or is it actually doing 400/3200?
It's really doing ISO 400/1600. You haven't specified what camera model you're using, but the available recovery ISOs are restricted depending on the particular model. I know that my 50D tops out at ISO 1600. The technical reason for this restriction is that Dual ISO is implemented by changing the CMOS-gain registers, while the ISOs greater than 1600 on the 50D have the same CMOS gain as ISO 1600 (a different amplifier gets you to the higher ISOs).
That and ISOs above 1600 on most cameras are useless.
Quote from: ayshih on January 31, 2014, 05:48:29 PM
It's really doing ISO 400/1600.
I admit the necessity for this is beyond me, if I wouldn't be so busy atm I'd add a patch to my local ml that locks dual_iso @iso100 - if I need faster shutter speeds the return of invest diminishes and I simply bite the bullet and shoot in default raw setting w/o the need for cr2hdr postprocessing time and white balance hassle.
I'm happy with 100/800. And I suspect with the ADTG gains, using 200 or even 400 ISO as the base won't be so bad, it will just be that pesky photon noise.
To me, postprocessing isn't a hassle, having to redo the shoot or dig detail out of the noise is :P
edit: BTW, CMOS values 663, 773, 883, 993 produce some other ISO values. I can't recall exactly, but with 1 of them I got ISO 300 and another looked to be over ISO 3200.
edit2: I would have been using 603, 703, 803, 903 for the recovery ISO.
Quote from: Audionut on January 31, 2014, 08:02:24 PM
I'm happy with 100/800. And I suspect with the ADTG gains, using 200 or even 400 ISO as the base won't be so bad, it will just be that pesky photon noise.
With recent cr2hdr versions, I also have good experiences with 100/1600 resulting in 14.5ev dr, that's on 6d...
... at for the effect of mini_iso I still have no clue at all how much this will really boost dual_iso, but I hope sooner or later alex will tell the user crowd :-). In the next days, I'll calibrate the 6d for the latest mini_mo so at least this part is covered on all current camera models.
Quote from: Audionut on January 31, 2014, 08:02:24 PM
edit: BTW, CMOS values 663, 773, 883, 993 produce some other ISO values. I can't recall exactly, but with 1 of them I got ISO 300 and another looked to be over ISO 3200.
Yes, would be very nice to map their approximate ISO and dynamic range and seeing if they are worth using. I guess raw spotmeter on a static scene should be enough to estimate the ISO.
Quote from: Marsu42 on January 31, 2014, 08:11:30 AM
...yamlmo
Config presets. They need some major refactoring to remove the restart requirement (basically, attaching the toggle functions to the config entries instead of menu entries), but it's doable (as long as the two presets use exactly the same set of modules loaded). Module reloading on the fly is next to impossible to do without resource leaks IMO.
Quote from: Marsu42 on January 31, 2014, 08:30:13 PM
... at for the effect of mini_iso I still have no clue at all how much this will really boost dual_iso
I have a clue here: http://www.magiclantern.fm/forum/index.php?topic=10111.msg97541#msg97541
Quote from: a1ex on January 31, 2014, 09:01:50 PM
Yes, would be very nice to map their approximate ISO and dynamic range and seeing if they are worth using.
I was getting different results up through 1133, 2233 etc also. I haven't had a chance to look at the data in a reasonable fashion and mainly just mentioned it so I didn't forget about it.
Here is the output of 1 of them (I think this is 663 (603)).
Noise levels : 5.10 4.29 4.32 5.06 (14-bit)
ISO difference : 2.51 EV (569)
Black delta : -3.05
Dynamic range : 11.32 (+) 11.07 => 13.58 EV (in theory)
ISO overlap : 5.8 EV (approx)
Noise level : 3.16 (16-bit), ideally 3.16
Dynamic range : 13.76 EV (cooked)I was using ADTG gains and CMOS[4] so you can ignore the ISO value. But this is a half stop between 2 EV and 3 EV. The DR numbers were in the middle of the full stop EVs also :)
My first thought was increasing the ISO overlap. For me, 100/1600 is generally a little to far apart, 100/800 generally gives me almost the same results (maybe I don't dig into the shadows enough, or I am more inclined to just crush them in post), well we get 0.4 EV back in the midtones with ADTG, so if I can find an ISO difference of 3.5 EV this should give me almost the same DR as 100/1600 with another 0.4 EV (0.8 EV total vs Canon ISO) in the midtones.
Here are the ISO results either side of the above.
Noise levels : 4.77 4.31 4.29 4.76 (14-bit)
ISO difference : 1.96 EV (388)
Black delta : -9.79
Dynamic range : 11.32 (+) 11.17 => 13.13 EV (in theory)
ISO overlap : 6.4 EV (approx)
Noise level : 4.69 (16-bit), ideally 4.69
Dynamic range : 13.19 EV (cooked)Noise levels : 5.52 4.30 4.32 5.54 (14-bit)
ISO difference : 2.97 EV (781)
Black delta : -1.11
Dynamic range : 11.32 (+) 10.95 => 13.92 EV (in theory)
ISO overlap : 5.4 EV (approx)
Noise level : 2.58 (16-bit), ideally 2.58
Dynamic range : 14.06 EV (cooked)I haven't had a chance to look at the actual output yet and judge differences. It obviously needs further investigation, but not sure I will have the time until next week.
edit: Disregarding dual_iso, I think these other ISOs will be useful in general shooting also. With real half stop ISOs, there should be no excuse for not getting the exposure nailed in the field :D
Quote from: Marsu42 on January 31, 2014, 08:30:13 PM
With recent cr2hdr versions, I also have good experiences with 100/1600 resulting in 14.5ev dr, that's on 6d...
Yes, but are you just trusting the DR numbers or judging the output?
100/800 vs 100/1600 gives me these
typical results.
Noise levels : 5.52 4.30 4.32 5.54 (14-bit)
ISO difference : 2.97 EV (781)
Black delta : -1.11
Dynamic range : 11.32 (+) 10.95 => 13.92 EV (in theory)
ISO overlap : 5.4 EV (approx)
Noise level : 2.58 (16-bit), ideally 2.58
Dynamic range : 14.06 EV (cooked)Noise levels : 7.57 4.34 4.35 7.54 (14-bit)
ISO difference : 3.97 EV (1563)
Black delta : 0.76
Dynamic range : 11.31 (+) 10.50 => 14.47 EV (in theory)
ISO overlap : 4.3 EV (approx)
Noise level : 1.77 (16-bit), ideally 1.77
Dynamic range : 14.60 EV (cooked)(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/800.jpg) (https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/1600.jpg)
I apparently gained 0.54 EV in the shadows which I can't see, but I lost 1 EV of midtone detail.
There is a point in the shadows where a pattern noise emerges, and no amount of higher ISO for the recovery seems to fix this. I assume it's a result of not being able to get any more detail from the base ISO. ie: You suffer aliasing in the highlights (only 1 ISO with detail), and this pattern noise is a result of only 1 ISO having good
useable detail in the shadows.
Would be very interesting to plot the SNR with a dual ISO shot. However, you can't use my SNR algorithm for this, because the dual ISO output is basiclly an adaptive blend of the two ISOs. So, the blending will go towards full-res image - both ISOs - in high-detail areas to minimize aliasing and bring back resolution, and will go towards the higher ISO to minimize noise.
SNR in the log is estimated from optical black area (which gets processed in the same way as the entire image). There, the blending factor is almost entirely biased to the higher ISO (you can check by comparing output noise level with the "ideally" value, which is from the higher ISO only).
So, probably the cleanup effect should be obvious in out-of-focus and really dark areas. The estimated DR is likely the best-case improvement (I should probably simulate the worst-case one too).
Quick check with a recent test shot from Marsu.
Quote
Dynamic range : 11.88 (+) 11.32 => 14.36 EV (in theory)
Dynamic range : 14.60 EV (cooked) // defaults
Dynamic range : 12.34 EV (cooked) // hacked cr2hdr to output fullres only
So... the real improvement was between 0.45 and 2.25?
(estimated improvement from dual ISO menu was probably 2.44 from my offline simulation at 100/800 with DxO data)
Some SNR testing would show exactly what's going on. To get the ground truth image, you can use CeroNoice with a bracketed set of images like the ones from engardeknave. You may even build fake dual ISO DNGs from two bracketed shots, just to study the behavior of the converter (I'll probably try it).
edit: here's a quick and dirty tool for creating fake dual ISOs from two regular images: fake_dual_iso.exe (http://a1ex.magiclantern.fm/bleeding-edge/fake_dual_iso.exe)
Quote from: Audionut on February 01, 2014, 07:20:04 AM
Yes, but are you just trusting the DR numbers or judging the output?
I'm *only* judging the output, and maybe there's something to be gained here by heavier "single-shot-hdr" style postprocessing outside acr and 100% proper auto-ettr exposure, I usually just underexpose manually with dual_iso which is quickest for dynamic scenes.
When I enable dual_iso it's for a specific purpose: to capture at least some details in blown whites, either in a sunlit fur rim or a white spot on a dark-grey horse - and using 100/1600 instead of 100/800 lets me underexpose some more. I never use dual_iso "just" to improve the general shadow noise, for that the postprocessing time, lower detail, dual storage cr2+dng and the current wb/tint problems isn't "worth it" to me personally.
Quote from: a1ex on February 01, 2014, 08:05:01 AMfake_dual_iso.exe (http://a1ex.magiclantern.fm/bleeding-edge/fake_dual_iso.exe)
Um, zis ist an elf application, Mr. Torvalds, if you name it as .exe you're creating some expectancy under the Windows guys around here :-p
F5
Quote from: Marsu42 on February 01, 2014, 09:31:19 AM
When I enable dual_iso it's for a specific purpose: to capture at least some details in blown whites, either in a sunlit fur rim or a white spot on a dark-grey horse - and using 100/1600 instead of 100/800 lets me underexpose some more. I never use dual_iso "just" to improve the general shadow noise,
ISO doesn't fix photon noise.
But yes, dual_iso is excellent for small changes in the EV of the scene. Expose for the highlights and let dual_iso take care of the shadows. Lazy mans exposure :P
Now that I have a ground truth image, I ran a quick test on the interpolation methods to see how much resolution you are really losing in the extreme highlights or shadows (worst case):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/ground-truth.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/nointerp.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/mean23.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/amaze-edge.jpg)
1) ground truth (used as the high-ISO image)
2) half-resolution image (2 lines sampled, 2 lines discarded and interpolated as nearest neighbour)
3) mean23 interpolation (some sort of bilinear; fast).
4) amaze-edge interpolation (default setting in cr2hdr)
Images 3 and 4 were both interpolated from 2.
Full DNGs:
DUAL7603.CR2 (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/DUAL7603.CR2) from engardeknave as ground truth
out.dng (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/out.dng) (fake dual ISO, open with dcraw)
nointerp.dng (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/nointerp.dng)
mean23.dng (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/mean23.dng)
amaze-edge.dng (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/amaze-edge.dng)
To get these DNGs, I've modified the cr2hdr source to disable all other processing except interpolation.
To get these crops:
ufraw-batch --temperature=4000 --exposure=4.5 DUAL7603.CR2 --out-type=ppm --output=- | convert - -crop 500x500+1790+1435 ground-truth.jpg
ufraw-batch --temperature=4000 --exposure=5.5 amaze-edge.dng --out-type=ppm --output=- | convert - -crop 500x500+1790+1435 amaze-edge.jpg
ufraw-batch --temperature=4000 --exposure=5.5 mean23.dng --out-type=ppm --output=- | convert - -crop 500x500+1790+1435 mean23.jpg
ufraw-batch --temperature=4000 --exposure=5.5 nointerp.dng --out-type=ppm --output=- | convert - -crop 500x500+1790+1435 nointerp.jpg
And now, a clear proof that we have reached the 16-bit limit (so adding floating-point DNG output may be a good idea). Here's the output from amaze-edge, first image is the same as before, the others were darkened before saving the DNG by 2, 3 and 4 stops (as it happens with ISO 100/400, 100/800 and 100/1600):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/amaze-edge-0ev.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/amaze-edge-2ev.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/amaze-edge-3ev.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/amaze-edge-4ev.jpg)
edit: half of the issue was that my modified ufraw sacrifices 2 shadow bits for clip=film, so I've re-created the images with --clip=digital which doesn't have the bug. The issue is still there, but not quite as bad.
ufraw-batch --clip=digital --temperature=4000 --exposure=5.5 amaze-edge.dng --out-type=ppm --output=- | convert - -crop 500x500+1790+1435 amaze-edge-0ev.jpg
ufraw-batch --clip=digital --temperature=4000 --exposure=7.5 amaze-edge-2ev.dng --out-type=ppm --output=- | convert - -crop 500x500+1790+1435 amaze-edge-2ev.jpg
ufraw-batch --clip=digital --temperature=4000 --exposure=8.5 amaze-edge-3ev.dng --out-type=ppm --output=- | convert - -crop 500x500+1790+1435 amaze-edge-3ev.jpg
ufraw-batch --clip=digital --temperature=4000 --exposure=9.5 amaze-edge-4ev.dng --out-type=ppm --output=- | convert - -crop 500x500+1790+1435 amaze-edge-4ev.jpg
Full DNGs:
amaze-edge-2ev.dng (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/amaze-edge-2ev.dng)
amaze-edge-3ev.dng (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/amaze-edge-3ev.dng)
amaze-edge-4ev.dng (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/amaze-edge-4ev.dng)
Repeated the experiment with DUAL7605.CR2 (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/DUAL7605.CR2) as overexposed image (considered ground truth for shadows), and the previous one (DUAL7603.CR2 (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck/DUAL7603.CR2)) as the underexposed image. This example is a bit closer to reality, no more deep shadows, no more bit problems. I'm showing 100% crops.
Ground truth (overexposed image), and another one underexposed by 4 stops with shutter (so the overlap is similar to 100/1600, but the noise difference is roughly 1 stop larger):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck2/ground-truth.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck2/underexposed.jpg)
Interpolations:
nearest neighbour (half of the lines from overexposed images), mean23, amaze-edge:
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck2/nointerp.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck2/mean23.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck2/amaze-edge.jpg)
(notice the detail reconstructed by amaze-edge from only half of the lines)
Full-res and half-res blending:
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck2/fullres.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck2/halfres.jpg)
Full-res simply combines both exposures after scaling and offseting them to match. Notice some more detail recovered, at the expense of noise. This is the worst case performance regarding DR.
Half-res blending is a weighted average between the two exposures (after interpolation with amaze-edge), as described in the PDF, figure 6. The noise is much better, at the expense of aliasing and lost detail.
And the final output is basically a weighted average between the last two:
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck2/final-output.jpg)
ufraw-batch --temperature=4000 --exposure=4.5 DUAL7603.CR2 --out-type=ppm --output=- | convert - -crop 900x600+1400+1360 underexposed.jpg
ufraw-batch --temperature=4000 --exposure=0.5 DUAL7605.CR2 --out-type=ppm --output=- | convert - -crop 900x600+1400+1360 ground-truth.jpg
ufraw-batch --temperature=4000 --exposure=1.5 nointerp.dng --out-type=ppm --output=- | convert - -crop 900x600+1400+1360 nointerp.jpg
ufraw-batch --temperature=4000 --exposure=1.5 amaze-edge.dng --out-type=ppm --output=- | convert - -crop 900x600+1400+1360 amaze-edge.jpg
ufraw-batch --temperature=4000 --exposure=1.5 mean23.dng --out-type=ppm --output=- | convert - -crop 900x600+1400+1360 mean23.jpg
ufraw-batch --temperature=4000 --exposure=5.5 fullres.dng --out-type=ppm --output=- | convert - -crop 900x600+1400+1360 fullres.jpg
ufraw-batch --temperature=4000 --exposure=5.5 halfres.dng --out-type=ppm --output=- | convert - -crop 900x600+1400+1360 halfres.jpg
ufraw-batch --temperature=4000 --exposure=5.5 final-output.dng --out-type=ppm --output=- | convert - -crop 900x600+1400+1360 final-output.jpg
Caveats:
- This was a fake dual ISO, done with shutter bracketing, not ISO (if you have a similar bracket with ISO, let me know)
- Still, the results are valid regarding resolution and noise recovery (how close it can get to the overexposed image?)
- I'm only showing the shadow performance (relevant for noise); feel free to experiment with highlights, but I think aliasing it's the same
- For some reason, my DNG files in ufraw are rendered 1 stop darker than CR2's with the exact same content (figure out why; for now I added 1 stop of exposure compensation when developing the DNGs)
My conclusions:
- deep shadows will benefit from more bits or floating point output (from previous post)
- there is room for improvement regarding noise in low-detail areas (compare final output with half-res blending)
- resolution loss is noticeable at 1:1, so it's not a substitute for tripod bracketing (compare final output with ground truth)
- however, resolution loss in worst case should not be obvious in practice (compare amaze-edge with ground truth; note that amaze-edge interpolates from half of the lines). Also compare amaze-edge with final output and notice how resolution improves in midtones.
- when you have to underexpose in order to capture the highlights, go ahead and use it (compare final output with underexposed)
You've increased photon noise with shutter. The results would be even better without it.
The shadow areas in Full Res and Weighted Average blends I find extremely distracting. It's the noise I was describing earlier also.
If you could apply half res blending on flat textures, and apply weighted average blending on the detailed textures, with some edge detection, and weight the blending towards full res on edge detail, that should help to give the best of both worlds?
Yes, the weighted average is weighted by both exposure and local details (so on the underexposed res charts from Ted it will increase noise but it will avoid aliasing). In midtones I use only full-res, but this balance can be improved.
I would start by tweaking fullres_start, fullres_transition, and playing around with alias map. These parameters are not exposed on the command line (you can turn off alias map, but here it doesn't improve the noise, so the first two parameters are the first suspects).
If you have a set of brackets with ISO, I can use that one instead.
Quote from: a1ex on February 01, 2014, 01:33:36 PM
- resolution loss is noticeable at 1:1, so it's not a substitute for tripod bracketing (compare final output with ground truth)
- however, resolution loss in worst case should not be obvious in practice (compare amaze-edge with ground truth; note that amaze-edge interpolates from half of the lines). Also compare amaze-edge with final output and notice how resolution improves in midtones.
- when you have to underexpose in order to capture the highlights, go ahead and use it (compare final output with underexposed)
This - good summary and exactly my conclusion after using dual_iso for some time, it's terrific for moving high-dr scenes and nice for added safetly if there's no time for a 100% proper exposure.
From my shots, the resolution loss shows most in semi-blown highlights with fine details, at least my human perception isn't tuned to shadow detail - as long as it's not clipped to black, it's fine.
Quote from: Marsu42 on February 01, 2014, 07:52:46 PM
it's terrific for moving high-dr scenes
Exactly. I don't like to carry a tripod, and for hand-held shots with sun in front of you it's essential.
I'm looking into increasing the bit depth at postprocessing stage and overcoming the 16-bit limitation, stay tuned.
Quote from: a1ex on February 01, 2014, 07:59:31 PM
I'm looking into increasing the bit depth at postprocessing stage and overcoming the 16-bit limitation, stay tuned.
I hope it'll work in acr, I remember you having problems with the tags or the format was only supported in the dng sdk?
Anyway, I'm looking forward to another opportunity to re-convert my dual_iso cr2 stock, adding up how many times I did that I contributed significantly to global warming by now :-p
@A1ex - sorry for not responding sooner, and thanks for the help.
Quote@erik: it's indeed a white level problem. I should provide a way to override it in the command line, because right now it's autodetected. It's important to have it correct - or a little underestimated - during the conversion; I've changed it in white_detect_brute_force to 10000 and the output seems fine.
Is there a way to pass a saturation max and white point level argument to cr2hdr?
And is there room to add WP_delta/time variable?
1) you quoted the answer
2) what is a "WP_delta/time variable" ?
Quote from: a1ex on February 01, 2014, 07:59:31 PM
and for hand-held shots with sun in front of you it's essential.
Anything above 8 EV of dynamic range is essential. Once you get past 6 EV of DR, you start to gain benefits.
Quote from: a1ex on February 01, 2014, 03:08:47 PM
Yes, the weighted average is weighted by both exposure and local details (so on the underexposed res charts from Ted it will increase noise but it will avoid aliasing). In midtones I use only full-res, but this balance can be improved.
I would start by tweaking fullres_start, fullres_transition, and playing around with alias map. These parameters are not exposed on the command line (you can turn off alias map, but here it doesn't improve the noise, so the first two parameters are the first suspects).
The trick will be applying halfres in the shadows for the least noise, since this will increase the true DR of the scene, not just the numbers, and applying fullres on edge detail to prevent aliasing. I'll start tweaking and see if I can find something that's any better.
Quote from: ayshih on January 31, 2014, 05:48:29 PM
It's really doing ISO 400/1600. You haven't specified what camera model you're using, but the available recovery ISOs are restricted depending on the particular model. I know that my 50D tops out at ISO 1600. The technical reason for this restriction is that Dual ISO is implemented by changing the CMOS-gain registers, while the ISOs greater than 1600 on the 50D have the same CMOS gain as ISO 1600 (a different amplifier gets you to the higher ISOs).
Ok thanks. Yea sorry about that, I forgot. I have a 5d mk2
Recently I decided to try this out when I was out for a party at a museum. The dance floor was pretty much pitch black, so I decided to use a flash. I noticed during post afterward, when I load the files in Lightroom 5 and shooting people, I tend to take down the highlights, resulting in the skin gets VERY "sun-burned". More so than regular raw files (obviously, cause the highlight recovery is insanely effective now).... Besides taking down the red/yellow/orange levels to try and compensate and recover the highlights, is there anything you guys would recommend to keep the "sun-burn" under control?
I'm loving this level of control of DR I have now, its so addicting to just shoot Dual Iso all the time!! Thank you again for this awesome tool!!
Also, one other quick question I had. When exposing for a Dual ISO shot, are you guys exposing for the base ISO, or the recover ISO, or somewhere in between?
Expose to the right for ISO 100, then bump the recovery ISO to fill the shadows (400-1600, depending on how much contrast you have).
Tip: auto ETTR does it for you.
Another test with a fake dual ISO from two ISO-bracketed shots from Audionut: 100.CR2 (https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/100.CR2) and 1600.CR2 (https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/1600.CR2).
fake_dual_iso 100.CR2 1600.CR2
cr2hdr out.dng --debug-blend
mv out.DNG final-output.dng
Highlights:
ufraw-batch --temperature=3500 --green=0.8 --exposure=0.5 100.CR2 --out-type=ppm --output=- | convert - -crop 1200x800+425+2500 hi-100.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=-3.5 1600.CR2 --out-type=ppm --output=- | convert - -crop 1200x800+425+2500 hi-1600.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=1.5 fullres.dng --out-type=ppm --output=- | convert - -crop 1200x800+425+2500 hi-fullres.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=1.5 halfres.dng --out-type=ppm --output=- | convert - -crop 1200x800+425+2500 hi-halfres.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=1.5 final-output.dng --out-type=ppm --output=- | convert - -crop 1200x800+425+2500 hi-final-output.jpg
100 and 1600 (left is ground truth for highlights):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/hi-100.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/hi-1600.jpg)
full-res and half-res (reconstructed from the fake dual ISO shot)
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/hi-fullres.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/hi-halfres.jpg)
final output (reconstructed from the fake dual ISO shot)
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/hi-final-output.jpg)
Midtones:
ufraw-batch --temperature=3500 --green=0.8 --exposure=2.5 100.CR2 --out-type=ppm --output=- | convert - -crop 1200x800+1750+2650 mid-100.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=-1.5 1600.CR2 --out-type=ppm --output=- | convert - -crop 1200x800+1750+2650 mid-1600.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=3.5 fullres.dng --out-type=ppm --output=- | convert - -crop 1200x800+1750+2650 mid-fullres.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=3.5 halfres.dng --out-type=ppm --output=- | convert - -crop 1200x800+1750+2650 mid-halfres.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=3.5 final-output.dng --out-type=ppm --output=- | convert - -crop 1200x800+1750+2650 mid-final-output.jpg
100 and 1600 (ground truths):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/mid-100.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/mid-1600.jpg)
full-res and half-res (reconstructed from the fake dual ISO shot)
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/mid-fullres.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/mid-halfres.jpg)
final output (reconstructed from the fake dual ISO shot)
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/mid-final-output.jpg)
Shadows:
ufraw-batch --temperature=3500 --green=0.8 --exposure=6.5 100.CR2 --out-type=ppm --output=- | convert - -crop 1200x800+3650+2600 shad-100.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=2.5 1600.CR2 --out-type=ppm --output=- | convert - -crop 1200x800+3650+2600 shad-1600.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=7.5 fullres.dng --out-type=ppm --output=- | convert - -crop 1200x800+3650+2600 shad-fullres.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=7.5 halfres.dng --out-type=ppm --output=- | convert - -crop 1200x800+3650+2600 shad-halfres.jpg
ufraw-batch --temperature=3500 --green=0.8 --exposure=7.5 final-output.dng --out-type=ppm --output=- | convert - -crop 1200x800+3650+2600 shad-final-output.jpg
100 and 1600 (right is ground truth for shadows):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/shad-100.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/shad-1600.jpg)
full-res and half-res (reconstructed from the fake dual ISO shot)
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/shad-fullres.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/shad-halfres.jpg)
final output (reconstructed from the fake dual ISO shot)
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/rescheck3/shad-final-output.jpg)
Feel free to repeat this test on your own bracketed images (you don't need any source code modifications; simply change your crop windows, exposures, white balance, and then paste the commands in the terminal).
Tools you need: fake_dual_iso.exe (http://a1ex.magiclantern.fm/bleeding-edge/fake_dual_iso.exe), cr2hdr, ufraw (I used ufraw-mod), ImageMagick and a Unix-like shell.
Happy pixel peeping!
P.S. try getting a better blending between fullres and halfres (compared to my final output). You may try getting as close to the ground truths as you can.
Can you update fake_dual_iso to accept tiff?
I'd like to test my theory of applying noise reduction on the dark ISO.
It's not quite straightforward. I think the easiest way is to split the tif into RGB, save each one as pgm, load them in fake_dual_iso.c with the existing routine, and redo the bayering.
Here's the source if you want to try: http://a1ex.magiclantern.fm/bleeding-edge/fake_dual_iso.c
To build it, copy the Makefile rules from cr2hdr, but use this source file instead.
Even if you denoise from dxo, you will get a linear (RGB) DNG, so you need to write the code to load it.
Therefore, I believe the really easy way is to implement the denoise filter in cr2hdr (in the "dark" image, probably before interpolation). Try a bilateral filter, it's fairly easy to write and shouldn't blur edges too much.
when reviewing these pictures in the camera they will look all banded and stuff right, it wont look right till post processing?
Quote from: joebone on February 06, 2014, 04:58:57 PM
when reviewing these pictures in the camera they will look all banded and stuff right, it wont look right till post processing?
Yep.
Quote from: joebone on February 06, 2014, 04:58:57 PM
when reviewing these pictures in the camera they will look all banded and stuff right, it wont look right till post processing?
There is/was an experimental patch to display part of the picture merged in camera - it's still available in the famed Tragic Lantern :-p though at least I never got it working on the 6d, but in ML it's not in the current dual_iso module... probably for good reason.
Personally, I've gotten some training in evaluating dual_iso shots in camera (sharpness & exposure) so I'm not so desperate for an immediate preview anymore ... I'd rather have the wb in acr correct, but I hope we acr users will figure something out sooner or later.
How do I get Dual ISO to work with video on the 50D. When I turn video on, it won't let me use dual iso. It says [MOVIE] This feature requires you shooting RAW. I'm using the latest nightly and have Raw enabled... What am I doing wrong? I can only seem to shoot dual iso stills.
Unless I'm mistaken, the 50D (along with other cameras of the same generation) cannot make use of Dual ISO in movie mode because it does not use ISO registers in the same way as in photo mode. The error message is misleading because Dual ISO can be disabled because the ISO register is not defined, not just because RAW recording is not enabled.
I would have been using mini_iso on these. BL 64, WL 16380.
https://dl.dropboxusercontent.com/u/34113196/ML/_46A3766.CR2
(https://dl.dropboxusercontent.com/u/34113196/ML/_46A3766.jpg)
The same thing happens with different recovery ISOs.
Here is a preview of 100/6400 for instance.
(https://dl.dropboxusercontent.com/u/34113196/ML/_46A3784.jpg)
It converts fine with the upcoming 20-bit engine (I should clean it up and post an update).
Output black level is 240 with both engines.
Quote from: a1ex on February 11, 2014, 07:49:31 AM
It converts fine with the upcoming 20-bit engine (I should clean it up and post an update).
That's of course interesting and I'm looking forward to it, just one note: If the fp files are significantly larger than 16bit I wouldn't mind if you keep maintaining the smaller version, I'm saying this because I know some people don't care about disk space, but some do.
Maybe you'll add some branching to cr2hdr to automatically decide if 16bit is enough for this specific shot or a higher bit depth is required?
Nope, the output is still 16-bit, only the internal processing will be 20-bit.
If I'll add an option for output, that would be floating point DNG (and the default will be still 16-bit).
I didn't even think to check the 20bit version.
(https://dl.dropboxusercontent.com/u/34113196/ML/cr2hdr20.jpg)
Quote from: a1ex on February 11, 2014, 08:35:01 AM
Nope, the output is still 16-bit, only the internal processing will be 20-bit.
If I'll add an option for output, that would be floating point DNG (and the default will be still 16-bit).
Is 20-bit is a limitation?
(I'm thinking about 32-bit internal -> 32-bit integer FITS file output for linear work purposes)
BTW 16-bit floating point DNG output should be nice, indeed.
I use some lookup tables (from linear to log and back), and for this, 20-bit is already on the large side.
Floating point makes sense on CeroNoice - http://www.magiclantern.fm/forum/index.php?topic=9581
Indeed, 2*12-bit - overlap should fit in 20-bit.
Mathematically it makes sense for 3 images or more, so not the case with dual_iso.
Quote from: a1ex on February 11, 2014, 04:19:09 PMI use some lookup tables (from linear to log and back), and for this, 20-bit is already on the large side.
Does this 20bit processing obsolete the --soft-film option you recently introduced to cr2hdr, or is it still worth experimenting with that?
It looks like it might fix one problem of acr: even with highlight recovery @max some dual_iso files still have to much contrast in the highlights so I need to use tone curves in LR - which might lower iq, or is this about the same like your cr2hdr curve?
20bit is just the internal processing. The output could still contain information pushing the limits of 16bit. Here, --soft-film will help.
When I put my canon iso to 100 and dual_iso's iso to 1600, then take a photo, it comes out with horizontal lines in my LCD - as expected. However, when I go to my comp and put the CR2 into cr2HDR.exe, it processes and outputs a DNG with only iso 100 (very dark)... I try to use Camera Raw to pull it back up in brightness, but it doesn't go enough, and it is super noisy.
any ideas?
Do you have your highlights exposed close to saturation (clipping)?
No. I'm in my room with a computer monitor which is super blown out, unless I go to iso100 with the Canon ISO, which is what I did, and then ML with 1600 on the dual_iso
You should ensure that you have your base ISO (ISO 100 in this case) exposure at close to saturation. That is, the really bright bits in the scene should be at the right hand side of the histogram.
Why?
Shot noise (https://en.wikipedia.org/wiki/Shot_noise) is not affected by ISO. The recovery ISO (ISO 1600 in this case) doesn't fix the shot noise, it only cleans up the noise inherent in the camera electronics. Higher ISOs are cleaner then lower ISOs: http://www.clarkvision.com/articles/evaluation-canon-5diii/
In post processing, dual_iso files have an extended dynamic range that is pushing the limits of the 16bit output. So here, if you have your highlights exposed at say -1.5 EV below saturation, you lose out on 3 fronts.
First: Your captured scene contains an increased amount of shot noise (then it would if the scene was exposed correctly for the highlights).
Second: Your shadows are digitally stored with to few bits. <-- research bit depth/tonal range
Third: Your post processing software may struggle to correctly render the brightness.
A computer monitor in a dark room will result in a dynamic range well in excess of 11 EV. Here, the dark areas of the scene will suffer from shot noise even with the highlight exposure set correctly. Here, dual_iso will clean the electronic noise added by the camera, but cannot clean the shot noise.
Been messing around with Dual_iso raws for the past week, this totally changes my shooting style! THANK YOU!!!
Just a question, cause I can't seem to find a master list anywhere, but I saw that the 5d3 has the ability to record dual_iso raw video, but I can't seem to figure out how to enable it in the 5d2. I'm going to assume that its not doable at this time on the 5dmk2?
What about the 6d?
Hey guys, I installed magic lantern on my 6D a couple weeks ago and still can't figure out why post processing dual ISO images doesn't work. I am on a mac running lion. Got the cr2hdr, dcraw and exif tools installed. cr2hdr is in my app folder, dragged the cr2hdr installation packaged into my app folder and installed it from the to make sure that dcraw and exif tools install in the same directory. When i drag and drop a dual iso image into cr2hdr I get this message:
cr2hdr // Beta 1.4
Input file : DUAL9549.CR2
Error: dcraw output is not a valid PGM file Logfile : DUAL9549.txt
THE END
the log file created has this in it:
Input file : DUAL9549.CR2
Canon EOS 6D detected
Full size : 4104 x 4104
Active area : 4104 x 2736
I have been reading through all the threads and have been googling for an answer on how to get it to work for over a week but no success.
Please help!!!
What am I doing wrong?
Thanks in advance.
Quote from: swinxx on January 30, 2014, 11:51:19 PM
@a_d_:
i have just read that some new dual iso conversion code is available..
is there an update for the mac tool planned in the near future? would be really great.
thx.
That last commit (https://bitbucket.org/hudson/magic-lantern/commits/2f4e2df465d7481c91cf5ee9ca46f6bbaf0a6532) was back in december.
@awesnap
Hmm...How to explain?
For example:
Photo Mode
0x404b3b5c ISO 100
0x404b3b6a ISO 200
...
14 Bytes Step Register between the ISO 100 and 200, same 14 Bytes between 200 and 400 ...
Movie Mode
0x404b4590 ISO 100
0x404b4590 ISO 200
The Register is unchanged in Movie Mode, so it's not possible to have Dual ISO for video on 5D2.
@reflectphoto
I don't own a 6D, but I think you seem to use some kind of SRAW, you need to switch to RAW.
No matter what I do, on my 5D2 I can't seem to get a difference in Dual ISO. I shoot a regular raw and then do something like 100/800 iso in dual iso, convert using cr2hdr and they both look the same.
Use 100/1600 and make an interior shot covering a window area, too. Place a darker object like a wood tablett with visible wood grain in a darker corner of the frame. Expose window area properly (without blown highlights). Take a Dual-ISO shot and another one at ISO 100 without blown highlights again.
Try to process your files to make the wood's grain visible.
If you're still unable to see a difference you may want to upload the CR files to a file hoster and link them here. We will look after that one then.
Ciao
Walter
PS: Daylight necessary.
I tried this for the first time and just want to say thank you :)
It does need some getting used to and a lot of finetuning, but once I got the hang of it, the results were stunning. My trusty old 50D isn't everything but state of the art when it comes to high ISO, so I'm very surprised how clean 100/1600 and even 100/3200 look compared to single ISO shots.
There is some noise in the shadows when pushing the recovery to the extreme, but it's far less annoying because it's very regular. The loss of resolution is pretty bad, but sometimes 4M well-exposed pixels are better than 15MP of crushed shadows.
If shadow priority > highlight priority, increase ISO separation.
You may be surprised at the gains with lower recovery ISOs. Also, ISO doesn't effect shot noise.
Woaah! Dual ISO all the way! Excelente work! Love the concept, am blown away by the results! ;-)
Quote from: Audionut on February 19, 2014, 07:41:02 PM
If shadow priority > highlight priority, increase ISO separation.
You may be surprised at the gains with lower recovery ISOs. Also, ISO doesn't effect shot noise.
I'm probably not understanding how the interpolation actually works. Why should the recovery ISO not affect the noise at all? Sure, the low ISO lines will always give less noise in the lights and mid-tones, but why should the high ISO lines have no influence on noise at all? Of course, that would be mostly in the shadows, but in the end, noise doesn't just disappear.
I did another test just now and I'm surprised that there seems to be no significant difference in noise between 100/400 and 100/3200 (still a huge difference to straight 100). I can see that the separation is basically a trade-off between more dynamic range (high separation) and resolution/sharpness (low separation)... I just don't understand how the noise plays into it all or rather, why it doesn't.
If anyone is experiencing errors post-processing in Windows 7 (dcraw.exe not recognized):
Add the folder to system path, as explained:
http://geekswithblogs.net/renso/archive/2009/10/21/how-to-set-the-windows-path-in-windows-7.aspx
:)
Quote from: halbmoki on February 20, 2014, 01:19:32 PM
Why should the recovery ISO not affect the noise at all?....................but why should the high ISO lines have no influence on noise at all?
Shot noise (http://en.wikipedia.org/wiki/Shot_noise#Optics).
ISO only reduces camera induced noise from the downstream electronics (amplifier/ADC).
The sensor in the 5D3 for instance, has a reported dynamic range of 14.7 stops (http://www.clarkvision.com/articles/evaluation-canon-5diii/).
Unfortunately (for us), the downstream electronics are only capable of capturing up to 11.4 stops of dynamic range,
at any one time.
At low ISOs, the priority is towards highlights. Here, the highlights and midtones are captured with maximum quality. No highlights are ever clipped unless through exposure (shutter/aperture) decisions by the user, but the cutoff point from shot noise dominance in the image, to camera induced noise is steep. Shadow detail suffers.
As we move through higher ISOs, the priority shifts from highlight detail to shadow detail. Here, highlight detail is simply clipped 1 EV for every full stop ISO bump,
always. By increasing the gain of the downstream electronics, they become more sensitive to lower signals. This increased sensitivity means that the shadow point (of lower ISOs) gets shifted higher in the detail capturing range of the ADC. Remember, the ADC can only capture 11.4 stops at any one time, so by throwing away 1 EV at the top end, we gain some in the low end.
Unfortunately (again), the loss of highlights vs the gain in shadows is not linear. So we always throw 1 stop away at the top end, but for each increasing ISO bump we gain less in the shadows.
Consider the dynamic range measurements (http://www.clarkvision.com/articles/evaluation-canon-5diii/) for the 5D3. If the ratio between highlight loss and shadow gain was linear, the measured dynamic range for all ISOs would be the same.
In dual_iso, when we increase ISO separation, we lose 1 stop of highlight resolution,
always. And as shown above, we do not gain 1 stop of shadow detail.
If shadow detail > highlight detail, increase separation.
And this is limited to the dynamic range of the sensor,
and shot noise. Here, once we hit around 14.7 stops of dynamic range, increasing the ISO separation further simply reduces highlight resolution further, without any gain in the shadows,
at all.
My limit to noise in an ISO 100 exposure is around 8.5 stops. If the dynamic range of the scene is only 10 stops, I can choose to lose 2 stops of highlight resolution and gain the required dynamic range for the scene. What I
wouldn't do, is set the recovery ISO to 1600,
just for the sake of it, and throw away an extra 2 stops of highlight resolution, for a net result of no gain in the shadows. I only wanted 10 stops to begin with.
Where motion and/or Depth of Field requirements clamp exposure, then further consideration should be made. Here, if the exposure limits require the use of say ISO 400 for ETTR (http://www.magiclantern.fm/forum/index.php?topic=5693.0), then 2 stops of highlight detail have already been clipped. The new sensor dynamic range, for all intents and purposes, has become 12.7 stops. Not only that, but the biggest gains in the shadows have already been made, just in an attempt to push the highlights in the scene towards saturation.
Here, the ratio of highlight loss to shadow gain is even less, just for the first recovery ISO bump. Instead of losing 2 stops of highlight resolution and gaining say 1.8 stops of shadow detail with ISOs 100/400, now, for the same 2 stop difference in base/recovery ISOs, we still lose 2 stops in the highlights, but we only gain 1.4 stops in the shadows. It should be obvious at this point, that increasing the lower ISO in dual ISO use,
decreases the usefulness of dual_iso itself.
If you've made it this far, hopefully you can see how the loss of highlight resolution vs the gains in the shadows, is completely dependent on each ISO setting (base/recovery). And at some point, regardless of anything, no further detail in the shadows can be obtained.
If you don't care about resolution loss, then just set the recovery ISO to 3200 and call it a day. In an upcoming article, I will (hopefully) show ways of where you can work around the highlight resolution loss with shooting techniques.
Quote from: Audionut on February 20, 2014, 03:46:40 PM
(...)
The sensor in the 5D3 for instance, has a measured dynamic range of 14.7 stops (http://www.clarkvision.com/articles/evaluation-canon-5diii/).
Unfortunately (for us), the downstream electronics are only capable of capturing up to 11.4 stops of dynamic range, at any one time.
(...)
Be careful with comparisons; 14.7 stops DR published by Roger N. Clark is not a
measured DR, Roger took the maximum FWC (Full Well Capacity) and the lower RON (Read On Noise). However the maximum FWC value is taken from ISO 100 and the minimum RON value is taken from
ISO 25600 ISO 6400, so this number of 14.7 stops is
very theoretical (supposes that we can have maximum FWC and minimum RON in the same time/shoot) and never confirmed by experimentations, even with full calibration post process.
Plus, if we speak of RON we speak of downstream electronics, so in both of cases with this methodology we don't get the sensor full/maximum DR. And beside it's
difficult to compare DR numbers in real world with regular photography shoot conditions without take consideration of FPN (Fixed Pattern Noise).
Quote from: SpcCb on February 20, 2014, 05:03:39 PM
However the maximum FWC value is taken from ISO 100 and the minimum RON value is taken from ISO 25600,
Actually, the minimum RON value was taken from ISO 6400.
Quote(note: limit read noise to ISO that give at least 8 stops dynamic range)
Based on my own testing, I would agree with Roger's findings. This
sensor dynamic range may not be accurate to the tenth degree, however, the
intent of the post remains the same.
If the reader determines to use photographic dynamic range (the limit of noise in the shadows based on personal preference), rather then engineering dynamic range, then the sensor dynamic range measurement shifts to another point entirely, subject to personal preference.
Quote from: SpcCb on February 20, 2014, 05:03:39 PM
without take consideration of FPN (Fixed Pattern Noise).
Actually, this isn't a consideration with dual_iso for 2 reasons. First, the higher ISO minimises it's effect, secondly, it is my understanding that a1ex performs banding correction in cr2hdr.
Quote from: SpcCb on February 20, 2014, 05:03:39 PM
Plus, if we speak of RON we speak of downstream electronics, so in both of cases with this methodology we don't get the sensor full/maximum DR.
If you have evidence to show otherwise, then I look forward to your results.
Else, do we get to a point where the DR limits of the entire system are reached? Of course we do. Whether that point is at the DR limits of all components in the system is irrelevant. Semantics.
Thanks, Audionut. I think, I do understand a little more now. Not only did I misunderstand shot noise but also almost everything about how a digital camera works...
This also explains, why 100/3200 is just not useful with my 50D. No matter at which base ISO, a recovery ISO above 1600 doesn't give any increase in dynamic range anymore. The "best" useful setting for me is 100/800 with +2EV of DR and 8EV of overlapping midtones.
I guess that would give me about 13 stops of useable range at best, so stop nitpicking, you 5D3 owners ;)
Quote from: halbmoki on February 20, 2014, 05:51:32 PM
Thanks, Audionut. I think, I do understand a little more now. Not only did I misunderstand shot noise but also almost everything about how a digital camera works...
Your welcome :)
The biggest issue, IMH
umbleO, is the reliance on the exposure triangle in a digital system.
The exposure triangle weights ISO with the same priority as shutter/aperture in exposure decisions. This leads the reader to assume that ISO effects exposure.
ISO 100 - 1/100s - f/4.0
ISO 1600 - 1/1600s - f/4.0The exposure triangle tells us that these 2 settings on a digital camera will both produce the same result. Well, technically, they do produce the same rendered brightness. In actual effect, the 2 exposures are completely different. The ISO 100 exposure has a SNR 4 x greater (in the highlights/midtones) then the ISO 1600 exposure, due to the effect of shot noise.
http://en.wikipedia.org/wiki/Shot_noise
QuoteSince the standard deviation of shot noise is equal to the square root of the average number of events N, the signal-to-noise ratio (SNR) is given by:
SNR = N/sqrtN = sqrtN.
In the shadows, the SNR = infinity for the ISO 100 exposure vs the ISO 1600 exposure, until such point is reached where the exposure falls within the reduced dynamic range limits of the higher ISO.
In other words, the ISO 100 exposure delivers 10.9 stops of dynamic range for a Canon 5D3, but the ISO 1600 exposure only delivers 10.1 stops of dynamic range. So not only have you reduced the SNR of the exposure with ISO 1600 due to shot noise, but you have clipped shadow detail also.
In a film system, the exposure triangle was much more useful since changing the roll of film was a pita. In a digital system, exposure should be set simply to the limits of motion, depth of field and sensor saturation. ISO should only be used to ETTR.
Quote from: halbmoki on February 20, 2014, 05:51:32 PM
This also explains, why 100/3200 is just not useful with my 50D. No matter at which base ISO, a recovery ISO above 1600 doesn't give any increase in dynamic range anymore. The "best" useful setting for me is 100/800 with +2EV of DR and 8EV of overlapping midtones.
I guess that would give me about 13 stops of useable range at best, so stop nitpicking, you 5D3 owners ;)
Correct. ISO 3200 only gives you 0.1 stops of shadow detail vs ISO 1600. Either you clip 1 stop of highlight headroom in non dual_iso usage, or you increase highlight resolution loss by 1 stop with dual_iso. In either case, 0.1 stop of shadow detail probably wasn't worth it. The measured dynamic range results for the 50D (http://www.sensorgen.info/CanonEOS_50D.html) suggest that the same occurs for ISO 1600 vs ISO 800 also.
For me, my most useful settings are usually 100/800 also. This provides the most overall useful compromise between highlight resolution and total dynamic range. However, I have been experimenting with lower ISOs (for the recovery ISO) in situations that only require a reduced dynamic range. Here, the highlight resolution loss is less, and the dynamic range requirements of the scene are still fulfilled.
Of course, the final decision rests in the hands of the operator. Having a sound understanding of the pros vs cons increases ones ability to maximise results. :)
Quote from: Audionut on February 20, 2014, 05:14:33 PM
Actually, the minimum RON value was taken from ISO 6400.
Indeed, 14.7 is from FWC/2.5 (ISO 6400), not from FWC/2.05 (ISO 25600).
I thought Roger took the lower value as he speaks top of his table, but he took from ISO 6400 (He does not explain why, mystic stuff?).
Thank you to point this.
Quote from: Audionut on February 20, 2014, 05:14:33 PM(...)
If you have evidence to show otherwise, then I look forward to your results.
Evidence is evident and it's not only my results; a lot of analysis confirm that (ex.
Noise Analysis in CMOS Image Sensors, Hui Tian, 2000).
Quote from: SpcCb on February 20, 2014, 07:05:11 PM
(He does not explain why, mystic stuff?).
Not so mystic!
Quote(note: limit read noise to ISO that give at least 8 stops dynamic range)
I do not know why he chose this limit of 8 stops.
Quote from: SpcCb on February 20, 2014, 07:05:11 PM
Evidence is evident and it's not only my results; a lot of analysis confirm that (ex. Noise Analysis in CMOS Image Sensors, Hui Tian, 2000).
Confirms what exactly? That your nitpicking is valid because I chose to label the dynamic range limits of the system as a whole, as simply sensor dynamic range? Which still does not change the conclusion of my post mind you, and only serves to satisfy your desire for correct terminology.
If this other evidence shows the true dynamic range of the sensor, then I would ask you to extend to me, the same curtsey I provided to the reader above, in both the length of my explanation, and the linking of evidence to support my post. When this evidence is forthcoming, then I will happily change my description to better reflect a technical accuracy.
Quote from: Audionut on February 20, 2014, 07:20:42 PM
(...) That your nitpicking is valid because I chose to label the dynamic range limits of the system as a whole, as simply sensor dynamic range? Which still does not change the conclusion of my post mind you, and only serves to satisfy your desire for correct terminology.
(...)
???
I think you miss understand me.
Since DR is simply log2(saturation/stdev), then sensor dynamic range is simply log2(saturation / the stdev of the highest ISO that produces an increase in shadow detail).
QuotePlus, if we speak of RON we speak of downstream electronics, so in both of cases with this methodology we don't get the sensor full/maximum DR.
Since we are speaking of downstream electronics, then the true and accurate DR of the sensor is meaningless. With downstream electronics producing a limiting factor, the sensor could have a true and accurate DR of 100 stops for all we care.
Does the sensor having 100 stops of DR in this situation effect your exposure/ISO decisions. Of course not. Because you are limited by the DR of other components in the chain. For all intents and purposes, the sensor has a DR of 14.7 stops.
Any misunderstanding on my part, is a direct result of your short post, with counter claims that were based on a misunderstanding of the reported results. And no linking documentation to support your claims. He said, she said.
Apparently, evidence is evident. Well if that is the case, then my post is sufficient evidence ;D and that should be evident :P
I am happy to be corrected in matters of correct terminology, and accurate understanding, but I require more then simply, your word, and some riddles. :)
Either way, I have changed the post to describe the dynamic range as "reported", instead of "measured". I trust, based on the explanations I have provided, that this is sufficient, and we can move on.
I thing that all these calculations on DR increase by using Dual-ISO are overestimating the real world visual effect because they are based on per pixel data while the correct is to normalize for displayed at equal size and equal apparent detail. It's "screen" vs "print" DR in DxO's terms .. we use print DR in our (based on DxO data) comparisons don't we ?).
As soon as at the dark areas only half the pixels are used then the print-DR gain is 0.5 stops less than if all pixels were used.
So in the case of 5D3 if the per pixel "screen-DR" gain of dual ISO 100/3200 is 3.5 stops the "print-Dr" gain will be 3.0 stops.
Here's how much the DR calculations are overestimating the real-world result: http://www.magiclantern.fm/forum/index.php?topic=7139.msg99437#msg99437
I didn't do the math, but I've posted the tools needed to reproduce the results. The 0.5-stop loss does make sense though (should I update the DR estimations from ML menu with it?)
Thanks for the link A1ex, very interesting !!.
I have to clarify that what I wrote is about using mixted ISO shots and the properties of the data in the DualISO CR2s. The final result (the DualISO DNGs) additionally include some kind of software denoise and some excellent algos for Black Level normalization + line/banding elimination + .. so a part of DR is regained :)
I really hope that we could use the same on normal CR2s ..
Quote from: Audionut on February 20, 2014, 08:18:45 PM
(...)
Any misunderstanding on my part, is a direct result of your short post, with counter claims that were based on a misunderstanding of the reported results. And no linking documentation to support your claims. He said, she said.
(...)
I totally understand and I totally agree that I could not pretend to be referent in the domain. Plus I admit doing errors sometimes (human factor, and I like it BTW).
So, let's see what we are talking about...
We are speaking about
CMOS and
DR, and
DR with dual_iso.mo in this particular case.
Some times ago in my job I spoke about close stuff with guys from the Stanford University, I think it could help here:
CMOS can be schematized by a 2D matrix sensor device where pixel voltage is read out one row at a time to column storage capacitors,
then read out using the column decoder and multiplexer. Row integration times are echeloned by row/column readout time.
I'm sure a lot of persons here already know it, it's just to introduce the context to be sure we will speak about the same thing.
Beside, it will clearly be a simplified version here; if you need more precisions please see references at the end.
We would like to get the maximum DR from this device, so the first thing to do is to analyse what is reducing the DR.
In our activity _daylight photography (I mean with short exposure time) with
regular thermal conditions (I mean what nobody use his camera @-40°C here, except maybe 1 or 2 persons with me count inside)_ we can find: TN (Temporal Noise), FPN (Fixed Pattern Noise), dark signal (also called dark current, etc. but for a question of physic I prefer to use signals, currents are for electronic) and spatial sampling and low pass filtering.
TN is caused by photodetector and MOS transistor thermal, shot, and 1/f noise. It can be lumped into three additive components: Integration noise (due to photodetector shot noise), RN (Reset Noise) and RON (Read Out Noise). Noise increases with signal, but so does the SNR (Signal to Noise Ratio) and under dark conditions presents a fundamental limit on sensor DR (Dynamic Range).
FPN is the spatial variation in pixel outputs under uniform illumination due to device and interconnect mismatches over the sensor and have two components: Offset and gain. FPN is most visible at low illumination because offset FPN is more important than gain FPN). This is why with a1ex we try to introduce new pedestal definition to optimize this point.
Dark signal comes from the leakage current at the integration node (i.e. current not induced by photogeneration) due to junction and transistor leakages. It limits the image sensor DR by introducing dark integration noise (due to shot noise), varying widely across the image sensor array causing FPN that cannot be easily removed (a1ex is working on this point right know, it's a hard task) and reducing signal swing.
So we could synthesis all of this in a photo-current to output charge model:
Q
0 = Q
(i) ⊕ Q
Shot ⊕ Q
Reset ⊕ Q
Readout ⊕ Q
FPNWhere:
Q
(i) is the sensor transfer function and is given by: {1/q(it
int) electrons for 0 < i < qQ
sat/t
int & Q
sat for i >= Q
sat/t
int} with Q
sat is the well capacity
Q
Shot is the noise charge due to integration (shot noise) and has average power 1/q(i
ph+i
dc)t
intHowever, to calculate SNR and dynamic range we use the model with equivalent input referred noise current.
Since it is linear we can readily find the average power of the equivalent input referred noise I
n (i.e. average input referred noise) power, to be:
σ
In² = q²/t
int²(1/q*(i
ph+i
dc)t
intσ
r²) A² where σ
r² = σ
reset²+σ
RON²+σ
FPN²
SNR is the ratio of the input signal power to the average input referred noise power, so using the average input referred noise power expression we get:
SNR(i
ph) = 10log
10(i
ph²/(q²/t
int²(1/q*(i
ph+i
dc)t
intσ
r²))
DR is defined as the ratio of the largest nonsaturating input signal to the smallest detectable input signal.
The analog output of the camera is subsequently quantized via A/D conversion to obtain a digital image. The number of gray levels in the image and the gain of the A/D convertor are usually adjusted such that the maximum gray level i
max corresponds to the FWC (Full Well Capacity) and the minimum level i
min corresponds to the minimum signal (RON) detectable by the sensor. The process of quantization itself introduces an additional noise, but we will ignore its contribution for simplicity (sorry, too long to explain here).
The largest nonsaturating signal given by:
i
max = qQ
sat/t
int- i
dcThe smallest detectable input signal defined as standard deviation of input referred noise under dark conditions σ
In(0) (the zero here refers to i
ph = 0), which gives:
i
min = q/t
int√[1/q(i
dct
intσ
r²)]
So the DR of the digitized image can be written as:
DR = 20log
10(i
max / i
min)
Hence,
DR = 20log
10(qQ
sat/t
int- i
dc)/(q/t
int√[1/q(i
dct
intσ
r²)])
DR with dual_iso.mo means we have to look at a DR from a kind of SVE (Spatially Varying Exposure) image.
Hence, DR of an SVE camera is:
DR
SVE = 20log
10[(i
max / i
min) (e
max / e
min)]
I don't replace all variables in the equation for reading consideration, it could be too complicate here in the forum.
As we can see when we take a picture with dual_iso.mo, each exposure is uniformly quantized but the set of exposures together produce a non-uniform quantization of scene radiance (i.e. we see horizontal lines on the camera screen). As noted by Madden (B. Madden, Extended Intensity Range Imaging. Technical Report MS-CIS-93-96, Grasp Laboratory, University of Pennsylvania, 1993), this non-uniformity can be advantageous as it represents a judicious allocation of resources (bits). Though the difference between quantization levels increases with scene radiance, the sensitivity to contrast remains more or less linear. This is because contrast is defined as brightness change normalized by brightness itself.
We now determine the total number of gray levels captured by an SVE imaging system. Let the total number of quantization levels produced at each pixel be
q and the number of different exposures in the pattern be
K. Then, the total number of unique quantization levels can be determined to be:
Q = q + ∑{K-1; K=0} R [(q-1) - (q-1) (e
K / e
K-1)]
Where R
(x) rounds-off to the closest integer.
This last equation is useful to compute ADU values from dual_iso images. Maybe a similar algorithm is used by ML and cr2HDR to compute DR gain from dual_iso; to be honest I don't take a look on the code yet.
Ref.: You can find full development of this in scientific papers plus some other references included:
_. High Dynamic Range Image Sensors, A. Gamal, Department of Electrical Engineering of Stanford University
_. High Dynamic Range Imaging: Spatially Varying Pixel Exposures, K. Nayar, Department of Computer Science of Columbia University & T. Mitsunaga, Media Processing Laboratories of Sony Corporation
_. E. Ikeda, Image data processing apparatus for processing combined image signals in order to extend dynamic range, September 1998
_. A. Morimura, Imaging method for a wide dynamic range and an imaging device for a wide dynamic range, October 1993
_. R. A. Street, High dynamic range segmented pixel sensor array, August 1998
_. Y. T. Tsai, Method and apparatus for extending the dynamic range of an electronic imaging system, May 1994
I appreciate that you spent the time to clarify
some of the issues I raised.
QuoteWe would like to get the maximum DR from this device, so the first thing to do is to analyse what is reducing the DR.
And herein lies the crux of the problem. While I appreciate that the content you posted may "clearly be a simplified version",
compared to extended equations that might span 3 square city blocks, they are probably still significantly above the comprehension of most readers of this forum. Certainly, the maths equations you described, are well above
this readers comprehension.
Here, I
had attempted to increase the knowledge (of the reader), of the processes involved in using dual_iso, from almost nil, to a satisfactory level suitable in obtaining excellent results, in the field. Based on the pros and cons of dual_iso itself.
You have attempted to increase the knowledge (of the reader), from the basic understanding I set forth, to that of a university professor.
Baby steps ;)
QuoteDR is defined as the ratio of the largest nonsaturating input signal to the smallest detectable input signal.
That is the engineering definition, yes.
Now consider this. Does the engineering definition of the DR, describe the
useful DR available to the photographer?
At some future date, I will attempt to describe a number of processes that will help the photographer best use ML, in a manner that is designed to increase the readers comprehension from almost nil, to a level that should hopefully provide an increase in the output quality of the photographers images.
I would appreciate it greatly, if you did not attempt to extend this comprehension to a level that requires an intimate knowledge, of the physics involved, in
that thread. Your extended explanations will only serve to help a significantly small number of the target audience, and will
simply serve to confuse the greater
target audience.
That is not to imply that I want to limit conversation, I simply ask that extended technical explanations are conducted in a separate thread. If you do conduct your own explanations in a separate thread, I, for one, look forward to attempting to increase my knowledge of the subject, even further.
Quote from: Audionut on February 21, 2014, 08:16:40 AM
You have attempted to increase the knowledge (of the reader), from the basic understanding I set forth, to that of a university professor.
Several months ago, DPReview added a forum for "Photographic Science and Technology", which tends to be rather esoteric.
http://www.dpreview.com/forums/1061
Would that kind of forum be something to consider for ML?
Quote from: Audionut on February 21, 2014, 08:16:40 AM
(...) Does the engineering definition of the DR, describe the useful DR available to the photographer?
(...)
Good question.
I understand that for users, ML have to be 'friendly'. We had discussions with a1ex about this offrecord to make best choices about how ML get access to special functions.
In fact, engineering tools are good; all ML developers here are a kind a engineer.
Plus, maths tools are good too; we should not be afraid by them.
Let me try to explain why:
Good observation is good science. Every time I can I repeat it. We need observations, experimentations, tests, etc.
However, without maths tools, observations are hard to interpret, it takes more times to understand what appends and/or how to correct engineering at the source (ML functions).
I don't say we 'have to' use complex maths etc.: I just say without some tools like these, progress will be hard and will take long times at this point where ML is.
a1ex made useful tools like that; ADTG_GUI, RAW_DIAG, etc. Of course it is hard to use for most of users, but it does not mean that we have to be afraid by them.
Then, it is why I think I was misunderstand some posts ago; it was not to find fault and depreciate someone, it was trying to help by underline where it should be.
The big challenge is that ML starts to access to very complex stuff since some times and it is equally more complex to help with simple stuff.
So, does the DR equation with FPN, RON, TN, .. considerations will be useful for photographers: Yes! :)
Photographers (users) maybe not understand (don't need to?) what appends with the ML on background, but it is clear that if we don't care off this, photographs (images/pictures) will not be as 'nice' as they could be.
For example, it clearly looks that with 5D2, with last offset/WL/ADTG/.. optimizations, we should pass over the 12 stops wall on a single frame with some FPN reduction. And with dual_iso there will be a gain too, of course. The 11.5, 12.0 or 12.5 stops numbers maybe not be relevant overall for photographs, but when they will see deep clean blacks (offset opt + FPN reduction), rich high lights and smooth middle tones (WL+ ADTG opt) on their photographs, it's quite sure that they see the gain.
QuoteArguing with an engineer is a lot like wrestling in the mud with a pig. After a few hours, you realize that he likes it.
;)
@SpcCb
Engineering dynamic range is useful, because it is not subjective (http://en.wikipedia.org/wiki/Subjectivity). It is not subject to personal opinion.
Apart from that, we are on completely different wavelengths. My intention in the initial post was not to school the reader on the benefits of accurate engineering dynamic range, and it's usefulness. It was to increase the readers understanding on basics. Of course, our definition of basics is surely many moons apart ;) I am not sure in how many more different ways I can try and explain this to you. As I tried to suggest many posts ago, how about we simply move on.
@a1ex
I think the biggest failing with most engineers, and of course, I speak generally here, is that they become fixated with results based entirely on equations. They are not willing to trust the senses that they have at their disposal.
I will use this example.
PSNR (http://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio) is used in video encoding for the same reason E-DR is used here. It does not suffer from subjectiveness. However, this same lack of subjectiveness is the primary reason why it is not the determining factor on the quality of an video encoder.
Consider x264 (http://en.wikipedia.org/wiki/X264). The initial development cycle consisted primarily of coding based on the results of PSNR. This is obviously useful, as the initial code is designed based on a metric that is not subjective. However, humans are highly subjective beings. Primarily, when Jason Garrett-Glaser (http://x264dev.multimedia.cx/about) joined the development team, the focus on coding shifted to subjectiveness. It was at this time that optimisations such as Adaptive quantisation (http://forum.doom9.org/showthread.php?t=130030&highlight=adaptive) (AQ), Psychovisual Rate–distortion optimisation (http://forum.doom9.org/showthread.php?t=138293&highlight=rdo) (Psy-RDO) and Macroblock-tree rate control (http://forum.doom9.org/showthread.php?t=148686&highlight=tree) (MB-Tree) were implemented in x264.
QuoteTry out my new AQ method. Warning: It is very powerful and will butcher your PSNR, though it has little effect on SSIM. Don't judge by the metrics, judge by appearance.
These optimisations are based entirely on subjectivity. They are, combined, one of the primary reasons why x264 is considered by many, the leading H.264 spec encoder available today.
It has been my experience, that engineers tend to work better with individuals who are adapt at using their senses to determine effects, not measurable with metrics. Think of this as a symbiotic relationship (http://en.wikipedia.org/wiki/Symbiosis).
Think of our relationship. Most of the time I am just stupid. But every now and then, I am useful. Mostly, because I do not think like an engineer.
Quote from: a1ex on February 21, 2014, 06:14:21 PM
QuoteArguing with an engineer is a lot like wrestling in the mud with a pig. After a few hours, you realize that he likes it.
;)
:D
I hope you don't have something worse about physicists! :)
Of course I have:
Quote
Mathematician: Pi is thenumber expressing the relationship between the circumference of a circle and its diameter.
Physicist: Pi is 3.1415927 plus or minus 0.00000005
Engineer: Pi is about 3.
=> now you can see where the misunderstanding is coming from :P
And I actually copy some code from a physicist:
https://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html
https://code.google.com/p/rawtherapee/source/browse/rtengine/amaze_demosaic_RT.cc
Quote from: a1ex on February 21, 2014, 10:59:53 PM
Of course I have:
QuoteMathematician: Pi is thenumber expressing the relationship between the circumference of a circle and its diameter.
Physicist: Pi is 3.1415927 plus or minus 0.00000005
Engineer: Pi is about 3.
=> now you can see where the misunderstanding is coming from :P
Well seen! ;D
And it confirms that I'm definitely not an engineer, nor a mathematician!
Quote from: a1ex on February 21, 2014, 10:59:53 PMAnd I actually copy some code from a physicist:
https://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html
(...)
Nice reading, and I note that you find an other physicist agree with me:
Quote from: Emil Martinec(...)
If the DR of the downstream components were not a limiting factor, then when recording at ISO 100, the highlights would be available, but the read noise would be predominantly due to the sensor itself and thus have the same value in electrons as the read noise at ISO 1600. For the 1D3, ISO 100 read noise, limited by the downstream components, is about 24 electrons; ISO 1600 read noise, coming primarily from the sensor, is only four electrons. Because of this factor of six difference in read noise, the full sensor dynamic range is over two stops higher (about 14.0) than the camera delivers (about 11.7) due to those downstream components.
This hidden extra dynamic range is what Roger Clark calls "sensor dynamic range" -- (...)
So what can we do to increase dynamic range other than Dual ISO? I know there's the ongoing hacking of the Digic registers, EDMAC, etc to squeeze half a stop out, but what's stopping us from getting more?
@a1ex
your last cr2hdr converts this ISO 100/+2EV
https://www.dropbox.com/s/xcaclw1jxwhnv82/2014-02-23_5089.CR2
into dng which still has dualISO lines like input cr2 file, could you check it, please?
c:\dualISO>cr2hdr.exe 2014-02-23_5089.CR2
cr2hdr: a post processing tool for Dual ISO images
Last update: 835b724 on 2013-12-10 13:06:11 UTC by email:
Merged OtherOnePercent/tragic-lantern-6d into unified
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (def
ault)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
Input file : 2014-02-23_5089.CR2
Camera : Canon EOS 50D
Full size : 4832 x 3228
Active area : 4770 x 3177
Black borders : 62 left, 51 top
Black level : 995
ISO pattern : ddBB GBRG
White levels : 13310 11413
Noise levels : 6.83 8.07 8.85 7.59 (14-bit)
Black delta looks bad, skipping correction
ISO difference : 0.29 EV (122)
Black delta : 0.00
Dynamic range : 10.82 (+) 10.20 => 10.49 EV (in theory)
Looking for hot/cold pixels...
Hot pixels : 226
Cold pixels : 141
AMaZE interpolation ...
Amaze took 6.07 s
Edge-directed interpolation...
Semi-overexposed: 33.07%
Deep shadows : 2.84%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap : 7.5 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 31.47 (16-bit), ideally 29.90
Dynamic range : 10.61 EV (cooked)
Black adjust : 7
Output file : 2014-02-23_5089.DNG
Quote from: orim on February 23, 2014, 05:00:46 PM
@a1ex
your last cr2hdr
Last update: 835b724 on 2013-12-10 13:06:11 UTC by email:
Merged OtherOnePercent/tragic-lantern-6d into unified
I see different.
QuoteLast update: d0ac769 on 2014-01-23 10:13:39 UTC by alex:
cr2hdr: experimental white level routine with two separate levels ...
Both versions have the bug.
It's already fixed in my local copy, but I'd like to understand what exactly happened before publishing the fix. If you run it with --iso-curve, it looks like a strong nonlinear response of the sensor (which is not true).
Thanks, alex,
so now i have the file, which can not be processed, till update, right.
(yes, i downloaded your cr2hdr again :-)
And what about opposite situation, where my non dualISO files are processed into dngs?
Why is that happening/how to prevent it?
Do you have an example? How do the log files look like when you process a non dual ISO shot and it gets mistaken for one?
Alex,
here is the example cr2: https://www.dropbox.com/s/dena1a1bu3dn3hb/2014-02-22_4912.CR2
i want to share it as short as possible - please, let me now u downloaded it.
it is Tv priority, ISO auto = 1600, +2/3 Ev
c:\dualISO>cr2hdr.exe 2014-02-22_4912.CR2
cr2hdr: a post processing tool for Dual ISO images
Last update: d0ac769 on 2014-01-23 10:13:39 UTC by alex:
cr2hdr: experimental white level routine with two separate levels ...
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (def
ault)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
Input file : 2014-02-22_4912.CR2
Camera : Canon EOS 50D
Full size : 4832 x 3228
Active area : 4770 x 3177
Black borders : 62 left, 51 top
Black level : 993
ISO pattern : ddBB GBRG
White levels : 9000 9000
Noise levels : 19.43 17.66 19.05 17.40 (14-bit)
Doesn't look like interlaced ISO
ISO difference : 0.00 EV (100)
Black delta : -0.55
Dynamic range : 8.85 (+) 8.69 => 8.69 EV (in theory)
Looking for hot/cold pixels...
Hot pixels : 778
Cold pixels : 1686
AMaZE interpolation ...
Amaze took 3.67 s
Edge-directed interpolation...
Semi-overexposed: 0.00%
Deep shadows : 24.08%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap : 5.8 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 106.69 (16-bit), ideally 59.52
Dynamic range : 8.23 EV (cooked)
Black adjust : -17
Output file : 2014-02-22_4912.DNG
hi guys,
Don't know if this is the appropriate topic but, I would like to share a dual_iso picture just to show one of the big advantages of using the dual_iso module.
Kitty scratching in HDR! May be plain to some, I loved the final result! ;)
http://imagizer.imageshack.us/v2/1600x1200q90/32/hqp8.jpg
For some reason, no matter what I do I cannot get the dual ISO working.
I tried this inside my house multiple times. I shoot the interior and outside window at the same time. ISO 100 from Canon, to expose the outside of the window, and ISO 800 for ML for the interior. I then run the cr2hdr tool to make a DNG which is supposed to make an image with a lot of dyn range improvement, but there is none. Not noticeable like you all show in your photos.
I compare it to an original photo that the camera takes, and they look the same pretty much.
5d Mark 2
Tried it a few times throughout the last week or two. I don't know what I did wrong?
Thank you,
Mr D
Hello mrd777,
When you take a dual_iso picture, can you see some banding? Does the picture look weird to you? If so, camera wise, you're doing it right. If so, let's get on the Barracuda software.
Use the BarracudaGUI.exe with the "attached" cr2hdr.exe and dcraw.exe in the same folder. Then simply run barracuda, you get a DNG file and then it's up to the Light Room or Photoshop to give it that extra punch in color, that's basically it.
Edit: Oh, and if you are using ETTR, try dual_iso module with it off, sometimes for some reason, it equals the ISO values for ex. 100/100 = 0 DR ::)
Hi madness,
Thanks for the reply. I see the scan lines if that is what you mean. So it looks like it works. So I'm guessing I missed the fact that I need to use Barracuda.
I get my camera back from a friend in a few days. Will try then.
Thank you!
Barracuda is only a gui for the original cr2hdr command line, but you're welcome ;-)
Compare a standard ISO 100 shot with a dual_iso shot. You need to boost exposure and shadows to see the difference.
@mad.eos
There is a thread for dual_iso photos here (http://www.magiclantern.fm/forum/index.php?topic=7402.0).
I've probably missed something but not having any luck here.
5D MKIII iso set to 100, dual iso turned on at recovery iso set to 1600. Screen looked interlaced. Extracted .RAW with RAW Magic (MAC OS) and dragged dng's onto cr2hdr - it ran through the files but errored with "Doesn't look like interlaced ISO " and deleted all the dng's.
Please advise and sorry in advance for any stupidity on my part.
Just doubled checked and I noticed the after setting the recovery iso it then switches back to the canon iso...
Do you have Ettr enabled? If so, uncheck it and try again.
Is there any way to have dualISO mlv files "tagged"? When using ETTR connected with dualISO we often get mixed files (normal and dualiso) and the only way to know if the clip is dualISO or not is to open one of the DNGs (which takes a lot of time with dozens of files). If we have DI files with some different name it would be possible to easly filter such a files out of the rest and process them with any dual iso app (or script) right away. Is metadata of MLV module has a "room" for such a tagging?
EDIT: Maybe something like 'custom file prefix' but for movies.
I have two problems.
First one:
cr2hdr does not work in command line for me with all pictures.
(https://dl.dropboxusercontent.com/s/ec87unrnckbiogj/cmd2.PNG?dl=1&token_hash=AAHrjfFrIm1kKiTd7U96xlnDpWED8uKfeSQobhetb4ej0Q)
The command "exiftool"(dcraw) is wrong written or could not be found. But both exe are located in the same folder with cr2hdr.exe
Second one:
So I used the barracudaGUI and it works with it, but not with this this picture https://www.dropbox.com/s/y6fybg1i2ta6149/IMG_2524.CR2
Other user reported, that he also can not process it with cr2hdr.
Input file : IMG_2524.CR2
Camera : Canon EOS 650D
Full size : 5280 x 3528
Active area : 5208 x 3476
White level : 12500
Black borders : 72 left, 52 top
Black level : 1992
ISO pattern : BdBd RGGB
Interlacing method not supported
ISO blending didn't work
This picture is taken with 650D and FEB 20. build
My OS is Windows 8.1
hi!
is there any way to extract the two "exposures" so i can combine them manually, as i do any other HDR?
best woud be if i could get two raw-files exactly as i would have shot two separate images like traditional HDR.
i already tried the --debug-blend -option but all the images i get from that look exactly the same. except the what is supposed to be the bright picture which is exactly the same except the hilights have a pink tint.
any suggestions?
my camera is a 550d and i use a ml version less than a month old
Develop the output RAW with two different exposure compensations, and blend them in the classic way.
Next update will have an option to burn some exposure compensation in the DNG file (will compress the highlights and hopefully extract a little more detail from shadows).
I just tried the new 20bit processing on a couple of my dual_iso files. The good news: I don't see any regressions! The bad news: It looks exactly the same as before :-p ... where do I look for changes, i.e. what's the visible benefit of the increased resolution?
There's no increased resolution; the difference may be in extreme shadows (if you ever got posterization). The most obvious difference is with my first fake_dual_iso test (http://www.magiclantern.fm/forum/index.php?topic=7139.msg99133#msg99133) (which is quite extreme, 6 EV between the two exposures).
If you feel you are close to the 16-bit limit, try --soft-film.
Quick sample (fake dual ISO 6EV):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/20bit/fake6ev-1900-1450-16bit.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/20bit/fake6ev-1900-1450-20bit.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/20bit/fake6ev-1900-1450-20bit-baked4ev.jpg)
1. 16-bit engine (522eb2e), developed with +10 EV in ufraw-mod, --clip=digital
2. 20-bit engine (b13fcd5), developed with +10 EV in ufraw-mod, --clip=digital
3. 20-bit with --soft-film=4, developed with +6 EV in ufraw-mod, --clip=digital
The --soft-film option compresses the highlights and bakes an exposure compensation in the DNG. With this trick, it uses more bits for shadow detail and you get a brighter image to start with. The baked curve looks like this:
(http://a1ex.magiclantern.fm/bleeding-edge/soft-film-curves.png)
For baking a soft film curve, you need to know the WB in advance, at least roughly (read the multipliers from ufraw, for example). In all the examples I used 4000K/green=0.8, which has the multipliers 1.899, 1, 2.173. Since the default WB for these baked curves is 2,1,2, this fits under the engineering definition of Pi (which is 3), so I didn't bother changing the multipliers for now.
Let's play with it a bit more:
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/20bit/fake6ev-1900-1450-20bit-baked8ev.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/20bit/fake6ev-1900-1450-20bit-baked9ev.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/20bit/fake6ev-1900-1450-20bit-baked10ev.jpg)
4. 20-bit with --soft-film=8, developed with +2 EV in ufraw-mod, --clip=digital
5. 20-bit with --soft-film=9, developed with +1 EV in ufraw-mod, --clip=digital
6. 20-bit with --soft-film=10, developed with +0 EV in ufraw-mod, --clip=digital
Now, let's try to bake the same WB as the one used for developing, to see how much it matters:
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/20bit/fake6ev-1900-1450-20bit-baked10ev-wb.jpg)
7. 20-bit with --soft-film=10 --wb=1.9,1,2.17, developed with +0 EV in ufraw-mod, --clip=digital
Notice the subtle WB change in highlights.
Alright, so we got pretty good highlight/shadow detail now. How does it compare with ufraw-mod's --clip=film?
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/20bit/fake6ev-1900-1450-16bit-film.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/20bit/fake6ev-1900-1450-20bit-baked4ev-film.jpg)
8. 16-bit engine, developed with +10 EV in ufraw-mod, --clip=film
9. 20-bit with --soft-film=4, developed with +6 EV in ufraw-mod, --clip=film
In image 8, notice how ufraw-mod sacrifices 2 shadow bits in order to get better highlight detail (compare with 1). That's a limitation of its 16-bit processing engine. Floating-point raw processors would not suffer from this => consider the highlight compression as a trick to bypass the bit depth limits in either the DNG container or in the raw processor itself.
So, you have noticed this curve is exactly the soft-film curve from ufraw-mod (http://www.magiclantern.fm/forum/index.php?topic=9597), and now you have the option to bake it in the DNG (you can consider it as a pre-grading). You can bake it completely (images 6,7) or just a part of it (9). Experiment and see what works best.
Experimental binary: cr2hdr-20bit.exe (http://acoutts.com/a1ex/cr2hdr-20bit.exe)
This example was extreme (I chose it to push the code beyond reasonable limits). In practice, the improvement of the 20-bit engine is probably minor, I'm not sure if it's worth the small speed penalty and increased memory usage. You tell me (if you find differences, just post some examples).
Happy pixel peeping!
Quote from: a1ex on March 06, 2014, 12:09:42 AMIn practice, the improvement of the 20-bit engine is probably minor, I'm not sure if it's worth the small speed penalty and increased memory usage. You tell me (if you find differences, just post some examples).
Right, that explains it :-) ... with increased resolution I meant the internal processing depth, I know there won't be any additional magic done to dual_iso files that actually exchanges the Canon for a Nikon sensor. As for speed, at least on my box cr2hdr is so slow it hardly makes any difference so you may as well keep the new version just to be on the safe side.
Worked great for me didnt seem any slower anyway. I dont know if im taking advantage of the 20bit through lightroom or acr cuz i think they only do 16bit (this was probably discussed but im sure i missed it) but results are still impressive! Well done!
Quote from: hjfilmspeed on March 06, 2014, 04:33:38 PM
I dont know if im taking advantage of the 20bit through lightroom or acr cuz i think they only do 16bit
No matter, 20bit internal processing gives you the high quality, high end feeling that was simply missing from the old and shaggy cr2hdr versions :-p
Actually I did encounter a real-world improvement in a picture from what I shot this weekend. It looks pretty much like my example from above, so I've re-converted that picture with --soft-film=4 and got the shadow detail back.
It was just a plain old 100/1600 with deep shadows, exposed with ETTR (nothing fancy). If anyone is interested, I can develop that image with a few different settings to show the differences.
Quote from: a1ex on March 06, 2014, 05:31:50 PM
It was just a plain old 100/1600 with deep shadows, exposed with ETTR (nothing fancy). If anyone is interested, I can develop that image with a few different settings to show the differences.
Well, if it's different from the sample shots above I guess it might be interesting to see where dual_iso and cr2hdr have their limits with the current 16bit output, i.e. what is just barely recoverable and what only shows after applying a pre-processing curve.
I always love me a good pixel peeping session! Again great work! Im forever impressed
This 20bit version with soft-film=2 really solved a severe colour blocking problem I faced in a skyline HDR of Melbourne.
With the old version the sky consisted of huge pink / blue squares. Now it is all smooth.
GREAT :-)
Remember the thread about when it's convenient use dual ISO?
I need to re check the possibilities of this module...
@a1ex
What edge-detection method cr2hdr use? EECI? I find one very interesting paper, of three koreans, in proposing the new method of edge detection method and the refinement scheme, called "DVBDM" or "Directional Variances Based Demosaicing Method". See here (http://www.thinkmind.org/download.php?articleid=mmedia_2014_5_10_50043).
The algorithm described is 37,8% more faster than EECI (avarage), and have more 2.95 dB of result on Color PSNR measurement.
I was wondering if it would be possible to adapt something to use in cr2hdr... but unfortunately I am not mathematician to do it.
*only problem of that, which is used is 5x5 CS.
Is there a 20bit version for mac? I have a di shot with some trees and the sun and after conversion i have many pink aeras..
I have the same problem like mad77 i can see the scanlines in the CR2 file, but when i start cr2hdr.exe the dng becomes the same like the original raw, without dual iso.
Quote from: norbyg570 on March 13, 2014, 12:35:51 PM
I have the same problem like mad77 i can see the scanlines in the CR2 file, but when i start cr2hdr.exe the dng becomes the same like the original raw, without dual iso.
You should provide some samples.
Greetings and Salutations
I see some users have been playing with the 600D to get the Duel ISO for HDR somewhat working. I did read the original PDF so its a little confusing if this actually works for the 600D? Or if it will at some point..deciding on a new camera to buy :)
http://www.magiclantern.fm/forum/index.php?topic=8329.0
here are some samples:
the RAW CR2 dual ISO
(http://imagizer.imageshack.us/v2/800x600q90/585/xitp.jpg)
the DNG file from dual ISO
(http://imagizer.imageshack.us/v2/800x600q90/132/e7pp.jpg)
By samples, I meant the dual_iso.CR2 file.
Nevermind.
Now open that DNG file in your favorite image editor. Boost the exposure. Take note of the noise in those dark areas.
Do you have the same image taken with normal ISO settings? If you do, open that file in your image editor also, and boost exposure. Note the difference in noise compared to the dual_iso file?
Now take the dual_iso DNG, and post process (tone map) that file as needed. Job done!
The reason why dual_iso images are so dark, is that you have captured highlights that are significantly brighter then the midtones/shadows. So you need to post process them to bring the midtones/shadows higher.
You can do the same with standard ISO also, but note the large difference in noise.
I would like to ask about 'custom file prefix' option in dualISO module. It seems that this works only with cr2 for now (and its buggy according to description), is it possible to activate it for movies? I tried and it doesnt affect movie file names. With the new apps and scripts it would be very useful to pick up dualiso files easly and fast from couple houndreds mixed MLVs.
thanks! i will try it this weekend and post the difference
Quote from: kgv5 on March 14, 2014, 09:25:40 AM
I would like to ask about 'custom file prefix' option in dualISO module. It seems that this works only with cr2 for now (and its buggy according to description), is it possible to activate it for movies? I tried and it doesnt affect movie file names. With the new apps and scripts it would be very useful to pick up dualiso files easly and fast from couple houndreds mixed MLVs.
Indeed! I just implemented dual iso video support in my converter and that would be neat! Though I think there IS a way of detecting automatically with metadata but that would be MUCH easier..
is anyone having this kind of flicker?
https://vimeo.com/89151079
any thoughts on how to remove it?
Try this one: www.magiclantern.fm/forum/index.php?topic=8850
Hello everyone, i want to ask, why my Canon 1100D using Magic Lantern latest nightly build Dual ISO shoot will like this? It look many line
(http://i51.photobucket.com/albums/f356/xbboyx/Untitled-1_zps01443b5f.jpg)
Quote from: riclim on March 15, 2014, 05:05:04 AM
Hello everyone, i want to ask, why my Canon 1100D using Magic Lantern latest nightly build Dual ISO shoot will like this? It look many line
Read the white paper first on dual iso before you use this and ask questions. It on the first post.
heres the white paper on it. https://bitbucket.org/hudson/magic-lantern/downloads/dual_iso.pdf
Quote from: ted ramasola on March 15, 2014, 05:32:23 AM
Read the white paper first on dual iso before you use this and ask questions. It on the first post.
heres the white paper on it. https://bitbucket.org/hudson/magic-lantern/downloads/dual_iso.pdf
Ok, thanks, i will read soon.
Quote from: ted ramasola on March 15, 2014, 05:32:23 AM
Read the white paper first on dual iso before you use this and ask questions. It on the first post.
heres the white paper on it. https://bitbucket.org/hudson/magic-lantern/downloads/dual_iso.pdf
Hi friend, i read it already, i saw that "3 ADTG/CMOS register tweaking", but it showing explain some code, i can't understand, i'm still confuse of it.
And i read many page of this topic too, i found some people is after shooting Dual ISO as CR2 photo, then drag the file to "cr2hdr.exe" then will be convert to DNG file with dual ISO combie.
Dual ISO is only success with this cr2hdr.exe software or got any solution can direct perfect shoot in the camera?
Hope someone can let me know the clearly answer, thanks ;)
Quote from: a1ex on July 16, 2013, 06:33:50 PM
Postprocessing
- For CR2 files: cr2hdr.c (https://bitbucket.org/hudson/magic-lantern/src/tip/modules/dual_iso/cr2hdr.c) (Windows:cr2hdr.exe (http://acoutts.com/a1ex/cr2hdr.zip)). It requires dcraw and (optional) exiftool in your executable path (they are in the zip). Drag and drop the CR2 files over cr2hdr.exe.
Update by RenatoPhoto:
For Mac download:
Mac GUI for cr2hdr: https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr_app.zip
dcraw and exif tool: https://bitbucket.org/a_d_/magic-lantern/downloads/OSX_cr2hdr.zip
Rename cr2 to capitals CR2 using: http://namechanger.en.softonic.com/mac
Also, once you've renamed them to .CR2, the newest CR2HDR allows you to drag and drop the entire folder to the icon.
---------------------
For automatic script using Multiple Instances of cr2hdr see the excellent:
http://www.magiclantern.fm/forum/index.php?topic=8520.0
- For RAW video files: latest raw2dng.exe (http://acoutts.com/a1ex/raw2dng.exe). extract the DNGs with any method you like (e.g. raw2dng), then drag the DNG files over cr2hdr.exe
- To get a natural HDR look: try my automatic color grading script (http://www.magiclantern.fm/forum/index.php?topic=7022).
Also, please don't create duplicate posts in the forum.
Quote from: Audionut on March 15, 2014, 06:40:51 AM
Also, please don't create duplicate posts in the forum.
ok, thanks and sorry for the duplicate posts.
For this DualISO shooting, it is only the way using the CR2 file convert to DNG file using "cr2hdr.exe"? No other way, right?
What's the matter with cr2hdr?
Quote from: Audionut on March 15, 2014, 06:56:47 AM
What's the matter with cr2hdr?
Maybe i misunderstanding of the Dual ISO function, i think it is direct adjust and shoot will get effect view in camera.
But i shoot every Dual ISO photo will get whole line texture picture like the photo i share.
Now i understand the Dual ISO function photo after shoot the CR2 file then need using computer cr2hdr.exe convert to DNG file only get the perfect Dual ISO image.
(http://i51.photobucket.com/albums/f356/xbboyx/Untitled-1_zps01443b5f.jpg)
Every Dual ISO photo will have line texture. Yes.
Because it captures extra dynamic range in this way.
Process with cr2hdr. Gives DNG. This DNG looks like Normal Shot. But this DNG has much less noise.
Raise exposure/shadows in post process.
Friend, thank you so much, now i 100% understand of the function already. :D
After shoot the CR2 with dual ISO, then use cr2hdr.exe make the DNG file, it can open in photoshop like RAW file, it will less noise and it can adjust recovery/shadow until a perfect HDR photo, awesome !!!
;)
Nice works, thanks again ;D
I'm using Nightly Build Feb-2014-02-18. Question: Does Dual ISO works on 60D?
Thanks.
http://nanomad.magiclantern.fm/jenkins/features.html
Why such an old build?
Quote from: a1ex on March 15, 2014, 12:35:55 AM
Try this one: www.magiclantern.fm/forum/index.php?topic=8850
I don't have bridge :-\ bummer, i'll have to wait for a cr2hdr fix
Quote from: Audionut on March 15, 2014, 04:21:57 PM
http://nanomad.magiclantern.fm/jenkins/features.html
Why such an old build?
Thanks Audionut, that matrix helps me to understand the features on Nightly Build. Are they updated in cases where new features for a camera?
I'll try and download/test the latest build.
Thanks again> :))
Does anyone think we are so crazy to check all that stuff manually? :D
I meant to say "Are they updated in cases where there is new or added ML feature or functionality?". I'm just saying....
They are generated by the build script (it greps the source code and checks what it finds enabled where).
Quote from: Audionut on March 15, 2014, 04:21:57 PM
http://nanomad.magiclantern.fm/jenkins/features.html
This features. html was built on Jan 21, 2014. Has there been an update since then that would indicate what works or not with the 6d?
Hit F5, it was a broken redirect. You should get redirected here: http://builds.magiclantern.fm/#/features
Quote from: riclim on March 15, 2014, 10:54:03 AM
Friend, thank you so much, now i 100% understand of the function already. :D
Not quite - dual_iso has some drawbacks, you're not getting something for nothing: There is reduced resolution on both sides of the histogram, so if one side matters more to you you'd probably better off with a normal cr2 and accept noisier shadows . Plus dual_iso has the whole cr2hdr postprocessing overhead and wb in acr is currently broken.
I have eos m. Dual iso work properly on video but how I take picture with dual iso? I'm noob. :D
Edit: now i got it. :D
Quote from: a1ex on March 16, 2014, 10:48:33 AM
Hit F5, it was a broken redirect. You should get redirected here: http://builds.magiclantern.fm/#/features
Thanks.
I did notice there were two columns for 700d. I infer one of those should be 70d? (the blank column?)
Ah, the script got confused by upgrading 700D 111->113.
cr2hdr is a single core program right?
On my macbook it stays around 97-99% consistently.
How hard would it be to make it multi-threaded so that it could utilise all cores? maybe the windows version is multi-threaded?
There are plenty of options here (http://www.magiclantern.fm/forum/index.php?board=56.0), for running multiple instances of cr2hdr.
I didn't really get what specifically you were thinking of, but I found this terminal line
open -n -a "APPLICATION NAME"
open -n -a "cr2hdr"
Do that a couple of times = win
Also, there seems to be a limit to the number of files which the cr2hdr program will accept, somewhere between 1750 and 2000 (didn't test too closely). When you hit this number, the program simply will not start running and you have to cancel, close the program and choose a smaller number of files.
Has anybody else found this to be true?
My apologies.
I thought I had seen an application for mac that opened multiple instances of cr2hdr for faster processing. Having a quick look through the forum though, doesn't appear that there are any applications.
If you accomplish a nice workflow to allow so, please feel free to make a post, and I can sticky it in the post processing section for others to benefit.
Regards,
Audionut.
No worries, worst case I'll use parallells or dual boot.
Right now I can open as many windows (osx) as I want (probably wouldn't be faster after 8, as I have quad core hyper thread), but the main problem is feeding the windows as you have to sit ready for when the processing is done. Would certainly be nice to just load all the folders with the .dngs and let it crunch through those files overnight.
Got a citation for the dual ISO paper :P
HDR reconstruction for alternating gain (ISO) sensor readout (http://vcl.itn.liu.se/publications/2014/HKU14/draft.pdf)
Nice. :)
The ultimate gain strategy imo, would be individual gain of each pixel group. Of course, that's not something possible atm, but I imagine the camera makers could implement it without to much hassle. I'm not holding my breath though.
I expect your paper on CMOS/ADTG to generate significant interest.
[a1e13] A1EX: Dynamic range improvement for canon
dslrs with 8-channel sensor readout by alternating iso
during sensor readout. Technical documentation, url:
http://acoutts.com/a1ex/dual_iso.pdf, July 2013.
hrhr, congratulations :)
and whats the difference to your algorithm now?
i cant see any visual difference except shadows being pushed.
edit: ah in the piano view it shows a bit improvement on aliasing
I'm writing to them as we speak; would be very nice to try their algorithm on my test suite (which simply contains the troublesome images reported in this thread).
Can't tell yet where the shadow difference is coming from; right now, my best guess is that it has to do with the algorithm for matching the image brightness (I simply match the histograms, which is fairly good because it removes most of the noise, but meanwhile I found out the histograms are distorted - read: convolved - by noise in shadows, and my only correction to that is a stronger weight in the highlight range).
For pixel peeping, here's a full-res version of their test image:
http://vcl.itn.liu.se/publications/2014/HKU14/result.png
(from http://vcl.itn.liu.se/publications/2014/HKU14/ )
Quote from: a1ex on March 19, 2014, 11:18:38 AM
Got a citation for the dual ISO paper :P
HDR reconstruction for alternating gain (ISO) sensor readout (http://vcl.itn.liu.se/publications/2014/HKU14/draft.pdf)
Congrats!!! I woot woot woot!
Yey a1ex! Congrats m8!! :D
Hi -
I've been following this thread for a few months, but I'm still a noob.
The use of 20 bit FP for combining the images makes sense to me, but then you possibly lose some of that advantage when mapping back to 16 bit. How about an option to output in 32 bit TIFF? Recent versions of ACR read that format, so you wouldn't lose any information.
Thanks for all your work.
My dual iso clips seem to have the last horisontal line flickering and I'm not sure what I'm doing wrong.
Processed with cr2hdr 2.0 for mac
Does anybody have the same experience?
Same
I have been on the fence about using ML on my 5D Mark III because I didn't want to downgrade the 1.2.3 firmware. Now that there is a version out that works on this firmware, I installed it. I wanted to try out the dual ISO so I took a shot with some deep shadows to see how it would do. The results are pretty astounding. I'm actually surprised that the camera at the default setting is so noisy at ISO100 (I exposed for the very bright window purposely so I would have to crank up the shadows in post). Enabling Dual ISO (100-1600) dramatically reduced the grain and increased the detail. I used cr2hdr on my Windows 7 laptop with no issues at all. Here is a 100% crop of with and without...
(http://www.stevendempseyphotography.com/images/compare.jpg)
Quote from: a1ex on March 19, 2014, 12:03:51 PM
For pixel peeping, here's a full-res version of their test image:
http://vcl.itn.liu.se/publications/2014/HKU14/result.png
(from http://vcl.itn.liu.se/publications/2014/HKU14/ )
their method seems to introduce blur in both 100-800 and 100-1600, like a thin softfocus to my eyes... they even mention this near the end of the paper: "However using isotropic filter kernel can introduce blur and color artifacts in the result" - does this mean they'd get sharp results by removing the aa filter?
grats on the citation, and if they work/share with you, even better
Quote from: aseek on March 21, 2014, 08:20:12 AM
does this mean they'd get sharp results by removing the aa filter?
Probably, but it would apply to all situations, not simply whatever they are doing.
Forgive me if this has already been discussed.
Dual ISO is not useful to me for video due to the crawling aliasing I see at boundaries between 1/2 res highlights and mid/shadows, or 1/2 res shadows and mids/highlights - when the camera or scene is in motion.
The question is whether it's possible to perform dual ISO in shadows only, leaving highlights at full resolution? I figure that this would sacrifice 1/2 the DR benefit, but I hypothesize that my main objection for dual ISO for video would be improved in that there would be no aliasing between highlights and mids, but only between 1/2 res shadows and highlights/mids.
This is based on my assumptions that the aliasing between 1/2 res highlights and 1/2 res shadows is the most noticeable and disagreeable, and that aliasing between 1/2 res highlights and mids is more noticeable that aliasing between 1/2 res shadows and mids.
[EDIT] Or do I accomplish this myself by setting Dual ISO to, say, 2.4EV DR gain (100/800) and underexposing by -1.2EV?
This would require two things:
1) knowing in advance where are the highlights and where are the shadows
2) the ability to configure gain per-pixel
What you can do right now is to tweak FPS to avoid crawling, and use raw zebras to make sure the aliasing occurs in out-of-focus areas only.
Note that right now, the algorithm does not exploit any kind of temporal information. Contributions are, of course, welcome in this direction, and as a starting point for research, I could mention the super-resolution technique.
I'll need to re-read your paper to understand why resolution must be sacrificed at both highlights and shadows. I assumed that if the high-ISO highlights are not clipped, then they could be kept, rather than replaced with line-doubled low-ISO highlights.
If the high-iso data is not clipped, it's called "midtone" and rendered in full resolution.
As an example, I'd think the aliasing in #1 could look more like #2, if highlights were not half-res. And #3 could be clean.
(http://i.imgur.com/qfSiwH3.jpg)
Do you have a DNG frame that I could diagnose?
Quote from: a1ex on March 21, 2014, 10:07:49 PM
If the high-iso data is not clipped, it's called "midtone" and rendered in full resolution.
So to accomplish full-res highlights, do I, say, run DualISO at 100/800 and expose to the right minus 1.2EV?
Or if I run 800/100 and expose to the right, are high-ISO highlights automatically preserved?
Quote from: a1ex on March 21, 2014, 10:13:25 PM
Do you have a DNG frame that I could diagnose?
http://popspring.com/drop/M21-1222_00038.dng
Use raw zebras. Solid color appear where both ISOs are clipped, weak zebras appear where only one ISO is clipped (half-res) and no zebras appear where the scene is exposed properly.
In shadows, the issue is not as bad because data from the lower ISO can be still used, even if it's noisy. But in highlights, clipped data is pretty much synonymous to "I have no idea what might be there" (well, a clipped pixel does give a lower bound for guessing its value, but that's pretty much it).
About your DNG:
- I couldn't reproduce issue 1 (check your debayering method, I tried all of the methods from ufraw and rawtherapee, and only eahd seemed to struggle a little)
- issue 3 should be handled by the amaze-edge interpolation, can you upload the interlaced DNG so I could troubleshoot further?
I used ACR to debayer.
Here's the original: http://popspring.com/drop/M21-1222_00038_interlace.dng
QuoteUse raw zebras. Solid color appear where both ISOs are clipped, weak zebras appear where only one ISO is clipped (half-res) and no zebras appear where the scene is exposed properly.
I assume that by 'solid color' you mean black zebras?
By this I would expect black zebras when clipping at 100/100, since both ISOs must be clipped. But that's not the case, as the initial zebras are green.
In either case, I can't corroborate the statement through experimentation. Here is my repro:
1) Set Dual ISO to 100/100
2) Open iris until first zebras appear
3) Close iris 1/3 stop (zebras disappear)
4) Set Dual ISO to 100/800
5) Expect: zebras to appear, since the ISO 800 lines must now be clipping
6) OBSERVE: no zebras appear.
Reproduced (it's a bug, probably over-optimization for speed).
Workaround: take a still picture to check the exposure.
Got some improvement in edge aliasing.
Before/after:
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/stevefal/M21-1222-735-before.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/stevefal/M21-1222-735-after2.jpg)
I've also included an anti-posterization trick (http://www.magiclantern.fm/forum/index.php?topic=10895.msg107972#msg107972), which should improve shadow detail a little (no samples yet).
Updated binary: cr2hdr-20bit.exe (http://acoutts.com/a1ex/cr2hdr-20bit.exe)
Did not run any tests, so watch out for regressions.
edit: I think I've just created a regression, bug #1 in stevefal's sample.
edit2: fixed and reuploaded.
a1ex, can you build one for Mac possibly?
Nope, but I uploaded the source code too.
Call me a dummy but can you paste the link here? I need a project for today and compiling a program would be it!
old algorithm (http://upload.g3gg0.de/pub_files/6777ce8de478154444a32549a3805914/dual_iso_old.PNG)
new algorithm (http://upload.g3gg0.de/pub_files/13b6667819dcba7ca5bd7c14c11e3510/dual_iso_new.PNG)
looks a lot better at the edges.
image is a bit darker, but rarely visible.
good job :)
btw here (http://upload.g3gg0.de/pub_files/01a7469e43d77d11fd4ab080d562789c/cr2hdr.exe) is the windows .exe for this build.
use it on your own risk ;)
Quote from: a1ex on March 21, 2014, 11:36:31 PM
Updated binary: cr2hdr-20bit.exe (http://acoutts.com/a1ex/cr2hdr-20bit.exe)
Did not run any tests, so watch out for regressions.
edit: I think I've just created a regression, bug #1 in stevefal's sample.
edit2: fixed and reuploaded.
These last versions give many totally dark pixels (I mean absolute 0 without subtracting BL) spread over the frame and the OB areas,
- and the third line up from the bottom is corrupted with most pixels again at level 0 and some at 65535 ..
Sample is http://filebin.net/qlh1lwzjwt/DUAL3185.CR2
Quoteedit: I think I've just created a regression, bug #1 in stevefal's sample.
edit2: fixed and reuploaded.
I haven't been able to get cr2hdr-20bit.exe working yet. I use cr2hdr for OSX.
In either case, I want to point out that the aliasing I saw in #1 of my sample became prominent after I lifted the shadows. So I'm not sure how the fix looks under the same circumstances.
Bug: I can confirm latest 20bit outputs the last scanline of the dng as trash (I tested both 6d & 60d). The latest version also has some other regressions (i.e. I think it used to work better, but I didn't test all older cr2hdr) with detecting the white point on some tricky shots, I already pm'ed alex about that.
I am cross-linking a sample cr2 archive here, originally for the dual_iso wb thread, but I also made this archive for alex to have some 6d (vs. 60d) samples with a proper color checker. Beware, 600mb archive:
NEW FIXED ARCHIVE (60d shots were taken with mini_iso by mistake): https://bitbucket.org/Marsu42/ml-aiso/downloads/dual_iso_samples_6d_60d_FIXED.zip
Quote from: Marsu42 on March 24, 2014, 08:27:22 AM
Bug: I can confirm latest 20bit outputs the last scanline of the dng as trash
Same with 600D (g3gg0 compile).
Ok I dont get what im doing wrong, I drag the files over cr2hdr and its not doing anything? confused.
Quote from: g3gg0 on March 22, 2014, 04:29:27 PM
old algorithm (http://upload.g3gg0.de/pub_files/6777ce8de478154444a32549a3805914/dual_iso_old.PNG)
new algorithm (http://upload.g3gg0.de/pub_files/13b6667819dcba7ca5bd7c14c11e3510/dual_iso_new.PNG)
looks a lot better at the edges.
image is a bit darker, but rarely visible.
Is this from a1ex build above, or your own code?
The reduction in aliasing is excellent.
Quote from: Audionut on March 25, 2014, 03:23:22 AM
Is this from a1ex build above, or your own code?
The reduction in aliasing is excellent.
this is of course alex' latest change at this time.
(i uploaded a compiled one as his original post hadnt an .exe D/L)
Quote from: combat squirrel on March 25, 2014, 01:28:08 AM
Ok I dont get what im doing wrong, I drag the files over cr2hdr and its not doing anything? confused.
The g3gg0 compile is command line, no bath (see a1ex compile above it).
@a1exIs possible do a software for apply the function "--soft-film=%f" for non-dual_iso photos? I really like it, but sometimes I don't shot dual-iso...
For simulating analog photos, the presets do a curve push the white point to down, and the information is loss, but with soft-film, no.
Yes, my modified version of ufraw (ufraw-mod (http://www.magiclantern.fm/forum/index.php?topic=9597)) uses the same soft-film curve.
OT1: SET+MainDial exposure compensation from ML uses the same curve.
OT2: I submitted it to ufraw (http://sourceforge.net/p/ufraw/bugs/362/) a few months ago, but I guess my samples were not very convincing, or I submitted it in the wrong place, because there was no feedback from them.
Yeah, I see your mod of UFRAW, really impressive. But, I think putting the UFRaw in the middle of the workflow is a little bit complicated. I mean, obviously for images refined process ("fine art", II dont like this term), this is very useful, but for editing events (aka weddings, birthdays, etc), I think it is inefficient (for speed processes).
I was thinking of a bin batch file, like a cr2hdr, but that only applied their algorithm for soft-film one would export lossless DNG. Would it be possible? I personally consider very useful to use, for example, with the presets VSCO for a wedding or something.
Quote from: a1ex on March 26, 2014, 12:17:32 AM
OT2: I submitted it to ufraw (http://sourceforge.net/p/ufraw/bugs/362/) a few months ago, but I guess my samples were not very convincing, or I submitted it in the wrong place, because there was no feedback from them.
Looks like the 2 developers are no longer active.
Quote from: Luiz Roberto dos Santos on March 26, 2014, 01:13:26 AM
for editing events (aka weddings, birthdays, etc), I think it is inefficient (for speed processes).
What could be more efficient than a batch process that you start, go out and have a beer, come back in a few hours and find everything almost done?
www.magiclantern.fm/forum/index.php?topic=7022
(well, still have to automate photo selection, white balance, rotation, cropping, maybe denoising, but at least I get the colors right out of the box for 80% of pics, mixed normal and dual iso files)
(I had an attempt to automate rotation with the built-in electronic level - partial success, the implementation was only good as proof of concept, was imprecise and crashing often)
Lossless DNG: interesting idea (some sort of a raw pre-cooker). This could include many other things like FPN correction, deflicker (via the soft-film curve), highlight recovery, bad pixel fix, raw operations like average/median/max and so on.
BTW, you can hack the cr2hdr source to accept regular cr2's (just trick it into believing it's a dual ISO file, let it accept ISO 100/100, enter some valid line configuration, like 0,1,1,0, and optionally skip all the processing and jump to output).
A few updates to cr2hdr-20bit.exe (http://acoutts.com/a1ex/cr2hdr-20bit.exe)
- new exposure matching algorithm (no longer using histogram matching, but medians - should be more robust to noise and outliers)
- new white level heuristic (10th max for low ISO, 100th max for high ISO, minus safety margins - should fix cases where the signal doesn't clip harshly to white)
- fixes green tint (wrong black level) on 650D and maybe others
- fixes shots heavily overexposed at the top edge (6D and maybe others)
- fixes a line of gibberish at the bottom
- a tiny bit less noise (nothing worth showing though)
- from ayshih: fixed EXIF WB info to match Canon tags (http://www.magiclantern.fm/forum/index.php?topic=10265.msg108171#msg108171) (note: Canon WB tags may be wrong with dual iso, but at least they will get copied to the DNG)
Quote from: IliasG on March 23, 2014, 12:39:58 AM
These last versions give many totally dark pixels (I mean absolute 0 without subtracting BL) spread over the frame and the OB areas
Could not reproduce this one, can you show a crop where I should look?
For any Mac user, if you have Wine installed, move cr2hdr-20bit.exe, dcraw.exe, and exiftool.exe to an empty folder, move all your Dual-ISO CR2s to that folder, open up Terminal and write "wine cr2hdr-20bit.exe *".
It works.
Proof...(http://s17.postimg.org/s1s2anfnz/0_W5_A9730.jpg)
Quote from: a1ex on March 26, 2014, 03:20:47 PM
Could not reproduce this one, can you show a crop where I should look?
The crop, used exe , and two dngs one with the guilty cr2hdr-20bit (
did not try the last one) and one with the official cr2hdr from the first post (no problem there).
Edit: black pixels exist with the last version also,
not sure if they are the same ..
http://filebin.net/3379r86b43
http://filebin.net/3379r86b43/dual3185-patched.dng-crop00-738620-300.2.png
crop is 0,0 - 738,620 .. there are at least four black pixels visible in this crop ..
With the last build ..
- the crappy line at the bottomn disappeared
- (after excluding the 2 top lines) there are 122-125-126-130 black pixels in R-G1-B-G2 same as with the previous 20bit versions
With b13fcd5 (original 20-bit version), the darkest pixel from (5:620, 1:740) is 7884. With e6bba36 (today's version), it's 8090. Checked with dcraw -4 -E DUAL3185.DNG followed by octave:
im = imread('DUAL3185.pgm');
crop = im(5:620, 1:740);
min(crop(:))
I am on win-vista32 bit, is your result on linux ?. did you checked the dng that I uploaded ?.
Time for some pixel peeping - how dual ISO postprocessing (cr2hdr) evolved from the first public release until today?
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/batman-850-380.jpg)
More samples here: https://www.dropbox.com/sh/sta87fj2pghv6o4/vPXNvbhZdJ
(when reviewing them, I realized the 20-bit improvement is a lot smaller than I thought)
Amaze-edge looks the best to my eye.
https://dl.dropboxusercontent.com/sh/sta87fj2pghv6o4/LpuMU0LdTh/100_1600_dots-2100-2500.jpg?token_hash=AAGS40r4pFlArYQWiBzw4hNdPR1E8QhOm8tejubqAWFbzA
If your evaluation criteria is only the aliasing in that particular test chart, yes.
I looked at the group of 8 and pretty much eliminated the left group of 4. Among the right group of 4 i looked at aliasing and how the smoothing was already at the point of removing parts of detail. And also which handled color noise best. And also "color smear", for lack of a better term. Greenish noise is "slightly" more obvious in latest and previous compared to amze-edge.
The left group of 4 had FPN clearly.
Quote from: a1ex on March 26, 2014, 09:53:10 PM
Time for some pixel peeping - how dual ISO postprocessing (cr2hdr) evolved from the first public release until today?
On a subjective scale and w/o comparing pixels, I'd say the latest cr2hdr versions can deal even with my most oddly exposed shots and show amazing ability to raise shadow detail out of nowhere - well done!
The wb green/magenta problem persists atm as the latest exif copy attempt doesn't really work for me, but I'm used to find some reasonable wb settings by now manually and use the acr wb brush to remove tint casts. My one possible worry is that the 6d is somehow a bit different than 5d3 as I seem to be much more concerned about this, but then again maybe I'm just shooting different scenes.
Quote from: a1ex on March 26, 2014, 09:53:10 PM(when reviewing them, I realized the 20-bit improvement is a lot smaller than I thought)
Never mind, 20bit sounds cool, and even if it only helps a bit what is all that multi-core multi-gb ram stuff inside my laptop for :-p ... I don't even play games anymore!
Quote from: a1ex on March 26, 2014, 07:37:31 AM
What could be more efficient than a batch process that you start, go out and have a beer, come back in a few hours and find everything almost done?
I will test it ;D
Quote from: a1ex on March 26, 2014, 07:37:31 AM
Lossless DNG: interesting idea (some sort of a raw pre-cooker). This could include many other things like FPN correction, deflicker (via the soft-film curve), highlight recovery, bad pixel fix, raw operations like average/median/max and so on.
Yeah, I don't thought on first moment for this way, but this is a really good idea.
Quote from: a1ex on March 26, 2014, 07:37:31 AM
BTW, you can hack the cr2hdr source to accept regular cr2's (just trick it into believing it's a dual ISO file, let it accept ISO 100/100, enter some valid line configuration, like 0,1,1,0, and optionally skip all the processing and jump to output).
I will do this, I find it very useful.
Quote from: a1ex on March 26, 2014, 09:53:10 PM
Time for some pixel peeping - how dual ISO postprocessing (cr2hdr) evolved from the first public release until today?
Is not possible do the SSIM (structural similarity) analysis? As "original source", it would be possible to use supersampling/super-resolution, with various pictures, as in the case PhotoAcute (http://photoacute.com/tech/superresolution_faq.html).
That way we do not fall into the subjectivity...
Quote from: Luiz Roberto dos Santos on March 26, 2014, 11:14:16 PM
Yeah, I don't thought on first moment for this way, but this is a really good idea.
Btw I'd like to +1 this - dealing with high dr shots is a pita in current acr, the highlight recovery isn't enough so you need to apply a custom tone curve which is a big hassle.
For some shots, I'm currently experimenting with --soft-film even though the "baked in" wb doesn't sound too appealing. But if it really has no (other) drawbacks, it might be a good solution for high dr non dual_iso shots, too, at least until Adobe can handle these shots as DxO can.
Quote from: Luiz Roberto dos Santos on March 26, 2014, 11:14:16 PM
Is not possible do the SSIM (structural similarity) analysis? As "original source", it would be possible to use supersampling/super-resolution, with various pictures, as in the case PhotoAcute (http://photoacute.com/tech/superresolution_faq.html).
That way we do not fall into the subjectivity...
You can generate ground-truth images from two bracketed shots (lookup fake_dual_iso earlier in this thread). Maybe combine them with CeroNoice to get a noise-free estimation.
Regression: After running 20bit over my collection 2/3 of the shots are broken with Lightroom. This is NOT visible when viewing the dng with some other viewer like XnView, but in acr it's about +1ev brighter than legacy 16bit and has a somehow screwed tone curve. I'm sending alex a problem sample, but anyone beware.
I've noticed LR having a problem also. ACR reads them fine. Since it's intermittent, I haven't had a chance to do sufficient testing.
Quote from: Audionut on March 28, 2014, 04:12:44 PM
I've noticed LR having a problem also. ACR reads them fine. Since it's intermittent, I haven't had a chance to do sufficient testing.
I'm really puzzled atm - but it also might be that the 16bit version was broken all along, which means I have to re-process all of my 1500 dual_iso shots. Yippee.
I haven't noticed any problems with compatibility, with the 16bit build. Only the latest 20bit source.
Quote from: Audionut on March 28, 2014, 04:34:01 PM
I haven't noticed any problems with compatibility, with the 16bit build. Only the latest 20bit source.
I usually use 100/800 and underexpose for highlights, and I always had to add heavy +ec in Lightroom to get decent results (and of course adjust the broken wb). It might be a 6d specific problem... doh. The results are also ok after re-adjusting lr for the latest 20bit, but a couple of days work what you get for using bleeding edge software on an unsupported model :-p
Honestly, I think the biggest problem, is that you may simply be underexposing to much.
dual_iso does not fix shot noise (https://www.princeton.edu/~achaney/tmve/wiki100k/docs/Shot_noise.html). If you need clean shadows, it's not simply a case of increasing ISO to remove electronic noise, you also have to expose sufficiently also. Exposure is controlled solely with lens diameter, shutter, and aperture.
When you attempt to capture highlights, that you would otherwise simply blow to white, without dual_iso, you're more likely to lower exposure to capture these highlights. Here, you are pushing the midtones/shadows further into shot noise dominance. Note: Generally, this situation would be pushing the exposure into read noise dominance, but since dual_iso lowers the read noise significantly, shot noise becomes more relevant.
If you're underexposing these highlights, that you would otherwise blow to white, then the situation becomes worse. You've underexposed your midtones with dual_iso (to capture the highlights), and then you are underexposing even further.
You almost always have to increase exposure in LR, because you are underexposing your midtones with dual_iso.
Lets me try and show an example. Let's say that this is our scene lighting.
+------+------+
Lets say that the above represents 14 EV of dynamic range. The middle + represents the midtones, and these midtones are 7 EV below the highlights. Remember, normal ISO is rated for around 11 EV of DR, and this is at the noise floor of the camera. The DR capability of the camera, where noise levels are deemed to be significant, is actually closer to 9 EV of DR.
So in normal circumstances, you might expose as such.
+------+---+
In other words, you sacrifice 2-3 stops of highlight detail, to expose your midtones correctly. In raw ADU's, the midtones are around 4 EV from sensor saturation (overexposure).
So where we use dual_iso to capture these highlights (that would otherwise be overexposed), the midtone point has been shifted 2-3 stops lower (instead of being 4 EV below sensor saturation, it is now 6-7 EV below sensor saturation). Hence, in PP, you need to increase exposure 2-3 stops to compensate. This isn't a deficiency with dual_iso, it is an expected outcome. a1ex's soft-film highlight compression can be used to automatically compress these highlights, so that the midtone point of your exposure (the rendered image), is more closely aligned with the midtone point of a standard Canon image.
Here, you might not need to increase the exposure in PP, because soft-film is applying an exposure correction automatically. I will agree though, that in it's current state, soft-film is difficult to use, because it requires manual intervention regarding the burned WB values. a1ex has mentioned that he is interested in using WB processing, to automatically determine the "correct" value for soft-film use.
Remember, raw based processing, is all about the raw processor determining what the 0's and 1's are supposed to represent. What you have to do in PP is irrelevant. The determining factor, is ensuring that your exposure is correct, during capture.
Quote from: Audionut on March 28, 2014, 05:21:23 PM
Honestly, I think the biggest problem, is that you may simply be underexposing to much.
We had this discussion, I often just want absolute highlight safety for wildlife shots and thus the max. dynamic range, my goal is not to have optimal shadow noise but to prevent clipping in changing light. No matter that, dual_iso and cr2hdr should just work, no matter how you expose.
Quote from: Marsu42 on March 28, 2014, 05:48:11 PM
We had this discussion, I often just want absolute highlight safety for wildlife shots and this the max. dynamic range, my goal is not to have optimal shadow noise but to prevent clipping in changing light.
That's fine. You are free to make whatever exposure decisions you want. ;)
Quote from: Marsu42 on March 28, 2014, 05:48:11 PM
No matter that, dual_iso and cr2hdr should just work, no matter how you expose.
It does.
Quote from: Audionut on March 28, 2014, 05:51:59 PM
That's fine. You are free to make whatever exposure decisions you want. ;)
That's good to hear, and of course I'm always open to suggestions from our local wildlife experts :-p
Quote from: Audionut on March 28, 2014, 05:51:59 PM
It does.
Indeed. On 6d, since yesterday.
Quote from: Marsu42 on March 28, 2014, 06:13:32 PM
Indeed. On 6d, since yesterday.
That's cool. I hope you enjoy dual_iso. What a shame it only started working for you, yesterday.
any updates to cr2hdr for video? (no flicker mode)
Quote from: Audionut on March 28, 2014, 06:35:17 PM
That's cool. I hope you enjoy dual_iso. What a shame it only started working for you, yesterday.
Well, not quite as the awb for underexposed shots is still broken (and even for normal exposures it's other than Canon) - but I think it's getting there. At least I now know why I with the 6d and what I use it for was so worried about some dual_iso oddities other than the big 5d3 folk.
Girls, girls, you're both pretty. Why don't the two of you just grow up and shut up if you can't be polite in a public forum. Just PM each other and get it over with.
My two cents. Cheers!
Quote from: NedB on March 28, 2014, 08:35:20 PM
Girls, girls, you're both pretty. Why don't the two of you just grow up and shut up if you can't be polite in a public forum. Just PM each other and get it over with.
Thanks for your input Ned, I'll keep your post in mind as an inspiration as you could also have pm'ed your critique this to us - but now I know who's really grown up around here :->
Ok, that's enough off topic fun and games, from all of us.
I'm supposed to be setting an example ::)
Minor update to the 20-bit cr2hdr to fix a regression in bright/dark line detection (thanks g3gg0 for a sample). See my previous posts for download link.
AMAZING PROGRESS!!!
Hi, first of all thanks for the amazing effort to all who are contributing to ML!
Now, as I'm not sure where to post I'll ask here: On my 7D, using the latest nightly build (3-31) and the builds I've tried before, dual iso doesn't seem to be working in liveview, sometimes there are a few in a series of images that are dual iso, but roughly 80% aren't, just ordinary base iso files. When not using liveview all is well and all files are dual iso. Is there something I'm missing, some setting I've overlooked? I've tried searching this forum but to no avail.. I'm trying to use it for pano's, so LV would be nice to have, and having a pano with mixed dual and normal iso images just doesn't work ;)
Any help would be really appreciated!
Do you mean regular pictures in LiveView, or silent pictures?
Also make sure you are using photo mode LiveView, not movie mode.
A1ex, April 1st tomorrow... Sensor Up...!!!! ;D
Thanks for the reply. Regular or silent doesn't seem to make a difference, when I shoot using te viewfinder I get dual iso files, when I switch to LV and don't change anything else I get normal iso files. I'm not in movie mode either.
Alright; will check next week. Meanwhile you can file it on bitbucket on the issues page.
Quote from: budafilms on March 31, 2014, 10:12:59 PM
A1ex, April 1st tomorrow... Sensor Up...!!!! ;D
https://nikonhacker.com/viewtopic.php?f=2&t=1840 :P
Quote from: budafilms on March 31, 2014, 10:12:59 PM
A1ex, April 1st tomorrow... Sensor Up...!!!! ;D
Wait a minute what?
Thanks a1ex, I've filed it on bitbucket.
With DNG at least in ACR/Lightroom, there is the option to embed a (color) calibration profile. Would it be possible to add the soft film curve as (part of) this embedded profile so users could dynamically switch it on and off w/o the need to re-process with cr2hdr?
Currently, the embedded profile after cr2hdr is "Adobe Standard", or more likely there is no embedded profile at all so Lightroom falls back to "Adobe Standard".
Quote from: a1ex on March 31, 2014, 11:20:26 PM
Poisson d'avril from budafilms :P
Ha! oh duh im slow.......
Quote from: a1ex on March 31, 2014, 11:20:26 PM
Poisson d'avril from budafilms :P
Jajajajaj :D
I am imagine something like sensor protocol?
It's a huge thing....
Quote from: kgv5 on February 26, 2014, 09:57:05 AM
Is there any way to have dualISO mlv files "tagged"? When using ETTR connected with dualISO we often get mixed files (normal and dualiso) and the only way to know if the clip is dualISO or not is to open one of the DNGs (which takes a lot of time with dozens of files). If we have DI files with some different name it would be possible to easly filter such a files out of the rest and process them with any dual iso app (or script) right away. Is metadata of MLV module has a "room" for such a tagging?
EDIT: Maybe something like 'custom file prefix' but for movies.
+1
and also can someone fix the flickering problem with dual iso video? i really dig dual iso video but i don't know how to code, i don't think its a big thing to do but i've been wrong in the past.
cheers
Quote from: a1ex on March 26, 2014, 03:20:47 PM
A few updates to cr2hdr-20bit.exe (http://acoutts.com/a1ex/cr2hdr-20bit.exe)
..
Could someone post a working link? I got 403 forbidden code when I clicked the link. Would like to try the 20-bit version.
Use right mouse button -> Save as ...
Any Update for cr2hdr mac users?
Quote from: Walter Schulz on April 02, 2014, 04:29:08 PM
Use right mouse button -> Save as ...
Unknown server error.. Can other people download it just fine?
Maybe your antivirus software doesn't like this one.
@a1ex: ZIP/RAR would be fine. Will prevent file version/date problems, too.
Quote from: Walter Schulz on April 02, 2014, 07:25:23 PM
@a1ex: ZIP/RAR would be fine. Will prevent file version/date problems, too.
And reduces the bandwidth requirements by about 2/3rds.
The latest version is mirrored here:
https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/cr2hdr-20bit.zip
Some progress with dual iso postprocessing for video / panoramas / timelapse / whatever else needs matching exposures between the frames (it should no longer flicker). To use it, specify the
--same-levels option on the command line.
cr2hdr-20bit.exe (http://acoutts.com/a1ex/cr2hdr-20bit.exe)
This is not a deflicker algorithm, so it requires manual exposure settings. It simply tries to keep the same difference between WhiteLevel and BlackLevel for all the files passed in the same command line. More details on the commit message (https://bitbucket.org/hudson/magic-lantern/commits/c0c8fecfadbb).
On Windows, you may have some trouble passing a bunch of files on the command line (you can't use wildcards there, and for drag and drop you may have to create a batch file that also passes --same-levels), but I didn't try. On Mac/Linux it should work without much trouble.
I also did a small shell script to render a Dual ISO video to AVI: dualisovideo.sh (https://bitbucket.org/hudson/magic-lantern/src/cr2hdr-20bit/modules/dual_iso/dualisovideo.sh). You may want to tweak it for each video (include exposure, white balance and other options), but it should do the job.
Here's an example, rendered with the above script (an older sample from papkee, which flickered a lot before this fix). Oddly enough, it's from 650D, but I didn't apply any pink dot fix...
This workflow is not very fast - for a 100-frame test video at 1920x1080 it took only 15 minutes, and for this 150-frame clip at 1472x626 it took 8 minutes (this includes the entire conversion from the captured .raw file to the final .avi file). There's a lot room for improvement in cr2hdr, and you don't need any hacking skills to do that (hint, hint).
I already did some small speed optimizations (you can check the changelog).
Quote from: a1ex on April 08, 2014, 08:58:40 PM
On Windows, you may have some trouble passing a bunch of files on the command line (you can't use wildcards there)
For all your windows scripting needs, use either cygwin+bash or the free tcc/le (4dos/4nt successor) and run "select cr2hdr.exe --same-levels [dual*.cr2]": http://jpsoft.com/all-downloads/downloads.html
Nice! I guess msys/mingw can do the same thing then (they also have a bash-like shell).
Here's another script for rendering dual iso videos, this time with tonemapping via enfuse:
dualisovideo-enfuse.sh (https://bitbucket.org/hudson/magic-lantern/src/cr2hdr-20bit/modules/dual_iso/dualisovideo-enfuse.sh).
For the sample clip, it took 12 minutes (so the time needed for ufraw+enfuse becomes comparable to the time needed for cr2hdr).
I uploaded two versions, with and without tonemapping, so you can check whether the tonemapping process introduces any flicker.
BTW, make sure you use ffmpeg from the ffmpeg web site (http://www.ffmpeg.org/download.html) (the one that comes with ubuntu is actually the libav fork, and it's buggy at least for this video).
Quote from: a1ex on April 08, 2014, 10:24:03 PM
Nice! I guess msys/mingw can do the same thing then (they also have a bash-like shell).
To be correct: Cygwin has *the* latest and original bash shell which works 99.9% like in other unix flavors as cygwin emulates a full posix system. You can even make the ntfs file system case-sensitive through a windows registry entry... that's why I think having cygwin as a full ml dev environment is a good idea as it doesn't need admin rights to run unlike a vm.
Problems on the dev side usually only occur when trying to use autoconf/automake scripts that aren't aware of cygwin just like some some still ignore macosx/bsd. On the user side mixing cygwin with dos/windows programs without taking the different line endings into account might be tricky as cygwin also uses the unix-like lf. Different behavior otherwise is very far in between, most prominently with awk inline scripts which I fixed here
https://bitbucket.org/hudson/magic-lantern/pull-request/467/fix-cygwin-compilation-docs
Thanks a1ex! Do you have the apple binary?
Quote from: a1ex on April 08, 2014, 08:58:40 PM
Some progress with dual iso postprocessing for video / panoramas / timelapse / whatever else needs matching exposures between the frames (it should no longer flicker). To use it, specify the --same-levels option on the command line.
Excellent work. To give an idea of the changes.
Here are 2 exposures from a panorama, that are correctly brightness matched with the latest cr2hdr.
(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/forum/_46a0142.jpg) (https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/forum/_46a0143.jpg)
And here is the same 2, using an old cr2hdr.
(https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/forum/_46a0142.jpg) (https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/forum/badlevels.jpg)
This makes batch processing significantly easier, since you don't also have to brightness match images.
edit: Removed outdated build.
idea:
cr2hdr + bracketing of 3(2,3,4...n) photos
simultaneously stapled dualiso and N pictures with different exposures
-->Output one dng file
possible to do this?
Take a look here: http://www.magiclantern.fm/forum/index.php?topic=9581
Main problem: you will need more than 16 bits, so you'll need a floating point DNG. But the only program that can open them is Adobe SDK (not Lightroom nor ACR). In some extreme cases, even the plain dual iso files will push the limit of the 16-bit integer DNGs.
(well, you can also open floating point DNGs with my hacked copy of ufraw (http://www.magiclantern.fm/forum/index.php?topic=9597), but that's far from a proper solution)
Quote from: a1ex on April 14, 2014, 12:26:27 PM
Take a look here: http://www.magiclantern.fm/forum/index.php?topic=9581
Main problem: you will need more than 16 bits, so you'll need a floating point DNG. But the only program that can open them is Adobe SDK (not Lightroom nor ACR). In some extreme cases, even the plain dual iso files will push the limit of the 16-bit integer DNGs.
(well, you can also open floating point DNGs with my hacked copy of ufraw (http://www.magiclantern.fm/forum/index.php?topic=9597), but that's far from a proper solution)
Interesting as Lightroom and ACR can definitely open floating point TIFFs (both 16 and 32 bits per component)...
I believe it only opens linear (demosaiced) DNGs, but didn't try to output this kind of DNGs yet.
But you can do the conversion with the DNG SDK, as described here: http://www.magiclantern.fm/forum/index.php?topic=9581.msg92195#msg92195
Quote from: jpaana on April 17, 2014, 08:48:15 AM
Interesting as Lightroom and ACR can definitely open floating point TIFFs (both 16 and 32 bits per component)...
Is this true for many or most of the LR / ACR versions, or the more recent versions like LR 5.x or ACR 8.x (which share the same "engine").
My impression is that A1ex doesn't use LR / ACR, but I could very well be mistaken on that.
I tried DualISO at Canon 5D MArkIII with new ML.
I read before that vertical resolution will be decreased, but I was surprised that all ok with resolution.
Just wanted to share - I used Dual ISO mode outdoors today on a real-estate shoot where you were under a lot of big large oak trees (lots of shadows and dark areas) but the sky was a beautiful bright blue out, and the pictures just came out, so, so awesome. The dynamic range has always been my #1 complaint of my T3i and this seriously has fixed it. Its not over processed looking or anything, it just works awesome.
Not a technical response given the nature of this thread but just wanted to say thank you x9999999 ;D
Is there any real chance to see Dual-Iso Raw Video on a 7d in the near future?
I played with dual iso for a while, and now I' am addicted to it.
Few days ago I had the necessity to restore an image of my OS (winxp-Sp3- 32bit).Now I can't get cr2hdr working anymore.
I have spent several hours in the last two days downloading and installing a dozen different zip and files from any source I could find on the net. I had re-read a lot of post here, on ML forum, about dual iso.
I j had luck to convert 1 single dual iso image by starting cr2hdr from command line, but processing one file at a time from command, is too boring task.
I also tried Barracuda Gui, but it crash....
Possible questions/solutions:
1) do anyone know where to download a working version for win xp? (I'm just guessing it could be an Os dependant issue)
2) is it there some other file required that I'am missing in my OS installation (apart cr2hdr.exe/ dcraw.exe /exiftool.exe?)
3) is it cr2hdr.exe the only tool to process dual iso cr2, or an alternative exist?
4) is it possible to process several files with cr2hdr.exe from command line?
Thanks for any help
(I feel I'am dumb for asking this, sorry :-(
Is there any way to use dual iso and end up with a cdng file?
5d3 1.2.3
mac osx mavericks
Best
David
Quote from: ToniX on April 27, 2014, 07:22:44 PM
Few days ago I had the necessity to restore an image of my OS (winxp-Sp3- 32bit).Now I can't get cr2hdr working anymore.
I have spent several hours in the last two days downloading and installing a dozen different zip and files from any source I could find on the net. I had re-read a lot of post here, on ML forum, about dual iso.
The first post of this thread, contains the latest stable build of cr2hdr for windows.
You can drag and drop a bunch of files onto the exe, and it will batch convert all of them.
@a1ex tried it last days the first time, and was a little bit scared of the first converting run. putting here, doin that, getting this.. but in combination with the lr-plugin by kichetof its a nobrainer-oneclick-thing. nobel peace prize 2014 for you and your code and 2015 for kichetof for the one-click-solution. really. now it seems i have to investigate time to code the video-dualISO feature into my converter.
Quote from: chmee on April 28, 2014, 02:06:37 AM
@a1ex tried it last days the first time, and was a little bit scared of the first converting run. putting here, doin that, getting this.. but in combination with the lr-plugin by kichetof its a nobrainer-oneclick-thing. nobel peace prize 2014 for you and your code and 2015 for kichetof for the one-click-solution. really. now it seems i have to investigate time to code the video-dualISO feature into my converter.
finally! glad to hear this!
Quote from: Audionut on April 27, 2014, 10:14:40 PM
The first post of this thread, contains the latest stable build of cr2hdr for windows.
You can drag and drop a bunch of files onto the exe, and it will batch convert all of them.
Audionut..
... it doesn't work: cr2hdr just flash and then close when I drag a file on it, while dcraw say that "not valid win32 application" ... (that's strange because dcraw 9.19 is working fine, for istance ), that's make me think is a 64 bit version...
It works fine here.
Create a new folder on a hard drive. Extract the contents of the cr2hdr zip to that folder (cr2hdr.exe, dcraw.exe, exiftool.exe). Don't use any other versions.
Copy a dual_iso file to that folder.
Drag and drop the CR2 over the cr2hdr exe.
What version of windows are you using?
:D Audionut, I am investigating why that version (as any other version I could find) it doesn't work anymore for me (I've got it work before, I am using win xp pro sp3 32 bit).
Beside the fact that cr2hdr.exe just appear and then disappear when I drag a dual iso image on it, I also noted that the dcraw.exe in this version is not a valid 32bit application, so I can argue it can't work.
Possible corrupted files?
Recent update
I've got cr2hdr.exe work again, but very strange way...
I dowloaded and installed as many older dcraw.exe versions from here
http://www.insflug.org/raw/Archives, until cr2hdr started to run....
and now it run perfectly, even if I remove dcraw.exe ....
does it make sense?
BTW, I'm really happy I can continue shoot dual iso photo. :) :D :D
I really thought I understood Dual-ISO.
'All' I wish to do is A-ETTR to look after the highlight and simply add on a fixed dual that I specify, eg 100/800.
I set midrange and shadow to off in ETTR and dual to on.
I set min shutter to 1/100, but in my experiments shutter is 1/200 via ETTR.
I take an ETTR reading and dual doesn't kick in, ie I get 100/100.
As I say I simply wany dual to always 'add on top of' ETTR.
Can someone point out what I'm doing wrong?
Cheers
Set dual iso link to off, set recovery iso to +3, done.
Damn it Alex, you posted before I had a chance to say how stupid my question was :-o)
I realized soon after my post I simply had to decouple the ETTR link to Dual.
As I say, a bad 'ML Monday'.
Cheers
ToniX
I suppose you could have multiple versions and some of them is in"path" so windows will find that.
Search order sarts from same path as default IMHO.
Quote
cr2hdr should work out of the box:
In terminal enter:
sudo apt-get install build-essential
hg clone -r unified https://bitbucket.org/hudson/magic-lantern
cd magic-lantern/modules/dual_iso
make cr2hdr
sudo cp cr2hdr /usr/local/bin
hmm doesn't work... am I missing something?
make cr2hdr
../../modules/Makefile.modules:22: warning: overriding commands for target `delete_if_failed'
../../modules/Makefile.modules:22: warning: ignoring old commands for target `delete_if_failed'
../../modules/Makefile.modules:39: warning: overriding commands for target `module_strings.h'
../../modules/Makefile.modules:39: warning: ignoring old commands for target `module_strings.h'
../../modules/Makefile.modules:42: warning: overriding commands for target `README.rst'
../../modules/Makefile.modules:42: warning: ignoring old commands for target `README.rst'
../../modules/Makefile.modules:45: warning: overriding commands for target `dual_iso.mo'
../../modules/Makefile.modules:45: warning: ignoring old commands for target `dual_iso.mo'
../../modules/Makefile.modules:50: warning: overriding commands for target `dump_asm'
../../modules/Makefile.modules:50: warning: ignoring old commands for target `dump_asm'
../../modules/Makefile.modules:69: warning: overriding commands for target `hgstamp'
../../modules/Makefile.modules:69: warning: ignoring old commands for target `hgstamp'
[ gcc ] cr2hdr
cr2hdr.c: In function 'module_get_string':
cr2hdr.c:344:34: error: '__module_strings_MODULE_NAME' undeclared (first use in this function)
cr2hdr.c:344:34: note: each undeclared identifier is reported only once for each function it appears in
cr2hdr.c: At top level:
module_strings.h:1:13: warning: '__module_string_a_name' defined but not used [-Wunused-variable]
module_strings.h:2:13: warning: '__module_string_a_value' defined but not used [-Wunused-variable]
module_strings.h:3:13: warning: '__module_string_b_name' defined but not used [-Wunused-variable]
module_strings.h:4:13: warning: '__module_string_b_value' defined but not used [-Wunused-variable]
module_strings.h:5:13: warning: '__module_string_c_name' defined but not used [-Wunused-variable]
module_strings.h:6:13: warning: '__module_string_c_value' defined but not used [-Wunused-variable]
module_strings.h:7:13: warning: '__module_string_d_name' defined but not used [-Wunused-variable]
module_strings.h:8:13: warning: '__module_string_d_value' defined but not used [-Wunused-variable]
module_strings.h:9:13: warning: '__module_string_e_name' defined but not used [-Wunused-variable]
module_strings.h:10:13: warning: '__module_string_e_value' defined but not used [-Wunused-variable]
make: *** [cr2hdr] Error 1
gcc version 4.6.4 (Ubuntu/Linaro 4.6.4-1ubuntu1~12.04) 64bit
What happens if you do this change?
#define MODULE_STRINGS_SECTION
+#define MODULE_NAME MODULE_NAME
#include "module_strings.h"
I figured I missed python-docutils
after I installed it, it worked.
thanks for fast response though!
will alter my raw linux post accordingly, and get a binary up there.
@a1ex After a five days of real-life testing and rapture I got a few ideas of further improvement.
- Add dual iso settings data directly in cr2 meta. Why: after shooting thousand of photos I can't remeber which settings I used and can't learn which is the best. And it will be easier to batch-postprocess photos if I sort them by "second ISO". Maybe just add few keywords ("subject" exif field), no need for custom tags.
- Sequence "No Dual ISO shot / Dual ISO shot ". Why: I not always sure if Dual ISO is good for the scene or not. And on dynamic events I have no time to open settings and enable/disale dual iso for each shot. Is it too hard to implement by your judgement?
- Sequence "Dual ISO RAW / Small Jpeg Low Iso / High Iso". Why: nowadays I have to convert cr2 on my desktop before I know I made it ok. But often I'm in travel weeks from any PC, and it's a huge chance I mess with settings and understand it when too late. But by looking at (yes, even unaligned) jpeg previews I will got an idea what's going on in a place.
1. how?
2. it's already there, from the very first implementation
3. not sure I understand what you mean
@a1ex,
1. Maybe text sidecar file(s) if it is not possible to write metadata directly. I can postprocess it later with LR plugin.
2. Sorry, couldn't find it. I'll check again.
3. Two jpegs as previews with two ISO settings. I realize it would be impossible to implement.
1. Right, sidecar could work. We could start with a generic metadata library, I guess XMP is the obvious choice, to accept various settings that normally are in EXIF. Once we understand how to change the CR2 EXIF, we can modify the backend. Some related discussion here: http://www.magiclantern.fm/forum/index.php?topic=6226
3. The embedded JPEG is generated from the raw data, and it's possible to modify it. Here's an experiment on this: it does a simple dual iso processing right in the camera, so you can get the preview adjusted for either the high or the low ISO: http://www.magiclantern.fm/forum/index.php?topic=8667
However, the code is quite fragile, slow, and it's very likely to cause weird issues, including data loss. ML has to modify the raw buffer to generate the JPEG preview, then undo the changes before saving the CR2. All of this happens in Canon's picture taking tasks (SCS state & friends), which are very complex.
I might revisit it once I'll understand the picture taking process better.
Until then, raw histogram and raw zebras are your friends. Maybe also ML's playback exposure adjustment, sharpness 0, and monochrome picture style.
Quote from: a1ex on May 01, 2014, 09:13:10 PM
We could start with a generic metadata library
One step further towards some keyword tags in the sidecar file :-)
Quote from: a1ex on May 01, 2014, 09:13:10 PMand monochrome picture style.
Good thinking there, I think this deserves a "set picstyle to monochrome" setting in dual_iso to eliminate the magenta/cyan distraction!
Quote from: Marsu42 on May 02, 2014, 12:29:40 AM
I think this deserves a "set picstyle to monochrome" setting in dual_iso to eliminate the magenta/cyan distraction!
From a grayscale JPEG, I even hope to be able to get a decent preview for checking the focus :D
(if you are bored, upload a screenshot of a dual iso picture, taken with monochrome style and sharpness 0, and zoomed in playback at 100% on some sharp detail; I might use it as test data for the algorithm)
Quote from: a1ex on May 02, 2014, 12:34:37 PM
if you are bored
In that case, I'd work on my modules :-p ... though I could probably manage a pull request for monochrome picstyle on dual_iso_is_active(), but I'd really really rather have you do it in 10 seconds.
The added goodie of this solution is that (at least on 6d) you can make the camera display a ! in the *viewfinder* that is otherwise inaccessible to ML so you could immediately see if dual_iso is enabled or not!
Good point.
Meanwhile, I think I've got the preview correction (https://bitbucket.org/hudson/magic-lantern/commits/5412e3373adf) working fairly well, with a very simple algorithm: if the pattern repeats every N lines (usual values are from 2 to 5, depending on zoom level), filter it with a Nx1 box blur twice (which will average out the lines), then sharpen the result with a 3x1 filter to bring back some details. Works on color shots too, but keeps the weird cast :)
Quote from: a1ex on May 02, 2014, 03:54:44 PM
Good point. [...] Meanwhile, preview correction working fairly well [...] but keeps the weird cast
I cannot test it atm since my laptop's sd card reader is going crazy, but sounds like a reasonable approach. Still, I'd +1 for the monochrome option since (next to the ! sign) the color cast is really distracting and doesn't help much to predict the final look, even with some dual_iso experience - ymmv.
Done. It took a little more than 10 seconds, and it's not yet fully idiot-proof.
I'm starting to see the need for a backend for persistent storage, separate from the config file. Config can be enabled or disabled, but this persistent storage should be always on.
Common scenario: you change some Canon setting (here, the picture style), and you want to undo that change at shutdown (or, if the shutdown was unclean, at next reboot). The GPS trick from 6D is similar (change at shutdown, remember you changed it, restore at next boot). H.264 sound, now disabled by FPS override, could make use of it too (since you end up with sound disabled if you take the battery out while recording). Same for the AF button (picture taking routines, like bracketing, will move it to the back button to disable autofocus, so if you take the battery out in the middle of a bracket, you may end up with AF button misconfigured).
The file prefix is not persistent IIRC (need to double-check on 5D3, but on 60D and 5D2, Canon will reset it back to IMG_ at next boot).
A higher-level API could try to run all these cleanup routines at shutdown, and if the shutdown was not clean, they can be scheduled for next boot.
Battery discharge rate (now saved in the config file) could make use of this persistent storage too.
Quote from: a1ex on May 02, 2014, 05:55:38 PM
Done. It took a little more than 10 seconds, and it's not yet fully idiot-proof.
Yeah, that's why I like delegating tasks :-p because in my ML experience things are seldom as easy as they appear: multitudes of different situations pop up out of nowhere, not to mention the support for all eos models which you'll never see or own. Still, you're the most qualified person for dealing with the unexpected, thanks :-) !
Hi All,
Sorry if this isn't the best place to post this but i'm trying to test out one of your recent cr2hdr-20 bit updates but am having a bit of a battle with the dcraw and exiftool.
What happens when I drop a CR2 or DNG onto the cr2hdr executable is the command prompt flashes open then closes immediately. However I caught it open in this screen shot so perhaps it will give some clue as to the problem. Again i'm sure this is basic but you could just save my bacon if you can help me get this working.
https://www.dropbox.com/s/shzqgvmk6lbbwr3/Screen%20Shot%202014-05-04%20at%2014.43.52.png (https://www.dropbox.com/s/shzqgvmk6lbbwr3/Screen%20Shot%202014-05-04%20at%2014.43.52.png)
Looks like the CR2 files are on a network path; try copying them to a local directory.
Quote from: a1ex on May 04, 2014, 09:59:12 PM
Looks like the CR2 files are on a network path; try copying them to a local directory.
... or assign a drive letter to the unc path, many windows programs still cannot handle them. You'll still copy the temporary files over the network two times, if that's a problem.
Speaking of which: Probably cr2hdr deserves an option to assign a temp working directory where to copy/process the intermediary files, people could also use a ram disk or their fastest drive where the temp dir should be. This would also be handy if processing dual_iso files directly from the sf/cf card.
cr2hdr is processor bound.
If cr2hdr cached a number of files in a ram drive, it may speed up processing by some very small percentage. Pointless imo.
Quote from: Audionut on May 05, 2014, 02:59:49 AM
cr2hdr is processor bound.
Not if you're processing the files directly from the cf/sd card as the temp files also end up there, you have to copy the files first off the card or you're killing the card with additional r/w cycles and the speed is crawling. This and the unc path problem imho are enough to warrant a "temp file path" option.
Quote from: joshuamk on May 04, 2014, 03:52:32 PM
Hi All,
Sorry if this isn't the best place to post this but i'm trying to test out one of your recent cr2hdr-20 bit updates but am having a bit of a battle with the dcraw and exiftool.
What happens when I drop a CR2 or DNG onto the cr2hdr executable is the command prompt flashes open then closes immediately. However I caught it open in this screen shot so perhaps it will give some clue as to the problem. Again i'm sure this is basic but you could just save my bacon if you can help me get this working.
https://www.dropbox.com/s/shzqgvmk6lbbwr3/Screen%20Shot%202014-05-04%20at%2014.43.52.png (https://www.dropbox.com/s/shzqgvmk6lbbwr3/Screen%20Shot%202014-05-04%20at%2014.43.52.png)
Hi Joshua,
I had this exact same problem the other night trying to get it to run on XP via Bootcamp.
After much frustration, I did get it working by doing the following:
I downloaded an earlier version of cr2hdr from https://bitbucket.org/a_d_/magic-lantern/downloads (https://bitbucket.org/a_d_/magic-lantern/downloads). The one dated 2013-10-04.
I then took the dcraw and exiftool files and copied them into c:/windows/system32.
Cr2HDR is a command line application. It relies on dcraw and exiftool. Basically, by placing them in the system32 folder, the command line knows exactly where to find them.
Finally, I downloaded the most current version of Cr2Hdr from the dual ISO thread page here:
http://www.magiclantern.fm/forum/index.php?topic=7139.0 (http://www.magiclantern.fm/forum/index.php?topic=7139.0). Go ahead and delete the dcraw and exiftool that come with it.
You should now be able to use the application like normal -- by dragging your files over it.
Hope this solution works well for you like it did for me.
-Daniel
Hi All,
Thank you all for the feedback 10 points! RTLdan your step-by-step was well appreciated.
The images I get out of the 20 bit version are much better. Just waiting on an after effects output then we'll see how the aliasing stands up to scrutiny when played back.
Can't thank you enough I'm glad I got involved in the forums.
Just processed a bunch of images and there is some improvement over the original mac GUI that I have been using (as you would expect).
But I do seem miles away from the kind of smooth edges that A1ex had demonstrated in the dng of the batman figure. http://www.magiclantern.fm/forum/index.php?topic=7139.1925 (http://www.magiclantern.fm/forum/index.php?topic=7139.1925)
Here are two sections from a frame i have been testing cr2hdr on. the first was done using the original mac GUI which was released on 02/12/2013 and the second I processed today with what I was sure was the latest build. But now looking at them I am questioning that I have this right.
https://www.dropbox.com/s/2gt40u0k80nx459/crop1.jpg (https://www.dropbox.com/s/2gt40u0k80nx459/crop1.jpg)
https://www.dropbox.com/s/80py07v30n2qid4/crop%20new.jpg (https://www.dropbox.com/s/80py07v30n2qid4/crop%20new.jpg)
I'm certain i'm making an error here because there is barely any difference regarding the aliasing in particular (which to me is the first hurdle to getting usable dual iso video).
The batman image was taken using the same iso spacing and from what I can tell had more edge aliasing to contend with.
How can I get better results? Which build should I be using to re-create the results that have been demonstrated so far.
Rotate your camera by 45 degrees ;)
haha nice. If only that were an option. Prefer not to shoot video in 'tallscreen'
Hey guys, ive been going crazy here, i been trying to use the latest: cr2hdr-20bit.exe a1ex has provided, it is stored in "C:\0" along side dcraw.exe and exiftool.exe files in the same folder, some dual iso dng files process fine, but then i get to a certain file and it cause cr2hdr-20bit.exe to crash forcing me to click the close program button, it is not just one video, its many, it works completely fine with older versions of cr2hdr, like the 2013.10.04 version, also 2014.01.23, but they dont have the "same levels" parameter options that i need to keep the sequence flicker free. i found other versions of dcraw, exiftool, replaced them, same thing, so im assuming its cr2hdr.exe file, or something wrong with me, i even tried on another computer, rebooting it, cant think of anything else, i shot another video, same thing, it will crash on certain frames, im very sure the frames are fine as they have the same dng file size as its neighbouring files and they open fine in photoshop, bridge, etc, tried running with the --same-levels parameters initially, same results if i clicked and dragged dng files onto the original exe, or the shortcut with the parameters, tried individual files, all the same, tried redownloading your latest cr2hdr file, same
Hope to hear something back because im lost, i dont understand why this version crashes, am i the only one?
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
In particular, upload a broken CR2.
here you go:
https://www.dropbox.com/sh/d5g3jlugmqqkjdl/8-fIwn3kRg
It doesn't crash here. Do you have a screenshot when it crashes? Does it crash if you try to process only that file, or only when you try to process many files at once?
A1ex, the Linux version that's in the linux ML tools post, is the 20 bit one? Or there are just .exe builds of it? I use linux and I'm trying the 20 bit by wine, but I would like to build it native if possible...
Great work! Thanks!
You can identify it from the changeset number (it prints this at startup), then look it up in Bitbucket.
But on Linux it should be very easy to build your own (make cr2hdr).
Well, the previous version build up without problems in my netbook with Ubuntu, but I didn't see the 20 bit one, I'll look for it. In my desktop with opens use, it builds, but doesn't work, I have to find some time to find why... But meanwhile, with wine in my desktop one is much faster than my poor netbook with native... :-)
I tried the "cr2hdr same levels" build, what I found in this post (http://www.magiclantern.fm/forum/index.php?topic=7139.msg110882#msg110882) to post process the pieces of a spherical panorama and 2 from the 7 dngs got vertical stripes after the drag and drop (all of the 7 cr2s).
My camera was in M mode, Dual ISO 100/1600, f8, 1/30 and -1 EV, I'm using the nighlty apr. 14.
In Lightroom: shadow +69 and highlight -69. Exported to jpg 100% quality, 300dpi produces this (https://www.dropbox.com/s/53qs0fylrp9jpiv/stripes.jpg). (deleted file, problem solved see #2031)
Am I doing something wrong or is this a bug?
Looks like a bug, can you upload a CR2?
yepp, it's here (https://www.dropbox.com/s/ji72riy1z2zkv5t/IMG_2135.CR2). (deleted file, problem solved see #2031)
sorry, the stripe is vertical if we see in landscape view, but my camera was in portrait direction because of the panoramic head. :P
Could not reproduce (tried with the 16-bit one and two 20-bit versions). Can you post the conversion log?
It happens with that cr2hdr bulid, where the same levels command is embeded.
QuoteFor anyone interested, here is the build from a1ex above, with the --same-levels command parameter set by default.
https://dl.dropboxusercontent.com/u/34113196/Dual-ISO/forum/cr2hdr-same-levels.zip
This will allow you to simply drag and drop files on the exe, for auto processing with --same-levels.
With the 20-bit version and with the version what is in the main post I can't see the stripes.
How to save the conversion log and how could I compile a 20-bit cr2hdr, where the same levels command is embeded? I'm on windows.
To save the conversion log, you can use copy/paste. I no longer need it; it's just FYI.
To enable drag and drop with some custom options (in this case, --same-levels), you can create a .bat file (no need to compile a custom binary). I think the syntax is something like this:
cr2hdr --same-levels %*
Found the problem:
The first good revision is:
changeset: 10751:4c6fa8b9faba
branch: cr2hdr-20bit
user: a1ex
date: Wed Apr 09 16:44:35 2014 +0300
summary: cr2hdr: fix overflows in RGGB/GBRG/bright/dark detection
updated the folder with screenshots of the crash, versions of cr2hdr that crash and dont crash, and dcraw
https://www.dropbox.com/sh/d5g3jlugmqqkjdl/8-fIwn3kRg
i tried it on two computers, maybe im using wrong version of dcraw or something? im using the one from the first page of this forum topic
cant think of any other variables, hope you can give some advice as to what im doing wrong
exiftool located in the directory where cr2hdr and dcraw are?
i removed it, the first page said it was optional, i tried it with it being in the same folder, same results, but i will try again and include it in the dropbox folder
update: yup, tried it with exiftool in same folder, same results as before with 20bit cr2hdr (crashes), but with 2014.01.23 version, converts correctly, added exit tool to folder: https://www.dropbox.com/sh/d5g3jlugmqqkjdl/8-fIwn3kRg
Same error message?
Thanks for the explanation a1ex.
I have this "cr2hdr-20bit-same-levels.bat" file
cr2hdr-20bit --same-levels %*
pause
in the folder, where my cr2s are and I setup the path environmental variable for the cr2hdr folder (cr2hdr-20bit.exe, exiftool.exe and dcraw.exe), so it should run well, but I get this:
Unknown option: --same-levels
update: I think I have to define the input and output somehow and after that run the "--same-levels" command only for the dngs or how?
Just updated
cr2hdr-20bit.exe (http://acoutts.com/a1ex/cr2hdr-20bit.exe).
Fixes:
- fixed some valgrind warnings (ShaunWoo, can you try?)
- yet another algorithm for exposure matching, which should solve some corner cases like this (http://a1ex.magiclantern.fm/bleeding-edge/isoless/stripe-fix-before-after.jpg)
New features:
- --skip-existing (https://bitbucket.org/hudson/magic-lantern/commits/7fad96785821644a05dfb9d4212791e6e1bc7706): if you pass a large number of files in a single command line, this will skip the input files for which the output file already exists (useful if it crashes in the middle, and you want to resume). There is a small question for you in the commit message. (https://bitbucket.org/hudson/magic-lantern/commits/7fad96785821644a05dfb9d4212791e6e1bc7706)
- --embed-original (https://bitbucket.org/hudson/magic-lantern/commits/26b7a08d2540b62c72bce3f40ad929e4e14367a5): this moves the original CR2 into the output DNG, so you'll be able to re-process the DNG with either different options or with another version (of course, newer than this one). Discussed here (http://www.magiclantern.fm/forum/index.php?topic=8336).
- --wb=graymed (https://bitbucket.org/hudson/magic-lantern/commits/8b19cff1db594907ffb14422e67e04692dbff00d): another auto white balance algorithm (median of R-G and B-G). I still like --wb=graymax, so I made this one the default (will battle-test it these days). Discussed here. (http://www.magiclantern.fm/forum/index.php?topic=10265.75)
@surami: try again with this one.
oh wow, congrats a1ex, you fixed it, now i will try with the parameter same levels, in a batch convertion of 100 files from 0 upwards, hopefully it wont crash and no flickering, if all good, ill be very happy, well done dude, few notes and suggestions:
may i ask, what was the issue, did the screen shot help figure it out?
few days ago before i knew the cause of flickering i used the script for running multiple instances of cr2hdr, i asume it wont work since the same levels needs to be used within the initial single process, right? if thats the case, how about instead of using "same levels", we assing a specific variable to run with a parameter that sets the exposure, so this way each instance using the script would process the same level, speeding up the processing for video sequences greatly (hopefully its not something i missed in the forum)
the above i believe is necessary for longer sequences with many frames, if i try to click and drag even a 100 files, it will give me an error saying file path too long
also, how about putting a list of versions of cr2hdr in the very first post, i found the actual list of versions, but its the code only not the exe file, and im no where near a coder so i wouldnt know what to do with the code, im sure it would benefit many people
maybe theres a way to put the latest file in the first post? i gotta be honest it took me a few days to figure out the flickering cause, at first i found its ACR, shadow and highlight changes, then cr2hdr, i processed some NON dual iso sequences and there was no flickering what so ever, (even with process 2012 in acr), so i dont think its ACR issue, so yeah, would be useful for many people i think if the latest file was in the first post
big respects for your work and the speed of your fixes, just making suggestions for problems ive encountered as a regular user that im sure many other users would experience (finding the right information. i.e latest cr2hdr file)
:)
NOTE FOR THOSE WHO DO NOT KNOW ABOUT RUNNING PARAMETERS, make a shortcut of the "cr2hdr-20bit.exe" file, right click on shortcut > properties, in the target, copy and paste --same-levels at the end of the taget, if the target has spaces in the url, put " at the beginning of the url and at the end of it, for example:
"C:\camera raw dual iso converter\cr2hdr-20bit.exe" --same-levels
putting this info here to save time for people who dont know (like me)
also do you know anyway of evening out the flickering in the dngs after processing them with the old cr2hdr that didnt have the same levels parameters
Right, using --same-levels with parallel processing is a little harder (because all the files are analyzed at the end, and then it decides a white level that hopefully matches all of them). But it can be done after a regular conversion, like this:
exiftool *.DNG "-WhiteLevel<BlackLevel"
exiftool *.DNG "-WhiteLevel+=50000"
You may need to fiddle with the values though; lower the value if you get pink highlights, increase it if you think you can squeeze some more highlight detail.
Thank you for your responce, just processed 100 files, seems the exposure is the same, but the luma changes, ill post process with after effects ACR now
When you have some free time could you please go over the complete messege i sent you above please, i think i highlight some points many other people would question and come across, especially high frame counts, (i cant process 600 for example using 1 process)
and thank you again for your batch processing advice, but i really dont understand what to do with those commands, any chance you could link me to a discussion, guide, or anything that would point me in the correct direction? do i still use cr2hdr and the script for multiple instances? where would i put in those parameters, in the script?
Hope you understand my ignorance in coding
@ShaunWoo: I just thought out the same as you, that there is no need for .bat file, it works with shortcut. :)
@a1ex: Thanks for the update. The same levels command works with the shortcut trick, but the results for a spherical panorama (situation: daylight, indoor: no electriclights, outdoor: full bright sunshine) isn't enough good. As I see the dngs, they should be analyzed again after the same levels algorithm and there should be a match overall brightness trick after that, which would be based on the same leveled dngs' histograms. This algorithm should find the middle one from the batch processed dngs and there should be an exposure compensation (something like deflicker). Now I make the matching manually and the result is verygood. Tomorrow (today) I have much to do, but as I have time, I will upload the a full set of the sphere pieces and I'll send a pm with a download link for testing.
Quote from: ShaunWoo on May 06, 2014, 11:26:37 PM
we assing a specific variable to run with a parameter that sets the exposure
What should the exposure be? Are you aware of the problems from setting the white/black levels, to high/low?
Quote from: ShaunWoo on May 06, 2014, 11:26:37 PM
maybe theres a way to put the latest file in the first post?
For the 20 bit version? The 20 bit version is still somewhat experimental.
Thanks for the shortcut trick.
isnt there a default exposure? like when you open with ACR its on 0, so i asume theres a default, am i wrong? if that is true, then is there a way to set on default= 0, or something like that, rather than the program auto adjusting it
im having a difficult time because i need to process around 700 frames sequence but i cant since it says the file path is too long, i am able to process 500, but what is the solution for this problem, does anybody know? :)
Quote from: ShaunWoo on May 07, 2014, 03:15:11 AM
isnt there a default exposure? like when you open with ACR its on 0, so i asume theres a default, am i wrong? if that is true, then is there a way to set on default= 0, or something like that, rather than the program auto adjusting it
You mean midtones at 0 EV? Because dual_iso files are rendered dark with default settings?
Quote from: Audionut on March 28, 2014, 05:21:23 PM
You almost always have to increase exposure in LR, because you are underexposing your midtones with dual_iso.
Lets me try and show an example. Let's say that this is our scene lighting.
+------+------+
Lets say that the above represents 14 EV of dynamic range. The middle + represents the midtones, and these midtones are 7 EV below the highlights. Remember, normal ISO is rated for around 11 EV of DR.
So in normal circumstances, you might expose as such.
+------+---+
In other words, you sacrifice 2-3 stops of highlight detail (overexpose them), to expose your midtones correctly. In raw ADU's, the midtones are around 4 EV from sensor saturation (overexposure).
So where we use dual_iso to capture these highlights (that would otherwise be overexposed), the midtone point has been shifted 2-3 stops lower (instead of being 4 EV below sensor saturation, it is now 6-7 EV below sensor saturation). Hence, in PP, you need to increase exposure 2-3 stops to compensate. This isn't a deficiency with dual_iso, it is an expected outcome. a1ex's soft-film highlight compression can be used to automatically compress these highlights, so that the midtone point of your exposure (the rendered image), is more closely aligned with the midtone point of a standard Canon image.
Here, you might not need to increase the exposure in PP, because soft-film is applying an exposure correction automatically.
The files are being rendered correctly, because there is significantly more data above the midtone point, then without dual_iso.
You can use --soft-film to compress the highlights.
"Highlight/shadow handling", (struct cmd_option[]) {
{ (int*)&soft_film_ev, 1, "--soft-film=%f", "bake a soft-film curve to compress highlights and raise shadows by X EV\n"
" (if you use this option, you should also specify the white balance)"},However IIRC, soft-film is nonlinear. So this would not be useful in a situation where your rendered images need to be exposure matched.
Quote from: ShaunWoo on May 07, 2014, 03:15:11 AM
im having a difficult time because i need to process around 700 frames sequence but i cant since it says the file path is too long, i am able to process 500, but what is the solution for this problem, does anybody know? :)
Quote from: a1ex on May 06, 2014, 11:33:20 PM
Right, using --same-levels with parallel processing is a little harder (because all the files are analyzed at the end, and then it decides a white level that hopefully matches all of them). But it can be done after a regular conversion, like this:
exiftool *.DNG "-WhiteLevel<BlackLevel"
exiftool *.DNG "-WhiteLevel+=50000"
You may need to fiddle with the values though; lower the value if you get pink highlights, increase it if you think you can squeeze some more highlight detail.
Rather then fiddling with values, I would probably process a bright and dark image with cr2hdr --same-levels, note the black/white levels printed, and use these with exiftool.
If you use the shortcut trick from above, you should just be able to drag and drop all the files onto the shortcut. With 700 files though, and --same-levels requiring single threaded processing, again, I would probably just process a bright and dark image, and use the printed output with exiftool. This way, you can process the files with cr2hdr, with a multi-threaded solution (significantly faster), and just use exiftool at the end.
in acr process 2012, just below white balance, its exposure first, i was refering to that, im afraid im not as well educated as you guys, but as far as im seeing theres exposure, contrast, shadows, highlights, etc, i dont see a midtone option in acr, so yeah, i was refering to absolute exposure, but i guess its different with cr2hdr
what do you mean process a bright and dark image? do you basically mean the dual iso file? note the black/white levels printed, would they be printed as results for each dng processed individually in cr2hdr?
and nope, when i click and drag around 700+ files onto the cr2hdr, or the shortcut for it, it will give me the error, file name too long, im very sure youve come across that error as ive seen people post about it before in this forum, but i havent seen a solution, if you havent i can post a screenshot, so im still limited to around 500 files
oh and i think i understand some of what you said now, you say process a black and white level image, im sure you must mean a dual iso now, once that is process, cr2hdr will print on cmd the black and white levels on screen, right? and use this setting printed with the rest of the sequence using the script that runs multiple instances of cr2hdrm with the code a1ex posted?:
exiftool *.DNG "-WhiteLevel<BlackLevel"
exiftool *.DNG "-WhiteLevel+=50000"
just change the variable with the one printed in cmd?
however once again i dont know what to do with this code, where do i input it in, is it for the cr2hdr file? use it the same way as the --same-levels parameter? the only way i know how to run THAT parameter alone is single line, how would i input two line code in there
also you say use exiftool at the end, do you mean after the cr2hdr process? what is the purpose of this if the file has already been processed and ready for PP
There are 2 ways to process dual_iso files that need to be exposure matched.
cr2hdr --same-levels
exiftool *.DNG "-WhiteLevel<BlackLevel"
exiftool *.DNG "-WhiteLevel+=50000"
Exiftool (http://www.sno.phy.queensu.ca/~phil/exiftool/).
So, rather then using --same-levels with cr2hdr, you can use exiftool. And yes, use exiftool after processing with cr2hdr. And since you are using exiftool, you
do not need to use --same-levels with cr2hdr.
Quote from: ShaunWoo on May 07, 2014, 06:19:30 AM
in acr process 2012, just below white balance, its exposure first, i was refering to that, im afraid im not as well educated as you guys, but as far as im seeing theres exposure, contrast, shadows, highlights, etc, i dont see a midtone option in acr, so yeah, i was refering to absolute exposure, but i guess its different with cr2hdr
I don't understand what you want to do with this "absolute exposure".
Quote from: surami on May 07, 2014, 12:26:41 AM
The same levels command works with the shortcut trick, but the results for a spherical panorama (situation: daylight, indoor: no electriclights, outdoor: full bright sunshine) isn't enough good. As I see the dngs, they should be analyzed again after the same levels algorithm and there should be a match overall brightness trick after that, which would be based on the same leveled dngs' histograms.
The --same-level option is meant for pictures taken with the same exposure settings in the camera (in manual mode). It's not a substitute for deflickering.
Deflickering for panoramas is very hard, because, when deflickering, the algorithm must know which parts of the image are going to overlap. The robust statistics from my deflicker algorithm (median, percentiles) are not robust enough for matching two shots with very different framing.
Also be careful of the flicker introduced by ACR. If in doubt, try to process the images with ufraw (none of its adjustments will introduce flicker). Make sure you are using manual white balance.
Quote from: ShaunWoo on May 07, 2014, 06:19:30 AM
when i click and drag around 700+ files onto the cr2hdr, or the shortcut for it, it will give me the error, file name too long
If you reach the command-line length limit, I'm afraid the only options are:
- don't use --same-levels, but run exiftool to equalize the levels after a regular conversion
- try an operating system with a larger limit (mine has around 2 MB, Windows has 8K or 32K (http://joshyjf.wordpress.com/2010/10/21/max-length-of-command-line-arguments/))
- bug Microsoft to increase the limit (you paid for it, so they should help you, right?)
If you don't want to fiddle with levels, your best bet is to copy them from an overexposed shot from your sequence. If you have used manual exposure settings in the camera, and you want the exposures matched when rendering, you need to have the WhiteLevel - BlackLevel (their difference) constant for all your files (and that's why my exiftool commands do).
For example, to copy the settings from one image to all the others:
# read the levels from a reference image
exiftool IMG_1234.DNG -WhiteLevel -BlackLevel
White Level : 47752
Black Level : 2032
# now, copy the levels difference (WhiteLevel - BlackLevel) to all other files in the sequence
exiftool *.DNG "-WhiteLevel<BlackLevel"
exiftool *.DNG "-WhiteLevel+=45720"
Updated cr2hdr-20bit.exe (http://acoutts.com/a1ex/cr2hdr-20bit.exe) as discussed here (http://www.magiclantern.fm/forum/index.php?topic=8336.msg114278#msg114278) (mostly tweaked the --embed-original functionality).
a1ex can you provide a new binary for Mac too?
Nope, but I think you may ask @kichetof here, since he uses Mac: http://www.magiclantern.fm/forum/index.php?topic=11056.0
$ ./cr2hdr
cr2hdr: a post processing tool for Dual ISO images
Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
cr2hdr: print a message when overwriting the output file
Download cr2hdr-20bit (https://bitbucket.org/kichetof/lr_cr2hdr/downloads/cr2hdr.30a5132.zip) for Mac only with last commit 30a5132 ;)
@a1ex thanks for having solved the warnings on compilation!
Quote from: kichetof on May 07, 2014, 06:36:39 PM
$ ./cr2hdr
cr2hdr: a post processing tool for Dual ISO images
Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
cr2hdr: print a message when overwriting the output file
Download cr2hdr-20bit (https://bitbucket.org/kichetof/lr_cr2hdr/downloads/cr2hdr.30a5132.zip) for Mac only with last commit 30a5132 ;)
@a1ex thanks for having solved the warnings on compilation!
Well i drag a file on top the cr2hdr icon and nothing happened...
what am i missing?
(old version of cr2hdr for mac is installed)
Quote from: kichetof on May 07, 2014, 06:36:39 PM
$ ./cr2hdr
cr2hdr: a post processing tool for Dual ISO images
Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
cr2hdr: print a message when overwriting the output file
Download cr2hdr-20bit (https://bitbucket.org/kichetof/lr_cr2hdr/downloads/cr2hdr.30a5132.zip) for Mac only with last commit 30a5132 ;)
@a1ex thanks for having solved the warnings on compilation!
Many thanks!
@DavidSh I've this error, so it's normal you don't have anything... I think it's a problem with dcraw, maybe to old version I'll find tonight if I've the time
$ ./cr2hdr _MG_2824.CR2
cr2hdr: a post processing tool for Dual ISO images
Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
cr2hdr: print a message when overwriting the output file
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
Input file : _MG_2824.CR2
Camera : Canon EOS 5D Mark III
Camera model : Canon EOS 5D Mark III
dcraw could not open this file
My version of dcraw
dcraw
Décodeur de photo raw "dcraw" v9.20
par Dave Coffin, dcoffin a cybercom o net
@dubzeebass happy to help!
-----------------------------------------
Solved!
You need dcraw 9.21 (download dcraw 9.21 for MAC Only (https://bitbucket.org/kichetof/lr_cr2hdr/downloads/dcraw9.21.zip) compiled on my mac)
@a1ex: I uploaded a comparison (https://www.dropbox.com/s/jvylh2uq0tc8yhl/comparison.zip) (deleted, see #2059), where you can see what I speak about. I processed the cr2s (shooting -> M, Dual ISO: 100/1600, f8, 1/30s, wb: daylight) with the newest cr2hdr-20bit without --same-levels command and after that I imported them into Lightroom. On the histogram jpgs you can see, that the pieces of the sphere panorama has the same settings (wb, exp, shad, highl), I exported them as 16bit TIF and stiched them in PTGui, so I got the panorama_01.jpg (It is resized and I cut the bottom part of the panorama). You can see the dark part (computer desk, chair, window, couch, till door on the right and a part of the ceiling) on the panorama. Take a look at the panorama_02.jpg, at this processing I pushed up the exposure manually (let's say expose to the middle) at the darker pieces and the problematic part looks much better after the stiching.
So if I see the histogram at the second processing all of the pictures are shifted to the middle. Is there a way to analyze the full histogram + setup a base exposure shift for all dngs (let's say +1,5EV) + after that setup a reference picture manually (let's say, this picture is the perfect middle) + after that shift (+/-) the other pictures exposure to the middle automatically? Of course there will be some difference between the pictures after the export (at panorama_02.jpg I had minimal differences), but PTGui done the rest shift at blending/stiching.
Update: When I shoot this panorama, I metered the light in 2 directions at -1EV (to the lightest and to the darkest) in AV mode and after that I switched to M mode and setup the avarage between this two.
Can you upload the CR2 files for shots 1 and 2? (these two seem quite different)
I already explained that histogram matching in this case does not make sense.
Yes, I will send the download link for you in pm as soon as it will be uploaded.
Quote from: kichetof on May 07, 2014, 07:56:38 PM
@DavidSh I've this error, so it's normal you don't have anything... I think it's a problem with dcraw, maybe to old version I'll find tonight if I've the time
Thanks man,
Best
David
Quote from: surami on May 07, 2014, 09:49:32 PM
Yes, I will send the download link for you in pm as soon as it will be uploaded.
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/surami-pano.png)
The exposure is matched in areas that overlap (check the spotmeter values). Therefore, the feature works as designed.
Thanks for test a1ex, now it's clear.
hey again everyone, i been trying to find some guide for exiftool with no luck, can anyone point me to a guide or a discussion for it please? i dont understand how to use it or input commands or anything, i run the .exe, click and drag a single file or a sequence onto it, it shows up some text, then an
-- More -- Lines:
with numbers at the end of lines, i cannot input text, only numbers, so i dont see how i am supposed to input the code that a1ex provided me to even out the levels to make the sequence "same levels" rather than using cr2hdr's same levels, so i can process sequenceses with 500+ frames,
also, the script for running multiple instances, now crashes every version of cr2hdr that is ran, does the script need to be updated for the latest version?
hope to hear some feedback
if it helps i can provide screenshot if needed of anything, just let me know what you need them off
if it helps, im using, windows 7 64bit ultimate, 5d mk3, mlv conversion using mystic PP with after effects
http://www.sno.phy.queensu.ca/~phil/exiftool/
From the commandline:
exiftool --help
to transfer text to file
exiftool --help>>help.txt
and open help.txt in your editor of choice.
If you downloaded exiftool from queensu you will get "EXIFTOOL(-k).EXE". All strings in brackets will work as arguments. In this case you will execute "EXIFTOOL.EXE -k". To get rid of it just rename or create a copy just named "EXIFTOOL.EXE" and work with this file.
Ciao
Walter
It bothered me yesterday, that I can't shoot a nice Dual ISO sphere panorama, so I tried it again at nearly the same light conditions and the result is quite good. (I think I didn't meter the light correctly the day before yesterday and I set up my camera wrongly, thats why there was a big dark part on that panorama.)
What I did now:
1. Whitebalance: daylight; picturestyle: standard, f8, ISO 100, in ML: Dual ISO 100/800
2. AV mode: metered (evaulative metering) the light in the room (turned around with the panorama head) in every direction, so this way I found the max. and a min. shutter speeds. I didn't touch the exposure compensation slider. (The day before yesterday it was -1EV.)
3. Switched to M mode: adjusted the shutter speed exactly between the max. and min. values (so this is the middle). There was 11 steps, so I originated from the metered min. and turned the wheel by 5 steps.
4. Shot the panorama.
5. Processed the cr2s with the lates cr2hdr-20bit without any command parameters.
6. I imported the dngs into Lightroom and made this adjusments: whitebalance: 5000; highlight: -100; shadows: +100, whites: -10; blacks: -10 and removed the cromatic aberration. I didn't touch the exposure. Exported to 16bit TIF, 300dpi.
7. Aligned the TIFs in PTGui and created the panorama as a 8 bit TIF.
8. I opened the panorama in Photoshop and adjusted the shadows by: 5% and pushed up the brightness to 21 and saturatoin to 10. After that I saved as jpg, quality 12.
Here is a small resized result.
(https://dl.dropboxusercontent.com/s/gevbr0e37l3c5v1/panorama_02_small.jpg) (deleted)
Sorry for this big image here, how can I thumbnail it, I saw here at many post? (see #2064)
You can use the [img width=] tag. Check the edit of your post.
Rather then metering the midtones of the entire scene (Canon Av mode), you might want to look at AETTR (http://www.magiclantern.fm/forum/index.php?topic=5693.0).
AETTR will meter for the highlights. So in your above shot, you would not have the blown out areas, in the window on right, and both floor areas. With dual ISO, the shadows remain clean.
In that scene (recovering all highlights), I would try dual ISO 100/1600.
Thanks for the "width" tag.
I also thought about the AETTR, but in this situation there is too much difference between the frames. If I start to shoot with AETTR always on, then in the 1st direction (from the 6 horizontal + 1 zenit) it will be fine after the sample shots (2/3 shots till get the right settings). The problem is there, that the panorama head turns automatically by 60 degrees after 1 shot, so the AETTR sample shooting sequence will start again and after the 1st sample shot the head turns again... and of course the next frame also different from the previous so everything will be bad. :) So it's a shoot-move-shoot head.
The shooting round would be fine till the 2nd direction, it looks like this:
0. (0 degree): AETTR always on, making the sample shots to get the right settings.
1. (0 degree): Start the panorama shooting sequence, result a perfect AETTR shot.
2. (60 degree): Head moved to the next frame, but totally other light conditions, so the AETTR starts again to get the right settings, but at the 1 shot head turns...
3. (120 degree): AETTR believes the it is at the 60 degree direction, but not. :D so again...
I don't know what to do, but I think there is solution somehow, that's why I started to experiment with middle shutter value.
Update: The best would be to fix an avarage AETTR setting somehow for all, and fix the Dual ISO too and shoot everything (all the 7) with same settings. Maybe this way:
1. AV mode to find the middle manually by turn around.
2. Find that direction where the expo comp slider shows 0.
3. Switch to M mode: AETTR always on, make the sample shooting sequence and fix the perfect one somehow and setup Dual ISO 100/1600. Can this be done somehow?
Something like this or I don't know. Maybe you understand what I think.
If you don't care about the images being exposure-matched in overlapped area (I assume you don't, otherwise you would have used M mode and --same-levels), just enable Post Deflicker, and meter each image as you wish (Av or ETTR).
Of course, in this case, the recommended way to use ETTR is on the SET key (and meter before each shot). The Always On mode should be used when the exposure is unlikely to change between shots (because it meters from the previous shot).
I got the sense that the OP did want exposure matching, since he was looking for some middle exposure setting.
In which case, use AETTR just for the brightest exposure. That is to say, with the camera positioned to capture the really bright window, in the above scene (the one that is overexposed), use AETTR to find the optimal shutter/aperture combination. That window is the brightest object in the scene, so we know that this is exposed as far right of the histogram as possible (with AETTR).
In M mode (turn off AETTR), with the shutter/aperture combination from AETTR earlier, I would simply enable dual ISO 100/1600, and capture the frames needed.
Process the dual ISO images with cr2hdr and the --same-levels setting.
i downloaded the version you just linked me to, and this one doesnt even let me type in anything, the only thing it lets me do is press enter which will reveal more information on exiftool, if i press the p character, it will display "Lines:", but if i type in numbers, any other characters, they wont come up, so i cant enter --help, with the version i originally had and the one you provided me with, i tried dragging and dropping in a DNG file, and it just gives me a list of file types, and at the end:
-- More -- Lines: 22814260000400
really seems confusing just to correct exposures for a sequence :(
@kichetof
Hi man,
Have you find out if its a dcraw problem preventing the cr2hdr 20bit to work on mac?
Is there something a user like me can do to help?
Best
David
@ShaunWoo: Exiftool is a so called "commandline" tool. It's "environment" is call CLI (command line interface).
You're running Windows 7/8?
Open Explorer (no, not Internet Explorer), mark the folder where exiftool is located. Now press SHIFT, open context menu (mouse right click) and you have an option "Open command window here". Do so and a whole new world will open up. Well, "new" may not be the proper description ... <eg>.
You will see a black window with some white/grey letters like
C:\users\yourname\downloads\exiftool>_
where the "cursor" is blinking constantly. Now type
exiftool --help
This command line was - once upon a time - the personal computer's very interface for users to access their programs, files, interfaces.
You can run CMD.EXE to access CLI, too. Use commands
cd <directory name>
cd \
cd ..
to access the directory you want.
Ciao
Walter
@DavidSh you need to update dcraw ;) (replace file to /usr/bin/ or use "which dcraw" to locate its exact folder)
Quote from: kichetof on May 07, 2014, 07:56:38 PM
Solved!
You need dcraw 9.21 (download dcraw 9.21 for MAC Only (https://bitbucket.org/kichetof/lr_cr2hdr/downloads/dcraw9.21.zip) compiled on my mac)
@a1ex could you add a way into cr2hdr.c to check if dcraw and exiftool exist into the same folder of cr2hdr exec ? (on Mac or Linux, when you execute the command without ./ exec comes from PATH but not from the same folder, I know: I don't learn you anything :))
With that, we can include into the same folder dcraw, dcraw.exe, exiftool, exiftool.exe and use the right version (most useful for my LR plugin with the newest version of dcraw needed)
Ouch! I think it will not possible unless if we can compile an executable of exiftool with embedded libraryIts works but I need to copy all lib files into the same folder of exiftool :)
$ ls -R
File Image exiftool
./File:
RandomAccess.pm
./Image:
ExifTool ExifTool.pm
./Image/ExifTool:
AES.pm ITC.pm Photoshop.pm
AFCP.pm Import.pm PostScript.pm
AIFF.pm InDesign.pm PrintIM.pm
APE.pm JPEG.pm Qualcomm.pm
APP12.pm JPEGDigest.pm QuickTime.pm
ASF.pm JVC.pm README
Apple.pm Jpeg2000.pm RIFF.pm
BMP.pm Kodak.pm RSRC.pm
BZZ.pm KyoceraRaw.pm RTF.pm
BigTIFF.pm LNK.pm Radiance.pm
BuildTagLookup.pm Lang Rawzor.pm
Canon.pm Leaf.pm Real.pm
CanonCustom.pm M2TS.pm Reconyx.pm
CanonRaw.pm MIE.pm Ricoh.pm
CanonVRD.pm MIEUnits.pod Samsung.pm
CaptureOne.pm MIFF.pm Sanyo.pm
Casio.pm MNG.pm Scalado.pm
Charset MPC.pm Shift.pl
Charset.pm MPEG.pm Shortcuts.pm
DICOM.pm MPF.pm Sigma.pm
DNG.pm MWG.pm SigmaRaw.pm
DPX.pm MXF.pm Sony.pm
DV.pm MakerNotes.pm SonyIDC.pm
DarwinCore.pm Matroska.pm Stim.pm
DjVu.pm Microsoft.pm TagInfoXML.pm
EXE.pm Minolta.pm TagLookup.pm
Exif.pm MinoltaRaw.pm TagNames.pod
FLAC.pm Nikon.pm Theora.pm
FLIR.pm NikonCapture.pm Torrent.pm
Fixup.pm NikonCustom.pm Unknown.pm
Flash.pm Nintendo.pm Vorbis.pm
FlashPix.pm OOXML.pm WriteCanonRaw.pl
Font.pm Ogg.pm WriteExif.pl
FotoStation.pm Olympus.pm WriteIPTC.pl
FujiFilm.pm OpenEXR.pm WritePDF.pl
GE.pm PDF.pm WritePNG.pl
GIF.pm PGF.pm WritePhotoshop.pl
GIMP.pm PICT.pm WritePostScript.pl
GPS.pm PLIST.pm WriteQuickTime.pl
GeoTiff.pm PNG.pm WriteXMP.pl
Geotag.pm PPM.pm Writer.pl
H264.pm PSP.pm XMP.pm
HP.pm Panasonic.pm XMP2.pl
HTML.pm PanasonicRaw.pm XMPStruct.pl
HtmlDump.pm Pentax.pm ZIP.pm
ICC_Profile.pm PhaseOne.pm iWork.pm
ID3.pm PhotoCD.pm t1
IPTC.pm PhotoMechanic.pm
./Image/ExifTool/Charset:
Arabic.pm Latin.pm MacCyrillic.pm MacLatin2.pm PDFDoc.pm
Baltic.pm Latin2.pm MacGreek.pm MacRSymbol.pm ShiftJIS.pm
Cyrillic.pm MacArabic.pm MacHebrew.pm MacRoman.pm Symbol.pm
Greek.pm MacChineseCN.pm MacIceland.pm MacRomanian.pm Thai.pm
Hebrew.pm MacChineseTW.pm MacJapanese.pm MacThai.pm Turkish.pm
JIS.pm MacCroatian.pm MacKorean.pm MacTurkish.pm Vietnam.pm
./Image/ExifTool/Lang:
cs.pm es.pm ja.pm ru.pm zh_tw.pm
de.pm fi.pm ko.pm sv.pm
en_ca.pm fr.pm nl.pm tr.pm
en_gb.pm it.pm pl.pm zh_cn.pm
Quote from: kichetof on May 08, 2014, 07:47:40 PM
@DavidSh you need to update dcraw ;) (replace file to /usr/bin/ or use "which dcraw" to locate its exact folder)
@a1ex could you add a way into cr2hdr.c to check if dcraw and exiftool exist into the same folder of cr2hdr exec ? (on Mac or Linux, when you execute the command without ./ exec comes from PATH but not from the same folder, I know: I don't learn you anything :))
With that, we can include into the same folder dcraw, dcraw.exe, exiftool, exiftool.exe and use the right version (most useful for my LR plugin with the newest version of dcraw needed)
Ouch! I think it will not possible unless if we can compile an executable of exiftool with embedded library
$ ./exiftool
Can't locate Image/ExifTool.pm in @INC (@INC contains: ./lib /Library/Perl/5.16/darwin-thread-multi-2level /Library/Perl/5.16 /Network/Library/Perl/5.16/darwin-thread-multi-2level /Network/Library/Perl/5.16 /Library/Perl/Updates/5.16.2 /System/Library/Perl/5.16/darwin-thread-multi-2level /System/Library/Perl/5.16 /System/Library/Perl/Extras/5.16/darwin-thread-multi-2level /System/Library/Perl/Extras/5.16 .) at ./exiftool line 30.
Thanks Man,
does cr2hdr have to be in the same folder (usr/bin)? because drag n drop option does not working
Quote from: DavidSh on May 08, 2014, 08:24:10 PM
Thanks Man,
does cr2hdr have to be in the same folder (usr/bin)? because drag n drop option does not working
Nop! Only dcraw need to be into /usr/bin
Quote from: kichetof on May 08, 2014, 08:46:47 PM
Nop! Only dcraw need to be into /usr/bin
and drag n drop works for you?
I can only use command line, and i dont know how and if command line works with folders or multiple files..
Quote from: DavidSh on May 08, 2014, 08:51:51 PM
and drag n drop works for you?
I can only use command line, and i dont know how and if command line works with folders or multiple files..
/usr/bin is a private folder, you need to copy with root privileges.
Before copy, make a backup of your actual version
sudo mv /usr/bin/dcraw /usr/bin/dcraw.9.20 #<-- I like to put the version of the file I backup, to know your version simply run dcraw into terminal
cd /folder/where/you/downloaded/dcraw
sudo cp dcraw /usr/bin/
Quote from: kichetof on May 08, 2014, 09:27:39 PM
/usr/bin is a private folder, you need to copy with root privileges.
Before copy, make a backup of your actual version
sudo mv /usr/bin/dcraw /usr/bin/dcraw.9.20 #<-- I like to put the version of the file I backup, to know your version simply run dcraw into terminal
cd /folder/where/you/downloaded/dcraw
sudo cp dcraw /usr/bin/
Kichetof,
Thank you for your kind a detailed answer, much appreciate.
But i wrote about me not able to use cr2hdr as a gui with drag n drop dual iso files.
I had no problems to copy the dcraw to usr/bin with command line.
Best
David
Oh sorry ! hum I doesn't try. I only use in terminal or LR ;)
Quote from: kichetof on May 08, 2014, 11:31:59 PM
Oh sorry ! hum I doesn't try. I only use in terminal or LR ;)
Never mind man, you already helped a lot ! :)
is there a way to use terminal command line on a folder instead of one file? i looked at the option and didnt find any...
yes you can but I doesn't try it (just write now)
#!/bin/bash
for i in *.cr2
do
cr2hdr $f
done
Or if you use lightroom, use my plugin :)
cr2hdr *.CR2
8) its too late for my brain!
So simple! Thanks a1ex
Got flicker free Dualiso video working yesterday with the 20 bit cr2hdr on Mac, thanks again! I had to run a command the set the same whitelevel for all of them and then zero flickering. Great job guys!
Did anyone experience strange artifacts happening on long exposures? I did a 60 second shot at 100/400 and it contains an awful lot of colorful triangles.
See example (http://szmigieldesign.pl/s/dualisotriangles).
They are stuck/hot pixels - the longer the exposure the more you are going to have. For a 60s photo that many is not unusual. (They are triangles b/c of the demosaicing algo, pixel's values are somewhat interpolated from their neighbors, so one hot pixel has an effect on several pixels around it). You really should do dark frame subtraction for exposures that long.
Also, using dual iso here I would say is completely inappropriate. If your going to have the shutter open that long anyway and your subject is completely motionless, might as well just take another, shorter exposure to add to your DR, if there really is that much DR.
To fix this particular shot, try --really-bad-pix.
Unfortunately I don't have originals to try different processing parameters, but I'll do some testing on different shots, thanks a1ex!
Considering how many bad pixels arise on long exposures, it would be nice to have this parameter kick in automatically while processing. But since people have different preferences (and cameras) it may be best to implement such feature on Barracuda GUI or LR plugin as a user programmable switch.
dmilligan, I always thought that demosaicing algorithms are mostly square-based (eg 2x2). I was surprised to see triangles ;-) As for dual being inappropriate for this shot - agreed, I'm just testing different ML settings to do a usability study in order to know when some features might be helpful and when not. Regular ISO 100 shot would work well too.
Hey again guys, about exiftool, using it to even out the levels, i got the tool to read the levels of a reference image by:
exiftool IMG_1234.DNG -WhiteLevel -BlackLevel
White Level : 47752
Black Level : 2032
but im curious, about this:
exiftool *.DNG "-WhiteLevel<BlackLevel"
exiftool *.DNG "-WhiteLevel+=45720"
thats two lines of codes, also, how do i process the above code to a sequence file, is it done one by one? is there a way to do it for all files, as it done 1 line of code:
exiftool *.DNG "-WhiteLevel<BlackLevel"
said 1 file was processed
then the next line, same, 1 file processed
UPDATE:
i dont know why i didnt think of this sooner, i feel so silly now:
ok i finally got the solution, if youre limited to around 500 files (file path too long etc), make a folder with 1 character name in root of c drive, say 0, make each individual subfolders for everyclip extracted from your shoot, and within each folder, select all dng files, (DONT SELECT WAV), rename with one character say "1", windows will rename them in same order), this way it uses less characters, and now you will be able to process very long sequences
for running multiple instances, its best used when you have a shoot with many clips, so dont use the script, instead, run cr2hdr with same levels parameter, for every clip in the collection rather than for every frame, so this way you will still have many instances, utilizing the CPU, running for every clip rather than frame
So i believe i figured out how to run the exif tool same levels process, as the above method is too time consuming, HOWEVER BEFORE YOU READ ON, it seems to have made no exposure differences, and it still flickers, but the command that a1ex given me was executed and the files were reprocessed (I BELIEVE)
Heres the mini guide i wrote for myself, so you guys maybe can point out the mistakes:
so, go into windows explorer folder with the dual iso processed files with different levels that flicker, make sure exiftool.exe is in the same folder, whilst holding shift right click in the explorer window blank area > open command window here, right click at the top of the window > edit > paste:
exiftool M28-1423.000000.DNG -WhiteLevel -BlackLevel
M28-1423.000000.DNG is the variable, copy and paste the filename that has the most correct exposure of your sequence, then to insert the command in cmd, copy the command, right click on top of the cmd window > edit > paste, press enter, it will give you the white and black levels, take away the smaller number from the big one
now copy and paste this command into cmd
exiftool "-WhiteLevel+=27808" C:\0\8
the number 27808 is the first variable, which is the answer that you get when you take the small number from the large number, and the C:\0\8 is the location of the files that have different levels, processed dngs, press enter, and it will process all files making new files and renaming the original file extensions, once done you can delete the old renamed files.
seems like it didnt make any exposure changes at all, still flickering
You probably need to use something like this.
exiftool C:\0\8\*.DNG "-WhiteLevel+=27808"
trying your code now, compiled in after effects, rendering, and i get an error on frame 37, with an error:
After Effects error: Photoshop file format error (-30504).
( 45 :: 35 )
deleted the processed folder with your code, tried again, exact same result, gonna render 39 frame onwards
nope, seems all frames onward were corrupt, both times of processing, viewed all frames in adobe bridge, it was flickering every 2 frames or so, at extreme conditions, around 2 exposure steps id say, tried opening 39th frame on wards, photoshop cannot open, corrupt, with my previous code, no corruption, all files processed, however there was flickering still
Do anyone could help me why I'm getting a huge magenta cast and low contrast when opening the dng dual iso in Photomatix?... here are the files:
Original RAW: http://www50.zippyshare.com/v/22736945/file.html
Dual iso DNG: http://www49.zippyshare.com/v/63977107/file.html
Anyone knows if there is any way to process the dng in Photomatix without this problem?
Camera: 5D MK2
For dual iso I use the last version of cr2hdr-20bit.exe
The DNG opens fine here in Adobe.
https://dl.dropboxusercontent.com/u/34113196/ML/_MG_9565.jpg
Suggest you contact HDRsoft support: http://www.hdrsoft.com/support/index.html
Thanks Audionut, yes on photoshop works, just wandered to know why it didnt work in photomatix... I emailed them, so let see what they answer
Generally, color casts are from an incorrectly identified black level. This should be an easy fix for them. :)
This was mentioned some time ago but i believe it went without attracting much attention.
I was filming real estate video lately and used dualiso video for some shots inside the building to have more dynamic range for windows. During the post it is a real huge pain to find those dualiso files among the rest (and there are houndreds of files). I could of course watch every mlv in mlvplayer (i did that, very time consuming, had to write down the numbers) and than checked those files to proceed in raw video converter. Bigger problem is when files are processed - in resolve it is very hard to guess which is processed dual iso and which is normal (especially when you loose you notes from previous step:). I am still struggling with this.
Could it be possible to make custom file prefix work with mlvs?
Devs, please :) That would be a huuuge time saver.
With cinelog profile i could get rid of most of the color fringing so dualiso video is realy usable but with this this one major quirk.
Thanks
Quote from: kgv5 on May 17, 2014, 12:56:18 PM
This was mentioned some time ago but i believe it went without attracting much attention.
I was filming real estate video lately and used dualiso video for some shots inside the building to have more dynamic range for windows. During the post it is a real huge pain to find those dualiso files among the rest (and there are houndreds of files). I could of course watch every mlv in mlvplayer (i did that, very time consuming, had to write down the numbers) and than checked those files to proceed in raw video converter. Bigger problem is when files are processed - in resolve it is very hard to guess which is processed dual iso and which is normal (especially when you loose you notes from previous step:). I am still struggling with this.
Could it be possible to make custom file prefix work with mlvs?
Devs, please :) That would be a huuuge time saver.
With cinelog profile i could get rid of most of the color fringing so dualiso video is realy usable but with this this one major quirk.
Thanks
Dual Iso has different name, Have you use .MLV or .RAW?
I use MLVs, 5d3, 1.1.3. Checked newest version from 17 may, files are named like this: M17-2329.MLV and seems that there is no difference between normal and dual iso. Also checked "custom file prefix" to be sure but doesn't work for movies.
@budafilms - what file names do you get for dual iso movies?
I'm wondering if there's any speed vs. iq tradeoff in current cr2hdr? That's because at least on my laptop it's so slow anyway I'll let it process the files when I'm not around ... so if a "--ultra-slow-highest-quality" setting would be there I'd certainly use it :-)
I think we can count on a1ex making it maximum IQ out of the box :)
These should make it faster.
{ &interp_method, 1, "--mean23", "average the nearest 2 or 3 pixels of the same color from the Bayer grid (faster)
{ &chroma_smooth_method, 0, "--no-cs", "disable chroma smoothing" },
{ &fix_bad_pixels, 0, "--no-bad-pix", "disable bad pixel fixing (try it if you shoot stars)" },
{ &use_fullres, 0, "--no-fullres", "disable full-resolution blending" },
{ &use_alias_map, 0, "--no-alias-map", "disable alias map, used to fix aliasing in deep shadows" },
{ &use_stripe_fix, 0, "--no-stripe-fix", "disable horizontal stripe fix" },
@a1ex: German language support, please!
(http://s4.postimg.org/cywod7cwp/Bild_2.jpg) (http://postimg.org/image/cywod7cwp/)
Semi-overexposed = "Halb-überbelichtet"? Urks! "Scheinbare Überbelichtung"?
Deep shadow = "Tiefer Schatten"? ... naja ...
Dynamikbereich (src1), Dynamikbereich (src2) = "Quelle 1", "Quelle 2". No clue what's meant by that.
Other finetuning needed? Open to suggestions!
Ciao
Walter
More help needed. Pretty sure I messed up this one:
TTAmaze=use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
TTAmaze=Verwende einen De-Mosaik-Zwischenschritt (AMaZE), gefolgt von kantenbetonter Interpolation (Standardeinstellung)
Ciao
Walter
Finally I've got an answer from Photomatix for the issue of magenta cast, this is their email:
"appears that the issue comes from the fact that the DNG files produced by Magic Lantern are missing two DNG tags that Photomatix needs in order to be able to interpret the color information of the DNG data. The Magic Lantern people would need to make their firmware write these two tags before Photomatix Pro would be able to support those files.
These are the missing tags:
CFAPlaneColor tag 50710
CFALayout tag 50711
Once these two tags are added it may be that Photomatix Pro will be able to immediately handle the files, or we may need to take another look at that time to try and correct the reading once the information is available."
Could be possible to add that two tags!?
These tags are optional according to the DNG spec (http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec.pdf), so... yeah, better let the free software guys work around problems in nonfree programs.
Anyway, adding these tags is quite easy, you only have to edit chdk-dng, fill the missing tags, and try it. Since I don't use Photomatix, I'll let you do this part and submit a pull request if it works.
Quote from: a1ex on May 28, 2014, 07:35:27 AM
These tags are optional according to the DNG spec (http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec.pdf), so... yeah, better let the free software guys work around problems in nonfree programs.
Well, you cannot really call it a problem since they're free to depend on official, but optional tags if they require them for one reason or another and all camera out there excluding ml write them - which seems to be the case or they'd have added a workaround by now.
Still, it unfortunately shows that ml still seems to be considered a "hack", not widespread and not to be taken seriously enough to add consideration for ml features (dual_iso, mlv) into commercial, closed-source software :-\
Quote from: a1ex on May 28, 2014, 07:35:27 AM
Anyway, adding these tags is quite easy, you only have to edit chdk-dng, fill the missing tags
I would have thought adding static exif tags would best be done in cr2hdr esp. since as I just learned the silent dng don't contain much exif data anyway?
When reading an optional tag, the proper thing to do is to use its default value, as described in the DNG spec.
Do you disagree with this?!
Quote from: a1ex on May 28, 2014, 11:45:22 AM
When reading an optional tag, the proper thing to do is to use its default value, as described in the DNG spec. Do you disagree with this?!
Right, if it does have a default value (which I didn't know, if I'd read long lists I'd start with the exiftool help page :-p) then clearly the photomatrix guys are to blame - I'd suggest the user in question replies to their "support" e-mail as such, suggesting to add the default fallback to their program. In reality of course, adding these to cr2hdr might be the realistic solution :-\
Quote from: Marsu42 on May 28, 2014, 01:46:16 PM
Right, if it does have a default value (which I didn't know, if I'd read long lists I'd start with the exiftool help page :-p) then clearly the photomatrix guys are to blame - I'd suggest the user in question replies to their "support" e-mail as such, suggesting to add the default fallback to their program. In reality of course, adding these to cr2hdr might be the realistic solution :-\
Hello, so what do you suggest me to do!?... is this a mistake of Photomatix?
Quote<Dark_Shikari> you can never go wrong by abiding by the spec, since if someone breaks the spec you can just blame them instead
<pengvado> fair enough. I'll allow the patch on the premise of passing the blame.
Quote from: citytrader on May 28, 2014, 04:03:08 PM
Hello, so what do you suggest me to do!?... is this a mistake of Photomatix?
Tell photomatrix to abide by the spec. You'll probably have more luck fixing the problem yourself, see here (https://bitbucket.org/hudson/magic-lantern/commits/0f20e1936e12cff0cd623569753ffb76e707e5f3), and here (https://bitbucket.org/hudson/magic-lantern/commits/2add8f75d844b3f7dcec6a2274cf1f3f8704d3f3) for hints.
This is a handy reference (http://www.magiclantern.fm/forum/index.php?topic=7940.0).
Quote from: citytrader on May 28, 2014, 04:03:08 PM
Hello, so what do you suggest me to do!?... is this a mistake of Photomatix?
I wouldn't mention the word "mistake" :-) but since you're in customer e-mail support contact with them anyway, there's no harm telling them that they're not observing the dng specification as there are default values for the tags in question when they don't exist in the file. Falling back to the default value would be the proper thing to do and easy to accomplish as well.
Quote from: citytrader on May 28, 2014, 04:03:08 PMHello, so what do you suggest me to do!?
Download Adobe DNGconverter 8.4, convert your ML-DNGs to whatever-DNGs and give them a try in Photomatix.
Prey, is there any drawback if converting the dual_iso unprocessed cr2 to dng (non-lossy of course) and *then* put them through cr2hdr?
Storing images as dng instead of cr2 always saves a few mb (mostly because the jpeg preview is dropped), and this can add up to a lot of saved space looking at my growing cr2 dual_iso archive. Alternatively, I'm still looking for a tool that erases the thumbnail from a cr2 and leaves the rest untouched :-p
AFAIK, there is no tool to remove thumbnails. Probably because so many things rely on the embedded thumbnail.
cr2hdr relies on the actual data, it doesn't care about exif or the file extension. As long as it's CR2/DNG :P
edit: Actually, a1ex likes robust solutions so much, that cr2hdr doesn't care about the extension at all. ;)
Quote from: Audionut on May 29, 2014, 05:55:50 PM
Probably because so many things rely on the embedded thumbnail.
They do? Other than in-camera preview I never found anything that uses it, it's just a waist of storage space. The one reason to keep it around is because you can extract it from the cr2 if the raw data is broken for some reason, but that's hardly any good with dual_iso shots.
Thanks for the answer concerning cr2/dng though, in this case I'll convert my cr2 to dng as any other cr2-processing programs like dxo cannot handle dual_iso files anyway.
Edit: fyi: the dng files are nearly 20% smaller than cr2.
Quote from: Marsu42 on May 29, 2014, 05:59:14 PM
They do? Other than in-camera preview I never found anything that uses it
Probably because most things don't advertise the fact that they do. ;) It's significantly faster to display the embedded preview, for a preview, then to decode the entire data stream.
Thanks, re: space. Personally, I prefer to stick to the original format and just upgrade HDDs, since they're cheap enough. I should probably convert all the dual ISO CR2s though. :)
Quote from: Marsu42 on May 29, 2014, 05:44:00 PM
Prey, is there any drawback if converting the dual_iso unprocessed cr2 to dng (non-lossy of course) and *then* put them through cr2hdr?
I actually do that. For archiving, I delete the dual ISO output files and keep the unprocessed dng's.
On Windows, be careful because they have the same extension, so the unprocessed one will probably get overwritten. On Linux, Adobe DNG converter outputs lowercase dng's, and cr2hdr outputs uppercase DNG's => no conflicts.
To solve this on Windows, one way would be to rename the unprocessed .dng back to .cr2, since dcraw doesn't care about the extension. Ugly, but gets the job done.
Quote from: a1ex on May 29, 2014, 06:12:58 PM
On Windows, be careful because they have the same extension, so the unprocessed one will probably get overwritten. [...]
To solve this on Windows, one way would be to rename the unprocessed .dng back to .cr2, since dcraw doesn't care about the extension. Ugly, but gets the job done.
Yes, I already noticed that, quite a catch since it destroys the original data w/o even a warning. Couldn't you just add an auto-rename or output path option to cr2hdr to solve this once and for all for everyone (yes-i-could-submit-a-patch)?
I'm getting lower case output here from Adobe, on a windows (8.1) machine.
Quote from: Marsu42 on May 29, 2014, 06:22:52 PM
Yes, I already noticed that, quite a catch since it destroys the original data w/o even a warning. Couldn't you just add an auto-rename or output path option to cr2hdr to solve this once and for all for everyone (yes-i-could-submit-a-patch)?
There is an option to not overwrite existing data: https://bitbucket.org/hudson/magic-lantern/commits/7fad96785821644a05dfb9d4212791e6e1bc7706
Quote from: Audionut on May 29, 2014, 06:23:21 PM
There is an option to not overwrite existing data: https://bitbucket.org/hudson/magic-lantern/commits/7fad96785821644a05dfb9d4212791e6e1bc7706
But even if you have to opt-in so you have to be already aware of the problem, and then it's no good because it would refuse converting the dual_iso original dng... the .cr2 rename method is imho rather hackish. I'd rather rename them to .mld or something :-p
Quote from: Audionut on May 29, 2014, 06:23:21 PM
I'm getting lower case output here from Adobe, on a windows (8.1) machine.
In theory, you can configure any windows ntfs volume to be case-sensitive, I did this because it helps mingw before I finally moved to cygwin. But very few users do that, usually windows is case-insensitive and .dng overwrites .DNG and vice versa.
Indeed it does. Adobe will prefix and not overwrite, but the output from cr2hdr will overwrite lowercase dng's.
Quote from: Marsu42 on May 29, 2014, 06:26:22 PM
But even if you have to opt-in so you have to be already aware of the problem, and then it's no good because it would refuse converting the dual_iso original dng... the .cr2 rename method is imho rather hackish. I'd rather rename them to .mld or something :-p
I understand and can see the cause for concern.
Personally, I would segregate with folder management. Originals in one folder, processed in another. BarracudaGUI (http://www.magiclantern.fm/forum/index.php?topic=9102.0) is great for this. But I'm used to being pedantic with file management. :P
Quote from: Audionut on May 29, 2014, 06:29:14 PMPersonally, I would segregate with folder management. Originals in one folder, processed in another.
I'm doing that right now with cr2 dual_iso originals on a backup hd and converted dng on a live hd right now. The nice thing about this solution that I just need to back up the non-dual_iso files from the live hd and am not duplicating data.
But folder management doesn't work if cr2hdr insists on overwriting the source .dng and you cannot rename the target or set an output folder :-\ ... cr2hdr simply needs an output path option. Or am I missing something here?
I think you might be alone on that one. All of the GUI's provide that option, which fits in with this philosophy.
My bold for emphasis.
Quote from: a1ex on May 15, 2014, 09:44:11 PM
Instead, I'm trying to build a community that does not just consume whatever we give to them, but I want this community to actually participate in the development process, help each other, and share the knowledge. We gave you some free software, we gave you a proof of concept that you found useful, and now we expect you all to take this software at the next level, and let us build upon your work, in the same way as you have built upon ours. I'm quite far from this utopian goal though, but this is the direction I want ML to go.
for people that have more than 500 files and cannot use same levels, i'd recommend using exiftoolGUI, way easier and fast.
Hi,
I have beem using Color filters for B&W photography, especificaly the 091 red filter, with CPL. Well, I receive very strips out of the cr2hdr-20bit, like the image bellow (--no-cs --full-res).
I put some files here (https://app.box.com/s/o6133rwcy8s5ry8kal2u). I think it will be very difficult to fix it, because the cr2hdr need to colors to interpret the photo, or not? Any ideia how to fix it?
Example frame here (http://s1.postimg.org/mpp73hn3z/MG_2267_dualiso_dualiso.jpg).
Try using the 20bit build from a couple of pages back.
I'm not seeing the problems you describe.
https://dl.dropboxusercontent.com/u/34113196/ML/forum_stuff/_MG_2268.jpg
Note that by using such a strong filter, the green and blue channels receive very little information.
I've tested the 20-bit version with a few infrared files, but these two are even stronger.
The result is a little noisier than I've expected. In channel mixer, I found this ratio to be close to the sweet spot, noise-wise: R=1, G=0.6, B=0.1 (at auto WB 1, 6.5, 20).
What crop window did you use for the test sample? I can't identify it in the CR2's...
edit: after playing a bit with your sample, here are some tips:
- your files have a very high SNR in the red channel, but quite poor in green and blue channels; unlike regular pictures, where the green channel is cleaner
- therefore, the first tip would be to disable chroma smoothing (--no-cs)
- related: internally I use a temporary demosaic to get a better interpolation in half-res areas; however, this demosaicing will take some noise from the green/blue channels and put it into the red channel, so let's disable this step (--mean23, which averages only pixels of the same color)
- in your flower sample, you have out-of-focus shadows, so we don't need to recover full-res detail there (--no-alias-map)
- in your flower sample, the resolution is not critical either, so let's also try --no-fullres
- in some cases, stripe correction may be less than ideal, so let's try disabling it (--no-stripe-fix)
Results (click for full-res):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/_MG_2269-small.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/_MG_2269-2200-3200.jpg)
Script used to render these crops (you may find useful for trying out things):
compare-options.py (http://a1ex.magiclantern.fm/bleeding-edge/isoless/compare-options.py)
Kalman analysis (optimal averaging formula, see http://robocup.mi.fu-berlin.de/buch/kalman.pdf ):
- let's assume constant additive noise, in arbitrary units normalized to get stdev=1 for the red channel
- green channel will have stdev=6.5 per pixel (from auto WB)
- blue channel will have stdev=20 (from auto WB)
- now, let's mix one red, two greens and one blue (as with dcraw -h) => green channel has stdev=6.5/sqrt(2) = 4.6 per RGGB cell
- variances: 1, 21, 400
- optimal blending factors (after auto wb, without color matrix): 1, 0.048, 0.0025.
Now, look at tip #2 from above: debayering will usually do some kind of channel mixing (depending of the algorithm), which will introduce noise. Let's minimize this. Under the additive additive noise hypothesis (same noise stdev for all pixels), this means the noise for each pixel should be mixed in equal proportions, so, let's do the debayer at UniWB (multipliers 1,1,1). This is not exactly a good idea for color images, but should be close to optimal noise-wise. In this case, the optimal blending factors would be 1, 0.3, 0.05, since we no longer have WB applied.
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/_MG_2269-small.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/_MG_2269-2200-3200-uniwb.jpg)
Proof that the difference is caused by debayering: develop with --shrink=2 (half-res without debayer), both WB settings, and the noise difference will vanish (auto-wb (http://a1ex.magiclantern.fm/bleeding-edge/isoless/_MG_2269-2200-3200-halfres.jpg) vs uniwb (http://a1ex.magiclantern.fm/bleeding-edge/isoless/_MG_2269-2200-3200-halfres-uniwb.jpg)).
Exercise to the reader: show (mathematically) that if you use AHD debayering performed at WB 1,6.5,20, and color matrix applied, the optimal blending factors for the channel mixers are close to 1,0.6,0.1 (my initial guess).
Would it be safe to use Dual ISO mode for a timelapses? taking maybe 300-500 shots at a time? Will this affect my sensors at all?
Daniel
Quote from: dpt90 on May 31, 2014, 01:19:50 PM
Would it be safe to use Dual ISO mode for a timelapses? taking maybe 300-500 shots at a time? Will this affect my sensors at all?
Daniel
No bad sensor exposure, same shutter count as a regular timelapse. So this improves on HDR timelapses in the shutter count area, because instead of 2+ (usually 3 or more) per frame for an HDR, you get increased DR with one shot. Just a heads up, as with HDR timelapse, your post processing time increases dramatically.
Quote from: dsManning on May 31, 2014, 05:14:30 PM
No bad sensor exposure, same shutter count as a regular timelapse. So this improves on HDR timelapses in the shutter count area, because instead of 2+ (usually 3 or more) per frame for an HDR, you get increased DR with one shot. Just a heads up, as with HDR timelapse, your post processing time increases dramatically.
I know the drawbacks/benefits of doing HDR in post lol.... What I mean is, will shooting in dual ISO mode put more strain on my cameras sensor at all? I've just hear that Dual ISO mode can put a lot of stress into the sensor causing problems with it. Or should I not even worry about it?
Quote from: dpt90 on May 31, 2014, 06:06:49 PMI've just hear that Dual ISO mode can put a lot of stress into the sensor causing problems with it.
May I ask, where you heard it and who told the tale? Link, please!
Ciao
Walter
Alex mentions it on the original post, page 1. I admit I got scared too, as I don't exactly understand what it's doing.
He wrote about possible hazards to the sensor, esp. frying it.
He doesn't say anything about additional stress during Dual ISO operation under standard conditions per se.
Side effects noticed by me: there seem to be more hot pixels than usual during long exposures, and the ISO alternation confuses the feedback loop (http://www.magiclantern.fm/forum/index.php?topic=10111.msg104546#msg104546) a bit.
About temperature: anybody can run an experiment and record it during a timelapse, for example, with and without dual iso; if there are differences, such a test should reveal them. I didn't run any temperature comparison, so I can't tell.
I took around 20000 pictures with dual iso on 5D3 by now (including some timelapse that I have yet to postprocess), so I guess it should be fine. Of course, that's not a guarantee.
Quote from: Walter Schulz on May 31, 2014, 06:12:10 PM
May I ask, where you heard it and who told the tale? Link, please!
Ciao
Walter
Mostly been hearsay by people who havent used ML before and also mentioned in the original post!
this is just my first time trying the nightly builds!!! didnt want to have a bad experience lol....
Quote from: a1ex on May 31, 2014, 10:56:37 PM
Side effects noticed by me: there seem to be more hot pixels than usual during long exposures, and the ISO alternation confuses the feedback loop (http://www.magiclantern.fm/forum/index.php?topic=10111.msg104546#msg104546) a bit.
About temperature: anybody can run an experiment and record it during a timelapse, for example, with and without dual iso; if there are differences, such a test should reveal them. I didn't run any temperature comparison, so I can't tell.
I took around 20000 pictures with dual iso on 5D3 by now (including some timelapse that I have yet to postprocess), so I guess it should be fine. Of course, that's not a guarantee.
This is good enough confirmation for me about it being safe to use. :)
Those noise differences from debayering at different WB are quite remarkable. Using the optimal blending, I prefer --csx2, as it retains good detail, and doesn't suffer those black spots.
One thing I am not certain on.
Quote from: a1ex on June 01, 2014, 08:18:09 AM
The color resolution in a debayered image is usually lower, but most algorithms are able to reconstruct full-resolution luma by exploiting inter-channel correlation.
It doesn't make sense why the luma resolution has to be reconstructed (at all). Are you talking about the efficiency differences in the color filter array?
It seems to me, that we should have full luma resolution, with the problems being the color resolution (since we only have 1/3 resolution of RB, and 1/2 resolution of G), and the noise problems associated with WB'ing that color information.
I tried playing with filters about 12 months ago, for the express purpose of reducing the sensitivity of the G channel (ETTR'ing RB :) ), but since it was some cheap POS filter from China, the results were not spectacular. I don't even seem to have sample images available anymore, but I do still have the filter. Might have to break it out and re-check some things.
Quote from: Audionut on June 01, 2014, 04:00:37 PM
It doesn't make sense why the luma resolution has to be reconstructed (at all)
With a naive demosaic algorithm that only interpolates from pixels of the same color, you don't get full luma resolution. You would get a luma resolution similar to upsampling some image by sqrt(2).
To interpolate missing green pixels, by using information from red and blue (since you don't have other hints), you need to make assumptions about how your captured data behaves locally (in the neighborhood used for interpolation). A common assumption is that chroma doesn't vary much locally, therefore we can smooth it out without much perceived loss in quality, and use the local variations in red/blue channels to reconstruct luma data.
Quote
In the second step the colors are updated by transferring information from the green to the red and blue channels. [...] It subsumes the observation, underlying most algorithms, that the high frequencies of the three channels are very similar.
Some stuff worth reading:
http://www4.comp.polyu.edu.hk/~cslzhang/paper/conf/demosaicing_survey.pdf
http://www.arl.army.mil/arlreports/2010/ARL-TR-5061.pdf
http://www.ipol.im/pub/art/2011/bcms-ssdd/
So, I call it "reconstructed" because it's not there from the beginning, but it's guessed (and most of the time it's guessed very well, unlike blind upsampling).
A good algorithm for debayering could probably also solve the chromatic aliasing because of the line-skipping. Maybe using some heuristic methods for recognising "unlikely" pixel-values. Combining this with some "superresolution" methods (using several frames as a1ex has linked to before) and the result could probably get very good.
Quote from: a1ex on June 01, 2014, 05:32:35 PM
With a naive demosaic algorithm that only interpolates from pixels of the same color, you don't get full luma resolution. You would get a luma resolution similar to upsampling some image by sqrt(2).
To interpolate missing green pixels, by using information from red and blue (since you don't have other hints), you need to make assumptions about how your captured data behaves locally (in the neighborhood used for interpolation). A common assumption is that chroma doesn't vary much locally, therefore we can smooth it out without much perceived loss in quality, and use the local variations in red/blue channels to reconstruct luma data.
Why not process it like a video signal? Brightness match the signal, based from the known properties of the CFA. ie Green pixels have a unit of 1, blue has a unit of 0.8, red has a unit of 0.7 (for instance).
This way we have full resolution luma detail, and we know that chroma isn't so important (perception), so we can interpolate that. It seems to easy, so clearly I have some reading to do. ;)
If you shoot a monochrome image, yes, that will work.
So why not a 2 stage process?
After the full luma resolution has been obtained, go back and process the chroma.
edit: Wait, I think I am missing the importance of "monochrome".
Quote from: a1ex on May 31, 2014, 10:32:48 AM
I found this ratio to be close to the sweet spot, noise-wise: R=1, G=0.6, B=0.1 (at auto WB 1, 6.5, 20).
Trying to find it for days... thanks.
Quote from: a1ex on May 31, 2014, 10:32:48 AM
What crop window did you use for the test sample?
The crop is 1:1, but this sample is from another image (the strips are the same of others). If you want to play with other files, I put others here (https://app.box.com/s/hrvltc0z4qzj9fu0zld0) (note: is just aleatory shots, test purpose).
Quote from: a1ex on May 31, 2014, 10:32:48 AM
Script used to render these crops (you may find useful for trying out things):
compare-options.py (http://a1ex.magiclantern.fm/bleeding-edge/isoless/compare-options.py)
Will be very userfull for some tests.
I don't know if it make sense, but I get best results with IGV algorithm on B&W photos [?].
Thanks for this huge explanation
A1ex, I obtain a good result now. And,
@Audionut, thanks too.
Quote from: Audionut on June 01, 2014, 04:00:37 PM
I tried playing with filters about 12 months ago, for the express purpose of reducing the sensitivity of the G channel (ETTR'ing RB :) ), but since it was some cheap POS filter from China, the results were not spectacular. I don't even seem to have sample images available anymore, but I do still have the filter. Might have to break it out and re-check some things.
So, about the color filters, Is not a good deal using it on digital? I buy this to use with film, like HP5+ Ilford, but I think: Why not use it on digital?
Ansel Adams was a big user of this filter (091 dark red), and he said it amplify (on analog) some scale of gray.
Of course, on Digital it not work in the same way, but I really like of cut 600nm and below of spectra.
For maximum information, the 'correct' way is not using a color filter, or using the violet (about the discution of ETTR RB, little absorption of gree channel)?
Can you share the final result and the settings? Just curious.
IGV looks interesting. I'm not yet convinced to switch to RawTherapee though (I simply couldn't get the same look as from ufraw+enfuse, which fits my taste). I might have better luck with Darktable though (they started from the ufraw codebase, and still use a few key routines from there), but had some trouble installing it.
For me Darktable, after trying all possibilities in Linux, has become THE tool. Worth trying, I think. Really useful, comfortable and full of powerful tools... I clearly prefer it to RawTherapee, personally.
Hey guys, i got a problem, last few shoots were perfect, camera raw, no crop mode, with dual iso, processed with cr2hdr20 fine, no flickering, all perfect, now i had a 3 day shoot, and tried to process dual isos, got this error:
bright dark detection error, iso blending didnt work
has anyone had this problem before? what do i do
i also tried to process with the old cr2hdr, didnt have any problems with the above, but it did flicker which is unacceptable for this production
im uploading these dng files for you to diagnose if you can please, really confused why as some files processes fine, some dont, just with this cr2hdr20, but i need this one as this one doesnt cause flickering (with same levels parameter)
heres the files:
https://db.tt/VYeGuQoU
so just to recap:
cr2hdr.exe processes all files fine but flickers
cr2hdr-20bit.exe doesnt process many files and gives the error code: "bright dark detection error, iso blending didnt work"
the shortcut file is what you drag the dng files onto to use same levels parameter
M23-1529.000000 WONT PROCESS WITH 20bit
M23-1529.000001 WONT PROCESS WITH 20bit
M23-1529.000002 WILL PROCESS WITH 20bit
M23-1529.000003 WILL PROCESS WITH 20bit
Concerning greymax: It's known the algorithm completely misses if the shot is underexposed. Now, some very smart people would say "just don't underexpose!" but for simple /me and probably others who expose for highlight safety: Probably there is a way to adjust the awb detection for these cases? If required, I can provide some dual_iso samples.
Quote from: Marsu42 on June 03, 2014, 04:45:34 PM
It's known ...
By whom and by what method is it known? (http://en.wikipedia.org/wiki/Weasel_word)
I didn't notice major issues in my samples (https://www.dropbox.com/sh/uz75fw290v4n7v7/ASbJRAUXk-), except for those requiring more than +10 EV of correction, like this one (https://www.dropbox.com/sh/uz75fw290v4n7v7/AAAHYBRNxO7WNmt0j-9Sds7Aa/000174-graymax.jpg) (and I hope you don't underexpose like that). However, this algorithm looks for a large area of gray color, so it sometimes gets it completely wrong (but not because of exposure).
The graymed algorithm is a bit closer to the traditional gray-world hypothesis; maybe it's worth trying that too.
Quote from: a1ex on June 03, 2014, 05:08:32 PM
By whom and by what method is it known? (http://en.wikipedia.org/wiki/Weasel_word)
Sorry for not providing an exact quote, I remembered you saying as much after introducing the dual_iso awb so I thought it's not necessary to provide exact proof. For my shots, greymax starts failing with ~2ev underexposure and results in tint off the chart, usually magenta.
Quote from: a1ex on June 03, 2014, 05:08:32 PMThe graymed algorithm is a bit closer to the traditional gray-world hypothesis; maybe it's worth trying that too.
Ok, I'll try that for a change. Probably the reason for the failure really is that it fails to find color areas which might not be surprising in nature shots.
So, if you set the camera on a tripod and shoot the same test scene at different exposure settings, you get different auto WB results?
I'd call this one a bug.
Quote from: a1ex on June 03, 2014, 05:24:25 PMSo, if you set the camera on a tripod and shoot the same test scene at different exposure settings, you get different auto WB results?
I didn't do systematic tests, just noticed borked greymax awb with undexposed shots. If you didn't look into that much yourself I'll try to find the time and do a proper test series with greymax and greymed.
A1ex
bright dark detection error, iso blending didnt work
do you have any advice on that, i saw loads of pages back that you fixed it, but for your new version of cr2hdr 20 the error comes up
Not yet, but confirmed the bug. Seems to be caused by extreme overexposure in the high ISO (that is, very little overlap to match them properly).
Quote from: Marsu42 on June 03, 2014, 05:26:34 PM
I didn't do systematic tests, just noticed borked greymax awb with undexposed shots. If you didn't look into that much yourself I'll try to find the time and do a proper test series with greymax and greymed.
I ran a test series with the dual_iso samples I uploaded earlier (https://bitbucket.org/Marsu42/ml-pull/downloads/dual_iso_samples_6d_60d_FIXED.zip) and graymax works ok with underexposure on these. Obviously it's really the algorithm failing on non-studio shots when a plain color area cannot be found. Knowing this, I'll use greymed in the future on my shots...
... but as an idea, maybe an smart auto-detection greymax/greymed would be possible if the tint is off the scale with greymax.
Quote from: a1ex on June 03, 2014, 07:06:49 PM
Not yet, but confirmed the bug. Seems to be caused by extreme overexposure in the high ISO (that is, very little overlap to match them properly).
ok thank you, may i ask if a fix is to be expected? i made the mistake on shooting a production in dual iso, even thought i tested it for a week still a new problem came about, and now i cannot continue it until this works, anyway i can help by any chance lol? researching seems to get me nowhere thought as im not a hacker or coder and cannot find any info on this either
but the thing that confuses me, shots, 0 and 1, compared to 2 and 3, are very similar, yet 0 and 1 dont process but 2 and 3 do
Quote from: a1ex on June 03, 2014, 05:08:32 PM
I didn't notice major issues in my samples (https://www.dropbox.com/sh/uz75fw290v4n7v7/ASbJRAUXk-),
Another sample for your list. --wb=graymed gets closer, but still a few thousand K wrong. Somewhere around 7000-7500K is about right IMO.
https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/_46A5433.CR2
Hey guys, hope i aint being a nuisance, but any update on the "bright dark detection error, iso blending didnt work" bug?
im really desperate for a fix
or would it be possible if the developers let us know if a fix is to be expected anytime soon, because if not we can move on from waiting and continue with an alternate method
update and notes and tutorial for the un educated people on the following problems:
"bright dark detection error, iso blending didnt work"
flickering when filmed with dual iso
using the older version:
2014.04.08\cr2hdr-20bit.exe, doesnt give the "bright dark detection error, iso blending didnt work" error
whilst
2014.05.06\cr2hdr-20bit.exe, DOES
so apply same levels in post yourself, as cr2hdr same levels parameter uses exiftool to apply same levels anyway
processes the 0 and 1 files, and to apply same levels i used, ExifToolGUI, run the program, navigate to first file of the sequence, on the right find BlackLevel and WhiteLevel, note down the values, select all the files (i found that if i select around 2000 files it doesnt process the command so theres a file ammount limitation, 1430 worked, so maybe 2k is the limit?) at top right of the gui program, click on exiftool direct, copy and paste:
-Exif:BlackLevel=2046
NOTE, the 2046 is the variable, enter the BlackLevel you noted down from first file, click on ok, itll process the files, and create a backup of the original files, if you dont want them backed up, go to options at top left > dont backup files when writing.
now repeat the procedure for the whitelevel:
-Exif:WhiteLevel=2046
again the 2046 is the variable, click on ok, once both the whitelevel and black level have been changed to match the first file of the sequence, there should be no flickering what so ever, i dont believe ACR caused the flickering in the first place, as when i applied ACR adjustments, even to extreme cases, like -100 highlights, +100 shadows, etc, no flickering was caused what so ever
Give Alex a break I bet he is already on that (judging from previous DUAL_ISO bugs).
In the meantime may I suggest to shoot alternating dual_iso and normal shots so at least to have a working raw file?
Quote from: tron on June 07, 2014, 02:31:40 AM
Give Alex a break I bet he is already on that (judging from previous DUAL_ISO bugs).
In the meantime may I suggest to shoot alternating dual_iso and normal shots so at least to have a working raw file?
+1
If you are unsure then do that. That is how I started shooting dual ISO. However, now in just shoot in dual ISO mode - no alternate files.
Hi,
I cannot convert files in DNG format. They are remaining as if they are not converted.
Quote from: djronbxs on June 08, 2014, 09:24:56 AM
Hi,
I cannot convert files in DNG format. They are remaining as if they are not converted.
Walk us through the steps. Make sure you list out each step as a bullet point and I will try to help.
However, before you do that, have you looked in the folder where the dualISO CR2 file was? If you exported right, the .DNG should be there. Go into windows explorer, navigate to that folder and look.
what I meant is that if I convert the file to dng with another software then i try to convert it from dual iso the photo is not being processed
Looks like a cr2hdr problem. @Mod: Please transfer offtopic to http://www.magiclantern.fm/forum/index.php?topic=7139.2150
We're no wizards here or paranormals. Give information to reproduce your problem.
- Program used to generate DNGs
- cr2hdr version used
- A link to a DNG having this problem
- Have you tried to run this file through cr2hdr?
And why do you have to create DNGs through this program?
Ciao
Walter
Quote from: djronbxs on June 09, 2014, 09:26:17 PM
what I meant is that if I convert the file to dng with another software then i try to convert it from dual iso the photo is not being processed
Why do you need to convert your dual ISO cr2 to dng using another program?
Hi, I'm new to this forum, so please forgive me if this is not posted in the correct thread.
I just installed ML on my 7D, mainly to take advantage of the dual ISO feature. Under relatively normal lighting conditions, it works fantastic; however last night, I was taking longer exposures on a tripod and noticed that all of my pictures taken with dual ISO have hundreds of random black marks scattered throughout the frame. I thought this was related to the sensor overheating, so I ran some benchmarks with and without dual ISO and with and without Long Exposure Noise Reduction. I figured the noise reduction would do the trick, but I was surprised to see that every photo taken in Dual ISO, regardless of whether or not noise reduction was activated, had these spots, all in the same places. Exposures without Dual ISO turned on were fine at both the low and the high ISO, with and without LENR.
Is dual ISO just overwhelming my sensor? Has anybody else had this issue?
Very interested to hear your thoughts. Thanks.
From the previous page:
Quote from: a1ex on May 31, 2014, 10:56:37 PM
Side effects noticed by me: there seem to be more hot pixels than usual during long exposures, and the ISO alternation confuses the feedback loop (http://www.magiclantern.fm/forum/index.php?topic=10111.msg104546#msg104546) a bit.
About temperature: anybody can run an experiment and record it during a timelapse, for example, with and without dual iso; if there are differences, such a test should reveal them. I didn't run any temperature comparison, so I can't tell.
Hope this is the right place to post this :)
I've been playing with dual-iso (5D3-123 branch) and noticed that the white balance Temp & Tint values in Lightroom (5.3) on the converted DNG file from cr2hdr did not match the original CR2 values.
After a bit of digging through the code, and playing with values I've found the following:
- the AsShotNeutral calculated using the RawMeasuredRGGB tag appears wrong, this tag gets used if the camera is set to Auto WB
- using the RedBalance & BlueBalance tags from the CR2 give the correct AsShotNeutral (as compared to a DNG converted using the Adobe DNG Convertor)
- using the WB_RGGBLevelsMeasured tag also gives the correct AsShotNeutral
The comments in exiftool-bridge.c state the RedBalance and BlueBalance values are not trustworthy - does anyone know why? They seem to work ok on my 5D3.
After fixing the AsShotNeutral value, the Temp and Tint were now closer to the original CR2 values in Lightroom; but still not exactly the same.
Looking more closely at the cr2hdr generated DNG file I noticed that only one color matrix is being saved - time to get the latest DNG code from CHDK ;)
After adding the second color matrix array, both calibration arrays, both forward matrix arrays and setting the correct value for illuminant1, I'm now getting consistent WB Temp and Tint values in Lightroom.
So far, the code changes to do this are in:
- src/raw.h (updated raw_info struct)
- src/chdk-dng.c (additional DNG tags)
- modules/dual-iso/cr2hdr.c (init raw_info)
- modules/dual-iso/dcraw-bridge.c (populate raw_info)
- modules/dual-iso/exiftool-bridge.c (get white balance)
Since there are changes to the core files I'm not sure what other impacts this may have - need to do some more testing.
Lastly, what's the preferred place to post patches for ML?
Phil.
Quote from: philmoz on June 13, 2014, 01:27:46 PM
Hope this is the right place to post this :)
For relevant discussion and testing, go to http://www.magiclantern.fm/forum/index.php?topic=10265.0.
Quote from: philmoz on June 13, 2014, 01:27:46 PM
...
The comments in exiftool-bridge.c state the RedBalance and BlueBalance values are not trustworthy - does anyone know why? They seem to work ok on my 5D3.
...
I wrote the metadata-based white-balance estimation. It's surprising to me that WB_RGGBLevelsMeasured is giving you a reasonable white balance; that did not appear to be true in general with the test shots (see the thread). I don't understand how changing the matrices managed to make your white balance consistent, so I'm curious about your changes.
a1ex coded a raw-data-based white-balance approach (http://www.magiclantern.fm/forum/index.php?topic=10265.msg111203#msg111203), but it's only in the
cr2hdr-20bit branch on Bitbucket, not in the
unified branch.
Quote from: philmoz on June 13, 2014, 01:27:46 PM
Lastly, what's the preferred place to post patches for ML?
Submit a pull request to the repository on Bitbucket: https://bitbucket.org/hudson/magic-lantern
Hehe, ayshih was faster :)
Indeed, you will want to try the cr2hdr-20bit branch, which includes some workarounds for this problem (both the exif approach from ayshih, and the custom WB algorithm from me).
To post patches, you can follow this tutorial: http://www.magiclantern.fm/forum/index.php?topic=7940.0
(in this case, you will probably want to submit your changes to the cr2hdr-20bit branch)
The problem with EXIF WB is that Canon's auto WB algorithm gets confused by dual iso. It's possible that some channels may be sampled from bright lines, and others from dark lines, and the exact mix may be camera-specific. However, if you can copy the manual WB from the CR2 to the DNG (which does not use Canon's auto WB algorithm), that would be a big improvement.
Modifying the raw_info structure will break the .RAW file format, which is not exactly desirable (there are still people who prefer it instead of MLV).
Hello
My coworker wants to 7D also installed Magic Lantern, but something we fail to install
I have a 550D and I have no problem with me
Of the instrument it is version 2.5 Firmware
Everything is done in the footsteps of the site
thank you
Sorry for the spelling I am not very good with English
Quote from: ayshih on June 13, 2014, 02:25:27 PM
For relevant discussion and testing, go to http://www.magiclantern.fm/forum/index.php?topic=10265.0.
Many thanks, will continue the discussion there.
Ignore most of what I posted previously - seems I was chasing the wrong problem :)
Phil.
Quote from: FOTOTO64 on June 13, 2014, 03:06:08 PM
Hello
My coworker wants to 7D also installed Magic Lantern, but something we fail to install
I have a 550D and I have no problem with me
Of the instrument it is version 2.5 Firmware
Everything is done in the footsteps of the site
thank you
Sorry for the spelling I am not very good with English
A 7D with firmware 2.0.5 will not install ML.
Get the 7D back to firmware 2.0.3 and try again.
Canon Amateur
thank you very much
;)
Hey if anyone wants to see a video i done with Camera Raw and Dual ISO
https://www.youtube.com/watch?v=qyicaOL9tl4
Quote from: djronbxs on June 09, 2014, 09:26:17 PM
what I meant is that if I convert the file to dng with another software then i try to convert it from dual iso the photo is not being processed
Why would you want to do that? The plugin does both the steps for you...
do you know guys if is there some software that cha transform the dual iso .raw video files directly in a cinema dng ora in a prores file?
Mlvmystic 0.5
I've just tried a timelapse with AETTR and Dual ISO, I have a question about the workflow.
After importing all my files, do I want to process all the dual ISO into DNGs before editing the image or do I edit the cr2 file, then process the dual ISO into DNGs? I'm assuming the first option but just wanted to make sure.
Any other workflowthat I may need to consider for AETTR Dual ISO timelapse vs AETTR timelapse? I'm using the LR Cr2DNG plugin from another thread and adobe bridge deflicker script if the ML deflicker XMP files aren't enough.
Quote from: LB7D on June 10, 2014, 05:30:21 AM
Hi, I'm new to this forum, so please forgive me if this is not posted in the correct thread.
I just installed ML on my 7D, mainly to take advantage of the dual ISO feature. Under relatively normal lighting conditions, it works fantastic; however last night, I was taking longer exposures on a tripod and noticed that all of my pictures taken with dual ISO have hundreds of random black marks scattered throughout the frame. I thought this was related to the sensor overheating, so I ran some benchmarks with and without dual ISO and with and without Long Exposure Noise Reduction. I figured the noise reduction would do the trick, but I was surprised to see that every photo taken in Dual ISO, regardless of whether or not noise reduction was activated, had these spots, all in the same places. Exposures without Dual ISO turned on were fine at both the low and the high ISO, with and without LENR.
Is dual ISO just overwhelming my sensor? Has anybody else had this issue?
Very interested to hear your thoughts. Thanks.
long exposure - black dots
prob hot pixels fix in camera.
I've been following ML for over a year now and find it invaluable, dual ISO included. I have a little workflow tweak suggestion that would help me a great deal, assuming that I've not missed the idea already being mentioned or that the hooks etc don't exist.
Where I live we get 320 days of sun a year, precisely the sort of weather where a large dynamic range is helpful, so much so that I often leave dual-iso on all day. However, I then forget to switch it off as the evening arrives and my ISO hits 1600+, until I see the results on the screen. Control via auto-ettr isn't really an option as I find it a bit slow at times. What I'd like to see is a switch that when you have dual-iso enabled, will auto-disable it above certain ISOs. For me that would be around 1600 and over, the point at which it starts to cause problems (green shadows etc).
Is that even feasible?
Hi all!
I've read through this entire thread once before...but I'm going to risk asking a possibly obvious question instead of
rereading all 88 pages.
I took a bunch of photos tonight. About half were dual iso, the other half were taken normally. I forgot to make a new folder when I was taking the dual iso ones, and I switched dual iso on and off quite a bit to do comparisons. Is there a way to easily identify which are which? I just pulled up the card in Canon DPP and I don't see the interlacing!?
Thanks again, and sorry if this question was addressed (multiple times) before.
-Daniel
Quote from: ShaunWoo on June 06, 2014, 06:27:49 PM
Hey guys, hope i aint being a nuisance, but any update on the "bright dark detection error, iso blending didnt work" bug?
im really desperate for a fix
or would it be possible if the developers let us know if a fix is to be expected anytime soon, because if not we can move on from waiting and continue with an alternate method
update and notes and tutorial for the un educated people on the following problems:
"bright dark detection error, iso blending didnt work"
flickering when filmed with dual iso
using the older version:
2014.04.08\cr2hdr-20bit.exe, doesnt give the "bright dark detection error, iso blending didnt work" error
whilst
2014.05.06\cr2hdr-20bit.exe, DOES
so apply same levels in post yourself, as cr2hdr same levels parameter uses exiftool to apply same levels anyway
processes the 0 and 1 files, and to apply same levels i used, ExifToolGUI, run the program, navigate to first file of the sequence, on the right find BlackLevel and WhiteLevel, note down the values, select all the files (i found that if i select around 2000 files it doesnt process the command so theres a file ammount limitation, 1430 worked, so maybe 2k is the limit?) at top right of the gui program, click on exiftool direct, copy and paste:
-Exif:BlackLevel=2046
NOTE, the 2046 is the variable, enter the BlackLevel you noted down from first file, click on ok, itll process the files, and create a backup of the original files, if you dont want them backed up, go to options at top left > dont backup files when writing.
now repeat the procedure for the whitelevel:
-Exif:WhiteLevel=2046
again the 2046 is the variable, click on ok, once both the whitelevel and black level have been changed to match the first file of the sequence, there should be no flickering what so ever, i dont believe ACR caused the flickering in the first place, as when i applied ACR adjustments, even to extreme cases, like -100 highlights, +100 shadows, etc, no flickering was caused what so ever
Hi, I agree this way is very useful for dual-iso video flickering prevenction... but I just figure out what is another cause of flickering, sometimes still visible even after the "ExiftoolGUI-FIX":
dual-iso videos have the dark-light stripes that exchange position every other frame, I mean you have one frame as (example) line 1-2, 5-6, 9-10, etc. at (example) 100 ISO and 3-4, 7-8, 11-12, etc. at 800 ISO, then the next frame is 2-3, 6-7, 10-11, etc. at 100 ISO and 0-1, 4-5, 8-9, etc. frames at 800 ISO. This cause a little flicker, sometimes visible in hi contrasty and/or hi-iso difference situation.
Lorenzo
Does anybody have accurate info on which camera's are currently able to run Dual ISO in raw video mode? I keep reading conflicting things.
Quote from: RTLdan on July 05, 2014, 11:48:37 AM
Hi all!
I've read through this entire thread once before...but I'm going to risk asking a possibly obvious question instead of
rereading all 88 pages.
I took a bunch of photos tonight. About half were dual iso, the other half were taken normally. I forgot to make a new folder when I was taking the dual iso ones, and I switched dual iso on and off quite a bit to do comparisons. Is there a way to easily identify which are which? I just pulled up the card in Canon DPP and I don't see the interlacing!?
Thanks again, and sorry if this question was addressed (multiple times) before.
-Daniel
Ok, ok, so maybe I was a little lazy last night after shooting and I didn't search hard enough. Did another google search today and came up with a reference to a dual iso tag that should be in the exif data. That sounds promising. Only problem is, using a photo that I'm almost positive I took as dual iso, I cannot see this tag using either Canon DPP or the command line exiftool.
Any ideas on where to look for the dual iso exif tag, and/or, a nice gui based windows exif viewer?
Thanks!
-Daniel
when you preview Dual ISO file, you should be able to see scan lines. if not zoom in in Light Room
Hallo,
I am a Fedora-Linux user and I tried to get cr2hdr to work.
First I tried to build it from the source using the method outlined here for Ubuntu:
https://bitbucket.org/hudson/magic-lantern/commits/50345b4d561151e4707c04e2b874e1f909bef6d9 (https://bitbucket.org/hudson/magic-lantern/commits/50345b4d561151e4707c04e2b874e1f909bef6d9)
But I gave up on the part of using dpkg to prepare the gcc compiler.
Luckily I came across the bitbucket archive of rufustfirefly
https://bitbucket.org/rufustfirefly/magic-lantern/downloads (https://bitbucket.org/rufustfirefly/magic-lantern/downloads)
where I can download a statically linked binary. Cool. 32 bit, I use 64, but it still works.
However when I call it with one of my files, it gives me:
[bastian@simpsus 20140712]$ ~/temp/cr2hdr-static.linux.x86.2014-06-19-f9a29c7c0685 IMG_0123.CR2
cr2hdr: a post processing tool for Dual ISO images
Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
cr2hdr: print a message when overwriting the output file
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
Input file : IMG_0123.CR2
Camera : Canon EOS M
Full size : 0 x 0
Active area : 0 x 0
Error: pgm width
which I cannot make any use of. Has somebody a hint for me to either
use cr2hdr on linux/fedora or
how to get rid of this error?
Thanks,
Bastian
Sounds similar to this: http://www.magiclantern.fm/forum/index.php?topic=7693.msg69865#msg69865
Hello everyone,
Since yesterday, I test DualISO mode on my DSLR.
In the menu, it is now possible to choose values as: 100, 200, 400, 800, 1600, ...
However, multiples of 160 ISO get better result to retrieve the dark tones, why not add: 160, 320, 640, 1250 ...?
To get there, I tested the functionality upside down, and I set the camera to ISO 2500, then DualISO on -5EV
Recovery of Shadows was better (less chromatic noise) than with the device in 100iso and DualISO 1600.
With 1250iso and DualISO -4EV and the results were also better.
What do you think?
Thanks
http://www.magiclantern.fm/forum/index.php?topic=10111.0
Quote from: Audionut on July 15, 2014, 01:31:11 PM
http://www.magiclantern.fm/forum/index.php?topic=10111.0
thanks, I try to understand the information on the Topic, even if it is difficult to translate the information in my native language.
But I can not understand why I got a better result with that in 1250 and 2500 to 1600 ...
Care to upload the CR2? 100/1600 and 100/2500.
2500/100 is more like 2500/80, since the intermediate gains are the ones of the base ISO. Dual ISO only operates on the full-stop amplifiers.
Shadow noise should be equal in 1250 and 1600. In 2500 is lower, because it's based on 3200.
Quote from: a1ex on July 13, 2014, 09:02:03 PM
Sounds similar to this: http://www.magiclantern.fm/forum/index.php?topic=7693.msg69865#msg69865
Thanks, the language setting solved it
That should be actually included in the source code, just making it cross-platform is a bit of a hassle.
Quote from: Audionut on July 15, 2014, 03:28:31 PM
Care to upload the CR2? 100/1600 and 100/2500.
Quote from: a1ex on July 15, 2014, 03:39:15 PM
2500/100 is more like 2500/80, since the intermediate gains are the ones of the base ISO. Dual ISO only operates on the full-stop amplifiers.
Shadow noise should be equal in 1250 and 1600. In 2500 is lower, because it's based on 3200.
Ok, I had not kept my tests yesterday, but I had increased only shadows on Camera Raw to see the difference (it was a mistake, see below)
Today I made a new test with these values: 800/100, 640/100 (more like 640/80 as stated a1ex) and 400/100
Here are the results (they are 100% crop of a dark area):
800iso :
(http://i.imgur.com/n8P2mgP.png)
high res : http://i.imgur.com/hHrpgUh.jpg (http://i.imgur.com/hHrpgUh.jpg)
640iso :
(http://i.imgur.com/t6PTQYk.png)
high res : http://i.imgur.com/RDJlhAW.jpg (http://i.imgur.com/RDJlhAW.jpg)
400iso :
(http://i.imgur.com/nnWv7Vx.png)
high res : http://i.imgur.com/sHbzzbs.jpg (http://i.imgur.com/sHbzzbs.jpg)
We can see that the three images do not have equivalent exposure if exposure increases equally, despite compliance photographic units (I had not seen the difference yesterday).
(a1ex & Audionut, merci pour vos réponses, et surtout un grand merci pour votre travail)
2 points to note:
The shadow slider in ACR is not linear. It will produce different results. Do not use it for brightness matching.
All exposure adjustments should be equal. All exposures are brightness matched by cr2hdr, the only difference being the level of noise in the shadows.
The exposures are matched from the lower ISO, ISO 100 in this case. The higher ISOs are only used to clean the shadows, they do not effect brightness rendering.
Quote from: a1ex on July 15, 2014, 03:39:15 PM
2500/100 is more like 2500/80, since the intermediate gains are the ones of the base ISO. Dual ISO only operates on the full-stop amplifiers.
Shadow noise should be equal in 1250 and 1600. In 2500 is lower, because it's based on 3200.
A1ex this is only for DUAL ISO? If not where I can get the table/values to know which is the maximum ISO with the same shadow noise?
I assume you are interested in RAW movie mode, where you only have full-stop ISOs, so check this graph: http://www.magiclantern.fm/forum/index.php?topic=10111.msg118553#msg118553
Quote from: a1ex on July 18, 2014, 09:11:44 AM
I assume you are interested in RAW movie mode, where you only have full-stop ISOs, so check this graph: http://www.magiclantern.fm/forum/index.php?topic=10111.msg118553#msg118553
Wow!
dr_1080 = [ 11.10 11.05 11.02
10.93 10.73 10.39 9.88 8.88 ];
I will go up with ISO!
I am trying to see if this is expected
I rented a Mark III to test dual ISO video (with proper card...).
With normal video I get frame A a certain exposure, and frame B another one - and A,B,A,B alternating
With RAW I don't understand what I got
I have a frame with a certain exposure then the other frame has a 2 scanline pattern, 2 scanlines same exposure (light) followed by 2 scanlines (dark) and so on. Here's a JPEG of what the bad frame(s) look like.
http://www.revisioneffects.com/bugreports/ML/testML.jpg (http://www.revisioneffects.com/bugreports/ML/testML.jpg)
I would expect either alternating exposure on a frame or scanline basis but not every 2 frames every 2 scanlines
I thought it might be the converter from .RAW to .dng but I tried different ones with same results
This was for the most part 1080P 30 FPS, I did follow the instructions
So the first question is with RAW video dual ISO what should I expect?
Is there sample .RAW files somewhere I can go through same conversion workflow to see it's not me introducing this problem?
Pierre
Quote from: Pierre Jasmin on July 19, 2014, 05:25:07 AM
I am trying to see if this is expected
I rented a Mark III to test dual ISO video (with proper card...).
With normal video I get frame A a certain exposure, and frame B another one - and A,B,A,B alternating
With RAW I don't understand what I got
I have a frame with a certain exposure then the other frame has a 2 scanline pattern, 2 scanlines same exposure (light) followed by 2 scanlines (dark) and so on. Here's a JPEG of what the bad frame(s) look like.
http://www.revisioneffects.com/bugreports/ML/testML.jpg (http://www.revisioneffects.com/bugreports/ML/testML.jpg)
I would expect either alternating exposure on a frame or scanline basis but not every 2 frames every 2 scanlines
I thought it might be the converter from .RAW to .dng but I tried different ones with same results
This was for the most part 1080P 30 FPS, I did follow the instructions
So the first question is with RAW video dual ISO what should I expect?
Is there sample .RAW files somewhere I can go through same conversion workflow to see it's not me introducing this problem?
Pierre
Wich software are you using to convert all the footage?
Mystic? Raw2cdng? RawMagic LITE? Dual ISO has different software previous color correction.
Hi guys! Been playing with dual iso movies again and come to the conclusion that 3x mode is working really good for this. Having issues converting in lightroom using cr2hdr(Alex) tool plugin from Kitchehof. Getting a white bottom pixel stripe or something. This is more evident when selecting "fullres" in post processing within lightroom. Decided to test a few other converters.
As it goes, for now MLVMystic seems to have both fullres and with no white pixel strip giving good results. My wish is to get the "fullres" to work with recent cr2hdr within lightroom. Is it hard to change this behaviour in cr2hdr? What is causing this?
These are crops 1:1
Lightroom(Kitchehof) "Fullres" conversion with pixel stripe,
(http://s2.postimg.org/6jd2pryft/Fullresblending.png)
Lightroom(Kitchehof) "nofullres" Still a little pixel stripe at the bottom.
(http://s2.postimg.org/5fsydtdsp/nofullres.png)
MLVMystic Giving good result without the any white pixel stripe
(http://s13.postimg.org/69st0bvhj/MLVMystic.png)
OSX_cr2hdr, an october version of the A.D mac drag and drop converter based on the cr2hdr before the white stripe problem
(http://s2.postimg.org/p1hfa0g7t/cr2hdr.png)
Quote from: Danne on July 19, 2014, 09:37:01 AM
Hi guys! Been playing with dual iso movies again and come to the conclusion that 3x mode is working really good for this. Having issues converting in lightroom using cr2hdr(Alex) tool plugin from Kitchehof. Getting a white bottom pixel stripe or something. This is more evident when selecting "fullres" in post processing within lightroom. Decided to test a few other converters.
As it goes, for now MLVMystic seems to have both fullres and with no white pixel strip giving good results. My wish is to get the "fullres" to work with recent cr2hdr within lightroom. Is it hard to change this behaviour in cr2hdr? What is causing this?
Lightroom(Kitchehof) "Fullres" conversion with pixel stripe,
http://s2.postimg.org/6jd2pryft/Fullresblending.png
Lightroom(Kitchehof) "nofullres" Still a little pixel stripe at the bottom.
http://s2.postimg.org/5fsydtdsp/nofullres.png
MLVMystic Giving good result without the any white pixel stripe
http://s13.postimg.org/69st0bvhj/MLVMystic.png
OSX_cr2hdr, an october version of the A.D mac drag and drop converter based on the cr2hdr before the white stripe problem
http://s2.postimg.org/p1hfa0g7t/cr2hdr.png
Hi Danne, I see your photogram too noisy. I don't know if this is normal, but for professional purposes could be a big problem. It's an opinion. But maybe you have reason for do this.
The noise is irrelevant to my questions about the white pixel border issue and cr2hdr.
It is three times as noisy since I,m experimenting with 3xmode. It is also a crop 1:1
I can't understand this (from the first page):
"Expose to the right for the lower ISO (usually ISO 100). Maybe darken 1 stop from there."
the metering looks like this : -2 ... -1 ... 0 ... +1 ... +2
Basically i should expose to the right (+1) and the darken 1 stop (go back?)
Let's say i have the sky and some shadow(y) parts
I usually use the spot mettering so I can get to pick which area i'm exposing for. Should I aim for the sky or for the shadow parts?
@mihail
If it helps, the way I use Dual-ISO is virtually always in conjunction with Auto-ETTR.
I use the SET setting for A-ETTR.
Thus my simple exposure workflow is this :
1. Comprise the scene
2. Take an A-ETTR exposure setting
3. Assuming you have also set Dual-ISO, say at 100/800 take your image
4. Ingest into Lightroom and use the CR2hdr plugin
5. Finish off in LR and Ps
Refinements will be setting Dual-ISO on every other image, and using the S/N setting, which I usually have zeroed out.
BTW here are a couple of iMages taken using A-ETTR and dual-ISO
http://photography.grayheron.net/2014/07/blue-hour-at-buffalo-thunder.html
Quote from: budafilms on July 19, 2014, 06:32:31 AM
Wich software are you using to convert all the footage?
Mystic? Raw2cdng? RawMagic LITE? Dual ISO has different software previous color correction.
You are correct I don't get the same result with Magic Lantern RAW - 1.2.0 and raw2cdn (windows)
With raw2cdn I get something more useful
I do get 2 dark lines, 2 light lines, 2 dark lines, 2 dark lines,...
Is this what I should expect?It's fine if it's what I am supposed to get, just making sure what I am supposed to get
Pierre
Quote from: Pierre Jasmin on July 20, 2014, 01:14:41 AM
You are correct I don't get the same result with Magic Lantern RAW - 1.2.0 and raw2cdn (windows)
With raw2cdn I get something more useful
I do get 2 dark lines, 2 light lines, 2 dark lines, 2 dark lines,...
Is this what I should expect?
It's fine if it's what I am supposed to get, just making sure what I am supposed to get
Pierre
Right! Have you change exposure, contrast, saturation and all that stuff in other software like ACR or DaVinci Resolve Lite (Free)?
Maybe you have read this: http://www.magiclantern.fm/forum/index.php?topic=7139.0
So does dual ISO only work on the 5D3 for video? It's greyed out for me on the 50D in video mode :/
Quote from: budafilms on July 20, 2014, 01:22:34 AM
Right! Have you change exposure, contrast, saturation and all that stuff in other software like ACR or DaVinci Resolve Lite (Free)?
Maybe you have read this: http://www.magiclantern.fm/forum/index.php?topic=7139.0
Not what I am saying - I understand HDR... see the link to the picture I posted earlier (2 dark lines, 2 light lines, 2 dark lines, 2 light lines)
Wait, not always, some RAW I shot I get alternate frames, not 2 scanlines dark, 2 light
Seems there is a combination of settings that changes what the output is (alternate frames of different levels or switching levels on same frame )
Quote from: Pierre Jasmin on July 20, 2014, 01:55:25 AM
Wait, not always, some RAW I shot I get alternate frames, not 2 scanlines dark, 2 light
Seems there is a combination of settings that changes what the output is (alternate frames of different levels or switching levels on same frame )
OK I see the white paper (which is well writen by the way) says the rational for two scanlines is for debayering - makes sense
I stilll have not figured out why I sometimes get alternating frames and sometimes alternating scanlines
That's said -
1) don't think this is an option but for in theory for alternating frames mov mpeg 4 has a lot of difficulty usually with alternating levels (any intra-frames schemes), a good hack would be to store the alternate files in separate video streams. (or perhaps RAW on CF while mpeg4 on SD card - but with different exposure settings).
2) I see that it's true that changing the IS0 (as in dual ISO) appears to work for the noise as advertised. Does that mean Canon has some analog gain that this affects? Or is this because of the +0.3 ev I see in parenthesis?
3) There is also a rationale for alternate frames (one can play with our tool DE:Flicker High Speed http://revisionfx.com/products/deflicker/ (http://revisionfx.com/products/deflicker/) - Alternate Mode (Method 6.) with alternate frames mode to see - although not as is designed for this as there is no let's say tone mapping done...). It would be nice if in the HDR Video menu there was 3 options: (OFF, alternate frame, alternate scanline).
4) I do like that the live view alternates (the way it switches) otherwise it's a bit hard to set two levels (sde by side would work too, would maybe be easier to set focus). I am going to rent another camera next month or so to play some more, this is way cool. Although as said in the white paper 4 stops is usually good enough for realistic HDR at least until we can shoot 16b float but trying this with about 10 different things, I failed a few times. Maybe what is really needed is to have one capture for black level (no clamping even if noisy) and one capture for white level (no clamping at 1.0 except except maybe if you shoot the sun in frame or some specular high light reflections of the sun). I am mentioning as a potential idea for user interface for this as the sequence of buttons to press in what order is easy to mess up, would be more natural to look at a dark patch and press OK, then a very white patch and press OK and dual Levels recording is set. Seems since the two things compared are pretty much the same pixels looked at between frames restoring a relative level curve to the other is pretty reliable in a post process so it's fine to deviate from the classical photography capture model.
5) I note that the sample .dng file on first entry of this thread does not seem to load in Adobe CC 2014 CRAW reader (at least in AE). In general with this stuff it's probably not extremely useful to set the controls on the image reader itself though.
6) I also had the issue with the rental camera that often times the HDR video recording auto stopped after a few seconds. Someone suggested that it's a battery worn rental camera. I was wondering too if I set up something that made that happen.
Thanks,
Pierre
You have a lot of ideas! You have a good level!
I think a Dev can read this and give you a good explanation.
I prefer shoot in Raw, it's something I can dominate and, maybe less dynamic range, but less noise too.
Using ETTR, it's a function that tell you if you need use Dual ISO or not. Maybe you can check that module, it's very, very good...
I have an answer for my own question (from another thread)
"
2) I see that it's true that changing the IS0 (as in dual ISO) appears to work for the noise as advertised. Does that mean Canon has some analog gain that this affects? Or is this because of the +0.3 ev I see in parenthesis?
"
A particular camera (sensor) has a point (like 3200 ISO) where it shifts from analog to digital gain.
i.e. so over 3200 one would be better to do it in post (at least if they shoot RAW)
Pierre
Off topic novels about installation of modules moved here: http://www.magiclantern.fm/forum/index.php?topic=12712.0
@Pierre Jasmin http://www.magiclantern.fm/forum/index.php?topic=10111.0
Hey Guys. I have a little Problem over here with Dual Iso. At first, u realy did a great job with this Modul, Big Thanks for that!
But now the Problem. I tryed the Dual Iso with the newest Version of Full Res Silent Pic (Imagine silent pics + 14ev dr... realy awesome).
So i took a dual Iso image on the normal way. I got a simple .cr2 file as always. (left side)
After that i took a Full Res Silent Pic as a 14Bit .DNG with the new Modul (right side)
(http://i.imgur.com/SROp4SM.jpg)
Both Images look OK for Dual Iso processing, or?(Yeha, the right pic is a little bit to bright, what ever ;) ) So i processed them both on the same way with cr2hdr.exe
But now look what i got after the process
(http://i.imgur.com/q5Fo34t.jpg)
The .cr2 image(still left side) looks fine but the Full Res Silent Pic .DNG(right side) looks very annoying.
Did i something wrong? Does some1 know's what happend here? Help pls!
Attero, you should upload the file so the experts can take a better looks
Ok im on it.
So, here are the Images
Standart Dual Iso capturing .cr2 --> https://www.dropbox.com/s/dw4yzgimi86vk80/Dual_Iso_CR2.CR2 (https://www.dropbox.com/s/dw4yzgimi86vk80/Dual_Iso_CR2.CR2)
Full Res Silent Pic capturing .DNG --> https://www.dropbox.com/s/e8umt8em8wl53oq/Dual_Iso_Silent_Pic.DNG (https://www.dropbox.com/s/e8umt8em8wl53oq/Dual_Iso_Silent_Pic.DNG)
Skip offsets are wrong in raw.c.
exiftool Dual_Iso_Silent_Pic.DNG -ActiveArea="50 142 3465 5344"
Ok. So, what can i do to fix this?
I just tried ETTR and DUAL ISO with Full-size Silent MLV
out of 5 shots only one were made into DUAL ISO (with stripes) and it were process OK with cr2hdr.exe
the stripe were removed and file size went from 37mb to 43mb
Just for experimenting I tried to process the remaining 4 with cr2hdr.exe and nothing were change.
NOTE: I used the new mlv_dump to extract DNGs from the MLV
QUESTION: Why only one got made into DUAL ISO?
Here are EXIF info
1) 1/15 sec F11.3 ISO 100
2) 1/1000 sec F11.3 ISO 100 this is the ONLY file have horizontal stripes (why is that?)
3) 1/1000 sec F11.3 ISO 100
4) 1/1000 sec F11.3 ISO 100
5) 1/166 sec F11.3 ISO 100
Probably the image was not noisy enough to require dual ISO. You can try increasing the SNR limits.
Thanks Alex for the explanation. Now I understand.
I have decided NOT to use Dual ISO for Full-size Silent because
Shutter speed is erratic (some frame may get darken at the top)
Extra steps in workflow
Keep it simple stupid (KISS) policy
Erratic? The shutter speed behavior was explained in great detail...
sorry wrong choice of word. I meant not consistent. for time-lapse we wouldn't want some frames have darker gradient at the top ... flicker
dark, dark, dark, light, dark, light, light ect...
Got it. I'm afraid it requires a custom deflicker algorithm that knows about this gradient.
Hi,
I have been playing with dual-iso and FullRes-SilentPic for the last view days to shoot time-lapse.
Now I would like to add ETTR to the process but I have 2 problems.
#1 ETTR does not have a "fastest shutter" setting to prevent it from using values that make no sense with FullRes-SP (<1/10 or 1/30)
#2 It seems that ETTR looks always at the entire picture to set the ISO and Shutterspeed. It would be cool if ETTR could be set to only measure the darker half of the DualISO Pics to be able to shoot bright sky and very dark parts in one image with DualISO 100/1600.
Just curious did you do ETTR FullRes-SP silent in DNGs or MLV
When you did DUAL ISO FullRes-SP only, didn't your shutter speed change a lot through out the shoot? If you did in DNG then you would't have EXIF info. But you may have notice/remember
Quote from: barepixels on July 27, 2014, 07:50:37 PM
Just curious did you do ETTR FullRes-SP silent in DNGs or MLV
I havent used ETTR until now - cam is shooting with ETTR in .MLV right now. The dual ISOs where recorded as .MLV (200+ frames, splited .MLV files with up to 6GB = 4+2GB)
Quote from: barepixels on July 27, 2014, 07:50:37 PMWhen you did DUAL ISO FullRes-SP only, didn't your shutter speed change a lot through out the shoot? If you did in DNG then you would't have EXIF info. But you may have notice/remember
With "DualISO only" you mean without ETTR? I dont know - I would asume not - but in one test I saw some flickering but havent looked into it any further till now.
edit:
a video of my tests is here: http://www.magiclantern.fm/forum/index.php?topic=12802.0
its the 3rd scene without clouds that flickers
Quote from: golem on July 27, 2014, 06:00:22 PM
It would be cool if ETTR could be set to only measure the darker half of the DualISO Pics
If ETTR doesn't "measure" the bright part of you picture, that part will just be blown highlights. If that's actually what you want, then set the highlight ignore to 50%.
Quote from: dmilligan on July 27, 2014, 09:29:04 PM
If ETTR doesn't "measure" the bright part of you picture, that part will just be blown highlights. If that's actually what you want, then set the highlight ignore to 50%.
Thats exactly what I want - I don´t need the full resolution in the bright clouds.
Right now I do this manually by not overexposing with 100 ISO and recording with 100/800
I thought about the 50% but that only works if the brighter image is 100% (all pixels) overexposed.
When used with Dual ISO, ETTR measures the darker sub-image only.
Quote from: a1ex on July 28, 2014, 08:00:40 AM
When used with Dual ISO, ETTR measures the darker sub-image only.
Yes, now it works - not sure what went wrong in my first test - thanks.
Any chance to ever get this working in Raw video mode for the 50D and 5D2?
Hi there. Thanks for DUAL ISO !!
Sorry, but I have since yesterday problems to convert the images correctly.
I have the RAW image in the dropbox.
The DNG file after conversion still has the lines can be seen. I have tried everything, but unfortunately can not find the error.
It was photographed with ISO 100/400
Bulb Timer 2min
Dark Frame
Thank you for your help and thank you for the great work that makes all your !!
Greetings
Christian
https://www.dropbox.com/sh/ocp8xl7q6u7a4jd/AABJQ7-LciZ3QFwmjTtCN2G-a
Edit:
The RAW was converted with the current cr2hdr.exe files with old files I still had, with the Lightroom plugin and without (ie directly in the folder)
@chris_1975:
It looks like you have chosen an extreme exposure situation, so cr2hdr-algorithm picks a wrong rendering-choice.
I tried with some cr2hdr-betas and it worked out fine:
Jpgs developed with ufraw
latest cr2hdr:
(http://img207.imagevenue.com/loc523/th_017714228_sepp_cr2hdr_122_523lo.jpg) (http://img207.imagevenue.com/img.php?image=017714228_sepp_cr2hdr_122_523lo.jpg)
cr2hdr-amaze-edge5:
(http://img12.imagevenue.com/loc177/th_017931546_sepp_amaze_edge5_122_177lo.jpg) (http://img12.imagevenue.com/img.php?image=017931546_sepp_amaze_edge5_122_177lo.jpg)
cr2hdr-20bit:
(http://img152.imagevenue.com/loc107/th_801812120_sepp_20bitcr2hdr_122_107lo.jpg) (http://img152.imagevenue.com/img.php?image=801812120_sepp_20bitcr2hdr_122_107lo.jpg)
Btw. from my experience Dual-ISO doesn't improve a longtime exposure at night
I copied below the output of the command window for further research by a pro ;D
E:\ufrawconvert>cr2hdr sepp.cr2
cr2hdr: a post processing tool for Dual ISO images
Last update: d0ac769 on 2014-01-23 10:13:39 UTC by alex:
cr2hdr: experimental white level routine with two separate levels ...
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default
)
Input file : sepp.cr2
Camera : Canon EOS 5D Mark III
Full size : 5920 x 3950
Active area : 5796 x 3870
Black borders : 124 left, 80 top
Black level : 1965
ISO pattern : BddB RGGB
White levels : 15301 15300
Noise levels : 10.82 8.34 8.34 10.74 (14-bit)
Black delta looks bad, skipping correction
ISO difference : 1.78 EV (344)
Black delta : 0.00
Dynamic range : 10.64 (+) 10.27 => 12.05 EV (in theory)
Looking for hot/cold pixels...
Hot pixels : 3080
Cold pixels : 162
AMaZE interpolation ...
Amaze took 6.84 s
Edge-directed interpolation...
Semi-overexposed: 0.00%
Deep shadows : 51.67%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap : 5.9 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 53.43 (16-bit), ideally 37.05
Dynamic range : 9.96 EV (cooked)
Black adjust : -5
Output file : sepp.DNG
1 image files updated
----------------------------------------------------------------------------------------------
E:\ufrawconvert>cr2hdr-20bit sepp.cr2
cr2hdr: a post processing tool for Dual ISO images
Last update: 26b7a08 on 2014-05-06 20:04:08 UTC by a1ex:
cr2hdr: experimental option to embed the original raw into the DNG
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directe
d interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default
)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (def
ault)
Input file : sepp.cr2
Camera : Canon EOS 5D Mark III
Camera model : Canon EOS 5D Mark III
Full size : 5920 x 3950
Active area : 5796 x 3870
Black borders : 124 left, 80 top
Black level : 2048
ISO pattern : BddB RGGB
White levels : 15212 13794
Noise levels : 10.81 8.32 8.34 10.73 (14-bit)
ISO difference : 1.98 EV (395)
Black delta : 1.98
Dynamic range : 10.63 (+) 10.09 => 12.07 EV (in theory)
Looking for hot/cold pixels...
Hot pixels : 1210
Cold pixels : 156
AMaZE interpolation ...
Amaze took 3.40 s
Edge-directed interpolation...
Semi-overexposed: 0.58%
Deep shadows : 82.23%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap : 5.6 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 143.94 (20-bit), ideally 140.87
Dynamic range : 12.51 EV (cooked)
Black adjust : -46
AsShotNeutral : 0.62 1 0.55, 4127K/g=1.20 (gray max)
Output file : sepp.DNG
sepp.DNG : updating EXIF
-----------------------------------------------------------------------------------------
E:\ufrawconvert>cr2hdr-amaze-edge5 sepp.cr2
cr2hdr: a post processing tool for Dual ISO images
Last update: af9a60f on 2013-10-24 18:44:01 UTC by alex:
cr2hdr: always use last 8 lines from top optical black bar (should fi...
Input file : sepp.cr2
Canon EOS 5D Mark III detected
Full size : 5920 x 3950
Active area : 5796 x 3870
White level : 12500
Black borders : 124 left, 80 top
Black level : 1965
ISO pattern : BddB RGGB
Noise levels : 10.82 8.34 8.34 10.74 (14-bit)
Estimating ISO difference...
ISO difference : 2.00 EV (399)
Black delta : 5.90
Interpolation : amaze-edge-chroma2x2-alias
AMaZE interpolation ...
Amaze took 6.79 s
Edge-directed interpolation...
Semi-overexpo'd: 0.64%
Deep shadows : 53.94%
Dynamic range : 10.30 (+) 9.93 => 11.92 EV (in theory)
Matching brightness...
Horizontal stripe fix...
Looking for hot/cold pixels...
Hot pixels : 877
Cold pixels : 171
Full-res reconstruction...
ISO overlap : 5.3 EV (approx)
Half-res blending...
Chroma filtering...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 9.79 (16-bit), ideally 9.53
Dynamic range : 12.07 EV (cooked)
Black adjust : -8
Output file : sepp.DNG
1 image files updated
THX Glubber...
now i tried it with "cr2hdr-hot" ... it works .... thanks...
regards chris
The long drawn out discussion about learning how to process dual ISO files, has been moved here: http://www.magiclantern.fm/forum/index.php?topic=13028.0
Hi! I see a bug in cr2hdr when converting from dualiso dng to regular dng.
I,m wondering if there is anything to do regarding this issue?
https://bitbucket.org/hudson/magic-lantern/issue/2091/cr2hdr-dual-iso-3x-zoom-mode-strange
I,ve been trying to get some answers before but with little success. Also wondering if where I,ve placed this issue-report on bitbucket is the right place?
Thanks
/D
Quote from: kichetof on May 07, 2014, 06:36:39 PM
$ ./cr2hdr
cr2hdr: a post processing tool for Dual ISO images
Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
cr2hdr: print a message when overwriting the output file
Download cr2hdr-20bit (https://bitbucket.org/kichetof/lr_cr2hdr/downloads/cr2hdr.30a5132.zip) for Mac only with last commit 30a5132 ;)
@a1ex thanks for having solved the warnings on compilation!
I am totally hooked on how good Dual ISO MLV's look on a Canon 5D3 to get nicer highlights, but I've broken my teeth on this for several days now and I'm dying to see the latest improvements in quality and less flicker in my files. I can't get this version to work on my mac and I'm very sorry to ask for help.
dcraw & exiftool are installed. The version from 4/10/13 works as expected.
It would be great if there was a complete package out that that people can run without any further installations. I'm trying every approach and I understand if a developer doesn't feel like pushing every update to the mac platform, but perhaps there's someone out there to help me out. Or point to other useful threads. Of course you can also reply here:
http://www.magiclantern.fm/forum/index.php?topic=13028.0
Quote from: dubzeebass on May 07, 2014, 07:46:20 PM
Many thanks!
Care to provide a step by step or checklist of all the stuff that should be installed in the tool chain to get this working.
If you teach me how to cross-compile a Mac binary from Linux, I can add it to the nightly build server. I wanted to include the postprocessing tools on that page for a long time.
I'm sorry. I can not be of too much much help there (yet). Sorry.
On the other hand, a small step in the that direction has been taken as I got the 20bit version of cr2hdr to work. And very happy about that!
The steps I took:
-Move to a computer with a different OS. OS X 10.9.4 instead of OS X 10.6.8 (the cr2hdr didn't run on 10.6.8 )
-Installed dcraw with this tutorial: http://vkphotoblog.blogspot.be/2014/05/dcraw-921-for-os-x-mavericks-users.html
(including xcode installation) (interesting read too)
-Move cr2hdr to the same folder as dcraw is in. Allows you to just type the cr2hdr command without specifying a directory.
-Install exiftool from here http://www.sno.phy.queensu.ca/~phil/exiftool/
-Run the program from the terminal
-Say fuuuuuuh...!
I will see if I can now also make it work on a OS X 10.6.8 machine.
Quote from: a1ex on August 25, 2014, 03:54:03 PM
If you teach me how to cross-compile a Mac binary from Linux, I can add it to the nightly build server. I wanted to include the postprocessing tools on that page for a long time.
Hi a1ex, I found OSXCross (https://github.com/tpoechtrager/osxcross) maybe it will works to compile it ! :)
(another way (http://wiki.lazarus.freepascal.org/Cross_compiling_OSX_on_Linux))If you need some help/files, I'll send to you !
Quote from: KurtAugust on August 25, 2014, 10:25:42 AM
I am totally hooked on how good Dual ISO MLV's look on a Canon 5D3 to get nicer highlights, but I've broken my teeth on this for several days now and I'm dying to see the latest improvements in quality and less flicker in my files. I can't get this version to work on my mac and I'm very sorry to ask for help.
dcraw & exiftool are installed. The version from 4/10/13 works as expected.
If you run cr2hdr into the Terminal, could you post the output ? And what's you dcraw version ? 9.21 works for cr2hdr 20bit
Quote from: kichetof on August 26, 2014, 11:56:10 AM
If you run cr2hdr into the Terminal, could you post the output ?
Illegal instruction
Quote from: kichetof on August 26, 2014, 11:56:10 AM
And what's you dcraw version ? 9.21 works for cr2hdr 20bit
9.19 works too for cr2hdr 20bit on 10.9.4 , but not on os x 10.6.8
Quote from: KurtAugust on August 26, 2014, 02:48:01 PM
Illegal instruction
Very strange ! It works well on my 2 mac
(10.9.4 too)Quote from: KurtAugust on August 26, 2014, 02:48:01 PM
9.19 works too for cr2hdr 20bit on 10.9.4 , but not on os x 10.6.8
To works on my mac, I need dcraw 9.21.
Quote from: OSXCross
From OSXCross:
Note: libc++ requires Mac OS X 10.7 or newer!
Maybe this is the reason, I don't know... I compiled on Mac OS X 10.9.3
tried dual iso video in normal video mode a few times (so I didn't use crop mode)
I always end up with visual horizontal stripes with a slightly brightness difference. (And yes I processed my files ;) (with CR2HDR v2.0)
And I don't use extreme values, mostly iso 100/800
Is it possible to get normal results with dual iso when not in crop mode ?
Or is impossible because the lineskipping and dual iso won't give a normal RGGB pixel readout in the same iso value ?
I don't see any stripes ;)
Ofcourse I will show some examples ;)
See the png's in this folder (It's not showing the entire frame, but I think it shows what I'm talking about)
https://drive.google.com/folderview?id=0B1BxGc3dfMDaZWxfLTlVQmxnU1k&usp=sharing
How did you get these png's? Were they straight out of the camera?
Those png's are screenshots of the processed frames (mlv-dump osx -> CR2HDR -> RawTherapee -> screenshot...)
I'm uploading some more files to the same folder now.
An original frame out of mlv-dump osx (renamed to CR2), and the same frame processed by CR2HDR (the dng).
This is recorded using the nightly build for 6d from March 15, 2014, using this build still a lot because it gives me the option for FPS override to 35fps.
Can you try with cr2hdr-20bit?
(you may have to run it under Wine, or maybe kichetof has a Mac binary here (http://www.magiclantern.fm/forum/index.php?topic=11056))
Found the Lightroom plugin cr2hdr v2.2 BETA 6 in the topic you linked.
And with 20bit export, the image is clean, no more stripe pattern !!! :D
How come the 20 bit export does work and normal export doesn't ???
huh wait, that's not the whole story.
Did process all frames now, and some of them are good and some of them are not good (horizontal stripes).
These are frames shot within the same clip, they share the exact same mlv file...
uploaded 2 original frames from the same MLV clip to the following folder, also uploaded their processed image by CR2HDR-20bit.
one has horizontal stripes, one doesn't... ???
https://drive.google.com/folderview?id=0B1BxGc3dfMDaZWxfLTlVQmxnU1k&usp=sharing
How come and is there a fix ?
In which of the two files do you see stripes?
( I'm not that good at pixel peeping, can't find any :P )
The last one has stripes
M27-1603_frame_000200-Processed-with20bitcr2hdr.DNG
Frame 200
You can see it below the reflecting sun on the car hood.
But if you download the file and push exposure, you can see it through the entire frame, did that for you and took a screenshot, see "stripes.png" :P
Also uploading a screenshot of frame 199 with the same exposure push, see "NoStripes(frame199).png" in the google drive folder.
The problem is here: https://bitbucket.org/hudson/magic-lantern/src/f12976885e6168813b69dd9acdde719f25045a67/modules/dual_iso/cr2hdr.c?at=cr2hdr-20bit#cl-1520
If you have octave, run it with --iso-curve to see where the problem comes from.
Suggestions welcome (math skills needed).
Too much formulas for bedtime :P, will look at it tomorrow
But I see it does handle each frame separately, for dual iso photo's this makes sense.
For dual iso video clips, it makes more sense to "find the factor and the offset for matching the bright and dark images" only for the first frame and treat all next frames the same.
Just thinking out loud dreaming :
I'm seeing a GUI(in Lightroom CR2HDR plugin) with a slider for manual matching the dark and bright area's :D
By viewing a realtime preview and moving the slider to match and get rid of any lines...
(probably takes a load of programming work :P )
I had some luck with this patch:
diff -r f12976885e61 modules/dual_iso/cr2hdr.c
--- a/modules/dual_iso/cr2hdr.c Tue Jul 15 08:45:33 2014 +0300
+++ b/modules/dual_iso/cr2hdr.c Thu Aug 28 00:39:38 2014 +0300
@@ -1546,10 +1546,11 @@
int pa = raw_get_pixel_20to16(x, y-2) - black;
int pb = raw_get_pixel_20to16(x, y+2) - black;
int pi = (pa + pb) / 2;
+ if (ABS(pa - pb) > 500) pi = clip; /* if it's likely to be aliasing, discard this pixel */
if (pa >= clip || pb >= clip) pi = clip;
interp[x + y * w] = pi;
int pn = raw_get_pixel_20to16(x, y) - black;
- native[x + y * w] = pn;
+ native[x + y * w] = pi < clip ? pn : clip; /* if interpolation failed, discard this pixel */
}
}
@@ -1582,8 +1583,8 @@
//~ int bmed = kth_smallest_int(tmp, n, n/4);
/* also compute the range for bright pixels (used to find the slope) */
- int b_lo = kth_smallest_int(tmp, n, n*98/100);
- int b_hi = kth_smallest_int(tmp, n, n*99.9/100);
+ int b_lo = kth_smallest_int(tmp, n, n*95/100);
+ int b_hi = kth_smallest_int(tmp, n, n*99/100);
/* median_dark */
n = 0;
The fitting algorithm is, in a nutshell:
- place the bright pixel values on the X axis and the dark ones on the Y axis
- discard clipped pixels
- take the median of bright pixels, the median of dark pixels, and draw this point on the graph
- choose the highlights (between two high percentile values)
- take the median of bright and dark pixels from highlights, and draw this second point on the graph
- draw a line through these points - its slope gives the ISO, and its offset is the black level difference between the two exposures
The patch also discards pixels likely to be aliasing, which give poor matches (these are usually outliers) and increases the number of pixels considered highlights (which should filter highlight outliers a bit better).
Can you suggest a better algorithm for robust linear regression?
"Can you suggest a better algorithm for robust linear regression?"
Knowing that dual iso processing is developed and fine tuned for probably more then a year...I doubt it :P.
But I will take a look at the code and maybe I will come up with something :D
Ok, here's a idea.
Don't know what exactly goes on in the code, so I might be wrong:
-I assume all pixels of the frame are read (so if you have a 20 megapixel image, it takes all 20 megapixels for calculation of the difference between the dark and bright image area.
-I'm sure white clipped pixels are discarded, I assume this also means that black clipped pixels are discarded.
Maybe the problem appears when many pixels are discarded.
I don't see anywhere in the code, that there's some compensating for the other ISO when throwing away clipped pixels.
I guess it would be appropriate that if you have too discard about half a million of white clipped pixels in the bright area(higher ISO), you also have to throw away half a million pixels in the dark area (throw away the exact same amount of half a million pixels, so discard half a million pixels with the highest brightness values (this will shift the Median of the non clipped ISO, I hope this Median comes closer to the Median of the other ISO).
If you don't compensate for throwing away clipped pixels in the non clipped ISO part of the image, you are not comparing identical images.
EDIT: Ofcourse the same goes for throwing away clipped shadow pixels, if you throw away half a million of clipped black pixels in the lower ISO, you als have to throw away the same amount of pixels in the higher iso, so throw away the same amount of pixels with the lowest brightness values.
There are two overlapping images: bright and dark. If a pixel from the bright image is clipped, the corresponding pixel from the dark image is not used.
However, to get perfect overlap, I've interpolated the two images with a very simple algorithm - so the two images are aliased (you will always compare a real pixel with an interpolated one). In this case, I believe the aliasing resulted in many outliers - the median, as used in midtones, has no trouble filtering them out, but in highlights there are only a few pixels selected, so outliers are more likely to cause trouble.
I ran a few tests over some images with the same problem, and the algorithm still struggles; what worked with the stock 20-bit version, now has stripes after the above changes. So I'm not there yet.
"If a pixel from the bright image is clipped, the corresponding pixel from the dark image is not used."
It's all even more clever designed than I expected :o
Just to understand.
The median from the mid tones is no problem, lot's of overlap and accuracy.
But you also need to know the exact difference in the highlights (so the brightness difference between the ISO's is not linear through the range ?)
The whole situation with dual iso is, you use the highlights from the lower iso and have them clipped in the higher iso... and yet you do need a way to compare the highlights...ironic...
"but in highlights there are only a few pixels selected, so outliers are more likely to cause trouble."
Can't you build something in like, if less then x(for example 100) pixels are usabla for comparing highlights, start again with slightly lower brightness values, until you get more then 100 pixels to compare.
In the worst case this moves you toward the mid tones and still won't give good results, but maybe it does work for most images ?
It's linear, and the thing we are talking about is just a linear regression. But you have a ton of outliers, so least squares is not going to work.
Here's the data set for your image (frame 200, cr2hdr-20bit unmodified; left is linear, before matching, right is log, after matching):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/levas/iso-curve-200.png) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/levas/iso-curve-200-log.png)
The green dots are what the algorithm selected as "highlights" (wrong choice).
Here's the correct choice (frame 199):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/levas/iso-curve-199.png) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/levas/iso-curve-199-log.png)
You can see in the right graph of frame 200, that the red line is off, indeed highlight point is wrong..blue dots are starting to cluster above the red line from the middle of the graph towards the highlights.
Did try to process the already processed frame 200 with CR2HDR again (processed image does still look like a dual iso file ;D), but it spits out the exact same result.
Just thinking out loud (dual iso is about a year old, so probably a lot has already been discussed in developing it, so sorry if things already have been suggested :) )
"I've interpolated the two images with a very simple algorithm"
I don't see the need for comparing two exact overlays pixel by pixel :-\ (probably missing something important here :P)
If I did know nothing about dual iso and someone gave me one of those raw dual iso images with the question to gave the brightness difference between the dark and light lines, I would do something like this:
-read the amount of dark pixels (let's say 10 megapixels) and sort them in order of value, plus throw out all values below 100(for example),let's say 100.000 pixels are thrown away because they are too low in value.
-read the amount of bright pixels (the other 10 megapixels) and sort them in order of value, plus throw out all values above 15000(for example), let's say 200.000 pixels are thrown away because they are too high in value.
So you end up with two strings of numbers ( 100, 100, 100, 101, 101, 110...14322, 14322, 14355 etc,)
Now compensate for throwing away pixels.
For the bright value string, discard the first 100.000 numbers in the string (assuming they are sorted out on value, this throws away the 100.000 darkest pixels in the bright image)
For the dark value string, discard the last 200.000 numbers in the string (assuming they are sorted out on value, this throws away the 200.000 brightest pixels in the dark image)
Now you have 2 strings of values to compare.
Now the difference in average of these 2 strings is the difference in brightness...
EDIT: so no pixel by pixel difference, but the average of all "bright" pixels compared to the average of all "dark" pixels.
But I'm probably overlooking something, because you're using 2 points(mid tones and highlights)
So maybe you can take the median(or average) of the first half of the two strings for mid tone point.
And the median(or average) of the second half of the two strings for highlight point.
That was one of the first algorithms I've tried, but it's worth revisiting. Updated the log graphs (magenta line is the result of comparing histograms directly at various percentiles).
This curve is biased at the sides (most likely because of different noise amounts in the two exposures), but the middle part seems unbiased. Worth trying on the other tricky test images.
comparing histograms directly makes more sense to me than comparing interpolated images with each other.
The middle of the magenta curve does a good job :D, you can see it nails it for frame 200!
You probably know this already.
But I believe there are formula's to get the exact slope from the middle of the magenta line.
Had this with math in high school (more the 15 years ago :P ), I believe you need the first derivative and set equalize it with zero ??? or something like that :P
Yea, second derivative - or take the pink curve where the second derivative is smaller than a threshold. Not sue if it is going to wok though - depends on exactly what that curve looks like and how close you need to get.
One way I have gotten around fitting data somewhat like this is iterative least squares. What I did was:
Do least squares, compute standard deviation.
Remove outliers more that 2 SD from fit
Refit
Remove outliers more than 1.5 SD from new fit (computed SD from initial fit, not new SD)
Refit
This could work depending on the data and how computationally expensive you can work with; moving in small steps does better, but you may only need 3. at SD 2, 1.75, 1.2 say.
You could also compute the pink curve, then only use the data where the second derivative of the pink curve is below a certain threshold (not sure how variable this will be with different ISO combinations though) and then use iterative least squares on the remaining data.
If the pink and red lines are supposed to always overlap in the center, you could also compute the pink curve and then choose say 40 points (or however many makes sense) on that curve and then use iterative LS to fit a line to those data. That would be a lot quicker than using all the data and you could do enough iterations to (hopefully) guarantee fitting the center part.
Not sure if any of this is actually helpful, but just in case...
Estimating the derivative locally is not exactly accurate.
Another problem with that curve is that - if the two exposures have different amounts of noise or aliasing - it becomes biased. Median is fine, but percentiles at the extreme may not be. This is why my previous algorithm first selected some highlights, then took their median. Here's a test case to show it (magenta curve is biased, but highlight median is not):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/levas/iso-curve-fail.png)
Am I right in saying that dual iso means that you only can compare the green channel, because it's available in both dark and bright lines ?
Red pixels and blue pixels are splitted to only the dark or bright image, so comparing brightness levels of either red or blue pixels is probably meaningless (A.K.A. noise) ?
I compare all of them, assuming they are all linear data.
It's all linear data.
But I can imagine you get biased results, cause they have different filters in front, red and blue light only passes.
A blue sky will appear bright in the blue channel and very dark (if it appears at all)in the red channel.
Let's say you have 3 stops of difference in dual iso, I can imagine for the blue sky, you get more than 3 stops difference between the red and blue channel because of the selective filters.
So? I compare reds with reds and blues with blues, not reds with blues.
But dual iso means alternating lines, so the bright image contains the blue pixels and the dark image contains the red pixels.
You're using interpolated images for comparing, so the difference may be not that big of a problem ???
And the results you get with CR2HDR-20bit are amazing, so it does a really good job.
But comparing interpolated data still sounds like a crazy idea, using created/fake data and treating it as hard actual data can't be the best solution.
And on the other hand it works like a charm for most dual iso images :D, weird ???
"So? I compare reds with reds and blues with blues, not reds with blues."
Just thinking, you're comparing reds with interpolated reds(interpolated from only red data from the other brightness image).
Actually you're comparing the red data with itself.
Edit: Not true, red and blue are both in the dark and bright image (See A1ex reply below)
Quote from: Levas on August 29, 2014, 10:47:15 AM
But dual iso means alternating lines, so the bright image contains the blue pixels and the dark image contains the red pixels.
Nope ;)
Looking at the white paper now, and see that it alternates every 2 lines :-[, that explains a lot.
How does it work for dual iso video, with the line skipping ?
Are both red and blue as wel in the dark image and bright image ?
EDIT: Looking at a video frame now, seeing both dark and bright lines are 2 pixels in height, so both red and blue are in the brigt and dark area's.
Update
Download latest cr2hdr-20bit below (http://www.magiclantern.fm/forum/index.php?topic=7139.msg127103#msg127103)
@Kitchehof. Awesome!
@Alex. Biggest thanks for this fix. Will try right now.
No more border issues converting dual iso moviefiles using 3x zoom mode thanks to Alex fix. in 20bit cr2hdr. Sweet!
(http://s10.postimg.org/iff78nk6x/M02_1558_C0000_00020_dualiso.jpg)
"fixed another border condition"
Testing this new CR2HDR build and I think "fixed another border condition" is not the only thing that has changed/fixed ;D
@Levas exactly :) this is the latest commit!
You can see all commits here (https://bitbucket.org/hudson/magic-lantern/commits/branch/cr2hdr-20bit) for cr2hdr-20bit!
Many thanks a1ex!
Thanks A1ex!
CR2HDR-20bit output has improved a lot for dual iso lineskipping video :D
@kichetof
I'm using this new build in your Lightroom plugin, opened the plugin in finder and replaced the CR2HDR20bit file with this new one, works !
Well, the ISO matching method from previous binary passed most of my tests, but not all. I went back to my previous algorithm, and replaced the highlight median with a brute force search, somewhat similar to RANSAC. The nice property of this algorithm is that it handles a huge number of outliers, as long as the relevant data points are clustered together, and the outliers are scattered all over the place.
Here's something that seems worth reading (I didn't read it yet, only looked at pictures): RANSAC4Dummies.pdf (http://vision.ece.ucsb.edu/~zuliani/Research/RANSAC/docs/RANSAC4Dummies.pdf)
Anyway, here's an update that passed all my tests: cr2hdr-20bit.exe (https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/)
And here are the tests: https://www.dropbox.com/sh/xfkizhu4lpoiuc8/AAC5GkEpZVacRUXcfrv1ikwVa?dl=0
(some very difficult images were 000166, 00017x, _46A0416, _46A3766, _UAL1304, IMG_2524; the previous binary failed at _UAL1304)
$ ./cr2hdr
cr2hdr: a post processing tool for Dual ISO images
Last update: 0eabcb0 on 2014-08-29 12:42:54 UTC by a1ex:
cr2hdr exposure matching: when all else fails, brute force prevails...
Download cr2hdr-20bit for Mac only with last commit 0eabcb0 ;)
Download here (https://bitbucket.org/kichetof/lr_cr2hdr/downloads/cr2hdr.0eabcb0.zip)
"When all else fails" Ha ha ha :D
I think I start off with this build from now on 8)
Or is there a reason not to starts off with this "when all else fails" build and is it (picture quality wise) better to first try the previous build(s) ???
First,
Thanks for improving my life with fun stuff called Magic Lantern!! :)
I have been testing the latest 2014-08-07 00:01:30 +0200 5D.123 build with dual iso.
I can not prosess any .dng files on PC.
Win 7 pro 64bit
5D Mark III
-I first converted the MLV files with Raw2Cdng, Mystic and Mirawviewer in Win 7 PC.
-I then tried converting all the different files with an "old" cr2hdr.exe I found on my pc, I downloaded it August 9th 2013.
-I then downloaded cr2hdr from the link provided in this thread/forum on page 1, tried both the regular one and the 20bit.exe
*20bit.exe converts my dng`s into something cracy.
It produces really messed up files, where the dual iso lines are still present and the colors are loco and pink-ish.
The regular c2hdr converts only 1 dng and then crashes, everytime.
The finished file is just as crazy as the ones that 20bit.exe makes.
Then tried with barracudaGUI, still crazy files coming out..
So,
after all this I tried with another MLV that was recorded with build, 5D.113 2014-07-08 01:11:45 +0200
Made dng`s with Mirawviewer and then dropped them all onto the 20bit.exe.....
Peeerfect with both old and new cr2hdr.exe`s.
That leaves me with not being able to convert dual iso made with the build I am on today, 5D.123 AND WINDOWS
I tried the same dng`s that with a Mac, and that worked without trouble.
Am I missing something?
https://www.dropbox.com/s/oh25s85xqc1a1el/M02-025500003.dng?dl=0 (https://www.dropbox.com/s/oh25s85xqc1a1el/M02-025500003.dng?dl=0)
https://www.dropbox.com/s/1x3w31wjfwpn7b5/M02-030000000.dng?dl=0 (https://www.dropbox.com/s/1x3w31wjfwpn7b5/M02-030000000.dng?dl=0)
from mlv.dump
https://www.dropbox.com/s/ytg6wrlxe89c5o1/M02-0255_frame_000000.dng?dl=0 (https://www.dropbox.com/s/ytg6wrlxe89c5o1/M02-0255_frame_000000.dng?dl=0)
An unprocessed DNG should be enough.
Is it possible to check the exposure in camera when shooting dual iso? Does previewing photos still work the same or is it weird looking? Tried searching, but couldn't find anything.
Use raw based exposure guides (http://www.magiclantern.fm/forum/index.php?topic=12096.0).
Quote from: Audionut on September 05, 2014, 11:46:49 AM
Use raw based exposure guides (http://www.magiclantern.fm/forum/index.php?topic=12096.0).
Does this mean that I have to use liveview to see exposure? I'm interested in dual iso for stills atm.
You can check you're exposure in live view and close it, and then take your still ?
Or stay in live view and make you're still...
Or don't enter LV at all and simply take a photo. The overlays show up in QR too
Hello,
I shoot quite bit of photos using dualiso and am developing them on my mac. The lightroom script is very nice, but on the mac I see a problem using the commandline tools for more interactivity (mainly me not realy understanding how to work with them or installing). That is why I thought using one of the already existing (commercial) programs might ease the workflow for me. So, I asked one company if they wouldn't be interested in supporting dualiso dng's. They told me that they don't know exactly, but that the dng's would lack two different tags to be opened correctly to start with.
These tags are:
CFAPlaneColor tag 50710
CFALayout tag 50711
I absolutely don't know what to do with this, but thought it might be of interest for everybody to pass on this information. I hope I didn't do anything wrong.
Regards,
Kontrakatze
Ok, I need some help here. Maybe a dumb question but... I made all the steps for processing my dual iso photo and in the end I cannot open the final DNG file. Which version of Photoshop and ACR do I need? I have Photoshop CS3 and ACR 4.6.
Thank you all.
QuoteOk, I need some help here. Maybe a dumb question but... I made all the steps for processing my dual iso photo and in the end I cannot open the final DNG file. Which version of Photoshop and ACR do I need? I have Photoshop CS3 and ACR 4.6.
Thank you all.
Try running the Dual ISO DNG through Adobe DNG Converter (FREE) and set the ACR to match what you have. I don't think you'll find anyone here actively using that old of a version of Photoshop. Beside all of the advances in photo processing in the last 5+ years.... it may be time to think about an upgrade. At least try Lightroom 5.6 it's free for 30 days...
If you want a free raw converter I can recommend RawTherapee.
http://www.rawtherapee.com
I'm using both Lightroom 5 and Raw Therapee.
Lightroom is a lot easier to use, but the more I get used to Raw Therapee, I think Raw Therapee gives me better results, especially with dual iso files (better debayering better false color elimination etc.).
Quote from: Stedda on September 14, 2014, 04:31:07 PM
At least try Lightroom 5.6 it's free for 30 days...
Or cost $9.99/month (https://creative.adobe.com/en/plans/photography?store_code=us) with Lightroom and Photoshop. Always up to date!
Lets say we do a full day of shooting a mix of regular and dual iso movie clips/shots. Anyway we can differentiate between the two when post processing?
Custom file prefix and/or metadata keyword "Dual-ISO".
Any discoveries recently that could lead to dual ISO working in 50D and 5D2 video?
QuoteDownload cr2hdr-20bit for Mac only with last commit 0eabcb0 ;)
Download here
I get the following exiftool error when using this build of cr2hdr:
Hexadecimal number > 0xffffffff non-portable at /usr/bin/lib/Image/ExifTool/Writer.pl line 2878.
Hexadecimal number > 0xffffffff non-portable at /usr/bin/lib/Image/ExifTool/Writer.pl line 2878.They open fine in ACR/LR but have no preview, so if I use the --compress option, which enables the .dng to be displayed in preview, it comes out like this-
(http://s30.postimg.org/n32y63y41/MG_1776.jpg)
@N/A try to update your version of exiftool (http://www.sno.phy.queensu.ca/~phil/exiftool/) and let me know !
Quote from: kyrobb on September 27, 2014, 07:37:22 PM
Any discoveries recently that could lead to dual ISO working in 50D and 5D2 video?
Nope.
Quote from: N/A on September 29, 2014, 05:13:43 PM
They open fine in ACR/LR but have no preview, so if I use the --compress option, which enables the .dng to be displayed in preview, it comes out like this-
If you mean OS X Preview, there's a known issue (http://www.magiclantern.fm/forum/index.php?topic=6218.msg126051#msg126051) where Preview apparently ignores the EXIF black level if it recognizes the camera model (and presumably uses some default black level), which can lead to color casts like you show.
Yup, exiftool 9.72. Looks the same in Xee, not sure about any other image viewers. Figured it was a black level issue, if its a known bug then that explains it.
hello a1ex,
i have a problem with a specific shot, perhaps you can take a quick look at it.
the shot is made with a canon 5dmk3 and there were no extreme settings applied.
after conversion the file gets some pinkish cast where the wood window fragments nearly touch each other.
here is the link to the dng: https://copy.com/3k6ep8dx7D1B3aNm
one original, converted with mlvfs, and the other one is converted with the lightroom plugin.
can i save that shot and get rid of that pinkish edges?
thank you
That's aliasing. I'm not sure what I can do about it, maybe identify high-contrast areas and desaturate them?
Hello A1ex!
Thank you, that sounds great, cause the problematic area is pinkish :)
another interesting observation is, that after conversion from untouched to converted dual iso dngs with lr plugin - in a frame pattern of A B A B A B A B.... after bringin up the shadows my video has
A.. greenish blacks and
B.. magenta blacks
in Davinci Resolve,
when i am at home i will post 2 frames where you can see that.?
is that a side effect of a too dark exposure?
thx swinxx
See related discussion (http://www.magiclantern.fm/forum/index.php?topic=12796.msg130197#msg130197). The white-balance calculation in cr2hdr is apparently being affected by the fact that which lines are at which ISO changes from frame to frame in dual-ISO video (in crop mode?), which is probably changing stuff like black levels.
There is another effect in video mode: at certain FPS settings, one frame ends up as dark-dark-bright-bright, the next one is dark-bright-bright-dark and so on.
And since the RAW video does not contain optical black data (for speed reasons), the black level is guessed. The guess may be different in the two cases (which may cause green/magenta difference in shadows).
The DNGs will tell exactly what's going on.
Useful test case?
https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/_MG_5689.CR2
(https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/_MG_5689.jpg)
Here is Senzazn two DNGs with whitebalance shifts.
From here
http://www.magiclantern.fm/forum/index.php?topic=12796.msg130197#msg130197
"Camera is a 5D Mark III. I used the CR2HDR20BIT algo through Magic Lantern RAW Video Converter 1.9.2 to process the MLV files to Dual Iso DNGs. Then when I opened up the DNGs in Lightroom, despite syncing them all to have the same white balance, each alternating DNG frame has a slightly different white balance. Here are two sample DNGs.
https://www.dropbox.com/s/kimafdub1vm9pm0/MLV_A002_271841_C00016.000000.dng?dl=0
https://www.dropbox.com/s/c4ac4t5qr5usy1w/MLV_A002_271841_C00016.000001.dng?dl=0"
Audionut > Nice! ;)
Edit: the CR2 looks... Strange (?); 19.1Mo, with no header informations.
Not sure if this is the right place to ask, but since cr2hdr belongs to the dual_iso module and I found this thread later, maybe I try and ask if someone can help me with my problem describe here?
http://www.magiclantern.fm/forum/index.php?topic=13548.0
Quote from: SpcCb on October 08, 2014, 07:00:42 PM
Edit: the CR2 looks... Strange (?); 19.1Mo, with no header informations.
I guess Canons lossless compression works well on these images. A quick scan through my collection sees another dual_iso image @ 100/1600: 28.1Mo.
Here is an non dual_iso image from the intervalometer sequence, five shots before the above: https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/_MG_5684.CR2
Not sure what you mean on the header info, all the exif data looks intact here, minus focal length/aperture (telescope).
An issue I found, when using the intervalometer, enabling/disabling dual_iso would not work for the next captured image, it was the next image after, that dual_iso became enabled/disabled.
Fault on my side; CRC error during download... (I use a crap VPN)
Re-download and it's OK. ;)
BTW I'm surprise of the size of the 6D file; with 5D2 'Moon' Dual ISO are very heavy, something like 30~40Mo, more than an regular CR2.
Maybe a question of noise... Or optical sampling/focus.
Resolution differences aside, the 6D is one of the best Canon performers regarding noise: http://www.clarkvision.com/articles/evaluation-canon-6d/index.html
I downgraded from a 5D3 to 6D for the noise performance, as I'm looking to move more into astro photography.
I also use ADTG tweaks :)
I used an NexStar 6 SE (cheap, all things considered), haven't made an equatorial mount yet, and missed prime focus (seeing conditions were poor), so there were no fine details to boost filesize.
The dual iso have something which is similar to interlaced video, in the interlaced video lines alternates in different fields with time difference, in dual iso lines alternate with light difference but no time difference. No matter this comparison, both interlaced video and dual iso video suffer from aliasing after separating the two images. The link below shows an excellent result for deinterlacing interlaced video, maybe this solution or a modified version can do a good job for dual iso:
http://forum.doom9.org/showthread.php?t=156028
Quote... 5dmkiii ... ... developments ... coming forward on this platform.
<- I'm glad this has been clearly stated!
I think ML should put this type of information very prominently on the home page - e.g. "Want to be on the edge - get a 5D3!!!" kind of thing. And do not be embarrassed to monetise the promotion - it truly would appear to be the best platform for advanced techniques, right?
I also believe there should be a user-friendly website built to help the simple photographer navigate and select the features that may be relevant to their method and their gear. I can see a clear potential for monetisation there too - e.g. a site that prompts to create and install a personalised custom build tailored to a registered user's profile. I personally only want a few useful features and rather not get confused by the choice between various semi-functional builds vs the latest "experimental bells and whistles" version irrelevant to my camera model and my requirements. E.g. I'd rather pay to get a simplified yet customised build that does no more than what I want and *not* have to study any esoteric complexities.
I believe that the majority of less technically minded photogs will be prepared to pay the price of a memory card to subscribe to such a personalised service as they may be intimidated by the complexity or lack of features in the full builds - the 2 year old one that's currently presented as the safe one vs the latest more complicated one. Think about it - a services that charges more for fewer but relevant features rather than trying to be an "All things for all men at all times" type of thing!
Hi a1ex,
I have been using ML for more than a year and am using dual ISO for roughly half of the photo shots.
I would like to suggest that you can add a flag to cr2hdr to force export DNG even it is not an interlaced ISO CR2 file. It will help a lot for people like me who is not using dual ISO all the time to standardise the workflow (working with DNG alone). I see your code that there is a debug routine to output the file untouched which is actually what it will do.
Identical settings.
ADTG[888x] 32x
Looks to me like the dual_iso exposure has the most accurate feedback. I guess this is because you have accurate white level from the higher ISO?
(https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/Canon.jpg) (https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/ML.jpg)
https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/_MG_6014.CR2
https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/_MG_6015.CR2
The second one should be marked as blown out, right? It's a white level problem (not detected correctly if you decrease the gain too much).
Zebras go weird at 31x, and white level drops at 30x.
RawDigger reports the whitelevel of the non dual_iso image as 14286, and since I still have aperture gains active, saturation should be 16283 (f/1.4). So it looks to me, the second one (dual_iso) has the correct exposure feedback, and the first image (non dual_iso) is incorrect.
I never noticed this behaviour on 5D3 (different exposure feedback for dual_iso/non dual_iso), and I took tens of thousands of images on that with various gains, and dual_iso.
Possible. I should be able to debug it in QEMU (will convert the CR2 to DNG and load it there).
This is quite interesting... definitely makes sense with the fact that the dual-ISO giving correct exposure feedbacks because its actually reading it 'deeper' within both ISO's as oppose to just one.
Quote from: DeafEyeJedi on October 17, 2014, 09:14:31 AM
This is quite interesting... definitely makes sense with the fact that the dual-ISO giving correct exposure feedbacks because its actually reading it 'deeper' within both ISO's as oppose to just one.
Not sure what you mean - the zebras in the dual ISO image from Audionut are wrong (it's a bug).
I have created homebrew tap for Mac OS X users, so that it is easier to build cr2hdr with dcraw.
https://github.com/ebeigarts/homebrew-magic-lantern
Quote from: a1ex on October 17, 2014, 09:53:48 AM
Not sure what you mean - the zebras in the dual ISO image from Audionut are wrong (it's a bug).
I guess I read that wrong from Audionut's images. For some reason I thought it meant that Dual-ISO would give much closer to perfect reading on the exposures regarding zebras than off from a single ISO.
:-\
Thank you a1ex and the other contributors for providing this tool to get rid of the nasty canon read noise!
I've done some test shots with the 50D and ran into rather more trouble than expected. Here are two files that are giving me trouble. I'm using the cr2hdr.exe from the first post on Win 7 32bit.
0052.cr2 won't process without error and 0065.cr2 keeps the interlacing lines through the processing.
_UAL0052.CR2 (http://www10.zippyshare.com/v/78711201/file.html)
_UAL0065.CR2 (http://www10.zippyshare.com/v/13483318/file.html)
Both of those images process fine with the latest 20bit experimental branch.
https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/cr2hdr_20bit.exe
Is there a 20 bit cr2hdr for OSX?
Quote from: kyrobb on November 17, 2014, 02:08:46 AM
Is there a 20 bit cr2hdr for OSX?
There are few options to choose from regarding OXS:
-@danne's app cr2hdr-r
-LR5 plugin
-a1ex's binary cr2hdr (use with Terminal)
-MLVFS
Is there a cr2hdr for windows?
Best regards and Thanks
First page
Thank you for your help and advice. All the best to dev's, ... I'm not all the time in front of my comp, more behind my camera. So please understand, ...
Keep on rockin'
Can the 7D do the alternating scan line HDR with the RAW video module?
or is that only possible with the 5d mk2/3?
http://www.magiclantern.fm/forum/index.php?topic=13597.0
I need some guidance - I've been using ML on T3i with Dual ISO pretty much ever sine it was released and had really good results.
I've since upgraded to a 5D3 (Merry Christmas to myself :D) with latest nightly build and the field is a bit different in that I set the base ISO via the Canon menu, and then teh secondary one via ML, which is fine, I got that part.
What I *am* a little confused about is, how exactly to set this based on my shooting style. I have learned you can use the +/- EV settings to couple with automatic ISO mode which seems very handy. In practice on editing the files though, I am not getting near the results and dynamic range I was on the T3i shooting say 100/800 and then manually under exposing a stop to get color and clouds in the sky in addition to well lighted scenery-this is basically my intended end use.
I have the 5D3 set to -3EV and I just can't seem to duplicate my previous results, I can't bring out the highlights.
If anyone can help me fully understand the menu choices and usage of this, I'd like to help write a small guide for other poor souls like myself :)
@HondaATC
Quick response as it's Christmas :-)
I have found the 'best' workflow to be A-ETTR and Dual.
That is set Dual to what you want, eg 100/800, say.
Set your ETTR parameters, eg I put all to 0, apart from % to cover any speculars.
Comprise and focus.
In manual. my choice, set the ETTR for the scene.
Take a dual capture.
Process in LR.
Hope this helps.
Cheers
Garry
Got it Garry, Thanks for the reply, I've been able to play around with it a bit and am getting good results again finally :)
Quick question - I am seeing a bit of funky pinkish/green bending in certain areas, basically in high contrast areas where you would have some Chromatic Aberrations with a typical photo. Not at all prevalent unless pixel peeping, but wanted to mention it none the less. Could the experts have a look see and let me know whether this is typical/expected or not? 5D3, with the new 16-35 F4L lens, using 100/1600 iso setting.
http://tinypic.com/r/o05u36/8
(http://i58.tinypic.com/o05u36.jpg)
Quote from: HondaATC on December 30, 2014, 08:46:54 PM
Quick question - I am seeing a bit of funky pinkish/green bending in certain areas, basically in high contrast areas where you would have some Chromatic Aberrations with a typical photo. Not at all prevalent unless pixel peeping, but wanted to mention it none the less. Could the experts have a look see and let me know whether this is typical/expected or not? 5D3, with the new 16-35 F4L lens, using 100/1600 iso setting.
Throw it out the window it's broken and you don't know how to read first posts and basically any post in this thread.
I only shot about 10,000 with my T3i and never had this issue, hence the question.
Quote from: HondaATC on December 30, 2014, 09:42:54 PM
I only shot about 10,000 with my T3i and never had this issue, hence the question.
Must be broken. Should throw it out or stop giving it so much LSD :)
Okay then. For anyone else, I ran a Chromatic Aberration removal on the photo in LR after playing with the sliders a bit and it went away entirely, and also on another photo I had in the back yard of some white fence rail. Its not like it had that effect on every one of them in a consistent manner like typical CA, so thats what I was asking.
The T3i had some aliasing jaggies and moire patterns in shingles from time to time which I am accustomed to, but never produced anything quite like this with the color banding.
Which sliders did you play with for CA removal?
I don't have the RAW files here at home to do a direct comparison, but I'll share what I did.
I went into develop mode, (LR 5.x), Down to lens correction. I applied the normal Lens distortion correction process, and then started playing around with these, and wound up with pretty decent results somewhere in this neighborhood (I went from memory to show here, ymmv).
http://tinypic.com/r/m81ov8/8
(http://i58.tinypic.com/m81ov8.png)
I do not have the RAW or completed photos I edited here at home, perhaps later today I'll be able to post a before and after.
Ok, thanks.
Thank you very much ! I use ML in China , it's so nice .
but my english is so poor ,hard to use it.
I wanted to give an update on my experience and success in removing the color banding in patterns on photos. I am in real-estate and therefore am around a lot of brick and patterns when using dual ISO. Attached is a screenshot with a before and after, when using some of the color correction and CA tools in Lightroom. The photo is at a 100% crop in these screen caps, a tiny bit of the green color banding in the brick pattern is still visible, but when you shrink it down to a usable size, you'll never notice it.
Hope this helps and is useful for others.
http://wgolightly.smugmug.com/Misc/i-tG8mnXH/0/O/DualISOCorrections.jpg
(http://wgolightly.smugmug.com/Misc/i-tG8mnXH/0/M/DualISOCorrections-M.jpg) (http://wgolightly.smugmug.com/Misc/i-tG8mnXH/A)
Can you upload that DNG?
Try giving this a whirl;
https://www.dropbox.com/s/db8w1nys24cb5wk/_CPB0798.dng?dl=0
Note: I am processing with the non 20bit version of CR2HDR.
Ah, my apologies. I'll slap myself, can you upload the original CR2.
I believe I may have already formatted my camera - I will check and see a bit later on.
Still had it; https://www.dropbox.com/s/kf0076boqzecpk5/_CPB0798.CR2?dl=0
Hi, i read dual-iso.doc, and found this passage:
QuoteUnfortunately, most other Canon cameras follow a different pattern for the CMOS #0 register,
without any obviously duplicate fields. As we know from Canon specs, most cameras have
a 4-channel sensor readout, except for 5D Mark III and 7D which have a 8-channel readout.
So this dual-iso trick, which originally worked only on 5dmk3 and 7d because of 8bit, is now working on all cameras, how you did achieve this?
p.s. in usual, not magic lantern, camera usage is there any advantage of 8bit readout?
Where did you get "8bit" from? That statement is talking about having 4 or 8 channels for readout. Which it turns out has nothing to do with how this works anyway (it was just suspected at the time this PDF was written).
Initially the 5D3 and 7D were the only ones with obviously duplicated register values in hex for CMOS ISO. For example:
0x00 => ISO 100
0x11 => ISO 200
0x22 => ISO 400
0x33 => ISO 800
0x44 => ISO 1600
and so forth.
We get dual ISO by setting those registers to something like this:
0x03 => ISO 100/800
0x14 => ISO 200/1600
0x42 => ISO 1600/400
The other cameras didn't have obviously duplicated values in hex like this, but if you look at them in binary, the patterns become apparent (the registers didn't repeat in groups of 4 bits, so it wasn't apparent from just looking at the values in hex, they might have instead repeated in groups of 3 or 5 or whatever, or there might have been extra bits as padding or other flags).
For example, here's what the 60D looks like:
0x00 => ISO 100
0x24 => ISO 200
0x48 => ISO 400
0x6c => ISO 800
0x90 => ISO 1600
I'll leave it as an exercise for you to figure out the repeating pattern (hint: write it out in binary).
Quote from: dmilligan on January 13, 2015, 02:02:23 PM
Initially the 5D3 and 7D were the only ones with obviously duplicated register values in hex for CMOS ISO.
Was just looking at 7D exif, is this why it has listed-
AutoISO: 100
BaseISO: 1600:edit: Erm nevermind, 600D has that too. What makes the 7d and 5m3 similar? Or is this a moot point? Just trying to understand code a little better.
@dmilligan: OK your hint was good! :)
0x00 => 0000 0000 => 000 000 00 (ISO 100)
0x24 => 0010 0100 => 001 001 00 (ISO 200)
0x48 => 0100 1000 => 010 010 00 (ISO 400)
0x6c => 0110 1100 => 011 011 00 (ISO 800)
0x90 => 1001 0000 => 100 100 00 (ISO 1600)
Thx for info, now i know how my 6d works :)
I'm looking magic lantern with dual iso function for my 5D MarkII.But i can't find the link.
Anyone knows?
Top of page -> Downloads
Have some dual ISO files with horizontal stripe artifacts as result of processing with cr2hdr-20bit on default settings. Enabling --no-stripe-fix option ... eliminates artifacts. Can't find info on --no-stripe-fix option - is it worth to enable it by default?
Here is original CR2 with this problem:
https://www.dropbox.com/s/bdtgmmpzdml54kn/D27A7666.CR2?dl=0
Very good catch.
If you don't have issues with this option enabled, just keep it on. It's supposed to fix exactly this type of artifacts, but this case is clearly a bug.
After inspecting a lot more images from that shot I find that this issue is bound to ISO and underexposure of image: higher ISO and darker image - stronger artifacts. Auto ISO mode was enabled - so I even not sure, what ISO saved in EXIF- upper or lower. I assume lower. On ISO up to 10000 it can be fixed via --no-stripe-fix. Higher than 10000 - stripes persist despite the option. But... hey, there is no or very little headroom to boost +3EV over ISO 10000. So, actually it's not a bug - probably, just sensor limits reached.
If you can select a few more samples with this issue, that would be helpful.
Collection of most representative files to illustrate the stripes issue:
https://www.dropbox.com/s/bdtgmmpzdml54kn/D27A7666.CR2?dl=0
https://www.dropbox.com/s/xxwmvs7opub9hnt/D27A7705.CR2?dl=0
https://www.dropbox.com/s/66vvrzmjejl8g9t/D27A7704.CR2?dl=0
https://www.dropbox.com/s/qg9oos89c935vvs/D27A7695.CR2?dl=0
https://www.dropbox.com/s/sodljwjf9vlufmm/D27A7691.CR2?dl=0
https://www.dropbox.com/s/857n3dmcg7jh89n/D27A7694.CR2?dl=0
https://www.dropbox.com/s/2ygtnvqhfn2bquc/D27A7690.CR2?dl=0
https://www.dropbox.com/s/k2ug7pwayxeb1gz/D27A7699.CR2?dl=0
Gracias. Please keep them for a while, since I won't be able to download all of them right now.
Quote from: mothaibaphoto on February 07, 2015, 02:47:24 PM
Collection of most representative files to illustrate the stripes issue:
https://www.dropbox.com/s/bdtgmmpzdml54kn/D27A7666.CR2?dl=0
https://www.dropbox.com/s/xxwmvs7opub9hnt/D27A7705.CR2?dl=0
https://www.dropbox.com/s/66vvrzmjejl8g9t/D27A7704.CR2?dl=0
https://www.dropbox.com/s/qg9oos89c935vvs/D27A7695.CR2?dl=0
https://www.dropbox.com/s/sodljwjf9vlufmm/D27A7691.CR2?dl=0
https://www.dropbox.com/s/857n3dmcg7jh89n/D27A7694.CR2?dl=0
https://www.dropbox.com/s/2ygtnvqhfn2bquc/D27A7690.CR2?dl=0
https://www.dropbox.com/s/k2ug7pwayxeb1gz/D27A7699.CR2?dl=0
i think ur using some extreme value for recovery ISO in ur method
correct me if im wrong
ru setting ur base ISO as 10000 n recovery ISO at what value?
in ML at the moment (Jan 29 NBuild) recovery ISO is maximum upto 6400
so if u set ur base ISO higher than or equal to recovery ISO (6400 in ur case) i am sure u will get such bands. So make sure ur base ISO (which u select from camera menu) is not more than (or equal to) ur recovery ISO
ie 100-6400, 1600-6400, but never 6400-6400 or 10000-6400
QuoteI get the following exiftool error when using this build of cr2hdr:
Hexadecimal number > 0xffffffff non-portable at /usr/bin/lib/Image/ExifTool/Writer.pl line 2878.
Hexadecimal number > 0xffffffff non-portable at /usr/bin/lib/Image/ExifTool/Writer.pl line 2878.
They open fine in ACR/LR but have no preview, so if I use the --compress option, which enables the .dng to be displayed in preview, it comes out like this-
(http://s30.postimg.org/n32y63y41/MG_1776.jpg)
I fixed the exiftool error by placing cr2hdr into my usr/bin folder. Still getting the pink pictures in Preview when processing 7d pictures with --compress, but interestingly my 600D dual iso's have a normal preview.
@babarasghar:
Recovery ISO was +3EV, base iSO - auto. I know, how to avoid this issue, I just want to share some strange things i discovered.
@mothaibaphoto thanks for ur reply. Im still learning this dual iso thingy. Have u observed any drop in sharpness?
@babarasghar:
Drop in sharpness due to what?
Dual ISO vs ordinal image - no, I don't see any lost details or moire on properly exposed low - ISO images. When I shoot on low ISO with Dual - I get wider dynamic range at no expense virtually. When I shoot on high ISO with Dual - I get artifacts complementary to noise at some point.
@mothaibaphoto thanks for ur response. Usually people talk here in highly complex "Developers" language so i thank u for talking is a simple manner. As i said i just explored these added features of ML despite using it on my 60D for last 2 years for just intervalometer, but on 6D im trying to understand these new features. Thats why u will see me in almost every thread on this forum trying to understand basics. This crop mode silent pictures i have still not understood.
As extraordinary as it gets... has anyone experience more DR if you were to do it by the eye with reading the meters (via viewvinder shows 3 stops UE) -- basically a "manual ETTR" type of mode because often times whenever I have ETTR (any types) enabled, it doesn't come out consistently IMO.
Dual-ISO would change to whatever the camera thinks its better and half of the time it goes to (ISO800/ISO800 or whatever the same ISO it may be) even so with the Dual-ISO 'UNLINKED' within Dual-ISO menu. Not sure if it's because I'm inaccurately doing something?
It's literally like I prefer to drive a stick shift than automatic in vehicles.
Hopefully this is a Human Error on my part.
I can be reprehensible at times... ha!
If not, doing it manually can be fun!
I should also note that Non Dual-ISO shooting works well with ETTR, as usual.
If you have Link to Dual ISO (http://www.magiclantern.fm/forum/index.php?topic=5693.0) set to off, make sure you also turn dual ISO off.
If it is choosing ISO 800/800, you don't have enough light. Either insufficient scene lighting, or shutter set incorrectly.
If you can expose more accurately then the ETTR module, you don't have it setup correctly.
Since this is more ETTR, then dual ISO, please continue discussion in the thread I linked above.
Cheers.
Got it @Audionut -- Actually saw that thread and at first I though it was more Dual-ISO related but after your pointers I'll try reading the entire thread thoroughly again and will stick to that thread instead.
Thanks!
Hi,
Maybe someone can take the time to clarify something for me?! I thought I had it all figured out, but for some reason, I am totally lost now. :-[ :-[ :-[
I thought that in order to get those nice HDR picture as posted in post #1, all you have to do is, shoot them in your camera, take the RAW file, drag it onto cr2hdr.exe and the end result will be a killer looking HDR .dng file??? I guess I am wrong, right?! The way it looks to me is, that I am THEN supposed to take the .dng file and import it to a Photo editing software like Lightroom or Photoshop and THEN create the final HDR image?! Is that correct?!
The other thing I am not getting is, what is the cr2hdr Lightroom plugin for?!
SO all in all, I am not sure what software does what and if any other steps have to be taken, in order to get those nice HDR images?!
thanks,
G
p.s. YES, I read this thread, not all of it, googled and tried what not, but I cannot figure it out! I am lost!
Neavissa
You might wish to reread what Dual ISO is all about.
Put simply, dual ISO achieves a two image bracket but in a single frame.
The two images are captured at the same time and interlaced in the image, hence the cr2hdr process is taking the interlaced data and creating a new digital negative, ie a DNG.
The interlaced images are separated in exposure space by different ISOs, ie the two ISO, interlaced brackets, have the same aperture and shutter speed.
The DNG negative will definately need post processing, as any RAW image will.
The advantage is that the dual ISO will have 2-3 stops of additional dynamic range.
For optimum capture use ETTR and dual together, ie push the tonal data to the right, without overexposing, and 'fill in' the shadows with dual ISO.
Quote from: garry23 on February 19, 2015, 12:26:54 AM
Neavissa
You might wish to reread what Dual ISO is all about.
Put simply, dual ISO achieves a two image bracket but in a single frame.
The two images are captured at the same time and interlaced in the image, hence the cr2hdr process is taking the interlaced data and creating a new digital negative, ie a DNG.
The interlaced images are separated in exposure space by different ISOs, ie the two ISO, interlaced brackets, have the same aperture and shutter speed.
The DNG negative will definately need post processing, as any RAW image will.
The advantage is that the dual ISO will have 2-3 stops of additional dynamic range.
Thanks for the reply. I got all of this so far, but I am still not sure what the difference is between using cr2hdr.exe, or the Lightroom plugin?! It seems like both "methods" achieve the same thing, right?!
QuoteFor optimum capture use ETTR and dual together, ie push the tonal data to the right, without overexposing, and 'fill in' the shadows with dual ISO.
I am going to have to look up how to use ETTR and dual together...
thanks,
G
Dual-ISO LR plug-in is in fact using cr2hdr/cr2hdr20.
It's kind of a wrapper with a GUI + integration in LR.
Neavissa
Walter has clarified the .exe use.
As for ETTR there are two basic ways of integrating it with dual.
You can get ETTR to automatically call it when it thinks it is required, ie when certain criteria are met.
Personally, I prefer to make this decision myself. For instance, I use ETTR to set the base exposure, at ISO 100, and take a dual with this ETTR exposure setting.
A nice feature is that you can tell duel to only take a dual every other shot, hence take two images and one will be dual and one will be non-dual.
I reach the limit on the command line length while try to run cr2hdr-20bit.exe --same-levels - "The filename or extension is too long". Please, folder, or file mask option.
Open commandline
subst X: C:\my_very_long_and_therefore_limited_path_length_which_will_cause_trouble_with_this_application_but_there_is_a_workaround
I have dual ISO video with several thousand frames in each MLV file. I want to process every shot with same black levels, or whatever else --same-levels do. I can't shorten pathnames anymore.
[ ] You have tested what I wrote.
You can assign any directory (=path name) to any drive letter not yet occupied.
You misunderstand the problem. MLV dump produce files with ten symbols at least. Like this one - 000000.dng. I need to pass several thousands filenames in one command line. Doesn't matter, how short the pathname to folder with all these files - the command line gets too long due to files quantity, not the redundantly long folder name i put them in.
When i deal with ffmpeg, or, let say adobe DNG converter, I can pass folder and/or file mask as parameter. I ask the same thing for HDR tool. There is NO easy workarounds.
you can accomplish this with xargs and pipes on *nix operating systems:
find /path/to/dir/with/images -name '*.dng' | xargs cr2hdr --same-levels
you can apply the same whitelevel to all DNGs with exiftool in the same way (this is all that --same-levels does)
find /path/to/dir/with/images -name '*.dng' | xargs exiftool "-WhiteLevel+=50000" -overwrite_original
there's probably a similar way to do this on windows, but I'm no windows shell expert
Wrong answer, because --same-levels only works if all the filenames are passed in a single command line. But xargs will pass each .dng to a separate instance of cr2hdr, which effectively cancels the effect of --same-levels.
And on *nix, command line length is less likely to be an issue anyway. On my system is 2 MB, and on Mac it seems to be around 256K (http://serverfault.com/questions/69430/what-is-the-maximum-length-of-a-command-line-in-mac-os-x).
For Windows, one has to implement a FindFirst/FindNext loop to handle wildcards. On *nix, these are expanded by the shell.
For the exiftool method, no workarounds should be required. Note the command is a little different:
http://www.magiclantern.fm/forum/index.php?topic=7139.msg114254
Find args is the best. What got me starting my unix playing thanks to dmilligan.
About same levels.
If you run command below it will take the whitelevel from your first file and apply to all the rest of your files. If you delete the line -overwrite original it will keep a copy of your file. I had some problems on external drive with exiftool.
exiftool -tagsfromfile *.DNG -exif:whitelevel -overwrite_original
Simply write cd drag your folder here exiftool -tagsfromfile *.DNG -exif:whitelevel -overwrite_original hit enter.
This mac app workflow will also correct same levels when run in folders with dng files.
http://www.magiclantern.fm/forum/index.php?topic=13512.0
*update. Sorry, didn,t notice you were looking for a windows solution.
Thank you all.
Apologies I didn't find this was explained in details before.
@Danne: Use the /D switch to change current drive in addition to changing current
directory for a drive :
cd /D [drag your external drive folder here]
I'm using Dual ISO on my EOS M quite successfully for a while and *really* like it. However, during some shots over the weekend I encountered a banding artifact after using cr2hdr.exe and cr2hdr20bit.exe (used several different versions, the result always was the same)
Here's a crop of the image:
https://www.dropbox.com/s/6rfiwz5gcawgcua/banding.jpg (https://www.dropbox.com/s/6rfiwz5gcawgcua/banding.jpg)
and here's the according debug black frame I saved.
https://www.dropbox.com/s/7edcyuc0yawq0wc/black.jpg (https://www.dropbox.com/s/7edcyuc0yawq0wc/black.jpg)
There's a --no-stripe-fix option which sounds like maybe this is already known? But unfortunately it seems as if it doesn't work.
Could someone give me a hint where to find the source code of cr2hdr?? Is this one the newest one? (bitbucket is not working right now, so I cannot open the link):
https://bitbucket.org/hudson/magic-lantern/branch/cr2hdr-20bit
I've some background in that field and would like to contribute if I'm able to get the stuff compiled.
Any suggestions? I actually traveled to the Anza Borrego desert staying in the desert during the night listening to coyotes and rattlesnakes to capture some night shots and would really like to rescue those pictures....
Can you upload the CR2?
Thanks for the quick reply a1ex. This one:
https://www.dropbox.com/s/yqpstzr1k30kxoh/_MG_2113.CR2?dl=0
is a similar shot with the same behavior after converting to DNG
New to ML. I have a question. After I export from LR, then what?
Post process.
okay I figured it out. I could not find the dng file. I was thinking it would automatically re import into LR.
Quote from: ganselm on February 23, 2015, 05:23:13 PM
[...] I encountered a banding artifact after using cr2hdr.exe and cr2hdr20bit.exe [...]
As you have guessed, the issue appears during the black level correction step (black_subtract, most likely the first half). It has nothing to do with --no-stripe-fix.
The OB areas appear correct though, so I'm not yet sure what causes it.Edit: found the problem - a hot pixel in the top optical black area! Solution: use median instead of averaging.
The source code link from your post is correct.
Wow a1ex, thanks for your quick fixing.
I'm currently travelling and will try to build the stuff when I'm back. Would you be able to build a Windows executable for me beforehand or do you know someone who has the stuff ready for compilation on a windows machine?
Thanks again!
I didn't fix it yet, only found the problem and suggested a fix.
a1ex committed d397c38 5 hours ago
but now you did =)
Updated cr2hdr-20bit (first post).
Fixes: this (http://www.magiclantern.fm/forum/index.php?topic=7139.msg139547#msg139547), these (http://www.magiclantern.fm/forum/index.php?topic=13212.msg140437#msg140437), this (http://www.magiclantern.fm/forum/index.php?topic=14613), this (http://www.magiclantern.fm/forum/index.php?topic=7139.msg140866#msg140866) and a few other minor things (https://bitbucket.org/hudson/magic-lantern/commits/branch/cr2hdr-20bit).
Thanks @a1ex for the kind update!
Quote from: a1ex on March 02, 2015, 12:32:02 AM
The second one should be marked as blown out, right? It's a white level problem (not detected correctly if you decrease the gain too much).
Noticed this issue today without any ADTG changes.
Is this something I might be able to diagnose with a little nudge in the right direction?
If you upload a CR2, I'll be able to load it in QEMU and see what happens. Here we need a pretty good heuristic, because once the gain drops too much, white is no longer clipping to a constant value - there's noise instead. The white level should be right below this noise in the clipped areas.
Something like this:
(http://a1ex.magiclantern.fm/bleeding-edge/apertus/response-curve/curve-gain1-20ms-40ms.png)
Our heuristic should figure out the white level from a single image (from histogram data), not from a nice graph like this. Not exactly an easy problem.
https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/_MG_2788.CR2
(https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/ISO100.png)
https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/_MG_2789.CR2
(https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/dual_ISO.png)
Just to confirm, no ADTG changes (module not even loaded). Base ISO 100 shot above, and then dual ISO image below (no other changes).
On closer inspection, there are some subtle changes in the histogram data also.
Great thanks for fixing this "Hot" (cold, actually) pixels issue. I did some pixel peeping and did not find any IQ degradation. Really impressive done, bravo !!!
Original:
(https://farm9.staticflickr.com/8612/16719563931_bf995548b0_o.jpg) (https://flic.kr/p/rts8RD)
Fixed:
(https://farm9.staticflickr.com/8586/16100880503_3ed37d67a3_o.jpg) (https://flic.kr/p/qwMdPZ)
Gd job on fixing the Hot pixels.
I have a stupid question though, I don't seem to be able to subscribe to this discussion, is there a way to do it ? or i just don't have the permissions?
That pixel correction handling is mind blowing.
Do you usually use ETTR with the dualISO?
Quote from: keepersdungeon on March 05, 2015, 12:37:59 PM
Do you usually use ETTR with the dualISO?
If you ask me, than answer is no. Why - described here (http://www.magiclantern.fm/forum/index.php?topic=12523.msg136363#msg136363)
Alright, thank you!
New cr2hdr compiled for Mac
https://www.dropbox.com/s/0yqh0baqur4wlcf/cr2hdr.zip?dl=0
Thanks for sharing @ N/A!
Anything to help.
Thx for hotpixel reduction .....
50sec long exposure ....
https://www.facebook.com/122338811269922/photos/a.122339334603203.1073741826.122338811269922/281093542061114/?type=1&theater (https://www.facebook.com/122338811269922/photos/a.122339334603203.1073741826.122338811269922/281093542061114/?type=1&theater)
I got confused a bit with all the posts so I apologize in advance for my stupid question. But is the hot pixel fix due to cr2hdr 20bit update or is the dualISO module has been updated?
Chris_1975, lovely picture.
Hi!
There,s one thing puzzling me with dual iso RAW and MLV. I compared the formats recording with a recent build, (about two weeks old, don,t have my camera here)
WHen shooting RAW all is fine giving me a file a little darker as expected. WHen shooting with exact same settings with MLV the file is brighter after postprocessing. It,s like it,s inverted? Am I doing an obvious mistake here?
examples shot in 100/1600 3x crop mode. Same result when shooting non crop mode. Will upload RAW/MLV examples if needed. (Short ones).
Converted with latest cr2hdr20bit build Last update: cc4340b on 2015-03-01 23:16:03 UTC by a1ex:.
The same goes for the former build. Shot with a 5d mark 3, firmware 1.1.3
DNG files
https://drive.google.com/file/d/0B4tCJMlOYfirYWl6WmF2ME9laWM/view?usp=sharing
3x crop RAW 100/1600
(http://s1.postimg.org/724ezrl97/Screen_Shot_2015_03_11_at_15_10_41.png) (http://postimg.org/image/724ezrl97/)
3x crop MLV 100/1600
(http://s1.postimg.org/kyh8pe6vv/Screen_Shot_2015_03_11_at_15_11_09.png) (http://postimg.org/image/kyh8pe6vv/)
Can you upload the source DNGs created by raw2dng / mlv_dump?
Dual_iso dng files, unprocessed from raw2dng and mlv_dump
https://drive.google.com/file/d/0B4tCJMlOYfirdXdmaktRUG1NRXM/view?usp=sharing
dual_iso RAW original(75mb)
https://drive.google.com/file/d/0B4tCJMlOYfirV0ktd2w3c0UtVjg/view?usp=sharing
dual_iso MLV originals (161mb)
https://drive.google.com/file/d/0B4tCJMlOYfirdWhMWXNJRWxNMW8/view?usp=sharing
Thanks
/D
I've got two DNGs with similar brightness.
Can you paste the conversion log?
It seems the problem is coming when I,m inside folders and letting c2hdr-r working in different levels. Therefore the conversion process works from parent folder. I have to check further. A little busy atm.
*Actually looks like it is only the first file that is giving strange results.
Here is a log from dual iso conversion. Also noticed some strange intermediate issue before it turns into a transcoded DNG (picture).
(http://s24.postimg.org/l32bco2sx/Screen_Shot_2015_03_11_at_17_25_22.jpg) (http://postimg.org/image/l32bco2sx/)
Log from faulty conversion dng files
Last login: Wed Mar 11 17:23:10 on ttys000
itgusercbook241:~ itguser$ /Users/itguser/Desktop/testMLV\:RAW/cr2hdr-r\ 5.3k_s.command ; exit;
ORIGINALS
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: 'M11-1138.MLV'
- Convert to DNG frames
- Output into 'M11-1138_1_2015-03-11_0001_C0000_'
File M11-1138.MLV opened
File M11-1138.M00 not existing.
Processing...
Vertical stripes correction:
1.000 1.000 1.000 1.002 0.999 0.996 0.997 0.999
Reached end of chunk 1/1 after 56 blocks
Processed 39 video frames
Done
mv: rename M11-1138_1_2015-03-11_0001_C0000_.wav to M11-1138_1_2015-03-11_0001_C0000.wav: No such file or directory
mv: rename M11-1138.M* to ../ORIGINALS/M11-1138.M*: No such file or directory
cr2hdr: a post processing tool for Dual ISO images
Last update: cc4340b on 2015-03-01 23:16:03 UTC by a1ex:
cr2hdr: fix parsing of --wb options
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
Input file : ./M11-1138_1_2015-03-11_0001_C0000_000000.dng
Camera : Canon EOS 5D Mark III
Camera model : Canon EOS 5D Mark III
Full size : 2048 x 1152
Active area : 2048 x 1152
Black borders : N/A
ISO pattern : BddB RGGB
White levels : 10000 14882
Noise levels : 8.00 8.00 8.00 8.00 (14-bit)
ISO difference : 3.89 EV (1487)
Black delta : 5.26
Dynamic range : 9.96 (+) 10.65 => 14.54 EV (in theory)
Looking for hot/cold pixels...
Hot pixels : 34
AMaZE interpolation ...
Amaze took 0.30 s
Edge-directed interpolation...
Semi-overexposed: 55.57%
Deep shadows : 4.55%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap : 3.1 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 8.00 (20-bit), ideally 8.00
Dynamic range : 15.96 EV (cooked)
AsShotNeutral : 0.53 1 0.59, 4793K/g=1.05 (gray max)
Output file : ./M11-1138_1_2015-03-11_0001_C0000_000000.DNG
./M11-1138_1_2015-03-11_0001_C0000_000000.DNG: copying EXIF from ./M11-1138_1_2015-03-11_0001_C0000_000000.dn6
grep: tee: No such file or directory
1 image files updated
find: *.dng: No such file or directory
cr2hdr: a post processing tool for Dual ISO images
Last update: cc4340b on 2015-03-01 23:16:03 UTC by a1ex:
cr2hdr: fix parsing of --wb options
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
Input file : .
cr2hdr: a post processing tool for Dual ISO images
Last update: cc4340b on 2015-03-01 23:16:03 UTC by a1ex:
cr2hdr: fix parsing of --wb options
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
Input file : ./M11-1138_1_2015-03-11_0001_C0000_000000.DNG
Cannot decode file .
cr2hdr: a post processing tool for Dual ISO images
Last update: cc4340b on 2015-03-01 23:16:03 UTC by a1ex:
cr2hdr: fix parsing of --wb options
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
Input file : ./M11-1138_1_2015-03-11_0001_C0000_000001.dng
cr2hdr: a post processing tool for Dual ISO images
Last update: cc4340b on 2015-03-01 23:16:03 UTC by a1ex:
cr2hdr: fix parsing of --wb options
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
Input file : ./M11-1138_1_2015-03-11_0001_C0000_000002.dng
Camera : Canon EOS 5D Mark III
Camera model : Canon EOS 5D Mark III
Full size : 2048 x 1152
Active area : 2048 x 1152
Camera : Canon EOS 5D Mark III
Camera model : Canon EOS 5D Mark III
Full size : 2048 x 1152
Active area : 2048 x 1152
Camera : Canon EOS 5D Mark III
Camera model : Canon EOS 5D Mark III
Full size : 2048 x 1152
Active area : 2048 x 1152
Black borders : N/A
Black borders : N/A
ISO pattern : BddB RGGB
ISO pattern : BddB RGGB
White levels : 10000 14882
White levels : 10000 14882
Doesn't look like interlaced ISO
cr2hdr: a post processing tool for Dual ISO images
Could be exiftool related. I apply a copy of the xmp from orignal dng files in the first file which I use after conversion to get applied to the rest of the files. This information clearly must be corrupt.
Thanks for listening. I,ll check on it when I get the time.
exiftool ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_000000.dng -X > /tmp/magic_l_MLV/all.xml -overwrite_original ;
find . -name ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_000000.dng -print0 | xargs -0 /usr/bin/cr2hdr20bit | tee /tmp/magic_l_MLV/cr2hdr20bit.log ;
if grep -q "Edge-directed interpolation..." tee /tmp/magic_l_MLV/cr2hdr20bit.log
then
Files are here converted
I apply the information below from first file to the rest of the files.
exiftool -tagsfromfile /tmp/magic_l_MLV/all.xml -all:all ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_000000.DNG -exif:whitelevel -overwrite_original
Hi!
I don,t know what I,ve been up to but is cr2hdr20bit keeping the metadata information from mlv dual iso movie dng even after conversion?? Has this always been? For RAW it,s hardcoded to 25fps after conversion. When I started messing with exiftool I,m certain also MLV files where stripped from metadata?
Is white/blacklevel same for movie files as well when filming MLV?
:o Tried an older build and sure it was hardcoding to 25fps. This is new stuff!! Great! I can erase a lot of stuff from the script.
REALLY Interesting information... Thanks for pointing these out @Danne!
Quote from: Danne on March 11, 2015, 03:11:51 PM
Hi!
There,s one thing puzzling me with dual iso RAW and MLV. I compared the formats recording with a recent build, (about two weeks old, don,t have my camera here)
WHen shooting RAW all is fine giving me a file a little darker as expected. WHen shooting with exact same settings with MLV the file is brighter after postprocessing. It,s like it,s inverted? Am I doing an obvious mistake here?
examples shot in 100/1600 3x crop mode. Same result when shooting non crop mode. Will upload RAW/MLV examples if needed. (Short ones).
Converted with latest cr2hdr20bit build Last update: cc4340b on 2015-03-01 23:16:03 UTC by a1ex:.
The same goes for the former build. Shot with a 5d mark 3, firmware 1.1.3
DNG files
https://drive.google.com/file/d/0B4tCJMlOYfirYWl6WmF2ME9laWM/view?usp=sharing
3x crop RAW 100/1600
(http://s1.postimg.org/724ezrl97/Screen_Shot_2015_03_11_at_15_10_41.png) (http://postimg.org/image/724ezrl97/)
3x crop MLV 100/1600
(http://s1.postimg.org/kyh8pe6vv/Screen_Shot_2015_03_11_at_15_11_09.png) (http://postimg.org/image/kyh8pe6vv/)
Hello!
Not sure, if it is the same issue, but I ran some tests and also saw, that the MLV - in comparison to a RAW- appears brighter after conversion. It does an amazing job in the shadows, but crops out the highlights (or is this expected? [I'm a noob]).
5DMK3 Nightly.2015Mar16.5D3113
MLV ISO 200 (no sharpening or noise reduction)
(http://s2.postimg.org/kxuw42k95/Screen_Shot_2015_03_25_at_14_39_19.png)
MLV DUAL ISO 100–1600 Conversion by cr2hdr-r (no sharpening or noise reduction)
(http://s12.postimg.org/nmjrbvyxp/Screen_Shot_2015_03_25_at_14_43_23.png)
Thank's in advance.
The brightness problem was due to me copying the orignal whitelevel setting to the created dual iso file with exiftool. RAW and MLV probably are identical in terms of quality and brightness. I fixed this issue a while ago. It was only affecting dual iso movie files.
Oh okay, I just found out my fault. Apparently it was the DUAL ISO conversion of cr2hdr-r.
I just used the Lightroom plugin and voilà!
(http://s13.postimg.org/yw5v7bgw7/Screen_Shot_2015_03_25_at_15_48_16.png)
What version of cr2hdr-r were you running?
Hello Danne :)
I conversed a dual-iso .DNG two ways. Once with cr2hdr-r_5.3u_s(beta), and once with the Lightroom Plugin.
At the beginning, I overlooked, that the Lightroom settings were totally different and thought it is clipping so much, sorry for that. But in the end, in comparison of the two DNGs with the same settings there is still a difference. the cr2hdr-r appears to be 0,4 stops brighter and the Lighroom dual iso conversion seems to be slightly detailed.
Here the comparison. Especially at the "m" letter.
For some reason the Lightroom-way has more detail, but also increases chromatic aberration?
Lightroom dual-iso plugin
(http://s7.postimg.org/o5fdvzqfv/Screen_Shot_2015_03_25_at_17_38_16.png)
cr2hdr-r_5.3u_s(beta)
(http://s28.postimg.org/ngkl0kal9/Screen_Shot_2015_03_25_at_17_38_26.png)
Did you update to A1ex latest version? I run the binary on default settings. Work for most scenarios. Thanks for info.
Hi,
I'm a Magic Lantern user on my T3 and now a 70D. I've been trying out the March 14 ML and I'm getting used to it. The problem I'm having is converting CR2 dual iso files to something usable.
I use a Mac with Yosemite, Lightroom 5, Photoshop CS6. I downloaded cr2hdr, in many versions including cr2hdr-r 1.8 app and 5.3u s, and the Lightroom plugin. I haven't been able to get anything to work. The programs either don't recognize the file type, don't do anything or put out a error message.
Help, please.
David
And that error message says what? Can you upload a sample CR2?
The 70D needs a custom cr2hdr. Look in the opening post of the 70D thread.
@dmilligan: errors like "Error to Export ...Error #256", here's a CR2: https://www.dropbox.com/sh/fhlcx2pin540ksp/AACv2BFb9tPWzDUWgA-GJ_LEa?dl=0
@Audionut: that's a .exe file and I don't have Lightroom for Windows.
@Audionut: I have Parallels with Windows 7. When I double try to open the cr2hdr.exe for the 70D a black dialog box with white letters flashes open, then disappears.
Worked fine here: http://i.imgur.com/OFWCnjq.jpg
I think @Danne can build mac versions. A polite PM might net you an mac version. However, the 70D is still in alpha status, so don't expect miracles.
Wow. So cutting edge and nerdy! 8) No wonder I couldn't get it to work, it wasn't invented yet. That could also explain the problems I've been having with MLV files. I'm sure looking forward to the Mac friendly w/ Yosemite, LR 5, 70D nightly build.
Never done a PM on this forum before, either; I feel like the whale in Hitchhiker's Guide to the Galaxy.
So excited to use ML for increasing the DR through dual iso, also amazed at the ETTR and RAW video possibilities.
I'll be glad to provide feedback to support the developers. As I use ML I'll be better at making suggestions.
Thanks for everybodys' help.
Update: Thanks Danne, and others, for working with me with instructions and script to get the dual iso CR2s processed on my Mac! Onward to dual iso and RAW video!
Hi!
It seems color matrices are fully missing from 70D when transcoding from CR2 to DNG with cr2hdr20bit.
You could run this script on the files after they are converted to DNG. (Not on the CR2 files). Put the script in the same location as your DNG files and double click.
https://drive.google.com/file/d/0B4tCJMlOYfirSTNNeV9JLXBkMms/view?usp=sharing
It will use exiftool to insert the correct color matrices.
If there,s any new cr2hdr version to compile I got all help from N/A on this. Hope this helps at least temporarily.
Looking at the transcoding log it says
"Input file : ./DSP20150327-00007.CR2
No camera model found with id 0x325"
/D
For the 70D, cr2hdr needs to be compiled from nikfreaks repo.
https://bitbucket.org/nikfreak/magic-lantern/src/ac3023b990592a47c15ff3c3cdd3b85b3f320f96/modules/dual_iso/?at=70D
Cool. Thanks for pointers
Hello guys, I just installed the ML Alpha4b firmware in my 70D - and been trying to test DualISO. I am using Mac for post-processing. So I have cr2hdr.app , dcraw (exec), and exiftool (exec) to be in one directory so I can double click on cr2hdr.app - which brings up a small cr2hdr window - and drag my dualISO CR2 file into it.
The cr2dhr almost immediately spits out "sh: exiftool: command not found sh: exiftool: command not found" message (yes it printed that message twice) but it did eventually create a DNG file . I opened up the DNG file in Adobe Camera Raw and it was HORRENDOUS .
I would like to share my CR2, the generated DNG file, and the text log generated by the cr2hdr but for some reason I couldn't insert my attachment here - could anyone share how to add attachment to my post here?
Thank you
Google is your friend (http://www.google.com.au/search?hl=en-AU&ie=UTF-8&source=android-browser&q=file+sharing&gws_rd=cr&ei=Cn0YVc3fI4Od8QXFhIJo).
Hello guys - I did try to research on how to do dualISO correctly - so I turned on the dualiso.mo and my DUALISO settings to 100 (on camera)/1600 (on ML), set the aperture and camera to Manual Mode.
I was expecting that all pictures taken under DUALISO would show interlacing lines. However, I notice that the shots I took are alternating between a normal ISO image (the 1st, 3rd, 5th, 7th shot, etc) and the actual dual iso image with those interlacing lines (2nd, 4th, 6th, 8th shot,etc). Is that correct?
I am still trying to figure out how to set the correct aperture for dual ISO so the DNG wouldn't come out so horrid... your inputs will be very much appreciated!
Don't alter settings without reading fineprint!
Dual ISO settings [Q] -> Alternate frames only -> OFF
hi AudioNut and danne -
first of all thank you guys for your tireless assistance in resolving ML issues with 70D - I for one appreciate it very much (as I believe everyone else on here too).
I did try to research on how to do dualISO correctly - so I turned on the dualiso.mo and my DUALISO settings to 100 (on camera)/1600 (on ML), set the aperture and camera to Manual Mode - and take RAW image. I notice that when taking pictures in dual ISO mode, the shots are alternating between a normal ISO 100 CR2 (the 1st, 3rd, 5th, 7th shot, etc) and the actual dual iso CR2 image with interlacing lines (2nd, 4th, 6th, 8th shot,etc). Is that correct? THANK YOU WALTER - I inadvertently changed that setting - now the Alternate Frame is OFF and I see interlaced lines in every image!
I am still trying to figure out how to change my aperture settings - by the way, here is my CR2 with the interlaced lines from my 70D which has really bad noise when I converted to DNG last night:
https://drive.google.com/file/d/0BwB_poTzzVK8RElTQ2Z1R0lndU0/view?usp=sharing
Quote from: Audionut on March 30, 2015, 12:32:18 AM
Google is your friend (http://www.google.com.au/search?hl=en-AU&ie=UTF-8&source=android-browser&q=file+sharing&gws_rd=cr&ei=Cn0YVc3fI4Od8QXFhIJo).
+1,000,000,000,000
Hi A1ex! I,m starting to use the lossless compress in cr2hdr20bit after downloading adobe dng converter. It,s very good and it creates half size of both CR2 dual iso and movie files. As usual when it comes to video there is a problem with linear conversion. There is an easy solution to this if implementing the "-l" option in the adobe dng converter command. Could this be updated in coming version perhaps?
Great find @Danne!
@Danne: not sure what you mean, can you explain the issue in detail?
I tried commandline option for dng converter "-l Output linear DNG files."
http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_commandline.pdf
When adobe dng conversion is done through cr2hdr20 it seems to use the default settings for compressed files. Works well and one don,t notice differences in CR2 files but for movie sequences one gets the same resulting flicker as without -same level option in cr2hdr20bit.
I tried to set same white and black level for the adobe transcoded file but with no success. However with setting "-l" I get total linear results.
Quote from: Audionut on March 29, 2015, 08:53:51 AM
For the 70D, cr2hdr needs to be compiled from nikfreaks repo.
https://bitbucket.org/nikfreak/magic-lantern/src/ac3023b990592a47c15ff3c3cdd3b85b3f320f96/modules/dual_iso/?at=70D
Here's a compile from the 70D branch along with a dcraw compile.
https://www.dropbox.com/s/37sn9wwhkrwedk9/cr2hdr-70d.zip?dl=0
Thanks!
Hi... I USING 70D AND ML INSTALLED.
I have problem now.
I don't know how to use unix file for 70d dual iso...
(https://www.dropbox.com/s/37sn9wwhkrwedk9/cr2hdr-70d.zip?dl=0) <- this
NEED HELP!!!
Are you on a Mac? Open a terminal window, drag and drop cr2hdr into it, then drag your dual iso cr2 into the window and press enter.
just tried dual_iso on my 50D (build from yesterday - 5th April) and I have some issues post processing it ...
often I get this error (Interlacing method not supported, ISO blending didn't work):
e:\bin\cr2hdr>cr2hdr.exe in\IMG_5704.CR2
cr2hdr: a post processing tool for Dual ISO images
Last update: d0ac769 on 2014-01-23 10:13:39 UTC by alex:
cr2hdr: experimental white level routine with two separate levels ...
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
Input file : in\IMG_5704.CR2
Camera : Canon EOS 50D
Full size : 4832 x 3228
Active area : 4770 x 3177
Black borders : 62 left, 51 top
Black level : 993
ISO pattern : BdBd GBRG
Interlacing method not supported
ISO blending didn't work
other time the cr2hdr works ok but the resulting DNG file is identical to CR2 file (the interlacing dark/light lines)
all three files (IMG_5704.CR2 that won't go trough cr2hdr, IMG_5703.CR2 that converts into bad IMG_5703.DNG) available here http://elco.crsndoo.com/files/ML/ML_dual_iso_problem_50D.zip (62M archive)
EDIT: the "Experimental 20-bit version: cr2hdr-20bit.exe" for e.g. reads this IMG_5704.CR2 properly and the output DNG is looking good
@arhi: are there any files that fail in the 20-bit version?
@a1ex - not so far, 20bit version so far worked perfectly
Hi!
At first: Thanks for doing all this work for Canon (Magic Lantern) community!
I'm using a 1100D with ML. I tried DualISO over the last few days to see how it works. But I'm absolutely unable to reproduce results such as shown in the start post. I took two shots, one with DualISO (100/1600 or so) an one just with 100. Then I copied to my PC and converted Dual CR2s with cr2hdr + barracudaGUI to DNG. But in RawTherapee there's almost no visible difference between DualISO.dng and NoDualISO.cr2!?
Even after Tone mapping to the extreme, all I get is a little bit less noise in the shadows. I looks like 0,3 EV (or so) more dynamic range, not like 2,5... :/
Before convertinmg to dng, the dualiso.cr2 looks better, with good exposed shadows.
Did I make something wrong or forget something? Please help...
Post the originals.
~40 mb? Dropbox?
yes
-> Dual ISO Test (https://www.dropbox.com/sh/id6siyjtxspwuaw/AAAwvECQr8Oo5asu-H8_t0jJa?dl=0)
No great picture, I just wanted to have much contrast...
The dual ISO image has the edge in noise. Although the noise in the CR2 increases the perceived sharpness for me.
The contrast isn't so bad in this image, you can tell easily because the regular CR2 is pretty clean. If you shot this same exposure with flat textures, the difference would probably be perceived as being greater. To put dual ISO through it's paces, find something with even greater contrast (maybe underexpose a little if you can't find a scene).
Here is an example of extreme contrast (http://www.magiclantern.fm/forum/index.php?topic=14892.msg144369#msg144369).
I played around with highlight and shadows sliders. Also with tone curve tool in acr. The dual iso file worked fine. Maybe find some more contrasty situations as suggested by Audionut to really see the difference.
Even in a scene with backlight (Directed towards the sun or in a dark room and bright window) seems not to be soo contrasty.
But yes, when i do an underexposure at -2 or -3 EV theres an extreme improvement in quality. But in which "real world" situations do I have that?
DualISO is working as expected. I just wanted to know that. Thanks...
@Danne/Audionut: I used exactly the same post processing settings for both pictures (Dual iso and single iso). I was expecting results like that: http://www.magiclantern.fm/forum/index.php?topic=7402.0
These scenes doesn't contain too much contrast, too. But now I guess these differences come from different post processing. Otherwise i can't explain...
Quote from: Myxir on April 19, 2015, 12:35:51 PM
But yes, when i do an underexposure at -2 or -3 EV theres an extreme improvement in quality. But in which "real world" situations do I have that?
The suggestion to underexpose was only to simulate a scene with greater dynamic range. By underexposing you are pushing more of the midtones down towards the shadows, where all of the noise is. In a large dynamic range scene this is what happens when you try and capture the highlights.
Dual ISO doesn't clean the midtones or highlights, only the deep shadows. If you don't need to capture deep shadows, stick to regular Canon images as you won't suffer any of the dual ISO drawbacks.
Quote from: a1ex]The output files contain 16-bit raw data in DNG format, that looks identical to a picture shot
in the same conditions at ISO 100. However, this DNG has a lot less noise in the shadows, and
therefore you can push the exposure a lot higher without getting massive noise (+6EV should
be quite clean).
Oh lol. Maybe I should have read that before. Now it's clear! :)
Although I think, +6EV is a little bit too much for my 1100D, even with DualISO...
By the way: If I change the DualISO settings from 100/800 to 800/100 it should save the highlights instead of shadows, right?
Yes, expose for the shadows and dual ISO will recover highlights.
Okay, perfect.
A few other questions:
Why can't I use ISO 6400 for Dual ISO? Even if I select 6400 from Canon menu and Dual ISO menu it says "3200/3200 -> Both isos are the same".
And what is if I select Highlight tone priority? ML says Canon analog ISO is 100, equivalent ISo is 200, but DuaISO says 100/x, not 200/x. What does that mean?
I can confirm that you can't set up 3200/6400 ( it will reset to 3200/3200 thus disabling it ), but you can set 6400 in ML and it yields 6400/3200.
HTP is only useful for JPG/H.264. With raw it simply underexposes one full stop.
ISOs above 3200 with dual ISO are pointless. For further information, start at the first post of this thread and enjoy the read.
Because when you increase ISO from 3200 to 6400 you "recover" 1EV of shadows but lose it on overall dynamic range (so you get nothing)? Even from 1600 to 3200 there's only 0.1 EV DR more...
Are there differences between different camera models? Because full frame cameras for example are a lot better at high ISOs...
http://www.magiclantern.fm/forum/index.php?topic=10111.0
In the first post of that thread, also check the section entitled "Any recommended reading?". The DxOMark site will list the dynamic range for all tested cameras.
ISO 1600 > ISO 3200 may net you 0.4 EV on some camera when you are using standard image capturing. With dual ISO you shouldn't expect to see that result. Each ISO is only half resolution, so you throw away one full stop of (full) midtone resolution, to "gain" 0.4 EV of half resolution shadow detail.
When you use recommended settings (100/800....100/1600), since the top couple of stops are generally white anyway (http://dpanswers.com/content/tech_zonesystem.php#zs), the resolution loss may not even be noticed. But when you start using extreme settings (100/3200), you're throwing away another full stop of midtone redolution, and this resolution loss is smack bang in the midtones. You probably want all available resolution for the midtones.
Where you use a low base ISO of say 100, the situation is probably even more dire. Here, you probably have your exposure (shutter/aperture) set for the base ISO. Since ISO doesn't effect shot noise (http://en.wikipedia.org/wiki/Shot_noise), not only are those last few points of EV only half resolution, but they are also heavy with shot noise.
There seems to be an issue with the created DNGs in Lightroom 6
https://www.dropbox.com/s/sj7htx3tr57y4op/Screenshot%202015-04-24%2017.12.37.png?dl=0
Shows the same created DNG file in Lightroom CC and in Lightroom 5. As you see, the interpretation is completely different. In the CC case everything is overexposed and cannot be brought back. I created the DNG an a Mac out of a Dual ISO file made by a 70D.
The original DNG file can be found in here: https://www.dropbox.com/sh/dnoiuhnqanjcbha/AADccyx0pHU2IiZhBCofNkM2a?dl=0
Does this occur with other Dual ISO DNGs as well?
Please keep issues regarding the 70D in the 70D thread. There's a reason it's still considered alpha.
Very well. Just to inform you I tested some of my older 550d dual-iso-dngs and they behave and look similarly in Lightroom 5 and CC, so indeed it seems to be an alpha issue with the 70d dngs. Should have tested that yesterday. Sorry for that.
Quote from: Audionut on April 24, 2015, 01:13:06 AM
http://www.magiclantern.fm/forum/index.php?topic=10111.0
In the first post of that thread, also check the section entitled "Any recommended reading?". The DxOMark site will list the dynamic range for all tested cameras.
I'll have a look.
Quote from: Audionut on April 24, 2015, 01:13:06 AM
http://www.magiclantern.fm/forum/index.php?topic=10111.0
ISO 1600 > ISO 3200 may net you 0.4 EV on some camera when you are using standard image capturing. With dual ISO you shouldn't expect to see that result. Each ISO is only half resolution, so you throw away one full stop of (full) midtone resolution, to "gain" 0.4 EV of half resolution shadow detail.
Estimated DR gained:
100/800: 2.2 EV
100/1600: 2.6 EV
100/3200: 2.7 EV
(100/6400: 2,7, too?)
Quote from: Audionut on April 24, 2015, 01:13:06 AM
http://www.magiclantern.fm/forum/index.php?topic=10111.0
When you use recommended settings (100/800....100/1600), since the top couple of stops are generally white anyway (http://dpanswers.com/content/tech_zonesystem.php#zs), the resolution loss may not even be noticed. But when you start using extreme settings (100/3200), you're throwing away another full stop of midtone redolution, and this resolution loss is smack bang in the midtones. You probably want all available resolution for the midtones.
Where you use a low base ISO of say 100, the situation is probably even more dire. Here, you probably have your exposure (shutter/aperture) set for the base ISO. Since ISO doesn't effect shot noise (http://en.wikipedia.org/wiki/Shot_noise), not only are those last few points of EV only half resolution, but they are also heavy with shot noise.
But what if I have an extreme (low light) situation where I may want to use not so usual setting? Such as 800/3200? Does it makes sense? And why do I get only ~0.5EV with that settings?
Edit: I see ISO 6400 is just 3200 with +1 EV digital ISO. So this would be an explanation (for me)...
Because the biggest gain is from ISO 100 to 200. Here you may gain 0.7-0.8EV. There's lots of noise at ISO 100, and the bump in sensitivity makes a big dent in the noise levels.
As ISO increases, the noise level is lower then the previous ISO, and so the reduction in noise level for each ISO bump reduces. As you note, when you go from ISO 800 to ISO 3200, the gain is only 0.5EV.
All other things being equal, you'll probably notice 0.5EV in the shadows. Higher ISOs also help to reduce any pattern noise (which are very visually distracting) in the images.
With dual ISO, 0.5EV in the shadows probably doesn't even come close.to outweighing the disadvantages. But it's your call.
From 400 to 3200 it's ~0.9EV. That should be noticeably.
I was thinking about using DualISO for some concerts or gigs. I know that I'm usually using very high ISOs, between 1600 and 6400. So I thought that I could use the second ISO to get the hang of high contrasts between spotlights and back/foregrounds.
Zanken already did this (http://www.magiclantern.fm/forum/index.php?topic=7402.msg115950#msg115950), but the links are broken and the pictures are gone. He had good experiences.
Quote from: ZankenI'm a gig/event photographer experimenting with dual ISO. It's been great in environments where there is such a strong contrast for things like stage lights and flash work. A couple of examples.
...
Quote from: ZankenThe big win for event flash photography is getting more DR in the backgrounds. Especially events like festivals where you notice that background lights colour channels cap out really quickly and everything just looks really smudged. Dual ISO has helped me immensly shooting manual flash in a pinch in this environment too - my TTL flash was stolen a while back and I haven't been able to afford a replacement.
Most of the examples here seem to be scenes which makes sense. We've all seen overexposed skies and can appreciate at a glance how big an improvement we are getting with Dual ISO. I question why most people haven't really discovered it for portrait/event/performance/street style stuff where the loss of detail across the frame matters less and your RAW files has much more flexibility in what you get out of a shot that is just snapped 'in the moment.'
If there is one downside for me though, it's shooting an event (200-400 shots) and leaving them to brew overnight, ultimately costing me 60mb each shot. Here I was thinking that buying a 6D would be a good move to preserve HDD space - hah!
Ive read through this forum and the LR plugin one, I can successfully take a DUALISO raw phot, but for the life of me, I am not able to enable the raw video (either 1.0 or 2.0MLV) with dual ISO... I just skimmed back through this forum to see if I was missing something, if I am I was unable to find it. So... can I shoot dual ISO RAW video on the 5d mk2?
If I can, please let me know which settings to enable,as my experimentation has so far not revealed the proper setup, each time I enable dual iso, raw video greys out, each time I enable raw video, dual iso greys out. Thanks in advance for any assistance.
They don't disable on my M1, although maybe you'd want to read the description and enable HDR video instead ;)
70D111B Dual-ISO CR2's converted to DNG's is now finally readable in Adobe... Mad Props to @Danne & @nikfreak!!!
(https://farm8.staticflickr.com/7662/17121751698_548f2675af.jpg) (https://flic.kr/p/s5Zsgu)
(https://farm8.staticflickr.com/7693/17122114400_43d5fcd7cb.jpg) (https://flic.kr/p/s62j5Y)
(https://farm8.staticflickr.com/7719/17283574236_6e0f2b14ed.jpg) (https://flic.kr/p/skhQtf)
Can any one give their experience how DualISO works in lets say weddings when you just take pictures of people moving/dancing, does it introduce any artefact for moving subject?
Quote from: Marius on May 05, 2015, 10:42:43 PM
Can any one give their experience how DualISO works in lets say weddings when you just take pictures of people moving/dancing, does it introduce any artefact for moving subject?
If you cared to read how DualISO works you should know the answer. Your answer is in the first post of this topic.
Trying to use dual iso on my canon 7d. I have enabled mlv raw, ettr and dual iso. Am trying to set an iso of 100 in the canon menu. But somehow its set to 3200. No matter how many times i do it, its set to 3200. There's a lot of banding in the resulting dng files. When i try to process it using the cr2hdr ui tool, the following message appears -
cr2hdr // Beta 1.4
Input file : M08-1022_frame_000001.dng
Input file : M08-1022_frame_000001.CR2 Canon EOS 5D Mark III detected Full size : 1728 x 972 Active area : 1728 x 972 Doesn't look like interlaced ISO
THE END
I am providing the link below for one of the frames in the video.
https://drive.google.com/open?id=0B_aIdQ5UVHpQUmN3N2dNbFZtV3M&authuser=0
Please help.
Dual ISO video not supported for 7D and 50D:
http://www.magiclantern.fm/forum/index.php?topic=13597.msg131347#msg131347
Oh! I missed that. Thanks for the update.
Updated cr2hdr with 70D support from nikfreak, and also with some fixes for future camera models.
At this stage, I think all major bugs from the 20-bit version were ironed out, so there should be no more reasons to use the old 16-bit one. Therefore, I've updated the Windows zip package with the latest version (also with latest dcraw and exiftool).
(note: the executable name is now just cr2hdr.exe)
Great news, @a1ex!
Awesome A1ex.
Great thank u A1ex!
Hello everyone! I have trouble shoot raw dual ISO. The problem is the pink dots that are common on canon 700d. I can shoot raw (no dual ISO) and get rid of the dots in MLVFS, chroma smothing 3x3 and bad pixel. I have in this example used the cr2hdr20bit. Hope someone can help me ?!
https://www.youtube.com/watch?v=jpN_MOvUSYE
This problem is not yet solved. I have a sample file, but if you can provide more, it may help.
One or two unprocessed DNG frames should be sufficient.
Sure, tell me what you need?! ok, how can i send it to you? pm?
https://www.dropbox.com/sh/382k6362pfb6pz1/AABC5UBbMODwGZugaXkcbsa8a?dl=0
Mac version 0c08758 (https://bitbucket.org/kichetof/lr_cr2hdr/downloads/cr2hdr.0c08758.zip) :)
Thanks a lot a1ex !!
Dreewling :P
Thanks Kitchehof!
It's Kristmas all over again! [emoji3]
5d Mark III
Dual ISO on, MLV Raw recording
and the latest:
-Nightly build
-raw2cdng.1.7.4
-cr2hdr
( Windows 7. 64 bit )
but the Dual ISO postprocessing isn't working. :'( Any suggestions?
Quote from: VanGogh7 on May 16, 2015, 06:52:58 PMbut the Dual ISO postprocessing isn't working. :'( Any suggestions?
Describe the problem.
You got the latest
cr2hdr from here too: https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/ right?
Yes, this is what I made described from Alex in this thread:
For RAW video files: extract the DNGs with any method you like (e.g. raw2dng or mlv_dump), then drag the DNG files over cr2hdr.exe
when I drag the whole files over cr2hdr.exe I got this error: the file name or extension is too long
No idea what this means!
(http://up.picr.de/21931734ue.png)
Try to aviod files and directories containing blanks.
Copy files to (example!) C:\temp and retry.
o.k. now it runs, but I have now a new error see image.
translated:
Input file C:\Temp\A\A.....dng
the command dcraw is write wrong or could not find..
( the same for exiftool)
Camera: Canon EOS 5D Mark III
????
(http://up.picr.de/21932286bq.png)
o.k. now I put the dngs on the same folder, it seems to work now. Wait a litte bit the process is slow...
the DNG files are due the repeated processing faulty. I tried to make a fresh converting of the MLV files to DNG and now it works.
One question, it seem necesary to put all .exe and DNG's in the same folder, right?
thanks Walter!
Quote from: VanGogh7 on May 16, 2015, 11:52:16 PM
One question, it seem necesary to put all .exe and DNG's in the same folder, right?
yes
or put them ( cr2hdr, dcraw and exiftool) in C:\WINDOWS\ ;)
then
SHIFT-rightclick on a folder ->
Open command line here ->
cr2hdr * ;)
Hi, I´ve used cr2dng.exe at several computers and I was very happy with it.
With Windows 8.1 (which I hate for several reasons) I fail. I do the same thing as always - having the converter and the .cr2 in the same directory and moving the cr2 onto the converter.
For a _very_ short time a dos-window opens and closes itself immediately. No conversion happens. No .dng file is created. It closes itself so fast that it is impossible to read anything. So in case there is an error message I don´t see it. The pause key doen´t help.
What can I do?
Make sure to have dcraw and exiftool in the same directory where cr2hdr is located.
Now it works again, thanks!
Hi I have just started using ML, on a 50D. At a recent airshow I used Dual ISO, with it set to alternate shots Dual/Normal. I have a few questions though. I had some problems downloading my files to my computer, and for some reason all the files managed to get deleted. I was able to use Test Disk/PhotoRec to recover all of my missing CR2 files and I have now loaded them into LR. I can visibly see which images are Dual ISO and which aren't. I am using the LR Plug In, but for quite a few of the images that are visibly obviously Dual ISO I receive an error message: This file dose not appear to be Dual ISO. Is there any way to force CR2HDR to process the file anyway? I would be willing to run the CR2HDR outside of LR if I had an easy way of telling which images were actually Dual ISO. I have also noted that I seem to have large numbers of images where it I have three Dual ISO exposures followed by three normal ISO exposures, but then there are also a number where they do indeed alternate. There are 2809 CR2 files from this one days shooting, so it would be really good to figure something out. I've yet to look at day to of the show, but that is only around 2400 images :D.
Alan
Upload one of the images that is dual ISO but not being processed. File sharing hosts (https://www.google.com/search?q=file+sharing).
If you take shots in rapid succession the alternate feature does not work. So if the next image is due to be a dual ISO image, and you take a burst of 6 images, all those 6 images will be dual ISO, and vice-verse if they were due to be regular images.
It's a shame that the alternating shot feature is fixed for bursts. I had been hoping to be able to get alternating shots to help with the comparison of the results. Unfortunately the conditions were quite poor for shooting most of the time, which reduced the DR quite a bit compared to "good" conditions.
I have put two CR2 files that were taken consecutively as part of a burst, they were both in the middle of the burst too, IMG_8650.CR2 would not convert, while IMG_8651.CR2 converted fine, I have also included the resulting .DNG, which was then processed in LR (processing not written to file). I have included another file IMG_9826.CR2 from a bit later in the day, which is also giving the error message. They are posted here on Dropbox (https://www.dropbox.com/sh/dggybd4i2jmh38e/AABrUJ3e21wgY1JRKfoZTG0Pa?dl=0).
I have also noted that the converted .DNG files are slightly larger, 4770 px on the long edge, than the CR2 files, which are 4752 px. I take it that is is normal?
Thanks
Alan
Sorry to have been a pain, it seems I have found the problem. It is the way I am trying to initiate the Export from LR. I created a Dual ISO export preset, with the log file turned off. If I use right click and select the preset from the list I get the error message. If I make a selection that causes the Export Dialogue to open, and then select the preset it works fine. I just used the Export Button at the bottom of the left side bar, and converted the problem files that I have posted above. Odd quirk, but I can certainly live with it.
Alan
Quote from: BigAl on June 04, 2015, 09:16:12 PMI have also noted that the converted .DNG files are slightly larger, 4770 px on the long edge, than the CR2 files, which are 4752 px. I take it that is is normal?
Yes. Canon shaves off a few pixels, ML doesn't.
I've got a problem with dual iso on my 5D mark II. Half of the photos were fine and half were corrupt. Latest cr2hdr software was used.
Here's the original, unprocessed file: https://www.dropbox.com/s/igkh8rvdqrqnexe/IMG_1569.CR2?dl=0
Here's how it looks: http://i.imgur.com/jYpTVip.jpg
And here's the processed original file: https://www.dropbox.com/s/xqy3kug0w8lcq7n/IMG_1569.DNG?dl=0
And how it looks: http://i.imgur.com/EMGE2ey.jpg
How is that so? Half fine and half corrupt? Is there a way to fix that?
Try to reproduce the issue on a different card.
Quote from: a1ex on June 12, 2015, 11:35:43 PM
Try to reproduce the issue on a different card.
I have no other cards unfortunately... The card I used was Transcend CF UDMA 7 1000x 32GB.
So is it a card issue? And there's no fix for the pictures I've already made?
Quote from: timkev on June 13, 2015, 10:10:48 AM
So is it a card issue?
Either a card issue, or a card reader issue. The embedded JPEG in the sample CR2 is not corrupted, so it's clearly not a sensor issue. Furthermore, dcraw reports:
IMG_1569.CR2: Corrupt data near 0x110109d
Quote
And there's no fix for the pictures I've already made?
If you still have the photos on the card, try downloading them with another card reader. If that doesn't work, it's hard.
Recovering a dual ISO image from a JPEG is not possible (http://www.magiclantern.fm/forum/index.php?topic=9145.0), though you may be able to recover a grayscale copy.
Your best bet is to figure out what bytes are damaged in the CR2 and recover them. There are some tools around that say they can do this, but I have no experience with them.
Hey. I struggle with dual iso video since yesterday, so I decided to make a post.
It works flawlessly in photo mode, but in video mode, I just don't get "dual iso dng", but normal dngs.
Dual iso is activated but is shown in gray, with the text bellow, in orange: "(MOVIE) This feature requires you shooting raw"
Of course I'm shooting raw. I seen on some older posts that you have to activate "hdr video" in video menu. But I don't have such option.
When I extract the dngs of the .mlv file obtained.. It's just not some dual iso files.
I don't get what I'm doing wrong... :-[
Camera? ML version?
Canon 7D with the latest nightly build.
Dual-ISO in video mode is not available on 7D: http://www.magiclantern.fm/forum/index.php?topic=13597.0
Ive been 3 hours trying to understand how to mix the 2 dngs in a single stream of dngs and I can`t understand anything on this thread. Look like chineese for me. Please any easy way of walkthrought like for dummies, for the 5D MARK 2. I have recorded a clip and converted to DNGs. Now what? I have downloaded cr2hdr app but when I drag the dngs give me an error like trying to reach "exitwhatever" module and dcraw, but I cant find this stuff to install in the forum and when I get something is like a text for the terminal or for a website in code that I dont really get what is that. Please, help.
By the way, how many frames per second can the 5D MArk 2 reach in HDR? Thanks
Windows or OS X? Windows here:
https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/18/
You have to download zip. Missing files are included.
OS X: https://bitbucket.org/kichetof/lr_cr2hdr/get/cr2hdr_unify.zip
Extract contents and use BIN directory.
THANKS!
I have used the bin directory and execute the 3 files in my mac. Now what? Where I have to drag the folder with the dngs? im lost. I does nothing doing it into the terminal after executing cr2hdr.
Thanks
You could try this one for mac
http://www.magiclantern.fm/forum/index.php?topic=15108.msg146822#msg146822
Also noticed you ask for HDR but that is completely different from Dual iso workflow so don,t really understand what you are trying to do here.
I thought it was the same thing, sorry.
I want to raise dynamic range in video maintaining 24 fps in 5D mark 2, like doing a HDR VIDEO? Can it be done? How.
Thanks!
You can work either with HDR or dual iso. They both have their ups and downs. I think you have to read and test out to see for yourself what you like best.
If you want to test these features out I created a scriptbased automated workflow which uses a1ex and g3gg0 binaries amongst others working with dual iso and hdr movie files. Here is link to the script.
cr2hdr-r (http://www.magiclantern.fm/forum/index.php?topic=13512.msg130562#msg130562)
Feel free to post questions in that thread.
jk i figured it out thanks everyone
Not sure why would you want to run the old nightly when it would be best to go with latest since it's usually most up to date. Are you running 113 or 123?
Are you still seeing the interlaced lines? Or are they not visible?
If not, did you load the proper modules needed within your ML settings?
Thanks for the response, I figured it out...it turned out to be a module issue.
I was having pink frame issues a while back; downgrading to the March nightlies fixed that issue for me. Admittedly, I haven't been checking up on the newer nightlies so I should probably get on that.
Fyi all in spite of cr2hdr offering lossy dng compression this doesn't quite work (for me). I've created a bug ticket: https://bitbucket.org/hudson/magic-lantern/issues/2333/cr2hdr-breaks-lossy-dng-aka-smart-previews
It's known that dual_iso dng files cause artifacts when converted to lossy dng in the regions of deep shadows that have been interpolated as half the the scanlines were clipped in the original raw file.
Apart from just being a "missing dng feature" in comparison to vanilla raw files, this problem has become more severe in recent Adobe apps since it's the base for the offline "smart preview" files. The free dng converter can be used to convert dng to lossy w/o Photoshop or Lightroom, it's basically 8bit jpeg-derived compression with the added benefit of keeping lossless wb adjustment.
I hope there's a way to do a minor cr2hdr adjustment to circumvent this problem. Alas, as Adobe's compression algorithm is closed source it might be tricky to identify the exact cause of the problem, but imho it's well worth a try. The dng converter seems to erroneously identify and compress these specific regions, maybe adding some noise or similar could be added to prevent this from happening?
Here's a sample how the lossy dng artifacts usually look: https://bitbucket.org/hudson/magic-lantern/issues/attachments/2333/hudson/magic-lantern/1438442876.44/2333/buggy_lossy-dng.png
@Marsu42: You trying to lossy compress with Adobe unprocessed Dual ISO? Why? I've read that lossy DNG compressed files no more Raw, it's demosaiced data. Demosaicing algorithm didn't expect to meet Dual ISO raw - this why artifacts, no?
Lossy Compress is possible with Lightroom plugin from @kitcheof...
http://www.magiclantern.fm/forum/index.php?topic=11056.msg151229#msg151229
Lossy compress is just an argument for cr2hdr. Kitchetof's plug-in is wrapped around cr2hdr and that's what most Dual-ISO tools do.
I'm aware of that, Walter.
I apologized for the slight misunderstanding and I thought I was just trying to be helpful by letting them know that it is possible.
I'm finding the dual ISO files a bit greener. 5D mark III 1.2.3 but build number doesn't matter because it's been all the way since I started using Dual ISO. Anyone notice this in the DNGs? WB is odd.
Do you mean that it doesn't show any greenish effect on 113 but it does 123?
Have you tested it on 113 yet yourself?
It looks fine to me here coming from the latest nightly 113...
Which converter did you use?
Quote from: mothaibaphoto on August 01, 2015, 07:33:49 PM
@Marsu42: You trying to lossy compress with Adobe unprocessed Dual ISO? Why? I've read that lossy DNG compressed files no more Raw, it's demosaiced data. Demosaicing algorithm didn't expect to meet Dual ISO raw - this why artifacts, no?
My understanding that a dual_iso file should be the same as a vanilla raw after it's reconstructed by cr2hdr. Adobe supports these 16bit raw files in ACR (Lightroom, Photoshop, DNG Converter), and I see no reason why lossy compression shouldn't work as well - though of course you can never be sure with closed source software.
It's an important feature for Adobe apps - smart previews enable you to do faster and offline editing, and lossy dng is great for archival of less important shots w/o resorting to jpeg. Btw the "standard" preview in Lightroom is broken as well with the same artifacts, seems to use the same compression algorithm.
On the bitbucket bug tracker, dmilligan guesses that this might have to do with the non-standard black level dual_iso files. I cannot say and I'm unable to change cr2hdr to test this, but lossy dng *does* work fine with mini_mo files which also have a custom black level (but break dxo's raw converter).
Quote from: dubzeebass on August 02, 2015, 12:30:28 AM
I'm finding the dual ISO files a bit greener. 5D mark III 1.2.3 but build number doesn't matter because it's been all the way since I started using Dual ISO. Anyone notice this in the DNGs? WB is odd.
I'm on 6d/60d, and I consider wb "more than odd". I cannot verify the theory that dual_iso and vanilla should just look alike, just with dual_iso having cleaner shadows. My current method is to let Lightroom figure it out with the "auto" setting on import which works better than the current cr2hdr detection attempts for guessing auto wb. The "auto" setting also gives you a hint in which way to correct files with a fixed wb setting from the camera.
Having said that, hdr pictures somewhat do look a bit odd in the shadows anyway (esp. concerning green/magenta tint), so that might be one cause of confusion and bogus bug reports.
Hello,
I have a little problem with dual ISO... with long exposures (1sec or more, though it seems to increase gradually), there are very strange artifacts in my images, as you can see from the image. This is a 1:1 crop from a 20sec exposure at ISO 100/800 with a 50D. These artifacts do not occur with the same exposure at a single ISO. Long exposure noise reduction has no influence at all.
Is this a known problem and is there any solution?
http://i239.photobucket.com/albums/ff252/mogul-/temp/IMG_8820-dualiso_zpslwk63idq.jpg (http://i239.photobucket.com/albums/ff252/mogul-/temp/IMG_8820-dualiso_zpslwk63idq.jpg)
I was geting some troubles processing dual iso DNG in the latest version of Rawtherapee, i tought it was a white level problem and asked some help in their forum.
It seens there are some double tags in the dual iso DNG and missing color matrix?(not sure if only mine, with latest ML nightly on 550D and latest cr2hdr)
"your dng file includes both tags with the same value (21). In this case the second one (50779) is given precedence, but there is no valid color matrix for second one. Only the matrix for 50771 exists in your dng.
I try to catch this case in rt source, but you should also report the bug to the guys who made the hack to produce the dual iso dng files."
@a1lex could you have a look in this post
rawtherapee.com/forum/viewtopic.php?f=1&t=6185
@toze3:
Latest cr2hdr from here: https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/ (https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/) right?
Do upload: Sample CR2 and Ouputed DNG.
Sample DNG will be found in linked thread.
Missing untouched CR2, though.
@toze. Could you upload an example dng?
Hi,
the problem with the DNG file is, that it contains the tags
50778 http://www.awaresystems.be/imaging/tiff/tifftags/calibrationilluminant1.html
and
50779 http://www.awaresystems.be/imaging/tiff/tifftags/calibrationilluminant2.html
Both have the value 21 and in this case Rawtherapee uses 50779. So far so good. But the corresponding color matrix for this case
tag 50722 http://www.awaresystems.be/imaging/tiff/tifftags/colormatrix2.html
does not exist in the DNG
I already prepared a fix for Rawtherapee, but IMHO, if tag 50722 does not exist, tag 50779 shouldn't exist too
Ingo
Certain programs implements tags and color matrix information. Provide a dng so people could check.
550D was used for sample.
Will try to reproduce with 650D.
Aware Systems offers a free TIFF tag viewer for Windows: http://www.awaresystems.be/imaging/tiff/astifftagviewer.html ... and it doesn't like DNG that much ...
Here's the link to the DNG (copied from RT forum) https://www.dropbox.com/s/r3k9t5pbvn6oe5i/_MG_1712_DI200_400.DNG?dl=0
Okay, Exiftool works.
Happens with 650D, too.
Tag 0xc621 (50721) Colormatrix1 exists but Colormatrix2 missing.
Workaround: Use exiftool to create tag Colormatrix2 and copy Colormatrix1 value into.
EDIT: Created issue/bug report #2343 https://bitbucket.org/hudson/magic-lantern/issues/2343/cr2hdr-not-generating-tag-colormatrix2
Quote from: Licaon_Kter on August 14, 2015, 12:46:23 PM
@toze3:
Latest cr2hdr from here: https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/ (https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/) right?
YES
Quote from: Licaon_Kter on August 14, 2015, 12:46:23 PM
Do upload: Sample CR2 and Ouputed DNG.
I don't have the same CR2 was the DNG posted but i can upload a new ones.
CR2: https://www.dropbox.com/s/1p3tbx44vl9msgg/_MG_2498.CR2?dl=0 (https://www.dropbox.com/s/1p3tbx44vl9msgg/_MG_2498.CR2?dl=0)
DNG: https://www.dropbox.com/s/83yju7334ibj19m/_MG_2498_DI100_800.DNG?dl=0 (https://www.dropbox.com/s/83yju7334ibj19m/_MG_2498_DI100_800.DNG?dl=0)
It happens in all my dual iso DNG.
Hope this helps
Quote from: Walter Schulz on August 14, 2015, 02:32:45 PM
Workaround: Use exiftool to create tag Colormatrix2 and copy Colormatrix1 value into.
How do i do that?
I think dcraw only outputs one color matrix tag.
If you,re on mac I could probably implement the insert of the color matrix 2 in my tool cr2hdr-r also for CR2 to DNG conversion. I already insert it it in movie dng sequences. Not familiar with the windows system anymore unfortunately.
For starters try to delete tag CalibrationIlluminant2:
Open command line/terminal in directory where Exiftool and test DNG are located and run
Exiftool -CalibrationIlluminant2= <name of your file>.DNG
Report results. Don't use originals but copies, please!
Quote from: Walter Schulz on August 14, 2015, 04:07:23 PM
For starters try to delete tag CalibrationIlluminant2:
Open command line/terminal in directory where Exiftool and test DNG are located and run
Exiftool -CalibrationIlluminant2= <name of your file>.DNG
Report results. Don't use originals but copies, please!
Just tested and now it looks good on rawtherapee
Can this be corrected on cr2hdr? or i have to use exiftool for all files if i want to use rawtherapee?
I suppose (!) it has to be fixed in cr2hdr. Danne has pointed to dcraw and I'm no programmer.
You can batch convert your files in a directory with
Exiftool -CalibrationIlluminant2= *.DNG
Some background:
original dcraw reads Colormatrix1 and Colormatrix2 (if present). If Colormatrix2 is present it uses Colormatrix2 else it uses Colormatrix1. It doesn't check CalibrationIlluminant1 or CalibrationIlluminant2 for value 21 (D65).
Modified dcraw in RT reads Colormatrix1 and Colormatrix2 (if present). Then it reads CalibrationIlluminant1 and CalibrationIlluminant2 (if present), checks which of them contains value 21 (for D65).
If both contain 21 (which doesn't make sense IMHO) it uses Colormatrix2 (which is not present in the dual iso dng). I already prepared a fix for this bug in RT. But it has to be tested before it will be merged into master branch.
Ingo
So, I fixed this a long time ago, it just hasn't been merged. The old chdk_dng.c code is not very good and this PR replaces it completely which should fix the issue: https://bitbucket.org/hudson/magic-lantern/pull-requests/603/dng-module-wip
You can compile cr2hdr from that branch and test if it solves the issue, it should.
Hi dmilligan. I compiled cr2hdr through codio from your branch. I creates a biary unix executable but when I try to run it in terminal I get the following.
/Users/itguser/Downloads/ml(1)/ml_dng/modules/dual_iso/cr2hdr: cannot execute binary file
You can't run a Linux binary on OSX ;)
Here are OSX binaries for cr2hdr and mlv_dump from dmilligan's ml_dng branch for testing:
https://drive.google.com/folderview?id=0BwCosJpHfhBsfmJudTAxekRiSzh3dEZMVnNaeTBGZE1yQ2hYMjVCRGR2dFlvbTRCbzhaazQ&usp=sharing
[EDIT: Added raw2dng]
hehe, gotta learn more compiling linux/mac.
But honestly. Dfort, your uploads worked totally fine! Super thanks. Would you upload the raw2dng as well? it,s in the lv_rec folder inside modules folder I believe. Or isn things changed in there?
Those binaries are awesome dmilligan! Tested right away.
Don,t forget to chmod u=rwx to give the binaries permission rights if you download.
/D
All of this is truly beautiful ... Definitely a good way to start the weekend -- Thanks guys!
Anyone out there able to compile for Windows?
Quote from: Walter Schulz on August 15, 2015, 08:55:13 AM
Anyone out there able to compile for Windows?
I'm also interested!! :)
Dfort updated with a compile of also raw2dng from dmilligan branch. See replay #2557. Enjoy.
Quote from: Walter Schulz on August 15, 2015, 08:55:13 AM
Anyone out there able to compile for Windows?
PM me on Monday if nobody responds. I'll dust off my old Windows laptop and give it try. It looks like all that's needed to compile the command line tools is Cygwin (https://cygwin.com/).
Ok--so I pulled out the Windows computer and started compiling the command line tools with Cygwin and even though it worked, turns out that you also need Cygwin installed in order to run the tools so that's not going to work for most people.
I installed MinGw and am going way down that rabbit hole--as expected.
I did manage to get a working version of dmilligan's ml_dng mlv_dump (https://drive.google.com/open?id=0BwCosJpHfhBsfl9EWDBQazFGdFVvMjJOd01WcUQ2amdXX1ZLNG9Rb3RNcTN3dHN2RTRxcjg) for Windows. I'll keep trying with cr2hdr and raw2dng.
@dfort:
your mlv_dump:
1. Produses empty wav (just header, 44 bytes - RIFF$ WAVEfmt Ђ» о data and nothing else )
2. Wrong lens F - number in ACR (original mlv_dump show correct F-Number)
3. Wrong temp 50000 in ACR (original embed 5500)
The rest seems OK...
Thanks for the feedback @mothaibaphoto
Can a Mac user check and see if you encounter the same issues?
Maybe the problem is with the way I set up MinGw? I still can't get raw2dng and cr2hdr to compile.
Until someone can build some proper Windows binaries, here are the Cygwin versions (https://drive.google.com/folderview?id=0BwCosJpHfhBsfkQ1V0pUV2pwWFhfWHB1LUR4ZGhSbE5ob1c4cEhVWVZIUEl2M21JRGIwclU&usp=sharing). You will need to put the .dll files in the same directory as the .exe files for them to work. Not too practical for production but maybe it will work for doing some quick tests of this:
Quote from: dmilligan on August 14, 2015, 10:34:55 PM
So, I fixed this a long time ago, it just hasn't been merged. The old chdk_dng.c code is not very good and this PR replaces it completely which should fix the issue: https://bitbucket.org/hudson/magic-lantern/pull-requests/603/dng-module-wip
You can compile cr2hdr from that branch and test if it solves the issue, it should.
By the way--I'm just trying to help out testers that can't compile for themselves. Looks like building these command line tools in Windows is more difficult than I anticipated.
Ok--I was able to compile raw2dng and cr2hdr for Windows. You can get all the Windows builds, including mlv_dump and Cygwin builds here (https://drive.google.com/folderview?id=0BwCosJpHfhBsfl9EWDBQazFGdFVvMjJOd01WcUQ2amdXX1ZLNG9Rb3RNcTN3dHN2RTRxcjg&usp=sharing).
Note that raw2dng and cr2hdr were cross compiled on a Linux system. I only have an old Windows Vista laptop and it can't run these binaries so there's no guarantee that it will work. mlv_dump won't cross compile on Linux but I was able to build it on Windows. By the way, the Linux system is on the same laptop that can dual boot into Windows or Ubuntu Linux.
Anyway--please report back so we know if this is working.
Cr2hdr.exe not working in 64-bit environment. Windows identifies it as 16-bit app and denies execution.
Quote from: Walter Schulz on August 17, 2015, 09:23:13 PM
Cr2hdr.exe not working in 64-bit environment. Windows identifies it as 16-bit app and denies execution.
+1 same for me
Yeah, it figures. My Windows/Linux laptop is too long in the tooth. I think these tools need to be compiled on a 64-bit system.
I'm trying to see if there's a way to run a MinGw gcc cross compiler on the Mac. In the meantime, do the other binaries work?
Alright--this one works. Cross compiled on a Mac.
cr2hdr.exe (https://drive.google.com/file/d/0BwCosJpHfhBsU0ZzZDRQMXViekU/view?usp=sharing)
Nice work dfort. Appreciated.
Thanks. Here's the whole bunch, cr2hdr, raw2dng and mlv_dump (https://drive.google.com/folderview?id=0BwCosJpHfhBsfl9EWDBQazFGdFVvMjJOd01WcUQ2amdXX1ZLNG9Rb3RNcTN3dHN2RTRxcjg&usp=sharing).
These are working on my old Windows laptop and they should also work on a new 64-bit system. (Used mingw-w64-i686)
Cr2hdr.exe works now. Tag-Issue solved.
But Win 8.1 "Smartscreen" is warning me about security threat. Never happened before.
Just as a reminder--if you are testing out cr2hdr, raw2dng and mlv_dump from dmilligan's ml_dng branch please make sure you post your comments to the open pull request:
https://bitbucket.org/hudson/magic-lantern/pull-requests/603/dng-module-wip
Once again, here are the Windows (https://drive.google.com/folderview?id=0BwCosJpHfhBsfl9EWDBQazFGdFVvMjJOd01WcUQ2amdXX1ZLNG9Rb3RNcTN3dHN2RTRxcjg&usp=sharing) and Mac (https://drive.google.com/folderview?id=0BwCosJpHfhBsfmJudTAxekRiSzh3dEZMVnNaeTBGZE1yQ2hYMjVCRGR2dFlvbTRCbzhaazQ&usp=sharing) versions.
Quote from: Walter Schulz on August 18, 2015, 12:04:24 AM
Cr2hdr.exe works now. Tag-Issue solved.
But Win 8.1 "Smartscreen" is warning me about security threat. Never happened before.
You shouldn't be downloading .exe files from strangers. Who knows what they might do to your computer.
Just kidding--I'm surprised you haven't gotten more security threat warnings.
Quote from: dfort on August 17, 2015, 11:39:48 PM
Thanks. Here's the whole bunch, cr2hdr, raw2dng and mlv_dump (https://drive.google.com/folderview?id=0BwCosJpHfhBsfl9EWDBQazFGdFVvMjJOd01WcUQ2amdXX1ZLNG9Rb3RNcTN3dHN2RTRxcjg&usp=sharing).
These are working on my old Windows laptop and they should also work on a new 64-bit system. (Used mingw-w64-i686)
Already tested and it's ok, except it's not the latest 20-bit version, but cr2hdr 16-bit and the camera white balance isn't correct, maybe because it doesn't have the option --wb=graymax
See the following picture, in the left is the cr2hdr 20 bit version, with tag corrected and in the right the DNG from this cr2hdr version
RAWTHERAPEE 4.2.276 with no processing with white balance set to camera. When in AUTO mode both looks equal
(http://i682.photobucket.com/albums/vv182/toze3/compare.png) (http://s682.photobucket.com/user/toze3/media/compare.png.html)
Also from the logs:
This new cr2hdr 16-bit
Last update: 76b5a32 on 2015-03-21 15:19:29 UTC by david:
dng: allow 64 bit compilation for raw2dng, mlv_dump, and cr2hdr
Unknown option: --wb=graymax
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
Input file : D:\Fotografia\RAW\0DISO_CR2_BKP\_MG_2498.CR2
Camera : Canon EOS 550D
Full size : 5344 x 3516
Active area : 5202 x 3465
White level : 12500
Black borders : 142 left, 51 top
Black level : 1965
ISO pattern : dBBd GBRG
Noise levels : 5.75 10.23 10.34 5.75 (14-bit)
ISO difference : 3.03 EV (817)
Black delta : 5.14
Dynamic range : 10.84 (+) 9.99 => 13.02 EV (in theory)
Looking for hot/cold pixels...
Hot pixels : 3
Cold pixels : 1
AMaZE interpolation ...
Amaze took 5.41 s
Edge-directed interpolation...
Semi-overexposed: 40.16%
Deep shadows : 4.08%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap : 4.8 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 4.39 (16-bit), ideally 4.36
Dynamic range : 13.23 EV (cooked)
Black adjust : -2
White balance determined from WB_RGGBLevelsMeasured
AsShotNeutral : 0.453714 1 0.809524
CR2HDR 20-bit
Last update: 0c08758 on 2015-05-09 19:25:05 UTC by a1ex:
cr2hdr: Makefile commands to create a zip package for Windows
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
Input file : D:\Fotografia\RAW\0DISO_CR2_BKP\_MG_2498.CR2
Camera : Canon EOS 550D
Full size : 5344 x 3516
Active area : 5202 x 3465
Black borders : 142 left, 51 top
Black level : 2048
ISO pattern : dBBd GBRG
White levels : 11738 12099
Noise levels : 5.74 10.23 10.34 5.79 (14-bit)
ISO difference : 3.03 EV (815)
Black delta : 2.87
Black adjust : 2
Dynamic range : 10.72 (+) 9.92 => 12.95 EV (in theory)
Looking for hot/cold pixels...
Hot pixels : 2
Cold pixels : 13620
AMaZE interpolation ...
Amaze took 2.23 s
Edge-directed interpolation...
Semi-overexposed: 43.64%
Deep shadows : 4.95%
Horizontal stripe fix...
49: offset too large (-4315)
50: offset too large (41078)
Full-res reconstruction...
ISO overlap : 4.7 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 66.08 (20-bit), ideally 65.99
Dynamic range : 13.20 EV (cooked)
Black adjust : -30
AsShotNeutral : 0.51 1 0.57, 4557K/g=0.98 (gray max)
Good report toze3. Looks like dmilligan worked on this before the 20-bit version of cr2hdr was ready.
Please keep checking the comments and changes on the pull request and let me know if you need another build to test.
https://bitbucket.org/hudson/magic-lantern/pull-requests/603/dng-module-wip
Quote from: toze3 on August 18, 2015, 11:11:02 AM
Already tested and it's ok, except it's not the latest 20-bit version
dmilligan just merged his code with the 20-bit version. I'm not able to open the Windows executable on my old laptop but for anyone who wants to do some testing--
Windows cr2hdr-20bit (https://drive.google.com/file/d/0BwCosJpHfhBsVVZ4YkFXYjVoQkk/view?usp=sharing) Macintosh cr2hdr-20bit (https://drive.google.com/file/d/0BwCosJpHfhBsZTBGNGlnaFF4Sjg/view?usp=sharing)
Quote from: dfort on August 19, 2015, 03:54:55 AM
dmilligan just merged his code with the 20-bit version. I'm not able to open the Windows executable on my old laptop but for anyone who wants to do some testing--
Windows cr2hdr-20bit (https://drive.google.com/file/d/0BwCosJpHfhBsVVZ4YkFXYjVoQkk/view?usp=sharing) Macintosh cr2hdr-20bit (https://drive.google.com/file/d/0BwCosJpHfhBsZTBGNGlnaFF4Sjg/view?usp=sharing)
It's a 16-bit app not compatible with my 64-bits machine...
Mac version cr2hdr works perfectly.
When it comes to white balance and mlv_dump I can see calculations for this in dng folder but numbers in actual dng files are always way off. Might send in a bug report if this issuposed to work same as in mlvfs?
Quote from: toze3 on August 19, 2015, 11:29:42 AM
It's a 16-bit app not compatible with my 64-bits machine...
That's odd, I'll look into it. The Mac version seems to be working.
Anyone else able to compile cr2hdr for Windows willing to help out?
I've been struggling to get a Windows binary of cr2hdr 20-bit version with the ml_dng changes applied. I do have a version built in the Cygwin environment which means you will need to download the .dll files for it to work. If anyone is interested in trying it out--here is the link (https://drive.google.com/folderview?id=0BwCosJpHfhBsfnBQM05EbHNnTzRnTkpVTHNpZnN6ZTUwSUJKejJIczhmRGtOclNTYm5DQVk&usp=sharing).
Quote from: dfort on August 20, 2015, 01:44:43 AM
I've been struggling to get a Windows binary of cr2hdr 20-bit version with the ml_dng changes applied. I do have a version built in the Cygwin environment which means you will need to download the .dll files for it to work. If anyone is interested in trying it out--here is the link (https://drive.google.com/folderview?id=0BwCosJpHfhBsfnBQM05EbHNnTzRnTkpVTHNpZnN6ZTUwSUJKejJIczhmRGtOclNTYm5DQVk&usp=sharing).
I put the dll on the same paste that cr2hdr, and it gives an error (translated from portuguese -> entering point not found (dynamic library)
One question, why is the file half the size (in KB) compared with the a1ex original 20-bit version (186KB vs 366KB)
It could be missing things?
Quote from: toze3 on August 20, 2015, 10:19:34 AM
I put the dll on the same paste that cr2hdr, and it gives an error (translated from portuguese -> entering point not found (dynamic library)
That's why I dislike distributions that require dynamically linked libraries. I'm still struggling getting MinGW to compile the 20-bit version.
Portuguese? Are you in Portugal or Brazil or someplace else? I visited Porto and Lisboa. I can speak Spanish but the extent of my Portuguese is "muito obrigado."
Quote from: toze3 on August 20, 2015, 10:19:34 AM
One question, why is the file half the size (in KB) compared with the a1ex original 20-bit version (186KB vs 366KB)
It could be missing things?
Sure, it is missing the libraries. Basically the libraries can be either statically or dynamically linked. Static linking includes the libraries in the executable file so the size of the .exe file is bigger than a dynamically linked version. So far I've only been able to compile dmilligan's ml_dng-cr2dng20bit in Cygwin which doesn't allow statically linked libraries.
I've had my best success at compiling ML code on a Mac. I didn't get any errors or warnings when cross compiling a Windows cr2hdr.exe but obviously it isn't working. Too bad, all the other command line tools seemed to cross compile fine.
Good news everyone--
Windows binary cr2hdr 20-bit version from the ml_dng branch is working. At least it launches on my system. Please try it out.
You can get it here (https://drive.google.com/file/d/0BwCosJpHfhBsUHZQaUNuWV9fTkE/view?usp=sharing).
This is a MinGW cross compile and no .dll's are required.
You, re tha man!
Just tested it on Windows here at work and I can also confirm that it works...
Seriously @dfort you're a pure workhorse and THANK you for finally closing the gap between Windows & Mac!
Great Job, Mate! [emoji108]
Anyone tested the new cr2hdr 20-bit on the 70D yet?
Tested a 70D file and it works!
Yuperonies!
All ok here!!
Thanks
Yeah seems gd! Thank you
Im confused.
I downloaded plugin (and exe file later on) and both doesnt work?
Ive got plugin enabled on my Ligthroom6 plugins list, and still no option to process files on 'right click. menu.
I drop CR2 files on exe file, still nothing.
Win 7, 64 bit.
I checked, those files are dual iso.
What am i doing wrong?
- Adobe's documentation about LR export plug-in location is just wrong and kichetof relied on that one. If you don't believe, you may download LR SDK 6 and try to run any of its export plug-ins in C:\TEMP.
If you want to have ML plug-in visible in context menu you have to place cr2hdr.lrplug in your Lightroom program folder in subdir Modules.
You can use your "unvisible" LR plug-in by using "Export ...". You have to select ML plug-in from drop-down list and press Export button.
If you have managed to make it visible in context menu you will see there is no user interface to change setting. If you want to change settings you have to use "Export ...".
- I suppose you downloaded your plug-in version from first post in LR plug-in thread. Sidenote: Why did you place your question here and not there? Anyway, this one is outdated and won't support 70D. Look into BIN subfolder and if there is a file CR2HDR20.exe you want to download and use http://www.filedropper.com/lrplugin
-
Taken ith my EOS M
(https://farm6.staticflickr.com/5713/21102619384_628b726d63_h.jpg) (https://flic.kr/p/y9Lrxf)The Sun Goes Down - ML Dual ISO (https://flic.kr/p/y9Lrxf) by jojobernard523 (https://www.flickr.com/photos/jojobernard523/), sur Flickr
Look at change list for recent builds:
https://builds.magiclantern.fm/jenkins/job/7D.203/218/changes
Changes 11 to 123 (sic!) are about cr2hdr.
Unfortunately there is no job for creating binaries. Eager to test this. Anyone able to compile for OS X and Windows?
Looks so sick! Definitely eager to test this one out.
Boss didn't play too nice w me and my personal 7D is broken (long story short it got soaked on a shoot my fault) and just found another used one online for $450 that includes a 24 pancake along w 2 batteries and a 64GB card.
Smells like good news are coming soon for 7D owners...
It's only the 20bit branch merged into the unified branch. So nothing new here, a1ex already made a binary available.
The rest is boring dev stuff from dfort, nothing exciting for an end user. :P Unless you ever wanted to compile cr2hdr yourself, in which case it probably just got a whole bunch easier. ;)
Just updated and built the latest cr2hdr. Here is the zip file containing cr2hdr.exe, dcraw.exe and exiftool.exe.
https://bitbucket.org/daniel_fort/magic-lantern/downloads/cr2hdr-Windows_2015-10-01.zip
More boring stuff to do, create a Mac binary--as soon as my system finishes installing El Capitan.
Excellent progress as usual, @dfort!
Ok boys and girls, here's the latest cr2hdr for Mac (new and improved 20-bit version) from the unified branch:
https://bitbucket.org/daniel_fort/magic-lantern/downloads/cr2hdr-Macintosh_2015-10-01.zip
Also bundled with the latest dcraw (Revision: 1.476) and exiftool (10.02).
cr2hdr with the cream on top 8). Thank you dfort for compiling, and thanks A1ex.
Tested and confirmed it is in fact creamy as it gets and special thanks to @a1ex for getting this done!
Thanks!!! These days I was wondering were to find the latest versions inside this forum. And you just post them!!! Great!!!
Hi all,
I'm a landscape photographer and impressed by the dual-ISO features in ML. However, the effective resolution of the post processed photo will be affected. Therefore I will still use the GND filter to adjust the DR issue. I want to check whether there is a simple multiple ISO setting feature that just take first half with ISO=100 (example) and second half as ISO=1600 to act like a Hard GND filter? If not, I will study the code and see how could have such feature to be developed.
PS: I'm new to the forum but would like to contribute too. :)
Regards,
lamhk
Would never had been able to take this picture without Dual ISO
http://kameratrollet.se/2015/10/02/manuppgang-over-angelholm/
I also tried two different exposure and to add the moon in the other one, but the Dual ISO image was the best.
@ Sapporo - Beautiful picture! What was your ISO setting (100-1600)? What about shutter speed, aperture and lens used?
Excellent shot of the Moon @Sapporo!
http://regex.info/exif.cgi is a good tool. 0.4 sec, f/2.8, 200 mm and ISO 3200. I think I used ISO 100 as a recovery ISO, but I am not sure.
kanonbild Sapporo :)
Tack (Thanks) Danne, DeafEyeJedi and JADURCA.
Whoop whoop ... a new updated version of crdhdr for Mac ... Thanks ! :)
But how do i install it please ? Do i need to dump the older stuff first, if so ... how ?
Any ideas DeafEyeJedi (you always seem to know all the answers !) :D
@NitromanX a little more information would help.
How are you using it? Command line, cr2hdr-r or the Lightroom plugin?
Where do you have it installed? If you are using it from the command line type "which cr2hdr" to get the installed path.
You can dump the old one or simply overwrite it with the new one.
Finally, from where did you download the new updated version? There's a chance that you don't really have the updated version.
Hi dfort ! Sorry, i forgot there are other ways than just using it as a Lightroom Plug In - which i do.
I'm not much of a computer whizz so don't use Command Line.
From what i remember i just added the plug in to Lightroom. However, when i downloaded the latest version from here 1.10.15 6.15pm : https://bitbucket.org/daniel_fort/magic-lantern/downloads/cr2hdr-Macintosh_2015-10-01.zip
It contained other stuff like cr2hdr, dcraw, exiftool and the folder called lib. Not sure what all that stuff is ?
Is this the right download and what do i do to install in to Lightroom as a new Plug In (to replace older version) ?
Thanks for your help ... you guys are great. :)
You downloaded the command line tools. I actually did a small update to the plugin pull request this morning so I posted a new version. Look in here for the latest:
https://bitbucket.org/daniel_fort/magic-lantern/downloads
or here:
https://bitbucket.org/daniel_fort/lr_cr2hdr/downloads
Fantastic dFort ! That is awesome ...
I've just downloaded the LR Plug in from the new url will try it out asap.
What has changed from original Lightroom Plug In ?
Can i also ask what advantages the LR Plug in may have over the actual stand alone Mac App - if any ?
I'm guessing i can now delete the older LR Plug In and just use this one as it's a stable release ?
Thanks again ! You have made things so much easier for us all. :)
This one uses the latest cr2hdr (now with 20-bits standard) along with the latest dcraw and exiftool. The Mac version no longer needs to have dcraw and exiftool installed on the system because it uses what is inside the plugin package like the Windows version does.
If you want to see all the changes check out the pull request:
https://bitbucket.org/kichetof/lr_cr2hdr/pull-requests/12/cr2hdr_unify_merge/diff
As far as what advantages the LR plugin has--well, it makes it easy to work with dual iso shots straight out of the camera and into Lightroom which is the workflow most still photographers that I know use. You can of course convert the dual iso shots before importing them using either the command line or a stand alone app. If you are shooting dual iso video you should check into cr2hdr-r (http://magiclantern.fm/forum/index.php?topic=13512.msg130562#msg130562) instead.
Thanks dFort ... incredible helpful again. I'd better get out there and try it ... :)
Have some footage illustrating (probably) this problem:
http://www.magiclantern.fm/forum/index.php?topic=12796.msg130147#msg130147
That discussion finished with nothing, but the phenomenon seems to exist.
First example shot on MKII with FRSP, frames 52-55 of several hundred shot extracted with mlv_dump -f option.
https://dl.dropboxusercontent.com/u/76052543/DualISOFlickering/20150923081158_f_52_55.MLV
What I found:
Flickering can appear and disappear during one take and cannot be fixed in post (I didn't find the way). It not looks like WB changing, just the cloud appear and disappear in the sky. Something about contrast.
Any setting --wb=, including custom, doesn't affect the result.
Every odd frame has "Horizontal stripe fix... offset too large" message but --no-stripe-fix doesn't affect the result.
--same-levels doesn't affect the result too.
Even and odd frames has vastly different Black delta and Black adjust values - something like
Black delta : 24.18
Black adjust : 255
and
Black delta : 13.17
Black adjust : 181
I don't know though, what it means and whether it matter.
Second - MKIII dual ISO video.
https://dl.dropboxusercontent.com/u/76052543/DualISOFlickering/M19-0541.MLV
No clear even-odd patterns here, really looks like WB changing. No way to cure, same as above.
Fortunately, these problems occurs very rare and noone really valuably shot didn't touched yet.
My first Dual-ISO Photos from SL1/100D ...
•Shot w 16-35L II @ ISO 100/1600
•Converted w latest cr2hdr.3.0BETA3.3 plugin from @kichetof in LR6
(https://farm6.staticflickr.com/5642/22290943418_946dbdbc88_n.jpg) (https://flic.kr/p/zXLUSE) (https://farm6.staticflickr.com/5767/22478664975_f8f108fa3e_n.jpg) (https://flic.kr/p/Afn2Xe) (https://farm1.staticflickr.com/620/22465288162_c1592d3d6f_n.jpg) (https://flic.kr/p/Aebtuw)
•Dual-ISO helped pushed this up to 13.28 stops of DR -- not bad for a $275 Body! 8)
(https://farm1.staticflickr.com/685/21856167174_5613976c29_n.jpg) (https://flic.kr/p/zimz3w)
Isn't it enough that you love the girl and the dog? Do you absolutely have to flaunt that tiny would-be cam SL1 - piece of plastic and vinyl eyes - nogood. Toy camera... minimum, midget, not to be taken serious, bla bla.
Ya, I'm convinced. I'm gonna have myself a SL1. You know what that tiny piece of ... shi...gold ... could do with a Canon 10-22? 15-85? 50 1.8? In a pocket in Paris with the tiny 40mm? And it has ML!!!!!!!!
And then you do Dual Iso? Backlit? Reproducing this with HDR would be a challenge. I showed your pics to a bunch of Sony and Panasonic guys. They talked about rolling me in feathers and tar for showing them something they are not able to reproduce. Rumors have it they are talking about Panic Lantern and Frantic Lantern... But all the good programmers and devs are ML guys.
Dual Iso takes 'bracketing' to a new dimension. ML makes it possible. Now it's up to the win/mac/linux people to build software, that exploits the full potential of DI. You show the way. And Boy must you love the girl and the dog. The wind caught her hair on the last photo and gave her a Chaplin moustache. Isn't it wonderful? Blow it up and put it over the mantlepiece. It's a picture of pure love.
At least I love the 3 of them. The pictures, of course. And the dog. And the camera. And...
You lucky dog, you! This SL1 was in masterhands.
Thanks @axelcine and here are a few more (100/1600) from the same body but this time with an old Nikon 135 2.8 glass...
(https://farm6.staticflickr.com/5686/21884470293_903d972dcd_n.jpg) (https://flic.kr/p/zkRCzk) (https://farm1.staticflickr.com/630/22318822679_ba3475a89a_n.jpg) (https://flic.kr/p/A1eNpz)
Look at the lady's eye lashes and the kitten's frisks. Not bad for a $40 lens I scooped up from a garage sale. Manually spot on!
Ha ha--good writing axelcine. You should consider a career in it. ;)
There's a separate topic for sharing dual iso photos (http://www.magiclantern.fm/forum/index.php?topic=7402.0) but as long as we're sharing numbers:
Camera : Canon EOS M
Full size : 5280 x 3528
Active area : 5208 x 3476
Black borders : 72 left, 52 top
Black level : 2038
ISO pattern : dBBd RGGB
White levels : 13440 12258
Noise levels : 7.93 43.24 44.91 7.91 (14-bit)
ISO difference : 5.02 EV (3245)
Black delta : -6.37
Black adjust : 82
Dynamic range : 10.49 (+) 7.83 => 12.85 EV (in theory)
AMaZE interpolation ...
Amaze took 2.77 s
Edge-directed interpolation...
Semi-overexposed: 74.38%
Deep shadows : 12.86%
Horizontal stripe fix...
1: offset too large (12422)
35: offset too large (-5916)
36: offset too large (-6180)
37: offset too large (-5987)
38: offset too large (-5451)
39: offset too large (-5319)
51: offset too large (-7154)
Full-res reconstruction...
ISO overlap : 3.0 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 70.59 (20-bit), ideally 70.57
Dynamic range : 13.34 EV (cooked)
Black adjust : -26
AsShotNeutral : 0.54 1 0.71, 4961K/g=1.38 (gray max)
A cheaper camera and even more dynamic range--well not by much. Uh oh, the gauntlet has been thrown!
Quote from: dfort on October 28, 2015, 09:47:50 PM
A cheaper camera and even more dynamic range--well not by much. Uh oh, the gauntlet has been thrown!
HaHa you will always be a true comedian, Dan!
Quote from: dfort on October 28, 2015, 09:47:50 PM
There's a separate topic for sharing dual iso photos (http://www.magiclantern.fm/forum/index.php?topic=7402.0)
Thanks for the friendly reminder... I can get lost easily just about anywhere, huh? :P
hi there .
Im new on the forum and i didn't understand properly the postprocessing workflow for those who use Dual ISO in video mode.
So , once i convert the raw files in DNG i drag these to cr2hdr.exe and i should get the images without the lines . Is that right?
Than in the GUI it says that the cr2hdr.exe needs dcraws as well. What for? and how do i use it? thank you
Assuming you are on Windows, right?
Hi Deaf , and thank you for answering . Yes im on windows.
So here i got the issue.
I made some shot with dual iso settings and i downloaded on my PC cr2hdr ( which in the some folder got dcraw and exiftool ) and raw2dng.exe.
One i downloaded my .CR2 files on the computer i dragged them into raw2dng program but it shows me the following error "This ain't a lv_rec RAW file".
So then i tried to open them in Lightroom and exporting in DNG with success. Now i got all the DNGs but once i drag them in cr2hdr app it doesn't work.
Thanks for the help ...i'm looking forward to use this amazing feature.
Anyone got problems with latest camera raw?
i did a few tests a week ago, worked great, but then i've reinstalled my pc, and now i have photoshop 2015.0.1 and ACR 9.2.0.486, and my dualIso's DNG show up really overexposed. (exposure slide can't fix it)
but they show properly when viewing with acdsee
im i doing something wrong? my camera is a 70D
(http://i.imgur.com/3I8vKWt.jpg)
I have been also able to screenshot what happens in the prompt.
Once i drag the DNGs in the propt it says:
Input file : C:\FOLDER (where i got the picture)
"dcraw" it is not recognized as an internal or external command , an executable program or a batch file
"exiftool" is not recognized as an internal or external command , an executable program or a batch file
Camera: Canon EOS Mark III
dcraw could not open this file
I translated everything from italian and i would like to remember that i dragged the DNGs exported from lightroom since raw2dng didn't work.
Thank u again
From first post: "[...] Windows: cr2hdr.exe). It requires dcraw and (optional) exiftool in your executable path (they are in the zip)". So place dcraw and exiftool into the same directory where you put cr2hdr.
I got them in the same folder.
i also put them in the same folder where i got the DNGs
Open command prompt (CMD.EXE) and type
cd \FOLDER
Then press ENTER key.
Type
cr2hdr.exe <name of DNG>.DNG
and press ENTER.
Results?
Nothing it tells me that:
cr2hdr is not recognized to be an internal or external command , an executable program or a batch file
You see cr2hdr in Explorer? If so: Type dir cr2hdr* followed by ENTER key and tell results.
if i do that it just shows me the folder with the 3 files inside ( cr2hdr , dcraw ,exiftool )
Could it be that the conversion made with lightroom is not the same made with raw2dng? sorry but im not a computer genius .
I read everywhere that people make the conversion with raw2dng but it doesnt work for me...could this be the problem? just wondering...
You aren't following Walter's instructions. Do EXACTLY what he tells you.
Do you know how to find your way around your system using the command prompt? If you can get into that directory where cr2hdr.exe, drcraw.exe and exiftool.exe are located you should be able to simply type cr2hdr and it will print out some options.
Okay. So you have a subdirectory named cr2hdr inside your FOLDER directory. Now copy contents from directory cr2hdr to FOLDER. Dcraw.exe, Exiftool.exe and cr2hdr.exe. And now you may proceed.
Dfort: He did. But he made a mistake about exe's location not knowing what the difference between dir cr2hdr and exe cr2hdr.exe is and what consequences there are.
Ok here is the result :
C:\Users\User\desktop\cr2hdr>cr2hdr.exe 7K7A3907.dng
cr2hdr: a post processing tool for dual iso images
Last update: 0c08758 on 2015-05-09 19:25:05 UTC by a1ex:
cr2hdr: Makefile commands to create a zip package for windows
Active options:
--amaze-edge : use a temporary demosaic step <AMaZE> followed by edge-directe
d interpolation <default>
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased.....
>
--wb=graymax : set asShotNeutral by maximizing the number.....
input file : 7K7A3907.dng
7K7A3907.dng: No such file or directory
File not found: 7K7A3907.dng
Camera : Canon EOS 5D Mark III
dcraw could not open this file
Guys i did it and a thank u a lot....now the only problem remain the fact that i havent been able to convert with raw2dng...but i had to go trough lightroom and export everything in dng first
If you are going through Lightroom anyway, why not make it easy on yourself and use the cr2hdr Lightroom plugin?
http://magiclantern.fm/forum/index.php?topic=11056.0
Oh--and raw2dng is not for CR2 files so there's no need to get into that.
I didnt know the existence . Thank you...is there any other trustworthy program to converte cr2 into dng without going trough lightroom? thanks
For Mac OS X I'm assuming you're on...
If so, @Danne's app 'cr2hdr-r' is incredible and yet so easy to use!
Sorry you're on Windows and I can't help there.
Look in the very first post of this thread.
You mean the:
Quote- Reference converter: cr2hdr.c (Windows: cr2hdr.exe). It requires dcraw and (optional) exiftool in your executable path (they are in the zip). Drag and drop the CR2 files over cr2hdr.exe.
- For RAW video files: extract the DNGs with any method you like (e.g. raw2dng or mlv_dump), then drag the DNG files over cr2hdr.exe
That's too easy. You know how we love to complicate things.
@LucaDeFranco it looks like you don't have dcraw.exe in your path. With Windows having cr2hdr.exe, dcraw.exe and (optional) exiftool.exe in the same directory should do the trick. You can also modify your path -- a.k.a. Environmental Variables in Windows. There are lots of tips online that can help you with that.
Hello guys, I have a problem here, now I can't convert the CR2 to DMG with cr2hdr. I'm doing exactly the same thing I was doing before but now doesn't work. Recently I upgrade my system to Mac OS El Capitan, I don't know if this is the problem but now I just get one .txt file saying this: (my camera is a Canon 7D)
------------------------
cr2hdr: a post processing tool for Dual ISO images
Last update: 2f4e2df on 2013-12-02 17:45:36 UTC by alex:
cr2hdr: fix DNG compression under Windows
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
Input file : IMG_9439.CR2
Camera : Canon EOS 5D Mark III
Full size : 0 x 0
Active area : 0 x 0
--------------------------------------------
The software says this:
-------------------------------------------
cr2hdr // Beta 2.0
Input file : IMG_9439.CR2
sh: dcraw: command not found sh: exiftool: command not found sh: dcraw: command not found Error: dcraw output is not a valid PGM file Logfile : IMG_9439.txt
THE END
----------------------------------
Thanks
César Oliveira
Whoa--that's an old build you are using. It looks like the "official" download linked here (https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/) is also outdated.
Try the one I compiled when cr2hdr-20bit was merged into the unified branch:
https://bitbucket.org/daniel_fort/magic-lantern/downloads
It also looks like you're missing exiftool and dcraw. I included them in the zip file that I posted.
A couple more things. There is an issue with cr2hdr not being able to ID cameras and sometimes it seg faults. Follow these links for more information:
https://bitbucket.org/hudson/magic-lantern/issues/2403/cr2hdr-always-reporting-5d3-as-camera
http://magiclantern.fm/forum/index.php?topic=16166
Hello dfort, thank you for answering me. I downloaded your files but I can't open it. cr2hdr open in terminal. What I need to do?
Where can I find a tutorial?
Thank you
-> First post in his thread.
Quote from: CesarOliveiraPhotography on November 14, 2015, 12:17:03 AM
Hello dfort, thank you for answering me. I downloaded your files but I can't open it. cr2hdr open in terminal. What I need to do?
Where can I find a tutorial?
What Walter said -- though to get the most out of it requires some skills on the terminal.
On the Mac put cr2hdr in /usr/local/bin so it will be in your path. If you don't know what I'm talking about then you can still use it if you put cr2hdr in the same directory as the files you want to process the difference is that instead of starting it by typing "cr2hdr" (without quotes) you will need to type "./cr2hdr". Open the terminal and type "cd" then drag the folder that contains the dng files you want to process, hit return. Now that you are in that directory type "./cr2hdr" and the name of the file you want to process. You can also type "./cr2hdr *" to process everything in the folder. cr2hdr will only process the .dng and .cr2 dual_iso shots.
By the way, a1ex just committed some changes to cr2hdr so I posted new builds in my download area:
https://bitbucket.org/daniel_fort/magic-lantern/downloads
Thanks dfort. Is it still the unified 20 bit version ? :)
There are also a new version from a1ex, the build 22.
https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/ (https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/)
Already tested and it's OK
Experienced a bug with the latest cr2hdr binary.
Older RAW file. Doesn,t seem to parse the color matrix 1 looking through exiftool. Filmed with a 5d mark 3 back in the days :).
Here are the files
https://drive.google.com/file/d/0B4tCJMlOYfirSmhNY1l3bGIzdGc/view?usp=sharing
https://drive.google.com/file/d/0B4tCJMlOYfircUd1NlY4MG0zVmc/view?usp=sharing
Quote from: Danne on November 17, 2015, 06:29:28 PM
Experienced a bug with the latest cr2hdr binary.
Older RAW file. Doesn,t seem to parse the color matrix 1 looking through exiftool. Filmed with a 5d mark 3 back in the days :).
I can't open M30-1635_1_2015-11-17_0001_C0000_000011.DNG with any of the tools that I've got.
M30-1635_1_2015-11-17_0001_C0000_000006.dng is a dual_iso shot that the latest cr2hdr seems to convert properly. What's the problem?
The version @toze3 pointed to seems to be for Windows only and I'm on OS-X so I wasn't able to test it out though I think I'm up to date on the latest version.
Quote from: tron on November 15, 2015, 01:56:17 PM
Thanks dfort. Is it still the unified 20 bit version ? :)
Yes, the 20-bit version has been merged into the main branch so from now on cr2hdr is 20-bit.
What happened to tag ColorMatrix1? Undefined in M30-1635_1_2015-11-17_0001_C0000_000011.DNG (viewed by ExiftoolGUI). Different DNG definition in both files: Readable one has 1.1.0, unreadable 1.3.0.
Do you have used different tools to get those DNGs?
Just guessing, but maybe dcraw and exiftool updated...
Sorry.
I used later build from dfort bitbucket and it won,t convert properly.
https://bitbucket.org/daniel_fort/magic-lantern/downloads
QuoteLast update: 8b9190b on 2015-11-14 16:44:35 UTC by a1ex:
cr2hdr: identify camera model from string (should fix issue #2403 ...
Should I update exiftool?
*Replaced also with included exiftool from dfort but still not working.
Try using the files included in dfort's zip file. Exiftool, dcraw, cr2hdr
EDIT: dfort is just running some code changes to cr2hdr.
I did. No cegar ;).
Do you get the second file to convert with his included files? Your on windows right.
Your DNG is out of order and there is no use playing around with it. How do you create this dng? It's converted from dual-iso clip, isn't it?
That is the DNG example I get when converting with dforts stuff. It,s just to show the problem. The second file is converted from raw2dng before using cr2hdr and it is very playable.
Can you make your source file accessible?
EDIT: I'm running Windows and OS X.
Found a file that isn,t humongous. MLV files works fine, RAW does not.
https://drive.google.com/file/d/0B4tCJMlOYfirQ1VtZXJscUZPcXM/view?usp=sharing
Thanks! And confirmed: dforts build messes up ColorMatrix1. I downloaded latest lr2hdr.plugin zip from kitchetof's bitbucket repository and converted your file and opened it in PS.
Good to know. The one on my download page has these commits that have not been merged into the main branch yet. Thanks for being the crash test dummies on this!
https://bitbucket.org/hudson/magic-lantern/commits/3a7553747e4f10984c9c35df3ed1fbd94c8aaefd
https://bitbucket.org/hudson/magic-lantern/commits/a5de7ee1cf619c5df45fb01122b9462dc77b3e3e
https://bitbucket.org/hudson/magic-lantern/commits/dd7f91d38e9ed2d3618d8014949b16e3b073d339
https://bitbucket.org/hudson/magic-lantern/commits/8b9190b7b0c57297c35eaaae4da26df74b12cc07
I'm not sure what the proper protocol is. There's no pull request yet. Perhaps comment on the commits so they won't get merged into unified? In the meantime I'll mark the versions in my bitbucket download area.
Ok--there's now a 2015-10-01 version and a 2015-11-14 testing version. I re-compiled the Nov. 14 version just in case that the raw video issue is coming from a mix up when I built the binaries.
https://bitbucket.org/daniel_fort/magic-lantern/downloads
Thanks! ColorMatrix1 issue not present in test version.
You mean the November 14 version? Really??!! That means that the problems you were seeing in the previous version had an error on my part and a1ex's commits are fine. Gee, imagine that. :o
We should probably continue using the November 14 test version and if it proves stable ask for that branch to be merged into unified. There is still a segmentation fault that goes back several versions. The discussion is in this topic (http://magiclantern.fm/forum/index.php?topic=16166).
Hello ML.
Dual ISO is new to me. I've tinkered with it a while back, but never fully explored it. Recently, I used it in a vacation rental video, and I hope to get feedback.
These jpgs are one frame from a RAW (not MLV) Dual ISO video. I used the LR Plugin to interlace.
5DM3
1920x1080
30fps
ISO difference : 3.97 EV (1567)
Original - Dual ISO difference : 3.97 EV (1567)
(http://s29.postimg.org/u44s9sblv/Dual_ISO.jpg) (http://postimg.org/image/u44s9sblv/)
Before Editing
(http://s29.postimg.org/l7u05ukzn/Interlaced.jpg) (http://postimg.org/image/l7u05ukzn/)
After Editing - In Lightroom
(http://s29.postimg.org/bm0fpjttv/Processed.jpg) (http://postimg.org/image/bm0fpjttv/)
Settings
(http://s12.postimg.org/biv6vniax/Settings.jpg) (http://postimg.org/image/biv6vniax/)
Is it normal to have to raise the exposure +2 EV and the shadows to almost 80? Seems very extreme to me.
Also, can some one clarify why the interlaced image is so dark? The pre interlaced version is much lighter.
If anyone has ideas or tips for me, I'd appreciate it!
Here are the details form the log file.
Camera model : Canon EOS 5D Mark III
Full size : 1920 x 1080
Active area : 1920 x 1080
Black borders : N/A
ISO pattern : BddB RGGB
White levels : 10000 14882
Noise levels : 8.00 8.00 8.00 8.00 (14-bit)
ISO difference : 3.97 EV (1567)
Black delta : 33.54
Dynamic range : 9.96 (+) 10.65 => 14.62 EV (in theory)
AMaZE interpolation ...
Amaze took 0.18 s
Edge-directed interpolation...
Semi-overexposed: 3.87%
Deep shadows : 89.44%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap : 3.0 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 8.00 (20-bit), ideally 8.00
Dynamic range : 15.96 EV (cooked)
AsShotNeutral : 0.61 1 0.53, 4048K/g=1.13 (gray max)
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs3x3 : apply 3x3 chroma smoothing in noisy and aliased areas
--no-bad-pix : disable bad pixel fixing (try it if you shoot stars)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
--same-levels : Adjust output white levels to keep the same overall exposure
for all frames passed in a single command line
(useful to avoid flicker - for video or panoramas)
@NickZee
I'm sure others will add further comments, however, from what you have posted it looks normal to me. Remember what dual ISO is. Think of it as a 'normal' exposure interlaced with an over exposed one. This is why the interlaced looks over exposed. The Dual ISO capture should only be seen as a 'data gathering' stage, ie don't try and assess it from a quality perspective.
It is thus usual to have to boost the exposure and/or shadows in post and play around with whites/blacks, and of course noise reduction.
I personally use the ETTR + Dual ISO as a killer combination for still photography, ie I don't touch video.
BTW I have written about my ML experiences on my blog, which you are welcome to scan at photography.grayheron.net. Please note I write this blog for personal enjoyment and accept it is not always the best view of ML or photography :-)
@NickZee:
You're getting about 16 stops of DR... Which is usually more than what you can ask for especially coming from a 5D3.
Looks fine to me as well.
@garry23 @DeafEyeJedi
Very appreciative gents!
Thank you.
You can also try crushing down the Blacks a bit more to give it more contrast and less noise...
a1ex did some bug fixes on cr2hdr. For those of you who like to live on the cutting edge, here are the latest binaries:
https://bitbucket.org/daniel_fort/magic-lantern/downloads
Hint: https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/changes
Hello. Your Magic Lantern is a perfect tweak for Canon. When used on Canon EOS 700D I thought I used a new camera. Please would it be possible to create GUI support for Photos APP on OSX EI Capitan as you have it created for Lightroom? Or cooperate with MacPhun and direct support in their new application for bracketing photoshots - Aurora HDR - http://aurorahdr.com/ for direct support your dual iso files and compilated Dual ISO shots. Also is there a possibility for iOS/Android mobile application?
Magic Lantern Forum » Using Magic Lantern » Post-processing Workflow » HDR and Dual ISO Postprocessing
Hi guys, I'm a fairly recent convert to ML and would like to say thanks for providing unbelievable tools for free!
I have recently started experimenting with dual ISO timelapses and am excited about the results, but I ran into an issue on a recent attempt.
I have attached a short video that shows my problem, which is unusual discoloration and lines above the horizon in the left half of the frame. Could anyone provide ideas on what is happening and how to fix it?
As for the settings they started at 100/800 ISO, and I accidentally set max aperture time to 1/15 second when kicking off the timelapse. Since I was at F22 I would imagine shutter speed was maxed at 1/15 by sun set, and the ISO was raising. I could find specifics later if need be.
Thanks again for the help everyone!
https://www.youtube.com/watch?v=JkOvmBE2QKg (https://www.youtube.com/watch?v=JkOvmBE2QKg)
It would be a good idea to post an original, unconverted CR2 from a frame that exhibits the issue.
Hi, is there an easy way to detect a dual iso CR2 file?
I was looking at some exif metadata, I noticed that some dual-iso CR2 files I own have a different BaseIso and AutoIso exif value, so at first I was thinking it was a way to detect them, but in fact I have some non dual-iso CR2 that have also different BaseIso and AutoIso values (why?) and some dual-iso CR2 that have the exact same value for BaseIso and AutoIso. So, this is not the way to get them.
Is there something in any metadata that can help me to detect them? I want to write a script that walk a directory full of CR2 files and convert dual iso ones automatically. Is it possible? :)
If you have a directory full of CR2 files, simply run them all through cr2hdr, and only the ones that are actually dual ISO will get converted.
You're right: You have to rely on cr2hdr.
Caveat: Cr2hdr will try to detect Dual-ISO files but will sometimes fail because of vastly overexposed/underexposed stripe sets.
QuoteHi, is there an easy way to detect a dual iso CR2 file?
You can sort them depending on the output message. For instance if it says "doesn,t look like dual iso" move those to a new directory.
Another one alternating flickering example (https://dl.dropboxusercontent.com/u/76052543/DualISOFlickering/M28-1449_fl.MLV) in addition to i've reported before (http://www.magiclantern.fm/forum/index.php?topic=7139.msg155488#msg155488)
It has to do with white balance or subtle color differences. Not sure. A1ex has mentioned something about it before I believe. I noticed this from another camera before and I believe in MLV format. It has not to do with white/black levels levels. You could post printscreens of picture 9 and 10 to show the difference. Or post a bug report https://bitbucket.org/hudson/magic-lantern/issues?status=new&status=open
Well, you posted about it yourself here :)
http://www.magiclantern.fm/forum/index.php?topic=12796.msg130175#msg130175
Here is one answer from A1ex
http://www.magiclantern.fm/forum/index.php?topic=12796.msg130982#msg130982
@Danne
Quote from: Danne on December 21, 2015, 05:30:01 PM
You could post printscreens of picture 9 and 10 to show the difference.
I shared a short MLV with just a dozen of frames - I believe it's better than printscreens.
Quote from: Danne on December 21, 2015, 05:30:01 PM
Or post a bug report https://bitbucket.org/hudson/magic-lantern/issues?status=new&status=open
I have to deal with bitbucket - account and so on... Ok, I will go that way next time if it's more convinient for developers to get info there.
Quote from: Danne on December 21, 2015, 05:30:01 PM
Well, you posted about it yourself here :)
No, it wasn't me, it was senzazn12 :)
Quote from: Danne on December 21, 2015, 05:30:01 PM
Here is one answer from A1ex
http://www.magiclantern.fm/forum/index.php?topic=12796.msg130982#msg130982
Actually it's not answer, it's request for some more examples.
If you mean "to avoid this effect, fine-tune the FPS until the image gets stable" -
I didn't noticed anything while filming with black and white crop mode preview.
Quote from: mothaibaphoto on December 21, 2015, 06:50:20 PM
If you mean "to avoid this effect, fine-tune the FPS until the image gets stable" -
I didn't noticed anything while filming with black and white crop mode preview.
Yes, I had learned this in the hard way back then when trying to avoid flickering was due to NOT using Fine-Tune FPS to get the perfect shutter speed which definitely helps reduce the flickering (it's even noticeable on the LCD LiveView on the camera as well) so yes it would be a good idea to use this method as suggested by @a1ex, @Danne and everyone else.
@DeafEyeJedi:
Examples, I uploaded, was shot on two cameras - MKII&MKIII, in FRSP photo and crop video modes. How can I fine - tune FPS in all these modes? Can you explain? When shoot in FRSP there is natural flickering, easy fixable in post. How can I differentiate natural flickering from this one on LCD? Nohow. And when i shoot crop mode i see just b/w picture. The main frustration with this issue is you can't understand, that something wrong until you convert your files on computer and it's too late to fine-tune something.
You should probably go out of the b&w mode before you start to film and only be in regular 3x mode if filming in crop mode. The problem as I see it is even if you set the white balance to match the next picture there is a nuance shift in color. Extract picture 9 and 10 and warm both up in ACR. Try fine tuning and to stay off b&w to see if that helps.
Quote from: Danne on December 22, 2015, 09:32:29 AM
You should probably go out of the b&w mode before you start to film and only be in regular 3x mode if filming in crop mode.
Why?
This eliminate this issue? - doubtable, as it exists not just in video mode, but in FRSP too.
Let me see if the flickering happens this time or not? - doubtable, too, plus, issue can happen somewhere in the middle of the shot.
But this definitely don't let me see the correct framing, which equal to disabling preview at all. I tend to disable preview on static shots anyway.
Quote from: Danne on December 22, 2015, 09:32:29 AM
The problem as I see it is even if you set the white balance to match the next picture there is a nuance shift in color.
Yes, something shifts and every time it looks differently, i'm not sure what shifts exactly, but it is not fixable in ACR or with deflicker.
Quote from: Danne on December 22, 2015, 09:32:29 AM
Extract picture 9 and 10 and warm both up in ACR. Try fine tuning and to stay off b&w to see if that helps.
I don't understand, what so special with 9 and 10 picture - why not 2 and 3? I really don't understand what does it mean "Try fine tuning". And if I disable b/w preview - how can I get, does it helps, or not - I shot hundreds videos and just couple with this issue among them all.
The main reason I share this shots not to survive it with any possible efforts, but to help developers to find the potential bug.
With lots of tests and an F1.4 lens I think noise with even the lowest settings 100/400 creates grain/noise that is more than wanted... I just want to bring in +1-2EV without additional noise in standard scenarios e.g a living room.
Is such possible with dual iso or is such just a dream?
Well, the lowest setting is going to have the most noise. If you want less noise, use a higher recovery ISO (higher ISOs have less noise per photon).
Technically you can't "bring in 1-2EV" without decreasing the noise b/c then you wouldn't actually be "bringing in 1-2EV". Dynamic range is defined by the noise floor, so reducing noise and increasing the dynamic range are actually the same thing. Dual ISO decreases noise or increases DR, whichever way you want to think about it, but they are the same thing.
Hi there , i got an issue i can't be able to solve. IM A WINDOWS USER.
Im trying to shoot HDR videos and i read on some guides that ones u got the RAW file out of your camera u have to process it with AVISYNTH , here instead i read that i should extract the files ( i use RAWtoCDNG) than drag them into CR2HDR and once processed open them into your editing program.
In the first case i downloaded AVISYNTH but im not able to use it since it doesnt show as a program but as a .dll file.
In the second case once a do everything and i do the CR2HDR process if i drag these output files ( which when i use it in picture mode i dont get any touble) into adobe premiere i get an error.
Also , reading the tutorial given in the beginning of the post , talking about avisynth it talkes about the need to have a "video work folder" which i have no idea what it is.
Is there any solution? am i mistaking during the process? thank you very much for the support
Please clarify: Do you use HDR option in movies or do you record video with Dual-ISO?
As far as understanding Avisynth, here's a good place to start: http://avisynth.nl/index.php/Main_Page (http://avisynth.nl/index.php/Main_Page)
Basically, you create a text file with Avisynth instructions in it, and open the file in another program (a video editor, processor or player) which interprets it as a video file.
Thank you for answer :
I'm using Dual Iso option in recording and in the first lines of this posts it talks about AVISYNTH...then there is a guide for photo mode and video mode....with photo mode there is no problem...with the video one i get the problem that i illustrated above.
Do i need to use avisynth ( which i don't know how ) or just follow the part that says:
"Postprocessing
- Reference converter: cr2hdr.c (Windows: cr2hdr.exe). It requires dcraw and (optional) exiftool in your executable path (they are in the zip). Drag and drop the CR2 files over cr2hdr.exe.
- For RAW video files: extract the DNGs with any method you like (e.g. raw2dng or mlv_dump), then drag the DNG files over cr2hdr.exe" ?????
The guide i've been reading is this one : Tutorial - Guide to using Dual ISO Mode on your Canon 5D Mark III
THank you
Guys...All I want is a simple application or plugin that can allow me to work with Dual ISO files. I know I'm probably asking a lot, but, my dream is to not have to learn what binaries are, where they go, how to code, having to type suffix IMG_1234 Dualiso nonsense, or bend over backwards trying to get things to work. The lightroom plugin on mac yields Error #256 and CR2HDR just says 'THE END' when dragging files in. Tried getting MLP to work but there are no clear instructions for the less code-savvy folk like myself.
Regarding MLP.
1 - install by double clicking MLP(.workflow)
No need to download additionals. Cr2hdr provided already.
2 - create a folder and fill it with your dual iso files. Right click on the folder and choose MLP from the menu. Do this twice as first time creates a folder structure. Second run starts processing.
F#@$-ing brilliant. Shows how much I know about computers :-[
I owe you one. I needed this tip a long time ago.
Any other tips on ways to get the most out of editing dual ISO images?
I'm specifically wondering if editing the dual ISO images SOOC in lightroom despite the crazy colors and stripe lines does anything differently than editing the DNG spat out by MLP
You can't use LR's RAW engine to process unconverted Dual-ISO files. Why do you think there is a plug-in if it is not needed?
Quote from: Walter Schulz on February 19, 2016, 12:23:09 PM
You can't use LR's RAW engine to process unconverted Dual-ISO files. Why do you think there is a plug-in if it is not needed?
I've already tried the plugin as stated before in my previous comment. Everytime I faced error #256. Anyways, your condescension is greatly appreciated. :P
Error occured when (which action) with which version?
Quote from: Walter Schulz on February 19, 2016, 01:44:11 PM
Error occured when (which action) with which version?
Lightroom 5. Mac Mavericks. File > Export > Sidebar Magic Lantern cr2hdr autosettings > Enter > Task bar loaded twice followed by: Unable to export file error #256
cr2hdr v3.0
Plug-In Manager -> Status
Path? Version? Status?
Quote from: Walter Schulz on February 19, 2016, 05:59:12 PM
Plug-In Manager -> Status
Path? Version? Status?
Applications/Adobe Photoshop Lightroom 5.app/contents/plugins/cr2hdr.lrplugin
This plugin is enabled, but has encountered some problems
failed to find a place for the imported file.
I'm guessing LR doesn't recognize the way the images are titled
Version info in Plug-in Manager, please.
hello to all, i'm new and i have a little problem :-X. i have install ML on canon 6d 1.1.6. but i don't find the dual iso in the menu :'( :'( :'( :'( :'( :'( :'( :'(. why? don't work with te 1.1.6.? thank's to all help me
Modules tab -> dual_iso.mo
for the pictures work? sorry for my english
.
.
.
i find it. thanks thanks thanks and thanks ;) ;) :) :) :D :D :D :D ;D ;D ;D ;D :-* :-* ;) ;) :) :) :) :)
.
.
.
.
sorry, but the file is .cr2, how to convert to dng? for postprocessing? you have a guide to workflow shot-pc-convert and postprocessing? thanks
First page, first post. And
http://magiclantern.fm/forum/index.php?board=14.0
Thanks you 8)
Hi there,
Does anyone know when dual iso on the 5Dmkii will be available?
I've been trying to use HDR video but it's only 720p.
So you've enabled the dual_iso module and restarted and?
I enable dual iso then restart, than when I go to the menu 'dual iso' is greyed out and says 'this feature requires you shooting raw'. But raw is already enabled. Is it know to be possible to shoot dual iso video with 5dmkii?
Sorry, no Dual-ISO in movie mode for 50D, 7D and 5D2.
привет) кто нибудь сталкивался с артефактами в виде полос на dualiso?
это он не смог свет вытянуть или из за вертикального кадра может? (подозреваю)
есть у кого опыт?
Canon 5D2 обрабатывал cr2hdr.exe
hi) somebody faced with the artifacts in the form of stripes on dualiso?
This he could not pull or light due to the vertical frame can? (I suspect)
Does anyone have experience?
(http://cs629403.vk.me/v629403320/3cb13/XfO9Lm_NGhg.jpg)
thanks
на горизонтальных кадрах вроде все нормально
horizontal frames feels good
(http://cs629403.vk.me/v629403320/3cb1a/sFi8CLrh2pY.jpg)
блин! и где отключается автоповорот на марке!! ??))
and which turns off auto-rotation on the mark!! ??))
@tosher
How are you processing your dual iso shots?
Способ обработки вы используете?
Hi everyone,
I am having a lot of trouble with this dual ISO in video. I was wondering if anyone knew the solution!
I am processing using raw2cdng.1.7.5 to turn MLVs into DNGs, and then cr2hdr to "process" the dual ISO to get rid of banding. Exiftool.exe and dcraw.exe are in the path.
Before processing, the DNGs look banded as they should:
(http://i.imgur.com/ETeYl8M.jpg)
But after cr2hdr is done with them, why are they turning out like this:
(http://i.imgur.com/e8JC6Br.jpg)
The directory they're in looks like this now:
(http://i.imgur.com/k3Ahydh.jpg)
Thank you!
According to my friend's 5D2 test footage that I shot in Dual-ISO (100/1600) which shows a lot of green cast (or possible incorrect metadata for Blacklevels and/or more others?) ... Any of you guys getting those as well?
Dual-ISO
(https://farm2.staticflickr.com/1528/25818932995_c60a7d61b2.jpg) (https://flic.kr/p/FkwMga) https://vimeo.com/159129291
Non Dual-ISO
(https://farm2.staticflickr.com/1569/25792865616_4374713a9f.jpg) (https://flic.kr/p/FiebkA) https://vimeo.com/159129292
Files were converted with @Danne's MLP and here are the samples files (MLV plus a single DNG from each) from 5D2 below for those that want to play.
DNG's -- https://mega.nz/#F!qgFDVIYY!Qe6XZc8FyVnbM6C4_OtdsA
MLV's -- https://mega.nz/#F!Wo8SDLaR!KD98tGfnzJMHlLy-cllB1A
*EDIT*
For those that are curious ... I got the 5D2 Dual-ISO Module from this thread below respectfully...
http://www.magiclantern.fm/forum/index.php?topic=16854.msg164124#msg164124
Here's some valuable info posted by @reddeercity that I felt was worth sharing in this particular thread...
http://www.magiclantern.fm/forum/index.php?topic=16854.msg164349#msg164349
Quote from: Tai.Fighter on March 15, 2016, 02:51:57 AM
I am having a lot of trouble with this dual ISO in video. I was wondering if anyone knew the solution!
I am processing using raw2cdng.1.7.5 to turn MLVs into DNGs, and then cr2hdr to "process" the dual ISO to get rid of banding. Exiftool.exe and dcraw.exe are in the path.
...
Whoa, something is definitely wrong here. I haven't used raw2cdng but your DNG file looks about right before processing through cr2hdr. Where did you get your cr2hdr.exe, dcraw.exe and exiftool.exe binaries? I've got a package on my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads) that you might try out. If that doesn't do it maybe you can upload one of those DNG files so we can take a look at it?
Seeing that you're on Windows you might also look into MLVFS. It can process Dual ISO while creating "virtual" DNGs. It also eliminate focus pixels that show up on certain cameras when shooting MLV.
Quote from: DeafEyeJedi on March 16, 2016, 09:17:57 AM
Dual-ISO
(https://farm2.staticflickr.com/1528/25818932995_c60a7d61b2.jpg) (https://flic.kr/p/FkwMga)
Updated cr2hdr source code to address this issue. Tested on the above example DNG, and on a 600D video DNG with similar black level issues.
The fix has two parts:
- use black level from EXIF if optical black data is not available (this should give results identical to MLVFS - not tested)
- analyze the shadows areas of the image; if it finds crushed blacks, the black level is lowered.
The first part was not needed so far because all the ML cameras that could record dual iso in video mode had the black level 2048. 5D2 is different.
The second part is actually a heuristic, and may help if EXIF data is incorrect. It worked on my test samples, but I'm not sure how well it will perform on other images. I'm also not sure how well it works with --same-levels (in theory, if crushed blacks are found, the best guess for black level should be the lowest value from all images).
The algorithm can be used in regular raw files as well (to check for incorrect black level).
On this test image, I've got:
Black level : 1795 is too high, using 1792
and the change actually reveals some extra shadow details (1795 vs 1792):
(http://a1ex.magiclantern.fm/bleeding-edge/isoless/M14-1732-black1795.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/isoless/M14-1732-black1792.jpg)
If it will pass my test suite, I'll upload a binary version as well; until then, you need to compile it.
I'm going to ask what I know everyone wants to ask but is too afraid to do so.
Does this increase the possibility that other cameras will also be able to shoot dual iso raw video? (7D??? :D)
Ignore this if u think this is a troll post :'<
Thanks dfort. I got the binaries from the ML downloads page, which is about as official of a source I could find... I'll take a shot using your bitbucket download and report back before investigating those other options, thank you so much for helping!
UPDATE:
I took screenshots of the cr2hdr that I tried before, and your bitbucket download. The one from ML download page gave me this:
(http://i.imgur.com/JBtXevb.jpg)
I tried your bitbucket, and got this instead:
Input file : C:\Users\USER\Desktop\New CDRHR\M14-201600066.dng
Camera : Canon EOS 5D Mark III
Full size : 1920 x 1080
Active area : 1920 x 1080
Black borders : N/A
ISO pattern : BBdd RGGB
White levels : 10000 5000
Noise levels : 8.00 8.00 8.00 8.00 (14-bit)
Doesn't look like interlaced ISO
ISO blending didn't work
screenshot:
(http://i.imgur.com/z7paSHQ.jpg)
@Tai.Fighter
It looks like the first cr2hdr build you tried says, "Doesn't look like interlaced ISO" the kept on processing and the second build stops after the "Doesn't look like interlaced ISO" message. In both cases cr2hdr is telling you that these aren't Dual ISO DNG files. If processing continues and treats them as Dual ISO the results will appear incorrect.
I did a search for "raw2cdng dual iso" and it looks like that's probably the problem. Try another conversion program like mlv_dump. This was updated recently--could someone supply a link to the latest Windows build? I'm only finding old "official" builds.
Quote from: Danson Delta-40 on March 17, 2016, 08:10:39 PM
I'm going to ask what I know everyone wants to ask but is too afraid to do so.
Does this increase the possibility that other cameras will also be able to shoot dual iso raw video? (7D??? :D)
Ignore this if u think this is a troll post :'<
I don't think that's a troll post--the 7D is about the same vintage as the 5D2 so maybe it will work. Of course someone has to be the guinea pig.
My google-fu sucks, but I can't find MLV_dump for windows anywhere... I'd be willing to take an old official one if you have it!
Try MLVFS (http://www.magiclantern.fm/forum/index.php?topic=13152.0) for Windows it dose dual iso
or MLVProducer (http://www.magiclantern.fm/forum/index.php?topic=15271.0) it has some dual iso support.
Quote from: Tai.Fighter on March 18, 2016, 03:41:39 AM
My google-fu sucks, but I can't find MLV_dump for windows anywhere... I'd be willing to take an old official one if you have it!
10 seconds to find
search box upper left hand corner of the web page , type
mlv_dumpIt was Number 8 in the result window
http://www.magiclantern.fm/modules/modules/mlv_dump.zip/mlv_dump.zip
I posted the latest cr2hdr:experiment for Mac and Windows in my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads/cr2hdr_experiment_2016-03-17.zip). This is mainly for testing the Video - dual iso for 5D2 (http://magiclantern.fm/forum/index.php?topic=16854.25) but it is of course open for anyone willing to live on the bleeding edge.
I also posted the latest version of mlv_dump for Windows (https://bitbucket.org/daniel_fort/magic-lantern/downloads/mlv_dump_Windows_2016-03-17.zip). Seems like I'm having troubles compiling a Mac version after rebuilding my development environment. Anyway--getting off topic.
@Tai.Fighter please don't double post: http://magiclantern.fm/forum/index.php?topic=16890.msg164418;topicseen#msg164418
Excellent timing @a1ex (as usual) and Thanks @dfort for sharing the latest cr2hdr binary.
Thanks @a1ex for the fix and thanks @dfort for the new complied cr2hdr.
Just tested with BarracudaGUI for dual iso on PC workstation ,
process the dual iso without any problem , there is a little difference in the mid tone area
there seems to more mid detail with MLVFS but more shadow detail with the update CR2HDR.
One think I notice is with cr2hdr process file in resolve I had to use less exposure then mlvfs process file
by about half , e.g. cr2hdr was about 2.5 where mlvfs was 4.6 , interesting .
plus I also notice the black level are the same but white is lower cr2hdr is 40000 , mlvfs is 60000
I guess that why the mid are brighter .
Non process Dual ISO 100/800 1856x928 from 5D2
(https://farm2.staticflickr.com/1564/25247778443_1ff8070722.jpg) (https://flic.kr/p/Et4sUT)
Non_process_M13-2146.MLV.frame_000000 (https://flic.kr/p/Et4sUT) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
New CR2HDR process though BarracudaGUI
(https://farm2.staticflickr.com/1712/25575943190_1eed090cb4.jpg) (https://flic.kr/p/EY4oRL)
New_CR2HDR_M13-2146.MLV.frame_000000 (https://flic.kr/p/EY4oRL) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
MLVFS process dual iso on windows7
(https://farm2.staticflickr.com/1630/25243826364_6a4db33c1a.jpg) (https://flic.kr/p/EsHd6A)
M13-2146_000000 (https://flic.kr/p/EsHd6A) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Old CR2HDR , wrong white & black levels
(https://farm2.staticflickr.com/1680/25876518485_5f39f24790.jpg) (https://flic.kr/p/FqBVqr)
Old_CR2HDR_M13-2146_000000 (https://flic.kr/p/FqBVqr) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Link to my dropbox with Dual ISO Cdng if any one would like check them out
Non process dual ISO --> https://www.dropbox.com/s/qnzst1571q766pu/Non_process_M13-2146.MLV.frame_000000.dng?dl=0
New CR2HDR dual ISO https://www.dropbox.com/s/3w24ifc4opurmgi/New_CR2HDR_M13-2146.MLV.frame_000000.DNG?dl=0
Old CR2HDR dual ISO https://www.dropbox.com/s/3al9oe8oerahtzv/Old_CR2HDR_M13-2146_000000.DNG?dl=0
MLVFS dual ISO (PC ver.) https://www.dropbox.com/s/c7punwsgezdmopt/M13-2146_000000.dng?dl=0
Great progress.
Here is a version of MLP with the experiment cr2hdr build for 5D mark 2 (and other cams to it seems). It will work right out of the box for developing the dng files. Havn,t had time to test it out and don,t own a 5d mark 2. (thanks dfort for compiling)
MLP_5D2
https://www.dropbox.com/s/vawstdji3yif8sk/MLP_5D2.zip?dl=0
Works like this
1 - Download and then double click MLP(.workflow) to install
2 - Right click and select MLP from the contextual menu on a folder containing dual iso MLV or RAW or MLV-lite footage. Do this twice cause first time MLP creates a folder structure then quits. The second time will start processing. Same white level(avoid flicker) will be applied to all files and it works multithreaded.
For this example, the ideal black level appears to be 1794 (or 7176 on a 16-bit image), if we assume the shadowed area on the second black PC is pure black. To my eye, 1793 looks best.
So, this is a good example where my algorithm went too far.
The exposure difference is most likely from the different white levels. From the raw data, MLVFS is closer to the truth; the difference should be visible in specular highlights.
Quote from: dfort on March 18, 2016, 06:42:30 AM
I posted the latest cr2hdr:experiment for Mac and Windows in my bitbucket download area (https://bitbucket.org/daniel_fort/magic-lantern/downloads/cr2hdr_experiment_2016-03-17.zip).
Windows 8.1 will not allow me to run this exe for security reasons ...
Right click -> properties -> Unblock ?
(https://obinshah.wordpress.com/2012/07/16/mass-unblock-files-remove-this-file-came-from-another-computer-warnings/)
What, you mean that your Windows doesn't trust my Macintosh?
(Oops--that remark is going to cost me another 900 seconds.)
@a1ex & crew -- instead of double posting but strongly felt this was worth mentioning for you guys to follow-up on what we have been digging through in regards to Dual-ISO videos for 5D2 here below...
http://www.magiclantern.fm/forum/index.php?topic=16854.msg164500#msg164500
@dfort
QuoteI don't think that's a troll post--the 7D is about the same vintage as the 5D2 so maybe it will work. Of course someone has to be the guinea pig.
I'm happy to be the guinea pig if you can provide the relevant updated modules.
I have a 7D. Gimme something to test bruh.
Ok guinea pigs, someone is going to have to do some investigation where to find these numbers for the 7D.
is_5d2 = 1; /*waza57:* For 5D2 distance = 0 , only one register in which doing changes */
FRAME_CMOS_ISO_START = 0x404b4590; // CMOS register 0000 - for LiveView, ISO 100 (check in movie mode, not photo!)
FRAME_CMOS_ISO_COUNT = 5; // from ISO 100 to 1600
FRAME_CMOS_ISO_SIZE = 0; // distance between ISO 100 and ISO 200 addresses, in bytes
I'm not really sure if there's more to dig up but maybe start a new topic like waza57 did for the 5D2 (http://magiclantern.fm/forum/index.php?topic=16854.0).
Here's the diff so you can see what changes were done to get Dual ISO working on the 5D2:
https://bitbucket.org/daniel_fort/magic-lantern-waza57/branches/compare/waza57/magic-lantern:dual_iso_video_5d2%0Dwaza57/magic-lantern:unified#diff
Quote from: dfort on March 19, 2016, 04:34:28 PM
I'm not really sure if there's more to dig up but maybe start a new topic like waza57 did for the 5D2 (http://magiclantern.fm/forum/index.php?topic=16854.0).
There is an old thread covering Dual ISO in video mode porting to 50D and 7D:
http://www.magiclantern.fm/forum/index.php?topic=13597
Thanks for the friendly reminder @Walter Schulz -- digging this old thread BIG time!
Quote from: dfort on March 18, 2016, 11:06:38 PM
In any case, that cr2hdr I posted is for testing so it looks like you got to be the crash test dummy :D
I don't have any 5D2 Dual ISO MLV files but the posted tests do look very promising. Maybe the 7D and 50D are next?
It's still online from my earlier
post (http://www.magiclantern.fm/forum/index.php?topic=16854.msg164286#msg164286post) if you would like to play with. I'm also trying to get ahold of my co worker to shoot some more Dual-ISO footage from his 5D2 just to have options for us to troubleshoot this bad boy down.
Quote from: Danne on March 18, 2016, 11:20:50 PM
Isn,t David using his own created dual iso algoritm in MLVFS?
Sean, can you try the same clip with the regular MLP and check for dropped frames?
Ah, well the one thing I'm surprised about is the fact that @dmilligan hasn't updated his cr2hdr binary yet (which I thought was needed in order for 5D2 Dual ISO to work?) however I then ran another test running with just MLVFS and default settings for cr2hdr20-bit and applied Cinelog-C in AE so here's what it looks like...
https://vimeo.com/159564896
Also here's the quick comparisons of in between MLP and MLVFS...
https://vimeo.com/159564926
@Danne -- I am currently rendering the same test files from 5D2 with the latest MLP from original post of your thread just to see what happens with the random dropped/pinkish frames on both the DNG's and ProRes4444 ... Stand by!
*EDIT*
Just finished spitting out DNG's and ProRes4444 which showed these same 'Green Cast' issues from few days ago BUT no dropping/pinkish frames.
https://vimeo.com/159566907
It seems with previous version of cr2hdr which shows green cast and the new compiled cr2hdr version shows random pinkish frames but no green cast ... Weird?
@a1ex -- How exactly can we narrow down the culprit on the latest cr2hdr binary re:5D2?
I have built and re-built cr2hdr from the cr2hdr branch on more than one Mac and verified that it keeps tripping up on the same frames from DeafEyeJedi's sample 5D2 files. Has anyone tried to convert these files on Windows or Linux?
DNG's -- https://mega.nz/#F!qgFDVIYY!Qe6XZc8FyVnbM6C4_OtdsA
MLV's -- https://mega.nz/#F!Wo8SDLaR!KD98tGfnzJMHlLy-cllB1A
Does anyone have another short Dual ISO test shot on a 5D2 they can post to see if this is a recurring issue or maybe just something with that one sample?
Ok I'll bit :D
I'll test the mlv & dng's and report back
Quote from: dfort on March 20, 2016, 01:54:12 AM
Does anyone have another short Dual ISO test shot on a 5D2 they can post to see if this is a recurring issue or maybe just something with that one sample?
I can make same Dual ISO from my 5D2 , any thing in particular you would like to see ?
Quote from: reddeercity on March 20, 2016, 02:20:20 AM
I can make same Dual ISO from my 5D2 , any thing in particular you would like to see ?
The file you used for testing is fine as long as it is short enough to upload somewhere.
Ok , I'll have to make new ones as the other ones are too big 4+GB .
I'll do this shortly.
Also I check out @DeafEyeJedi dual ISO MLV file & no problem , they look right
Extracted with MLVFS for PC & process with BarracudaGUI for CR2HDR (PC) and also process dual iso with MLVFS.
I'll post image later, so I don't think there anything wrong with your new complied CR2HDR at least with PC on windows7 pro
So as it seems ... it's only an issue on the OS X side?
I got few more 5D2 Dual-ISO samples at home.
Currently watching 'Ben-Hur' at the Segerstorm Center for the Arts. Silent film mixed with Symph0ny is quite remarkable!
@dfort Here you go , 5D2 MLV Dual ISO 100/800 1856x1044 23.976p 693MB file (about 300 frame)
https://www.dropbox.com/s/utb5b9hvfzuxr52/M19-2200.MLV?dl=0
I check on my PC , mlvfs & BarracudaGUI for cr2hdr , everything looks correct .
Anyone else that wants to play around with the files , be my guest
I'll keep the link up for 2 Weeks :)
Ok, here's what I'm getting on the Mac.
I thought I was on to something when several of the frames in reddeercity's MLV were corrupt when run through mlv_dump followed by the testing version of cr2hdr but the same frames were not corrupt when run through MLVFS without the dual iso setting then through the testing cr2hdr. (Well not 100%--only one frame was currupted.)
In other words I thought I had it narrowed down to a problem with mlv_dump but going through the same workflow on DeafEyeJedi's sample MLV came up with the same errors with MLVFS as it did with mlv_dump.
One workflow that is working fine with these 5D2 files is using MLVS to make the DNG's and convert the Dual ISO.
I wouldn't rule out an issue with my mlv_dump and cr2hdr builds--I recently rebuilt the development environment on my MacBook Pro, but the compile seemed to complete fine. The MLVFS I'm using was also compiled on my system though with a different compiler. In addition, it looks like the Windows binaries I cross-compiled on the Mac are working fine. Go figure.
Maybe it is because I'm using gcc 5.3.0? Should I roll back to a previous version?
Seems to be a problem also on mac just tried to use mlv_dump and getting errors with my 5d2 dual iso.
mlvfs is find thou .
@reddeercity
Where did you get the cr2hdr experimental binary? Is that the one I compiled?
Quote from: dfort on March 21, 2016, 12:17:28 PM
@reddeercity
Where did you get the cr2hdr experimental binary? Is that the one I compiled?
Yes Sir got for Here (http://www.magiclantern.fm/forum/index.php?topic=7139.msg164439#msg164439)
Edit: just a note , the images I posted came from MLV Viewer to view .mlv (and .raw) files on windows (C#) (http://www.magiclantern.fm/forum/index.php?topic=8447.msg77759#msg77759) as Dng's then process with your Win build of cr2hdr thought BarracudaGUI (Very fast)
@dfort @reddeercity
QuoteI have built and re-built cr2hdr from the cr2hdr branch on more than one Mac and verified that it keeps tripping up on the same frames from DeafEyeJedi's sample 5D2 files. Has anyone tried to convert these files on Windows or Linux?
DNG's -- https://mega.nz/#F!qgFDVIYY!Qe6XZc8FyVnbM6C4_OtdsA
MLV's -- https://mega.nz/#F!Wo8SDLaR!KD98tGfnzJMHlLy-cllB1A
Does anyone have another short Dual ISO test shot on a 5D2 they can post to see if this is a recurring issue or maybe just something with that one sample?
My test results processing the MLV's in a W10 environment. MLV file names were M14-1732 and M14-1733. A third MLV file from reddeercity m19-2200 was also tested.
Initial observation is that mlv file m14-1733 is not a Dual ISO mlv file. A visual inspection of the individual DNGs does not show the tell tale bright / dark horizontal lines of the alternate ISO's. Also, none of the apps used for testing recognised the DNGs as Dual ISO. If it was filmed in Dual ISO, then I'd suggest that some other setting in camera has overridden the alternate ISO value.
For files M14-1732 and m19-2200:
MLVP with Dual ISO enabled
- Was able to export a video file. Playback showed that that the individual frames had been processed for dual ISO successfully. In M14-1732 there did appear to be a bit of fringing? around the window frame.
- Exported cDNGs were not processed for dual ISO
MLVFS Dual ISO enabled
- From the virtual folders, I was able to successfully view dual ISO processed DNGs and successfully extract them to local HD directory.
MLVP with Dual ISO disabled and cr2hdr
- Dragging and dropping dngs from the dokan virtual drive directly onto cr2hdr resulted in cr2hdr going through the motions but it was unable to overwrite the original dng with the dual ISO processed image.
- Dragging and dropping dngs directly from the dokan virtual drive into a local directory on my HD and then processing them through cr2hdr successfully processed them as dual ISO images. Note that this resulted in the source dngs being overwritten with the dual ISO processed dngs.
- In both of the above bullet points the warning "Exiftool - not recognised as an internal or external command, operable program or batch file", "Exiftool couldn't update dng data" was display. Exiftool was in the same directory as cr2hdr so in light of a better explanation, I assume that this is because the source file was a dng that was being overwritten by the process.
Barracuda
- No joy here as this app only accepts .cr2 files as input
Edit: Correction. I was able to get this app to process the dng frames after extracting the individual frames to a local HD directory and renaming the file extensions from .dng to .cr2
Edit2: Correction. If I run this app via the desktop shortcut created at installation time vs from its installation directory, then dragging and dropping dngs into the app works fine.
MLRawViewer v1.4.3
- Allows preview of video but cannot process dual ISO.
- Allows export to dng but cannot process dual ISO.
- The following error was logged for reddeercity's file M19-2200.mlv
MlRawViewer v1.4.3
(c) Andrew Baldwin & contributors 2013-2014
Using GLFW
Loading 'D:\\MLVs\\M19-2200.MLV'
Set indexed. Frames missing: 2
M19-2200.WAV D:\MLVs\M19-2200.WAV
FAILED TO FIND FRAME AFTER SCAN 212
FAILED TO FIND FRAME AFTER SCAN 213
MLV Converter 1.9.2
- Extracts the dng frames 1st, then processes the extracted dngs for dual ISO (overwriting them). A very slow process (5 seconds per frame to convert to dual ISO on my notebook).
There is a restricted parameter set for the dual ISO conversions. - Can produce a proxy mov video using temporary tiff files generated from the dual ISO processed dng files. There was a noticeable colour shift in the proxy video vs the source dng images.
Hope this helps.
Thanks for the test
Quote from: Ottoga on March 22, 2016, 06:21:05 AM
Barracuda
- No joy here as this app only accepts .cr2 files as input
No , It take Cdng,s & DNG's Trust me ! Just drag & drop on the batch list that's how I process all of my Dual ISO on Windows.
Quote from: Ottoga on March 22, 2016, 06:21:05 AM
MLRawViewer v1.4.3
- Allows preview of video but cannot process dual ISO.
- Allows export to dng but cannot process dual ISO.
- The following error was logged for reddeercity's file M19-2200.mlv
MlRawViewer v1.4.3
(c) Andrew Baldwin & contributors 2013-2014
Using GLFW
Loading 'D:\\MLVs\\M19-2200.MLV'
Set indexed. Frames missing: 2
M19-2200.WAV D:\MLVs\M19-2200.WAV
FAILED TO FIND FRAME AFTER SCAN 212
FAILED TO FIND FRAME AFTER SCAN 213
Funny I never got any error , All DNG's, are accounted for no missing frames but I only ! use
MLVFS On Mac & Windows
It the Only convertor that up to date . All the other convertor have not been updated.
MLRawviewer v.1.43 is obsolete now
Last updated 2014/12/12 , in fact most are at least with the new dual ISO for 5D2.
I guess the only other up to date app is MLVProducer but the dev. wrote his own code for extracting dng's
So I guess I'll kept using MLVFS so not to have any problems until the other convertor catches up on updates at least with dual iso
Regarding file M19-2200.MLV from reddeercity https://www.dropbox.com/s/utb5b9hvfzuxr52/M19-2200.MLV?dl=0
mlv_dump metadata says 214 frames but the files contains only 211. Don,t know if that has anything at all to do with the problem here.
Size : 0x00000034
Ver : v2.0
GUID : 2648616435936133618
FPS : 23.976000
File : 0 / 1
Frames Video: 214
Frames Audio: 46
The flicker from the experimental cr2hdr is caused by a faulty black level number. I just lowered the number from Black Level 459008 to Black Level 7172 with exiftool and the file was fine again.
Here,s a problematic sample https://drive.google.com/file/d/0B4tCJMlOYfirTXVBUVkzYXJzNHc/view?usp=sharing
Coming from this file https://www.dropbox.com/s/utb5b9hvfzuxr52/M19-2200.MLV?dl=0
I've had lots of success with MLP. It works and swiftly for dual-iso on the 5d2. I have also had some issues. On a few sequences - even though the still-interlaced preview indicates a healthy image - conversion through MLP renders totally blank, empty black frames.
I wasn't able to successfully export converted Dual-Iso DNGS with MLVFS. This may have just been me not being sure how. (Not sure when/where output is modified due to in-browser toggles.)
Is there a GUI version of the experimental CR2HDR for Mac floating around anywhere?
Here is a file - Converted with MLP on Mac - to test for anyone curious what the finished output looks like.
https://drive.google.com/file/d/0B1lCd952bWxQaV91NmtKZnU2Nnc/view?usp=sharing
(To save space I only included some of the frames in the sequence. Hopefully this doesn't contribute to another issue like the one @Danne pointed out.
I tried the newest MLVProducer (PC win7 64) but couldnt export processed dualiso files as CDNGs despite preview which looked ok and processed. CDNGs still have stripes after export. Does this feature works in build 2241.intel ?
@kgv5
QuoteI tried the newest MLVProducer (PC win7 64) but couldnt export processed dualiso files as CDNGs despite preview which looked ok and processed. CDNGs still have stripes after export. Does this feature works in build 2241.intel ?
MLVP is a work in progress. At this stage it will process dual ISO MLVs when exporting/rendering as a video (MOV format) only.
For now I suggest that you use MLVFS, cr2hdr or Barracuda_GUI.
@reddeercity
QuoteNo , It take Cdng,s & DNG's Trust me ! Just drag & drop on the batch list that's how I process all of my Dual ISO on Windows.
Your right, tried it on another PC and it works fine. May need to reinstall Barracuda_GUI on my main notebook.
QuoteFunny I never got any error , All DNG's, are accounted for no missing
MLRawViewer didn't display any error messages. The error identified was written into a log file that is located in:
C:\Users\[username]\.mlrawviewer - @danne has identified a metadata mismatch which was the cause of the
error message.
Quote from: Danne on March 22, 2016, 09:12:36 AM
Here,s a problematic sample https://drive.google.com/file/d/0B4tCJMlOYfirTXVBUVkzYXJzNHc/view?usp=sharing
Do you mind extracting me just the unprocessed frame from the test video?
Sure, the file in question unprocessed.
https://drive.google.com/file/d/0B4tCJMlOYfirU2ZrN1JxNnA4RjA/view?usp=sharing
You guys have been magically amazing on this journey in the past several days ... Absolutely stunning, I dig the collaborations and excellent team effort from all of you!
Quote from: Danne on March 22, 2016, 09:12:36 AM
The flicker from the experimental cr2hdr is caused by a faulty black level number. I just lowered the number from Black Level 459008 to Black Level 7172 with exiftool and the file was fine again.
Here,s a problematic sample https://drive.google.com/file/d/0B4tCJMlOYfirTXVBUVkzYXJzNHc/view?usp=sharing
Coming from this file https://www.dropbox.com/s/utb5b9hvfzuxr52/M19-2200.MLV?dl=0
Ah, ha ... Nice work Danne and now I'm gonna try this trick of yours with a few more new Dual-ISO test files from the co-worker's 5D2 that I saved.
Has this fix be implemented into your MLP app or not yet? Manually adjusting the Blacklevels via exiftool within Terminal is the only way atm, correct?
Thanks again, D!
Quote from: Ottoga on March 22, 2016, 06:21:05 AM
@dfort @reddeercity
My test results processing the MLV's in a W10 environment.
Wow, great report.
FYI--M14-1733 was not shot in Dual ISO. It was shot so we can compare it to M14-1732 which is a Dual ISO shot on the 5D2.
I've been trying all sorts of ways to compile cr2hdr experimental on the Mac and I keep getting those pesky frames. I thought it was just a Mac issue and possibly something I did wrong when compiling but I just tried it on Windows with the same results.
Thanks Danne for pointing out the Black Level issue. Here's the output (on Windows) from that MLV reddeercity posted.
First frame 26 which always processes fine:
Input file : M19-2200_frame_000026.dng
Camera : Canon EOS 5D Mark II
Full size : 1856 x 1044
Active area : 1856 x 1044
Black borders : N/A
Black level : 1793
ISO pattern : ddBB RGGB
White levels : 14176 14817
Noise levels : 8.00 8.00 8.00 8.00 (14-bit)
ISO difference : 3.14 EV (882)
Black delta : -29.52
Dynamic range : 10.60 (+) 10.67 => 13.81 EV (in theory)Next, frame 27 which is the one Danne pointed out:
Input file : M19-2200_frame_000027.dng
Camera : Canon EOS 5D Mark II
Full size : 1856 x 1044
Active area : 1856 x 1044
Black borders : N/A
Black level : 1793
ISO pattern : BBdd RGGB
White levels : 15917 14819
Noise levels : 8.00 8.00 8.00 8.00 (14-bit)
ISO difference : 3.14 EV (883)
Black delta : -28.92
Black level : 1793 is too high, using 114752
Dynamic range : 10.79 (+) 10.67 => 13.81 EV (in theory)
Doing more testing with the new cr2hdr Win7 platform, now I'm getting random frame with Wrong black level/No viewable image on 5d2 dual ISO video , no preview.
I would say at least 30% are bad , with exiftool I set the black level with this command line string on Win7 PC
exiftool -BlackLevel=7172 *.dng
damage dng exiftool metadata
Black Level : 459008
White Level : 64876
Default Crop Origin : 0 0
Default Crop Size : 1856 928
Active Area : 0 0 928 1856
Opcode List 1 : (Binary data 28 bytes, use -b option to extrac
t)
XMP Toolkit : Image::ExifTool 10.02
Subject : Dual-ISO
Exposure Time : 1/53
F Number : 16.0
ISO : 200
Frame before the damage dual ISO frame
Black Level : 7172
White Level : 64876
Default Crop Origin : 0 0
Default Crop Size : 1856 928
Active Area : 0 0 928 1856
Opcode List 1 : (Binary data 28 bytes, use -b option to extrac
t)
XMP Toolkit : Image::ExifTool 10.02
Subject : Dual-ISO
Exposure Time : 1/53
F Number : 16.0
ISO : 200
After repairing the black level , the image still doesn't look correct , black level is ok but looks really pixelated
I have put in my Dropbox the frame before the damage one , plus the un-process dual ISO of the damage one by cr2hdr
there are name by what did or not do
https://www.dropbox.com/s/goai3ghcr2uxki3/CR2HDR_M23-1414_000189.DNG?dl=0
https://www.dropbox.com/s/nfcc6u9efkrrbhv/CR2HDR_damage_M23-1414_000190.DNG?dl=0
https://www.dropbox.com/s/nqeahldxngfv0rn/Exiftool_7172_black_level_fixM23-1414_000190.DNG?dl=0
https://www.dropbox.com/s/kk8y7v8gz2cyf63/Original_M23-1414_000189.dng?dl=0
https://www.dropbox.com/s/wejv7rzzikfqtzv/Original_M23-1414_000190.dng?dl=0
Not sure if the Barracuda_GUI is the cause of the problem , didn't see any problems with MLVFS dual ISO but being it's
slow at conversion I which to Barracuda_GUI for more speed.
Edit: I used 200/1600 ISO , and one last thing in the Barracuda_GUI as it was processing it reported the dual ISO as 0/800
Last time I used dual ISO I tested it at 100/800 , maybe a problem ? more testing needed.
Good news , Dual ISO (5D2) works in 3Xcrop successfully record MLV raw at 2048x1024 23.976p +audio for about 20 seconds.
Found the problem , cr2hdr doesn't like 200/1600 ISO for same reason from 5d2 re-tested with 100/800 ISO and no problem ,
all dual ISO dng's converted with proper levels .
So is this a limitation of this experimental build of cr2hdr or a problem with the 5d2 Dual ISO module in video mode ?
Still Testing ......
[Post deleted by author]
@dfort Correction @dmilligan - a new headache for you :)
I was using reddeercity's dual_iso mlv clip to perform some time trials of mlvfs vs other dual_iso processing utilities when I stumbled across a potential issue with mlvfs.
My hardware ASUS s550c notebook, intel i5 processors (4 cores), Intel HD graphics, 6gb RAM, OS w10_Pro
Dual_ISO mlv clip used from this link: https://www.dropbox.com/s/utb5b9hvfzuxr52/M19-2200.MLV?dl=0
The following links are for the same frame extracted from the mlv with hot/bad pixel fixing and chroma smoothing enabled only from mlvfs_x64, mlv_dump and raw2cdng respectively. Note the mlvfs images were just dragged and dropped from the virtual dokan folder to a local HD folder.
from mlvfs_x64: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11635&authkey=!AJoTpvZKJdKgdxs&ithint=file%2cdng
from mlv_dump: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11658&authkey=!AGfXcTSg0lAm7DU&ithint=file%2cdng
from raw2cdng: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11654&authkey=!AP3YXZLggtyjtLQ&ithint=file%2cdng
If you compare the images you will see that in the mlvfs extracted frame, there is a band of horizontal discolouration in the lower half of the image that does not occur in the other two images.
I tried various combinations of bad pixel fix, vertical stipe fix and chroma smoothing in mlvfs to see if it would correct the issue to no avail.
Next in mlvfs, I enable the Full 20it dual_iso with Amaze interpolation, Alias map ON and Fullres blending ON. The following is a link to the processed image:
https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11607&authkey=!AHl0M4oRKUS65fQ&ithint=file%2cdng (https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11607&authkey=!AHl0M4oRKUS65fQ&ithint=file%2cdng)
When inspecting this image you will se that the horizontal discolouration is still visible in the image. I tried again with combinations of Alias Map and Fullres Blending ON/OFF to no avail.
Lastly, I switched from Amaze interpolation to Mean32 and voilà! the frame was processed cleanly as per the following example.
https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11619&authkey=!AJnOkHpyiK5v_fI&ithint=file%2cdng (https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11619&authkey=!AJnOkHpyiK5v_fI&ithint=file%2cdng)
Is this an issue? Clearly, not a major one as I was ultimately able to successfully process the dual_iso mlv using mlvfs. However, based on my observations during this testing exercise, a couple of questions are raised my mind:
- For MLVFS, what is different in the logic that converts the individual frames to dngs vs mlv_dump and raw2cdng?
It seems to me that this is where the problem lies. - It is my understanding that Amaze is a superior interpolation algorithm/process than mean32. So why didn't it work?
Some links to additional frames extracted from the mlv clip and demonstrating the above.
MLVFS unprocessed dngs: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11591&authkey=!AJnfDU17yD92grI&ithint=folder%2cdng (https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11591&authkey=!AJnfDU17yD92grI&ithint=folder%2cdng)
mlv_dump unprocessed dngs: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11657&authkey=!AOuzH6UPxPb-0R8&ithint=folder%2cdng (https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11657&authkey=!AOuzH6UPxPb-0R8&ithint=folder%2cdng)
raw2cdng unprocessed dngs: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11592&authkey=!ABIMtCaTULWJie8&ithint=folder%2cdng (https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11592&authkey=!ABIMtCaTULWJie8&ithint=folder%2cdng)
mlvfs dual_iso using Amaze: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11589&authkey=!AEWf0uZEcrkT0P0&ithint=folder%2cdng (https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11589&authkey=!AEWf0uZEcrkT0P0&ithint=folder%2cdng)
mlvfs dual_iso using Mean32: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11590&authkey=!AHf8g2U5lutfPTQ&ithint=folder%2cdng (https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11590&authkey=!AHf8g2U5lutfPTQ&ithint=folder%2cdng)
Quote from: Ottoga on April 01, 2016, 08:30:29 AM
@dfort - a new headache for you :)
Not my headache. MLVFS is dmilligan's project and the MLV file where those files you are working with originated on a 5D2 (shot by reddeercity) using an experimental build that enables dual iso video on that camera. I believe it was shot at 200/1600 ISO which is another problem reported by reddeercity. I tried the MLV file with MLVFS on my Mac and it works fine. No strange green stripes like you are seeing. So not only don't I have a Windows system that can run MLVFS, I also don't have a 5D2 I can use to test. No headaches!
However, you bring up a good point--Amaze shouldn't be showing that strange artifact. It must be an issue with MLVFS on the PC and it should be reported on that topic.
What I do find interesting is that your mlv_dump processed DNG looks very good. Everyone else seems to be having issues--probably because they are using the mlv_dump that I built and posted on my bitbucket download area. Where did you get your mlv_dump binary?
Finally, people were reporting issues with raw2cdng combined with dual iso. Have you tried processing those raw2cdng processed dual iso DNG files in cr2hdr?
@dfort
I pm'd you.
QuoteHave you tried processing those raw2cdng processed dual iso DNG files in cr2hdr?
No, but I can give it a go and report back.
Just to make sure we're on the same page, we're using reddeercity's 5D2 dual iso MLV file for this test:
https://www.dropbox.com/s/utb5b9hvfzuxr52/M19-2200.MLV?dl=0
So far it looks like we're getting pretty much the same result with mlv_dump and cr2hdr binaries compiled on different systems and on different operating systems.
@Ottoga -- I have not encountered that artifact you pointed out with MLVFS on the Mac.
@dfort
QuoteJust to make sure we're on the same page, we're using reddeercity's 5D2 dual iso MLV file for this test:
https://www.dropbox.com/s/utb5b9hvfzuxr52/M19-2200.MLV?dl=0
Correct
QuoteI have not encountered that artifact you pointed out with MLVFS on the Mac.
It may be a Windows or w10 specific issue. I don't have a MAC or W7 machines so I can't do any parallel testing.
QuoteHave you tried processing those raw2cdng processed dual iso DNG files in cr2hdr?
Here are the results.
For those reading, this test result is specifically referring to dual_iso dng frames extracted from a dual_iso video.
mlvfs frame processed through cr2hdrThe same band of horizontal discolouration in the lower half of the image that is present in the original extracted frame is retained in the processed frame.
No errors logged by cr2hdr
Original unprocessed frame: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11735&authkey=!AAdkihGIrgX66lA&ithint=file%2cdng
cr2hdr processed frame: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11635&authkey=!AJoTpvZKJdKgdxs&ithint=file%2cDNG
cr2hdr run time text: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11737&authkey=!AISReBXRWPvt5Gc&ithint=file%2ctxt
mlv_dump frame processed through cr2hdrSuccessfully processed with no errors logged by cr2hdr.
Original unprocessed frame: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11718&authkey=!ABhj6JPqDUwNQrI&ithint=file%2cdng
cr2hdr processed frame: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11658&authkey=!AGfXcTSg0lAm7DU&ithint=file%2cDNG
cr2hdr run time text: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11739&authkey=!AFJZ5yUIETuwZUA&ithint=file%2ctxt
raw2cdng_v1.6.5 and 1.7.8 frame processed through cr2hdrcr2hdr didn't like the frames extracted by either version of raw2cdng displaying a number of 'offset too large" messages during processing. The output files generated are essentially a discoloured distortion of the source dng. They visually still look like dual-iso's but are no longer recognised as such by cr2hdr. The examples below are from v1.7.8.
Original unprocessed frame: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11758&authkey=!ACLei2iSHagG5mQ&ithint=file%2cdng
cr2hdr processed frame: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11695&authkey=!AIhe-SmbWq1W29Y&ithint=file%2cDNG
cr2hdr run time text: https://onedrive.live.com/redir?resid=C1A12E009BA4F636!11738&authkey=!AEPQG2ghgiDWvqM&ithint=file%2ctxt
Post processing conclusions of dual_iso from the sample video in my w10 environment......
- mlvfs has a potential problem with the initial frame extract that that causes corruption within the pre-processed dual_iso frame. post processing of the frame using amaze interpolation fails to correct this however mean23 interpolation did. Note that @dfort has advised that he is not seeing this issue in his Apple MAC environment
- mlv_dump coupled with cr2hdr resulted in a clean extract and successful post processing of the dual-iso frame
- raw2cdng coupled cr2hdr: a visual inspection of the extracted frames a successful extraction however cr2hdr encounters offset problems and fails to process the extracted frame correctly
Remember that these test results from one source video with capture parameters that may or may not have contributed to the results. If you are into dual_iso video then I urge you to perform similar tests and give feedback to the devs. Given good feedback and examples will enable them the tools available and provide a consistent post processing experience for all supported platforms.
Works just fine here.
In the first post of this thread a1ex said:
Histograms, ETTR, playback tricks
- ETTR integrates with dual ISO (just enable both). It will use dual ISO only in high dynamic range scenes. Adjust the SNR limits to let the algorithm know what exactly is a "high dynamic range scene" and to fine-tune the balance between midtone/shadow noise and highlight aliasing.
I enabled both DualISO and ETTR on my 550D, but even in normal scenes I get DualISO RAW. Where am I wrong?
Sounds like you need to adjust the SNR limits.
Is there a detailed guide somewhere for how to use these features or it's only in ML general manual?
The best we have is this: http://www.magiclantern.fm/forum/index.php?topic=11269.0
Auto ETTR is pretty well documented, just follow the links.
Very good, thank you very much!
Trimis de pe al meu X2_Soul folosind Tapatalk
Hi guys, what's the latest with the 5d Mark II Dual-Iso stuff?
Is there a newer version of the dual-iso.mo?
Is there a better workflow than MLP? I wasn't able to find the hacked CR2HDR.
my canon 50d has the lowest iso at 200. Since multiples of 160 are ideal for less grain does this mean my lowest option in real iso is 320?
Also, canon digital iso says highly recommended 0, however i can never get it below 1. is this due to 200 iso limit by canon ?
So, if i don't want to go any higher than 1600 my best setup is 200/1600 ?
i was able to get real iso as low as 50? are there disadvantages/advantages to this is relation to dual iso?
You may want to consult your manual about turning off HTP (Highlight Tone Priority) and revisit ISO settings after.
On the 50D, the lowest native ISO appears to be ISO 200. There's some ISO results for the 50D in this thread (http://www.magiclantern.fm/forum/index.php?topic=10111.0). That thread also details things regarding ISO multiples of 160.
Hey guys,
I've used ML for several years and have been getting more into timelapses recently, using auto ETTR to shoot my first holy grail sequence which turned out beautifully. The past few days I started adding dual ISO to the mix and, while the individual images look incredible, I've been having some issues getting it to work seamlessly in timelapses.
I've spent the past couple days searching the forums but I haven't found my issue addressed, which is: doesn't allowing ML to turn dual ISO on and off as needed introduce an incorrectable color/exposure change in the final result due to the extra processing step (cr2hdr)? I just got back from a test holy grail timelapse and compared sequential (10s) images before and after ML enabled dual ISO as it got darker. With identical exposure and raw settings, the two images show a huge change in color and exposure. Also the dual ISO images have a few more pixels (5202x3465 vs 5184x3456). I know my SNR values were probably wrong as it actually disabled dual ISO as it got even darker, but I think the question is still valid in changing light conditions.
I'm shooting on a T2i using the latest ML (2016Jul09), and BarracudaGUI pointed at the cr2hdr executables from the latest zip. First I tried feeding the raw CR2s into Barracuda, and the dual ISO images were extremely different. Then I read someone on the forums say they converted all CR2s to DNGs with ACR, then Barracuda, which turned out closer, but still noticeably different.
Has no one else encountered these problems? Am I just missing something in shoot settings or in post?
Here's a comparison of two of these images, both converted to DNG, run through cr2hdr, and set to same raw settings: http://i.imgur.com/Rg0LpE0.jpg
Thank you for these incredible tools, can't wait to get this working!
I have shot a few timelapses with mixed dual/regular ISO, but didn't manage to render them yet, so I'm not able to fully answer your question right now.
Your sample looks like it needs some deflicker. Try this script: http://www.magiclantern.fm/forum/index.php?topic=8850
If it doesn't do the trick, you might also need a black level adjustment (which can be found with a linear fit), or (hope it's not the case) you might have to use some sort of histogram matching (if the response curves are not exactly linear).
If all else fails, shooting all the pictures in a timelapse at ISO 100/1600, and letting ETTR adjust only the shutter, should result in a set of images easier to deflicker. The --same-levels option in cr2hdr might also help.
These are just my expectations, as I didn't actually try these settings in real-world scenarios.
Thanks for the info. Yes, it needs deflicker, but I'm quite sure the flicker is due to switching to dual ISO; the images on either side of the transition show absolutely no flicker. It's more than an exposure change though; if I correct the brightness I can get it very close, but there is still a subtle difference in color. If possible I want to try to find shoot settings that don't cause that flicker to begin with and give me a very linear post process workflow, so the last option sounds like exactly what I'm looking for. How would I force ETTR to only adjust the shutter without changing ISO? Even with link to dual ISO disabled, ETTR still adjusts the base ISO.
Hello Magic Lantern team,
I'm trying hard to get DUAL ISO to work, without much success.
I must do something wrong in the workflow or i do not understand how to use the feature, although i read a lot of material.
When trying to get most data out of the sensor, I'm shooting with AETTR and dual iso (100-1600).
That way I'm getting a lot of highlight information, but the blacks and shadows are pitch black.
When I try to recover the data, I must pull them all the way to +100 in order to see something in the shadows,
but that of course insert a lot of noise to it. So much i cant use the picture for almost anything.
Am I doing something wrong?
Does the DR is just too big that even DUAL ISO 100-1600 cant handle it?
Should i expouse the image without AETTR?
I'm using 70D, latest CR2HDR, ACR 9.1.1 or LR 6 (same results).
I want to use it so much, help will be very appreciated.
Thanks.
Here is an example of DNG file (after CR2HDR) I cannot bring to life:
https://www.dropbox.com/s/sdj6zgf8u77xken/DUAL0794_100_1600.DNG?dl=0
Thanks.
Quote from: Sooner266 on July 24, 2016, 08:05:14 AM
It's more than an exposure change though; if I correct the brightness I can get it very close, but there is still a subtle difference in color.
I expect a difference in black level - there is some guesswork going on. When switching the ISOs back and forth, the low ISO ends up having a slightly higher black level, and their average is what you get in exif.
If you upload the two raw files that you expect to be identical, I can take a look.
Quote from: Sooner266 on July 24, 2016, 08:05:14 AM
How would I force ETTR to only adjust the shutter without changing ISO? Even with link to dual ISO disabled, ETTR still adjusts the base ISO.
There isn't an option for that right now; the max ISO in ETTR is set in Canon menu, but I doubt you can set it to 100. Editing the source code is probably the best option right now. You can tweak the settings so it will be less likely to select higher ISOs (e.g. disabling SNR and highlight ignore), but this will most likely result in underexposure.
Quote from: a1ex on July 25, 2016, 05:08:50 AM
If you upload the two raw files that you expect to be identical, I can take a look.
Editing the source code is probably the best option right now.
I appreciate it, and I'm sure with enough cr2hdr tweaks we could get those two particular shots to converge, but those tweaks would go out the window if the dual ISO EV ratio (or other settings) changed, wouldn't they? Indeed my lowest max ISO is 400.
Yes I think looking to the shoot code would be a better solution. I'd already forked it to work on a couple more options for ettr timelapses, so I'll also see about locking iso entirely. Just could not get the ubuntu virtual box to compile modules due to its age (not a linux person), so I'm going to try my luck with cygwin.
Anyone can help please to my previous post?
Quote from: Sooner266 on July 25, 2016, 07:06:32 AM
I appreciate it, and I'm sure with enough cr2hdr tweaks we could get those two particular shots to converge, but those tweaks would go out the window if the dual ISO EV ratio (or other settings) changed, wouldn't they?
Of course; I only wanted to find out what kind of mismatch is there (and whether it can be corrected with only a black level adjustment, or it needs something more complex).
Quote from: Sooner266 on July 25, 2016, 07:06:32 AM
Yes I think looking to the shoot code would be a better solution. I'd already forked it to work on a couple more options for ettr timelapses, so I'll also see about locking iso entirely.
Nice! You may want to look at (and build on top of) this branch: https://bitbucket.org/hudson/magic-lantern/branch/ettr-bulb
Quote from: jhon_wane on July 26, 2016, 07:47:59 PM
Anyone can help please to my previous post?
Not sure how else to help other than to assume that this was due to an user error? All I think of is that you most likely shot this Dual-ISO under exposed at least in this case since you're aiming for the shadows, right?
Which converter did you use to spit this DNG out? Do you still have the original Dual-ISO CR2?
According to my tests with your file it tells me that you shot this under exposed by 2-3 stops & possible more (I had to push the exposure up in ACR)? I also applied Cinelog-C just to see how much it helps with noise. Fun stuff that you can even use this for Photo work (Thanks @Andy600) and let alone with Dual-ISO even if it wasn't intended for! ;)
(https://c4.staticflickr.com/9/8815/27952778563_e5f83cfec8.jpg) (https://flic.kr/p/JA6icZ) (https://c4.staticflickr.com/9/8670/28502588851_e08693d7af.jpg) (https://flic.kr/p/KqFdDH)
Perhaps next time try to have the ETTR reads indicating 0.0 (rather than -1.3 or +1.50) on your LCD screen while exposing for ETTR. I've never been a fan of AETTR maybe because I'm just plain lazy and refuse to learn how to use it. I mean who wouldn't want to control the lighting manually?
Remember it's better to protect the shadows than the highlights since it's RAW and you can just pull the exposure down in post to preserve the noise levels (even takes away too) while keeping the highlights so I try to keep this method in mind when I go out and about.
Thanks,
I know I must be doing something wrong.
Here is the original file:
https://www.dropbox.com/s/z4y1nw4anvbd5wm/DUAL0794.CR2?dl=0
I use CR2HDR latest versions in order to convert it to DNG (using BarracudaGUI).
Can you please help me and explain what is the best workflow for taking a picture using DUAL ISO? I'm very confused.
I understand that I should expose for the lower (base) ISO.
I thought I need to get all the highlights without clipping (that is why I used AETTR) and the recovery ISO (1600) will take care for the shadows - Am I wrong (apparently - yes..)?
So...
Should I expose for the highlights? shadows?
Should I avoid any clipping - shadows and highlights?
Sorry for the newbie questions,
10x again.
There,s actually nothing wrong with the file. Try set exposure for correct highlights(dark shadows) and set something like 100-1600 for dual iso. Contrasty situations.
In post pull out the shadows and pull down the highlights. Test, test, test. And post your results.
Quote from: jhon_wane on July 26, 2016, 09:22:30 PM
Should I expose for the highlights? shadows?
Should I avoid any clipping - shadows and highlights?
Only you can decide what to clip and what to keep in the picture. Raw zebras will tell you exactly what is completely clipped (pure white), or only one or two color channels clipped (can be recovered if color accuracy is not critical), or too noisy (SNR below some threshold). When you enable Dual ISO, the RAW-based zebras will also tell you which parts of the image are covered only by the lower ISO (that is, where it's likely to get aliasing artifacts).
For ETTR, in this particular case, you could allow it to clip some more highlights (to get better shadow detail). The issue here was the sun being much larger than your highlight ignore threshold.
I have in mind some changes to ETTR that would identify such strong highlights and clip them, but didn't manage to implement them yet. Will keep your CR2 to use it as test data.
Thanks a lot guys-
for the tips and for your time trying analyze my picture.
I will keep trying and update as soon as I can.
Cheers.
Id the cr2hdr.exe from Dec. 7, 2013 the "latest / greatest"?
Seems like there was a 20-bit semi-experimental version that was supposed to be compiled. Or is that the Dec 7, 2013 version?
You want to read first post of this thread ...
hello
I´m quite new to dual iso, maybe the issues I have here were discussed before .. I have 2 problems with my dual iso shots:
1. some merged frames differ quite much in brightness but I can´t see that in the the original dual iso dngs. It appears for 20 or so frames in a sequence every few 100 frames. I uploaded exaple dngs here (original and merged) https://www.dropbox.com/sh/dvwp3f2xkpkzlmh/AAAMo2z1bBiuxXjhvDHYMw2Ha?dl=0
2. the merged files appear to have a color temperature shift towards blue. That´s not really a big problem but maybe there´s an easy way to fix this..
thanks
Hi! I have 2 questions:
1:
Is possible to combinate Canon Highlight Tone Priority mode with Dual ISO for max DR?
2:
And... is there any new improvments with shadows/highlites/blacks without loosing details and informations? - It's really BIG FAIL
1. No (There are reasons why we've been instructed to have HTP off).
2. Not sure what ur asking but it depends on which ISO combo (100/800 or 100/1600) you use along with how you expose it and also depends on which camera you use
2. Seems like he wants to have Dual-ISO benefit without trade-off.
Dual ISO with actual performance is not good for any types of hobby or professional photography... This "massive dr improvments" is like "massive miss of the best shots" I want the best performance as possible from my gear (EOS 700D) and with Dual-ISO is like lottery for few geeks. Not for hobby enthusiastic photographers or... professionals.
Guess that makes me a Geek that won the lottery. All my years as a professional photographer doesn't count?
Of course Dual ISO works. You just need to know how to use it and just as important, when it is better not to use it.
Sent from my iPhone using Tapatalk
+1 @dfort
Quote from: xorpi on September 30, 2016, 06:52:58 PM
Dual ISO with actual performance is not good for any types of hobby or professional photography... This "massive dr improvments" is like "massive miss of the best shots" I want the best performance as possible from my gear (EOS 700D) and with Dual-ISO is like lottery for few geeks. Not for hobby enthusiastic photographers or... professionals.
I have sold thousands of Dual ISO photos. I'd recommend you to experiment a lot, compare results, refine your exposure calculations, use zone system for metering areas, anticipate where your shadows will fall after running cr2hdr, and start again until you find your method. Once you know where to measure and in which zone the opposite side of the histogram will fall you'll be able to predict results.
Check my portfolio www.ramonclemente.com/portfolio (http://www.ramonclemente.com/portfolio), most of the photos are Dual ISO.
Spot on, @dfort and perhaps it takes a true geek to at least understand how to use (or not to use) such features properly, right? [emoji6]
Simple trial and errors goes a long way and for good reasons as @R said it perfectly!
Thanks for the friendly reminder @xorpi!
Quote from: xorpi on September 30, 2016, 06:52:58 PM
Dual ISO with actual performance is not good for any types of hobby or professional photography...
This is just FUD. I'm using dual_iso since it was released and use it on most outdoor shots nowadays. When in doubt, a little wasted dr and semi-clipping with dual_iso beats completely blown highlights and low-res blacks drowned in noise.
With recent Canon sensors ml supports (like 6d) you get excellent iq, 16bit raw files and can actually match the dynamic range of the scene and the image by selecting the correct iso spread (100/200, 100/400, ...) = optimal bit-res usage. Even on older sensors like the 18mp crop the shadow noise is ok up to 100/800, but with hugely improved dr.
As you need to back up your shots anyway, there's no additional file space requirement (back up the dual_iso dng, work on the cr2hdr-processed dng). The only hassle is the broken auto-wb, the striped preview and the time to process the files.
QuoteThe only hassle is the broken auto-wb
Reenter CR2 white balance back into a dual iso dng can be done quite easily. Every CR2 file stores WB RGGB Levels As Shot. For instance:
WB RGGB Levels As Shot : 1739 1024 1024 19311024/1739=0,588844163312248
1024/1931=0,528107271789582
Then inject the tag with exiftool.
exiftool "-AsShotNeutral=0,588844163312248 1 0,528107271789582" dualiso.dng
I don,t know how to do this in C but since the CR2 is always there in front of you I don,t see why this can,t be done inside cr2hdr.
Calculations coming from Chmee and a lot of trial and error from my part when building scripts.
Quote from: Danne on October 01, 2016, 01:38:20 AM
Reenter CR2 white balance back into a dual iso dng can be done quite easily. Every CR2 file stores WB RGGB Levels As Shot.
I don,t see why this can,t be done inside cr2hdr.
Agreed. This would be nice. Should be doable.
--wb=exif
;)
See also http://www.magiclantern.fm/forum/index.php?topic=10265 (in particular, posts from ayshih and Marsu42)
Last time I went through --wb=exif this was happening :).
http://www.magiclantern.fm/forum/index.php?topic=16024.msg155670#msg155670
I since deleted that particular CR2 but I could probably dig it out from some folder if necessary. In the thread is the solution coming from using dcraw -i -v multiplier method.
Here, --wb=exif gives the same results as dcraw -i -v. I probably missed your post back then, so please re-upload the file (or other file that has the same issue).
I will see If I can find that particular file or if I can reproduce the issue otherwise. Meanwhile.
Tested just now three files like this.
The auto wb CR2 file might be worth checking. All three files except AWB was giving similar results as dcraw -i -v.
https://drive.google.com/file/d/0B4tCJMlOYfirLVVXZXIzNnQzOTA/view?usp=sharing
Checked in acr
Auto
CR2 6400 +35
DNG 6050 +27 (dcraw -i -v)
DNG 3850 +10 (--wb=exif)
2500 kelvin
CR2 2500 +3
DNG 2400 -1
10000 kelvin
CR2 10000 + 9
DNG 9300 +1
Auto (dcraw -i -v through MLP)
6050 +27
(https://s16.postimg.org/v23yyrd9h/Screen_Shot_2016_10_01_at_09_24_41.png)
Auto (--wb=exif)
3850 +10
(https://s16.postimg.org/5s97bo6hx/Screen_Shot_2016_10_01_at_09_26_11.png)
Here we go. Found the CR2. Issue reproduced.
Problematic file
https://drive.google.com/file/d/0B4tCJMlOYfirb1FqTzVNRXZzVkk/view?usp=sharing
dcraw i -v
(https://s13.postimg.org/x1l4qtxbr/Screen_Shot_2016_10_01_at_10_02_50.png)
--wb=exif
(https://s13.postimg.org/j96pv76k7/Screen_Shot_2016_10_01_at_10_02_02.png)
exiftool DUAL2695.CR2 | grep RGGB
Measured RGGB : 144 1024 1024 1136
WB RGGB Levels As Shot : 2036 1024 1024 1990
WB RGGB Levels Auto : 2036 1024 1024 1990
WB RGGB Levels Measured : 2036 1024 1024 1990
WB RGGB Levels Daylight : 2001 1024 1024 1667
WB RGGB Levels Shade : 2300 1024 1024 1421
WB RGGB Levels Cloudy : 2153 1024 1024 1535
WB RGGB Levels Tungsten : 1440 1024 1024 2485
WB RGGB Levels Fluorescent : 1753 1024 1024 2389
WB RGGB Levels Kelvin : 1324 1024 1024 2774
WB RGGB Levels Flash : 2231 1024 1024 1511
Raw Measured RGGB : 219 243 2608 1555
WB RGGB Levels : 2036 1024 1024 1990
cr2hdr --wb=exif:
White balance : from RawMeasuredRGGB
AsShotNeutral : 0.90 1 0.60
ayshih's comments in exiftool-bridge.c:
//If WB mode is not Auto, use WB_RGGBLevelsAsShot values
//If WB mode is Auto, read WB_RGGBLevelsMeasured values
// (not WB_RGGBLevelsAuto values because they can be temperature-shifted)
// Use WB_RGGBLevelsMeasured values if they have a significant difference between the G channels
// Otherwise, use RawMeasuredRGGB values
https://bitbucket.org/hudson/magic-lantern/pull-requests/453/white-balance-fix-for-dual-iso/diff
Looks like what works on one camera may cause trouble on others, so I think we need tricky test cases for more camera models in order to solve this puzzle.
What are we looking for? Checking auto wb cr2hdr seems to use dcraw auto wb when camera set to AWB(?) But viewing CR2 files you can match AWB output much closer with multipliers. The numbers aren,t exact but it seems close enough. To me :).
Here is the same awb test from an eos M.
Original CR2 (5050 +29)
dcraw -i -v (4650 +26)
(https://s22.postimg.org/scs6c7ds1/Screen_Shot_2016_10_01_at_10_32_30.png)
cr2hdr --wb=exif (3800 +18)
(https://s22.postimg.org/4lxur7jwh/Screen_Shot_2016_10_01_at_10_31_56.png)
I bet ayshih did this change for a reason, but he tested on 50D. Here's the exif info from his sample file:
http://www.magiclantern.fm/forum/index.php?topic=10265.msg107458#msg107458
Aha. A few samples from a 50D to start with then. Anybody?
Auto WB, 2500K and 10000K?
50D samples:
---------------
WB auto, dual ISO enabled 100/800 (https://dl.dropboxusercontent.com/u/44995840/WB-50D/IMG_3550.CR2)
WB auto, dual ISO disabled (https://dl.dropboxusercontent.com/u/44995840/WB-50D/IMG_3551.CR2)
WB 2500K, dual ISO enabled 100/800 (https://dl.dropboxusercontent.com/u/44995840/WB-50D/IMG_3546.CR2)
WB 2500K, dual ISO disabled (https://dl.dropboxusercontent.com/u/44995840/WB-50D/IMG_3547.CR2)
WB 10000K, dual ISO enabled 100/800 (https://dl.dropboxusercontent.com/u/44995840/WB-50D/IMG_3542.CR2)
WB 10000K, dual ISO disabled (https://dl.dropboxusercontent.com/u/44995840/WB-50D/IMG_3543.CR2)
Same scene, with an EOS M:
---------------------------------
WB auto, dual ISO enabled 100/800 (https://dl.dropboxusercontent.com/u/44995840/WB-EOSM/IMG_1624.CR2)
WB auto, dual ISO disabled (https://dl.dropboxusercontent.com/u/44995840/WB-EOSM/IMG_1625.CR2)
WB 2500K, dual ISO enabled 100/800 (https://dl.dropboxusercontent.com/u/44995840/WB-EOSM/IMG_1626.CR2)
WB 2500K, dual ISO disabled (https://dl.dropboxusercontent.com/u/44995840/WB-EOSM/IMG_1627.CR2)
WB 10000K, dual ISO enabled 100/800 (https://dl.dropboxusercontent.com/u/44995840/WB-EOSM/IMG_1628.CR2)
WB 10000K, dual ISO disabled (https://dl.dropboxusercontent.com/u/44995840/WB-EOSM/IMG_1629.CR2)
Thanks josepvm. I,m on a limited time schedule atm but I managed to work your files. The relation is the same. Auto wb dcraw -i -v match much closer than --wb=exif.
CR2 5750 +35
DNG 5900 +34
DNG 4100 -7 --wb=exif
Conclusion over here is that reliable number to work with are the WB RGGB Levels As Shot values. The other stuff, at least, from my testing way back, I couldn,t make real sense of. So, what steps ahead? Getting back Ayshih in the discussion?
Read through some more interesting things and I wonder if this would be the solution? The missing matrices.
http://www.magiclantern.fm/forum/index.php?topic=10265.msg118799#msg118799
*Added all the recommended tags but it didn,t alter acr white balance settings.
QuoteThe solution is to include the ColorMatrix2, CameraCalibration1, CameraCalibration2, CalibrationIlluminant2, ForwardMatrix1 and ForwardMatrix2 values for the camera in the DNG file. It also requires setting CalibrationIlluminant1 to 'Standard Light A' and CalibrationIlluminant2 to D65.
This?
Yes. Was trying adding the more complete set of tags to see if that would refine white balance interpretation in acr but made no difference.
Despite the very long and interesting thread around white balance through other calculations than the simple WB RGGB Levels As Shot division I cannot see any closer match than coming from those numbers.
I digged a little further and I think I,m on to something. The camera calibration tags seems to be the final piece of what is needed for that final calibration of white balance. This tag is different for every camera and you obtain it from a converted dng file through good old adobe dng converter. Notice both camera calibration tags are the same.
CR2 4800 +7
DNG 4600 +6 (without missing tags)
DNG 4800 +7 (with added tags) Exact match!
(Calibration numbers possible to include to AsShotNeutral tag with simple addition? Havn,t tested)
*update (Calibration numbers possible to include to AsShotNeutral tag with simple division tested working)
exiftool "-CameraCalibration1=1.0255 0 0 0 1 0 0 0 0.9766" /Users/Daniel/Desktop/untitledfolder/O42A0230.DNG
exiftool "-CameraCalibration2=1.0255 0 0 0 1 0 0 0 0.9766" /Users/Daniel/Desktop/untitledfolder/O42A0230.DNG
These tags were added. Now next we could see if any tags could be excluded.
Color matrix 1
Color matrix 2
CameraCalibration1(cameraspecific)
CameraCalibration2(cameraspecific)
ForwardMatrix1
ForwardMatrix2
CalibrationIlluminant1 to 'Standard Light A'
CalibrationIlluminant2 to D65
This should conclude the WB RGGB Levels As Shot to be the correct value to calculate dng multipliers from.
On a sidenote wouldn,t it be possible to match perfect muliplier numbers in mlvfs and mlp(mlv_dump), raw2cdng if knowing cameracalibration value?
Hi... and any new performance update for blacks/highlites/shadows?
What problem(s) do you have with current version? (images, not grumbling (http://www.magiclantern.fm/forum/index.php?topic=7139.msg172736#msg172736))
Please read this (http://www.chiark.greenend.org.uk/~sgtatham/bugs.html) and this (http://www.catb.org/~esr/faqs/smart-questions.html) before replying.
Images:
https://www.flickr.com/photos/martinfrano/23185583551/in/album-72157660572707536/
https://www.flickr.com/photos/martinfrano/23669973776/
- Less details and sharpeness on black/shadows areas of these photos. It's like blur or something... - under the castle and second photo - in the dark/shadow areas...
and chromatic abberations:
https://www.flickr.com/photos/martinfrano/22044619463/
CANON EOS 700D.
You didn't read the above links, did you?
Less details and chromatic abberations compared to what?
Please upload the original CR2 and the reference images (ground truths) for the defects mentioned.
Chromatic aberration is a defect of the lens. How could Dual-ISO introduce chromatic aberration?
And some loss of vertical resolution in extreme shadows and highligths is an unavoidable tradeoff of the Dual-ISO method. But the loss is not very high, and the overall result is a lot better than having all the shadow's noise that you get without Dual-ISO.
But if you need maximum detail, the solution is simple: don't use Dual-ISO and shoot bracketed exposures instead.
I suppose he confuses aliasing artifacts and chromatic aberrations.
Yes.. I need maximum details :D LOL...
Please, you should approach this according to the development plan of the whole product and focus on best parts. Everyone who will come explore magic lantern do so because of ability to encrease dynamic range using dual-iso, that is still not finished properly. More like a toy than professional instrument. Its sad that programmers who developed this do not wish to optimize this for production use.
My recommendation:
Optimize and finalize dual-iso (without loosing resolutions and details) as format for ML platform with dynamic range boost. With that use AutoETTR system for dual-iso (as replacement for HDR Bracketing) - Dual-iso as native format of the Magic Lantern firmware.
Integrate various modules from other developers on the forum directly to the Magic Lantern - great one is Focus Stacking + Autofocus + Intervalometer.
Along with that please consider usage and planning of modules emulation in runtime to view results from dual-iso. Eventually Android app which can view this as Android app is connected.
In case you would be able to finish this during one year, which is no problem for other dev teams, Magic Lantern would become good firmware for normal use, not 30 unusable experiments.
Have a nice day ;)
Quote from: xorpi on October 03, 2016, 03:27:24 PM
My recommendation:
Optimize and finalize dual-iso (without loosing resolutions and details) as format for ML platform with dynamic range boost.
With that use AutoETTR system for dual-iso (as replacement for HDR Bracketing) - Dual-iso as native format of the Magic Lantern firmware.
That's impossible. Dual-ISO uses half of the sensor rows to capture extreme shadows and highlights. And this produces an unavoidable loss of resolution on those areas. There's no way to optimize anything to change this fact.
You are asserting that Dual-ISO is not optimized and not prepared for serious work, but you don't have any idea of how Dual-ISO works, and It seems that you don't care at all of trying to understand it.
Dual-ISO is optimized and finished enough for a lot of users here, we use it frequently, and take great advantage of it. But Dual-ISO is not a replacement for bracketing when you need to use the full resolution of your camera sensor.
And you are not providing any data and any help to the developers to improve the flaws you say Dual-ISO has.
Quote
Integrate various modules from other developers on the forum directly to the Magic Lantern - great one is Focus Stacking + Autofocus + Intervalometer.
Experimental modules need to be tested and fully implemented for all ML supported cameras, before being added to nightly builds. Otherwise there is a big risk of bricking cameras. And I suppose you will not be happy if you see your camera bricked...
But you have said in another thread that you don't care at all about helping the ML community with programming, building, and testing experimental code, you only want a finished product, for free, and in less than a year ...
So the conclusion is: you are a troll, and with your posts you are only making the developers to waste their valuable time. And you are using this forum to insert links to your photo galleries, advertising yourself for free.
You are not apporting anything good to this comunity, so don't expect to receive any consideration or any help here.
This is the last time that I waste my time also, replying to your messages.
Quote from: josepvm on October 03, 2016, 04:25:55 PM
So the conclusion is: you are a troll, and with your posts you are only making the developers to waste their valuable time. And you are using this forum to insert links to your photo galleries, advertising yourself for free.
You are not apporting anything good to this comunity, so don't expect to receive any consideration or any help here.
This is the last time that I waste my time also, replying to your messages.
+1
Quote from: xorpi on October 03, 2016, 03:27:24 PM
Please, you should approach this according to the development plan of the whole product and focus on best parts. Everyone who will come explore magic lantern do so because of ability to encrease dynamic range using dual-iso, that is still not finished properly. More like a toy than professional instrument. Its sad that programmers who developed this do not wish to optimize this for production use.
@xorpi before you start acting all bossy and blaming magic lantern for your unsatisfied photos, start small with the stuff that u can fix. Maybe, I don't know, work on your stamping for example in your post-process...
Quote from: josepvm on October 02, 2016, 02:38:46 PM
Chromatic aberration is a defect of the lens. How could Dual-ISO introduce chromatic aberration?
In theory, I guess it's possible that cr2hdr confuses that ca-reduction algorithm of some postprocessing software which are tuned for specific, untouched raw files, i.e. dual_iso shots could have more ca left. However, I have never observed this effect myself (using acr) even with cheap fast lenses, but I didn't try to provoke it either to prove or disprove the possibility.
Quote from: josepvm on October 02, 2016, 02:38:46 PMSee also http://www.magiclantern.fm/forum/index.php?topic=10265 (in particular, posts from ayshih and Marsu42)
Good advice, that :-p ... I faintly remember me stating that the color reproduction of very different iso levels (like iso100/iso1600 on a crop camera) is a source of confusion in any case, i.e. you cannot expect this to behave like a perfect iso100 bracketing-hdr shot with expanded dynamic range.
Alas, I admit I was nitpicking when I started using dual_iso, but since I do nature shots and no one except me knows how the original scene looked anyway a perfect color reproduction or wb doesn't matter that much to me. When in doubt about the wb, the "auto" setting in acr (i.e. Lightroom or Photoshop) works just fine as a starting point, and not getting blown whites or drowned blacks is more valuable than anything else.
These are the tags which seems necessary for correct white balance(check below). The other tags mentioned before aren,t needed. I,m trying to fetch CameraCalibration tags in CR2 but somehow it only shows when converted in adobe dng converter. If anyone have any clues here? I tried checking here http://lclevy.free.fr/cr2/
CameraCalibration1(cameraspecific)
CameraCalibration2(cameraspecific)
CalibrationIlluminant1 to 'Standard Light A'
CalibrationIlluminant2 to D65
What is essential too is the WB RGGB Levels As Shot tag in the CR2 file. Put the multiplier in "-AsShotNeutral" tag. Could look like this.
ex. 1
WB RGGB Levels As Shot : 1464 1165 1165 2725
1165/1464=multiplier
1165/2725=multiplier
ex 2
WB RGGB Levels As Shot : 1165 1024 1024 3226
1024/1165=multiplier
1024/3226=multiplier
I have updated MLP with a new bash script which will produce the camera calibration number if adobe dng converter is installed.
Here,s the bash script
https://drive.google.com/file/d/0B4tCJMlOYfirQnZTUWN4UVpSOXc/view?usp=sharing
Dude this is smoking hot and thanks for sharing the bash script @Danne!
I wonder if it is actually due to missing specific tag(s) because we "haven't told" cr2hdr what's to look for in the first place prior to processing Dual-ISO files and only if we had knew then this WB issue would be resolved somehow, right?
Quote from: Danne on October 04, 2016, 06:01:40 AM
These are the tags which seems necessary for correct white balance(check below). The other tags mentioned before aren,t needed. I,m trying to fetch CameraCalibration tags in CR2 but somehow it only shows when converted in adobe dng converter. If anyone have any clues here?
Doh - that's closed source for you. There have been attempts to get the correct wb from tags here before, all doomed b/c obviously we lack the secret sauce ingredient :-\
Adobe seems to have managed to make use of the maker notes, probably using some insight straight from Canon we don't have. It might even be that there's no single algorithm to convert the values, but actually a model-specific lookup table as for new cameras there has to be a new acr version.
Quotewe lack the secret sauce ingredient
It,s only the Camera calibration tag numbers that are missing. Once they are put together with the above mentioned tags and the WB RGGB Levels As Shot value recalculated to dng multipliers everything seems fine. Are you saying Camera calibrations tags are matrixed and calculated in a secret sauce :) revealed in a dng file or are they maybe hardcoded into the cr2 coming straight from the factory where it was calibrated but only told to adobe?
hey guys just if someone feels like replying to my message at least with something like "you can find solutions to this in this forum" or "your camera seems to be broken" then I would might have a hint how to go on from here.. thanks
Quotehello
I´m quite new to dual iso, maybe the issues I have here were discussed before .. I have 2 problems with my dual iso shots:
1. some merged frames differ quite much in brightness but I can´t see that in the the original dual iso dngs. It appears for 20 or so frames in a sequence every few 100 frames. I uploaded exaple dngs here (original and merged) https://www.dropbox.com/sh/dvwp3f2xkpkzlmh/AAAMo2z1bBiuxXjhvDHYMw2Ha?dl=0
2. the merged files appear to have a color temperature shift towards blue. That´s not really a big problem but maybe there´s an easy way to fix this..
thanks
Seems that white level isn,t the same. Usually the problem when there is flicker in dual iso files. If you,re on mac you can use MLP http://www.magiclantern.fm/forum/index.php?topic=13512.0
If on windows not sure how raw2cdng or MLVFS(windows version) handles these problems.
You can also apply same white level in post with for instance exiftool. Search here at the forum. Should be several posts here about it. Mlvproducer also handles dual iso files for windows. And of course cr2hdr itself has a same level switch which could be used.
thanks Danne! I´ll check it..
Quote from: Armetz on October 04, 2016, 02:51:04 PM1. some merged frames differ quite much in brightness
I browsed the files in your merged "dual" folder and everything's fine and as it's supposed to be :-o
Quote from: Armetz on October 04, 2016, 02:51:04 PMthe merged files appear to have a color temperature shift towards blue
The wb of dual_iso is dodgy, you need to set a manual wb which is no problem as it's lossless on raw data.
Concerning the dual_iso dynamic range: Where does cr2hdr get the data to calculate the theoretical dynamic range from? I had a glance at the source code, and it seems to have to do with black/white-points and some mysterious "dark noise", but I don't understand it.
I'm asking b/c the 7d has higher dynamic range than the 60d according to dxo data, but the output of cr2hdr shows a little less. The resulting difference is nothing to write home about, even a 5d2 isn't a lot different at 100/800 according to cr2hdr output (but full frame can be used at 100/1600 while crop falls apart).
http://sensorgen.info/CanonEOS-7D.html
http://sensorgen.info/CanonEOS-60D.html
But I'm wondering: Why doesn't dxo's nominal higher dr @100 plus highrer dr @ 800 add up to a better dual_iso result than on 60d? Is it b/c of the higher noise of the older 7d1 (it has more "pattern" noise, too)? Or is the photographic reality for preventing clipped highlights and raising shadows completely different from what cr2hdr says anyway?
It looks at the optical black area (see dcraw -4 -E) and computes the noise levels from there, assuming that's the read noise. Then, it uses the simplified DR formula: log(white-black) - log(black_stdev), which is another approximation.
Note the stdev is heavily influenced by hot pixels. Maybe I should replace it with some robust statistic (https://en.wikipedia.org/wiki/Statistical_dispersion).
Quote from: a1ex on October 16, 2016, 10:18:41 AM
It looks at the optical black area (see dcraw -4 -E) and computes the noise levels from there, assuming that's the read noise. [...] Note the stdev is heavily influenced by hot pixels. Maybe I should replace it with some robust statistic (https://en.wikipedia.org/wiki/Statistical_dispersion).
So basically, this means the dr given by cr2hdr doesn't really reflect the actual dr of the image. No big deal as the picture itself is important, but it's a puzzler b/c the read noise spec from sensorgen of the 7d @ iso 100 is way lower than the 60d, but in cr2hdr it's the other way round. The 7d is said to have more "pattern" noise, maybe that is an influence on the stdev, too, like hot pixels?
Btw: What's up with the 20bit version, I'm using that one for ages, but it isn't merged. Was the reason that it's a lot slower but doesn't show reasonable iq gain from the 16bit version, or are there any other problems left with 20bit?
Yes, pattern noise increases the stdev as well. Do you have some sample images to show the difference?
The 20-bit version is merged.
Are you asking about those few changes from here (https://bitbucket.org/hudson/magic-lantern/branch/cr2hdr)? These are not merged because they didn't pass my test suite.
Quote from: a1ex on October 16, 2016, 11:16:01 AM
Yes, pattern noise increases the stdev as well. Do you have some sample images to show the difference?
Well, I can send you two random 100/800 shots from the 60d and 7d1 that show the different "theoretical dr" line. But as my 60d is currently broken (I'll have someone look at the fuses, but that'll take time) I can only give you real "requested" test images from the 7d1.
What would be interesting to know which one of all the crop sensors before the 80d is how good for dual_iso. Sure, maybe the difference is minor and maybe 0,5ev, but if someone like me really wants Canon crop with enhanced dr and doesn't really care which old model to buy, it might matter.
Again, only looking at the sensorgen/dxo data the 7d1 should do best... even though it has a bad rep b/c of the pattern noise (which is on 5d2, too). I haven't tested how much the pattern noise affects dual_iso and the amaze algorithm, as it only appears visibly after pushing the shadows a *lot* which shouldn't be necessary using dual_iso.
Quote from: a1ex on October 16, 2016, 11:16:01 AM
The 20-bit version is merged.
Wupps, I missed that :-o
Quote from: a1ex on October 16, 2016, 11:16:01 AM
Are you asking about those few changes from here (https://bitbucket.org/hudson/magic-lantern/branch/cr2hdr)? These are not merged because they didn't pass my test suite.
I didn't try these either, but I'm keeping all my original non-cr2hdr'ed shots around in case some improvements are made :-)
Good evening,
I did not read 115 pages, but today I played with the Dual-ISO feature and got an interesting banding issue when using the new 20bit conversion.
Here the result with the classic cr2hdr:
(https://s12.postimg.org/tet4fqkot/normal.jpg)
Here the result when converting with the cr2hdr-20bit :
(https://s22.postimg.org/8d3hbaa0x/20bit.jpg)
My question now is: Why do I get such a bad result with the 20bit converter?
Best regards,
Chris
20bit is merged into a unified cr2hdr. Could you tell us what build you are using and what program used to convert? Upload the original footage?
Sure,
I used the version of cr2hdr from here:
http://www.magiclantern.fm/forum/index.php?topic=11056.0 (http://www.magiclantern.fm/forum/index.php?topic=11056.0)
So the setup was:
Lightroom Dual ISO Converter 2.1
Lightroom Dual ISO Converter (cr2hdr) plugin (v3.0 BETA3).
Adobe Lightroom CC Version 2015.7
In the plugin I have chosen (via the export dialog):
1) For the first picture "cr2hdr (classic)"
2) For the second picture "cr2hdr-20bit (Experimental)"
The original footage can be found here:
http://www.file-upload.net/download-12038109/IMG_5983.CR2.html
I had a lot of pictures, but as the other conversions were all OK, I saw the banding issue with the 20bit version in this picture. So I thought this report might be helpful in development.
You are using an outdated plug-in. Newest one does not have a 20-bit option (because 20bit version is the only one supported after merging).
Most likely you are using outdated cr2hdr binaries, too.
Visit https://bitbucket.org/kichetof/lr_cr2hdr/downloads and download cr2hdr.3.0BETA3.3.lrplugin.zip
Oh, thank you for this information,
I only found the thread mentioned, but did not know that there is a newer version of the plugin available. With the supplied cr2hdr (in the bin directory) [Version 1779727 from 2015-09-30] all is working well. But I do not know if it is now 20bit or not, because there is no option anymore to choose from.
By the way: I also cannot compress the DNG file. Is this normal that I can not choose this from the options? (grayed out)
As said before: There is no other version. 20bit version has been merged/unified and is the only one around.
And if you are using LR you don't have to compress the file. LR will do it after changing metadata (we do not know why LR does it and if is unintended or not). It's lossless compression, however. No lossy compression with LR available.
Perfect. Thank you so much for your help. I think hereby all my questions are answered. Maybe there is a possibility to link to the newest version, because searching for DualIso will lead you to the thread where the old version is linked.
So, thanks again.
Dankeschön Walter :-)
If on mac there,s a program I started building here if you want to try it out feel free to do so.
https://bitbucket.org/Dannephoto/cr2hdr
Forum thread
http://www.magiclantern.fm/forum/index.php?topic=15108.msg146822#msg146822
Regarding compressing options they require you install adobe dng converter.
Thanks Danne for your suggestion. Unfortunately I do not have a MA. But maybe this will be useful one day...
Is it available for Canon 5D classic?
As the first post reads "most cameras" that does not mean all Canon cameras, right? What about the old 5d?
Quote from: mosaster on November 09, 2016, 04:41:21 PM
Is it available for Canon 5D classic?
As the first post reads "most cameras" that does not mean all Canon cameras, right? What about the old 5d?
It's no longer maintained but there's a working build available here check it out
http://builds.magiclantern.fm (http://builds.magiclantern.fm)
Ps: Not sure if the dualiso module works on it though.
Hey Guys!
is there a special way to convert the CR2s?
I'm not sure if somebody mentioned to use the cr2hdrConverter + some extra software to get better colors etc..
Reply #2860 shouldn,t be too far away?
When I export photos with cr2hdr, the file becomes 44M.What information is added？Interpolated pixels?
If you use dual ISO without cr2hdr decoding can get 16bit uncompressed photos,why is the photos using dual ISO taken as the same size as the photos taken without dual ISO?Does it say that the increase in bit depth does not cause an increase in the size of the photo?Or increase the value of bit depth equal to the loss of pixels?
Dual-ISO is 14 bit digital output.
Pixel interpolation is part of de-mosaicing process and not applied to RAW/DNG.
Canon's CR2 files do have losless compression. Therefore you will have smaller file sizes capturing uniform scenes (fog) compared to scenes where compression fails (sand).
ML Dual-ISO DNG does not do compression. Therefore you will get exactly the same file sizes because every single sensor pixel data value is written to file.
Devs are experimenting with losless DNG compression, though.
Thanks!
Now that Dual-ISO is14bit digital output,why use cr2hdr export images, the image becomes 16bit?Is this 16bit interpolated by cr2hdr?
Windows, cr2hdr.exe and wildcards ...
Maybe a way to enable wildcards running GCC (IANAP): https://gcc.gnu.org/ml/gcc-patches/2017-01/msg01299.html
A1ex, is this a solution or is it necessary to compile sources using Visual Studio?
Looks good to me. No idea whether Visual Studio libraries offer a similar workaround.
What I am doing wrong?
CR2 before processing:
(http://download.maydanik.de/cr2_zoom_fit.JPG)
Crop
(http://download.maydanik.de/cr2_zoom_100.jpg)
After processing in cr2hdr
(http://download.maydanik.de/after_processing.JPG)
Log:
cr2hdr: a post processing tool for Dual ISO images
Last update: ab1e90c on 2015-11-24 09:50:19 UTC by a1ex:
cr2hdr: moved safeguard from median_int_wirth to kth_smallest_i...
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
Input file : DUAL0022.CR2
Camera : Canon EOS 5D Mark III
Full size : 5920 x 3950
Active area : 5796 x 3870
Black borders : 124 left, 80 top
Black level : 2046
ISO pattern : dBBd RGGB
White levels : 15154 13824
Noise levels : 7.87 35.56 34.86 7.83 (14-bit)
ISO difference : 5.92 EV (6038)
Black delta : 1.76
Black adjust : -6
Dynamic range : 10.71 (+) 8.37 => 14.29 EV (in theory)
Looking for hot/cold pixels...
Hot pixels : 81
Cold pixels : 10476
AMaZE interpolation ...
Amaze took 5.14 s
Edge-directed interpolation...
Semi-overexposed: 3.07%
Deep shadows : 96.18%
Horizontal stripe fix...
Full-res reconstruction...
ISO overlap : 3.0 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 30.61 (20-bit), ideally 30.60
Dynamic range : 14.74 EV (cooked)
Black adjust : 2
AsShotNeutral : 0.57 1 0.55, 4308K/g=1.09 (gray max)
Output file : DUAL0022.DNG (already exists, overwriting)
DUAL0022.DNG : copying EXIF from DUAL0022.CR2
Doesn't look like you are doing anything wrong.
Quote from: dmilligan on March 11, 2017, 02:36:28 AM
Doesn't look like you are doing anything wrong.
Here is a link to CR2 file - http://download.maydanik.de/DUAL0022.CR2
if everything right, why the final result too dark?
seems something wrong with bright extraction. Here is snapshot of bright.dng after processing in debug mode.
(http://download.maydanik.de/bright.jpg)
That's the nature of dual-ISO, you need to lift the shadows in post to obtain your end result. Normally lifting shadows this deep results in a lot of noise, but a dual-ISO image has increased dynamic range and hence the shadows are much cleaner. Here's what I get with after dragging a couple of sliders in Lightroom for your image - note the relatively clean shadows combined with retention of highlight detail:
(http://www.redyeti.net/external/DUAL0022-dualiso.jpg)
Quote from: emayd on March 11, 2017, 07:26:30 AM
seems something wrong with bright extraction. Here is snapshot of bright.dng after processing in debug mode.
Looks OK to me - the debug DNGs are saved after the two sub-images are matched in brightness (that is, the bright image is darkened to match the low-iso one).
From help:
dark.dng the low-ISO exposure, interpolated
bright.dng the high-ISO exposure, interpolated and darkened
Quote from: chris_overseas on March 11, 2017, 09:39:57 AM
... Here's what I get with after dragging a couple of sliders in Lightroom ...
I haven't LightRoom - I have Adobe PS. I've used file utility: cr2hdr.exe
Is there any possibility to process without LR?
Lightroom sliders are present in Camera RAW.
Quote from: Walter Schulz on March 11, 2017, 10:39:20 AM
Lightroom sliders are present in Camera RAW.
Does LR plugin works in PS? or where I can found PS plugin for Windows, and how to install it in PS?
Does it enough to copy cr2hdr LR plugin files in PS plugin directory?
Camera raw sliders are present in both lightroom and photoshop ;).
Simply doubleclick your dng. It should open up in camera raw.
Quote from: emayd on March 11, 2017, 01:12:20 PM
Does LR plugin works in PS?
LR and PS do have a very different design there. Plug-ins are incompatible.
Good afternoon. I have a problem - when converting several hundred files taken from DUALISO among which come in the usual way - they get out the "doesn't look like interlaced iso" sign and the process stops before clicking OK. How to make sure that such files are skipped automatically, without confirmation, so that I do not have to spend several hours sitting at the computer.
Tnx.
If you can click OK, that means you are using a GUI wrapper to do the conversion. In this case, please ask in the relevant forum thread for the GUI you are using.
The command-line tool already skips unrecognized files without confirmation.
Oh not, I've been using the old 7d1 for some time now to expand dynamic range of the old-ish sensor, and just discovered that with 800/100 setting there are still stripes after cr2hdr processing :-( ... obviously the algo has trouble telling noise from stripes. I've even discovered some shots from my good ol' 60d affected by this problem.
Strangely, the old 14 bit algo does better, leaving more of the 7d1's vertical noise stripes in the sky instead of dual_iso's horizontal stripes. But even the old algo doesn't do as I'd expect it to.
... 14bit:
(http://der-tierfotograf.de/img-tmp/14bit.png)
... 20bit:
(http://der-tierfotograf.de/img-tmp/20bit.png)
I'll pm the source cr2 to alex or anyone wanting to help, pretty please, don't tell me these shots are broken :-(((
Coming from here:http://www.magiclantern.fm/forum/index.php?topic=19300.msg183053#msg183053
Quote
deafeyejedi
Has anyone notice an issue re: Dual-ISO 14-bit lossless MLV seems to be spitting out with correct black levels but incorrect for original 14-bit MLV's. Could this be related to the new mlv_dump not corresponding properly to cr2hdr or at least confused with each other atm?
https://www.flickr.com/photos/50068152@N05/33870168051
https://www.flickr.com/photos/50068152@N05/33842109492
QuoteAnyway here are some short samples in two of each (138 MB spat out w ease from latest cr2hdr.app -- Thanks @Danne!) and all shot within crop_rec @ 3.5k: https://mega.nz/#!OwV2DTLC!rb39cVOrAlXXuSer2M6fw3vSuvMfiSG508hewIQJvmg
Answer:
I checked your four provided examples. Nice to see compressed files processes fine. Your last two uncompressed 14bit files. The last of the two works fine but the other one is giving incorrect black level(M12-1330.MLV). Probably concidental or can you reproduce this issue?
I Have problem with Canon 700D
when i turn on dual iso module automatically ISOless LV err(8 ) appear on screen
(https://image.ibb.co/g1ZDkk/Capture.jpg)
and the module do not work anymore
WHY THIS HAPPENING ?
any Ideas? a1ex >:(
kidding i love you :D but i need solution plz :'(
https://bitbucket.org/hudson/magic-lantern/issues/2427/700d-dual_iso-isoless-ph-err-15#comment-30000194
(I've played with a 700D last summer, but didn't encounter it)
Anyone else having problems compiling cr2hdr?
https://bitbucket.org/hudson/magic-lantern/issues/2727/cr2hdr-fails-to-build
Quote from: dfort on June 02, 2017, 03:48:45 PM
Anyone else having problems compiling cr2hdr?
https://bitbucket.org/hudson/magic-lantern/issues/2727/cr2hdr-fails-to-build
[EDIT] Never mind, g3gg0 fixed (https://bitbucket.org/hudson/magic-lantern/commits/4126b560c22b9a7e577e0b8862e29adb4e44f1ef) it.
I suppose I'm just too stupid or something: I've just installed the latest ML build an my 6D (Firmware 1.16) but there's no Dual ISO option in the Expo menu? Did I forget something?
Thanks
Modules tab.
Editing this image I noticed a kind of ghost pattern at the bottom that seems to come from the wood table. Is this known and normal?
5D Mark II 2.1.2, Lr 6.7, Dual ISO plugin 3.0-170316-02
(http://thumb.ibb.co/mR4haF/MG_5269_dualiso.jpg) (http://ibb.co/mR4haF)
(http://thumb.ibb.co/fM5Yhv/MG_5269_dualiso_2.jpg) (http://ibb.co/fM5Yhv)
image hosting (http://imgbb.com/)
Try --no-stripe-fix and show me the CR2.
--no-stripe-fix softens the problem but creates new stripes through all the image clearly visible in dark areas.
(http://thumb.ibb.co/kmsW4a/MG_5269_dualiso_2_no_stripe_fix.jpg) (http://ibb.co/kmsW4a)
Here's the original Dual ISO RAW download link:
https://www.amazon.es/clouddrive/share/bg65hvtYaeodNzBKjIODaOb9KuBam1sSEYdZg53b7sC (https://www.amazon.es/clouddrive/share/bg65hvtYaeodNzBKjIODaOb9KuBam1sSEYdZg53b7sC)
Gracias, will take a closer look later. The problem is likely from the ISO matching routine (ideas welcome).
Another example of stripes appearing after cr2hdr. In this case stripes appear in a column about 50px wide in the left third of the image.
(http://thumb.ibb.co/hjyY2v/MG_7461_dualiso.jpg) (http://ibb.co/hjyY2v) (http://thumb.ibb.co/j1hFFF/MG_7461_dualiso_2.jpg) (http://ibb.co/j1hFFF)
Here's the link to the Dual ISO RAW, unfortunately I don't keep the original RAW, just the processed after cr2hdr:
https://www.amazon.es/clouddrive/share/3DjuCaSJLRcLTcP9oKvgjrdH4Hx7PLr9Vwub8q90rTv
Hi guys,
not sure whether this is the right forum or better in the scripting area:
I would like to automatically capture three images in a row:
A Dual ISO with fixed ISO values, then another image with deactivated Dual ISO and low ISO and then the latter with the high ISO. I.e. three images with fixed aperture and exposure etc., but changed ISO and activated and then deactivated Dual ISO.
Did anyone try that out already or can give me a hint on how to script that?
Thanks a lot!
With the lua_fix experimental build, it's as simple as:
-- Dual ISO bracket
menu.set("Expo", "Dual ISO", 1)
camera.iso = 100
menu.set("Dual ISO", "Recovery ISO", "1600")
msleep(500) -- fixme - settings are not applied right away
camera.shoot()
menu.set("Expo", "Dual ISO", 0)
msleep(500) -- fixme - settings are not applied right away
camera.iso = 100
camera.shoot()
camera.iso = 1600
camera.shoot()
Hm, really got to dive into lua. I´d like to build a script that records 2 seconds of MLV of every iso on the camera and when done exits the scripts. I´d use the files for darkframe storage workflows.
Quote from: Danne on August 04, 2017, 05:04:26 PM
Hm, really got to dive into lua. I´d like to build a script that records 2 seconds of MLV of every iso on the camera and when done exits the scripts. I´d use the files for darkframe storage workflows.
+100
Sorry guys tp be a party pooper once again but for some reason I'm having issues trying to get cr2hdr to respond properly to 12/14-bit lossless shot in Dual-ISO.
At first I thought it was an issue directly to the experimental build for SL1/100D but turns out it may be for all cameras capable of recording Lossless compression.
Correct me if I'm wrong but either way here are the sample files from SL1/100D for those who are curious.
14-bit (non-lossless):
1) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%231_SL1_14-bit_1728x972_samples.zip
2) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%232_SL1_14-bit_DUAL-ISO_1728x972_samples.zip
10-bit (non-lossless):
3) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%233_SL1_10-bit_1728x972_samples.zip
10-bit (non-lossless) Dual-ISO:
4) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%234_SL1_10-bit_DUAL-ISO_1728x972_samples.zip
5) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%235_SL1_10-bit_1632x552(60p)_DUAL-ISO_samples.zip
14-bit (Lossless) Dual-ISO:
6) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%236_SL1_14-bit_Lossless_DUAL-ISO_1736x976_samples.zip
12-bit (Lossless) Dual-ISO:
7) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/SL1_12-bit_lossless_Dual-ISO_1800x1008p_Sample.zip
Here's another post from earlier with three more samples: https://www.magiclantern.fm/forum/index.php?topic=16040.msg190646#msg190646
I only recently discovered dual iso, and as you've all been saying for years, it's nothing short of amazing.
My trouble is that in every video I shoot, there's loads of flickering - the exposure seems to change noticeably from frame to frame, making the video seem to 'flicker' when played at normal speed. I've read the entire thread, and haven't been able to find a working solution, apart from faulty black and white levels, which should be fixed by exiftool. I tested this, reading a few of the (cr2hdr) processed DNGs using exiftool, and can confirm that the white and black levels remain constant across frames, including those exhibiting the apparent change in exposure, at 8192 and 65128, respectively.
I suppose this can be changed by constantly altering the exposure in post, but it will be very laborious to have to do it throughout the length of each video, so I'm hoping the latest cr2hdr has an improved algorithm to detect these variations.
All footage is shot on a 5D3, on the October 27 4k_lossless branch build. At first, I processed it with Switch, then to confirm with the mlv_dump + cr2hdr combination, as documented on the original post in this thread, using the --same-levels argument.
I'm on macOS Sierra (10.12.6), and have tried the cr2hdr in the OP, which is quite old, if I remember, as well as the one from Danne's Switch, which is:
Last update: ab1e90c on 2015-11-24 09:50:19 UTC by a1ex:
cr2hdr: moved safeguard from median_int_wirth to kth_smallest_i..
As far as I can tell, there have been a number of changes to cr2hdr.c in the repo, which I've tried building, but end up with build errors:
make cr2hdr
/bin/sh: /Volumes/Data HD/Users/sacha/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3: No such file or directory
[ gcc ] cr2hdr
cr2hdr.c:59:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
MODULE_STRINGS()
^
cr2hdr.c:59:17: error: expected ';' after top level declarator
MODULE_STRINGS()
^
;
cr2hdr.c:573:51: error: use of undeclared identifier 'dual_iso_strings'
printf("Last update: %s\n", module_get_string(dual_iso_strings, "Last update"));
^
cr2hdr.c:1334:46: warning: if statement has empty body [-Wempty-body]
if(system("octave --persist rggb.m"));
^
cr2hdr.c:1334:46: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:1472:46: warning: if statement has empty body [-Wempty-body]
if(system("octave --persist bddb.m"));
^
cr2hdr.c:1472:46: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:1748:51: warning: if statement has empty body [-Wempty-body]
if(system("octave --persist iso-curve.m"));
^
cr2hdr.c:1748:51: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:2184:55: warning: if statement has empty body [-Wempty-body]
if(system("octave --persist fullres-curve.m"));
^
cr2hdr.c:2184:55: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:2518:56: warning: if statement has empty body [-Wempty-body]
if(system("dcraw -d -r 1 1 1 1 edges.dng"));
^
cr2hdr.c:2518:56: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:2531:59: warning: if statement has empty body [-Wempty-body]
if(system("dcraw -d -r 1 1 1 1 edge-map.dng"));
^
cr2hdr.c:2531:59: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:2557:21: error: function definition is not allowed here
{
^
cr2hdr.c:2571:31: warning: implicit declaration of function 'edge_interp' is invalid in C99 [-Wimplicit-function-declaration]
int pi0 = edge_interp(dir);
^
cr2hdr.c:2824:51: warning: if statement has empty body [-Wempty-body]
if(system("octave --persist mix-curve.m"));
^
cr2hdr.c:2824:51: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:3308:55: warning: if statement has empty body [-Wempty-body]
if(system("octave --persist soft-film.m"));
^
cr2hdr.c:3308:55: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:3652:34: warning: if statement has empty body [-Wempty-body]
if(system("octave wb.m"));
^
cr2hdr.c:3652:34: note: put the semicolon on a separate line to silence this warning
11 warnings and 3 errors generated.
dcraw-bridge.c:101:3: warning: unused variable 'unique' [-Wunused-const-variable]
} unique[] = {
^
1 warning generated.
amaze_demosaic_RT.c:983:10: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value
[-Wabsolute-value]
if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;}//a better result was obtained from the neighbors
^
amaze_demosaic_RT.c:983:10: note: use function 'fabs' instead
if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;}//a better result was obtained from the neighbors
^~~~~
fabs
amaze_demosaic_RT.c:983:35: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value
[-Wabsolute-value]
if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;}//a better result was obtained from the neighbors
^
amaze_demosaic_RT.c:983:35: note: use function 'fabs' instead
if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;}//a better result was obtained from the neighbors
^~~~~
fabs
amaze_demosaic_RT.c:1215:10: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value
[-Wabsolute-value]
if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
^
amaze_demosaic_RT.c:1215:10: note: use function 'fabs' instead
if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
^~~~~
fabs
amaze_demosaic_RT.c:1215:35: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value
[-Wabsolute-value]
if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
^
amaze_demosaic_RT.c:1215:35: note: use function 'fabs' instead
if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
^~~~~
fabs
4 warnings generated.
make: *** [cr2hdr] Error 1
Can anyone who's been able to build on the mac help with either the errors, or to share their build environment settings/dependencies? I have installed dcraw and exiftool, and Python's docutils (someone's build failed because of this dependency).
If developing dualiso movie files in Switch the flicker issue should be taken care of by having the same black level added in post.
Thanks for the reply @Danne, I didn't know this.
I do all the post in Resolve, but have not ever changed the black level; I didn't even know about it. Do I set it to the value reported by exiftool?
Do you know where to set it in Resolve?
Sorry for the silly questions...
It's inside the dng metadata but it's all automated in Switch from start to finish. Just head over to (m) or (ms) to get to mlv_dump menu and uncheck disable dualiso automation and then run Switch.
Uncheck dualiso automation? I thought I read that was there to disable dualiso processing, on a folder you know doesn't contain any dual iso MLVs?
Either way, as I wrote, I subsequently processed one MLV manually, using mlv_dump, then cr2hdr manually, as in the original post in this thread, using the --same-levels switch (no puns intended Danne).
With this procedure, I can confirm that white and black levels are stable, across a number of frames, including those where there is an obvious darkening and lightening in exposure - showing up as flicker. That's why I didn't report this as a Switch issue.
So, if the recent (last two years' worth) check-ins to cr2hdr.c have improved the flicker-suppressing algorithm, I'd like to compile and test the latest cr2hdr, but am running into compiling errors (3 posts ago).
Otherwise, it's something I'm doing that's throwing the algorithm off, as no-one else is reporting flickering in all their dual-iso videos.
Can someone help?
The commits after the pre-built binary are these (https://bitbucket.org/hudson/magic-lantern/branch/cr2hdr), mostly about black level handling. They were not included in the build because they failed my test suite back then (and didn't look closer to narrow down).
I don't currently have a better method for handling videos, other than --same-levels, but a few tools borrowed my deflicker algorithm (http://www.magiclantern.fm/forum/index.php?topic=10496.0) lately (darktable, MLVFS, mlv_dump on steroids, maybe others) and might be worth trying. Of course, the algorithm is trivial to port into cr2hdr, just didn't look into it yet.
Quote from: DeafEyeJedi on September 30, 2017, 12:22:35 AM
Sorry guys tp be a party pooper once again but for some reason I'm having issues trying to get cr2hdr to respond properly to 12/14-bit lossless shot in Dual-ISO.
At first I thought it was an issue directly to the experimental build for SL1/100D but turns out it may be for all cameras capable of recording Lossless compression.
Correct me if I'm wrong but either way here are the sample files from SL1/100D for those who are curious.
.....
I had that problems with some clips shooted in dual-iso 12-bit, and I just find the solution using the "batch_mlv_in_out" (thank guys!!!):
https://www.magiclantern.fm/forum/index.php?topic=10526.0 (https://www.magiclantern.fm/forum/index.php?topic=10526.0)
I find a solution by converting the mlv lossless compressed files (using the mentioned mlv_dump batch plug-in) with option 2 (-d) and 4 (specifying 14 bits) ant then give theese 14bit files to the dual-iso process.
Let me know if that works for you too!!
Thanks @a1ex, I'll try mlv_dump on steroids. MLVFS seems random to me; sometimes it works, sometimes it crashes under the load, and it's hard to say which it's going to be until you've waited a long time.
I had no idea darktable had this feature, I must investigate. MLV App definitely has it, and it's an elegant and powerful app, but I can't find any documentation about how the flicker suppression works, though their 'Deflicker target' setting clearly makes a difference, making the footage darker overall, but I haven't been able to fix it altogether yet. I assumed it built off cr2hdr.
Deflicker option will alter only this tag in the dng file:
Baseline Exposure Offset : 0
It´s usually set to 0 but will vary as it deflicker the footage selecting deflicker option. The changes will take affect in acr/lightroom/AE. I thought it´s main purpose was to use it with timelapse footage, not dualiso files. Now if it was ported and used with white level tag for instance it might work but then again same level tag sets the same white level for all the dualiso files preventing it from flicker so the simpliest solution is to use exiftool/exiv2 in or the same level tag in cr2hdr.
Now I think mlvfs is applying the same white level tag to all movie dualiso files so flicker won´t be an issue there, maybe mlv_dump on steroids is doing that too. As I said already, Switch will correct the white level tag as well as keeping the originally shot white balance along with multiprocessing which makes it fast to work with.
Quote from: Danne on November 29, 2017, 05:20:22 AM
As I said already, Switch will correct the white level tag as well as keeping the originally shot white balance along with multiprocessing which makes it fast to work with.
Bang on! ;D
Hi @Danne, @DeafEyeJedi,
I hope neither of you think I was criticising Switch? It's great, it works well, I can set it and walk away, to come back to processed files. Thank you for this software.
My trouble is the constant flickering - on almost every shot I've taken so far - and this is my search to find ways to fix it.
Using Exiftool (and exiv2), I can't see the Baseline Exposure Offset tag as having been set, on DNG (dual iso) files, processed by either Switch, or the mlv_dump + cr2hdr combination.
This is what I do see happening across 2 frames which exhibit flicker - the first of the two is subtly darker than the second one:
App | Frame 513 | Frame 514 |
mlv_dump (unprocessed) | BL: 2047, WL: 16200 | BL: 2047, WL: 16200 |
cr2hdr (with --same-levels) | BL: 8192, WL: 65128 | BL: 8192, WL: 65128 |
Switch (with (s) same-levels) | BL: 8192, WL: 63768 | BL: 8192, WL: 63768 |
MLV App (can't remember deflicker target value) | BL: 8188, WL: 64800 | BL: 8188, WL: 64800 |
As you can see from the above, all three approaches result in changed Black Level and White Level values, relative to the 'original', unprocessed .dng file (out of mlv_dump), and they all differ from each other. However, they are also all equal across frames, resulting in very visible flickering.
In Switch you don't set anything when developing dualiso files. Only drop mlv files in a folder and run it. You need uncheck the before mentioned setting in mlv_dump menu for dualiso automation to get activated. This way you will be able to run dualiso and non dualiso mlv files together. Only then you'll get the non flickering files in Switch.
If you are able to set same levels from a menu in Switch you are suppose to work only with CR2 files, different story.
QuoteUsing Exiftool (and exiv2), I can't see the Baseline Exposure Offset tag as having been set, on DNG (dual iso) files, processed by either Switch, or the mlv_dump + cr2hdr combination.
Forget about the
Baseline Exposure Offset tag and deflicker target values when processing dualiso files.
So, I've been doing it wrong all this time, by going into the (hdr?) submenu.
I' try again, exactly as you suggest, by re-enabling dual-iso automation and running it , with no other options.
Thanks @Danne!
Good luck and let us know how it goes.
Quote from: Danne on November 29, 2017, 05:20:22 AM
Deflicker option will alter only this tag in the dng file:
Baseline Exposure Offset : 0
Yes right no actual raw data is altered.
Member of 'raw_info' struct 'exposure_bias[2]' array is set according to a1ex's algorithm and pushed to DNG header. If this dng tag is not interpreted by raw developing software then deflicker effect is not gonna be noticeable.
Edit: this is the case in the MLV App. Only tag is added to DNG header when exporting codec is cdng. If exporting is not cDNG, then 'Baseline Exposure Offset' is not taken in account by proggie's image processing part ( yet ;) )
Hi @Danne, @bouncyball. I've processed the MLV with Switch again, using no extra flags, and can still 'see' the problem.
On the other hand, what you and @a1ex have said is only now making sense; it's the developing software that I haven't questioned. You're all right that that's where I see the problem manifest itself. In my case, I'm developing these [c]DNG files in Resolve, and that looks like where this (mis)interpretation is happening.
How are the rest of you managing, with dual-iso in Resolve, to hide flickering?
Could you post examples, printscreens, dng files etc?
Sure.
https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000000.DNG
(https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000000.DNG)
https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000001.DNG
(https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000001.DNG)
https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000002.DNG
(https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000002.DNG)
https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000003.DNG
(https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000003.DNG)
https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000004.DNG
(https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000004.DNG)
https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000005.DNG
(https://www.rojoynegroclub.com/images/tmp/M26-1444_1_2017-11-26_0001_C0000_000005.DNG)
In just those 6 frames, you can see at least three exposure changes.
Sorry, unable to recover the input file from output images...
That said, here's a quick deflicker script in octave, that should work on any DNG sequence (after filling in the correct black/white levels):
# exiftool *.DNG -BlackLevel
black = 8192
# exiftool *.DNG -WhiteLevel
# value not critical; todo: read from exif
white = 60000
# EV above black (exposure compensation)
# 0 EV = 1 DN
target_median = 8
for f = dir('*.DNG')'
f.name
im = read_raw(f.name);
g1 = im(1:2:end,2:2:end);
med_g1 = median(g1(:));
med_ev = log2(med_g1 - black);
ec_ev = target_median - med_ev;
ec_linear = 2 ^ ec_ev;
new_white = black + white / ec_linear;
system(sprintf("exiftool '%s' -overwrite_original -WhiteLevel=%d", f.name, new_white));
end
# debug: where's the median in the last image?
# show 12 stops logarithmically
g1log = (log2(max(1, g1 - black)) - log2(white - black) + 12) * 255 / 12;
# mark the median with a red overlay
ovr(:,:,1) = ovr(:,:,2) = ovr(:,:,3) = g1log;
red = ovr(:,:,1);
red(g1 > med_g1 - 10 & g1 < med_g1 + 10) = 255;
ovr(:,:,1) = red;
imwrite(uint8(ovr), 'median.jpg');
# test with:
# dcraw -W -b 4 *.DNG
# ffplay -loop 0 M26-1444_1_2017-11-26_0001_C0000_%06d.ppm
This adjusts the exposure using the WhiteLevel tag (to my knowledge, all raw converters should respect it).
It deflickers the building (look at median.jpg to check where exactly it meters the exposure on this image), but there's still some flicker in the sky.
The sky flicker goes away by metering on the clouds (e.g. at the 95th percentile):
p95_g1 = prctile(g1(:), 95);
p95_ev = log2(p95_g1 - black);
ec_ev = target_median + 5 - p95_ev;
but then, the building will flicker.
You'll probably need some nonlinear deflicker curve - maybe by blending somehow the two (linear) exposure compensations.
You'll also need read_raw.m:
function im = read_raw(filename)
dcraw = 'dcraw -c -4 -D';
system(sprintf('%s "%s" > tmp.pgm', dcraw, filename));
im = double(imread('tmp.pgm'));
system('rm tmp.pgm');
end
Results on this image sequence: deflicker.zip (http://a1ex.magiclantern.fm/bleeding-edge/deflicker/deflicker.zip) (both median and 95th percentile).
Try blending the two - keep the midtones and shadows from the former, and highlights from the latter.
edit: enfuse does the trick :)
combined script (deflickering both midtones and highlights):
# exiftool *.DNG -BlackLevel
black = 8192
# exiftool *.DNG -WhiteLevel
# value not critical; todo: read from exif
white = 60000
# EV above black (exposure compensation)
# 0 EV = 1 DN
target_median = 12
target_highlight = 14
# prepare output directories
# make sure you don't have anything important here!
system("mkdir -p high med out")
system("rm -f high/* med/* out/*")
function adjust_white(filename, dirname, black, white, metered, target)
ev = log2(metered - black);
ec_ev = target - ev;
ec_linear = 2 ^ ec_ev;
new_white = black + white / ec_linear;
system(sprintf("exiftool '%s' -Directory='%s' -WhiteLevel=%d ", filename, dirname, new_white));
end
for f = dir('*.DNG')'
f.name
im = read_raw(f.name);
g1 = im(1:2:end,2:2:end);
med_g1 = median(g1(:));
p95_g1 = prctile(g1(:), 95);
adjust_white(f.name, "med", black, white, med_g1, target_median);
adjust_white(f.name, "high", black, white, p95_g1, target_highlight);
# render the images
system(sprintf("dcraw -W 'med/%s' 'high/%s'", f.name, f.name));
ppm = [f.name(1:end-4) ".ppm"];
png = [f.name(1:end-4) ".png"];
system(sprintf("enfuse 'med/%s' 'high/%s' -o 'out/%s'", ppm, ppm, png));
end
# debug: where's the median in the last image?
# show 12 stops logarithmically
g1log = (log2(max(1, g1 - black)) - log2(white - black) + 12) * 255 / 12;
# mark the median with a red overlay
# and the 95% percentile with a cyan overlay
ovr(:,:,1) = ovr(:,:,2) = ovr(:,:,3) = g1log;
red = ovr(:,:,1);
red(g1 > med_g1 - 10 & g1 < med_g1 + 10) = 255;
red(g1 > p95_g1 - 100 & g1 < p95_g1 + 100) = 0;
ovr(:,:,1) = red;
imwrite(uint8(ovr), 'metered-areas.jpg');
# test with:
# ffplay -loop 0 out/M26-1444_1_2017-11-26_0001_C0000_%06d.png
Results from second script: deflicker-enfuse.zip (http://a1ex.magiclantern.fm/bleeding-edge/deflicker/deflicker-enfuse.zip)
Metered areas (median and 95th percentile):
(http://a1ex.magiclantern.fm/bleeding-edge/deflicker/metered-areas.jpg)
Was the input MLV flicker-free before processing? (if so, that must be a bug in cr2hdr)
Hi @a1ex,
QuoteSorry, unable to recover the input file from output images...
Quite right, here's a cut-down, thus lighter, MLV file:
https://www.rojoynegroclub.com/images/tmp/M26-1444-30-frames.MLV (https://www.rojoynegroclub.com/images/tmp/M26-1444-30-frames.MLV)
I'll have to read through your shell script and instructions carefully and test. That and the zip file you sent.
QuoteWas the input MLV flicker-free before processing? (if so, that must be a bug in cr2hdr)
Hard to say, since I apparently have no idea how to monitor it properly; MlRawViewer is old, Footage is, well, dead in the water. In MLV App, my present favourite (thanks for the great software Illia, Masc and company!) it obviously plays back with flickering when I bring up the exposure, which is what 75% of the image needs. If I leave it with no exposure change, the image is quite dark, and it's hard to tell.
Is there a recommended way to preview/monitor the MLV file prior to processing? Sorry if it's an obvious question...
Link not working
Quotehttps://www.rojoynegroclub.com/images/tmp/M26-1444-30-frames.MLV
Not Found
You're right, the upload failed, sorry. The link works now, though.
Thanks, works. Interesting as always when bightness shifts and maybe also colors? Hard to tell what´s going on and if anyone knows where to look into this ingenious code it´s a1ex.
Vanilla processing:
mlv_dump M26-1444-30-frames.MLV --dng --no-fixcp --no-stripes
cr2hdr *.dng --same-levels
dcraw -W -b 4 *.DNG
ffplay -loop 0 M26-1444-30-frames_%06d.ppm
There is some flicker - let's see if it's natural flicker or bug in cr2hdr. MlRawViewer (1 month old (http://www.magiclantern.fm/forum/index.php?topic=9560.msg191965#msg191965) is too old, really?) doesn't show any obvious flicker.
The debug options in cr2hdr won't help much, as they are all after matching the exposure between the two sub-images (that is, after scaling the raw data). Let's go back to octave:
for f = dir('*.dng')'
f.name
im = read_raw(f.name);
g1 = im(1:4:end, 2:4:end);
imwrite(uint8(g1 / 64), [f.name(1:end-4) ".jpg"]);
end
Oh no, the "crawling (http://www.magiclantern.fm/forum/index.php?topic=9456)" effect!
phase = 0;
for f = dir('*.dng')'
f.name
im = read_raw(f.name);
g1 = im(1+phase:4:end, 2-mod(phase,2):4:end);
phase = mod(phase+1, 4);
imwrite(uint8(g1 / 64), [f.name(1:end-4) ".jpg"]);
end
system("ffplay -loop 0 M26-1444-30-frames_%06d.jpg")
No flicker obvious to me (just lots of aliasing, as we are previewing at 480x270, without any filtering), so it must be a bug in cr2hdr. Let's try the experimental changes.
cd modules/dual_iso
hg up e72eb09ffa30 -C
make cr2hdr; sudo cp cr2hdr /path/to/executables
Running the first script gives similar result (still no luck).
TODO: fix the math in cr2hdr.c, match_exposures. This is going to be tricky.
Using the data from each sub-image (dark or bright) gives no flicker. The issue appears after matching the brightness between them. The issue is this variation in fitting results:
Linear fit : y = 0.063813*x + 108.308081
Linear fit : y = 0.053660*x + 142.179592
Linear fit : y = 0.062154*x + 105.105966
Linear fit : y = 0.063109*x + 102.554616
Linear fit : y = 0.061640*x + 115.000017
Linear fit : y = 0.063022*x + 114.077421
Linear fit : y = 0.064436*x + 101.677465
Linear fit : y = 0.063109*x + 102.554616
Linear fit : y = 0.063109*x + 108.302178
Linear fit : y = 0.063285*x + 113.235900
Linear fit : y = 0.060960*x + 110.928908
Linear fit : y = 0.061044*x + 109.146648
Linear fit : y = 0.061725*x + 114.726728
Linear fit : y = 0.061129*x + 118.265047
Linear fit : y = 0.062327*x + 104.553750
Linear fit : y = 0.063022*x + 102.833685
Linear fit : y = 0.062587*x + 111.972896
Linear fit : y = 0.057352*x + 132.357319
Linear fit : y = 0.057992*x + 122.541289
Linear fit : y = 0.057912*x + 117.377329
Linear fit : y = 0.053069*x + 140.393025
Linear fit : y = 0.053290*x + 145.259863
Linear fit : y = 0.058883*x + 119.573745
Linear fit : y = 0.061554*x + 105.396036
Linear fit : y = 0.061982*x + 111.904583
Linear fit : y = 0.063548*x + 108.517969
Linear fit : y = 0.064525*x + 101.649343
Linear fit : y = 0.064346*x + 94.735446
Linear fit : y = 0.062674*x + 111.319366
Linear fit : y = 0.064974*x + 106.084113
Linear fit : y = 0.063109*x + 102.175959
If you have some other footage that shows flicker (in particular, extreme cases), I'd like to see it as well (small cuts like this are OK).
QuoteMlRawViewer (1 month old is too old, really?) doesn't show any obvious flicker.
I stand corrected; I was sure development of MlRawViewer was discontinued, but am happy to be wrong in this case.
QuoteIf you have some other footage that shows flicker (in particular, extreme cases), I'd like to see it as well (small cuts like this are OK).
I have plenty; I was worried that something's wrong with my 5D3 or my workflow since I seem to be getting flickering in most if not all dual-iso footage, while no-one else seems to be reporting anything. Perhaps it's because I push it beyond your recommendations of 400/1600 or 200/1600 and go all the way to 100/1600, leaving fewer midtones?
Unfortunately, I've deleted a lot of test clips, taken while I was learning, and experimenting with, this feature. Here are more clips from the same shoot, though:
https://www.rojoynegroclub.com/images/tmp/M26-1449.MLV (https://www.rojoynegroclub.com/images/tmp/M26-1449.MLV)
https://www.rojoynegroclub.com/images/tmp/M26-1450-100-frames.MLV (https://www.rojoynegroclub.com/images/tmp/M26-1450-100-frames.MLV)
https://www.rojoynegroclub.com/images/tmp/M26-1451-166-frames.MLV (https://www.rojoynegroclub.com/images/tmp/M26-1451-166-frames.MLV)
https://www.rojoynegroclub.com/images/tmp/M26-1454-48-frames.MLV (https://www.rojoynegroclub.com/images/tmp/M26-1454-48-frames.MLV)
https://www.rojoynegroclub.com/images/tmp/M26-1455-66-frames.MLV (https://www.rojoynegroclub.com/images/tmp/M26-1455-66-frames.MLV)
https://www.rojoynegroclub.com/images/tmp/M26-1458-52-frames.MLV (https://www.rojoynegroclub.com/images/tmp/M26-1458-52-frames.MLV)
https://www.rojoynegroclub.com/images/tmp/M26-1502-312-frames.MLV (https://www.rojoynegroclub.com/images/tmp/M26-1502-312-frames.MLV)
https://www.rojoynegroclub.com/images/tmp/M26-1503-144-frames. (https://www.rojoynegroclub.com/images/tmp/M26-1503-144-frames.)MLV
The last two are more extreme, as they are looking towards the sun. I think the setting are still the same (100/1600), but can't remember for sure.
Update: still aliased like hell, but flicker-free! Enjoy the power of robust statistics!
http://builds.magiclantern.fm/jenkins/view/Utilities/job/cr2hdr-20bit/
I expect this to solve many problems caused by imprecise exposure matching, not just flicker (the most common one being horizontal stripes). You may want to run it with --no-stripe-fix (this workaround might not be needed any more - to be tested). There are a few other (small) changes I didn't manage to test yet, but they appear harmless at first sight.
Rendering script used for testing:
# assumes only 1 MLV is present in the working directory (no more, no less)
mlv_dump *.MLV --dng --no-fixcp --no-stripes
cr2hdr *.dng --same-levels --no-stripe-fix
dcraw -W -b 4 *.DNG
ffplay -loop 0 $(basename *.MLV .MLV)_%06d.ppm
TODO: make good use of this to solve the aliasing issue: http://www.magiclantern.fm/forum/index.php?topic=20999
Caveat: I did not run my "usual" test suite, as it would require swapping the hard drive, so please consider the latest build highly experimental (and try the previous one in case of problems). As usual, feel free to report issues - in particular, troublesome test images.
Great work. Here´s a binary for mac users:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/cr2hdr
Tested against one test file from smasry.
This is an interesting added option:
Troubleshooting options:
--force : force processing even if the image is not Dual ISOQuoteTODO: make good use of this to solve the aliasing issue: http://www.magiclantern.fm/forum/index.php?topic=20999
This would be great for dualiso.
Thanks, @a1ex, can't wait to test this! And thanks, @danne for already building the mac cr2hdr binary!
Yes, aliasing is bad; I should have invested in a wider lens and shot this in crop mode. It wouldn't have been too terrible then.
As for the aliasing reduction using your script, hugin, enfuse and all the other options listed in that thread, it looks fantastic, just one problem: we lose the RAW image, leaving less data to work on, in the colour-correcting stage. I hope I'm wrong on this issue, would a 16-bit TIFF or EXR may give us the same leeway as a 16-bit DNG?
Well, would be nice to see some comparison tests on this theme. I updated the hugin/enfuse script just now by the way.
Quote from: smasry on December 04, 2017, 09:24:08 AM
Yes, aliasing is bad; I should have invested in a wider lens and shot this in crop mode. It wouldn't have been too terrible then.
It would have been just as bad - exercise for the reader.
Hint: it was already discussed.
If spatial aliasing is an issue, but you have no problems with temporal aliasing, I suggest looking into the plain old HDR video. The optical flow tricks are MUCH easier to use on that kind of footage.
Moved to own thread...
@Dane
"Here´s a binary for mac users:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/cr2hdr"
So what do we do with This to be sure Switch Uses It?
http://www.magiclantern.fm/forum/index.php?topic=15108.msg194147;topicseen#msg194147
Regarding aliasing in videos:
Here's a very simple octave script that only works on static scenes (with nearly no motion) with crawling (http://www.magiclantern.fm/forum/index.php?topic=9456.0) artifacts. It's based on an older idea (http://www.magiclantern.fm/forum/index.php?topic=7139.msg60613#msg60613) from stevefal, and seems to work quite well on some of the clips posted lately (as long as the motion from one frame to another is next to none).
You'll now see with your own eyes what exactly a "slight temporal artifact" is.
This script also serves as a minimal proof of concept code for Dual ISO blending (useful for understanding how the algorithm works, without having to dig through thousands of lines of code). In other words, this script is very useful as documentation. It operates on unprocessed 14-bit .dng frames (as extracted with mlv_dump), but requires some metadata from cr2hdr logs.
# file name prefix
# M26-1444-30-frames_000000.dng -> prefix = "M26-1444-30-frames_"
prefix = "M26-1444-30-frames_"
# "crawling" phase - possible values: 0, 1, 2, 3
# from ISO pattern on the first rendered frame (cr2hdr log), or trial and error:
# dBBd = 0
# ddBB = 1
# BddB = 2
# BBdd = 3
p = 3
# hardcoded - copy from cr2hdr log
black = 2047 # black level (14-bit)
white = 15000 # white level (14-bit), can be approximate, must be slightly lower than actual clipping point
gain = 15.3 # from "Linear fit" messages - gain part (actually ISO ratio)
offset = 24 # from "Linear fit" messages - offset part, invert sign
black_adj = 3 # fine-tune until blacks look right (fixme: why is it needed?)
# from start+1 to end-1 (frame numbers to process)
# we need one frame before and one after
for i = 1:29
# fixme: read each image only once
a = read_raw(sprintf("%s%06d.dng", prefix, i-1));
b = read_raw(sprintf("%s%06d.dng", prefix, i));
c = read_raw(sprintf("%s%06d.dng", prefix, i+1));
# print the pattern for current frame (cross-check with cr2hdr log)
# for example: cr2hdr *.dng | grep pattern
# cross-check with: octave crawl.m | grep pattern
iso_pattern = circshift("dBBd", [0 p])
# rebuild hi-iso data for frame b, taking bright fields from frames a and c
# fixme: this only works when there's little or no motion between the frames
# this recovers the full resolution from perfectly static (!) scenes
# but introduces massive aliasing whenever something moves
hi = b;
hi(mod(p+0,4)+1:4:end,:) = a(mod(p+0,4)+1:4:end,:);
hi(mod(p+3,4)+1:4:end,:) = c(mod(p+3,4)+1:4:end,:);
# rebuild lo-iso data for frame b, taking dark fields from frames a and c
lo = b;
lo(mod(p+1,4)+1:4:end,:) = c(mod(p+1,4)+1:4:end,:);
lo(mod(p+2,4)+1:4:end,:) = a(mod(p+2,4)+1:4:end,:);
# cancel black delta (see "Find black delta..." in cr2hdr.c)
# fixme: incorrect offset scaling in cr2hdr logs
black_delta = 2 * offset * 16 / (gain + 1)
hi -= black_delta / 2;
lo += black_delta / 2;
# remove black level, match image brightness
# (darken hi to match lo)
hi -= black;
lo -= black;
hi /= gain;
# approximate dual iso blending (smooth transition from lo-iso to hi-iso)
# fixme: compute noise profiles and use Kalman optimal averaging formula
T = @(x) min(max((x - 3000) / 8000, 0), 1);
F = @(x) ((sin((x-0.5)*pi)+1)/2).^4;
f = F(T(hi .* gain));
x = lo .* f + hi .* (1-f);
plot(1:15000, F(T(1:15000)));
# restore the black level
x += black;
# fine-tune black level
x += black_adj;
# check image levels
percentiles = [10 50 90 99 99.9 99.99 99.999];
levels = round(prctile(x(:), percentiles))';
printf("%g\t", percentiles); printf("\n");
printf("%d\t", levels); printf("\n");
# output as PGM -> DNG
# todo: clean way to save a DNG from octave
# pgm2dng: http://a1ex.magiclantern.fm/bleeding-edge/pgm2dng.c
# note: pgm2dng hardcodes a black level of 2048 and a white level of 16382
# multiply 14-bit image and black/white levels by 4 to get 16-bit output
out = sprintf("frame%d.pgm", i);
imwrite(uint16(x * 4), out);
system(sprintf("pgm2dng %s", out));
system(sprintf("exiftool %s -BlackLevel=%d -WhiteLevel=%d -overwrite_original", [out(1:end-4) ".DNG"], black * 4, white * 4));
# update the phase for next frame
# todo: is the "crawling" always going in this direction?
p = mod(p+1, 4);
end
Caveat: it requires tuning for every single clip (mostly copying stuff from cr2hdr log). It is currently tuned for M26-1444-30-frames.MLV (linked earlier). You'll also need pgm2dng (http://a1ex.magiclantern.fm/bleeding-edge/pgm2dng.c) (a very hackish tool I use for experiments, to get DNGs out of octave).
This also gives fairly good results on the clip from 70MM13 (p=0, gain=7.86, offset=10.61).
Possible improvements:
- find out where there is motion (how? aliasing may easily confuse any algorithm I can think of) and use regular interpolation in these areas.
- apply motion compensation (optical flow) and interpolate only the missing bits (difficult)
Have fun!
Here's a couple of frames extracted from test shots of iso 53 vs 100...
I did nothing to avoid pink highlights, simply reducing analog gain to look for noise improvement, and got a good result!
Both recordings are using identical conditions and camera settings, one with iso regs enabled, the other disabled. Exposure was adjusted in post to match.
These images have NO noise reduction of any kind. I raised the raw white level in rawtherapee to reduce the pink highlights but only a quick adjustment to make it less distracting.
I'll be using this!
(http://thumb.ibb.co/gKv04b/iso100.jpg) (http://ibb.co/gKv04b)
(http://thumb.ibb.co/eDJyqG/iso53.jpg) (http://ibb.co/eDJyqG)
Has anyone experienced problems with last version (2017-12-04 00:01) of cr2hdr processing 5DII files?
I notice two things in Lightroom that I think they happen since I updated cr2hdr.
First, it seems the processed Dual ISO DNG it has lost orientation info as now LR doesn't turn my images automatically.
Second, LR is unable to find a lens profile of the DNG processed with cr2hdr but it can find it automatically in the original Dual ISO file.
Anyone has noticed this also?
Upload an example CR2 and version of cr2hdr? Windows, mac?
- unprocessed dualiso: https://www.amazon.es/clouddrive/share/2hOTnDMQlaR59dlhFOp5faiVnOftB4sWivsdVIiHL5K
- processed dualiso: https://www.amazon.es/clouddrive/share/N5smrclgCuyIQOPP5bO80qnh0TMaryMC825RSzsGakM
Things I have noticed since build 2017-12-04 00:01.
1) Metadata loss. Unprocessed dng has the right metadata.
- Exposure: 1/125 at f8.0
- Focal Length: 30mm
- Model: Canon EOS 5D Mark
- Lens: EF 17-40 f4 L USM
Processed dual iso has lost metadata.
- Exposure: 1 sec at f1.0
- Focal Length: ?
- Model: Canikon
- Lens: ?
So Lr warns that it's unable to locate a matching lens profile automatically.
2) Even after manually assign lens profile something seems wrong. Before the last build 90% I needed -2 points of distortion correction for the 17-40mm. Now I need about +10. ?¿?¿?¿ Weird.
3) Orientation info seems lost? Before last build Lr automatically rotate the images to right orientation, now I have to do it manually.
Lr 7.1 on Windows 8.1.
I used the latest version for mac and it transcodes just fine over here, camera name and lens name is represented. Are you using cr2hdr directly or some other app?
(https://s14.postimg.cc/4fm83zck1/Screen_Shot_2018-01-27_at_17.48.11.png)
Quote from: Danne on January 27, 2018, 05:50:45 PM
I used the latest version for mac and it transcodes just fine over here, camera name and lens name is represented. Are you using cr2hdr directly or some other app?
I'm using Dual ISO Converter plugin for Lightroom. Could it have something to do the fact that I've downloaded last build of cr2hdr and overwrite the cr2hdr found in the Lr plugin?
Well, everything is possible.
Quote from: Walter Schulz on January 27, 2018, 07:24:02 PM
@R: Sharing an unprocessed Dual-ISO file might be helpful ... content of plug-in bin dir, too. Exiftool, dcraw, cr2hdr
Like the one here? --> https://www.magiclantern.fm/forum/index.php?topic=7139.msg196445#msg196445
( https://www.amazon.es/clouddrive/share/2hOTnDMQlaR59dlhFOp5faiVnOftB4sWivsdVIiHL5K )
The ..\Adobe Lightroom Classic CC\cr2hdr.lrplugin\bin looks like this:
(https://thumb.ibb.co/ggkvMw/Clipboard_1.jpg) (https://ibb.co/ggkvMw)
Quote from: R on January 27, 2018, 07:31:01 PM
Like the one here? --> https://www.magiclantern.fm/forum/index.php?topic=7139.msg196445#msg196445
Must be alcohol induced blindness ...
Problem with snapshot: No reliable version info. Hard to tell what we are trying to reproduce without your files.
About https://www.magiclantern.fm/forum/index.php?topic=7139.msg196445#msg196445 (https://www.magiclantern.fm/forum/index.php?topic=7139.msg196445#msg196445), it turned out to be incompatibilities between Lr plugin and exiftool .
cr2hdr.3.0-170316-02-MLV-DEV Plugin uses exiftool 10.43 and cr2hdr-win 2017-12-04 00:01 comes with exiftool 10.02. If you want to update the plugin bin folder with new releases of cr2hdr-win, just replace cr2hdr.exe and dcraw.exe but not exiftool.exe.
Quote from: theBilalFakhouri on April 27, 2017, 09:45:37 AM
I Have problem with Canon 700D
when i turn on dual iso module automatically ISOless LV err(8 ) appear on screen
(https://image.ibb.co/g1ZDkk/Capture.jpg)
and the module do not work anymore
WHY THIS HAPPENING ?
any Ideas? a1ex
dual_iso ISOless ( 8 ) & (15) err finally solved for Canon 700D that have this problem!
The problem was in CMOS (0) Registers at ISO 100, It was different in my camera.
So I changed it to match the code in dual_iso.c
is_700d = 1;
FRAME_CMOS_ISO_START = 0x4045368e;
FRAME_CMOS_ISO_COUNT = 6;
FRAME_CMOS_ISO_SIZE = 34;
PHOTO_CMOS_ISO_START = 0x40452444;
PHOTO_CMOS_ISO_COUNT = 6;
PHOTO_CMOS_ISO_SIZE = 16;
It's Working!
(https://thumb.ibb.co/coTW1S/VRAM0.png) (https://ibb.co/coTW1S)
(https://thumb.ibb.co/jPeh87/VRAM1.png) (https://ibb.co/jPeh87)
Now the time to try it:
magiclantern-crop_rec_4k.2018Apr11.700D115_Iso_Less_8_15_resolved (https://drive.google.com/open?id=1GJLxGz_edhYdJLWfGf96CRFLEaaT18vA) (Updated with 10/3/2018 changes)
Issue related to ISOless message on 650D? Any advice how to corner this?
Quote from: Walter Schulz on February 11, 2018, 02:10:52 PM
Issue related to ISOless message on 650D? Any advice how to corner this?
Yes, I think it's the same way I did you can do, it's about CMOS 0 Registers at ISO 100, Some cameras have different registers than others for
the same model, (I don't know yet if it will be more than two Registers for same model, waiting 700d users to feedback)
So you have to see your registers on your camera by adtg_gui.mo (https://builds.magiclantern.fm/modules.html) (Assuming adtg_gui.mo is working on canon 650d)
1-Set ISO at 100
2-Set camera on movie mode 1080/24
3-After loading the modules, turn on adtg_gui in Debug submenu
4-Turn on dual_iso
5-Go to PLAY mode then go back to liveview
6-Check your CMOS 0 Registers data and write it down (In the white box)
(https://preview.ibb.co/cRdQT7/VRAM0.png) (https://ibb.co/ktAHan)
7-Now go to photo mode
8-Take a picture and Check your CMOS 0 Registers data and write it down again.
--> Now in dual_iso.c
This value for movie mode
FRAME_CMOS_ISO_START = 0x********;
and this for photo mode:
PHOTO_CMOS_ISO_START = 0x********;
Leave the (0x) and change the others to the values you wrote, Compile and test it in the camera!
Quote from: theBilalFakhouri on February 11, 2018, 02:58:08 PM
1-Set ISO at 100
2-Set camera on movie mode 1080/24
3-After loading the modules, turn on adtg_gui in Debug submenu
4-Turn on dual_iso
5-Go to PLAY mode then go back to liveview
6-Check your CMOS 0 Registers data and write it down (In the white box)
I got 0x4045368e on my 700D, just like you did:
(https://farm5.staticflickr.com/4753/40248216311_e64a8a6541.jpg) (https://flic.kr/p/24jACtZ)
Quote from: theBilalFakhouri on February 11, 2018, 02:58:08 PM
7-Now go to photo mode
8-Take a picture and Check your CMOS 0 Registers data and write it down again.
I got 0x40452044:
(https://farm5.staticflickr.com/4750/39350752365_4d17f9c6ef.jpg) (https://flic.kr/p/22XhTHF)
So following your instructions it looks like my camera is fine with the current code:
modules/dual_iso.c
else if (is_camera("700D", "1.1.5"))
{
is_700d = 1;
FRAME_CMOS_ISO_START = 0x4045328E;
FRAME_CMOS_ISO_COUNT = 6;
FRAME_CMOS_ISO_SIZE = 34;
PHOTO_CMOS_ISO_START = 0x40452044;
PHOTO_CMOS_ISO_COUNT = 6;
PHOTO_CMOS_ISO_SIZE = 16;
CMOS_ISO_BITS = 3;
CMOS_FLAG_BITS = 2;
CMOS_EXPECTED_FLAG = 3;
}
However, your 700D is different?
Quote from: theBilalFakhouri on February 11, 2018, 12:53:02 PM
PHOTO_CMOS_ISO_START = 0x40452444;
It's Working!
It doesn't make sense that these registers are different on the same model camera. This is what happens if I use your build on my camera:
(https://farm5.staticflickr.com/4740/38438371420_28aa86840a.jpg) (https://flic.kr/p/21yEGDJ)
@dfort I really don't know what makes these registers different. Maybe Canon did some tiny edits between the models or the country of manufacture?
Assuming if it true, my camera was made in Taiwan, what is yours? crazy theory.
When I adjusted the photo register I wrote it wrong by mistake, I got ISOless err(3), it was after correcting the video register and only dual_iso was working in video mode.
Okay, can we add these both registers in the code? If yes, this will be cool! :D , if not, so why? What is the problem exactly?
1) the cause for this variation is unknown; are these two really the only possible addresses?
2) Canon firmware startup is non-deterministic; maybe it's that; maybe the startup mode or Canon settings will make a difference, no idea
3) if outside LV, the error can be probably reproduced in QEMU; if so, that can be used to solve the mystery.
Quote from: theBilalFakhouri on February 13, 2018, 09:31:01 PM
Maybe Canon did some tiny edits between the models or the country of manufacture?
Assuming if it true, my camera was made in Taiwan, what is yours? crazy theory.
Not so crazy, my Canon EOS Rebel T5i (a.k.a. 700D/Kiss X7i) was made in Japan. I've got an MLV file you shared and see this in mlv_dump -v:
Block: IDNT
Offset: 0x00000108
Number: 3
Size: 84
Time: 3.970000 ms
Camera Name: 'Canon EOS 700D'
Camera Serial: '59310CBFB6' (383075008438)
Camera Model: 0x80000326
While on my camera the same block looks like this:
Block: IDNT
Offset: 0x00000108
Number: 3
Size: 84
Time: 0.797000 ms
Camera Name: 'Canon EOS REBEL T5i'
Camera Serial: '775328882' (32031017090)
Camera Model: 0x80000326
I'm pretty sure that ML is identifying cameras using the "Camera Model" hex code (in propvalues.h?) but maybe in this case a different identifier needs to be used?
Quote from: a1ex on February 13, 2018, 09:46:07 PM
3) if outside LV, the error can be probably reproduced in QEMU; if so, that can be used to solve the mystery.
Ran @theBilalFakhouri build on my 700D ROM in QEMU and got this:
(https://farm5.staticflickr.com/4719/25384937867_3bcece99bd.jpg) (https://flic.kr/p/EFbrCx)
Here is the QEMU log (https://pastebin.com/R0fF1fbn).
Quote from: dfort on February 13, 2018, 08:23:57 PM
I got 0x4045368e on my 700D, just like you did:
(https://farm5.staticflickr.com/4753/40248216311_e64a8a6541.jpg) (https://flic.kr/p/24jACtZ)
@dfort
Sorry but it's different, look at the picture again you missed it.
It's 0x4045328e the same as dual_iso.c not 0x4045368e.
Quote from: a1ex on February 13, 2018, 09:46:07 PM
1) the cause for this variation is unknown; are these two really the only possible addresses?
Maybe yes, we are waiting for more users to make sure.
Oops--sorry. Looks like mine is the same as what is currently in modules/dual_iso.c and both of your addresses are offset by 0x400. Wonder if that's affecting anything else.
This is the theBilalFakhouri ROM dump from his Taiwanese 700D running the official current Nightly build from the Jenkins server in QEMU:
(https://farm5.staticflickr.com/4742/40272890152_6e8ddaf2fd.jpg) (https://flic.kr/p/24mM6a3)
Quote from: Walter Schulz on February 11, 2018, 02:10:52 PM
Issue related to ISOless message on 650D? Any advice how to corner this?
I believe that the 650D ROM I have is yours. It was saved with the language set to German. Are you seeing this using the current Nightly build?
(https://farm5.staticflickr.com/4724/39421201435_95c8848eae.jpg) (https://flic.kr/p/234vXLP)
Quote from: dfort on February 17, 2018, 04:40:30 PM
I believe that the 650D ROM I have is yours. It was saved with the language set to German. Are you seeing this using the current Nightly build?
Tried my best to make this error appear but no avail with latest build (2018Feb04). Retried with the one before (2017Dec07) and got it again after power off and powering on again.
Issue solved, I suppose?
EDIT: And here we go again. Switched Picture Style to UserDef1 (Marvels Advanced 3.4) and err(2) appears.
EDIT2: Not that easy to reproduce. With former builds I had to reset ML settings to get rid of it. Now I have troubles to provoke the error. Not persistent anymore.
Ok, it looks like I can see the error in QEMU but can't get adtg_gui working in QEMU so if you could run the test theBilalFakhouri posted (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197495;topicseen#msg197495) on your 650D maybe we can track this down. On the 700D the addresses for both FRAME_CMOS_ISO_START and PHOTO_CMOS_ISO_START were offset by 0x400 but when I tried it on the 650D it didn't work. Looks like once we get the CMOS[0] value from your camera using adtg_gui (on the modules downloads page (https://builds.magiclantern.fm/modules.html)) we should be able to "fix" it--at least for your camera. Need some help from a1ex for this to work on all cameras. So far it looks like it only affects the 700D and 650D but maybe there are others?
Unable to perform test.
tcc: error: undefined symbol 'unpatch_memory'
tcc: error: undefined symbol 'patch_hook_function'
[E] failed to link modules
Are you using the crop_rec_4k build? I think the lua_fix branch also works. Just in case I'll post an iso-research branch build for you on my my downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/).
@theBilalFakhouri & dfort:
Thank you for support and build but numbers shown with Adtg_gui.mo are identical to those in dual_iso.c.
Additional info: Error shows up with ISO200/800, too.
@Walter Schulz
Okay, you have followed all the steps (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197146#msg197146)as it is, now can you do it again with dual_iso turned off?
A question What is working with dual_iso on your 650D and what is not working?
e.g. The module is not working at all? or in some specific settings like in video mode only ? or at 100/400 is working fine but at 200/800 isn't working?
And also try this build (https://drive.google.com/open?id=1R5iOqI7x5RUjKCnYJk9cg3nskLjLf3Gv) based on crop_rec_4k_mlv_lite_snd branch and feedback please.
Wonder if the addresses change depending on which picture profile is selected.
Made another 650D QEMU run, this time on the qemu branch with some of the debugging goodies turned on. The "ISOless PH err (2)" is showing up consistantly here. Maybe this QEMU log will help?
https://pastebin.com/WPP9V90x
@theBilalFakhouri - I tried your build in QEMU and am still getting the "ISOless PH err (2)"
Quote from: theBilalFakhouri on February 19, 2018, 07:56:34 PM
@Walter Schulz
Okay, you have followed all the steps (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197146#msg197146)as it is, now can you do it again with dual_iso turned off?
Dual ISO module loaded, not activated in Expo tab: Same registers.
Quote from: theBilalFakhouri on February 19, 2018, 07:56:34 PM
A question What is working with dual_iso on your 650D and what is not working?
e.g. The module is not working at all? or in some specific settings like in video mode only ? or at 100/400 is working fine but at 200/800 isn't working?
Everything is working fine until err(2) occurs. With this error on screen cam will do "normal" stills without interlaced lines.
Error is not exclusive to 100/xxx. Happens in 200/xxx, too.
Quote from: theBilalFakhouri on February 19, 2018, 07:56:34 PM
And also try this build (https://drive.google.com/open?id=1R5iOqI7x5RUjKCnYJk9cg3nskLjLf3Gv) based on crop_rec_4k_mlv_lite_snd branch and feedback please.
Unable to run adtg_gui.mo together with dual_iso.mo. Tried adtg_gui from Daniel's repository and a1ex/hudson. Later gives undefined symbol 'get_ms_clock_value' as you mentioned before. (https://www.magiclantern.fm/forum/index.php?topic=19300.msg197507#msg197507)
Error happens but it's err(8) now!
Quote from: Walter Schulz on February 20, 2018, 03:15:22 PM
Dual ISO module loaded, not activated in Expo tab: Same registers.
Sorry my fault I meant to activate it when I said "enable" this is important step. So try again and if you have tried it already and it didn't work:
Looks like the problem only happened in photo mode. In video mode it's working correctly with no problem right?
Quote from: Walter Schulz on February 20, 2018, 03:15:22 PM
Error happens but it's err( 8 ) now!
I changed the two registers in the build to the registers on this page (https://bitbucket.org/hudson/magic-lantern/issues/1781/650d-%3Cbr%20/%3Edual_iso-isoless-ph-err-2), maybe it's the same problem with you?
@dfort Can you add the same codes on the page (https://bitbucket.org/hudson/magic-lantern/issues/1781/650d-dual_iso-isoless-ph-err-2) to dual_iso.c for 650d?
//is_address_right must use after all other value setted
//despite the FRAME_CMOS_START and PHOTO_CMOS_START
//return 1 means true, 0 meas false
static uint8_t is_address_right(uint32_t address,uint16_t count,uint16_t size,uint16_t expected,char *model)
{
if(streq(model,"650D"))
{
for(uint i=0;i< count;i++)
{
uint16_t raw=*(uint16_t *)(address+i*size);
uint32_t flag= raw & 0x3;
//if start_addr is right,all flag will be 11
//else raw is 0x0 0x4046,flag is 0x0 or 0x10 bad address
if(flag != expected)
return 0;
}
return 1;
}
else
{
return 0;
}
} *100 0x803 0x4049f144 or 244
*200 0x827 0x4049f154 0x24
*400 0x84b 0x4049f164 0x24
*800 0x86f 0x4049f174 0x24
*1600 0x893 0x4049f184 0x24
*/
is_650d = 1;
uint32_t photo_cmos_start_set[]={0x4049f144,0x4049f244};
uint32_t frame_cmos_start_set[]={0x404a038e,0x404a048e};
FRAME_CMOS_ISO_COUNT = 6;
FRAME_CMOS_ISO_SIZE = 0x22;
PHOTO_CMOS_ISO_COUNT = 6;
PHOTO_CMOS_ISO_SIZE = 0x10;
CMOS_ISO_BITS = 3;
CMOS_FLAG_BITS = 2;
CMOS_EXPECTED_FLAG = 3;
//determin the PHOTO_CMOS_ISO_START VALUE
if(is_address_right(photo_cmos_start_set[0],PHOTO_CMOS_ISO_COUNT,PHOTO_CMOS_ISO_SIZE,CMOS_EXPECTED_FLAG,camera_model_short))
{
PHOTO_CMOS_ISO_START = photo_cmos_start_set[0];
}
else
{
PHOTO_CMOS_ISO_START = photo_cmos_start_set[1];
}
//determin the FRAME_CMOS_ISO_START VALUE
if(is_address_right(frame_cmos_start_set[0],FRAME_CMOS_ISO_COUNT,FRAME_CMOS_ISO_SIZE,CMOS_EXPECTED_FLAG,camera_model_short))
{
FRAME_CMOS_ISO_START=frame_cmos_start_set[0];
}
else
{
FRAME_CMOS_ISO_START=frame_cmos_start_set[1];
}
I tried but I got errors when I compiled it because I don't have any coding skills yet.
Maybe it will solve the problem and if not:
I think the problem is while you are trying to get the register from photo mode because when I tried for the first time I got different register and it didn't solve the problem on my 700d, after the second try I got another register for photo mode at the same settings and it worked! (Maybe I changed something between the tries).
Try again: Disable all the modules from loading except adtg_gui & dual_iso
1-Go to photo mode without live view.
2-Enable dual_iso (ISO: 100 in canon menu 800 in dual_iso).
3-Enable adtg_gui.
4-Take normal .cr2 picture
5-If the photo still holding press half shutter then 6-Press trash to get back to ML and check CMOS 0
Try the same thing in other ways like take JPEG instead of .cr2, if nothing worked I really don't know what the problem exactly.
Quote from: theBilalFakhouri on February 20, 2018, 08:50:57 PM
Sorry my fault I meant to activate it when I said "enable" this is important step. So try again and if you have tried it already and it didn't work:
Sorry, I'm confused. You wrote "dual_iso turned off" above. Now I have no idea what modules to load and (if module dual iso loaded) intended cam status.
1) Turning module dual_iso.mo off and redo register check with/without restart?
2) Loading module dual_iso.mo (ON), restart cam and activate or not activate or activate and deactivate Dual-ISO before checking registers?
Quote from: theBilalFakhouri on February 20, 2018, 08:50:57 PM
Looks like the problem only happened in photo mode. In video mode it's working correctly with no problem right?
Can't say because I'm not that much into video and actually did Dual-ISO movie tests maybe once or twice.
Quote from: theBilalFakhouri on February 20, 2018, 08:50:57 PM
I changed the two registers in the build to the registers on this page (https://bitbucket.org/hudson/magic-lantern/issues/1781/650d-%3Cbr%20/%3Edual_iso-isoless-ph-err-2), maybe it's the same problem with you?
You want me to compile this, right?
The issue can be reproduced with theBilalFakhouri's ROM in QEMU.
First step: what Canon code writes to this address range? (0x40452044 on the "good" firmware).
# this goes into 700D/debugmsg.gdb
# note: 0x40000000 is the uncacheable memory flag, so both addresses refer to the same physical memory location
watch *0x40452044
watch *0x452044
# run with:
./run_canon_fw.sh 700D,firmware="boot=1" -s -S & arm-none-eabi-gdb -x 700D/debugmsg.gdb
No results that way.
Second try: use our own logging framework (-d ram). The logs obtained that way would be huge, so add this into logging.c, eos_log_mem, right after the call to should_log_memory_region_verbosely:
if ((addr & 0x0FFFFFF0) != 0x00452040)
{
return;
}
./run_canon_fw.sh 700D,firmware="boot=1" -d ram |& grep -a -C 5 ram
...
[EEPROM-DMA]! [0x340000] -> [0x40450E00] (0xBB860 bytes)
[ram] at Startup2:FF139C24:FF139C24 [0x40452040] <- 0xFF : was 0x0; 8-bit
...
A-ha! This data is copied from the serial flash.
On the bad firmware:
[EEPROM-DMA]! [0x340000] -> [0x40451200] (0xBB860 bytes)
Alright, so next step would be to find out where 0x40450E00 or 0x40451200 is allocated, and how to find a pointer to it. Grep is usually good for finding needles in the haystack, so...
./run_canon_fw.sh 700D,firmware="boot=1" -d ram,io,calls,tail,sflash,debugmsg |& grep -a -C 5 -i 40450E00
...
call 0xFF13AB98(340000, 40450e00, bb6dc, 100000) at [Startup2:ff12087c:ff123344]
call 0xFF139FFC(340000, 40450e00, bb6dc, c0820404) at [Startup2:ff13abfc:ff120880]
[ Startup2:ff13a028 ] (00:01) [SF] readSerialFlashWithQuad Dest 0x40450e00 Src 0x340000 Size 767708
call 0xFF139AE8(bd000, 40450e00, bc, ffffff3e) at [Startup2:ff13a3f4:ff13ac00]
[SFDMA] at Start:FF139BE8:FF139BE8 [0xC0530060] <- 0x40450E00: Transfer memory address
[EEPROM-DMA]! [0x340000] -> [0x40450E00] (0xBB860 bytes)
All of the above addresses are good candidates to be looked up in the disassembly.
Stack trace for the call to readSerialFlashWithQuad:
./run_canon_fw.sh 700D,firmware="boot=1" -d callstack -s -S & arm-none-eabi-gdb -x 700D/debugmsg.gdb
...
(gdb) b *0xff13a028
...
Breakpoint 4, 0xff13a028 in ?? ()
(gdb) print_current_location_with_callstack
Current stack: [14bf60-14bb60] sp=14be88 at [Startup2:ff13a028:ff13ac00]
0xFF0C35B0(0, ff0c35b0, 19980218, 19980218) at [Startup2:bba0:14bf58] (pc:sp)
0xFF1232FC(14bf44, 1, 340000, 1000) at [Startup2:ff0c3634:14bf28] (pc:sp)
0xFF1206E8(450d3c, 450d64, 23b24, 0) at [Startup2:ff123340:14bf10] (pc:sp)
0xFF13AB98(340000, 40450e00, bb6dc, 100000) at [Startup2:ff12087c:14bed0] (pc:sp)
0xFF139FFC(340000, 40450e00, bb6dc, c0820404) at [Startup2:ff13abfc:14beb0] (pc:sp)
In particular, 0xFF1232FC is... PROPAD_CreateFROMPropertyHandle : FIX.
Wait, what?! All these years, we've been writing in the middle of Canon's property data structures without even knowing about it?! Guess we were lucky, as I believe the FIX settings block is probably not written back to the permanent memory at shutdown... (one small piece added to this puzzle (https://www.magiclantern.fm/forum/index.php?topic=19369)).
Back to our quest: who allocates memory for this? There are some calls to alloc_dma_memory around, so let's trace them with gdb:
b *0x16e8c
commands
silent
print_current_location
KRED
printf "alloc_dma_memory(0x%x)\n", $r0
log_result
KRESET
c
end
b *0xFF1232FC
commands
silent
print_current_location
KRED
printf "PROPAD_CreateFROMPropertyHandle(0x%x, 0x%x, 0x%x, 0x%x, 0x%x, 0x%x, 0x%x)\n", $r0, $r1, $r2, $r3, MEM($sp), MEM($sp+4), MEM($sp+8)
KRESET
c
end
(./run_canon_fw.sh 700D,firmware="boot=1" -d debugmsg -s -S & arm-none-eabi-gdb -x 700D/debugmsg.gdb) |& grep -a -C 50 -i 40450E00
...
[ Startup2:ff0c3634 ] PROPAD_CreateFROMPropertyHandle(0x14bf44, 0x1, 0x340000, 0x1000, 0x100, 0x100000, 0x1000000)
[ Startup2:ff1230a8 ] alloc_dma_memory(0x1000ff)
[ Startup2:ff1230ac ] => 0x40450dc4
...
[ Startup2:ff13a028 ] (00:01) [SF] readSerialFlashWithQuad Dest 0x40450e00 Src 0x340000 Size 767708
...
Found where it's allocated, now to figure out where the result is stored. Or, maybe better, as I prefer to avoid messing with the property data structures - refactor with the crop_rec method for overriding these registers. That would require finding the cmos_write stub for all camera models.
Reference for the above commands: README.rst (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/README.rst#rst-header-debugging-with-gdb) - Debugging with GDB.
Quote from: a1ex on February 20, 2018, 10:16:38 PM
Found where it's allocated, now to figure out where the result is stored. Or, maybe better, as I prefer to avoid messing with the property data structures - refactor with the crop_rec method for overriding these registers. That would require finding the cmos_write stub for all camera models.
Wouldn't that be the same as CMOS_WRITE_FUNC that is used in adtg_gui? Seems like adtg_gui worked for "fixing" the 700D belonging to theBilalFakhouri but we might be trying to hit a moving target with Walter's 650D.
@Walter Schulz
Okay sorry my fault again :D:
After loading the two modules which are dual_iso and adtg_gui only
In the first try, do it (the steps (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197146#msg197146)) with dual iso turned on in Expo menu.
In the second try, do it with dual iso turned off in Expo menu.
If the registers didn't changed, so then try to compile the codes from that page (https://bitbucket.org/hudson/magic-lantern/issues/1781/650d-dual_iso-isoless-ph-err-2) which I put it in the last post (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197610#msg197610). And maybe your problem will be resolved!
@a1ex
Wow it's huge.
If I can help with my cam, just tell me that.
@dfort: yes, it's the same function. If these functions become widely used, it would make sense to add them to stubs.S, along with some generic callbacks that other modules (dual_iso, crop_rec etc) could use to override these register values. Need to think a bit about it, as it's not a straightforward change (at least not for me).
Something like this would be also welcome for the ISO tweaks (to implement them in a way that doesn't conflict with existing features - something which was holding me back, besides the complexity of that topic).
Quote from: a1ex on February 20, 2018, 10:16:38 PM
The issue can be reproduced with theBilalFakhouri's ROM in QEMU.
Same with Walter's 650D ROM in QEMU.
I have a 700D.115, purchased in Germany (as 700D.113). On my camera the two addresses also have an offset of 0x400.
With the nightly builds I always got "ISOless PH err(15)".
I changed the addresses accordingly => now dual iso works for me :)
Maybe this helps...
Tried in my 5d Mark II and I get these strong bands in photos :o
https://www.dropbox.com/s/2br0b39s5125lsv/Image%20139.png?dl=0 (https://www.dropbox.com/s/2br0b39s5125lsv/Image%20139.png?dl=0)
For analysis original file is needed.
And if you worried about those horizontal bands: That's what Dual-ISO is all about. You have to postprocess Dual-ISO files with cr2hdr or or other utilities able to deal with interlaced ISO. See post-processing forum and read first post of this thread, too.
Quote from: vicshift on March 20, 2018, 01:37:35 PM
Tried in my 5d Mark II and I get these strong bands in photos :o
https://www.dropbox.com/s/2br0b39s5125lsv/Image%20139.png?dl=0 (https://www.dropbox.com/s/2br0b39s5125lsv/Image%20139.png?dl=0)
I use this https://www.magiclantern.fm/forum/index.php?topic=9102.0
I have replaced cr2hdr in Barracuda folder with this cr2hdr https://builds.magiclantern.fm/utilities.html
Thanks, it is clear now. :D
Quote from: Walter Schulz on March 20, 2018, 01:39:45 PM...or other utilities able to deal with interlaced ISO.
There are others?
Quote from: Walter Schulz on March 20, 2018, 01:39:45 PM
...or other utilities able to deal with interlaced ISO.
> There are others?
Would someone who knows comment please? I'm sure that others would be interested to know that there are other utilities able to process dual ISO files.
Are you on mac? Switch maybe works for you.
Quote from: Danne on April 07, 2018, 12:41:45 PMAre you on mac?
No Windoze 7.
Thanks for the suggestion, though.
Hello,
first of all welcome and kudos for all participating in development of ML! That's a great work!
The Dual ISO feature was the most exciting one, wanting me try ML out. Unfortunately, years ago I had an unsupported 450D and now I'm still quite unlucky with both of my actual bodies (700D & 70D) which should theoretically be supported.
700D
The camera was bought late 2016. Its S/N starts with 42*, so it's most probably (https://www.the-digital-picture.com/Canon-Lenses/Canon-Lens-Aging.aspx) manufactured June 2016 and thus it's a late production model. Originally delivered with FW 1.1.4, later updated to 1.1.5. I was using ML versions for both firmware versions.
On all versions of ML for FW 1.1.4 - 1.1.5, activating the Dual ISO leads to an immediate error "ISOless PH err(x)". Like in https://bitbucket.org/hudson/magic-lantern/issues/2427/700d-dual_iso-isoless-ph-err-15 (https://bitbucket.org/hudson/magic-lantern/issues/2427/700d-dual_iso-isoless-ph-err-15) but with a different error number (can't recollect now and the camera is away but I can reproduce it anytime if it helps). Trying to restart camera, change mode (M, Av, P...), altering settings in the original firmware did not help. I tried a dozen of times during the 1,5 year of when the camera is in my possession. Never made it working.
Does anyone feel fond of digging in (A1ex?)? How could I support you? Should I prepare ROM and RAM dumps?
70D
Serial number starts with 26*, so the camera is probably from February 2015. Much newer than when porting to 70D started. Two versions of ML for 70D, now merged into one build, were developed for cameras manufactured in 2013 (S/N were starting with 0* and 12*).
Here it seems to work on the camera level fine. The problem appears when converting the cr2 into dng using the provided cr2hdr (https://builds.magiclantern.fm/jenkins/job/cr2hdr-20bit/). I have tried builds 18 and 27. Both behave the same.
The file after conversion seems to be the bright frame only. All highlights, which are clearly visible in the cr2 are gone in dng. I am using ACR 10 in my workflow but tried also LR 6 with the same result (including usage of LR plugin). However, FastStone Viewer does not show this behaviour, it opens the dng file preserving both highlights and shadows, so it made me think that maybe ACR interprets the file differently. But... you all seem to use ACR or LR, so it should work. I tried also some dual iso cr2 samples from different forums, converting them on my own with cr2hdr and they opened fine, so it seemed to be related to my camera.
Original CR2 from my 70D with AFAIR 100/1600 Dual ISO (1:1 crop):
(https://s25.postimg.cc/64kfzlpsv/01_70d_dual_iso_cr2_100pct_crop.png)
Final DNG after processing with cr2hdr build 27 with default settings. The highlights are missing.
(https://s25.postimg.cc/htofnk91r/02_70d_dual_iso_dng_100pct_crop.png)
Here final DNG after darkening in ACR. You can already recognize artifacts due to reduced resolution and... again the missing highlights.
(https://s25.postimg.cc/qbxvrw7un/03_70d_dual_iso_dng_100pct_crop_artifacts.png)
Fun fact: when running cr2hdr with --debug-blend the fullres_smooth.dng looks perfect (it seems to include the highlights from dark frame and shadows from bright frame). I believe the only difference between fullres_smooth and final dng should be the lack of EXIF in fullres_smooth. So, I tried stripping the final dng of the complete EXIF (exiftool -all= xxx.dng; btw awesome manual but tl;dr) and... yes, now this file opens in ACR fine, same as fullres_smooth. Here my full appreciation, the hack is gorgeous! I am pleased to see the significant improvement in shadows pushed by +4EV whereas the partial loss of vertical resolution is for me a low price to pay, having a 20Mpx sensor. Finally my camera can compete with my wife's Pentax K3 in the noise category. ;D
Here fullres_smooth.dng (or final with stripped down EXIF). All data is in!
(https://s25.postimg.cc/g1vgso54f/04_70d_dual_iso_dng_fullressmooth_100pct_crop.png)
The next step could be analyzing what fields are set in EXIF and track the one property down, which causes ACR to cut the highlights. I believe maybe there is something about the bit depth, causing ACR to overflow in the bitdepth it is using?
Summary: the problem is camera specific, as other dual iso cr2, which I downloaded from internet (e.g. from 650D) are opening after conversion with my toolchain properly. My 70D is, however, affected.
Could you please have a look at the problem, so the converter works out of the box, preventing the others falling into the same issue as me?
Attachments:
original CR2 [removed, PM if you need it]
final DNG processed with cr2hdr build 27 [removed, PM if you need it]
conversion log:
c:\PortableApps\MagicLanternDualISO>cr2hdr IMG_3925.CR2
cr2hdr: a post processing tool for Dual ISO images
Last update: 06fae84 on 2017-12-03 23:53:11 UTC by alex:
cr2hdr: experiment - yet another exposure matching algorithm
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpol
ation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
Input file : IMG_3925.CR2
Camera : Canon EOS 70D
Full size : 5568 x 3708
Active area : 5496 x 3670
Black borders : 72 left, 38 top
Black level : 2048
ISO pattern : dBBd RGGB
White levels : 13485 12090
Noise levels : 6.48 11.06 11.45 6.39 (14-bit)
Linear fit : y = 8.1888*x - 1.39
Linear fit : y = 8.1838*x - 1.25
Linear fit : y = 8.1826*x - 1.25
Linear fit : y = 8.1817*x - 1.24
Linear fit : y = 8.1806*x - 1.24
ISO difference : 3.03 EV (818)
Black delta : 4.33
Black adjust : -0.0
Dynamic range : 10.81 (+) 9.78 => 12.81 EV (in theory)
Looking for hot/cold pixels...
Hot pixels : 63
Cold pixels : 10379
AMaZE interpolation ...
Amaze took 2.24 s
Edge-directed interpolation...
Semi-overexposed: 29.70%
Deep shadows : 55.25%
Horizontal stripe fix...
38: offset too large (18327)
3707: offset too large (-8555)
Full-res reconstruction...
ISO overlap : 4.8 EV (approx)
Half-res blending...
Chroma smoothing...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level : 71.72 (20-bit), ideally 71.59
Dynamic range : 13.32 EV (cooked)
Black adjust : -0.0
AsShotNeutral : 0.49 1 0.60, 5222K/g=0.96 (gray max)
Output file : IMG_3925.DNG (already exists, overwriting)
IMG_3925.DNG : copying EXIF from IMG_3925.CR2
Found something:
White Level : 53940
Normal White Level : 11765
Specular White Level : 12277
Does it help if you delete the last two tags, or set them to the same value as White Level? (sorry, I don't use ACR)
BTW, Adobe's own dng_validate renders this DNG properly, without discarding any highlights. RawTherapee, Darktable and dcraw are also rendering the file properly.
There's a bug in older builds of adobe which messes up the 70D tag in acr in dng files even though it's a perfectly valid file. Seems fixed in later versions. It's possible to change 70D to 6D tag and it opens. Didn't check the provided file.
@ArcziPL
I had the same problem (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197139#msg197139) on 700D, Fortunately it resolved and you can test the build in the same page (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197139#msg197139) make sure you are on 1.1.5 canon firmware and report back please.
@a1ex @Danne
Brilliant, the tag UniqueCameraModel changed from "Canon EOS 70D" to "Canon EOS 6D" or actually something artificial like e.g. "Canon EOS 70D DualISO" indeed causes the image to open properly in ACR 9 and 10. It means they really treat the data differently for just this camera model. Funny.
Regarding the tags NormalWhiteLevel and SpecularWhiteLevel -- they have some influence. I changed them to the value of 53940, later to the max 65535 (exiftool reports, that the type of this property is int16u). There are then some more details in highlights visible but still not all.
I have checked how does Adobe DNG converter modify EXIF when converting a standard non-DualISO CR2 into DNG.
Here an example:
cr2:
ExifToolVersion : 10.02
FileName : IMG_3923.cr2
Directory : .
FileSize : 23 MB
FileModifyDate : 2018:04:02 14:18:07+02:00
FileAccessDate : 2018:04:11 19:48:41+02:00
FileCreateDate : 2018:04:11 19:48:41+02:00
FilePermissions : rw-rw-rw-
FileType : CR2
FileTypeExtension : cr2
MIMEType : image/x-canon-cr2
ExifByteOrder : Little-endian (Intel, II)
ImageWidth : 5472
ImageHeight : 3648
BitsPerSample : 8 8 8
Compression : JPEG (old-style)
Make : Canon
Model : Canon EOS 70D
PreviewImageStart : 91316
Orientation : Horizontal (normal)
PreviewImageLength : 1410814
XResolution : 72
YResolution : 72
ResolutionUnit : inches
ModifyDate : 2018:04:02 14:18:05
Artist :
Rating : 0
Copyright :
ExposureTime : 1/50
FNumber : 4.5
ExposureProgram : Manual
ISO : 200
SensitivityType : Recommended Exposure Index
RecommendedExposureIndex : 200
ExifVersion : 0230
DateTimeOriginal : 2018:04:02 14:18:05
CreateDate : 2018:04:02 14:18:05
ComponentsConfiguration : Y, Cb, Cr, -
ShutterSpeedValue : 1/49
ApertureValue : 4.6
Flash : Off, Did not fire
FocalLength : 17.0 mm
MacroMode : Normal
SelfTimer : Off
Quality : RAW
CanonFlashMode : Off
ContinuousDrive : Single, Silent
FocusMode : One-shot AF
RecordMode : CR2
CanonImageSize : n/a
EasyMode : Manual
DigitalZoom : None
Contrast : Normal
Saturation : Normal
MeteringMode : Evaluative
FocusRange : Not Known
CanonExposureMode : Manual
LensType : Canon EF-S 17-55mm f/2.8 IS USM
MaxFocalLength : 55 mm
MinFocalLength : 17 mm
FocalUnits : 1/mm
MaxAperture : 2.8
MinAperture : 23
FlashActivity : 0
FlashBits : (none)
ZoomSourceWidth : 0
ZoomTargetWidth : 0
ManualFlashOutput : n/a
ColorTone : Normal
SRAWQuality : n/a
AutoISO : 100
BaseISO : 200
MeasuredEV : 9.25
TargetAperture : 4.5
TargetExposureTime : 1
ExposureCompensation : 0
WhiteBalance : Auto
SlowShutter : None
SequenceNumber : 0
OpticalZoomCode : n/a
CameraTemperature : 31 C
FlashGuideNumber : 0
FlashExposureComp : 0
AutoExposureBracketing : Off
AEBBracketValue : 0
ControlMode : Camera Local Control
MeasuredEV2 : 9.5
BulbDuration : 0
CameraType : EOS High-end
NDFilter : n/a
CanonImageType : Canon EOS 70D
CanonFirmwareVersion : Firmware Version 1.1.2
CameraOrientation : Horizontal (normal)
FirmwareVersion : 1.1.2
FileIndex : 3923
DirectoryIndex : 100
ContrastStandard : 0
SharpnessStandard : 3
SaturationStandard : 0
ColorToneStandard : 0
ContrastPortrait : 0
SharpnessPortrait : 2
SaturationPortrait : 0
ColorTonePortrait : 0
ContrastLandscape : 0
SharpnessLandscape : 4
SaturationLandscape : 0
ColorToneLandscape : 0
ContrastNeutral : 0
SharpnessNeutral : 0
SaturationNeutral : 0
ColorToneNeutral : 0
ContrastFaithful : 0
SharpnessFaithful : 0
SaturationFaithful : 0
ColorToneFaithful : 0
ContrastMonochrome : 0
SharpnessMonochrome : 3
FilterEffectMonochrome : None
ToningEffectMonochrome : None
ContrastAuto : 0
SharpnessAuto : 3
SaturationAuto : 0
ColorToneAuto : 0
FilterEffectAuto : n/a
ToningEffectAuto : n/a
ContrastUserDef1 : 0
SharpnessUserDef1 : 3
SaturationUserDef1 : 0
ColorToneUserDef1 : 0
FilterEffectUserDef1 : None
ToningEffectUserDef1 : None
ContrastUserDef2 : 0
SharpnessUserDef2 : 3
SaturationUserDef2 : 0
ColorToneUserDef2 : 0
FilterEffectUserDef2 : None
ToningEffectUserDef2 : None
ContrastUserDef3 : 0
SharpnessUserDef3 : 3
SaturationUserDef3 : 0
ColorToneUserDef3 : 0
FilterEffectUserDef3 : None
ToningEffectUserDef3 : None
UserDef1PictureStyle : Auto
UserDef2PictureStyle : Auto
UserDef3PictureStyle : Auto
CanonModelID : EOS 70D
ThumbnailImageValidArea : 0 159 7 112
AFAreaMode : Flexizone Single
NumAFPoints : 31
ValidAFPoints : 1
CanonImageWidth : 5472
CanonImageHeight : 3648
AFImageWidth : 5472
AFImageHeight : 3648
AFAreaWidths : 552 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AFAreaHeights : 736 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AFAreaXPositions : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AFAreaYPositions : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AFPointsInFocus : 0
AFPointsSelected : 0
TimeZone : +00:00
TimeZoneCity : London
DaylightSavings : Off
BracketMode : Off
BracketValue : 0
BracketShotNumber : 0
RawJpgSize : Large
WBBracketMode : Off
WBBracketValueAB : 0
WBBracketValueGM : 0
LiveViewShooting : On
FocusDistanceUpper : 1.63 m
FocusDistanceLower : 1.45 m
FlashExposureLock : Off
InternalSerialNumber : FA1848004
DustRemovalData : (Binary data 1024 bytes, use -b option to extract)
CropLeftMargin : 0
CropRightMargin : 0
CropTopMargin : 0
CropBottomMargin : 0
ExposureLevelIncrements : 1/3 Stop
ISOSpeedIncrements : 1 Stop
AEBAutoCancel : On
AEBSequence : 0,-,+
AEBShotCount : 3 shots
SafetyShift : Disable
AIServoTrackingSensitivity : Standard
AccelerationTracking : 0
AIServoFirstImagePriority : Equal priority
AIServoSecondImagePriority : Equal priority
AFAssistBeam : Emits
LensDriveNoAF : Focus search on
SelectAFAreaSelectMode : Disable; Flags 0x7
AFAreaSelectMethod : AF area selection button
OrientationLinkedAFPoint : Same for vertical and horizontal
ManualAFPointSelectPattern : Stops at AF area edges
AFPointDisplayDuringFocus : On
SuperimposedDisplay : Off
AFMicroadjustment : Adjust all by same amount; 0; 4; 7; 2; 10; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0
DialDirectionTvAv : Normal
MultiFunctionLock : Off; Quick control dial
ViewfinderWarnings : Monochrome, WB corrected
CustomControls : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AspectRatio : 3:2
CroppedImageWidth : 5472
CroppedImageHeight : 3648
CroppedImageLeft : 0
CroppedImageTop : 0
ToneCurve : Standard
Sharpness : 3
SharpnessFrequency : n/a
SensorRedLevel : 0
SensorBlueLevel : 0
WhiteBalanceRed : 0
WhiteBalanceBlue : 0
ColorTemperature : 5200
PictureStyle : Auto
DigitalGain : 0
WBShiftAB : 0
WBShiftGM : 0
MeasuredRGGB : 449 1024 1024 735
VRDOffset : 0
SensorWidth : 5568
SensorHeight : 3708
SensorLeftBorder : 84
SensorTopBorder : 50
SensorRightBorder : 5555
SensorBottomBorder : 3697
BlackMaskLeftBorder : 0
BlackMaskTopBorder : 0
BlackMaskRightBorder : 0
BlackMaskBottomBorder : 0
ColorDataVersion : 10 (1DX/5DmkIII/6D/70D/100D/650D/700D/M)
WB_RGGBLevelsAsShot : 2180 1024 1024 1646
ColorTempAsShot : 5600
WB_RGGBLevelsAuto : 2180 1024 1024 1646
ColorTempAuto : 5600
WB_RGGBLevelsMeasured : 2180 1024 1024 1646
ColorTempMeasured : 5600
WB_RGGBLevelsDaylight : 2123 1024 1024 1733
ColorTempDaylight : 5200
WB_RGGBLevelsShade : 2485 1024 1024 1494
ColorTempShade : 7000
WB_RGGBLevelsCloudy : 2294 1024 1024 1606
ColorTempCloudy : 6000
WB_RGGBLevelsTungsten : 1485 1024 1024 2515
ColorTempTungsten : 3200
WB_RGGBLevelsFluorescent : 1824 1024 1024 2383
ColorTempFluorescent : 3733
WB_RGGBLevelsKelvin : 2123 1024 1024 1733
ColorTempKelvin : 5200
WB_RGGBLevelsFlash : 2356 1024 1024 1603
ColorTempFlash : 6185
AverageBlackLevel : 2049 2049 2049 2049
RawMeasuredRGGB : 371333 863562 865350 598106
PerChannelBlackLevel : 2049 2049 2049 2049
NormalWhiteLevel : 14580
SpecularWhiteLevel : 15092
LinearityUpperMargin : 10000
CustomPictureStyleFileName :
AFMicroAdjMode : Disable
AFMicroAdjValue : 0
VignettingCorrVersion : 0
PeripheralLighting : On
ChromaticAberrationCorr : On
PeripheralLightingValue : 66
OriginalImageWidth : 5472
OriginalImageHeight : 3648
PeripheralLightingSetting : On
ChromaticAberrationSetting : On
PeripheralIlluminationCorr : Off
AutoLightingOptimizer : Off
HighlightTonePriority : Off
LongExposureNoiseReduction : Off
HighISONoiseReduction : Standard
AmbienceSelection : Standard
MultiExposure : Off
MultiExposureControl : Additive
MultiExposureShots : 0
GrainyBWFilter : Off
SoftFocusFilter : Off
ToyCameraFilter : Off
MiniatureFilter : Off
MiniatureFilterOrientation : Horizontal
MiniatureFilterPosition : 0
MiniatureFilterParameter : 0
FisheyeFilter : Off
PaintingFilter : Off
WatercolorFilter : Off
HDR : Off
HDREffect : Natural
UserComment :
SubSecTime : 30
SubSecTimeOriginal : 30
SubSecTimeDigitized : 30
FlashpixVersion : 0100
ColorSpace : sRGB
ExifImageWidth : 5472
ExifImageHeight : 3648
InteropIndex : R98 - DCF basic file (sRGB)
InteropVersion : 0100
FocalPlaneXResolution : 6086.76307
FocalPlaneYResolution : 6090.15025
FocalPlaneResolutionUnit : inches
CustomRendered : Normal
ExposureMode : Manual
SceneCaptureType : Standard
OwnerName :
SerialNumber : 263058002874
LensInfo : 17-55mm f/0
LensModel : EF-S17-55mm f/2.8 IS USM
LensSerialNumber : 000003f54b
GPSVersionID : 2.3.0.0
ThumbnailOffset : 80600
ThumbnailLength : 10714
PhotometricInterpretation : RGB
SamplesPerPixel : 3
RowsPerStrip : 309
PlanarConfiguration : Chunky
StripOffsets : 2362388
StripByteCounts : 21303278
RawImageSegmentation : 1 2784 2784
SRawType : 1
Aperture : 4.5
DriveMode : Continuous Shooting
FileNumber : 100-3923
ImageSize : 5472x3648
Lens : 17.0 - 55.0 mm
LensID : Canon EF-S 17-55mm f/2.8 IS USM
Megapixels : 20.0
PreviewImage : (Binary data 1410814 bytes, use -b option to extract)
ScaleFactor35efl : 1.6
ShootingMode : Manual
ShutterSpeed : 1/50
SubSecCreateDate : 2018:04:02 14:18:05.30
SubSecDateTimeOriginal : 2018:04:02 14:18:05.30
SubSecModifyDate : 2018:04:02 14:18:05.30
ThumbnailImage : (Binary data 10714 bytes, use -b option to extract)
WB_RGGBLevels : 2180 1024 1024 1646
BlueBalance : 1.607422
CircleOfConfusion : 0.019 mm
DOF : 1.75 m (1.06 - 2.81 m)
FOV : 67.8 deg
FocalLength35efl : 17.0 mm (35 mm equivalent: 26.8 mm)
HyperfocalDistance : 3.37 m
Lens35efl : 17.0 - 55.0 mm (35 mm equivalent: 26.8 - 86.7 mm)
LightValue : 9.0
RedBalance : 2.128906
dng from Adobe DNG Converter:
ExifToolVersion : 10.02
FileName : IMG_3923.dng
Directory : .
FileSize : 19 MB
FileModifyDate : 2018:04:11 19:19:49+02:00
FileAccessDate : 2018:04:11 19:48:41+02:00
FileCreateDate : 2018:04:11 19:48:41+02:00
FilePermissions : rw-rw-rw-
FileType : DNG
FileTypeExtension : dng
MIMEType : image/x-adobe-dng
ExifByteOrder : Little-endian (Intel, II)
Make : Canon
Model : Canon EOS 70D
StripOffsets : 200698
Orientation : Horizontal (normal)
RowsPerStrip : 171
StripByteCounts : 131328
Software : Adobe DNG Converter 9.2 (Windows)
ModifyDate : 2018:04:11 19:19:49
ImageWidth : 5568
ImageHeight : 3708
BitsPerSample : 16
Compression : JPEG
PhotometricInterpretation : Color Filter Array
SamplesPerPixel : 1
PlanarConfiguration : Chunky
TileWidth : 256
TileLength : 256
TileOffsets : (Binary data 2823 bytes, use -b option to extract)
TileByteCounts : (Binary data 1979 bytes, use -b option to extract)
CFARepeatPatternDim : 2 2
CFAPattern2 : 0 1 1 2
CFAPlaneColor : Red,Green,Blue
CFALayout : Rectangular
BlackLevelRepeatDim : 2 2
BlackLevel : 2049 2049 2049 2049
WhiteLevel : 15000
DefaultScale : 1 1
DefaultCropOrigin : 12 12
DefaultCropSize : 5472 3648
BayerGreenSplit : 250
AntiAliasStrength : 1
BestQualityScale : 1
ActiveArea : 38 72 3708 5568
SubfileType : Reduced-resolution image
PreviewImageStart : 332026
PreviewImageLength : 53907
YCbCrCoefficients : 0.299 0.587 0.114
YCbCrSubSampling : YCbCr4:2:0 (2 2)
YCbCrPositioning : Co-sited
ReferenceBlackWhite : 0 255 128 255 128 255
XMPToolkit : Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37
CreatorTool : Adobe DNG Converter 9.2 (Windows)
MetadataDate : 2018:04:11 19:19:49+02:00
ApproximateFocusDistance : 1.51
FlashCompensation : 0
Firmware : 1.1.2
DateCreated : 2018:04:02 14:18:05.30
DocumentID : xmp.did:5305c71a-0826-3d42-b563-40ad6b1fcf98
OriginalDocumentID : 6D2FB39D88BAA646697AFE1DA1A160C8
InstanceID : xmp.iid:5305c71a-0826-3d42-b563-40ad6b1fcf98
Format : image/dng
HistoryAction : derived, saved
HistoryParameters : converted from image/x-canon-cr2 to image/dng, saved to new location
HistoryInstanceID : xmp.iid:5305c71a-0826-3d42-b563-40ad6b1fcf98
HistoryWhen : 2018:04:11 19:19:49+02:00
HistorySoftwareAgent : Adobe DNG Converter 9.2 (Windows)
HistoryChanged : /
DerivedFromDocumentID : 6D2FB39D88BAA646697AFE1DA1A160C8
DerivedFromOriginalDocumentID : 6D2FB39D88BAA646697AFE1DA1A160C8
ExposureTime : 1/50
FNumber : 4.5
ExposureProgram : Manual
ISO : 200
SensitivityType : Recommended Exposure Index
RecommendedExposureIndex : 200
ExifVersion : 0230
DateTimeOriginal : 2018:04:02 14:18:05
CreateDate : 2018:04:02 14:18:05
ShutterSpeedValue : 1/50
ApertureValue : 4.5
MaxApertureValue : 2.8
Flash : Off, Did not fire
FocalLength : 17.0 mm
SubSecTimeOriginal : 30
SubSecTimeDigitized : 30
FocalPlaneXResolution : 2452.118652
FocalPlaneYResolution : 2452.118652
FocalPlaneResolutionUnit : cm
CustomRendered : Normal
ExposureMode : Manual
SceneCaptureType : Standard
SerialNumber : 263058002874
LensInfo : 17-55mm f/?
ImageNumber : 0
DNGVersion : 1.4.0.0
DNGBackwardVersion : 1.1.0.0
UniqueCameraModel : Canon EOS 70D
ColorMatrix1 : 0.7546 -0.1435 -0.0929 -0.3846 1.1488 0.2692 -0.0332 0.1209 0.637
ColorMatrix2 : 0.7034 -0.0804 -0.1014 -0.442 1.2564 0.2058 -0.0851 0.1994 0.5758
CameraCalibration1 : 0.9534 0 0 0 1 0 0 0 0.9602
CameraCalibration2 : 0.9534 0 0 0 1 0 0 0 0.9602
AnalogBalance : 1 1 1
AsShotNeutral : 0.469725 1 0.622114
BaselineExposure : 0.25
BaselineNoise : 0.8
BaselineSharpness : 1.25
LinearResponseLimit : 1
CameraSerialNumber : 263058002874
DNGLensInfo : 17-55mm f/?
ShadowScale : 1
MacroMode : Normal
SelfTimer : Off
Quality : RAW
CanonFlashMode : Off
ContinuousDrive : Single, Silent
FocusMode : One-shot AF
RecordMode : CR2
CanonImageSize : n/a
EasyMode : Manual
DigitalZoom : None
Contrast : Normal
Saturation : Normal
MeteringMode : Evaluative
FocusRange : Not Known
CanonExposureMode : Manual
LensType : Canon EF-S 17-55mm f/2.8 IS USM
MaxFocalLength : 55 mm
MinFocalLength : 17 mm
FocalUnits : 1/mm
MaxAperture : 2.8
MinAperture : 23
FlashActivity : 0
FlashBits : (none)
ZoomSourceWidth : 0
ZoomTargetWidth : 0
ManualFlashOutput : n/a
ColorTone : Normal
SRAWQuality : n/a
AutoISO : 100
BaseISO : 200
MeasuredEV : 9.25
TargetAperture : 4.5
TargetExposureTime : 1
ExposureCompensation : 0
WhiteBalance : Auto
SlowShutter : None
SequenceNumber : 0
OpticalZoomCode : n/a
CameraTemperature : 31 C
FlashGuideNumber : 0
FlashExposureComp : 0
AutoExposureBracketing : Off
AEBBracketValue : 0
ControlMode : Camera Local Control
MeasuredEV2 : 9.5
BulbDuration : 0
CameraType : EOS High-end
NDFilter : n/a
CanonImageType : Canon EOS 70D
CanonFirmwareVersion : Firmware Version 1.1.2
OwnerName :
CameraOrientation : Horizontal (normal)
FirmwareVersion : 1.1.2
FileIndex : 3923
DirectoryIndex : 100
ContrastStandard : 0
SharpnessStandard : 3
SaturationStandard : 0
ColorToneStandard : 0
ContrastPortrait : 0
SharpnessPortrait : 2
SaturationPortrait : 0
ColorTonePortrait : 0
ContrastLandscape : 0
SharpnessLandscape : 4
SaturationLandscape : 0
ColorToneLandscape : 0
ContrastNeutral : 0
SharpnessNeutral : 0
SaturationNeutral : 0
ColorToneNeutral : 0
ContrastFaithful : 0
SharpnessFaithful : 0
SaturationFaithful : 0
ColorToneFaithful : 0
ContrastMonochrome : 0
SharpnessMonochrome : 3
FilterEffectMonochrome : None
ToningEffectMonochrome : None
ContrastAuto : 0
SharpnessAuto : 3
SaturationAuto : 0
ColorToneAuto : 0
FilterEffectAuto : n/a
ToningEffectAuto : n/a
ContrastUserDef1 : 0
SharpnessUserDef1 : 3
SaturationUserDef1 : 0
ColorToneUserDef1 : 0
FilterEffectUserDef1 : None
ToningEffectUserDef1 : None
ContrastUserDef2 : 0
SharpnessUserDef2 : 3
SaturationUserDef2 : 0
ColorToneUserDef2 : 0
FilterEffectUserDef2 : None
ToningEffectUserDef2 : None
ContrastUserDef3 : 0
SharpnessUserDef3 : 3
SaturationUserDef3 : 0
ColorToneUserDef3 : 0
FilterEffectUserDef3 : None
ToningEffectUserDef3 : None
UserDef1PictureStyle : Auto
UserDef2PictureStyle : Auto
UserDef3PictureStyle : Auto
CanonModelID : EOS 70D
ThumbnailImageValidArea : 0 159 7 112
AFAreaMode : Flexizone Single
NumAFPoints : 31
ValidAFPoints : 1
CanonImageWidth : 5472
CanonImageHeight : 3648
AFImageWidth : 5472
AFImageHeight : 3648
AFAreaWidths : 552 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AFAreaHeights : 736 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AFAreaXPositions : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AFAreaYPositions : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AFPointsInFocus : 0
AFPointsSelected : 0
TimeZone : +00:00
TimeZoneCity : London
DaylightSavings : Off
BracketMode : Off
BracketValue : 0
BracketShotNumber : 0
RawJpgSize : Large
WBBracketMode : Off
WBBracketValueAB : 0
WBBracketValueGM : 0
LiveViewShooting : On
FocusDistanceUpper : 1.63 m
FocusDistanceLower : 1.45 m
FlashExposureLock : Off
LensModel : EF-S17-55mm f/2.8 IS USM
InternalSerialNumber : FA1848004
DustRemovalData : (Binary data 1024 bytes, use -b option to extract)
CropLeftMargin : 0
CropRightMargin : 0
CropTopMargin : 0
CropBottomMargin : 0
ExposureLevelIncrements : 1/3 Stop
ISOSpeedIncrements : 1 Stop
AEBAutoCancel : On
AEBSequence : 0,-,+
AEBShotCount : 3 shots
SafetyShift : Disable
AIServoTrackingSensitivity : Standard
AccelerationTracking : 0
AIServoFirstImagePriority : Equal priority
AIServoSecondImagePriority : Equal priority
AFAssistBeam : Emits
LensDriveNoAF : Focus search on
SelectAFAreaSelectMode : Disable; Flags 0x7
AFAreaSelectMethod : AF area selection button
OrientationLinkedAFPoint : Same for vertical and horizontal
ManualAFPointSelectPattern : Stops at AF area edges
AFPointDisplayDuringFocus : On
SuperimposedDisplay : Off
AFMicroadjustment : Adjust all by same amount; 0; 4; 7; 2; 10; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0
DialDirectionTvAv : Normal
MultiFunctionLock : Off; Quick control dial
ViewfinderWarnings : Monochrome, WB corrected
CustomControls : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
AspectRatio : 3:2
CroppedImageWidth : 5472
CroppedImageHeight : 3648
CroppedImageLeft : 0
CroppedImageTop : 0
ToneCurve : Standard
Sharpness : 3
SharpnessFrequency : n/a
SensorRedLevel : 0
SensorBlueLevel : 0
WhiteBalanceRed : 0
WhiteBalanceBlue : 0
ColorTemperature : 5200
PictureStyle : Auto
DigitalGain : 0
WBShiftAB : 0
WBShiftGM : 0
MeasuredRGGB : 449 1024 1024 735
ColorSpace : sRGB
VRDOffset : 0
SensorWidth : 5568
SensorHeight : 3708
SensorLeftBorder : 84
SensorTopBorder : 50
SensorRightBorder : 5555
SensorBottomBorder : 3697
BlackMaskLeftBorder : 0
BlackMaskTopBorder : 0
BlackMaskRightBorder : 0
BlackMaskBottomBorder : 0
ColorDataVersion : 10 (1DX/5DmkIII/6D/70D/100D/650D/700D/M)
WB_RGGBLevelsAsShot : 2180 1024 1024 1646
ColorTempAsShot : 5600
WB_RGGBLevelsAuto : 2180 1024 1024 1646
ColorTempAuto : 5600
WB_RGGBLevelsMeasured : 2180 1024 1024 1646
ColorTempMeasured : 5600
WB_RGGBLevelsDaylight : 2123 1024 1024 1733
ColorTempDaylight : 5200
WB_RGGBLevelsShade : 2485 1024 1024 1494
ColorTempShade : 7000
WB_RGGBLevelsCloudy : 2294 1024 1024 1606
ColorTempCloudy : 6000
WB_RGGBLevelsTungsten : 1485 1024 1024 2515
ColorTempTungsten : 3200
WB_RGGBLevelsFluorescent : 1824 1024 1024 2383
ColorTempFluorescent : 3733
WB_RGGBLevelsKelvin : 2123 1024 1024 1733
ColorTempKelvin : 5200
WB_RGGBLevelsFlash : 2356 1024 1024 1603
ColorTempFlash : 6185
AverageBlackLevel : 2049 2049 2049 2049
RawMeasuredRGGB : 371333 863562 865350 598106
PerChannelBlackLevel : 2049 2049 2049 2049
NormalWhiteLevel : 14580
SpecularWhiteLevel : 15092
LinearityUpperMargin : 10000
CustomPictureStyleFileName :
AFMicroAdjMode : Disable
AFMicroAdjValue : 0
VignettingCorrVersion : 0
PeripheralLighting : On
ChromaticAberrationCorr : On
PeripheralLightingValue : 66
OriginalImageWidth : 5472
OriginalImageHeight : 3648
PeripheralLightingSetting : On
ChromaticAberrationSetting : On
PeripheralIlluminationCorr : Off
AutoLightingOptimizer : Off
HighlightTonePriority : Off
LongExposureNoiseReduction : Off
HighISONoiseReduction : Standard
LensSerialNumber : 000003f54b
AmbienceSelection : Standard
MultiExposure : Off
MultiExposureControl : Additive
MultiExposureShots : 0
GrainyBWFilter : Off
SoftFocusFilter : Off
ToyCameraFilter : Off
MiniatureFilter : Off
MiniatureFilterOrientation : Horizontal
MiniatureFilterPosition : 0
MiniatureFilterParameter : 0
FisheyeFilter : Off
PaintingFilter : Off
WatercolorFilter : Off
HDR : Off
HDREffect : Natural
CalibrationIlluminant1 : Standard Light A
CalibrationIlluminant2 : D65
RawDataUniqueID : 86EFA44EB616EFA79AED39C100000000
OriginalRawFileName : IMG_3923.CR2
CameraCalibrationSig : com.adobe
ProfileCalibrationSig : com.adobe
ProfileName : Adobe Standard
ProfileHueSatMapDims : 90 30 1
ProfileHueSatMapData1 : (Binary data 137575 bytes, use -b option to extract)
ProfileHueSatMapData2 : (Binary data 138379 bytes, use -b option to extract)
ProfileEmbedPolicy : Allow Copying
ProfileCopyright : Copyright 2012 Adobe Systems, Inc.
ForwardMatrix1 : 0.7763 0.0065 0.1815 0.2364 0.8351 -0.0715 -0.0059 -0.4228 1.2538
ForwardMatrix2 : 0.7464 0.1044 0.1135 0.2648 0.9173 -0.182 0.0113 -0.2154 1.0292
PreviewApplicationName : Adobe DNG Converter
PreviewApplicationVersion : 9.2
PreviewSettingsDigest : 0bdfc91724df61e7a1410aecd643c850
PreviewColorSpace : 2
PreviewDateTime : 2018:04:11 19:19:48+02:00
ProfileLookTableDims : 36 8 16
ProfileLookTableData : (Binary data 89143 bytes, use -b option to extract)
NoiseProfile : 7.76470588235294e-005 3.89674928757033e-007
NewRawImageDigest : 4f351b7729d00e95a28dd7c047c052f6
Aperture : 4.5
CFAPattern : [Red,Green][Green,Blue]
DriveMode : Continuous Shooting
FileNumber : 100-3923
ImageSize : 5568x3708
Lens : 17.0 - 55.0 mm
LensID : Canon EF-S 17-55mm f/2.8 IS USM
Megapixels : 20.6
PreviewImage : (Binary data 53907 bytes, use -b option to extract)
ScaleFactor35efl : 1.6
ShootingMode : Manual
ShutterSpeed : 1/50
SubSecCreateDate : 2018:04:02 14:18:05.30
SubSecDateTimeOriginal : 2018:04:02 14:18:05.30
WB_RGGBLevels : 2180 1024 1024 1646
BlueBalance : 1.607422
CircleOfConfusion : 0.019 mm
DOF : 1.69 m (1.07 - 2.76 m)
FOV : 66.6 deg
FocalLength35efl : 17.0 mm (35 mm equivalent: 27.4 mm)
HyperfocalDistance : 3.45 m
Lens35efl : 17.0 - 55.0 mm (35 mm equivalent: 27.4 - 88.7 mm)
LightValue : 9.0
RedBalance : 2.128906
It does actually alter some properties, but e.g. UniqueCameraModel stay the same. Both files look identical in ACR 9 and 10.
@theBilalFakhouri
Your build magiclantern-crop_rec_4k.2018Apr11.700D115_Iso_Less_8_15_resolved.zip with altered DualISO works for my camera well! First time! My 700D is also "Made in Taiwan", distribution for Germany from 2016. Thank you a lot! Now the question is, how to identify both camera versions, to be able to implement conditionally both adresses in the official build...
BTW, before trying your code blindly, I wanted to check if the registers' addresses match your. I started with enabling ADTG to find out, it doesn't load, requires a special branch, which according to the docu found, is only for 5D... A scavenger hunt. Uhhh, the barieer to jump into debugging and development seems to be huge. Your amazing knowledge is distributed among the forum topics, like here: over 100 pages for DualISO "only", with partially dead links and images. I believe weeks are needed to go through it to be up to date and manage to just configure the workflow... :(
Quote from: ArcziPL on April 11, 2018, 08:38:34 PM
Now the question is, how to identify both camera versions, to be able to implement conditionally both adresses in the official build...
Yes, that is a very good question.
Quote from: ArcziPL on April 11, 2018, 08:38:34 PM
...enabling ADTG to find out, it doesn't load, requires a special branch, which according to the docu found, is only for 5D... A scavenger hunt. Uhhh, the barieer to jump into debugging and development seems to be huge...
Not that huge. First make sure you're using a crop_rec_4k experimental build (https://builds.magiclantern.fm/experiments.html), download the adtg_gui module from the modules download page (https://builds.magiclantern.fm/modules.html), put it in the ML/MODULES directory on your card, activate the module and it should work.
If you want something more powerful, set up a development environment and compile the iso-research branch for your camera. It isn't hard at all to set up a development system. I made tutorials for both Macintosh (https://www.magiclantern.fm/forum/index.php?topic=16012.0) and Windows/Cygwin (https://www.magiclantern.fm/forum/index.php?topic=15894.0). Danne recently released a compiler tool for Mac (https://www.magiclantern.fm/forum/index.php?topic=21882.0) that makes it super easy to get set up and compile ML.
Good docu, thanks for sharing & linking! I would however start with debugger/emulator environment first and run my own dumps. Compiling ML is second in a row.
12bit MLV file used(13mb) cut in Mlv app:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M18-1354.MLV
Hi. Since recording rates are getting better with lossless but also with using the sd_uhs it seems even more interesting to get dual iso movie recording working also with at least 12bit files. While 14bit really looks awesome filming with 3xzoom mode 12bits are still problematic(check stripes after conversion):
(https://s7.postimg.cc/5ndplqtyj/Screen_Shot_2018-04-19_at_08_500px.png)
Checking dual iso code sources and it´s really some of the most impressive stuff I´ve seen. I don´t know where to start even looking for tweaks in there so I thought I´d post an example MLV if maybe Alex or somenone else wants to have a look.
Seem to be a problem in the High ISO (Clipping --Pink Hi-lights) Run the file though MLVProducer
There you can just pick hi ISO , low ISO , Balanced etc. .... when I adjusted just the Low ISO (100) seem fine
(https://preview.ibb.co/kZA1SS/small.png) (https://ibb.co/mhtzE7)
Thou it took some adjusting to clean it up , ended up using "curves" plus some other grading option
2047-5586 , are the reported while & black levels in mlvproducer -- shouldn't the black level be 2048 ?
I couldn't clean up the hi-ISO & together with the low ISO it's not possible clean up
With fullres blending "off" and AMaZE preview the MLVApp gives me this:
(https://i.imgur.com/2yoeFv7.jpg)
You probably already know, but the stripes seems to be miss-aligned. There's ~6 horizontal stripes that are not in the correct colors (alternate between yellow and blue, probably one channel of YUV chrominance is off).
Quote from: Danne on April 19, 2018, 07:59:27 AM
filming with [...] 12bits are still problematic
Answered here (https://www.magiclantern.fm/forum/index.php?topic=16040.msg196034#msg196034).
Thanks again a1ex.
@Danne
Now, latest MLV App supports your restricted to 12bit lossless dual iso MLV test clip posted above ;)
Without any additional processing applied:
(https://thumb.ibb.co/mpfN2S/M18_1354_frame_1.png) (https://ibb.co/mpfN2S)
regards
bb
Shit, you're kidding me. That was fast. Will it work with even lower bits too?
By the way. Could this be fixed in mlv_dump or cr2hdr as well B?
It should wok for less bits, please try and report. Not sure about cr2hdr ( it's a1ex's exclusive territory :) ), mlv_dump on steroids (or merged) does not have dual iso processing implemented at all.
Here is the sample with small corrections:
(https://thumb.ibb.co/jx7xmn/M18_1354_frame_1.png) (https://ibb.co/jx7xmn)
Edit: dual iso preview mode is broken and shows pinks a lot. Not important I guess.
Crazy good. Mlv app exports dual iso on the fly. Great addition. Will do some testing later.
I never use preview function...
Haven't used dual iso for almost 2 years, but I wanted to give it a try now with crop mode video (1:1) on the 6d.
But I get an error:
ISOless PH err(3)
So it obviously doesn't work anymore on the 6d, I have no idea for how long.
I'm using the "crop_rec_4k.2018Mar10.6D116.zip" build from the experiments download page.
@Levas
On 700D I had this issue and it was resolved. (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197139#msg197139)
Also I got once time
ISOless PH err(3) :
Quote from: theBilalFakhouri on February 13, 2018, 09:31:01 PM
When I adjusted the photo register I wrote it wrong by mistake, I got ISOless err(3), it was after correcting the video register and only dual_iso was working in video mode.
When ISOless PH err(3) appeared in 700D it was the photo mode not working but in video mode was working very well unlike you on 6D .
Did you try
Nightly Builds ? same error?
Is photo mode working ? and the normal uncropped mv1080 mode ?
And it's look like only the Taiwanese models for 700D affected by dual_iso errors:
Quote from: theBilalFakhouri on February 13, 2018, 09:31:01 PM
...my camera was made in Taiwan...
Quote from: ArcziPL on April 11, 2018, 08:38:34 PM
Your build magiclantern-crop_rec_4k.2018Apr11.700D115_Iso_Less_8_15_resolved.zip with altered DualISO works for my camera well! First time! My 700D is also "Made in Taiwan"...
@dfort's 700D camera was made in Japan which works correctly without these errors.
Can other 6D users confirm that?
A much longer explanation of what is going on is on Reply #2968 (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197612#msg197612). Looks like this is affecting 700D, 650D and 6D cameras depending (we suspect) on when and where they were manufactured.
... depending on what's stored in their settings area (properties) and where that stuff happens to be allocated at startup. It can happen on any camera model, but some of them were just lucky.
Conspiracy theories do not apply here; one has to find where that data structure ends up in the memory without guesswork.
Quote from: a1ex on April 23, 2018, 05:18:12 PM
Conspiracy theories do not apply here...
:P
The fix @theBilalFakhouri made on his 700D didn't work on Walter's 650D so yeah, we need a no guesswork solution.
Quote from: a1ex on February 20, 2018, 10:16:38 PM
Found where it's allocated, now to figure out where the result is stored. Or, maybe better, as I prefer to avoid messing with the property data structures - refactor with the crop_rec method for overriding these registers. That would require finding the cmos_write stub for all camera models.
Is this the same as the CMOS_WRITE_FUNC stub used in the adtg_gui module?
Yeah, that's the one, but don't have any proof of concept ready yet.
Disabled all modules on my 6d (Made in Japan, it says on the bottom).
Only activated dual iso module.
Got the 'ISOless PH err(3)' error both in video mode and photo mode...
So it doesn't work in any mode.
Will try some other builds of magic lantern.
My 650D does ISOless PH err(2) and I have to restore ML defaults to get rid of it. Will appear again after using Dual-ISO.
Try before switching to other builds.
My 2 cents, that is.
@Levas - have you tried what @theBilalFakhouri suggested (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197146#msg197146)? It might be a "fix" until we can resolve this conspiracy--I mean find where that data structure ends up in the memory.
@Dfort, Will try it out when I have some time for it, doesn't sound that complicated to try out.
Probably first will check ISO CMOS values with adtg_gui and what they are set to in sourcecode.
Hi all,
Awhile back I had downloaded what I need to take the .mov files and seperate the high and low ISO frames into seperate DNG files. Now for the life of me I cant find the download link of the software I need to do that. Possible for someone to point me in the right direction?
Thanks much!
You posted a question about HDR video (alternating ISO for odd/even frame number) into Dual-ISO (alternating ISO setting for every 2nd pixel line) thread. You're not the first one mixing up both techniques.
HDR described in User Guide is outdated and contains dead links mostly. Sorry! You may want to look into
https://www.magiclantern.fm/forum/index.php?topic=20025.msg201314#msg201314
https://www.magiclantern.fm/forum/index.php?topic=15271.0
Quote from: theBilalFakhouri on February 11, 2018, 02:58:08 PM
Yes, I think it's the same way I did you can do, it's about CMOS 0 Registers at ISO 100, Some cameras have different registers than others for
the same model, (I don't know yet if it will be more than two Registers for same model, waiting 700d users to feedback)
So you have to see your registers on your camera by adtg_gui.mo (https://builds.magiclantern.fm/modules.html) (Assuming adtg_gui.mo is working on canon 650d)
1-Set ISO at 100
2-Set camera on movie mode 1080/24
3-After loading the modules, turn on adtg_gui in Debug submenu
4-Turn on dual_iso
5-Go to PLAY mode then go back to liveview
6-Check your CMOS 0 Registers data and write it down (In the white box)
(https://preview.ibb.co/cRdQT7/VRAM0.png) (https://ibb.co/ktAHan)
7-Now go to photo mode
8-Take a picture and Check your CMOS 0 Registers data and write it down again.
--> Now in dual_iso.c
This value for movie mode
FRAME_CMOS_ISO_START = 0x********;
and this for photo mode:
PHOTO_CMOS_ISO_START = 0x********;
Leave the (0x) and change the others to the values you wrote, Compile and test it in the camera!
Tried this with my 6D which I bought in Japan. Frame CMOS I changed to 40e6190 and photo CMOS I changed to 404e4e06. Still ISOless PH 3. Only Magic-Lantern-4k crop mode has the issue. Not the Nightly build.EDIT: It works! It was only me who didn't read everything. New values in
CMOS 3 and not CMOS 0:
404e61b6
and
404e4e08
I'm a new to Magic Lantern, but have been getting on well with it, so I thought I would try out the duel ISO.
I have just done some tests on my Canon 5D MK3. Recording at either 11-14bit lossless, 16:9, 1920x1080. I'm having issues though, it stops recording after a few frames and buffers. I have a Komputerbay 64GB 1000x card, which normally works fine at higher resolutions. Does duel ISO have a heavy hit on the CPU, or am I doing something wrong?
@Walter Schulz Thank you so much !!
Thanks to dfort I'm now able to use ADTG_GUI with 650D.105!
Regarding a1ex's and dfort's post above in this page (and a1ex's posts some pages ago (quoted below)) I need some clarification: As far as I understood there is a structural deficit in dual_iso code. Tweaking value for PHOTO_CMOS_ISO_START will not be sufficient to get rid of ISOless error once and for all. It *might* work for a particuliar cam and not work for another.
Did I grab the concept, a1ex?
If I'm - somehow - correct (and if I'm not, too):
So, where to go from here? Anything we can do to support you?
Quote from: a1ex on February 20, 2018, 10:16:38 PM
Wait, what?! All these years, we've been writing in the middle of Canon's property data structures without even knowing about it?! Guess we were lucky, as I believe the FIX settings block is probably not written back to the permanent memory at shutdown... (one small piece added to this puzzle (https://www.magiclantern.fm/forum/index.php?topic=19369)).
[...]
Found where it's allocated, now to figure out where the result is stored. Or, maybe better, as I prefer to avoid messing with the property data structures - refactor with the crop_rec method for overriding these registers. That would require finding the cmos_write stub for all camera models.
Quote from: lureb74 on November 10, 2013, 06:17:13 PM
I wonder why is not possible to move more than x files over the cr2hdr-denoice.exe for processing (where x > 495) or you get a windows "The filename or extension is too long" error... someone have a solution?
Anyway around this issue?
Quote from: QUAKE on July 17, 2018, 04:38:11 AM
Anyway around this issue?
https://www.magiclantern.fm/forum/index.php?board=56.0
I made the mistake of processing a directory full of MLV files (using the excellent Switch.app on macOS) and then deleting the MLV files before checking the resulting DNG sequences. 4 of the DNG sequences were my first attempt at using dual ISO, and I did not have Switch set to enable dual ISO processing. So I have several DNG sequences that have the interlaced dark and light line look of unprocessed dual-ISO. I've checked my most recent backups, and unfortunately they were made before I copied these new MLV files to my RAID. It's a perfect storm of absent-mindedness on my part.
Do any of the available dual-ISO tools have the ability to process DNG sequences rather than MLV files? I figure the same information is right there in the DNG waiting to be utilized.
Thanks for any help or information.
You can drag a folder containing dng files onto Switch then open this in then main menu:
(d) cr2hdr dualiso processing(CR2)
then select:
(r) run Switch
Move on to the next dng folder when done.
Note that your dng files will be renamed and erased the in the process. Extension .dng will simply become .DNG so if you want to keep anything, make copies.
Yet another academic paper mentioning Dual ISO:
Reconstructing Interlaced High-Dynamic-Range Video using Joint Learning (http://vclab.kaist.ac.kr/tip2017/IEEETIP2017_Choi-07990042.pdf)
Experience enhancement:
Automatically disabling dual_iso in x10 zoom for better focusing in video mode ?
Makes sense, yet another easy coding task :)
Workflow tip: how to get a quick'n'dirty preview in the CR2 before the (time-and-space-consuming) cr2hdr step?
Method 1:
exiftool IMG_6706.CR2 -PreviewImage -b | convert - -resize 100%x25% -resize 100%x400% preview.jpg
exiftool IMG_6706.CR2 '-PreviewImage<=preview.jpg' # -overwrite_original
rm preview.jpg
Question for exiftool and/or bash experts: can this be done in one command, without a temporary file?
How it works: resizing on the Y axis gets rid of the stripes; resizing it back gives a nicer preview (could be useful when selecting a few "keepers" from a large number of files).
Run time: about 4 seconds, single thread on Core i7-7700HQ.
Method 2: use cr2hdr --fast (or --fast --fullres)
cr2hdr IMG_6706.CR2 --fast # or --fast --fullres
dcraw -b 16 -c IMG_6706.DNG | convert - preview.jpg
rm IMG_6706.DNG
exiftool IMG_6706.CR2 '-PreviewImage<=preview.jpg' # -overwrite_original
rm preview.jpg
Run time: about 9 seconds on the same system (or 20 seconds with full processing).
Of course, the above can be wrapped into scripts to process all files in one directory, tweaked to run in parallel and so on.
Pixel peeping:
- original preview from the CR2:
(https://a1ex.magiclantern.fm/bleeding-edge/isoless/preview/crop-preview.jpg)
- with method 1, using the following resize factors:
a) -resize 100%x50% -resize 100%x200%
b) -resize 100%x33x33% -resize 100%x300%
c) -resize 100%x25% -resize 100%x400%
(https://a1ex.magiclantern.fm/bleeding-edge/isoless/preview/crop-preview2.jpg) (https://a1ex.magiclantern.fm/bleeding-edge/isoless/preview/crop-preview3.jpg) (https://a1ex.magiclantern.fm/bleeding-edge/isoless/preview/crop-preview4.jpg)
- with method 2, using the following options for cr2hdr
a) --fast
b) --fast --fullres
c) default (full processing)
(https://a1ex.magiclantern.fm/bleeding-edge/isoless/preview/crop-fast.jpg) (https://a1ex.magiclantern.fm/bleeding-edge/isoless/preview/crop-fast-fullres.jpg) (https://a1ex.magiclantern.fm/bleeding-edge/isoless/preview/crop-full.jpg)
For my own use, I've ended up with this:
#!/bin/bash
# Dual ISO preview
# To process all files from a directory, with GNU parallel:
#
# ls *.CR2 | parallel dual_iso_preview.sh
# exit on error
set -e
# base file name, without extension
fn=$(basename $1 .CR2)
# backup old previews
dcraw -e $1 && mkdir -p original_previews && mv $fn.thumb.jpg original_previews/
# quick processing, just good enough for a half-res preview
cr2hdr $1 --fast --fullres
#dcraw -w -b 16 -c $fn.DNG | convert - $fn.jpg
ufraw-batch $fn.DNG --shrink=2 --exposure=auto --wb=camera --out-type=jpg --compression=70
# replace the preview with our own (caveat: it will delete the old preview!)
exiftool $fn.CR2 "-PreviewImage<=$fn.jpg" -overwrite_original
# cleanup
rm $fn.jpg
rm $fn.DNG
Make sure you've got backups before running this! (just in case)
Cool. Will check your latest tomorrow.
One fast way to extract previews is by using exiv2:
exiv2 -ep3 -l . *.{cr2,CR2}
Script below not exactly fast but will apply auto white balance(white balance not correct when dual iso) by roundtripping creating a hald clut and build a fake dng which dcraw happily takes to grab multipliers.
When I think of it we could easily get the exact white balance from the cr2 and apply it back to the preview jpg instead of using auto wb. Will check into this later.
#dependencies
#dcraw
#ffmpeg
#exiftool
#exiv2
exiv2 -ep3 -l . IMG_6706.CR2 && convert IMG_6706-preview3.jpg -resize 100%x25% -resize 100%x400% preview.jpg
ffmpeg -f lavfi -i haldclutsrc=8 -compression_algo raw -pix_fmt rgb24 -vframes 1 clut.tif
#create the reference tif
ffmpeg -i preview.jpg -compression_algo raw -pix_fmt rgb24 -vframes 1 reference.tif
#add needed dng tag. Will open up in dcraw but not in acr. Doesn´t matter, good old dcraw is all we need
exiftool -DNGVersion=1.4.0.0 -PhotometricInterpretation='Linear Raw' clut.tif reference.tif -overwrite_original
#now you have dcraw compatible dng /raw files
mv clut.tif clut.dng
mv reference.tif reference.dng
#add auto white balance
multiplier=$(dcraw -T -a -v c reference.dng 2>&1 | awk '/multipliers/ { print $2,$3,$4,$5; exit }')
#apply white balance and -H 2 to your hald clut
dcraw -v -T -H 2 -r $multiplier clut.dng
#here we go. Let´s apply auto white balance
ffmpeg -i preview.jpg -i clut.tiff -filter_complex '[0][1] haldclut' wb_preview.jpg
#done
exiftool IMG_6706.CR2 '-PreviewImage<=wb_preview.jpg' # -overwrite_original
#trash files
rm preview.jpg wb_preview.jpg IMG_6706-preview3.jpg clut.tiff clut.dng reference.dng
Quote from: Danne on April 19, 2018, 07:59:27 AM
12bit MLV file used(13mb) cut in Mlv app:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M18-1354.MLV
Hi. Since recording rates are getting better with lossless but also with using the sd_uhs it seems even more interesting to get dual iso movie recording working also with at least 12bit files. While 14bit really looks awesome filming with 3xzoom mode 12bits are still problematic(check stripes after conversion):
(https://s7.postimg.cc/5ndplqtyj/Screen_Shot_2018-04-19_at_08_500px.png)
Checking dual iso code sources and it´s really some of the most impressive stuff I´ve seen. I don´t know where to start even looking for tweaks in there so I thought I´d post an example MLV if maybe Alex or somenone else wants to have a look.
Hi Danne,
Which crop-build do you use to record video in 14-bit 3x + dual iso? I have both 5d3 1.1.3 and 1.2.3.
I still haven't moved to the experimental builds and want to try 3x and dual iso.
Quote from: Sapporo on May 24, 2018, 11:32:08 AM
EDIT: It works! It was only me who didn't read everything. New values in CMOS 3 and not CMOS 0:
404e61b6
and
404e4e08
Sorry to resurrect an old post but are you sure about this?
I'm bringing this up because I took a look at @Levas module and the changes for his camera were:
diff -r f4213dd747cd modules/dual_iso/dual_iso.c
--- a/modules/dual_iso/dual_iso.c
+++ b/modules/dual_iso/dual_iso.c
@@ -733,11 +733,11 @@
{
is_6d = 1;
- FRAME_CMOS_ISO_START = 0x40452196; // CMOS register 0003 - for LiveView, ISO 100 (check in movie mode, not photo!)
+ FRAME_CMOS_ISO_START = 0x404e6196; // CMOS register 0003 - for LiveView, ISO 100 (check in movie mode, not photo!)
FRAME_CMOS_ISO_COUNT = 7; // from ISO 100 to 6400
FRAME_CMOS_ISO_SIZE = 32; // distance between ISO 100 and ISO 200 addresses, in bytes
- PHOTO_CMOS_ISO_START = 0x40450E08; // CMOS register 0003 - for photo mode, ISO 100
+ PHOTO_CMOS_ISO_START = 0x404e4e08; // CMOS register 0003 - for photo mode, ISO 100
PHOTO_CMOS_ISO_COUNT = 7; // from ISO 100 to 6400 (last real iso!)
PHOTO_CMOS_ISO_SIZE = 18; // distance between ISO 100 and ISO 200 addresses, in bytes
The difference here is +0x94000 for both. On yours it is also +0x94000 for PHOTO_CMOS_ISO_START but it is +0x94020 for FRAME_CMOS_ISO_START.
Assuming this is an error in the post and the values changed by the same amount on both cameras we've got only two know possible values for the 6D. On the 700D we also have only two known possible values and the difference is +0x400 for both registers (//http://).
@Walter_Schulz -- A while back we were trying to get this working on the 650D but it seemed that camera would switch values. In addition, I assumed the offset was the same as the 700D, now I'm thinking it is probably different. Could you run the adtg gui module on your 650D following the instructions from @theBilalFakhouri (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197146#msg197146)? I'd like to see if we could figure out if CMOS register 3 switches between just two values.
I'd do it in QEMU but it doesn't show the CMOS registers.
(https://farm8.staticflickr.com/7922/46893980311_d3a3031fb2.jpg) (https://flic.kr/p/2erRUzV)
The solution is to avoid making assumptions about the addresses of these registers, i.e. to get rid of these hardcoded addresses. See reply #2968 (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197612#msg197612) and this topic (https://www.magiclantern.fm/forum/index.php?topic=23086). I find it hard to believe there are only 2 possible values on a given camera model. FRAME_SHUTTER_BLANKING constants should be probably handled in the same way (i.e. hardcoded addresses should be deleted).
The CMOS data is in the middle of Canon's property data structure, so playing around with prop_diag, or with Canon's property functions in QEMU, might reveal something. One possible step is identifying these properties (for example, by parsing a RAM dump around the known addresses with prop_diag).
Quote from: dfort on January 27, 2019, 03:41:23 PM
Sorry to resurrect an old post but are you sure about this?
I'm bringing this up because I took a look at @Levas module and the changes for his camera were:
diff -r f4213dd747cd modules/dual_iso/dual_iso.c
--- a/modules/dual_iso/dual_iso.c
+++ b/modules/dual_iso/dual_iso.c
@@ -733,11 +733,11 @@
{
is_6d = 1;
- FRAME_CMOS_ISO_START = 0x40452196; // CMOS register 0003 - for LiveView, ISO 100 (check in movie mode, not photo!)
+ FRAME_CMOS_ISO_START = 0x404e6196; // CMOS register 0003 - for LiveView, ISO 100 (check in movie mode, not photo!)
FRAME_CMOS_ISO_COUNT = 7; // from ISO 100 to 6400
FRAME_CMOS_ISO_SIZE = 32; // distance between ISO 100 and ISO 200 addresses, in bytes
- PHOTO_CMOS_ISO_START = 0x40450E08; // CMOS register 0003 - for photo mode, ISO 100
+ PHOTO_CMOS_ISO_START = 0x404e4e08; // CMOS register 0003 - for photo mode, ISO 100
PHOTO_CMOS_ISO_COUNT = 7; // from ISO 100 to 6400 (last real iso!)
PHOTO_CMOS_ISO_SIZE = 18; // distance between ISO 100 and ISO 200 addresses, in bytes
The difference here is +0x94000 for both. On yours it is also +0x94000 for PHOTO_CMOS_ISO_START but it is +0x94020 for FRAME_CMOS_ISO_START.
Assuming this is an error in the post and the values changed by the same amount on both cameras we've got only two know possible values for the 6D. On the 700D we also have only two known possible values and the difference is +0x400 for both registers (//http://).
@Walter_Schulz -- A while back we were trying to get this working on the 650D but it seemed that camera would switch values. In addition, I assumed the offset was the same as the 700D, now I'm thinking it is probably different. Could you run the adtg gui module on your 650D following the instructions from @theBilalFakhouri (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197146#msg197146)? I'd like to see if we could figure out if CMOS register 3 switches between just two values.
I'd do it in QEMU but it doesn't show the CMOS registers.
(https://farm8.staticflickr.com/7922/46893980311_d3a3031fb2.jpg) (https://flic.kr/p/2erRUzV)
else if (is_camera("5D2", "2.1.2"))
{
is_5d2 = 1;
PHOTO_CMOS_ISO_START = 0x404b3b5c; // CMOS register 0000 - for photo mode, ISO 100
PHOTO_CMOS_ISO_COUNT = 5; // from ISO 100 to 1600
PHOTO_CMOS_ISO_SIZE = 14; // distance between ISO 100 and ISO 200 addresses, in bytes
CMOS_ISO_BITS = 3;
CMOS_FLAG_BITS = 2;
CMOS_EXPECTED_FLAG = 3;
}
else if (is_camera("6D", "1.1.6"))
{
is_6d = 1;
FRAME_CMOS_ISO_START = 0x404e61b6; // CMOS register 0003 - for LiveView, ISO 100 (check in movie mode, not photo!)
FRAME_CMOS_ISO_COUNT = 7; // from ISO 100 to 6400
FRAME_CMOS_ISO_SIZE = 32; // distance between ISO 100 and ISO 200 addresses, in bytes
PHOTO_CMOS_ISO_START = 0x404e4e08; // CMOS register 0003 - for photo mode, ISO 100
PHOTO_CMOS_ISO_COUNT = 7; // from ISO 100 to 6400 (last real iso!)
PHOTO_CMOS_ISO_SIZE = 18; // distance between ISO 100 and ISO 200 addresses, in bytes
CMOS_ISO_BITS = 4;
CMOS_FLAG_BITS = 0;
CMOS_EXPECTED_FLAG = 0;
}
else if (is_camera("50D", "1.0.9"))dual_iso.mo https://drive.google.com/open?id=1IHa1ofE0ZdSWEj15Y6Xz4QB5fZrj3i1Z
I am using this setting with my old Japanese Canon 6D together with experimental build. I haven't updated the experimental build since I wrote that.
I am using the original setting with my new Swedish sold 6D together with nightly build.
@a1ex -- Understood but let me play in the shallow end a little longer before diving into the deep end.
@Sapporo -- Your module indeed has the values you posted but I'm still skeptical about your FRAME_CMOS_ISO_START address. Have you tried recording dual_iso raw video? I know this is something that not very many people do because of the aliasing issues--that is unless you're using the new experimental 1x3 crop_rec setting.
What I did was to change the FRAME_CMOS_ISO_START to a random value on the 700D and try it out. No error message. Then I put it back and did the same with the PHOTO_CMOS_ISO_START and got an 'ISOless PH err(2)' so it turns out that only a wrong photo mode value will trigger an error message.
Quote from: Walter Schulz on February 18, 2018, 06:55:59 PM
EDIT: And here we go again. Switched Picture Style to UserDef1 (Marvels Advanced 3.4) and err(2) appears.
EDIT2: Not that easy to reproduce. With former builds I had to reset ML settings to get rid of it. Now I have troubles to provoke the error. Not persistent anymore.
Seems logical that something like activating/deactivating third party picture styles would cause things to move around in memory. It also seems reasonable to assume that the LiveView and Photo CMOS 3 register would change by the same amount. I'm not getting into conspiracy theories here--the right thing to do is to get rid of these hardcoded addresses.
By the way, the reason I looked into this again was because one of the show stoppers on the EOSM2 port is not being able to find the value for PHOTO_CMOS_ISO_START using the tools available in the adtg_gui module.
Quote from: dfort on January 28, 2019, 01:08:40 AM
@Sapporo -- Your module indeed has the values you posted but I'm still skeptical about your FRAME_CMOS_ISO_START address. Have you tried recording dual_iso raw video? I know this is something that not very many people do because of the aliasing issues--that is unless you're using the new experimental 1x3 crop_rec setting.
Failed with video and Dual ISO.
Quote from: Sapporo on January 28, 2019, 06:47:56 PM
Failed with video and Dual ISO.
That what I suspected. Now try the module Levas posted:
Quote from: Levas on November 23, 2018, 10:57:50 PM
I've had the same error, and made the changes in dual_iso.c, compiled the file and it works on my 6d, this is the dual_iso.mo file:
https://drive.google.com/open?id=1l_DgNZH6Gh_rqQeXOme-kV6XGrvMcMC1 (https://drive.google.com/open?id=1l_DgNZH6Gh_rqQeXOme-kV6XGrvMcMC1)
Big chance it works on your 6d, but also a small chance that you still get the same error, since it appears to differ per model 6d ???
Or -- use the value he found and compile it yourself:
FRAME_CMOS_ISO_START = 0x404e6196;
Quote from: dfort on January 27, 2019, 03:41:23 PM
@Walter_Schulz -- A while back we were trying to get this working on the 650D but it seemed that camera would switch values. In addition, I assumed the offset was the same as the 700D, now I'm thinking it is probably different. Could you run the adtg gui module on your 650D following the instructions from @theBilalFakhouri (https://www.magiclantern.fm/forum/index.php?topic=7139.msg197146#msg197146)? I'd like to see if we could figure out if CMOS register 3 switches between just two values.
Rusty I am. Last time I compiled ... boy, feels like another life ... had to check some VMs to find a proper configuration ... and failed kicking Err2 out of the system. Will try to redo that after some sleep!
Low priority. We know that the values can change on the same camera. The 650D is a camera that it seems to happen more frequently than the others.
The right fix is getting rid of the hardcoded addresses. Haven't figured out how yet but we got some clues.
Quote from: dfort on January 28, 2019, 07:13:29 PM
That what I suspected. Now try the module Levas posted:
Or -- use the value he found and compile it yourself:
FRAME_CMOS_ISO_START = 0x404e6196;
Works. Thanks!
Great--another piece of the puzzle solved. The LiveView and photo mode values change by the same amount.
Using Danne's latest stable 5d3 build, I'm getting these vertical stripes on mid tones.
The lower the exposure, the more prominent the effect.
Provided is a well exposed night scene, and you can see the effect on the wall. I exaggerated the processing in mlvapp to showcase the issue, the receipt is included along with the MLV in the zip.
It's not a fatal flaw, especially at higher exposures, but it really shows at lower exposures, the kind I typically use.
I provided this well exposed example to show it isn't just a problem in the darkest case scenario. This is a definite issue when exposing for highlights.
It looks the same in resolve also...
https://drive.google.com/open?id=13N3-9q8o9GmfuUuEI5-9GkKrOhfdfLFc
(https://i.ibb.co/B3DCj6N/stripes-example-frame-1.jpg) (https://ibb.co/B3DCj6N)
I've just checked another approach: MLVProducer with DualISO processing (high-iso option). The result looks perfect.
Unless I'm mistaken, choosing that option negates dual iso and simply uses the high iso half, cutting the vertical resolution in half...
Absolutely (sorry for the messy post).
So the stripes come from the underexposed low ISO layer. That's just a usual noise pattern (like in pushed up darkframe) exaggerated while scaling. I've blended them manually, using low-iso layer for extreme highlights only.
right...
so i need to find a way to get this look without the stripes. i'm playing with different ISOs now to see what improvement can be made in that regard, but it would be nice to have a simple way to adjust the blending to minimize this also.
it would be beneficial to be able to do this interactively, and not via a command prompt with a pile of numbers, rendering blindly just to see what the settings do. is there such an option out there?
Seems that cr2hdr don't work on old versions of osx.
I've tried it in 10.7.5 and gives error:
Illegal instruction: 4
Is compiled on 10.13.6 and here it works ok.
Maybe is because of clang?
@scrax: Compatibility from a newer to an older OSX is often very bad. Better you compile on your old(est) system.
Quote from: masc on April 17, 2019, 01:12:54 PM
@scrax: Compatibility from a newer to an older OSX is often very bad. Better you compile on your old(est) system.
Yes, and it's really very old version :P
It's a core2duo with two Osx versions, the oldest is a backup of like 6 years ago and is not important to have cr2hdr working, was just a curiosity.
Problem is that even the last supported Osx can't install all deps for compiling ML (problems with python iirc) I think it is 10.10.something
But it works ok on High Sierra so I'm good for now.
Good news is that I've added it to MLTools and also update it with other updated utils. and ML files like XMP, UFR and so on. Only thing left to convert so are MLV files (but not sure if is needed, there are better option and not being a video isn't helping).
(as a reminder, had to modify vid2hdr.sh for make it work ok with mplayer).
This the new line (a couple of options seem to have changed name):
opts="vbitrate=12150000:mbd=2:keyint=132:v4mv:vqmin=3:o=luma_elim_threshold=-4:o=chroma_elim_threshold=7:lumi_mask=0.07:dark_mask=0.10:naq:vqcomp=0.7:vqblur=0.2:mpeg_quant"
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dual_iso_video-updates_8-21-2019.zip
updated 5D2 with dual-iso video , works in 3x3 (FHD 1856x1248) & 5xZoom (3xcrop_rec 2144x1074) @ 10bit raw
Very interesting Canon patent :D
https://www.canonrumors.com/patent-expanded-dynamic-range-using-dpaf-sensors/ (https://www.canonrumors.com/patent-expanded-dynamic-range-using-dpaf-sensors/)
"It should be noted that the patent does mention in-vehicle sensors, but can also be applied to any other type of sensor as well"
...pity that they only want to use it for cars
@Reddeercity. Thanks. But could you tell how you process the video? In my case, i got the dual iso video more noisy in shadows the regular video using mlv app.
But when i take photos and use cr2hdr the result is stunning.
In the first page i found:
mlv_dump clip.mlv --dng --no-fixcp --no-stripes
cr2hdr --same-levels *.DNG
So, how to do it using windows? I have cr2hdr, but don't know how to open the terminal.
AFAIK Windows version does not support wildcards, at least ATM. You won't be able to use it this way and command line length is limited in Windows. 2047 or 8191 characters, haven't tested cr2hdr limit yet.
For "terminal": In Windows terms it may be called "Command prompt", "command line", "CLI" (command line interface), "cmd" ... Startup search engine of your choice ...
Understood, thanks
Hello,
I'm starting to use Dual ISO to take some photos and I'd like to understand the best strategy to capture photo with it.
Since the sensor will be at ISO100 and then an higher ISO, should I under expose a little the image? This way, the lines at ISO100 will have correctly exposed highlights and little underexposed midtones, while the lines at ISO1600 will have correctly exposed shadows and a little overexposed midtones. Am I right? Is this the right way to take photos with Dual ISO?
About the post processing, from what I understood I'll be able to push the shadows more than usual without seeing much noise because there's more data in the dark pixels than in a normal photo. So in the end Dual ISO will let me brighten the darkest regions of a picture much more than usual. Is that right?
Are there any other post-processing strategy other than this?
Thank you very much
Quote from: GiamBoscaro on January 09, 2020, 02:30:43 PM
Since the sensor will be at ISO100 and then an higher ISO, should I under expose a little the image? This way, the lines at ISO100 will have correctly exposed highlights and little underexposed midtones, while the lines at ISO1600 will have correctly exposed shadows and a little overexposed midtones. Am I right? Is this the right way to take photos with Dual ISO?
No, because of shot noise (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/index.html#shotnoise):
The higher the illumination, the less apparent the shot noise; the lower the illumination, the more apparent it is. Quote from: GiamBoscaro on January 09, 2020, 02:30:43 PM
So in the end Dual ISO will let me brighten the darkest regions of a picture much more than usual. Is that right?
There will be less noise in the darkest regions than usual.
QuoteI'd like to understand the best strategy to capture photo with it.
IMHO, For photogrhy, you can't beat ETTRing with ML and dual-ISOing from there.
As for what ISO to use, that is a little camera dependent.
On a 5D3. I'm comfortable with 1600, as above this the 5D3 is ISO invariant and it's not worth pushing the ISO above this. https://clarkvision.com/articles/iso/
I have written about ML and photography on my blog at photography.grayheron.net
I'm trying to find an active link for the software that converts dual iso to a useable format for a MacBook. So far all the links are dead minus the Lightroom plugin, which I do not have and don't want to pay $120 a year for.
https://www.magiclantern.fm/forum/index.php?topic=15108.0
Drag a cr2 file on the icon. Be sure to get pass gatekeeper on first time install.
I'm having a problem using dual iso with crop_rec presets on the 6d.
I've recorded some clips with dual iso and my own made crop_rec presets, but there is something weird going on.
Not all frames are exactly the same, colorwise.
The difference is not very apparent in full scale 1:1. But you can see a slightly hue difference, some pictures are slightly warmer, colors are slightly different.
This is how the thumbnails look in rawtherapee, these are unprocessed dual iso files, the thumbnails don't represent the real colors seen in the 1:1 picture, it's exagerated, but it gives a hint there is something going on every other frame.
This was recorded with 2480x1396 at 25 fps preset dual iso 100/1600
(https://live.staticflickr.com/65535/49827195696_6e68d0e1d1_o.png)
Now when I use my 2688 x 1296 at 25 fps preset, I've get something very alike, but it happens 2 frames and then switches for 2 frames, same is visible in da vinci resolve in the thumbnails:
(https://live.staticflickr.com/65535/49827509762_651296ef67_o.png)
It is very apparent in the thumbnails of non processed dual-iso movie files. When looked at the image 1:1, you can't hardly tell the difference.
The color hue is slightly off or something.
checked black levels, white level and stuff with exiftool, but there is no difference in the files ???
Tested dual-iso with plain 5x zoom mode, no crop_rec preset and nothing weird going on, all thumbnails are the same, no difference ???
So the problem is when using the crop_rec presets on the 6d. The presets are not exactly 25.000 fps is it a timing problem ?
Anyone any idea what is going, and maybe how to fix it ?
Here's a 100mb sample MLV with 30 frames:
https://drive.google.com/open?id=1JiQVH6VEZCAf23QkihsHuWm5rUBzh9NZ (https://drive.google.com/open?id=1JiQVH6VEZCAf23QkihsHuWm5rUBzh9NZ)
Ok, thank you all for your massive help ;D
Dived some more into the problem described in the post above. And it appears to be related to the B-timer, you can fix the above within 4 units of the B-timer setting.
I can't imagine the 6d is the only cam that had this problem in the crop_rec presets.
You can check your presets, if you enable dual iso and a preset, the ML-preview should give a image with static dual-iso line/bars...if the dual-iso lines/bars move around...your preset is not 100% accurate for dual-iso, you can fix this by adjusting the B-timer, should be right within 4 units.
A good preset gives static dual-iso lines/bars in ML preview.
Gonna post an updated crop_rec module in the 6d topic now.
So be sure to check out your presets and see if you have static dual-iso lines/bars with dual-iso.
Interesting. The slight hue difference issue has been discussed from time to time. So if lines are correct output should be too? Gonna check into this one some more on my eosm when I can.
Never noticed any problems with the presets without dual-iso.
But I can imagine that it also gives minor, not much visible issues there (never noticed them actually ???)
The processing of the dual-iso files with cr2hdr probably makes the difference more visible.
Probably only affects dualiso files and in post processing.
Pretty cool. you can actually see timers go into static lines while changing a-timer on the fly.
Unfortunately I get bad side effects also when I set my preset to static:
https://www.magiclantern.fm/forum/index.php?topic=20025.msg226701#msg226701
In this case I was actually getting more consistent results with how the a-timer was set before.
Edit: I suspect 10bit file issue. I better jump off this wagon before the can of worms is reopened...
Edit: tested 12bit, already flicker free. Stay off 10bit on eosm while doing dualiso.
WOuld dual exposure be more impressive ? IS that possible so instead of different isos , it captures different shutter speeds ? FRom over 1000 to 40000000000 theres kinda no difference in shutter speeds so that might look fine ?
OR am i lame and late and its already been done ?
Hi, I'm migrating here my question from other topics (Switch (https://www.magiclantern.fm/forum/index.php?topic=15108.msg227670#msg227670) and MLVApp (https://www.magiclantern.fm/forum/index.php?topic=20025.msg227661#msg227661)) since I've found out that my problem are my somehow faulty dual-ISOs. After the conversion with Switch I can clearly notice the horizontal lines and an heavy flickering. I've tried also MLVApp and the result is even more weird: horizontal lines, flickering, but also bluish tint and vertical lines. The funny thing is that if I set a tonemapping in MLVApp, even if I'm converting to DNG, the tonemapping seems influencing the final DNG, so that in Davinci I can see the two DNGs (from Switch and from MLVApp) as completely different!
Images speak louder than words, so HERE (https://youtu.be/5QffZCwMPjk) the screen record of the two conversions.
HERE (https://drive.google.com/file/d/15YK2B4lvH6xKhRs0KN2vJIN3J3sIUa9T/view?usp=sharing) the first-frames of the two DNG-folders.
And finally HERE (https://drive.google.com/file/d/1czdKUAc5L4ua24b_qzruXuLWPM8yn3ku/view?usp=sharing) the original problematic dual-ISO MLV.
I'd like to know how to fix my dual-ISOs (and IF it's possible to do that).
Thanks really a lot.
Thank for sharing your videos of what you were experiencing re: Dual-ISO. Now I would like to ask which build was this shot with?
Also would you mind sharing some short MLV samples if you still have them. Would be nice to have a play with. Appreciated @adrjork!
Quote from: DeafEyeJedi on June 05, 2020, 01:49:22 AMI would like to ask which build was this shot with? Also would you mind sharing some short MLV samples if you still have them.
Thank to
YOU DeafEyeJedi and to all the guys here for your kind support.
My build is the old Nightly.2017Feb12.5D3123 (it's so old because I started a project in 2017 and it's still in progress...)
I haven't "short" original MLVs (all my MLVs, expecially dual-ISO MLVs, are 1 minute long and up). What I can do is re-linking THIS (https://drive.google.com/file/d/1czdKUAc5L4ua24b_qzruXuLWPM8yn3ku/view?usp=sharing) 6GB file, and adding 3 other "shortened" MLVs (I shortened 3 long MLVs with MLVApp hoping this is the correct workflow: I uploaded and cut-out each MLV at frame 100, then I un-checked the RAW Correction panel, then I exported DNGs uncompressed, then I used "Transcode and Import" function). HERE (https://drive.google.com/file/d/1z0RekDvN-D8PMTJmjM678F1XNl8M9ytG/view?usp=sharing) the link for these 3 short dual-ISO MLVs.
Thanks for your help, I deeply appreciate it.
Quote from: adrjork on June 03, 2020, 07:10:55 AM
The funny thing is that if I set a tonemapping in MLVApp, even if I'm converting to DNG, the tonemapping seems influencing the final DNG, so that in Davinci I can see the two DNGs (from Switch and from MLVApp) as completely different!
I can guarantee that tonemapping is not applied for DNG export in MLVApp, also if dualiso is used. Whatever you did, it must have another reason. But this is 100% impossible.
There is no unique rule how to preprocess a dual iso picture. There are different algorithms out there, and even if you change just a tiny little thing on such an algorithm you'll get a different result. The current algorithms are made for current builds. So if you use an old build, you should have more luck with an old processing tool.
.dng files are different ...
M11-1011_1_2018-09-11_0001_C0000_000000.DNG
Temperature 4450
Tint +30
M12-1011_1_2018-09-11_0001_C0000_000000.dng
Temperature 5150
Tint +17
Photoshop on Windows 10
Quote from: yokashin on June 06, 2020, 08:36:32 AM
.dng files are different ...
M11-1011_1_2018-09-11_0001_C0000_000000.DNG
Temperature 4450
Tint +30
M12-1011_1_2018-09-11_0001_C0000_000000.dng
Temperature 5150
Tint +17
What do you want to say with that numbers? These are two different clips. Many values in metadata are different. Who knows how ACR interprets metadata... Camera's WB values are not saved in the MLV, so they won't exist in the DNG.
Quote from: masc on June 06, 2020, 09:29:25 AM
What do you want to say with that numbers? These are two different clips. Many values in metadata are different. Who knows how ACR interprets metadata... Camera's WB values are not saved in the MLV, so they won't exist in the DNG.
I thought they were frames from the same film but from different programs ...
Quote from: yokashin on June 06, 2020, 01:06:18 PMI thought they were frames from the same film but from different programs ...
Yes indeed: the original MLV is the same, but converted to DNG via 2 different apps (Switch and MLVApp).
I took the 6GB file to experiment
Experiment 1
Opened in MLVApp, exported some frames as Uncompressed DNG.
RawTherapee saw the dng-s as raw files, but they were filled with 0 values.
Opened with my homemade tool i was able to see the values in the files. This tool ignores any metadata apart from resolution and raw values.
These dng-s cannot be processed with cr2hdr app directly. Converting with Adobe DNG converter usually solves this problem, but it faild to convert.
Experiment 2
The original file is uncompressed, made a compressed version with MVLApp. Exporting the compressed file's frames they could be successfully opened in RawTherapee. Valid values.
These DNG frames can be successfully processed with cr2hdr application. I processed them as group of 250 frames,4 times paralelled, default options
The processed frames can be stiched into a single MLV file with raw2mlv application.
I suspect there is some problem with the metadata/info fields in the original file. When frames exported, those info get tranferred into the dng, causing bad file.
Compressing the mlv file somehow solves this.
Done on Win10/x64
Here is the processed file
google drive link (https://drive.google.com/file/d/10-6kpYPMtGoViWWy2CIcuAyel2scfdWg/view?usp=sharing)
Incredible! Thank you really a lot, kayman1021.
Unfortunately I can't replay your "Experiment 2"... So please, may you explain to me Experiment 2 like if you are talking to a baby?
1. You uploaded the original MLV file into MLVApp and unchecked Enable RAW Correction? (Or the Correction remains enabled?)
2. In Export Settings you set codec MLV, Compress, uncheck Export audio, then clic Close (or Export audio is enabled?)
3. At this point, you pressed Export selected clip.
4. Now you uploaded the new compressed MLV into the same MLVApp and again unchecked RAW Correction enabled, right?
5. In Export Settings you set codec DNG Uncompressed with Davinci Resolve naming scheme, and Export audio unchecked, right?
6. At this point, you pressed Export selected clips, and so you obtained the DNGs folder.
7. Now you pointed Switch to the DNGs folder, and chose "(d) cr2hdr dualiso processing(CR2)", then "(f) Force dualiso processing"?
Please, explain your detailed workflow.
Thanks a lot.
1-4. is correct
5. In MLVApp i used Default Naming Scheme.(I renamed the files manually in 7., i think it does not matter)
6. is correct
7. I used cr2hdr.exe app on windows. I have no knowledge about Switch.
Drag&dropping all dng onto cr2hdr.exe give me a "filename or extension too long" error, so i renamed the files. The cause however wasn't this, but the list of filenames is too long to drag&drop.
cr2hdr works on a single core/thread, and i have a skylake i5-6400/4core. I made a few directories, each containing the exe, and 250 dng files
I drag&dropped each directory's 250 file onto the EXE, i let 4 processes run at once, giving full cpu usage
cut&pasted the DNG files to a common single folder
The other app is raw2mlv.exe for making a single MLV from the DNGs
This one too shows "filename or extension too long" error on drag&drop
However, this can be worked around from command line. LINK (https://www.magiclantern.fm/forum/index.php?topic=24631.msg222308#msg222308)
opening a command prompt in the dng folder, and running raw2mlv.exe *.dng -o output.mlv gave a single mlv file
MLVApp for compressing the original, raw correction off
MLVApp for extracting the frames as DNG, raw correction off
cr2hrd.exe for processing dual iso to single iso
raw2mlv for stiching the dng frames into a single mlv
btw this problem reminds me when i recorded dualIso anamorphic videos. MLVApp detected anamorphic video, and set the Transformation/Width Stretch to 3.0x
The frames exported this way were not recognizeable to cr2hdr, but when i set the stretch back to 1.0x, the exported frames could be processed. I think that too is stored somehow in the dng files
Quote from: kayman1021 on June 08, 2020, 05:46:18 PM
opening a command prompt in the dng folder, and running raw2mlv.exe *.dng -o output.mlv gave a single mlv file
This step can also be done in MLVApp, if you don't like the command line.
Quote from: kayman1021 on June 08, 2020, 05:46:18 PM
btw this problem reminds me when i recorded dualIso anamorphic videos. MLVApp detected anamorphic video, and set the Transformation/Width Stretch to 3.0x
The frames exported this way were not recognizeable to cr2hdr, but when i set the stretch back to 1.0x, the exported frames could be processed. I think that too is stored somehow in the dng files
Stretching metadata is saved in DNG, when exporting with MLVApp.
Quote from: kayman1021 on June 08, 2020, 05:46:18 PMMLVApp for compressing the original, raw correction off
MLVApp for extracting the frames as DNG, raw correction off
cr2hrd.exe for processing dual iso to single iso
raw2mlv for stiching the dng frames into a single mlv
Incredibly, I CAN'T obtain the same result... Still flickering, still lines... And I tested all in Windows, following step by step!
So, please kayman1021 may you very kindly make a couple of things for me?
1. Please, tell me the version of MLVApp you are using;
2. Please, link an archive with cr2hdr.exe and raw2mlv.exe you are using.
Thanks in advance.
@kayman1021
Upload an unprocessed dng file after your roundtrip and also a processed one to show your result. I doubt lines are gone. Maybe you are referring to simply being able to process the file without the force option?
@kayman1021:
As I wrote before, I tried your method without obtaining your clean result. I really don't know why, since I replicated step by step your guide.
The only explanation to this is that probably you are using a peculiar version (an old one?) of cr2hdr.exe
I tried everything, even a super-long workflow with Lightroom and LRTimelapse...
At the moment the best&fastest result I obtained is based on a petty trick:
1. In MLVApp I upload the uncompressed MLV, and I set Off Focus dots, Off Bad pixels, Off Vertical stripes, whilst Chroma smooth 2x2 and Dual ISO forced on Preview;
2. I export uncompressed DNGs;
3. In Davinci the first node is for RAW panel settings, then the second is a layer-node in Color mode: the "under" layer completely desaturated (B/N) and with a strong deflicker, while the "above" layer is a coloured still exported from a "grab" still. Adding just a bit of blur, the dirty trick works, but only because the shot is pretty motionless.
Conclusions: this is NOT the way to go...
tools used (https://drive.google.com/file/d/1SiTopmGJIvvoG1OeVE07DHGCyNBRGZH6/view?usp=sharing)
dng frames and an mlv (https://drive.google.com/file/d/18ySUENSNLQ6PneQ2CjIT7StVAQJYGLWc/view?usp=sharing)
@masc
I completely forgot mlvapp is capable of this, thanks for reminding
@Danne
Uploaded on "dng frames and mlv"
0_corrected_I_raw_corr_disabled.dng is that i made outside of mlvapp, the others are from the original, exported with different raw correction settings.
Mine does not look good as dng somehow, but i attached an mlv that contains the first frame of the round trip file. There is looks okay.
@adrjork
I have MLV App v1.11
Also i zipped the tools as "tools used"
Trying to wrap my head around this. So, the first dng is the one that converts successful?
0_corrected_I_raw_corr_disabled.dng
Could you tell me or upload a dng that you know for sure you are able to convert that looks ok? I just tried cr2hdr from 2016 but still getting stripes.I just need one of your dng files that you are able to convert, nothing else.
export_from_compressed_mlv (https://drive.google.com/file/d/15BcZt7huE6xAk-pX3zF7tWBI6M2OwPOk/view?usp=sharing)
It works...
I've no words kayman1021.
With your apps' version my dual-ISOs get solved. I don't even need raw2mlv because I uploaded the DNGs directly into Davinci... and it finally works.
Now, just to be fussy, passing throuh cr2hdr is a "pain-in-the-a@@" workflow: very-very-very time-consuming.
@Danne: is there any future chance to see this sort of "legacy" dual-ISO-conversion option in Switch?
If I know what version and from what branch I could look into what's going on. If it's even cr2hdr but probably is.
Shouldn't be too hard for a windows user to obtain this info.
Thanks for the file @kayman1021. It doesn´t wothout lines convert on my mac so it depends on your version of cr2hdr. If you could find from what branch and version I could look into it.
I'm not really a Windows user and I'm not a coder, anyway the working cr2hdr.exe has been created 4 Dec 2017, and HERE (https://bitbucket.org/hudson/magic-lantern/branch/cr2hdr) i found Commit 06fae84 (https://bitbucket.org/hudson/magic-lantern/commits/06fae84faeee81d6368091205d10868cf8d4cbe3?at=cr2hdr) dated 4 Dec 2017... Is it anything useful?
I am at work now, if arrive at home i'll check (5 hours from now)
It's probably the version on ML Downloads/Utilities (https://builds.magiclantern.fm/utilities.html)
No specified branch but should be included in the binary info I think.
Quote from: kayman1021 on June 10, 2020, 10:10:03 AMIt's probably the version on ML Downloads/Utilities (https://builds.magiclantern.fm/utilities.html)
Apparently yes: same creation date. Perhaps silly question: does libraw.dll matter somehow?
Quote from: Danne on June 10, 2020, 10:16:52 AMNo specified branch but should be included in the binary info I think.
Can you tell me how can I extract this info for you? (In OSX I opened the .exe with Keka, and now I have some docs in a folder. I opened them with an Hex editor and the only one not-in-alien-language is a doc named "19" in which there is written «GNU C 4.9.0 20130523 (experimental) -m32 -mtune=generic -march=pentiumpro -g -O2 -std=gnu99». Is it anything useful? I guess not...)
Ok, compiled from cr2hdr branch which seems to be the one used successfully here:
https://bitbucket.org/hudson/magic-lantern/branch/cr2hdr
I managed to convert the uploaded dng correctly with a mac binary so obviously it should work.
I updated Switch, please test:
https://bitbucket.org/Dannephoto/switch/downloads/Switch.dmg
Also a mac binary here:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/cr2hdr
EDIT: Tested Switch and problem still persists so @kayman1021 round trip to rebake the dng fixed it. We need to examine what data is missing originally. Phew, complicated stuff.
EDIT: Tested again and the mlv_dump not coming from bouncyball works straight out of the box with the updated version of cr2hdr. In Switch select (m) mlv_dump(MLV)
So, the problem seems related to mlv_dump version. cr2hdr doesn´t like the dng from the updated mlv_dump code.
@Danne: do you see the difference between the versions? Is it the same algorithm as in MLVApp? Worth to change it?
Yes. Mlv App has those issues. Will probably fix those flickery bugs and maybe finally dual iso will become reliable.
What works now in Switch for dual iso for both mlv_dump version are:
(05) do not fix cold pixels added!
(07) disable vertical stripes in highlights added!
Disable cold pixel and vertical stripes. Now cr2hdr chews the files normally again...
You almost did it, Danne. Great job anyway!
I tested 7 most problematic dual-ISO mlvs, and new Switch does work for 6, and fails only for 1.
That one gives a weird result: the converted dngs are a sequence that alternates bunches of good-dngs and bunches of bad-dngs. The bunches contain different numbers of dngs, for example the first bunches contain respectively 8-good-dngs, 2-bad-dngs, 8-good-dngs, 10-bad-dngs, 2-good-dngs, etc.
Good-dngs have extension in uppercase (.DNG) whilst bad-dngs have extension in lowercase (.dng).
The curious thing is that importing the dngs-folder into Davinci, the software can't recognize a unique sequence, instead it imports all the bunches separately (each one with 8, 2, 8, 10, 2, etc. dng-frames)
I uploaded the first 30 dngs of the folder HERE (https://drive.google.com/file/d/16B8uUijX5iUayZeLUNBIgjkC8fRMeTno/view?usp=sharing), and also a shortened version of the problematic mlv HERE (https://drive.google.com/file/d/1yR_AjyZ9J-3VQh-Y42NLy10Jtt_LI2Ss/view?usp=sharing) (the original file was 15GB, so I used MLVApp, RAW Correction disabled, codec MLV Fast Pass).
Usually when not all files are converted it means cr2hdr algoritm have a hard time detecting dualiso on the file. Trying a second time sometimes works but probably the file itself wasn´t a perfect dualiso example. Sorry, I can´t download atm, on a very slow connection. Could you upload maybe a single dng file that wouldn´t convert?
Tried a second chance with that single problematic file gave a perfect result.
I correct myself: you COMPLETELY did it.
Sincerely many congrats for your amazing work.
Are the flickering issues generally more pronounced with heavily underexposed shadows? I've got a telephoto shot of the full moon (well above the horizon, so very bright!) with some relatively dim buildings in the foreground. With 5D3 dual-iso at 100/1600, exposed for the moon at 100, I get crazy flickering with both MLV App and Switch. I've confirmed with exiftool the the black and white levels are constant. Besides the flickering, the individual processed DNGs look amazing!
Anyone tested Dual ISO in 48+ FPS mode?
for some reason i get a very weird flickering.
Hi, I am new to ML.
I just installed ML few days ago on my 6D and I have tryout dual iso module today which has got me some greyed out images.
As far as I know it needs a cr2hdr program to convert it to normal image and that is the only way to use it.
However, neither links of .c code/binary are still opened.
Any suggestion to do with my photos?
Thanks.
What if triple iso? just wondering how.. :D
Quote from: Danne on June 10, 2020, 01:04:16 PM
Yes. Mlv App has those issues. Will probably fix those flickery bugs and maybe finally dual iso will become reliable.
Ohh.. I just knew it, I thought it was dual iso module problem... thanks.
Hi,
I have just got an idea.. how if dual iso setting adjusted by interval?
for instance when we set iso to 100 with say 3 (interval or stop) then the second iso automaticaly set to 800
then when we set iso to auto, the second iso adjust also based on that interval if it's possible..
I hope you know what I mean.. :D
For video, with dual ISO, am I correct with understanding that the exposure you're seeing on the screen is showing the full dynamic range set by the two iso values? You're seeing both values at the same time?
So the exposure meters such as the histogram etc are metering correctly? I can expose normally?
Then when put into MLV, the exposure is set to the lowest ISO value you have chosen, but the data for the higher dual iso is still there in the data for you to recover in post process?
I am also running Dannes experimental build for the 5diii, using the 1:3 binning anamorphic preset. I noticed that the vertical lines are visible in the higher dual iso value areas in the image. So the darker parts that has been brightened up in PP have these lines.
This is normal due to the nature of noise and the binning, correct?
Not working on 6D 1.1.6
ISOless PH err(3)
Mo load failed.
But if I use 1.1.3 with old ML+old dualiso.mo, it works just fine with 6D.
May this information helps.
Quote from: Levas on April 22, 2018, 08:52:12 PM
Haven't used dual iso for almost 2 years, but I wanted to give it a try now with crop mode video (1:1) on the 6d.
But I get an error:
ISOless PH err(3)
So it obviously doesn't work anymore on the 6d, I have no idea for how long.
I'm using the "crop_rec_4k.2018Mar10.6D116.zip" build from the experiments download page.
Same problem as yours, but I change to 1.1.3 with an old copy of ML, dual iso.mo works fine again, don't know why.
Quote from: Levas on April 23, 2018, 08:43:10 PM
Disabled all modules on my 6d (Made in Japan, it says on the bottom).
Only activated dual iso module.
Got the 'ISOless PH err(3)' error both in video mode and photo mode...
So it doesn't work in any mode.
Will try some other builds of magic lantern.
1.1.3 with magiclantern-Nightly.2014Mar28.6D113
dual iso works fine.
Problem exists (at least) for 6D, 650D, 700D. According to a1ex it should be fixed by solving the common issue: https://www.magiclantern.fm/forum/index.php?topic=25976.msg231177#msg231177
I think something changed from 1.1.3 to 1.1.6 on 6d.
Anyway, the 'ISOless PH err(3)' for 6d is fixed in the builds I uploaded the last couple of years, although those aren't available on the experimental downloadpage.
In this post there's a linkt to the latest build for 6D I uploaded, should work for dual iso:
https://www.magiclantern.fm/forum/index.php?topic=25782.msg229094#msg229094 (https://www.magiclantern.fm/forum/index.php?topic=25782.msg229094#msg229094)
But be aware that this is a experimental build, this build also contains the SD_UHS speed hack module and has a special crop_rec module with more resolution options.
To know how it all works, you might want to read the whole topic, it's only 6 pages, so it's not that much. But a lot of info in there.
https://www.magiclantern.fm/forum/index.php?topic=25782.msg223031#msg223031 (https://www.magiclantern.fm/forum/index.php?topic=25782.msg223031#msg223031)
Comes to mind that the dual iso module in these builds should be compatible with the 4k build on the experimental downloadpage, so if you don't want to use all the stuff in the new build, you could copy the 'dual_iso.mo' in the modules folder of my build and overwrite it to the 4k experimental build you are using from the downloadpage.
Hi guys!
Is it possible in any build to take silent pictures from liveview with dualiso with an intervalometer.
This does not work for me, it shoots out only a raw frame without processing by dualiso.
It would be very cool for timelapses.
Thanks
Hi @dream951
Silent pictures with Dual ISO already works:
1-If you set "Silent Mode" to "Simple", you need to enable RAW video in this case to make Dual ISO works with silent pictures (this only works in movie mode, unless you want it in Photo mode for slower shutter speeds?)
(*Edit: it seems this doesn't work well with intervalometer, I got "Out of Memory" with RAW video enabled).
*Workaround: make Dual ISO works when taking "Simple" silent mode pictures, also when "Silent" mode is activated in movie mode without RAW video enabled (coding tasks).
2- If you are using "Full-res" silent pictures, it works out of the box, no RAW video required.
"Simple" silent mode has LiveView RAW resolution which is about ~1920x1280 or less (in none crop mode, depending on your camera) with Binning/Skipping, "Full-res" silent mode uses full pixels on sensor without Binning/Skipping, which one are you looking for?
Does MLV store RAW data in a linear gamma?
I'm reading that cameras that store RAW sensor data linearly, the dynamic range is limited to the bit depth. Whereas cameras that store RAW sensor data logarithmically, the dynamic range is not limited by the bit depth.
https://www.avsforum.com/threads/fake-raw-braw-proresraw-z-raw-whats-going-on.3157980/ (https://www.avsforum.com/threads/fake-raw-braw-proresraw-z-raw-whats-going-on.3157980/)
Example: In cameras that store sensor data linearly, capturing a 10bit signal, limits the available dynamic range to be captured to 10 stops.
Quote from: a1ex on July 16, 2013, 06:33:50 PM
Technical doc
- dual_iso.pdf (http://acoutts.com/a1ex/dual_iso.pdf) (in-depth description of how it works)
- See also: ADTG and CMOS discussion (http://www.magiclantern.fm/forum/index.php?topic=6751)
The technical doc link is down. However it can still be accessed here if anyone needs it: https://web.archive.org/web/20130821183804/http://acoutts.com/a1ex/dual_iso.pdf
@Milk The developers would have done it if there was an easy way to make the camera do it! 10 bit log would be almost indistinguishable from 14 bit.
Milk and Coffee 10,12 and 14 bits in raw on my canon M or 5d2 are hard to tell apart, 10 bits have noisey blacks so i use it when i have no choice and footage is too high res for continuous recording
Dynamic range is identical on all 3 bitdepths, i tested it, there wasnt more stuff and even if there was its just junknoise ,you can test it easily too.I use 12bits.
I tested this mode for the first time today, I seem to have got a few strange corrupted frames on 1080p mode.
Is this less likely to happen if I use a crop mode or is it suspectable to it regardless of mode?
Astro modified 550D owner, and new Magic Lantern user here.
I have the Dual ISO module successfully installed and running, and have taken a few one shot daylight HDR images.
Fast forward to using this in my deep sky tracked and stacked astrophotography workflow, and I am struggling with image calibration.
Basically, my flats are overcorrecting and not removing any dust bunnies. There also appears to be weird multi coloured colour casts.
I am at 100/1600, Dual ISO-wise for what it's worth.
And yes, I have tried calibrating the files with and without crwhdr and every combination in between, without success. Without flats, the overcorrection is gone, but then you run into all the other incomplete calibration issues this brings.
I know this is a long shot, as I'm not sure anyone without DSO astrophotography knowledge has any clue what I'm talking about, but there are no Dual ISO users in my astro circle I can call upon.
Thanks
I don't think you will find any information on that. Dual-ISO and astrophotography is IMHO not a combination anyone wants to use.
Yuvex,
For sure not a DSO photography expert here, but I know what you are talking about.
May I ask you why one should be using dual ISO for astrophotography? You are losing half vertical resolution in deep shadows (that is, the part you are most interested in) and deep highlights.
Alternatively, you can 'just' double your overall exposure time and take (if you really need to!) a set of shots at ISO 100 after taking the one at 1600. Without losing resolution.
Though I'm wondering what kind of DSO subject would need a 4-stop exposure bracket. But once again, I'm no expert here. Would be nice to learn something interesting new on astrophotography.
The way I see this, dual ISO makes extremely sense when you have a moving subject, that makes it impossible to apply standard HDR bracketing techniques. DSO's do not fall in this category ;)
But I may be missing something.
S
Ps: the way dual ISO is debayered is not trivial. I guess this could also make calibration impossible. Anyway, what kind of calibration workflow are you adopting? Are you going through flat darks as well?
Hi, thanks for responding.
Under normal circumstances, my 550D has a peak DR of 10.8 stops at ISO 100 (I normally image at ISO 400). By current standards, this is fairly mediocre.
I'm aware by stacking I am increasing my SNR, and also my DR, but why not try to maximize it further with Dual-ISO too? I am not green when it comes to astrophotography, so look to any avenue to better my improve my workflow.
Resolution is not usually an issue, as more often than not I BIN my data to increase SNR (my data is often oversampled out of the box, so I am not really losing anything but useless noise).
I have successfully taken a Dual-ISO widefield astro panning time lapse, but that needed no calibration frames. That dealt with bright foreground artificial lighting and the night sky very nicely. The problems lie when you apply calibration frames for deep sky.
Calibration wise, I use lights, temperature and time matched darks, flats (via an EL panel) and bias frames. All pre processed in Siril.
I only employ dark flats over bias for my dedicated cooled monochrome astro camera, as bias are sufficient for an old DSLR like the Canon.
Quote from: Walter Schulz on May 22, 2022, 07:09:54 PM
I don't think you will find any information on that. Dual-ISO and astrophotography is IMHO not a combination anyone wants to use.
It seems that way.
Maybe I'll just stick with terrestrial and uncalibrated time lapse.
Cheers
Why did CR2HDR export a DNG with a resolution of 384x252 while still being more than 37mb? what can i change to improve this?
Open DNG in RAWtherapee first. I think you will see a difference.
I was looking on the PhotosToPhotos.com website recently to see if there was information there on the dynamic range of the 5D Mark III with dual iso.
I was surprised that the 5D Mark III wasn't listed, (though there was info for the Mark II), so I contacted the website owner and offered to assist him with gathering data for the Mark III, which he has now posted to the website. Should you be interested, here's the link comparing the standard Mark III with ML dual-iso base settings of 100 and 400 ISO : https://www.photonstophotos.net/Charts/PDR_MagicLantern.htm#Canon%20EOS%205D%20Mark%20III,Canon%20EOS%205D%20Mark%20III_16(ML),Canon%20EOS%205D%20Mark%20III_16(ML400). The results suggest a dynamic range improvement of approx 1 bit over the 5D Mark II.
Its not possible to compare the 5D Mark III with the much more recent Canon R5 on that chart, but reading the separate chart for the R5 here : https://www.photonstophotos.net/Charts/PDR.htm#Canon%20EOS%20R5 suggests that the Mark III does well for dynamic range, albeit with a lower resolution.
Cool, thanks for getting another cam added to their list! Are there other missing cams that can do dual ISO?
On a sidenote. R5/R6 behaves practically like dualiso already included in the sensor output meaning shadow handling on these cameras is not even comparable anymore to digic V sensors.
Do I still pick up my eos m though. Hell yes.
Quote from: names_are_hard on March 23, 2023, 05:00:17 AM
Are there other missing cams that can do dual ISO?
A lot:
ML supported cams not listed at all:
1200D
1300D
500D
650D
EOS M2
Missing Dual-ISO data:
100D
1100D
50D
60D
700D
70D
EOS M
Missing data for EOS M, M2, 650D, (1200D/1300D) is no big deal for those in the know. They share 600D's sensor AFAIK. Same for 100D, 700D.
Hello,
Just to bump the thread or restart conversation a bit:
Courious how i get this "denim" looking pattern of noise with 5D2.
Dualiso 100/800:
(https://i.imgur.com/Gh6uMaZ.png)
link: https://i.imgur.com/Gh6uMaZ.png
https://i.imgur.com/0QAwetL.png
I hope i'm not imagining things? Anyway this is pretty cropped in so it's not so visible all the way out, but still courious.
Please link unprocessed file.
And describe your workflow a bit.
Sure thing, here it is https://ufile.io/p66hbqrg
If you preffer another file host i can use that, this one doesn't tease bady after clicking slow download.
I use barracudaGUI with latest cr2hdr from the site:
options used for conversion:
Last update: 06fae84 on 2017-12-03 23:53:11 UTC by alex:
cr2hdr: experiment - yet another exposure matching algorithm
Active options:
--amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax : set AsShotNeutral by maximizing the number of gray pixels (default)
After that i just import into Lightroom and that is it, it shows up like that, usually on areas where there is little contrast, smooth tones etc..
I think its some after effect of the dual iso alternating lines, when i shoot horizontal they are horizontal, when i rotate vertical they are vertical.
Quote from: Markelian on September 23, 2023, 05:24:46 PM
Sure thing, here it is https://ufile.io/p66hbqrg
If you preffer another file host i can use that, this one doesn't tease bady after clicking slow download.
That is true, not any teasing but also no download. Other hoster, please!
Maybe the window doesnt display properly if you have JS disabled (fair enough), but clicking options free download->slow download-> should instantly download without waiting.
I dont know any good ones off the top of my head because this one has always worked (so i ask what you prefer), i dont use ones that require account.
Here's a different one that i found that also doesnt tease on download https://uploadnow.io/f/DM5scG1
Try adding --force
and report back.
Sorry for long reply. I tried playing around with settings now, including adding --force, but the effect is always the same.
Near the bottom of the export log it does say "horizontal stripe fix..." so i guess it is a known problem, just i guess doesn't fix it for my camera?
Check this file: https://uploadnow.io/f/Fm65nWd
Loaded your file in MLVapp and applied Dual-ISO setting, then exported.
Very interesting, there the situation is reversed.
Basically no more lines in the sky area, but now the sealightpost and pier have the lines.
I thought maybe it was lightroom being like that, but opened now in every image viewing program i could find and it always renders like that.
Is it like that for you?
https://i.imgur.com/QPdELGT.png (https://i.imgur.com/QPdELGT.png)
Another thing, sometimes there's also a color chroma line issue thing:
(https://i.imgur.com/eoFYd7K.png)
https://i.imgur.com/eoFYd7K.png (https://i.imgur.com/eoFYd7K.png)
source: https://uploadnow.io/f/vW0W4bg (original, converted, and conversion log)
Had to add 5x5 smoothing to get rid of artifacts.
(https://abload.de/thumb/img_1880_frame_1szdrr.png) (https://abload.de/image.php?img=img_1880_frame_1szdrr.png)
I suppose we have a purple fringing issue here. Happens in overexposed out-of focus areas.
Would require a second shot without Dual-ISO to verify/falsify.
Yes, the 5x5 does seem to help it, thank you for the advice.
Yes there's a lot of fringing on those shot which i was wondering about too..
I tried to replicate it but now that i am home i can't get this behaviour to appear again in this form, at worst i just get normal CA's on high contrast edges (like bright background against dark foreground).