Dual ISO - massive dynamic range improvement (dual_iso.mo)

Started by a1ex, July 16, 2013, 06:33:50 PM

Previous topic - Next topic

0 Members and 4 Guests are viewing this topic.

[email protected]

I thought it was the same thing, sorry.

I want to raise dynamic range in video maintaining 24 fps in 5D mark 2, like doing a HDR VIDEO? Can it be done? How.

Thanks!

Danne

You can work either with HDR or dual iso. They both have their ups and downs. I think you have to read and test out to see for yourself what you like best.
If you want to test these features out I created a scriptbased automated workflow which uses a1ex and g3gg0 binaries amongst others working with dual iso and hdr movie files. Here is link to the script.
cr2hdr-r 
Feel free to post questions in that thread.

Tin_28

5DIII, 7D; Sigma 10mm ƒ/2.8, Canon 16-35mm ƒ/2.8 II, Canon 35mm ƒ/1.4, Canon 24-70mm ƒ/2.8 II, Canon 60mm ƒ/2.8, Canon 70-200mm ƒ/2.8 II, Sigma 120-300 ƒ/2.8

DeafEyeJedi

Not sure why would you want to run the old nightly when it would be best to go with latest since it's usually most up to date. Are you running 113 or 123?

Are you still seeing the interlaced lines? Or are they not visible?

If not, did you load the proper modules needed within your ML settings?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Tin_28

Thanks for the response, I figured it out...it turned out to be a module issue.

I was having pink frame issues a while back; downgrading to the March nightlies fixed that issue for me. Admittedly, I haven't been checking up on the newer nightlies so I should probably get on that.
5DIII, 7D; Sigma 10mm ƒ/2.8, Canon 16-35mm ƒ/2.8 II, Canon 35mm ƒ/1.4, Canon 24-70mm ƒ/2.8 II, Canon 60mm ƒ/2.8, Canon 70-200mm ƒ/2.8 II, Sigma 120-300 ƒ/2.8

Marsu42

Fyi all in spite of cr2hdr offering lossy dng compression this doesn't quite work (for me). I've created a bug ticket: https://bitbucket.org/hudson/magic-lantern/issues/2333/cr2hdr-breaks-lossy-dng-aka-smart-previews

It's known that dual_iso dng files cause artifacts when converted to lossy dng in the regions of deep shadows that have been interpolated as half the the scanlines were clipped in the original raw file.

Apart from just being a "missing dng feature" in comparison to vanilla raw files, this problem has become more severe in recent Adobe apps since it's the base for the offline "smart preview" files. The free dng converter can be used to convert dng to lossy w/o Photoshop or Lightroom, it's basically 8bit jpeg-derived compression with the added benefit of keeping lossless wb adjustment.

I hope there's a way to do a minor cr2hdr adjustment to circumvent this problem. Alas, as Adobe's compression algorithm is closed source it might be tricky to identify the exact cause of the problem, but imho it's well worth a try. The dng converter seems to erroneously identify and compress these specific regions, maybe adding some noise or similar could be added to prevent this from happening?

Here's a sample how the lossy dng artifacts usually look: https://bitbucket.org/hudson/magic-lantern/issues/attachments/2333/hudson/magic-lantern/1438442876.44/2333/buggy_lossy-dng.png

mothaibaphoto

@Marsu42: You trying to lossy compress with Adobe unprocessed Dual ISO? Why? I've read that lossy DNG compressed files no more Raw, it's demosaiced data. Demosaicing algorithm didn't expect to meet Dual ISO raw - this why artifacts, no?

DeafEyeJedi

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Walter Schulz

Lossy compress is just an argument for cr2hdr. Kitchetof's plug-in is wrapped around cr2hdr and that's what most Dual-ISO tools do.

DeafEyeJedi

I'm aware of that, Walter.

I apologized for the slight misunderstanding and I thought I was just trying to be helpful by letting them know that it is possible.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

dubzeebass

I'm finding the dual ISO files a bit greener. 5D mark III 1.2.3 but build number doesn't matter because it's been all the way since I started using Dual ISO. Anyone notice this in the DNGs? WB is odd.

DeafEyeJedi

Do you mean that it doesn't show any greenish effect on 113 but it does 123?

Have you tested it on 113 yet yourself?

It looks fine to me here coming from the latest nightly 113...

Which converter did you use?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Marsu42

Quote from: mothaibaphoto on August 01, 2015, 07:33:49 PM
@Marsu42: You trying to lossy compress with Adobe unprocessed Dual ISO? Why? I've read that lossy DNG compressed files no more Raw, it's demosaiced data. Demosaicing algorithm didn't expect to meet Dual ISO raw - this why artifacts, no?

My understanding that a dual_iso file should be the same as a vanilla raw after it's reconstructed by cr2hdr. Adobe supports these 16bit raw files in ACR (Lightroom, Photoshop, DNG Converter), and I see no reason why lossy compression shouldn't work as well - though of course you can never be sure with closed source software.

It's an important feature for Adobe apps - smart previews enable you to do faster and offline editing, and lossy dng is great for archival of less important shots w/o resorting to jpeg. Btw the "standard" preview in Lightroom is broken as well with the same artifacts, seems to use the same compression algorithm.

On the bitbucket bug tracker, dmilligan guesses that this might have to do with the non-standard black level dual_iso files. I cannot say and I'm unable to change cr2hdr to test this, but lossy dng *does* work fine with mini_mo files which also have a custom black level (but break dxo's raw converter).

Quote from: dubzeebass on August 02, 2015, 12:30:28 AM
I'm finding the dual ISO files a bit greener. 5D mark III 1.2.3 but build number doesn't matter because it's been all the way since I started using Dual ISO. Anyone notice this in the DNGs? WB is odd.

I'm on 6d/60d, and I consider wb "more than odd". I cannot verify the theory that dual_iso and vanilla should just look alike, just with dual_iso having cleaner shadows. My current method is to let Lightroom figure it out with the "auto" setting on import which works better than the current cr2hdr detection attempts for guessing auto wb. The "auto" setting also gives you a hint in which way to correct files with a fixed wb setting from the camera.

Having said that, hdr pictures somewhat do look a bit odd in the shadows anyway (esp. concerning green/magenta tint), so that might be one cause of confusion and bogus bug reports.

halbmoki

Hello,
I have a little problem with dual ISO... with long exposures (1sec or more, though it seems to increase gradually), there are very strange artifacts in my images, as you can see from the image. This is a 1:1 crop from a 20sec exposure at ISO 100/800 with a 50D. These artifacts do not occur with the same exposure at a single ISO. Long exposure noise reduction has no influence at all.

Is this a known problem and is there any solution?

http://i239.photobucket.com/albums/ff252/mogul-/temp/IMG_8820-dualiso_zpslwk63idq.jpg

toze3

I was geting some troubles processing dual iso DNG in the latest version of Rawtherapee, i tought it was a white level problem and asked some help in their forum.

It seens there are some double tags in the dual iso DNG and missing color matrix?(not sure if only mine, with latest ML nightly on 550D and latest cr2hdr)

"your dng file includes both tags with the same value (21). In this case the second one (50779) is given precedence, but there is no valid color matrix for second one. Only the matrix for 50771 exists in your dng.
I try to catch this case in rt source, but you should also report the bug to the guys who made the hack to produce the dual iso dng files.
"

@a1lex could you have a look in this post
rawtherapee.com/forum/viewtopic.php?f=1&t=6185

Licaon_Kter


Walter Schulz

Sample DNG will be found in linked thread.
Missing untouched CR2, though.

Danne


heckflosse

Hi,

the problem with the DNG file is, that it contains the tags
50778 http://www.awaresystems.be/imaging/tiff/tifftags/calibrationilluminant1.html
and
50779 http://www.awaresystems.be/imaging/tiff/tifftags/calibrationilluminant2.html
Both have the value 21 and in this case Rawtherapee uses 50779. So far so good. But the corresponding color matrix for this case
tag 50722 http://www.awaresystems.be/imaging/tiff/tifftags/colormatrix2.html
does not exist in the DNG
I already prepared a fix for Rawtherapee, but IMHO, if tag 50722 does not exist, tag 50779 shouldn't exist too

Ingo

Danne

Certain programs implements tags and color matrix information. Provide a dng so people could check.

Walter Schulz

550D was used for sample.
Will try to reproduce with 650D.
Aware Systems offers a free TIFF tag viewer for Windows: http://www.awaresystems.be/imaging/tiff/astifftagviewer.html ... and it doesn't like DNG that much ...


Walter Schulz

Okay, Exiftool works.

Happens with 650D, too.
Tag 0xc621 (50721) Colormatrix1 exists but Colormatrix2 missing.

Workaround: Use exiftool to create tag Colormatrix2 and copy Colormatrix1 value into.

EDIT: Created issue/bug report #2343 https://bitbucket.org/hudson/magic-lantern/issues/2343/cr2hdr-not-generating-tag-colormatrix2


toze3

Quote from: Licaon_Kter on August 14, 2015, 12:46:23 PM
@toze3:
Latest cr2hdr from here: https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/ right?

YES
Quote from: Licaon_Kter on August 14, 2015, 12:46:23 PM
Do upload: Sample CR2 and Ouputed DNG.
I don't have the same CR2 was the DNG posted but i can upload a new ones.
CR2: https://www.dropbox.com/s/1p3tbx44vl9msgg/_MG_2498.CR2?dl=0
DNG: https://www.dropbox.com/s/83yju7334ibj19m/_MG_2498_DI100_800.DNG?dl=0
It happens in all my dual iso DNG.

Hope this helps

Quote from: Walter Schulz on August 14, 2015, 02:32:45 PM

Workaround: Use exiftool to create tag Colormatrix2 and copy Colormatrix1 value into.


How do i do that?

Danne

I think dcraw only outputs one color matrix tag.
If you,re on mac I could probably implement the insert of the color matrix 2 in my tool cr2hdr-r also for CR2 to DNG conversion. I already insert it it in movie dng sequences. Not familiar with the windows system anymore unfortunately.