Author Topic: Dual ISO - massive dynamic range improvement (dual_iso.mo)  (Read 1693699 times)

daniel@danielchamorro.com

  • New to the forum
  • *
  • Posts: 26
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2525 on: July 09, 2015, 09:48:47 AM »
I thought it was the same thing, sorry.

I want to raise dynamic range in video maintaining 24 fps in 5D mark 2, like doing a HDR VIDEO? Can it be done? How.

Thanks!

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6861
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2526 on: July 09, 2015, 10:42:24 AM »
You can work either with HDR or dual iso. They both have their ups and downs. I think you have to read and test out to see for yourself what you like best.
If you want to test these features out I created a scriptbased automated workflow which uses a1ex and g3gg0 binaries amongst others working with dual iso and hdr movie files. Here is link to the script.
cr2hdr-r 
Feel free to post questions in that thread.

Tin_28

  • New to the forum
  • *
  • Posts: 4
  • oh boy
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2527 on: July 10, 2015, 05:00:01 AM »
jk i figured it out thanks everyone
5DIII, 7D; Sigma 10mm ƒ/2.8, Canon 16-35mm ƒ/2.8 II, Canon 35mm ƒ/1.4, Canon 24-70mm ƒ/2.8 II, Canon 60mm ƒ/2.8, Canon 70-200mm ƒ/2.8 II, Sigma 120-300 ƒ/2.8

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3397
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2528 on: July 10, 2015, 07:58:06 AM »
Not sure why would you want to run the old nightly when it would be best to go with latest since it's usually most up to date. Are you running 113 or 123?

Are you still seeing the interlaced lines? Or are they not visible?

If not, did you load the proper modules needed within your ML settings?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Tin_28

  • New to the forum
  • *
  • Posts: 4
  • oh boy
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2529 on: July 10, 2015, 08:46:17 AM »
Thanks for the response, I figured it out...it turned out to be a module issue.

I was having pink frame issues a while back; downgrading to the March nightlies fixed that issue for me. Admittedly, I haven't been checking up on the newer nightlies so I should probably get on that.
5DIII, 7D; Sigma 10mm ƒ/2.8, Canon 16-35mm ƒ/2.8 II, Canon 35mm ƒ/1.4, Canon 24-70mm ƒ/2.8 II, Canon 60mm ƒ/2.8, Canon 70-200mm ƒ/2.8 II, Sigma 120-300 ƒ/2.8

Marsu42

  • Contributor
  • Hero Member
  • *****
  • Posts: 1557
  • 66d + flashes
cr2hdr breaks lossy dng (aka smart previews)
« Reply #2530 on: August 01, 2015, 05:08:49 PM »
Fyi all in spite of cr2hdr offering lossy dng compression this doesn't quite work (for me). I've created a bug ticket: https://bitbucket.org/hudson/magic-lantern/issues/2333/cr2hdr-breaks-lossy-dng-aka-smart-previews

It's known that dual_iso dng files cause artifacts when converted to lossy dng in the regions of deep shadows that have been interpolated as half the the scanlines were clipped in the original raw file.

Apart from just being a "missing dng feature" in comparison to vanilla raw files, this problem has become more severe in recent Adobe apps since it's the base for the offline "smart preview" files. The free dng converter can be used to convert dng to lossy w/o Photoshop or Lightroom, it's basically 8bit jpeg-derived compression with the added benefit of keeping lossless wb adjustment.

I hope there's a way to do a minor cr2hdr adjustment to circumvent this problem. Alas, as Adobe's compression algorithm is closed source it might be tricky to identify the exact cause of the problem, but imho it's well worth a try. The dng converter seems to erroneously identify and compress these specific regions, maybe adding some noise or similar could be added to prevent this from happening?

Here's a sample how the lossy dng artifacts usually look: https://bitbucket.org/hudson/magic-lantern/issues/attachments/2333/hudson/magic-lantern/1438442876.44/2333/buggy_lossy-dng.png

mothaibaphoto

  • Senior
  • ****
  • Posts: 392
  • pesky kid
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2531 on: August 01, 2015, 07:33:49 PM »
@Marsu42: You trying to lossy compress with Adobe unprocessed Dual ISO? Why? I've read that lossy DNG compressed files no more Raw, it's demosaiced data. Demosaicing algorithm didn't expect to meet Dual ISO raw - this why artifacts, no?

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3397
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2532 on: August 01, 2015, 08:47:24 PM »
Lossy Compress is possible with Lightroom plugin from @kitcheof...

http://www.magiclantern.fm/forum/index.php?topic=11056.msg151229#msg151229
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 7508
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2533 on: August 01, 2015, 08:51:40 PM »
Lossy compress is just an argument for cr2hdr. Kitchetof's plug-in is wrapped around cr2hdr and that's what most Dual-ISO tools do.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3397
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2534 on: August 02, 2015, 12:12:14 AM »
I'm aware of that, Walter.

I apologized for the slight misunderstanding and I thought I was just trying to be helpful by letting them know that it is possible.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

dubzeebass

  • Senior
  • ****
  • Posts: 360
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2535 on: August 02, 2015, 12:30:28 AM »
I'm finding the dual ISO files a bit greener. 5D mark III 1.2.3 but build number doesn't matter because it's been all the way since I started using Dual ISO. Anyone notice this in the DNGs? WB is odd.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3397
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2536 on: August 02, 2015, 01:22:40 AM »
Do you mean that it doesn't show any greenish effect on 113 but it does 123?

Have you tested it on 113 yet yourself?

It looks fine to me here coming from the latest nightly 113...

Which converter did you use?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Marsu42

  • Contributor
  • Hero Member
  • *****
  • Posts: 1557
  • 66d + flashes
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2537 on: August 03, 2015, 10:05:48 PM »
@Marsu42: You trying to lossy compress with Adobe unprocessed Dual ISO? Why? I've read that lossy DNG compressed files no more Raw, it's demosaiced data. Demosaicing algorithm didn't expect to meet Dual ISO raw - this why artifacts, no?

My understanding that a dual_iso file should be the same as a vanilla raw after it's reconstructed by cr2hdr. Adobe supports these 16bit raw files in ACR (Lightroom, Photoshop, DNG Converter), and I see no reason why lossy compression shouldn't work as well - though of course you can never be sure with closed source software.

It's an important feature for Adobe apps - smart previews enable you to do faster and offline editing, and lossy dng is great for archival of less important shots w/o resorting to jpeg. Btw the "standard" preview in Lightroom is broken as well with the same artifacts, seems to use the same compression algorithm.

On the bitbucket bug tracker, dmilligan guesses that this might have to do with the non-standard black level dual_iso files. I cannot say and I'm unable to change cr2hdr to test this, but lossy dng *does* work fine with mini_mo files which also have a custom black level (but break dxo's raw converter).

I'm finding the dual ISO files a bit greener. 5D mark III 1.2.3 but build number doesn't matter because it's been all the way since I started using Dual ISO. Anyone notice this in the DNGs? WB is odd.

I'm on 6d/60d, and I consider wb "more than odd". I cannot verify the theory that dual_iso and vanilla should just look alike, just with dual_iso having cleaner shadows. My current method is to let Lightroom figure it out with the "auto" setting on import which works better than the current cr2hdr detection attempts for guessing auto wb. The "auto" setting also gives you a hint in which way to correct files with a fixed wb setting from the camera.

Having said that, hdr pictures somewhat do look a bit odd in the shadows anyway (esp. concerning green/magenta tint), so that might be one cause of confusion and bogus bug reports.

halbmoki

  • Freshman
  • **
  • Posts: 55
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2538 on: August 12, 2015, 05:56:48 PM »
Hello,
I have a little problem with dual ISO... with long exposures (1sec or more, though it seems to increase gradually), there are very strange artifacts in my images, as you can see from the image. This is a 1:1 crop from a 20sec exposure at ISO 100/800 with a 50D. These artifacts do not occur with the same exposure at a single ISO. Long exposure noise reduction has no influence at all.

Is this a known problem and is there any solution?

http://i239.photobucket.com/albums/ff252/mogul-/temp/IMG_8820-dualiso_zpslwk63idq.jpg

toze3

  • New to the forum
  • *
  • Posts: 13
DNG CR2HDR double tags and missing color matrix
« Reply #2539 on: August 14, 2015, 11:05:08 AM »
I was geting some troubles processing dual iso DNG in the latest version of Rawtherapee, i tought it was a white level problem and asked some help in their forum.

It seens there are some double tags in the dual iso DNG and missing color matrix?(not sure if only mine, with latest ML nightly on 550D and latest cr2hdr)

"your dng file includes both tags with the same value (21). In this case the second one (50779) is given precedence, but there is no valid color matrix for second one. Only the matrix for 50771 exists in your dng.
I try to catch this case in rt source, but you should also report the bug to the guys who made the hack to produce the dual iso dng files.
"

@a1lex could you have a look in this post
Code: [Select]
rawtherapee.com/forum/viewtopic.php?f=1&t=6185

Licaon_Kter

  • Hero Member
  • *****
  • Posts: 519
  • M
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2540 on: August 14, 2015, 12:46:23 PM »
@toze3:
Latest cr2hdr from here: https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/ right?



Do upload: Sample CR2 and Ouputed DNG.


Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 7508
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2541 on: August 14, 2015, 12:52:08 PM »
Sample DNG will be found in linked thread.
Missing untouched CR2, though.

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6861
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2542 on: August 14, 2015, 12:53:38 PM »
@toze. Could you upload an example dng?

heckflosse

  • New to the forum
  • *
  • Posts: 4
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2543 on: August 14, 2015, 01:20:11 PM »
Hi,

the problem with the DNG file is, that it contains the tags
50778 http://www.awaresystems.be/imaging/tiff/tifftags/calibrationilluminant1.html
and
50779 http://www.awaresystems.be/imaging/tiff/tifftags/calibrationilluminant2.html
Both have the value 21 and in this case Rawtherapee uses 50779. So far so good. But the corresponding color matrix for this case
tag 50722 http://www.awaresystems.be/imaging/tiff/tifftags/colormatrix2.html
does not exist in the DNG
I already prepared a fix for Rawtherapee, but IMHO, if tag 50722 does not exist, tag 50779 shouldn't exist too

Ingo

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6861
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2544 on: August 14, 2015, 01:47:39 PM »
Certain programs implements tags and color matrix information. Provide a dng so people could check.

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 7508
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2545 on: August 14, 2015, 01:57:14 PM »
550D was used for sample.
Will try to reproduce with 650D.
Aware Systems offers a free TIFF tag viewer for Windows: http://www.awaresystems.be/imaging/tiff/astifftagviewer.html ... and it doesn't like DNG that much ...

heckflosse

  • New to the forum
  • *
  • Posts: 4
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2546 on: August 14, 2015, 01:57:58 PM »
Here's the link to the DNG (copied from RT forum) https://www.dropbox.com/s/r3k9t5pbvn6oe5i/_MG_1712_DI200_400.DNG?dl=0

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 7508
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2547 on: August 14, 2015, 02:32:45 PM »
Okay, Exiftool works.

Happens with 650D, too.
Tag 0xc621 (50721) Colormatrix1 exists but Colormatrix2 missing.

Workaround: Use exiftool to create tag Colormatrix2 and copy Colormatrix1 value into.

EDIT: Created issue/bug report #2343 https://bitbucket.org/hudson/magic-lantern/issues/2343/cr2hdr-not-generating-tag-colormatrix2


toze3

  • New to the forum
  • *
  • Posts: 13
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2548 on: August 14, 2015, 03:39:12 PM »
@toze3:
Latest cr2hdr from here: https://builds.magiclantern.fm/jenkins/view/Other%20tasks/job/cr2hdr-20bit/ right?

YES
Do upload: Sample CR2 and Ouputed DNG.
I don't have the same CR2 was the DNG posted but i can upload a new ones.
CR2: https://www.dropbox.com/s/1p3tbx44vl9msgg/_MG_2498.CR2?dl=0
DNG: https://www.dropbox.com/s/83yju7334ibj19m/_MG_2498_DI100_800.DNG?dl=0
It happens in all my dual iso DNG.

Hope this helps


Workaround: Use exiftool to create tag Colormatrix2 and copy Colormatrix1 value into.


How do i do that?

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6861
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #2549 on: August 14, 2015, 04:04:43 PM »
I think dcraw only outputs one color matrix tag.
If you,re on mac I could probably implement the insert of the color matrix 2 in my tool cr2hdr-r also for CR2 to DNG conversion. I already insert it it in movie dng sequences. Not familiar with the windows system anymore unfortunately.