Author Topic: Dual ISO - massive dynamic range improvement (dual_iso.mo)  (Read 1839393 times)

Canon eos m

  • Hero Member
  • *****
  • Posts: 562
  • 5D Mark III, Canon EOS M
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #225 on: July 21, 2013, 11:40:59 AM »
Hi Audionut could you please help with the latest binary of the revised module code. Do not know how to do it myself. Running the dual ISO module that you updated last.
Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)

Digital Corpus

  • Freshman
  • **
  • Posts: 66
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #226 on: July 21, 2013, 12:08:46 PM »
If I got off my arse and compiled a 7D copy from the repo, I'd host it. I have no time for extra projects though and this would fall into that category. Hosting isn't a problem when I've pushed through 1 TB in 3 weeks off my home connection and Verizon hasn't cared...
7D w/ ML | Tokina ATX 11-16 | Canon 24 mm pancake | Canon 40 mm pancake | Canon 17-55 f/2.8 IS | Sigma 150-600 Sports

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #227 on: July 21, 2013, 03:24:12 PM »
How do you find which bits are iso and which bits are flag bits?

50D:

Code: [Select]
16C - 3200
16C - 1600
1FC - 1000
1B4 - 500 -320
94  - 250
04    100

10 11 0 1 1 00
11 11 1 1 1 00
11 01 1 0 1 00
01 00 1 0 1 00
  1 00

Also FRAME_CMOS_ISO_SIZE

6D reg0 doesn't move... have to check the stubs:

Code: [Select]

iso 100 - 0
800 - 33
1600 -44
3200 -55
6400 - 77
128k - 77
256K - 77


/* 00:00:05.898461  */     03 0000 40450E08
/* 00:00:25.234033  */     03 0033 40450E3E
/* 00:00:55.186093  */     03 0044 40450E50
/* 00:01:23.931307  */     03 0055 40450E62
/* 00:01:39.858203  */     03 0077 40450E74

 11 0011
100 0100
101 0101
111 0111

ap1hk

  • New to the forum
  • *
  • Posts: 5
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #228 on: July 21, 2013, 07:31:19 PM »
Hi,

I took still photo mostly and didn't use ML before as I thought ML was for movie mostly.

I believe this update is amazing and drove me to try ML finally. So far my test did show some improvements in DR but depending on situations sometimes the difference are not very apparent.

I would also like to report some suspected bugs when using the Dual_ISO module for photo.
1. Auto ISO doesn't work well with relative setting of Recovery ISO. It will give very badly under-exposed photo (e.g. I usually set at +4EV but even setting to negative the symptom is the same.)
2. The "alternative frames only" option will fail for continuous shots and sometimes it seems just completely stop taking @ dual_ISO at all after a while.
3. The converted DNG always shows a WB of 5500K and tint=17 when I imported them into Lightroom, independent of the original WB at shot.

Regards,

Peter

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #229 on: July 21, 2013, 09:16:13 PM »
1. Try a fixed recovery ISO (not relative).

2. It looks at file number (IMG_1234) and burst is not supported. Once you know that, it's completely predictable.

3. Check if exiftool is working (post a conversion log).

jpaana

  • New to the forum
  • *
  • Posts: 28
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #230 on: July 21, 2013, 09:47:49 PM »
Using cr2hdr from today (as of https://bitbucket.org/hudson/magic-lantern/commits/00ea74a66945b9626e0fe53807232af831d62269) compiled on OSX, I get a crash on one particular file:

Input file     : IMGL9830.CR2
Full size      : 5920 x 3950
Active area    : 5796 x 3870
Black borders  : 124 left, 80 top
ISO pattern    : BddB RGGB
Interpolation  : mean23-vsmooth3-contrast

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0xaa5e3408
0x00006967 in hdr_interpolate () at cr2hdr.c:1126
1126                   dark_smooth[x + y*w] = ev2raw[(raw2ev[a] + raw2ev[ b] + raw2ev[ b] + raw2ev[c]) / 4];
(gdb) p a
$1 = -30

Other files I've tried so far convert fine so I put the file up at https://www.dropbox.com/s/ruv13bn6ibkzm08/IMGL9830.CR2

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #231 on: July 21, 2013, 10:19:11 PM »
For some reason, dcraw was writing a pixel value of 65506 (come on, for 14-bit data, the range is 0-16383...). This also breaks my calculations (even the robust statistics part). The image is read correctly though.

I've converted the shot by hacking the source code a bit, but before providing a fix, I want to understand exactly what's going on.

http://acoutts.com/a1ex/IMGL9830.DNG
http://acoutts.com/a1ex/IMGL9830.jpg

jpaana

  • New to the forum
  • *
  • Posts: 28
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #232 on: July 21, 2013, 10:30:23 PM »
For some reason, dcraw was writing a pixel value of 65506 (come on, for 14-bit data, the range is 0-16383...). This also breaks my calculations (even the robust statistics part). The image is read correctly though.

I've converted the shot by hacking the source code a bit, but before providing a fix, I want to understand exactly what's going on.

http://acoutts.com/a1ex/IMGL9830.DNG
http://acoutts.com/a1ex/IMGL9830.jpg

Excellent, thank you!

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #233 on: July 21, 2013, 10:46:36 PM »
/* 00:00:05.898461  */     03 0000 40450E08
/* 00:00:25.234033  */     03 0033 40450E3E
/* 00:00:55.186093  */     03 0044 40450E50
/* 00:01:23.931307  */     03 0055 40450E62
/* 00:01:39.858203  */     03 0077 40450E74

This one looks really good.

Code: [Select]
        ?_CMOS_ISO_START = 0x40450E08;
        ?_CMOS_ISO_COUNT = at least 7; // from ISO 100 to 6400
        ?_CMOS_ISO_SIZE  =  18;

        CMOS_ISO_BITS = 4;
        CMOS_FLAG_BITS = 0;
        CMOS_EXPECTED_FLAG = 0;

If it works, the theory about 8-channel readout is invalidated (the 6D has 4 channels afaik).

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #234 on: July 22, 2013, 01:34:56 AM »
I get error 1 in photo & LV, reg not 0? So 7D + 5d3 = 8 channels and everything else = 4?

ap1hk

  • New to the forum
  • *
  • Posts: 5
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #235 on: July 22, 2013, 03:28:31 AM »
1. Try a fixed recovery ISO (not relative).

2. It looks at file number (IMG_1234) and burst is not supported. Once you know that, it's completely predictable.

3. Check if exiftool is working (post a conversion log).

Thanks.
1. Yes, autoISO with fixed recovery ISO or fixed ISO with relative ISO both okay. Just to report the combo I find not working.
2. IC. Thx.
3. I am using Mac OS and I compile cr2hdr myself. I found that the my exiftool is a perl interface to Image::ExifTool. There is no error message from stdout of cr2hdr, e.g.

Input file     : 5D3_6476.CR2
Full size      : 5920 x 3950
Active area    : 5796 x 3870
Black borders  : 124 left, 80 top
ISO pattern    : dBBd RGGB
Interpolation  : mean23-vsmooth3-contrast
ISO difference : 2.97 EV (784)
Black delta    : 2
ISO overlap    : 6.0 EV (approx)
Hot pixels     : 1055692
Output file    : 5D3_6476.DNG
    1 image files updated

For conversion log, do you mean those tmp.txt at conversion?
I will check if exiftool really work later.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #236 on: July 22, 2013, 06:38:35 AM »
I get error 1 in photo & LV, reg not 0?

From the dumps, it's register 3, right? (I didn't use the original dump format, but some custom hooks)

Try overriding it manually (just change that memory address so it reads 0x3 instead of 0x33, for example).

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #237 on: July 22, 2013, 09:44:00 AM »
Some comparison results with cr2 files. No question you get much cleaner shadows with less noise in dual iso. It,s pretty amazing to say the least. Done with "smooth" version. I think I used 100-1600 in dual iso. Can,t remember but I think I even went 100-3200 on another example and got nice results on this particular view.


dng original(dual iso)


dng dual iso processed in lightroom


regular cr2 processed in lightroom (not dual iso)


dng pressed shadow (dual iso)


cr2 regular pressed shadow

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #238 on: July 22, 2013, 11:01:07 AM »
Here,s an example where I went between I believe 100-3200 or even further. I,ll post a exif from the conversion app which show the ev settings. It,s like "turning on the lights"


dng original(dual iso)


dng processed in lightroom(dual iso)


Crop 1, not very sharp due to handshake but clean!(dual iso)


Crop 2(dual iso)



ev settings

Mike Bluestone

  • New to the forum
  • *
  • Posts: 4
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #239 on: July 22, 2013, 11:35:16 AM »
Dual ISO works well, great new function. Problem I'm having, which may be normal, is that when I activate Dual ISO the display is really "wavy", almost like the scan lines from out of sync artificial lights, but worse.
Would like to know if this is to be expected or is there a problem with my set up specifically?

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #240 on: July 22, 2013, 11:39:05 AM »
Read the first post. All info there

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3654
  • Blunt and to the point
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #241 on: July 22, 2013, 01:59:17 PM »
ISO 100/800




Base ISO 100 shot with same post processing.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #242 on: July 22, 2013, 02:04:49 PM »
@Danne: try processing the same shots with the latest cr2hdr. You should get much cleaner shadows (so I shouldn't have to read the text to know which is which ;) )

In particular, the black spots should no longer be there.

Canon eos m

  • Hero Member
  • *****
  • Posts: 562
  • 5D Mark III, Canon EOS M
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #243 on: July 22, 2013, 02:25:49 PM »
Hi Audionut could you please load the binary for the latest dual USO module. I am currently using the one you created last.
Canon 5D Mark III, Gopro Hero Blacks with 3D Casing, A Few Lenses, Adobe CC 2014, MacBook Pro, Windows 8 PC, Lots of Video Rig!

Started Nuke. Loved it but then the 15 day trial ran out. Back to After Effects and loving it :-)

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3654
  • Blunt and to the point
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #244 on: July 22, 2013, 02:31:08 PM »
a1ex maintains an updated cr2hdr binary in the first post.

There doesn't appear to be any updates to the module since my last build.  Only the cr2hdr binary.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #245 on: July 22, 2013, 02:41:51 PM »
Two examples with crops in extreme dual iso, 100-6400. As you can see it works fairly well and it blows the native cr2 miles, miles away. Even tried a 100-12800 iso but had to much magenta issues in certain mid-whites. Settings processed the same in lightroom for both the crg and dual iso dng. Had to drag the exposure slider all the way to the right with the dual iso-file. Probably could have gone a little further but lightroom stopped me from trying ;).


dng 100-6400 original



dng 100-6400 processed


dng 100-6400 cropped


cr2 iso 100 original


cr2 iso 100 processed


cr2 iso 100 crop


Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7154
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #246 on: July 22, 2013, 02:47:04 PM »
@Alex. Appears to be a new hdr converter from Alex. I will reconvert the pics in time. I,m on the countryside and slooow bandwidth. Today uploads took me two hour to complete.
Amazing work Alex.

Stedda

  • Hero Member
  • *****
  • Posts: 504
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #247 on: July 22, 2013, 03:17:51 PM »
Had to drag the exposure slider all the way to the right with the dual iso-file. Probably could have gone a little further but lightroom stopped me from trying ;).

Using the brush tool you can apply layered effects that would exceed 100%. You may already know this though.  :)
5D Mark III -- 7D   SOLD -- EOS M 22mm 18-55mm STM -- Fuji X-T1 18-55 F2.8-F4 & 35 F1.4
Canon Glass   100L F2.8 IS -- 70-200L F4 -- 135L F2 -- 85 F1.8 -- 17-40L --  40 F2.8 -- 35 F2 IS  Sigma Glass  120-300 F2.8 OS -- 50 F1.4 -- 85 F1.4  Tamron Glass   24-70 2.8 VC   600EX-RT X3

pipino

  • Just arrived
  • *
  • Posts: 1
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #248 on: July 22, 2013, 04:18:50 PM »
It seems a great idea!

which FW I've to load for double_iso on my 5DMK3?


JackDaniel412

  • New to the forum
  • *
  • Posts: 38
Re: Dual ISO - massive dynamic range improvement for 5D3 and 7D
« Reply #249 on: July 22, 2013, 05:39:56 PM »
Highlight problem with last raw2dng.



DNG file

http://dcmakecreative.altervista.org/ML/DNG000000.dng