Dual ISO - massive dynamic range improvement (dual_iso.mo)

Started by a1ex, July 16, 2013, 06:33:50 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

Onoxyne

Quote from: fotojohni on October 07, 2013, 06:01:18 PM

Not possible without dual iso.  This is a truly incredible development.  Now if only it didn't have to go line by line and instead could be a dual bayer pattern with 6 instead of 3. 

Also, any chance for an adaptive algorithm that measures scene contrast in live view then auto selects?   

Just putting out random ideas I bet you are already cooking up something even better.  can't wait.

fotojohni : what are the photo parameters ? Which camera ?

fotojohni

Quote from: Onoxyne on October 07, 2013, 09:33:10 PM
fotojohni : what are the photo parameters ? Which camera ?

Camera: 6D
ISO 100/1600
Lens: 50mm @ 1.4
VAF filter to reduce moire/aliasing
Shot in manual mode @ 1/45

For night shooting I switch recovery iso to 100 and shoot at 1600, then I can actually see what I am shooting in live view.  I didn't do that here though.

For grading I just went into ACR and bumped the exposure up 3.0 stops, boosted shadows etc.  I have a profile I like. 

akumiszcza

Quote from: a1ex on October 07, 2013, 02:00:47 PM
As I said earlier, the issue is in EXIF. So, just find the exiftool command that copies WB properly from the CR2 and I'll integrate it.

This is what I found, maybe something will be useful in the future:

I renamed the files mentioned before to norm.cr2, dual.cr2 and dual.dng.

Adobe Camera Raw shows the following temperature/tint values ("As shot"):
norm.cr2: 5200/+14
dual.cr2: 5850/+38
dual.dng: 4850/+46

As you can see dual.cr2 has it wrong, too, but better than dual.dng anyway.

I tried using exiftool commands, but none worked for me:
exiftool -tagsfromfile norm.cr2 -Canon:WhiteBalance -WB_RGGBLevelsAsShot -WB_RGGBLevelsAuto -WB_RGGBLevelsMeasured dual.dng
(mentioned somewhere by the author of exiftool)
exiftool -WhiteBalance#=5200 dual.dng

Then I used hex editor to alter dng on my own.
After setting crs:Temperature="5200" and crs:Tint="+14" ACR shows the proper values of dual.dng file. This could be a good way of changing dng directly, but I don't know how to read temperature and tint value from a file. I've tried exiftool, exiv2, dcraw and all of them had other formats. And, as I found out just before posting it, it only works if ACR already exported xmp values into dng (standard cr2hdr output does not have it). But if it's possible to add some xmp to dngs, then crs:Temperature and crs:Tint works.

Second thing I've noticed: dcraw -v -w norm.cr2 shows the following: multipliers 2.102539 1 1.365234 1.

Now the command dcraw -T -r 2.102539 1 1.365234 1 dual.dng creates dual.tiff with correct WB, so it might be possible to use this method somewhere in cr2hdr.c.

1drey

Sorry for stupid question.

I've dragged CR2 created with Ml and dual ISO, the window got appeared and seemingly the conversion happened, but I can't find a resulting DNG
Where it should be?

PS. 've wasted a plenty of time trying to use 32 bit dcraw in 64 bit environment. Probably it is necessary to mention that users with 64x Windows need proper dcraw

Marsu42

Quote from: 1drey on October 08, 2013, 10:09:04 AM
PS. 've wasted a plenty of time trying to use 32 bit dcraw in 64 bit environment. Probably it is necessary to mention that users with 64x Windows need proper dcraw

It isn't mentioned because it isn't true, x86 apps work just fine on x64 Windows. If you are really determined to try yourself - Get both versions of the current dcraw here: http://www.fastpictureviewer.com/downloads/

The dng file you seek ends up in the same folder as the cr2 after drag n' drop.

pest01

Hi to everyone, i download the new Windows:cr2hdr.exe and it doesn't seem to be working on my windows i get a dcraw.exe - System error that says <<cant start cygwin1.dll is missing>> how do i fix it or do i stay with the previous one that worked? Are there any major changes in those two?

Thanks!!

1drey

It is strange. I was getting " 'the application was unable to start correctly (0xc000007b)' " until 32 bit dcraw was replaced with x64 one.

And for some mysterious reason I don't see resulting DNG in the source folder. It was the first place where I expected it to appear.
Seems that *******.dng file was not created at all - I was searching it through the file search on my PCwith no avail.

akumiszcza

Quote from: 1drey on October 08, 2013, 10:33:50 AM
And for some mysterious reason I don't see resulting DNG in the source folder. It was the first place where I expected it to appear.
Seems that *******.dng file was not created at all - I was searching it through the file search on my PCwith no avail.

Put the .cr2 dual iso file in the same folder with cr2hdr.exe, dcraw.exe, exiftool.exe. Run cmd.exe there and execute "cr2hdr <yourfilename>". See if any errors appear.

1drey

Quote from: pest01 on October 08, 2013, 10:27:54 AM
Hi to everyone, i download the new Windows:cr2hdr.exe and it doesn't seem to be working on my windows i get a dcraw.exe - System error that says <<cant start cygwin1.dll is missing>> how do i fix it or do i stay with the previous one that worked? Are there any major changes in those two?

Thanks!!

I was fighting with the same issue.
Re-installed Cigwin, but started to receive a message  'the application was unable to start correctly (0xc000007b)'
Replaced dcraw with x64 version

akumiszcza

Quote from: pest01 on October 08, 2013, 10:27:54 AM
Hi to everyone, i download the new Windows:cr2hdr.exe and it doesn't seem to be working on my windows i get a dcraw.exe - System error that says <<cant start cygwin1.dll is missing>> how do i fix it or do i stay with the previous one that worked? Are there any major changes in those two?

Just replace dcraw.exe with a version not requiring cygwin1.dll or install cygwin in your system.

a1ex

Do you have a link to a non-cygwin dcraw.exe that also works for 6D, so I can put that one in the zip?

1drey

Quote from: akumiszcza on October 08, 2013, 10:37:13 AM
Put the .cr2 dual iso file in the same folder with cr2hdr.exe, dcraw.exe, exiftool.exe. Run cmd.exe there and execute "cr2hdr <yourfilename>". See if any errors appear.
Got this

akumiszcza

Quote from: a1ex on October 08, 2013, 10:47:48 AM
Do you have a link to a non-cygwin dcraw.exe that also works for 6D, so I can put that one in the zip?

I don't have 6D to check. I use some 9.17 version 576064 byte on my 50D. Some new builds of 9.19 are here: https://sites.google.com/site/manuelllorens/ – but I didn't test those.

akumiszcza

Quote from: 1drey on October 08, 2013, 10:48:43 AM
Got this


Looks like you're in another folder...

Open the folder in explorer. Check if all your files are here. shift+rightclick and choose "open command prompt here" (it does not work on every windows version) and check again. Or run cmd.exe and cd <yourfolderpath> before running the command.

Marsu42

Quote from: a1ex on October 08, 2013, 10:47:48 AM
Do you have a link to a non-cygwin dcraw.exe that also works for 6D, so I can put that one in the zip?

As I'm posting to the extend of feeling ignored :-o ... http://www.fastpictureviewer.com/downloads/ ... you gotta scroll down a bit or search the page for "dcraw"

Btw a multi-threaded dcraw is here, but as it's not the newest version it unfortunately will only work for older cameras (i.e. not the 6D) - but for the rest it should be faster: http://www.heliconsoft.com/heliconsoft-products/dcraw/

1drey

Quote from: akumiszcza on October 08, 2013, 11:05:23 AM
Looks like you're in another folder...

Open the folder in explorer. Check if all your files are here. shift+rightclick and choose "open command prompt here" (it does not work on every windows version) and check again. Or run cmd.exe and cd <yourfolderpath> before running the command.

It kind of helped! :)

How I get this:


If I replace x64 dcraw with one from archive - I get an error message ('couldn't start correctly' etc.)

a1ex

Updated the zip on main page with dcraw 9.19 32-bit from the link from Marsu42. Does it work?

@1drey: make sure you update ML for EOS M. Early versions produced files that can't be converted.

akumiszcza

I'd like to share my quick and very dirty way to get dual iso files from CF card as fast as possible (...)
Topic moved here: http://www.magiclantern.fm/forum/index.php?topic=8738.msg81768#msg81768
Please post your comments there.

painya

Quote from: akumiszcza on October 08, 2013, 11:45:14 AM
I'd like to share my quick and very dirty way to get dual iso files from CF card as fast as possible.
Normal steps are:
1. Get files from CF cards (it's a long process for me especially with USB 2.0 card reader and some slow 30MB/s cards).
2. Process them by cr2hdr (I use the script provided by engardeknave (http://www.magiclantern.fm/forum/index.php?topic=8520.0) without any prefix as I get quite many dual-iso files without prefix).
3. Rename them (I use exif date based system that moves filename.cr2 into something like a:\2013\10\07\20131007T121930filename.cr2).
4. Find dual-iso .cr2s in those created dirs and move them to dualiso subdir (I want to see only generated dngs while browsing, if something goes wrong I can alway go to dualiso subdir for original).

This takes a looong time.

Here is a zip file containing cr2hdr.exe, exiftool.exe, dcraw.exe (9.19, not requiring cygwin1.dll) together with dualiso.vbs (altered script by engardeknave), commands.cmd (this calls cr2hdr.exe and changes names), and runiffiles.cmd (a loop that calls dualiso.vbs if there's any file left to process).

Now, put all those in one directory, upload all .cr2 files in the same dir and run dualiso.vbs (you can change maxprocs to higher number for 4-core processors, but see KNOWN BUGS at the end). After some time you'll get all your files processed and put into different directories

Second method is even more dirty but gives you ability to run the script while getting files from the cards. Just start downloading from your card into the directory with the unzipped files and run runiffiles.cmd (after some bunch of .cr2 files are already there). The crazy script just checks constantly if there are any .cr2 files and calls dualiso.vbs then. So you can put all photos from several different cards and in the meantime the computer will process and rename them for you. You can leave the computer while moving the files and at the end everything will be done.

THESE SCRIPTS ARE PROVIDED AS IS AND I DON'T GIVE ANY WARRANTY IT WORKS AND DOES NOT DELETE YOUR PHOTOS SO PLEASE TEST IT ON SOME COPIES FIRST!!!!

KNOWN BUGS:
1. The script sometimes (especially at the beginning) calls more than maxprocs threads of cr2hdr.exe – that's probably because it tests for working cr2hdr.exe but calls commands.cmd, which gives some racing condition.
2. runiffiles.cmd sometimes never stops because some file already exists and exiftool won't move it, which makes runiffiles call dualiso.vbs again and it loops. I think it's because dualiso.vbs exits before all cr2hdr.exe processes are done. The easiest way to cope with this is to move the lonely files to other location, wait for runiffiles to stop and check if the files are already processed (should be).

ZIP FILE
http://bit.ly/dualisopack2

As I said, it's very dirty but WFM and saves me some time. Feel free to make it somewhat more user friendly and post it here.
I think this deserves its own thread in the post processing forum. Any one else?
Good footage doesn't make a story any better.

chris_overseas

I'm using a custom file prefix for my images, set via the standard Canon menus (5D mark 3). When using dual-ISO I would like to use the "DUAL" prefix instead, but enabling the "Custom file prefix" option in the dual-ISO settings doesn't seem to have any effect. All my dual-ISO images are still prefixed with my custom file prefix. Is that by design or is it a bug? I'm hoping it is a bug... :)
EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II

1%

He who gets there first, gets the prefix and keeps it. So if you have one from bracketing or one from dual ISO you won't be able to change it until whatever changed it releases it.

akumiszcza


a1ex

Sensor response was nonlinear... I may need to estimate ISO in log space.

First one solved.

Second one is harder and it seems to have issues with chroma smoothing.

Doyle4

What camera did you use akumiszcza? i noticed i had an image like your bottom sample on my 600D, only the one though.

akumiszcza


Quote from: Doyle4 on October 10, 2013, 02:19:41 PM
What camera did you use akumiszcza? i noticed i had an image like your bottom sample on my 600D, only the one though.

It's 50d. The problem on the bottom sample appeared in some previous photos in dark scenes, but only on small fragments.