Author Topic: Dual ISO - massive dynamic range improvement (dual_iso.mo)  (Read 2156632 times)

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1950 on: March 31, 2014, 06:04:23 AM »
AMAZING PROGRESS!!!

mmenno

  • New to the forum
  • *
  • Posts: 3
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1951 on: March 31, 2014, 04:33:09 PM »
Hi, first of all thanks for the amazing effort to all who are contributing to ML!

Now, as I'm not sure where to post I'll ask here: On my 7D, using the latest nightly build (3-31) and the builds I've tried before, dual iso doesn't seem to be working in liveview, sometimes there are a few in a series of images that are dual iso, but roughly 80% aren't, just ordinary base iso files. When not using liveview all is well and all files are dual iso. Is there something I'm missing, some setting I've overlooked? I've tried searching this forum but to no avail.. I'm trying to use it for pano's, so LV would be nice to have, and having a pano with mixed dual and normal iso images just doesn't work ;)

Any help would be really appreciated!

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1952 on: March 31, 2014, 09:57:44 PM »
Do you mean regular pictures in LiveView, or silent pictures?

Also make sure you are using photo mode LiveView, not movie mode.

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1953 on: March 31, 2014, 10:12:59 PM »
A1ex, April 1st tomorrow... Sensor Up...!!!!  ;D

mmenno

  • New to the forum
  • *
  • Posts: 3
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1954 on: March 31, 2014, 10:14:13 PM »
Thanks for the reply. Regular or silent doesn't seem to make a difference, when I shoot using te viewfinder I get dual iso files, when I switch to LV and don't change anything else I get normal iso files. I'm not in movie mode either.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1955 on: March 31, 2014, 10:35:42 PM »
Alright; will check next week. Meanwhile you can file it on bitbucket on the issues page.

A1ex, April 1st tomorrow... Sensor Up...!!!!  ;D

https://nikonhacker.com/viewtopic.php?f=2&t=1840 :P

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1956 on: March 31, 2014, 11:01:35 PM »
A1ex, April 1st tomorrow... Sensor Up...!!!!  ;D

Wait a minute what? 

mmenno

  • New to the forum
  • *
  • Posts: 3
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1957 on: March 31, 2014, 11:16:22 PM »
Thanks a1ex, I've filed it on bitbucket.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1958 on: March 31, 2014, 11:20:26 PM »
Wait a minute what?

Poisson d'avril from budafilms :P

Marsu42

  • Contributor
  • Hero Member
  • *****
  • Posts: 1557
  • 66d + flashes
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1959 on: March 31, 2014, 11:59:56 PM »
With DNG at least in ACR/Lightroom, there is the option to embed a (color) calibration profile. Would it be possible to add the soft film curve as (part of) this embedded profile so users could dynamically switch it on and off w/o the need to re-process with cr2hdr?

Currently, the embedded profile after cr2hdr is "Adobe Standard", or more likely there is no embedded profile at all so Lightroom falls back to "Adobe Standard".

hjfilmspeed

  • Senior
  • ****
  • Posts: 479
  • 5D III and IV
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1960 on: April 01, 2014, 12:13:00 AM »
Poisson d'avril from budafilms :P

Ha! oh duh im slow.......

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1961 on: April 01, 2014, 07:52:56 AM »
Poisson d'avril from budafilms :P

Jajajajaj  :D
I am imagine something like sensor protocol?
It's a huge thing....

arturochu

  • Member
  • ***
  • Posts: 129
  • Filmmaker based in Mexico City
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1962 on: April 02, 2014, 07:59:27 AM »
Is there any way to have dualISO mlv files "tagged"? When using ETTR connected with dualISO we often get mixed files (normal and dualiso) and the only way to know if the clip is dualISO or not is to open one of the DNGs (which takes a lot of time with dozens of files). If we have DI files with some different name it would be possible to easly filter such a files out of the rest and process them with any dual iso app (or script) right away. Is metadata of MLV module has a "room" for such a tagging?

EDIT: Maybe something like 'custom file prefix' but for movies.

+1

and also can someone fix the flickering problem with dual iso video? i really dig dual iso video but i don't know how to code, i don't think its a big thing to do but i've been wrong in the past.

cheers
Chu

tombu

  • New to the forum
  • *
  • Posts: 5
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1963 on: April 02, 2014, 04:16:28 PM »
A few updates to cr2hdr-20bit.exe

..


Could someone post a working link? I got 403 forbidden code when I clicked the link. Would like to try the 20-bit version.

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8542
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1964 on: April 02, 2014, 04:29:08 PM »
Use right mouse button -> Save as ...

DavidSh

  • Member
  • ***
  • Posts: 141
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1965 on: April 02, 2014, 04:47:18 PM »
Any Update for cr2hdr mac users?
600D | 5D3 | macOS Sierra | http://www.GentleDogMovie.com

tombu

  • New to the forum
  • *
  • Posts: 5
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1966 on: April 02, 2014, 05:39:32 PM »
Use right mouse button -> Save as ...
Unknown server error.. Can other people download it just fine?

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8542
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1967 on: April 02, 2014, 07:25:23 PM »
Maybe your antivirus software doesn't like this one.
@a1ex: ZIP/RAR would be fine. Will prevent file version/date problems, too.

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1968 on: April 03, 2014, 01:53:15 AM »
@a1ex: ZIP/RAR would be fine. Will prevent file version/date problems, too.

And reduces the bandwidth requirements by about 2/3rds.

The latest version is mirrored here:
https://dl.dropboxusercontent.com/u/34113196/ML/dual_iso/cr2hdr-20bit.zip

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1969 on: April 08, 2014, 08:58:40 PM »
Some progress with dual iso postprocessing for video / panoramas / timelapse / whatever else needs matching exposures between the frames (it should no longer flicker). To use it, specify the --same-levels option on the command line.

cr2hdr-20bit.exe

This is not a deflicker algorithm, so it requires manual exposure settings. It simply tries to keep the same difference between WhiteLevel and BlackLevel for all the files passed in the same command line. More details on the commit message.

On Windows, you may have some trouble passing a bunch of files on the command line (you can't use wildcards there, and for drag and drop you may have to create a batch file that also passes --same-levels), but I didn't try. On Mac/Linux it should work without much trouble.

I also did a small shell script to render a Dual ISO video to AVI: dualisovideo.sh. You may want to tweak it for each video (include exposure, white balance and other options), but it should do the job.

Here's an example, rendered with the above script (an older sample from papkee, which flickered a lot before this fix). Oddly enough, it's from 650D, but I didn't apply any pink dot fix...

This workflow is not very fast - for a 100-frame test video at 1920x1080 it took only 15 minutes, and for this 150-frame clip at 1472x626 it took 8 minutes (this includes the entire conversion from the captured .raw file to the final .avi file). There's a lot room for improvement in cr2hdr, and you don't need any hacking skills to do that (hint, hint).

I already did some small speed optimizations (you can check the changelog).

Marsu42

  • Contributor
  • Hero Member
  • *****
  • Posts: 1557
  • 66d + flashes
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1970 on: April 08, 2014, 09:55:00 PM »
On Windows, you may have some trouble passing a bunch of files on the command line (you can't use wildcards there)

For all your windows scripting needs, use either cygwin+bash or the free tcc/le (4dos/4nt successor) and run "select cr2hdr.exe --same-levels [dual*.cr2]": http://jpsoft.com/all-downloads/downloads.html

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1971 on: April 08, 2014, 10:24:03 PM »
Nice! I guess msys/mingw can do the same thing then (they also have a bash-like shell).

Here's another script for rendering dual iso videos, this time with tonemapping via enfuse:
dualisovideo-enfuse.sh.

For the sample clip, it took 12 minutes (so the time needed for ufraw+enfuse becomes comparable to the time needed for cr2hdr).


I uploaded two versions, with and without tonemapping, so you can check whether the tonemapping process introduces any flicker.

BTW, make sure you use ffmpeg from the ffmpeg web site (the one that comes with ubuntu is actually the libav fork, and it's buggy at least for this video).

Marsu42

  • Contributor
  • Hero Member
  • *****
  • Posts: 1557
  • 66d + flashes
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1972 on: April 09, 2014, 12:57:14 AM »
Nice! I guess msys/mingw can do the same thing then (they also have a bash-like shell).

To be correct: Cygwin has *the* latest and original bash shell which works 99.9% like in other unix flavors as cygwin emulates a full posix system. You can even make the ntfs file system case-sensitive through a windows registry entry... that's why I think having cygwin as a full ml dev environment is a good idea as it doesn't need admin rights to run unlike a vm.

Problems on the dev side usually only occur when trying to use autoconf/automake scripts that aren't aware of cygwin just like some some still ignore macosx/bsd. On the user side mixing cygwin with dos/windows programs without taking the different line endings into account might be tricky as cygwin also uses the unix-like lf. Different behavior otherwise is very far in between, most prominently with awk inline scripts which I fixed here

https://bitbucket.org/hudson/magic-lantern/pull-request/467/fix-cygwin-compilation-docs

dubzeebass

  • Senior
  • ****
  • Posts: 360
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1973 on: April 09, 2014, 04:52:39 AM »
Thanks a1ex! Do you have the apple binary?

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3657
  • Blunt and to the point
Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)
« Reply #1974 on: April 09, 2014, 05:38:58 AM »
Some progress with dual iso postprocessing for video / panoramas / timelapse / whatever else needs matching exposures between the frames (it should no longer flicker). To use it, specify the --same-levels option on the command line.

Excellent work.  To give an idea of the changes.

Here are 2 exposures from a panorama, that are correctly brightness matched with the latest cr2hdr.


And here is the same 2, using an old cr2hdr.



This makes batch processing significantly easier, since you don't also have to brightness match images.


edit:  Removed outdated build.