MLV Lite - The sweet taste of MLV with none of the extra calories™

Lua scripting updated


Twitter: @autoexec_bin    Facebook: magiclantern.firmware    Live chat: [email protected]

Want to help with Magic Lantern development, but have no coding skills?

Author Topic: Automatic color grading script for photos (uniform_exposure.py)  (Read 64332 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 8749
  • 5D Mark Free
Most of the time I'm too lazy to adjust colors in my photos, so I wrote a script that does this exactly the way I want. It gets roughly 80-90% of pics just right for my taste; for the others I can do manual adjustments.

The script is a few months old, so it was fine-tuned over the time. It's similar to the "Post Deflicker" feature from ML, but optimized for regular photos, not for timelapse.

What it does:
- applies exposure compensation to all photos (so the end result looks consistent)
- performs shadow and highlight recovery, with the "HDR from a single RAW" tonemapping trick and enfuse

Download: uniform_exposure.py or browse the Mercurial repository.

Usage: look in the comments. There are no command-line params; edit the script to tweak stuff.

Dependencies: python, numpy, dcraw, ufraw, enfuse and ImageMagick.

Sample photo developed with the script, from this DNG, without any post-processing after running the script:



How I've got that DNG, I'll tell you later these days.

Caveats:
- By default it develops at half-res, for speed. To use full resolution, set --shrink=1 instead of 2.
- I use a modified ufraw, so you may not get exactly the same result for this sample DNG.
- I didn't try it for timelapse. It may flicker a bit more than ML post deflicker, but you can try it.
- Don't expect it to be a speed demon. It will spend several seconds on a raw file. So, first select your shots that you want to keep, then fire the script and let it crunch your pics.
- For Windows, see this step-by-step guide.

Tips & tricks:
- To re-develop a picture with different settings, just delete that jpeg and run the script again.
- You can use any kind of raw files recognized by dcraw (not just Canon). I've tried only CR2 and DNG.

You can post sample pics here, I'd like to see how well it works for you.

Enjoy!
« Last Edit: December 01, 2013, 09:14:16 AM by a1ex »

escho

  • Senior
  • ****
  • Posts: 406
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #1 on: July 10, 2013, 09:20:34 PM »
great result, Alex.

If you enable the hot pixel reduction in ufraw with a sensitivity of about 10, than the 2 hot pixels on the lower right side will go, too, I think. :)

Edgar
EOS 600d
OpenSuse 13.1

Audionut

  • Moderators
  • Hero Member
  • *****
  • Posts: 3310
  • I have a 1Dx, I'm awesome.
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #2 on: July 13, 2013, 04:14:00 AM »
Any idea what's happening here on a win x64 machine?

Code: [Select]
raw\D46A7239.CR2:
Invalid Parameter - -type
C:\Python27\lib\site-packages\numpy\core\_methods.py:57: RuntimeWarning: invalid
 value encountered in double_scalars
  ret = ret / float(rcount)

Edgar Matos

  • New to the forum
  • *
  • Posts: 49
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #3 on: July 13, 2013, 05:08:05 AM »
Seems promising.

0_0 Testing mode_activated

Sijimo

  • New to the forum
  • *
  • Posts: 10
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #4 on: July 13, 2013, 11:43:18 AM »
Cool!
« Last Edit: July 13, 2013, 11:46:00 AM by Sijimo »

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 8749
  • 5D Mark Free
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #5 on: July 13, 2013, 10:36:47 PM »
Another sample, this time a video frame from 5D3:



(bottom left is ISO 100, top right is with a small ADTG trick, at the same shutter speed and aperture; both graded with the above script so I could compare them)

DNG

Still fine-tuning the details.

Audionut

  • Moderators
  • Hero Member
  • *****
  • Posts: 3310
  • I have a 1Dx, I'm awesome.
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #6 on: July 13, 2013, 11:10:13 PM »
top right is with a small ADTG trick

Makes a massive difference.

swinxx

  • Hero Member
  • *****
  • Posts: 597
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #7 on: July 13, 2013, 11:18:22 PM »
that looks really great

is it possible that i can test this with my 5dmk3...?

greets. swinxx

Greg

  • Senior
  • ****
  • Posts: 478
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #8 on: July 14, 2013, 12:10:28 AM »
 :o it looks great!

aaphotog

  • Senior
  • ****
  • Posts: 356
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #9 on: July 14, 2013, 03:49:23 AM »
Another sample, this time a video frame from 5D3:



(bottom left is ISO 100, top right is with a small ADTG trick, at the same shutter speed and aperture; both graded with the above script so I could compare them)

DNG

Still fine-tuning the details.
ISP ONE HUNDRED... Is the noisy one?
How'd it get so noisy?
Pitch black room, you got it that bright in post?

Greg

  • Senior
  • ****
  • Posts: 478
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #10 on: July 14, 2013, 04:11:04 AM »
It looks brighter ISO 100 with 5EV  :o

swinxx

  • Hero Member
  • *****
  • Posts: 597
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #11 on: July 14, 2013, 10:43:53 AM »
hello,

the question should not be: how did you get so noisy? :) ..

the question should be: how did you remove so much noise with your next masterpiece of code.. :)

i am thrilled!!!!
++++ great job

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 8749
  • 5D Mark Free
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #12 on: July 14, 2013, 10:50:45 AM »
Easy, with that theory that almost nobody believed: ISO 1600 is less noisy than ISO 100.

;)

Something along these lines: http://www.guillermoluijk.com/article/nonoise/index_en.htm

Audionut

  • Moderators
  • Hero Member
  • *****
  • Posts: 3310
  • I have a 1Dx, I'm awesome.
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #13 on: July 14, 2013, 12:19:26 PM »
Easy, with that theory that almost nobody believed: ISO 1600 is less noisy than ISO 100.

Lol.  Pretty tricky being able to keep the lighter bits at ISO 100!

More exciting code from the Dev team  :)

Greg

  • Senior
  • ****
  • Posts: 478
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #14 on: July 14, 2013, 01:58:23 PM »
It requires a few photos?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 8749
  • 5D Mark Free
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #15 on: July 14, 2013, 02:04:50 PM »
No, just one.

pascal

  • Freshman
  • **
  • Posts: 91
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #16 on: July 14, 2013, 02:57:46 PM »
You strive for perfection. I like that. I don't understand the tools process though.  :P

IliasG

  • Member
  • ***
  • Posts: 124
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #17 on: July 14, 2013, 11:07:30 PM »
Easy, with that theory that almost nobody believed: ISO 1600 is less noisy than ISO 100.

;)

Something along these lines: http://www.guillermoluijk.com/article/nonoise/index_en.htm

Well, that theory is pretty established, even Dr Emil Martinec made an experiment-presentation using the same idea you use
http://www.dpreview.com/forums/post/28749589
http://www.dpreview.com/forums/post/28750076

The miraculous is the way you gather all this info from a Canon boby in a single file !!!.

It's the best Canon users could hope .. a clear increase of at least 2-3 stops in DR. And this makes Canons Isoless cameras !!.

Can't keep waiting to learn what is this "small ADTG trick".


Redrocks

  • Member
  • ***
  • Posts: 151
  • 5D2 Windows
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #18 on: July 15, 2013, 01:35:15 AM »
what is this "small ADTG trick".

A1ex Delivers The Goodshit.

Audionut

  • Moderators
  • Hero Member
  • *****
  • Posts: 3310
  • I have a 1Dx, I'm awesome.
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #19 on: July 15, 2013, 02:42:13 AM »
It's the best Canon users could hope .. a clear increase of at least 2-3 stops in DR.

3 stops should be the expected increase in DR.

Quote from: a1ex
In particular, with the 5D Mark III, at ISO 100 you get 10.97 stops of dynamic range, and at
ISO 1600 you get 9.94 stops according to DxO [5]. When raising the ISO from 100 to 1600, you
are shifting the dynamic range towards the shadows by 4 stops and shortening it by 1 stop. In
other words, you throw away 4 stops of highlight data (they get clipped), and you get in return
3 stops of shadow detail.

Greg

  • Senior
  • ****
  • Posts: 478
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #20 on: July 15, 2013, 02:53:34 AM »
16 bit dng  :o


Audionut

  • Moderators
  • Hero Member
  • *****
  • Posts: 3310
  • I have a 1Dx, I'm awesome.
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #21 on: July 15, 2013, 08:03:26 AM »
3 stops should be the expected increase in DR.

I was way off the mark.

16 bit dng  :o

Indeed.

This isn't the exact process used by the code from a1ex, but should be a good representation of what to expect.

Single ISO 100 shot.


Enfused ISO 100 and ISO 1600 shots.


Of course, when a1ex lets us play with his code, it won't take 2 different shots to produce the enfused image above :)

edit:  And some comparisons of extreme shadow areas.

ISO 100 with +4 EV in post.


Enfused.



And some examples of not so extreme shadow areas.

ISO 100 with +4 EV in post.


Enfused.
« Last Edit: July 15, 2013, 08:30:56 AM by Audionut »

brapodam

  • Member
  • ***
  • Posts: 121
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #22 on: July 15, 2013, 09:04:20 AM »
I tried running the script and while I got the output files, opening them produces this error message: Could not load image 'imagename.jpg". Error interpreting JPEG image file (Suspension not allowed here)

Anyone knows what is going on? Screenshot:


The errors that appear in the command prompt seem to be related to EXIF info, which shouldn't affect the image
« Last Edit: July 15, 2013, 09:06:54 AM by brapodam »

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 8749
  • 5D Mark Free
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #23 on: July 15, 2013, 09:08:30 AM »
Can you upload dev.log and maybe try to open the intermediate files from tmp?

ImageMagick gave some warnings, but the exposure levels seem to be recognized. Probably something with enfuse (try running it manually on the images from tmp).

IliasG

  • Member
  • ***
  • Posts: 124
Re: Automatic color grading script for photos (uniform_exposure.py)
« Reply #24 on: July 15, 2013, 09:37:52 AM »
3 stops should be the expected increase in DR.

As Alex wrote, 3 stops is expected for 5DIII. It depends on the model .. for 50D using ISOs range 100-1600 DR increase is expected at a bit more than 2 stops.
It also depends on the ISOs used .. On 5DIII ISOs range 200 - 1600 will give 2 stops 100-3200 will give a bit more than 3 stops ..

 

courtesy