Automatic color grading script for photos (uniform_exposure.py)

[a1ex]

Most of the time I'm too lazy to adjust colors in my photos, so I wrote a script that does this exactly the way I want. It gets roughly 80-90% of pics just right for my taste; for the others I can do manual adjustments.

The script is a few months old, so it was fine-tuned over the time. It's similar to the "Post Deflicker" feature from ML, but optimized for regular photos, not for timelapse.

What it does:
- applies exposure compensation to all photos (so the end result looks consistent)
- performs shadow and highlight recovery, with the "HDR from a single RAW" tonemapping trick and enfuse

Download: uniform_exposure.py or browse the Mercurial repository.

Usage: look in the comments. There are no command-line params; edit the script to tweak stuff.

Dependencies: python, numpy, dcraw, ufraw, enfuse and ImageMagick.

Sample photo developed with the script, from this DNG, without any post-processing after running the script:



How I've got that DNG, I'll tell you later these days.

Caveats:
- By default it develops at half-res, for speed. To use full resolution, set --shrink=1 instead of 2.
- I use a modified ufraw, so you may not get exactly the same result for this sample DNG.
- I didn't try it for timelapse. It may flicker a bit more than ML post deflicker, but you can try it.
- Don't expect it to be a speed demon. It will spend several seconds on a raw file. So, first select your shots that you want to keep, then fire the script and let it crunch your pics.
- For Windows, see this step-by-step guide.

Tips & tricks:
- To re-develop a picture with different settings, just delete that jpeg and run the script again.
- You can use any kind of raw files recognized by dcraw (not just Canon). I've tried only CR2 and DNG.

You can post sample pics here, I'd like to see how well it works for you.

Enjoy!

[escho]

great result, Alex.

If you enable the hot pixel reduction in ufraw with a sensitivity of about 10, than the 2 hot pixels on the lower right side will go, too, I think.

Edgar

[Audionut]

Any idea what's happening here on a win x64 machine?

Code Select Expand

raw\D46A7239.CR2:
Invalid Parameter - -type
C:\Python27\lib\site-packages\numpy\core\_methods.py:57: RuntimeWarning: invalid
value encountered in double_scalars
  ret = ret / float(rcount)

[Edgar Matos]

Seems promising.

0_0 Testing mode_activated

[Sijimo]

Cool!

[a1ex]

Another sample, this time a video frame from 5D3:



(bottom left is ISO 100, top right is with a small ADTG trick, at the same shutter speed and aperture; both graded with the above script so I could compare them)

DNG

Still fine-tuning the details.

[Audionut]

top right is with a small ADTG trick

Makes a massive difference.

[swinxx]

that looks really great

is it possible that i can test this with my 5dmk3...?

greets. swinxx

[Greg]

  it looks great!

[aaphotog]

Another sample, this time a video frame from 5D3:



(bottom left is ISO 100, top right is with a small ADTG trick, at the same shutter speed and aperture; both graded with the above script so I could compare them)

DNG

Still fine-tuning the details.

ISP ONE HUNDRED... Is the noisy one?
How'd it get so noisy?
Pitch black room, you got it that bright in post?

[Greg]

It looks brighter ISO 100 with 5EV 

[swinxx]

hello,

the question should not be: how did you get so noisy? ..

the question should be: how did you remove so much noise with your next masterpiece of code..

i am thrilled!!!!
++++ great job

[a1ex]

Easy, with that theory that almost nobody believed: ISO 1600 is less noisy than ISO 100.



Something along these lines: http://www.guillermoluijk.com/article/nonoise/index_en.htm

[Audionut]

Easy, with that theory that almost nobody believed: ISO 1600 is less noisy than ISO 100.

Lol.  Pretty tricky being able to keep the lighter bits at ISO 100!

More exciting code from the Dev team 

[Greg]

It requires a few photos?

[a1ex]

No, just one.

[pascal]

You strive for perfection. I like that. I don't understand the tools process though. 

[IliasG]

Easy, with that theory that almost nobody believed: ISO 1600 is less noisy than ISO 100.



Something along these lines: http://www.guillermoluijk.com/article/nonoise/index_en.htm

Well, that theory is pretty established, even Dr Emil Martinec made an experiment-presentation using the same idea you use
http://www.dpreview.com/forums/post/28749589
http://www.dpreview.com/forums/post/28750076

The miraculous is the way you gather all this info from a Canon boby in a single file !!!.

It's the best Canon users could hope .. a clear increase of at least 2-3 stops in DR. And this makes Canons Isoless cameras !!.

Can't keep waiting to learn what is this "small ADTG trick".

[Redrocks]

what is this "small ADTG trick".

A1ex Delivers The Goodshit.

[Audionut]

It's the best Canon users could hope .. a clear increase of at least 2-3 stops in DR.

3 stops should be the expected increase in DR.

In particular, with the 5D Mark III, at ISO 100 you get 10.97 stops of dynamic range, and at
ISO 1600 you get 9.94 stops according to DxO [5]. When raising the ISO from 100 to 1600, you
are shifting the dynamic range towards the shadows by 4 stops and shortening it by 1 stop. In
other words, you throw away 4 stops of highlight data (they get clipped), and you get in return
3 stops of shadow detail.

[Greg]

16 bit dng 

[Audionut]

3 stops should be the expected increase in DR.

I was way off the mark.

16 bit dng 

Indeed.

This isn't the exact process used by the code from a1ex, but should be a good representation of what to expect.

Single ISO 100 shot.


Enfused ISO 100 and ISO 1600 shots.


Of course, when a1ex lets us play with his code, it won't take 2 different shots to produce the enfused image above

edit:  And some comparisons of extreme shadow areas.

ISO 100 with +4 EV in post.


Enfused.



And some examples of not so extreme shadow areas.

ISO 100 with +4 EV in post.


Enfused.

[brapodam]

I tried running the script and while I got the output files, opening them produces this error message: Could not load image 'imagename.jpg". Error interpreting JPEG image file (Suspension not allowed here)

Anyone knows what is going on? Screenshot:


The errors that appear in the command prompt seem to be related to EXIF info, which shouldn't affect the image

[a1ex]

Can you upload dev.log and maybe try to open the intermediate files from tmp?

ImageMagick gave some warnings, but the exposure levels seem to be recognized. Probably something with enfuse (try running it manually on the images from tmp).

[IliasG]

3 stops should be the expected increase in DR.

As Alex wrote, 3 stops is expected for 5DIII. It depends on the model .. for 50D using ISOs range 100-1600 DR increase is expected at a bit more than 2 stops.
It also depends on the ISOs used .. On 5DIII ISOs range 200 - 1600 will give 2 stops 100-3200 will give a bit more than 3 stops ..