Post Deflicker

[Audionut]

Deflicking is a process used to brightness match images.  This is useful in video, timelapses, or simply brightness matching multiple images (photos).

You may find this module useful for things like flicker free timelapse.


Loading the module

To use this feature, you must first load the module from the module menu.




After restarting the camera, the module is now available in the shoot menu.




Using the module

First, lets look at the options.




Sidecar file type:

• UFRaw:  Use sidecar files suitable for use in the open sourced UFRaw image processor.
• Adobe XMP:  Use sidecar files suitable for use in Adobe products.

Deflicker percentile:

• Where to meter for for deflicking.
• What percentage of brightness should be used for the brightness matching.

Deflicker target level:

• Desired exposure level for processed pics.
• Where to set the processed exposure level in raw EV.

[simonleung]

hello ,

where can i download the module ?

thx

[Audionut]

http://www.magiclantern.fm/forum/index.php?topic=12713.0

[c_joerg]

Interesting idea to create the XMP files directly in the camera. I have done something similar so far.
https://chdk.setepontos.com/index.php?topic=12790.0

I have been dealing with the topic for some time and see two disadvantages with this method:

1) If you determine the corrections in the post-processing, then you can also use selected areas of the image. If, for example, there is turbulent water in the foreground then it is not advisable to use these areas for deflickering because they create new flickering.

2) For the determination of a smooth background curve, it is very helpful if you can already include exposure values from the future.

So far I have always used the Exposure2012 values in the XMP file for deflickering. LRTimelapse uses the LocalExposure2012. There are two advantages to using LocalExposure2012:

1) You can still change the exposure in Lightroom without affecting the deflickering (No need of exiftool).

2) Lightroom apparently only uses 2 decimal places for the Exposure2012 no matter how many digits are entered in the XMP file. In the case of LocalExposure2012, all decimal places are apparently used.

The difference is not big but for the deflickern in many situations very helpful. The following example video shows how great the flicker is when the background changes by 0.04EV. Even 0.01EV are still to see. Therefore, it is already an advantage if you can do the deflickern with one decimal place more.

Here is an example of an entry from LRTimelapse. The gradients are chosen so that the slope probably has no influence. What I have not understood yet is that at LocalExposure2012, I only need to use a quarter of the Exposure2012 values.

Code: [Select]

   <crs:GradientBasedCorrections>
    <rdf:Seq>
     <rdf:li>
      <rdf:Description
       crs:What="Correction"
       crs:CorrectionAmount="1.000000"
       crs:CorrectionActive="true"
       crs:LocalExposure2012="-0.000001">
      <crs:CorrectionMasks>
       <rdf:Seq>
        <rdf:li
         crs:What="Mask/Gradient"
         crs:MaskValue="1.000000"
         crs:ZeroX="0.5"
         crs:ZeroY="-0.31"
         crs:FullX="0.5"
         crs:FullY="-0.3"/>         
       </rdf:Seq>
      </crs:CorrectionMasks>
      </rdf:Description>
     </rdf:li>
    </rdf:Seq>
   </crs:GradientBasedCorrections>

[a1ex]

Sorry, I don't use the XMP workflow, so cannot test the proposed changes. Would replacing crs:Exposure2012 with crs:LocalExposure2012 in Post Deflicker XMPs do the trick, or it requires some more?

Have a sample image where the moving subject (in your case, turbulent water) could not be avoided by adjusting the percentile? Have a short clip (30 frames or so) where metering on turbulent water creates flicker? (just curious, would like to play with such a DNG or CR2 sequence).

Indeed, the best place for deflickering is at post-processing stage (as you can adjust settings and repeat as needed); this in-camera implementation is more like a proof of concept. Back then, there were many examples showing strong flicker in LRTimelapse with ML intervalometer (even with the old bulb ramping implementation). I have no idea what algorithm LRTimelapse uses, or whether the issue was fixed in later versions, but I've got good results with median (or some other percentile) instead of plain average (percentiles are robust statistics, while average is not). The easiest way to fix (without having to write a cross-platform postprocessing tool from scratch, interfacing with a proprietary software I wasn't familiar with) was to save the exposure compensation values from camera and let the user decide later whether to use them or not. Since then, the algorithm was borrowed by other postprocessing tools (Bridge deflicker script, darktable, MLVFS, mlv_dump on steroids).

See also: https://www.magiclantern.fm/forum/index.php?topic=10496.0

[c_joerg]

Sorry, I don't use the XMP workflow, so cannot test the proposed changes. Would replacing crs:Exposure2012 with crs:LocalExposure2012 in Post Deflicker XMPs do the trick, or it requires some more?

It’s not enough just to change crs:Exposure2012 to crs:LocalExposure2012.
The full ‘GradientBasedCorrections’ block has to be used (code block from Reply #3)
And you have to divide the value by 4 (that is what i have not understood yet).

Have a sample image where the moving subject (in your case, turbulent water) could not be avoided by adjusting the percentile? Have a short clip (30 frames or so) where metering on turbulent water creates flicker? (just curious, would like to play with such a DNG or CR2 sequence).

I have scenes in which the problem occurs. For example, here at the end 1:10

If you would take the whole picture to Deflickern, then the sky begins to flicker at the end. In this case you should only use the sky to deflicker. With LRTimelapse you can specify an area in the image to calculate.

I'm not sure if you can do anything with my CR2 files. They come from a Canon G1x or S110. In addition, the exposure with CHDK was continuously changed in 1 / 96EV steps. In the EXIF data only the rounded 1/3EV values can be found. With my 6D I haven’t done any timelapse yet.

Indeed, the best place for deflickering is at post-processing stage (as you can adjust settings and repeat as needed); this in-camera implementation is more like a proof of concept.

I agree. That's the best way.
Even if the way in the camera is easier for the user.

Back then, there were many examples showing strong flicker in LRTimelapse with ML intervalometer (even with the old bulb ramping implementation).

Yes I have seen them. Could be a problem with wrong EXIF values? LRTimelapse need them correct. If there are not correct LRTimelapse get confused. I have the same problem with CR2 from CHDK.

I have no idea what algorithm LRTimelapse uses

I can now pretty much follow the results of LRTimelapse …

or whether the issue was fixed in later versions,

In that case only the visual deflicker workflow helps in LRTimelapse. This works only on calculated exposure values from RAW.


 

but I've got good results with median (or some other percentile) instead of plain average (percentiles are robust statistics, while average is not).

I do both in my post-processing workflow (median and average).

This is my workflow:
- calculating exposure values from a specific area in RAW Data.
- running a median over the values to remove outlier
- running an average to smooth background cure.
- calculate differences from background

[a1ex]

Can you share a CR2 from the youtube clip, or maybe a few consecutive flickering ones? As long as the raw data can be extracted, it should be fine. I'd expect ML deflicker algorithm to handle this particular case out of the box (as it would meter on the ground, not on the water), just want to double-check.

- calculating exposure values from a specific area in RAW Data.

There you should use median, not average. Even if it's a mostly flat area, a single specular highlight (just an example) will bias the average, but will have no effect on the median.

In other words, median (over image pixels) gives robust results when measuring exposure.

- running a median over the values to remove outlier
- running an average to smooth background cure.

I'm not talking about median on temporal axis; there you can ramp the exposure values as you wish, as long as you don't remove the high-frequency components (required for deflickering).

Applying average or median on the temporal axis (on the exposure corrections array) is likely to introduce flicker (at least that's what I'd expect to happen).

[c_joerg]

Can you share a CR2 from the youtube clip, or maybe a few consecutive flickering ones?

Yes of cource. I try to upload on my google drive tonight…

There you should use median, not average. Even if it's a mostly flat area, a single specular highlight (just an example) will bias the average, but will have no effect on the median.

Understood. I have not thought about that yet. Here I have always used the mean value.

I'm not talking about median on temporal axis; there you can ramp the exposure values as you wish, as long as you don't remove the high-frequency components (required for deflickering).

 
Just for understanding:
- The purple curve are the exposure values from RAW (selected area)
- The green curve is the calulatet background curve
(For that curve over time I use median and average. LRTimlapse use only average.)
- The red curve is the delta from green and purple curve. The deflicker values which comes into LocalExposure2012.




img upload

[c_joerg]

Unfortunately, that was the only one Scene I have only in JPG...

Enclosed the last 80 files of a scene from this YouTube (1:40) video.

Furthermore, there is a text file with file number and EV values in the ZIP file.
Tell me if you need more files… upload takes a while…

https://drive.google.com/open?id=1mmd-aIvkpbSYKLeRcRyCECtPq_c0F2I3

On the plot you can see how the flicker increase when the water goes higher…






Update Second link
https://drive.google.com/open?id=1QA78WXg5guVBJluY8EfhO-aeW4I2VM2m

[a1ex]

Ran this deflicker script modified as follows:

Code: [Select]

# dcraw -v IMG_0951.CR2
black = 512

# value not critical
white = 16383

# EV above black (exposure compensation)
# 0 EV = 1 DN
target_median = 11

for f = dir('*.CR2')'
    f.name
    im = read_raw(f.name);
    g1 = im(1:2:end,2:2:end);
    med_g1 = median(g1(:));
    med_ev = log2(med_g1 - black);
    ec_ev = target_median - med_ev;
    ec_linear = 2 ^ ec_ev;
    ec_ev
    system(sprintf('dcraw -h -W -b %f %s ', ec_linear, f.name));
end

# debug: where's the median in the last image?

# show 12 stops logarithmically
g1log = (log2(max(1, g1 - black)) - log2(white - black) + 12) * 255 / 12;

# mark the median with a red overlay
ovr(:,:,1) = ovr(:,:,2) = ovr(:,:,3) = g1log;
red = ovr(:,:,1); green = ovr(:,:,2); blue = ovr(:,:,3);
red  (g1 > med_g1 - 20 & g1 < med_g1 + 20) = 255;
green(g1 > med_g1 - 20 & g1 < med_g1 + 20) = 0;
blue (g1 > med_g1 - 20 & g1 < med_g1 + 20) = 0;
ovr(:,:,1) = red; ovr(:,:,2) = green; ovr(:,:,3) = blue;
imwrite(uint8(ovr), 'median.jpg');

# test with:
# ffplay -loop 0 IMG_09%02d.ppm

Result: only some minor flicker in the sky (when metering on the turbulent water). Couldn't locate the scene in the youtube clip to compare. Very interesting phenomenon, btw!

deflick-median.avi (metered on the water)

If metering on the sky, there's no more flicker, of course. Choosing the percentile can usually be made before recording the timelapse, as you know in advance what is expected to change brightness and what is expected to remain constant. The UI is not the best though - you can see some percentiles (in particular, median) with the Auto ETTR debug screens.

Code: [Select]

prctile(g1(:), 90) %  instead of median(g1(:))

deflick-p90.avi (metered at the 90th percentile - on the sky)

BTW - in this particular case, plain averaging (mean) gives very good results, too:
deflick-mean.avi

[c_joerg]

Ran this deflicker script modified as follows:

How do you run the code?
Looks very similar to my ‘MATLAB’ Code but it’s not ‘MATLAB’.
Do you have a code like this for focus peaking?

Code: [Select]

# dcraw -v IMG_0951.CR2
black = 512

# value not critical
white = 16383
Have you tried to get white and black with EXIFTOOL from CR2? I have only found black values.

Result: only some minor flicker in the sky (when metering on the turbulent water). Couldn't locate the scene in the youtube clip to compare. Very interesting phenomenon, btw!

OK sorry for that.
This video has very much post-processing to get the turbulent water shooth. As soon as the water touches the feet of the tripod, it will vibrate. If you want I can give you the original video.

Just a short overview about post-processing:
- Deflicker with LocalExposure2012 in Lightroom
- images aligned (image registration).
- median on 3 images to eliminate birds and individual events.
- smooth by averaging several pictures (motion blur).

Videos without lens correction from P&S looks awful….   

Code: [Select]

prctile (g1(:), 90) %  instead of median(g1(:))

I have not really understood what that has an impact if I use prctile instead of median…

in this particular case, plain averaging (mean) gives very good results, too:

So you think in that case it make no different if I just use the sky or the full scene with mean?

[a1ex]

Click on the links to find the answers. For example, to find where the 90th percentile falls in your image, click on this:

metered at the 90th percentile - on the sky

Generally, when you see something blue underlined on the web, it's clickable.

The red "line" is where the 90th percentile falls in your image (in other words, 90% of pixels are darker than that and 10% are brighter). If this falls onto a flat area (such as the sky), that's a good choice for deflickering.

If there aren't any outliers to bias the mean, it works just fine. Trimmean mean might actually be better than both (it's both robust - not affected by extreme values - and averages the middle values, so the result has good precision). May require some trickier scenes to see the difference.

[c_joerg]

The red "line" is where the 90th percentile falls in your image (in other words, 90% of pixels are darker than that and 10% are brighter). If this falls onto a flat area (such as the sky), that's a good choice for deflickering.

Ok, I understood that now…

I have now examined the entire scene. Once the complete picture and then only the sky.
Each mean, median and 90% percentile. Curves are normalizes by an offset.

My result:
If you use only the sky, then there are no differences between mean, median and 90% percentile. Also the 90% percentile for the full picture makes no different (of course the data comes from sky only).
If you use mean or median for the full picture you get some errors (Around picture 914). Not much (around 0.05EV) but enough to see it on a clear sky. I mean I would have scene where it was clearer to see. This video shows flicker from 0.04EV

.

I still think that if you have the opportunity to choose a sector (Like in LRTimelapse) that is the safest method.

Code: [Select]

im = read_raw(f.name); Where can I find the implementation from ‘read_raw’ ?

I’m doing it in the following way. Converting in a linear TIF

Code: [Select]

dcraw -4 -D –T And than I open the TIF.





 

[a1ex]

Ran this deflicker script [...]

Click on that link to find out the programming language and other requirements, including read_raw. It's pretty much the same as you did (dcraw -4 -D).