Dual ISO - massive dynamic range improvement (dual_iso.mo)

Started by a1ex, July 16, 2013, 06:33:50 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

DeafEyeJedi

Quote from: Danne on August 04, 2017, 05:04:26 PM
Hm, really got to dive into lua. I´d like to build a script that records 2 seconds of MLV of every iso on the camera and when done exits the scripts. I´d use the files for darkframe storage workflows.

+100
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

DeafEyeJedi

Sorry guys tp be a party pooper once again but for some reason I'm having issues trying to get cr2hdr to respond properly to 12/14-bit lossless shot in Dual-ISO.

At first I thought it was an issue directly to the experimental build for SL1/100D but turns out it may be for all cameras capable of recording Lossless compression.

Correct me if I'm wrong but either way here are the sample files from SL1/100D for those who are curious.

14-bit (non-lossless):

1) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%231_SL1_14-bit_1728x972_samples.zip

2) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%232_SL1_14-bit_DUAL-ISO_1728x972_samples.zip

10-bit (non-lossless):

3) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%233_SL1_10-bit_1728x972_samples.zip

10-bit (non-lossless) Dual-ISO:

4) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%234_SL1_10-bit_DUAL-ISO_1728x972_samples.zip

5) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%235_SL1_10-bit_1632x552(60p)_DUAL-ISO_samples.zip

14-bit (Lossless) Dual-ISO:

6) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/%236_SL1_14-bit_Lossless_DUAL-ISO_1736x976_samples.zip

12-bit (Lossless) Dual-ISO:

7) https://bitbucket.org/DeafEyeJedi/magic-lantern/downloads/SL1_12-bit_lossless_Dual-ISO_1800x1008p_Sample.zip

Here's another post from earlier with three more samples: https://www.magiclantern.fm/forum/index.php?topic=16040.msg190646#msg190646
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

smasry

I only recently discovered dual iso, and as you've all been saying for years, it's nothing short of amazing.

My trouble is that in every video I shoot, there's loads of flickering - the exposure seems to change noticeably from frame to frame, making the video seem to 'flicker' when played at normal speed. I've read the entire thread, and haven't been able to find a working solution, apart from faulty black and white levels, which should be fixed by exiftool. I tested this, reading a few of the (cr2hdr) processed DNGs using exiftool, and can confirm that the white and black levels remain constant across frames, including those exhibiting the apparent change in exposure, at 8192 and 65128, respectively.

I suppose this can be changed by constantly altering the exposure in post, but it will be very laborious to have to do it throughout the length of each video, so I'm hoping the latest cr2hdr has an improved algorithm to detect these variations.

All footage is shot on a 5D3, on the October 27 4k_lossless branch build. At first, I processed it with Switch, then to confirm with the mlv_dump + cr2hdr combination, as documented on the original post in this thread, using the --same-levels argument.

I'm on macOS Sierra (10.12.6), and have tried the cr2hdr in the OP, which is quite old, if I remember, as well as the one from Danne's Switch, which is:

Last update: ab1e90c on 2015-11-24 09:50:19 UTC by a1ex:
cr2hdr: moved safeguard from median_int_wirth to kth_smallest_i..


As far as I can tell, there have been a number of changes to cr2hdr.c in the repo, which I've tried building, but end up with build errors:


make cr2hdr
/bin/sh: /Volumes/Data HD/Users/sacha/gcc-arm-none-eabi-4_8-2013q4/bin/arm-none-eabi-gcc-4.8.3: No such file or directory
[ gcc      ]   cr2hdr
cr2hdr.c:59:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
MODULE_STRINGS()
^
cr2hdr.c:59:17: error: expected ';' after top level declarator
MODULE_STRINGS()
                ^
                ;
cr2hdr.c:573:51: error: use of undeclared identifier 'dual_iso_strings'
    printf("Last update: %s\n", module_get_string(dual_iso_strings, "Last update"));
                                                  ^
cr2hdr.c:1334:46: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist rggb.m"));
                                             ^
cr2hdr.c:1334:46: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:1472:46: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist bddb.m"));
                                             ^
cr2hdr.c:1472:46: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:1748:51: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist iso-curve.m"));
                                                  ^
cr2hdr.c:1748:51: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:2184:55: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist fullres-curve.m"));
                                                      ^
cr2hdr.c:2184:55: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:2518:56: warning: if statement has empty body [-Wempty-body]
            if(system("dcraw -d -r 1 1 1 1 edges.dng"));
                                                       ^
cr2hdr.c:2518:56: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:2531:59: warning: if statement has empty body [-Wempty-body]
            if(system("dcraw -d -r 1 1 1 1 edge-map.dng"));
                                                          ^
cr2hdr.c:2531:59: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:2557:21: error: function definition is not allowed here
                    {
                    ^
cr2hdr.c:2571:31: warning: implicit declaration of function 'edge_interp' is invalid in C99 [-Wimplicit-function-declaration]
                    int pi0 = edge_interp(dir);
                              ^
cr2hdr.c:2824:51: warning: if statement has empty body [-Wempty-body]
        if(system("octave --persist mix-curve.m"));
                                                  ^
cr2hdr.c:2824:51: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:3308:55: warning: if statement has empty body [-Wempty-body]
            if(system("octave --persist soft-film.m"));
                                                      ^
cr2hdr.c:3308:55: note: put the semicolon on a separate line to silence this warning
cr2hdr.c:3652:34: warning: if statement has empty body [-Wempty-body]
        if(system("octave wb.m"));
                                 ^
cr2hdr.c:3652:34: note: put the semicolon on a separate line to silence this warning
11 warnings and 3 errors generated.
dcraw-bridge.c:101:3: warning: unused variable 'unique' [-Wunused-const-variable]
} unique[] = {
  ^
1 warning generated.
amaze_demosaic_RT.c:983:10: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value
      [-Wabsolute-value]
                                        if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;}//a better result was obtained from the neighbors
                                            ^
amaze_demosaic_RT.c:983:10: note: use function 'fabs' instead
                                        if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;}//a better result was obtained from the neighbors
                                            ^~~~~
                                            fabs
amaze_demosaic_RT.c:983:35: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value
      [-Wabsolute-value]
                                        if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;}//a better result was obtained from the neighbors
                                                                     ^
amaze_demosaic_RT.c:983:35: note: use function 'fabs' instead
                                        if (fabsf(0.5-hvwt[indx>>1])<fabsf(0.5-hvwtalt)) {hvwt[indx>>1]=hvwtalt;}//a better result was obtained from the neighbors
                                                                     ^~~~~
                                                                     fabs
amaze_demosaic_RT.c:1215:10: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value
      [-Wabsolute-value]
                                        if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
                                            ^
amaze_demosaic_RT.c:1215:10: note: use function 'fabs' instead
                                        if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
                                            ^~~~~
                                            fabs
amaze_demosaic_RT.c:1215:35: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value
      [-Wabsolute-value]
                                        if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
                                                                     ^
amaze_demosaic_RT.c:1215:35: note: use function 'fabs' instead
                                        if (fabsf(0.5-pmwt[indx>>1])<fabsf(0.5-hvwt[indx>>1]) )
                                                                     ^~~~~
                                                                     fabs
4 warnings generated.
make: *** [cr2hdr] Error 1


Can anyone who's been able to build on the mac help with either the errors, or to share their build environment settings/dependencies? I have installed dcraw and exiftool, and Python's docutils (someone's build failed because of this dependency).

Danne

If developing dualiso movie files in Switch the flicker issue should be taken care of by having the same black level added in post.

smasry

Thanks for the reply @Danne, I didn't know this.

I do all the post in Resolve, but have not ever changed the black level; I didn't even know about it. Do I set it to the value reported by exiftool?

Do you know where to set it in Resolve?

Sorry for the silly questions...

Danne

It's inside the dng metadata but it's all automated in Switch from start to finish. Just head over to (m) or (ms) to get to mlv_dump menu and uncheck disable dualiso automation and then run Switch.

smasry

Uncheck dualiso automation? I thought I read that was there to disable dualiso processing, on a folder you know doesn't contain any dual iso MLVs?

Either way, as I wrote, I subsequently processed one MLV manually, using mlv_dump, then cr2hdr manually, as in the original post in this thread, using the --same-levels switch (no puns intended Danne).

With this procedure, I can confirm that white and black levels are stable, across a number of frames, including those where there is an obvious darkening and lightening in exposure - showing up as flicker. That's why I didn't report this as a Switch issue.

So, if the recent (last two years' worth) check-ins to cr2hdr.c have improved the flicker-suppressing algorithm, I'd like to compile and test the latest cr2hdr, but am running into compiling errors (3 posts ago).

Otherwise, it's something I'm doing that's throwing the algorithm off, as no-one else is reporting flickering in all their dual-iso videos.

Can someone help?

a1ex

The commits after the pre-built binary are these, mostly about black level handling. They were not included in the build because they failed my test suite back then (and didn't look closer to narrow down).

I don't currently have a better method for handling videos, other than --same-levels, but a few tools borrowed my deflicker algorithm lately (darktable, MLVFS, mlv_dump on steroids, maybe others) and might be worth trying. Of course, the algorithm is trivial to port into cr2hdr, just didn't look into it yet.

lureb74

Quote from: DeafEyeJedi on September 30, 2017, 12:22:35 AM
Sorry guys tp be a party pooper once again but for some reason I'm having issues trying to get cr2hdr to respond properly to 12/14-bit lossless shot in Dual-ISO.

At first I thought it was an issue directly to the experimental build for SL1/100D but turns out it may be for all cameras capable of recording Lossless compression.

Correct me if I'm wrong but either way here are the sample files from SL1/100D for those who are curious.

.....

I had that problems with some clips shooted in dual-iso 12-bit, and I just find the solution using the "batch_mlv_in_out" (thank guys!!!):

https://www.magiclantern.fm/forum/index.php?topic=10526.0

I find a solution by converting the mlv lossless compressed files (using the mentioned mlv_dump batch plug-in) with option 2 (-d) and 4 (specifying 14 bits) ant then give theese 14bit files to the dual-iso process.

Let me know if that works for you too!!

smasry

Thanks @a1ex, I'll try mlv_dump on steroids. MLVFS seems random to me; sometimes it works, sometimes it crashes under the load, and it's hard to say which it's going to be until you've waited a long time.

I had no idea darktable had this feature, I must investigate. MLV App definitely has it, and it's an elegant and powerful app, but I can't find any documentation about how the flicker suppression works, though their 'Deflicker target' setting clearly makes a difference, making the footage darker overall, but I haven't been able to fix it altogether yet. I assumed it built off cr2hdr.

Danne

Deflicker option will alter only this tag in the dng file:
Baseline Exposure Offset        : 0
It´s usually set to 0 but will vary as it deflicker the footage selecting deflicker option. The changes will take affect in acr/lightroom/AE. I thought it´s main purpose was to use it with timelapse footage, not dualiso files. Now if it was ported and used with white level tag for instance it might work but then again same level tag sets the same white level for all the dualiso files preventing it from flicker so the simpliest solution is to use exiftool/exiv2 in or the same level tag in cr2hdr.
Now I think mlvfs is applying the same white level tag to all movie dualiso files so flicker won´t be an issue there, maybe mlv_dump on steroids is doing that too. As I said already, Switch will correct the white level tag as well as keeping the originally shot white balance along with multiprocessing which makes it fast to work with.

DeafEyeJedi

Quote from: Danne on November 29, 2017, 05:20:22 AM
As I said already, Switch will correct the white level tag as well as keeping the originally shot white balance along with multiprocessing which makes it fast to work with.

Bang on!  ;D
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

smasry

Hi @Danne, @DeafEyeJedi,

I hope neither of you think  I was criticising Switch? It's great, it works well, I can set it and walk away, to come back to processed files. Thank you for this software.

My trouble is the constant flickering - on almost every shot I've taken so far - and this is my search to find ways to fix it.

Using Exiftool (and exiv2), I can't see the Baseline Exposure Offset tag as having been set, on DNG (dual iso) files, processed by either Switch, or the mlv_dump + cr2hdr combination.

This is what I do see happening across 2 frames which exhibit flicker - the first of the two is subtly darker than the second one:







AppFrame 513Frame 514
mlv_dump (unprocessed)BL: 2047, WL: 16200BL: 2047, WL: 16200
cr2hdr (with --same-levels)BL: 8192, WL: 65128BL: 8192, WL: 65128
Switch (with (s) same-levels)BL: 8192, WL: 63768BL: 8192, WL: 63768
MLV App (can't remember deflicker target value)BL: 8188, WL: 64800BL: 8188, WL: 64800

As you can see from the above, all three approaches result in changed Black Level and White Level values, relative to the 'original', unprocessed .dng file (out of mlv_dump), and they all differ from each other. However, they are also all equal across frames, resulting in very visible flickering.

Danne

In Switch you don't set anything when developing dualiso files. Only drop mlv files in a folder and run it. You need uncheck the before mentioned setting in mlv_dump menu for dualiso automation to get activated. This way you will be able to run dualiso and non dualiso mlv files together. Only then you'll get the non flickering files in Switch.
If you are able to set same levels from a menu in Switch you are suppose to work only with CR2 files, different story.

QuoteUsing Exiftool (and exiv2), I can't see the Baseline Exposure Offset tag as having been set, on DNG (dual iso) files, processed by either Switch, or the mlv_dump + cr2hdr combination.
Forget about the Baseline Exposure Offset tag and deflicker target values when processing dualiso files.

smasry

So, I've been doing it wrong all this time, by going into the (hdr?) submenu.

I' try again, exactly as you suggest, by re-enabling dual-iso automation and running it , with no other options.

Thanks @Danne!

Danne


bouncyball

Quote from: Danne on November 29, 2017, 05:20:22 AM
Deflicker option will alter only this tag in the dng file:
Baseline Exposure Offset        : 0
Yes right no actual raw data is altered.

Member of 'raw_info' struct  'exposure_bias[2]' array is set according to a1ex's algorithm and pushed to DNG header. If this dng tag is not interpreted by raw developing software then deflicker effect is not gonna be noticeable.

Edit: this is the case in the MLV App. Only tag is added to DNG header when exporting codec is cdng. If exporting is not cDNG, then 'Baseline Exposure Offset' is not taken in account by proggie's image processing part ( yet ;) )

smasry

Hi @Danne, @bouncyball. I've processed the MLV with Switch again, using no extra flags, and can still 'see' the problem.

On the other hand, what you and @a1ex have said is only now making sense; it's the developing software that I haven't questioned. You're all right that that's where I see the problem manifest itself. In my case, I'm developing these [c]DNG files in Resolve, and that looks like where this (mis)interpretation is happening.

How are the rest of you managing, with dual-iso in Resolve, to hide flickering?

Danne

Could you post examples, printscreens, dng files etc?


a1ex

Sorry, unable to recover the input file from output images...

That said, here's a quick deflicker script in octave, that should work on any DNG sequence (after filling in the correct black/white levels):

# exiftool *.DNG -BlackLevel
black = 8192

# exiftool *.DNG -WhiteLevel
# value not critical; todo: read from exif
white = 60000

# EV above black (exposure compensation)
# 0 EV = 1 DN
target_median = 8

for f = dir('*.DNG')'
    f.name
    im = read_raw(f.name);
    g1 = im(1:2:end,2:2:end);
    med_g1 = median(g1(:));
    med_ev = log2(med_g1 - black);
    ec_ev = target_median - med_ev;
    ec_linear = 2 ^ ec_ev;
    new_white = black + white / ec_linear;
    system(sprintf("exiftool '%s' -overwrite_original -WhiteLevel=%d", f.name, new_white));
end

# debug: where's the median in the last image?

# show 12 stops logarithmically
g1log = (log2(max(1, g1 - black)) - log2(white - black) + 12) * 255 / 12;

# mark the median with a red overlay
ovr(:,:,1) = ovr(:,:,2) = ovr(:,:,3) = g1log;
red = ovr(:,:,1);
red(g1 > med_g1 - 10 & g1 < med_g1 + 10) = 255;
ovr(:,:,1) = red;
imwrite(uint8(ovr), 'median.jpg');

# test with:
# dcraw -W -b 4 *.DNG
# ffplay -loop 0 M26-1444_1_2017-11-26_0001_C0000_%06d.ppm


This adjusts the exposure using the WhiteLevel tag (to my knowledge, all raw converters should respect it).

It deflickers the building (look at median.jpg to check where exactly it meters the exposure on this image), but there's still some flicker in the sky.

The sky flicker goes away by metering on the clouds (e.g. at the 95th percentile):

    p95_g1 = prctile(g1(:), 95);
    p95_ev = log2(p95_g1 - black);
    ec_ev = target_median + 5 - p95_ev;


but then, the building will flicker.

You'll probably need some nonlinear deflicker curve - maybe by blending somehow the two (linear) exposure compensations.

You'll also need read_raw.m:

function im = read_raw(filename)
    dcraw = 'dcraw -c -4 -D';
    system(sprintf('%s "%s" > tmp.pgm', dcraw, filename));
    im = double(imread('tmp.pgm'));
    system('rm tmp.pgm');
end


Results on this image sequence: deflicker.zip (both median and 95th percentile).

Try blending the two - keep the midtones and shadows from the former, and highlights from the latter.

edit: enfuse does the trick :)

combined script (deflickering both midtones and highlights):

# exiftool *.DNG -BlackLevel
black = 8192

# exiftool *.DNG -WhiteLevel
# value not critical; todo: read from exif
white = 60000

# EV above black (exposure compensation)
# 0 EV = 1 DN
target_median = 12
target_highlight = 14

# prepare output directories
# make sure you don't have anything important here!
system("mkdir -p high med out")
system("rm -f high/* med/* out/*")

function adjust_white(filename, dirname, black, white, metered, target)
    ev = log2(metered - black);
    ec_ev = target - ev;
    ec_linear = 2 ^ ec_ev;
    new_white = black + white / ec_linear;
    system(sprintf("exiftool '%s' -Directory='%s' -WhiteLevel=%d ", filename, dirname, new_white));
end

for f = dir('*.DNG')'
    f.name
    im = read_raw(f.name);
    g1 = im(1:2:end,2:2:end);
    med_g1 = median(g1(:));
    p95_g1 = prctile(g1(:), 95);
    adjust_white(f.name, "med", black, white, med_g1, target_median);
    adjust_white(f.name, "high", black, white, p95_g1, target_highlight);

    # render the images
    system(sprintf("dcraw -W 'med/%s' 'high/%s'", f.name, f.name));
    ppm = [f.name(1:end-4) ".ppm"];
    png = [f.name(1:end-4) ".png"];
    system(sprintf("enfuse 'med/%s' 'high/%s' -o 'out/%s'", ppm, ppm, png));
end

# debug: where's the median in the last image?

# show 12 stops logarithmically
g1log = (log2(max(1, g1 - black)) - log2(white - black) + 12) * 255 / 12;

# mark the median with a red overlay
# and the 95% percentile with a cyan overlay
ovr(:,:,1) = ovr(:,:,2) = ovr(:,:,3) = g1log;
red = ovr(:,:,1);
red(g1 > med_g1 - 10 & g1 < med_g1 + 10) = 255;
red(g1 > p95_g1 - 100 & g1 < p95_g1 + 100) = 0;
ovr(:,:,1) = red;
imwrite(uint8(ovr), 'metered-areas.jpg');

# test with:
# ffplay -loop 0 out/M26-1444_1_2017-11-26_0001_C0000_%06d.png


Results from second script: deflicker-enfuse.zip

Metered areas (median and 95th percentile):



Was the input MLV flicker-free before processing? (if so, that must be a bug in cr2hdr)

smasry

Hi @a1ex,

QuoteSorry, unable to recover the input file from output images...

Quite right, here's a cut-down, thus lighter, MLV file:

https://www.rojoynegroclub.com/images/tmp/M26-1444-30-frames.MLV

I'll have to read through your shell script and instructions carefully and test. That and the zip file you sent.

QuoteWas the input MLV flicker-free before processing? (if so, that must be a bug in cr2hdr)

Hard to say, since I apparently have no idea how to monitor it properly; MlRawViewer is old, Footage is, well, dead in the water. In MLV App, my present favourite (thanks for the great software Illia, Masc and company!) it obviously plays back with flickering when I bring up the exposure, which is what 75% of the image needs. If I leave it with no exposure change, the image is quite dark, and it's hard to tell.

Is there a recommended way to preview/monitor the MLV file prior to processing? Sorry if it's an obvious question...


smasry

You're right, the upload failed, sorry. The link works now, though.

Danne

Thanks, works. Interesting as always when bightness shifts and maybe also colors? Hard to tell what´s going on and if anyone knows where to look into this ingenious code it´s a1ex.