Regarding aliasing in videos:
Here's a very simple octave script that only works on
static scenes (with nearly no motion) with
crawling artifacts. It's based on an
older idea from stevefal, and seems to work quite well on some of the clips posted lately (as long as the motion from one frame to another is next to none).
You'll now see with your own eyes what exactly a "slight temporal artifact" is.
This script also serves as a minimal proof of concept code for Dual ISO blending (useful for understanding how the algorithm works, without having to dig through thousands of lines of code). In other words, this script is very useful as documentation. It operates on unprocessed 14-bit .dng frames (as extracted with mlv_dump), but requires some metadata from cr2hdr logs.
# file name prefix
# M26-1444-30-frames_000000.dng -> prefix = "M26-1444-30-frames_"
prefix = "M26-1444-30-frames_"
# "crawling" phase - possible values: 0, 1, 2, 3
# from ISO pattern on the first rendered frame (cr2hdr log), or trial and error:
# dBBd = 0
# ddBB = 1
# BddB = 2
# BBdd = 3
p = 3
# hardcoded - copy from cr2hdr log
black = 2047 # black level (14-bit)
white = 15000 # white level (14-bit), can be approximate, must be slightly lower than actual clipping point
gain = 15.3 # from "Linear fit" messages - gain part (actually ISO ratio)
offset = 24 # from "Linear fit" messages - offset part, invert sign
black_adj = 3 # fine-tune until blacks look right (fixme: why is it needed?)
# from start+1 to end-1 (frame numbers to process)
# we need one frame before and one after
for i = 1:29
# fixme: read each image only once
a = read_raw(sprintf("%s%06d.dng", prefix, i-1));
b = read_raw(sprintf("%s%06d.dng", prefix, i));
c = read_raw(sprintf("%s%06d.dng", prefix, i+1));
# print the pattern for current frame (cross-check with cr2hdr log)
# for example: cr2hdr *.dng | grep pattern
# cross-check with: octave crawl.m | grep pattern
iso_pattern = circshift("dBBd", [0 p])
# rebuild hi-iso data for frame b, taking bright fields from frames a and c
# fixme: this only works when there's little or no motion between the frames
# this recovers the full resolution from perfectly static (!) scenes
# but introduces massive aliasing whenever something moves
hi = b;
hi(mod(p+0,4)+1:4:end,:) = a(mod(p+0,4)+1:4:end,:);
hi(mod(p+3,4)+1:4:end,:) = c(mod(p+3,4)+1:4:end,:);
# rebuild lo-iso data for frame b, taking dark fields from frames a and c
lo = b;
lo(mod(p+1,4)+1:4:end,:) = c(mod(p+1,4)+1:4:end,:);
lo(mod(p+2,4)+1:4:end,:) = a(mod(p+2,4)+1:4:end,:);
# cancel black delta (see "Find black delta..." in cr2hdr.c)
# fixme: incorrect offset scaling in cr2hdr logs
black_delta = 2 * offset * 16 / (gain + 1)
hi -= black_delta / 2;
lo += black_delta / 2;
# remove black level, match image brightness
# (darken hi to match lo)
hi -= black;
lo -= black;
hi /= gain;
# approximate dual iso blending (smooth transition from lo-iso to hi-iso)
# fixme: compute noise profiles and use Kalman optimal averaging formula
T = @(x) min(max((x - 3000) / 8000, 0), 1);
F = @(x) ((sin((x-0.5)*pi)+1)/2).^4;
f = F(T(hi .* gain));
x = lo .* f + hi .* (1-f);
plot(1:15000, F(T(1:15000)));
# restore the black level
x += black;
# fine-tune black level
x += black_adj;
# check image levels
percentiles = [10 50 90 99 99.9 99.99 99.999];
levels = round(prctile(x(:), percentiles))';
printf("%g\t", percentiles); printf("\n");
printf("%d\t", levels); printf("\n");
# output as PGM -> DNG
# todo: clean way to save a DNG from octave
# pgm2dng: http://a1ex.magiclantern.fm/bleeding-edge/pgm2dng.c
# note: pgm2dng hardcodes a black level of 2048 and a white level of 16382
# multiply 14-bit image and black/white levels by 4 to get 16-bit output
out = sprintf("frame%d.pgm", i);
imwrite(uint16(x * 4), out);
system(sprintf("pgm2dng %s", out));
system(sprintf("exiftool %s -BlackLevel=%d -WhiteLevel=%d -overwrite_original", [out(1:end-4) ".DNG"], black * 4, white * 4));
# update the phase for next frame
# todo: is the "crawling" always going in this direction?
p = mod(p+1, 4);
end
Caveat: it
requires tuning for every single clip (mostly copying stuff from cr2hdr log). It is currently tuned for M26-1444-30-frames.MLV (linked earlier). You'll also need
pgm2dng (a very hackish tool I use for experiments, to get DNGs out of octave).
This also gives fairly good results on the clip from 70MM13 (p=0, gain=7.86, offset=10.61).
Possible improvements:
- find out where there is motion (how? aliasing may easily confuse any algorithm I can think of) and use regular interpolation in these areas.
- apply motion compensation (optical flow) and interpolate only the missing bits (difficult)
Have fun!
