Just playing with this dataset and https://github.com/pathak22/pyflow
Quote from: Levas on November 28, 2016, 01:46:10 PM
Do you mean you want to try some super resolution algorithms ?
I have about 45 frames of this castle, before I start panning to the right...
Uploading frame 0 to 45 right now, takes about half an hour.
Same link as before.
http://drive.google.com/drive/folders/0B1BxGc3dfMDaRmtKc2tOa3dHMTA?usp=sharing (http://drive.google.com/drive/folders/0B1BxGc3dfMDaRmtKc2tOa3dHMTA?usp=sharing)
- before: dcraw M27-1337-frame_000002.dng
- after: averaged with frames 1 and 3, warped with optical flow to match frame 2
(http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/M27-1337_frame_000002-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/M27-1337_frame_000002.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/M27-1337_frame_000002-a-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/M27-1337_frame_000002-a.jpg)
To reproduce the above result, get the files below, install the dependencies (follow comments and error messages), then type:
make -j2 M27-1337_frame_000002-a.jpg
or "make -j8" to render the entire sequence on a quad-core processor.
Makefile (use Firefox for copying the text; Google Chrome and Safari
will not work!)
# experiment: attempt to reduce aliasing on hand-held footage using optical flow
# requires https://github.com/pathak22/pyflow
# replace with path to pyflow repository
FLOW=python ~/src/pyflow/flow.py
# default target: render all frames as jpg
all: $(patsubst %.dng,%-a.jpg,$(wildcard M27-1337_frame_*.dng))
# render DNGs with dcraw
%.ppm: %.dng
dcraw $<
# helper to specify dependencies on previous or next image
# assumes the file name pattern is: prefix_000123 (underscore followed by 6 digits)
# fixme: easier way to... increment a number in Makefile?!
inc = $(shell stem="$1"; echo $${stem%_*}_$$(printf "%06d" $$((10\#$${stem//*_/}+$2))) )
# enable secondary expansion (needed below)
.SECONDEXPANSION:
# next or previous frames
%-n.png: %.ppm $$(call inc,%,1).ppm
$(FLOW) $^ $@
%-p.png: %.ppm $$(call inc,%,-1).ppm
$(FLOW) $^ $@
# average
%-a.png: %.ppm %-n.png %-p.png
convert -average $^ $@
# fallback rules: first / last file will only have "next" / "previous" neighbours
# FIXME: these rules may be chosen incorrectly instead of the above in some edge cases; if in doubt, delete them and see if it helps
%-a.png: %.ppm %-n.png
convert -average $^ $@
%-a.png: %.ppm %-p.png
convert -average $^ $@
# convert to jpg
%.jpg: %.ppm
convert $< $@
%.jpg: %.png
convert $< $@
# 100% crops
%-crop.jpg: %.jpg
convert $< -crop 400x300+900+650 $@
# careful if you have other files in this directory ;)
clean:
rm -f *.ppm *.jpg *.png *.npy
# do not delete intermediate files
.SECONDARY:
# example:
# make -j8
# make -j2 M27-1337_frame_000002-a.jpg
flow.py:
# Modified the demo from https://github.com/pathak22/pyflow
# -> just save the warped image and the computed flow; filenames from command line
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
# from __future__ import unicode_literals
import numpy as np
from PIL import Image
import time
import pyflow
import sys
try:
print("%s %s -> %s" % (sys.argv[1], sys.argv[2], sys.argv[3]))
except:
print("usage: %s input1.jpg input2.jpg output.npy" % sys.argv[0])
raise SystemExit
im1 = np.array(Image.open(sys.argv[1]))
im2 = np.array(Image.open(sys.argv[2]))
im1 = im1.astype(float) / 255.
im2 = im2.astype(float) / 255.
# Flow Options:
alpha = 0.012
ratio = 0.75
minWidth = 20
nOuterFPIterations = 7
nInnerFPIterations = 1
nSORIterations = 30
colType = 0 # 0 or default:RGB, 1:GRAY (but pass gray image with shape (h,w,1))
s = time.time()
u, v, im2W = pyflow.coarse2fine_flow(
im1, im2, alpha, ratio, minWidth, nOuterFPIterations, nInnerFPIterations,
nSORIterations, colType)
e = time.time()
print('Time Taken: %.2f seconds for image of size (%d, %d, %d)' % (
e - s, im1.shape[0], im1.shape[1], im1.shape[2]))
flow = np.concatenate((u[..., None], v[..., None]), axis=2)
np.save(sys.argv[3] + ".npy", flow)
import cv2
cv2.imwrite(sys.argv[3], im2W[:, :, ::-1] * 255)
Exercise for the reader: use more frames to compute the correction.
Have fun.
That footage looks familiar ;D
Interesting stuff to experiment with, seems to clear up a lot of the moire.
It seems very promising indeed. Amazing really. Not affecting sharpness. Never seen such results.
How does this affect the parts of the image that are without Moire?
Edit: Just clicked on the pictures.
Quote from: a1ex on November 19, 2017, 07:49:22 PM
- before: dcraw M27-1337-frame_000002.dng
- after: averaged with frames 1 and 3, warped with optical flow to match frame 2
Definitely looks promising. Fun times are ahead!
Trying to duplicate what you did here.
Didn't succeed.
Did this in terminal
git clone https://github.com/pathak22/pyflow.git
cd pyflow/
python setup.py build_ext -i
python demo.py
But with 'python setup.py build_ext -I'
I get the following error:
Quote
Traceback (most recent call last):
File "setup.py", line 8, in <module>
from Cython.Build import cythonize
ImportError: No module named Cython.Build
Did you try:
pip install Cython
You may need to install pip first. There are two ways to do this on Mac (or maybe more):
- sudo easy_install pip (then, sudo pip install Cython)
- without sudo: pip2 after installing homebrew and hg. After running the commands suggested by homebrew: pip2 install Cython; python setup.py build_ext -i
You'll also need PIL (sudo pip install Pillow / pip2 install Pillow), OpenCV (brew install opencv3), dcraw and ImageMagick (brew install dcraw imagemagick). Also requires some fiddling (e.g. copying code doesn't work well from Safari, python modules don't work out of the box - read brew's messages - and so on).
Tested on a Mac VM after installing this (http://www.magiclantern.fm/forum/index.php?topic=16012.msg191686#msg191686). The hardest part is getting used to the Mac UI (e.g. how do you open the Home directory in the file browser? how do you select all the files from a zip in order to drag them?!)
Tip: only experiment with one frame at a time (it's not very fast).
Wow, lots of information.
Ok, installed all the packages, did the following commands in terminal:
Quote
sudo easy_install pip
sudo pip install Cython
cd pyflow/
python setup.py build_ext -I
sudo pip install Pillow
brew install opencv3
brew install dcraw imagemagick
Now when I use the command:
make -j2 M27-1337_frame_000002-a.jpg
I get an error message:
Makefile:12: *** missing separator. Stop.
I'm not sure about my makefile, I assume I needed to make a file in Pyflow directory with the name 'makefile' (no extension) with the code you had in your first post.
To be sure the makefile is not in a wrong text format, rich tekst or something, I copied one from the Magiclantern directory and replaced the tekst with the tekst in your post.
I have Pyflow in my user home directory,
So this alteration should do ?
Quote
# replace with path to pyflow repository
FLOW=python ~/pyflow/Flow.py
Quote from: a1ex on November 20, 2017, 12:19:34 PM
copying code doesn't work well from Safari
Maybe you can report a bug to Apple after you get it working :D
edit: Google Chrome has the same issue, Firefox works.
edit: wget also has the issue, what the...
Read that the first time :P
Thought I got it right with using existing makefile from magic lantern and selecting the lines manually etc...
But now that I have copied the tekst of code with using the quote button on your post, to see plain tekst of your message, it works.
It does a lot, takes about 1 minute to calculate stuff for only averaging frame 2.
But at the end I get an error, no module CV2.
Quote
make -j2 M27-1337_frame_000002-a.jpg
python ~/pyflow/flow.py M27-1337_frame_000002.ppm M27-1337_frame_000003.ppm M27-1337_frame_000002-n.png
python ~/pyflow/flow.py M27-1337_frame_000002.ppm M27-1337_frame_000001.ppm M27-1337_frame_000002-p.png
M27-1337_frame_000002.ppm M27-1337_frame_000003.ppm -> M27-1337_frame_000002-n.png
M27-1337_frame_000002.ppm M27-1337_frame_000001.ppm -> M27-1337_frame_000002-p.png
Constructing pyramid...done!
Constructing pyramid...done!
Pyramid level 14
Pyramid level 14
Pyramid level 13
Pyramid level 13
Pyramid level 12
Pyramid level 12
Pyramid level 11
Pyramid level 11
Pyramid level 10
Pyramid level 10
Pyramid level 9
Pyramid level 9
Pyramid level 8
Pyramid level 8
Pyramid level 7
Pyramid level 7
Pyramid level 6
Pyramid level 6
Pyramid level 5
Pyramid level 5
Pyramid level 4
Pyramid level 4
Pyramid level 3
Pyramid level 3
Pyramid level 2
Pyramid level 2
Pyramid level 1
Pyramid level 1
Pyramid level 0
Pyramid level 0
Time Taken: 55.16 seconds for image of size (1026, 1824, 3)
Time Taken: 55.17 seconds for image of size (1026, 1824, 3)
Traceback (most recent call last):
Traceback (most recent call last):
File "/Users/magic_lantern/pyflow/flow.py", line 45, in <module>
File "/Users/magic_lantern/pyflow/flow.py", line 45, in <module>
import cv2
ImportError: No module named cv2
import cv2
ImportError: No module named cv2
make: *** [M27-1337_frame_000002-n.png] Error 1
make: *** Waiting for unfinished jobs....
make: *** [M27-1337_frame_000002-p.png] Error 1
Sorry for all, probably noob, questions :P
Interested in making this work, curious what it can do with high iso noise 8)
For as far I can tell, you're using pyflow to align images and once aligned, your averaging the frame with previous and next frame.
Is that what's happening ?
When you install opencv with brew, it will tell you to run some commands if you want to develop with it (or something along these lines). Once you do that, it should be able to find cv2 (aka OpenCV). At least that's how it worked here.
I've used a modified version of this for vertical pattern noise removal (https://www.magiclantern.fm/forum/index.php?topic=19429.msg193360#msg193360) (used more neighbouring frames, reduced the weight of frames without camera motion and only kept the vertical offsets from the correction). Regular noise reduction also works (advice: limit the correction to +/- some small quantity, e.g. 5 units on a 0-255 scale).
For more fun stuff, look at https://people.csail.mit.edu/celiu/ (watch the video on the front page).
Just got another sample file for testing; result:
Quote from: a1ex on November 20, 2017, 04:42:14 PM
dcraw -a -b 2:
Before (original, uncorrected): M18-1635_000003-dcraw.jpg (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/M18-1635_000003-dcraw.jpg)
Corrected with 1 frame before/after: M18-1635_000003-dcraw-a11.jpg (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/M18-1635_000003-dcraw-a11.jpg)
Corrected with 3 frames before/after: M18-1635_000003-dcraw-a33.jpg (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/M18-1635_000003-dcraw-a33.jpg)
Same example with RawTherapee, LMMSE:
Before (original, uncorrected): M18-1635_000003-RT-LMMSE.jpg (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/M18-1635_000003-RT-LMMSE.jpg)
Corrected with 1 frame before/after: M18-1635_000003-RT-LMMSE-a11.jpg (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/M18-1635_000003-RT-LMMSE-a11.jpg)
Corrected with 3 frames before/after: M18-1635_000003-RT-LMMSE-a33.jpg (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/M18-1635_000003-RT-LMMSE-a33.jpg)
Quote from: a1ex on November 20, 2017, 04:19:31 PM
When you install opencv with brew, it will tell you to run some commands if you want to develop with it (or something along these lines). Once you do that, it should be able to find cv2 (aka OpenCV). At least that's how it worked here.
Found the command you're talking about, it works now.
Thanks.
Time to do some testing with it :D
How did you solve opencv issue Levas? I´m getting:
Time Taken: 55.89 seconds for image of size (1026, 1824, 3)
RuntimeError: module compiled against API version 0xb but this version of numpy is 0x9
Traceback (most recent call last):
File "/Users/dan/pyflow/flow.py", line 45, in <module>
import cv2
ImportError: numpy.core.multiarray failed to import
make: *** [M27-1337_frame_000002-p.png] Error 1
dans-MBP:pyflow dan$
Probably need to install the latest numpy version as well...
*Update. Working now. Did:
sudo -H pip install numpy --upgrade
Also noticed LMMSE already handles color artifacts pretty well (https://www.magiclantern.fm/forum/index.php?topic=20995.msg193400#msg193400), so I've rendered the frames with RawTherapee and applied the same script. This is not fully automated - after batch processing with RT or any other external program, convert the images to ppm, delete the dcraw rule from the Makefile and run the remaining stuff. Result:
(http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/RT/M27-1337_frame_000002-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/RT/M27-1337_frame_000002.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/RT/M27-1337_frame_000002-a-crop.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/aliasing/RT/M27-1337_frame_000002-a.jpg)
Knew about the LMMSE method controlling color moire really good, THE reason I've learned to use RawTherapee ;)
But I'm wondering, how to convert the images to ppm :-[ (it's mentioned in your workflow as if every photo/videographer uses the ppm format :P)
And is there a way to export the final; image as tif file instead of jpg ?
convert foobar.tif foobar.ppm
I had some trouble saving 16-bit tif from OpenCV, but 16-bit PNG appears to work fine (convert it to tif afterwards). For 8-bit output, just change the extensions in the Makefile; for 16-bit output, it's something like: cv2.imwrite(out, im2.astype('uint16')); might also require scaling.
The default output format is 8-bit PNG, btw (and the default input is 8-bit PPM because that's what dcraw outputs by default). I've used JPEG just for uploading smaller files.
Anybody already find out how to do more then one previous and one next frame ?
Alex is using 3 previous/next frames here:
https://www.magiclantern.fm/forum/index.php?topic=20995.msg193400#msg193400 (https://www.magiclantern.fm/forum/index.php?topic=20995.msg193400#msg193400)
Very easy:
# similar to %-n.png and %-p.png, but using frames i+2, i-2, i+3 and i-3
%-n2.png: %.ppm $$(call inc,%,2).ppm
$(FLOW) $^ $@
%-p2.png: %.ppm $$(call inc,%,-2).ppm
$(FLOW) $^ $@
%-n3.png: %.ppm $$(call inc,%,3).ppm
$(FLOW) $^ $@
%-p3.png: %.ppm $$(call inc,%,-3).ppm
$(FLOW) $^ $@
# average current frame with 3 warped frames before and 3 after
%-a33.png: %.ppm %-n.png %-p.png %-n2.png %-p2.png %-n3.png %-p3.png
convert -average $^ $@
=> make whatever_001234-a33.png (or jpg, since there is a rule that converts from png to jpg)
A integration of it on HDRMerge would be cool...
ffmpeg has a very fast averaging filter. Here´s an example using the tblend filter averaging consecutive frames. Obviously it will produce a slight blur but maybe one can use something like minterpolate filter to build some in between frame.
You need ffmpeg and dcraw and then you can run the following command directly on two dng files to output highres tif files:
Averaged output
find . -maxdepth 1 -iname '*.dng' -print0 | xargs -0 dcraw +M -c -6 -W -q 3 | ffmpeg -f image2pipe -vcodec ppm -i pipe:0 -y -pix_fmt rgb24 -t 2 -vf tblend=all_mode=average AVERAGED%03d.tif
(https://s18.postimg.cc/5tqhm1055/Screen_Shot_2017-11-21_at_08.08.38.png)
Unaveraged output
find . -maxdepth 1 -iname '*.dng' -print0 | xargs -0 dcraw +M -c -6 -W -q 3 | ffmpeg -f image2pipe -vcodec ppm -i pipe:0 -y -pix_fmt rgb24 NOTAVERAGED%03d.tif
(https://s18.postimg.cc/rsww9899l/Screen_Shot_2017-11-21_at_08.09.05.png)
Averaged
(https://s18.postimg.cc/jbxdy84jt/Screen_Shot_2017-11-21_at_08.18.00.png)
Unaveraged
(https://s18.postimg.cc/ppmh1h1q1/Screen_Shot_2017-11-21_at_08.18.13.png)
ffmpeg tblend filter:
https://ffmpeg.org/ffmpeg-filters.html#blend_002c-tblend
Minterpolate:
https://ffmpeg.org/ffmpeg-filters.html#minterpolate
Seems there are more ways to play with this. By using enblend-enfuse and hugin(align image tool) one can elmininate aliasing and moiré. Inspired by a1ex post I did following.
Install:
brew install dcraw
brew cask install hugin
brew install enblend-enfuse
In a folder place two files let´s say for instance(skip one file in between):
M27-1337_frame_000001.dng
M27-1337_frame_000003.dng
Then run following command inside the folder containing the two dng files:
dcraw +M -6 -W -q 3 -T *.dng && /Applications/Hugin/Hugin.app/Contents/MacOS/align_image_stack -a aligned.tiff $(ls *.tiff | awk 'FNR == 1 {print}') $(ls *.tiff | awk 'FNR == 2 {print}') && enfuse -o enfused_aligned.tiff aligned.tiff0000.tif aligned.tiff0001.tif && rm aligned.tiff0000.tif aligned.tiff0001.tif
The final file will be called enfused_aligned.tiff and should be sharp and clean from aliasing and moiré.
The first file M27-1337_frame_000001.dng can be replaced with enfused_aligned.tiff so then it´s just a matter of building a script that will iterate a whole movie sequence working through the dng sequence/replacing in consecutive order.
enfused_aligned
(https://s18.postimg.cc/dcitlvqrd/Screen_Shot_2017-11-21_at_10.33.45.png)
original
(https://s18.postimg.cc/b7ygkshex/Screen_Shot_2017-11-21_at_10.34.09.png)
Quote from: Danne on November 21, 2017, 08:16:09 AM
maybe one can use something like minterpolate filter to build some in between frame.
ffmpeg -framerate 30 -i M27-1337_frame_%06d.ppm -filter_complex "minterpolate=fps=60:mi_mode=mci:me_mode=bidir" -y interp_%06d.ppm
M27-1337_frame_000000.ppm -> interp_000002.ppm
M27-1337_frame_000001.ppm -> interp_000004.ppm
M27-1337_frame_000002.ppm -> interp_000006.ppm
...
It's very fast; try changing some parameters (https://ffmpeg.org/ffmpeg-filters.html#minterpolate) to get better quality.
To keep only the interpolated frames:
ffmpeg -framerate 30 -i M27-1337_frame_%06d.ppm -filter_complex "minterpolate=fps=60:mi_mode=mci:me_mode=bidir,select=not(mod(n-1\,2))" -r 30 -y interp_%06d.ppm
Nice syntax. However sharpness is affected(possibly masking aliasing better). Compare with the alignment roundtrip(huging/enfuse):
ffmpeg
(https://s18.postimg.cc/dj5pgijtl/Screen_Shot_2017-11-21_at_12.23.10.png)
hugin/enfuse
(https://s18.postimg.cc/iht7v10h5/Screen_Shot_2017-11-21_at_12.25.10.png)
If this effort succeeds, does that mean that expensive VAF filters will no longer be necessary for the cropped Canon models? If that is the case, then this is really a big thing and I keep my thumbs pressed for all developers involved. Good work, guys!
I think it´s dependent to motion but handheld should work.
Quote from: IDA_ML on November 21, 2017, 02:42:51 PM
If this effort succeeds, does that mean that expensive VAF filters will no longer be necessary for the cropped Canon models? If that is the case, then this is really a big thing and I keep my thumbs pressed for all developers involved. Good work, guys!
For as far as I can tell, VAF filters where never necessary :P
http://www.magiclantern.fm/forum/index.php?topic=19687.msg185429#msg185429
That's the beauty of raw, many options in post proces ;D
Sorry for asking here and not googling :P
But is there an easy command line option, with convert, for converting all tif's in one direcory to ppm files ?
This reminds me of the image stacking experiment I made in the Large image, low fps vs. Small image, high fps (http://www.magiclantern.fm/forum/index.php?topic=15306.0) topic a while back. With optical flow you shouldn't get the motion artifacts.
Quote from: Levas on November 21, 2017, 10:50:59 PM
Sorry for asking here and not googling :P
But is there an easy command line option, with convert, for converting all tif's in one direcory to ppm files ?
See command "convert" inside the imagemagick package. If you're on debian-based gnu/linux system, use "sudo apt-get install imagemagick" and then "convert foo.tif bar.ppm".
@bpv5P: if you want to help, try reading the question (quoting it is not enough) and maybe also some of the previous posts.
On-topic:
https://www.cyberciti.biz/faq/bash-loop-over-file/
https://stackoverflow.com/questions/20483331/how-to-copy-and-rename-file-with-for-loop-in-bash-script
https://askubuntu.com/questions/60401/batch-processing-tif-images-converting-tif-to-jpeg
Spoiler:
for f in *.tif; do convert $f ${f%.tif}.ppm; done
@Levas
Note that dcraw can convert straight to ppm as well:
find . -maxdepth 1 -iname '*.dng' -print0 | xargs -0 -P 4 -n 1 dcraw +M -6 -W -w -q 3
This command will also work where a for loop or ls command will break with "argument list too long" issues hitting rooftop around 6000 files...
Thanks,
Never would have guessed this one:
for f in *.tif; do echo "Converting $f"; convert "$f" "$(basename "$f" .tif).ppm"; done
Now I'm trying to average out a whole batch of ppm files with flow.py
I thought, the command
make j8
would do batch process, but it does only the clean operation in the makefile :P
So the number behind the j is just the number of CPUjobs it starts ?
I have a sequence of files with the following name ' Test_000000.ppm' to 'Test_000121.ppm'
So the prefix should be according to the rules _6digits
What command will make all average files for the whole sequence ?
It's "make -j8", and unless you have edited the Makefile, it should render all frames.
Here's a nicer version of the "all" rule (it process frames sequentially and should be easier to understand, but all sub-frames required by one image are still done in parallel):
all:
for f in *.dng; do \
$(MAKE) $${f%.dng}-a.jpg; \
rm -f *.npy; \
done
If your inputs are .jpg (same as outputs), I suggest using case sensitivity to tell the difference between inputs and outputs (for example, make the inputs uppercase). For example:
# use *.JPG for input files and *.jpg for output files
%.ppm: %.JPG:
convert $< $@
all:
for f in *.JPG do \
$(MAKE) $${f%.JPG}-a.jpg; \
rm -f *.npy; \
done
That removes the need to run a bash for loop before "make".
When rendering all frames, you may also want to delete the intermediate files. To do so, just comment out .SECONDARY. The *.npy files are an exception, since they are not created with Makefile rules (so "make" won't know it has to clean them up), hence the "rm -f *.npy" in the above loop.
Thanks, works :D
(Had deleted all unused functions in the makefile, the 'all:' function was already deleted :-[, my fault :P)
Built a batch transcoding script aorund these dependencies:
brew install dcraw
brew cask install hugin
brew install enblend-enfuse
When run on a dng sequence it will end up with tif files which will be aligned and enfused hopefully reducing moiré. All dng files will be placed in a folder named tmp:
mkdir -p tmp
while grep 'dng' <<< $(find . -maxdepth 1 -iname '*.dng')
do
dcraw +M -6 -W -w -q 3 -T $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 1') & pid1=$!
dcraw +M -6 -W -w -q 3 -T $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 2') & pid2=$!
dcraw +M -6 -W -w -q 3 -T $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 3') & pid3=$!
dcraw +M -6 -W -w -q 3 -T $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 4') & pid4=$!
dcraw +M -6 -W -w -q 3 -T $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 5') & pid5=$!
dcraw +M -6 -W -w -q 3 -T $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 6') & pid6=$!
dcraw +M -6 -W -w -q 3 -T $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 7') & pid7=$!
dcraw +M -6 -W -w -q 3 -T $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 8') & pid8=$!
dcraw +M -6 -W -w -q 3 -T $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 9') & pid9=$!
dcraw +M -6 -W -w -q 3 -T $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 10') & pid10=$!
#wait for jobs to end
wait < <(jobs -p)
mv $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 1') $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 2') $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 3') $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 4') $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 5') $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 6') $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 7') $(find . -maxdepth 1 -iname '*.dng' | awk 'FNR == 8') tmp
/Applications/Hugin/Hugin.app/Contents/MacOS/align_image_stack -a aligned.tiff $(ls *.tiff | awk 'FNR == 1') $(ls *.tiff | awk 'FNR == 3') && enfuse -o $(ls *.tiff | awk 'FNR == 1' | cut -d "." -f1).tif aligned.tiff0000.tif aligned.tiff0001.tif && rm aligned.tiff0000.tif aligned.tiff0001.tif & pid1=$!
/Applications/Hugin/Hugin.app/Contents/MacOS/align_image_stack -a aligned02.tiff $(ls *.tiff | awk 'FNR == 2') $(ls *.tiff | awk 'FNR == 4') && enfuse -o $(ls *.tiff | awk 'FNR == 2' | cut -d "." -f1).tif aligned02.tiff0000.tif aligned02.tiff0001.tif && rm aligned02.tiff0000.tif aligned02.tiff0001.tif & pid2=$!
/Applications/Hugin/Hugin.app/Contents/MacOS/align_image_stack -a aligned03.tiff $(ls *.tiff | awk 'FNR == 3') $(ls *.tiff | awk 'FNR == 5') && enfuse -o $(ls *.tiff | awk 'FNR == 3' | cut -d "." -f1).tif aligned03.tiff0000.tif aligned03.tiff0001.tif && rm aligned03.tiff0000.tif aligned03.tiff0001.tif & pid3=$!
/Applications/Hugin/Hugin.app/Contents/MacOS/align_image_stack -a aligned04.tiff $(ls *.tiff | awk 'FNR == 4') $(ls *.tiff | awk 'FNR == 6') && enfuse -o $(ls *.tiff | awk 'FNR == 4' | cut -d "." -f1).tif aligned04.tiff0000.tif aligned04.tiff0001.tif && rm aligned04.tiff0000.tif aligned04.tiff0001.tif & pid4=$!
/Applications/Hugin/Hugin.app/Contents/MacOS/align_image_stack -a aligned05.tiff $(ls *.tiff | awk 'FNR == 5') $(ls *.tiff | awk 'FNR == 7') && enfuse -o $(ls *.tiff | awk 'FNR == 5' | cut -d "." -f1).tif aligned05.tiff0000.tif aligned05.tiff0001.tif && rm aligned05.tiff0000.tif aligned05.tiff0001.tif & pid5=$!
/Applications/Hugin/Hugin.app/Contents/MacOS/align_image_stack -a aligned06.tiff $(ls *.tiff | awk 'FNR == 6') $(ls *.tiff | awk 'FNR == 8') && enfuse -o $(ls *.tiff | awk 'FNR == 6' | cut -d "." -f1).tif aligned06.tiff0000.tif aligned06.tiff0001.tif && rm aligned06.tiff0000.tif aligned06.tiff0001.tif & pid6=$!
/Applications/Hugin/Hugin.app/Contents/MacOS/align_image_stack -a aligned07.tiff $(ls *.tiff | awk 'FNR == 7') $(ls *.tiff | awk 'FNR == 9') && enfuse -o $(ls *.tiff | awk 'FNR == 7' | cut -d "." -f1).tif aligned07.tiff0000.tif aligned07.tiff0001.tif && rm aligned07.tiff0000.tif aligned07.tiff0001.tif & pid7=$!
/Applications/Hugin/Hugin.app/Contents/MacOS/align_image_stack -a aligned08.tiff $(ls *.tiff | awk 'FNR == 8') $(ls *.tiff | awk 'FNR == 10') && enfuse -o $(ls *.tiff | awk 'FNR == 8' | cut -d "." -f1).tif aligned08.tiff0000.tif aligned08.tiff0001.tif && rm aligned08.tiff0000.tif aligned08.tiff0001.tif & pid8=$!
#wait for jobs to end
wait < <(jobs -p)
rm $(ls *.tiff | awk 'FNR == 1') $(ls *.tiff | awk 'FNR == 2') $(ls *.tiff | awk 'FNR == 3') $(ls *.tiff | awk 'FNR == 4') $(ls *.tiff | awk 'FNR == 5') $(ls *.tiff | awk 'FNR == 6') $(ls *.tiff | awk 'FNR == 7') $(ls *.tiff | awk 'FNR == 8')
done
Tested against enfuse/hugin to produce tifs with reduced aliasing. Would be nice to have it checked against optical flow/ffmpeg too. Running out of time here..
A test file here(1200D mv720):
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M23-1009_aliasing.MLV
develop settings in Switch:
ms (mlv_dump_on_steroids)
02 (cs2x2)
dng(opened in acr, sharpness set to 0)
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M23-1009_1_2017-11-23_0001_C0000_000005.dng
(crop)
(https://s33.postimg.cc/rxkpc8qb3/Screen_Shot_2017-11-23_at_09.40.56.png)
tif
(crop)
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M23-1009_1_2017-11-23_0001_C0000_000005.tif
(https://s33.postimg.cc/nofza2urj/Screen_Shot_2017-11-23_at_09.40.15.png)
On the theme super resolution I played with merging and aligning more and more images and the result gets better(crops):
My test file:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M23-1643_aliasing.MLV
Images 1_3
(https://s18.postimg.cc/z9pkaw0zt/1_3.png)
Images 1_2_3_4_5
(https://s18.postimg.cc/7mcuws0dl/1_2_3_4_5.png)
Images 1_2_3_4_5_6_7_8_9_10
(https://s18.postimg.cc/m5jzy7189/1_2_3_4_5_6_7_8_9_10.png)
Images 1_2_3_4_5_6_7_8_9_10_11_12_13_14_15
(https://s18.postimg.cc/6wu2kes49/1_2_3_4_5_6_7_8_9_10_11_12_13_14_15.png)
Full image width:
Images 1_3
(https://s18.postimg.cc/ft4uo6y49/Screen_Shot_2017-11-23_at_16.55.05.png)
Images 1_2_3_4_5_6_7_8_9_10_11_12_13_14_15
(https://s18.postimg.cc/dokhn3ort/Screen_Shot_2017-11-23_at_16.55.15.png)
Quote from: Danne on November 23, 2017, 04:50:14 PM
On the theme super resolution I played with merging and aligning more and more images and the result gets better(crops):
O0o0o0ohh hell yeah baby this is so bad ass!!!
Also is it just me but I noticed that even the shadows seems to boost up slightly in 'Images 1_2_3_4_5_6_7_8_9_10_11_12_13_14_15' or no? 8)
So this is super resolution using optical flow? Wonder how it would works with video. How long does it take to render each frame?
I'm using enfuse/hugin alignment. Rewrote script part to be able to work 15 files.
http://www.magiclantern.fm/forum/index.php?topic=20999.msg193531#msg193531
Well, processing between 5-10 images producing one image will take a while if running movie sequences.
if you can get something thats fast and better than normal with video everyone will be happy. build it into the new switch danne:)
Decided to split the topic:
http://www.magiclantern.fm/forum/index.php?topic=21089.msg193622#msg193622
Using the optical flow script, first post this topic, to enhance some old 320x240 motion jpg avi files.
upscaled the resolution 3 times each dimension, so 960x720 and then run the optical flow script, altered it to let it average out 17 frames (8 previous, 8 next).
The result is actually not bad. It still lacks some detail of course, but shadow noise and jpg blocking artifacts are all gone.
@Alex
In the makefile I see the average frame is made by the command convert.
But when I type convert in terminal, I don't see an option for average ?
I'm curious if there are more options instead of average, maybe median or something ?
Sounds cool - mind sharing a few frames?
For median and other methods for combining images: https://www.imagemagick.org/discourse-server/viewtopic.php?t=17519 and https://www.imagemagick.org/script/command-line-options.php#evaluate-sequence
Would be nice to pick or weight the images to average, based on sharpness, similarity to original, whether there's some slight camera movement... probably best done in a Python or Octave script.
Had to search for an avi without relatives in it, found one :D
I think this is a good example of what aligning and averaging can do.
Before (upscaled tif from 320x240 to 960x720 with Lightroom
(https://farm5.staticflickr.com/4581/24747285148_bf25935ed6_b.jpg) (https://flic.kr/p/DGQiG9)
Same frame, but averaged with 8 previous and 8 next frames:
(https://farm5.staticflickr.com/4584/24746833408_671ef4bd88_b.jpg) (https://flic.kr/p/DGMZpw)
Not very detailed, but much better then the original.
Think median will give more detailed results, will check out the links you send.
Great job Levas!
Could you please do some tests with my 5D2 footage? You can download it from the following link:
https://wetransfer.com/downloads/7747cb1b86614e01dc1acb9bf11bd2a220171124170652/f7fd5be4b588f945da4898c6c41feb9e20171124170652/f83ff8 (https://wetransfer.com/downloads/7747cb1b86614e01dc1acb9bf11bd2a220171124170652/f7fd5be4b588f945da4898c6c41feb9e20171124170652/f83ff8)
I think on this example "motion picture" means a totally different thing... This was my worst experience with moire/aliasing. Yes you can clean up a lot with chroma blur. i filmed this about a year ago, now im filming around F16 if im shooting with infinity focus, so now moire/aliasing is not that much of a problem. Im curious what can you do with this if you are using your method.
EDIT:
Got a averaged frame too, frame 9 averaged with 8 previous and 8 next.
Can't hardly see the difference with the TIF I got with RawTherapee.
For the pixelpeepers, 3 files on google drive, original DNG, the Rawtherapee TIF and the averaged DNG files:
https://drive.google.com/drive/folders/1yOffbd7OzRnr6LEKoJqXIddZ5bFLu03c?usp=sharing (https://drive.google.com/drive/folders/1yOffbd7OzRnr6LEKoJqXIddZ5bFLu03c?usp=sharing)
That one was rather easy 8)
The optical flow script wasn't even needed for this, only RawTherapee was used:
See this post for a youtube link to see the workflow:
http://www.magiclantern.fm/forum/index.php?topic=19687.msg185429#msg185429
Just to see what happens, I'm going to use the optical flow script on frame 9 too.
But that's gonna take a while, will upload it when its done.
(https://farm5.staticflickr.com/4518/38588810162_0397d7537e_o.jpg) (https://flic.kr/p/21MXJR9)
@Alex
Tested out median, I think it has a little edge compared to average, you get some fine noise/grain in the frame which gives it a little more sharpness perception.
I have uploaded all the 16 aligned PNG files, and the original PPM file for this particular frame on google drive:
Just in case you come up with some other methods of combining those 17 frames ;)
https://drive.google.com/drive/folders/1w9kYw-TgVdiJriVpnfkZ1J1PNv17n5mI?usp=sharing (https://drive.google.com/drive/folders/1w9kYw-TgVdiJriVpnfkZ1J1PNv17n5mI?usp=sharing)
I've also included the average and the Median files on google drive
Median:
(https://farm5.staticflickr.com/4518/37735276595_6066da6c4b_o.png) (https://flic.kr/p/Zuxa4a)
@Levas
For comparison reasons I tested your png files by transcoding them to tiffs and ran them against this script(enfuse/hugin). It will average 6 files not all the 17 png files:
http://www.magiclantern.fm/forum/index.php?topic=21089.msg193629#msg193629
enfuse/hugin
(https://s2.postimg.cc/xc3tdqnnd/Screen_Shot_2017-11-26_at_08.48.08.png)
Original
(https://s2.postimg.cc/kkpn7865l/Screen_Shot_2017-11-26_at_08.49.01.png)
@Alex,
Just curious, don't know if, and how much you did dive into the source files for pyflow (the 15 files in the SRC directory).
I see the files are readable code, so easy editable.
But do you know if it is possible to alter some settings in these files that motion is more blurred or something ?
When motion is in the frame, walking person for example, I end up with sharp curvy lines through the image, due to how the allignment and averaging is done.
Here's an example with lot's of those sharp curvy lines around the frame:
(https://farm2.staticflickr.com/1750/42467483062_df2c908eff_o.jpg)
Didn't dig much. There are some parameters, you can edit the source to adjust them, but to figure out what they do, one has to read the papers linked in the README. I've only skimmed the website.
For your issue, maybe some sort of thresholding (comparing the images before blending, to discard areas with differences above the noise floor) could work. You don't need to change the optical flow code for that - just the blending stage (I've used "convert -average"). You may even consider writing it from scratch (e.g. in octave).
ah yes, you're right, the whole average step is done in the makefile written by you...the alignment is done by the pyflow code and creates the aligned images, doesn't put them together.
Enough info for me to dig some further into the average step.
# average
%-a.png: %.ppm %-n.png %-p.png %-n2.png %-p2.png %-n3.png %-p3.png %-n4.png %-p4.png %-n5.png %-p5.png %-n6.png %-p6.png
convert -average $^ $@
Thanks :D
For now, this seems like a quick solution that gives rather good results:
I added the bold tekst to the convert line.
convert -average -despeckle -statistic minimum 3x3 blur 2x2 $^ $@
-Despeckle helps a little and since the curvy lines are mostly highlights, the statistic minimum works rather well, after that a little blur is added and most of it is gone.
Hope somebody is reading this:
I wanted to mess around with pyflow again, but I can't get it to work anymore :-\
I get this error message when running the make command:
Traceback (most recent call last):
File "/Users/magic_lantern/pyflow/flow.py", line 45, in <module>
import cv2
ImportError: dlopen(/usr/local/lib/python2.7/site-packages/cv2.so, 2): Library not loaded: /usr/local/opt/ilmbase/lib/libImath-2_2.23.dylib
Referenced from: /usr/local/Cellar/opencv/3.4.2/lib/libopencv_imgcodecs.3.4.dylib
Reason: image not found
[\code]
When I look into the directory mentioned in the error (Library not loaded: /usr/local/opt/ilmbase/lib/libImath-2_2.23.dylib)
I find a file named "libImath-2_4.24.0.0.dylib" so apparently I only have a newer version of the library on my system.
Error message says it's "Referenced from: /usr/local/Cellar/opencv/3.4.2/lib/libopencv_imgcodecs.3.4.dylib"
How to solve this error ?
Sorry for your time, got it working again :P
Used "brew uninstall --force opencv"
So all versions of opencv were deleted.
After that reinstalled opencv "brew install opencv3"