Magic Lantern Forum

Using Magic Lantern => Post-processing Workflow => HDR and Dual ISO Postprocessing => Topic started by: GedePasek on June 07, 2013, 03:48:05 PM

Title: Updated script for HDR Video in here.
Post by: GedePasek on June 07, 2013, 03:48:05 PM
Hi,

just started to use the Magic Lantern Firmware for HDR Video....

I use the scripts and software from http://wiki.magiclantern.fm/userguide#hdr_video

I saw that these script use JPG as intermediary files...

I have altered the scripts, so lossless file types are used, tif for the Light/Dark alternating frames, tif is a bit faster than jpg, but large files ofcourse, png was too slow.

Avisynth Virtualdub cannot use tif file sequences, but can use png, so enfuse "fuses" the tifs into 24 bit lossless png's, a bit slower, but hey, lossless...

Quality should be a bit better with this workflow...

Its easy to alter the scripts yourself, but you can download the altered scripts here:

http://gpwrite.com/magic-lantern-firmware-hdr-video-script-eos-5d/

Update: In addition color space conversions have been removed (see thread below) and the quality of the output has improved quite a lot.
Title: Re: Lossless File Method for HDR Video in here.
Post by: Audionut on June 07, 2013, 05:18:47 PM
Quote from: GedePasek on June 07, 2013, 03:48:05 PM
Avisynth cannot use tif file sequences

Yes it can (http://avisynth.org/mediawiki/ImageSource)

QuoteSupported formats are:

    (e)bmp, dds, ebmp, jpg/jpe/jpeg, pal, pcx, png, pbm/pgm/ppm, raw, sgi/bw/rgb/rgba, tga, tif/tiff.
    gif, exr, jp2, psd, hdr. [all of them require 1.7.8 DevIL.dll]
Title: Re: Lossless File Method for HDR Video in here.
Post by: deleted.account on June 07, 2013, 10:38:00 PM
The color space conversions from ffmpegsource2 YV12 to RGB then conversion back to YV12 is not lossless and totally pointless. You could remove it and save a bit of quality and speed.
Title: Re: Lossless File Method for HDR Video in here.
Post by: GedePasek on June 08, 2013, 02:49:56 AM
Ah, my mistake, I meant Virtualdub..... Virtualdub cannot import tiff sequences....

Quote from: Audionut on June 07, 2013, 05:18:47 PM
Yes it can (http://avisynth.org/mediawiki/ImageSource)
Title: Re: Lossless File Method for HDR Video in here.
Post by: GedePasek on June 08, 2013, 02:53:10 AM
I don't know about color space conversions, I just think that tif's are better than jpg's to use, since jpg's are lossy, and tifs are not.

(I tested the difference between the jpg's and tifs (by compressability in photoshop) and the tifs do contain more image information)

And that is the difference between my version of the scripts and the ones used in the wiki.

Quote from: y3llow on June 07, 2013, 10:38:00 PM
The color space conversions from ffmpegsource2 YV12 to RGB then conversion back to YV12 is not lossless and totally pointless. You could remove it and save a bit of quality and speed.
Title: Re: Lossless File Method for HDR Video in here.
Post by: Audionut on June 08, 2013, 05:18:53 AM
Quote from: GedePasek on June 08, 2013, 02:49:56 AM
Ah, my mistake, I meant Virtualdub..... Virtualdub cannot import tiff sequences....

Well, not directly it can't.  But it can through avisynth ;)

With regards to the color space conversions, y3llow is right.  You should decide where you want to perform the color space conversion and stay in that color space from there.

Image sources will be RGB.  So you're going RGB - YV12 - RGB - YV12?  Each time you convert color space you lose color precision.
Title: Re: Lossless File Method for HDR Video in here.
Post by: GedePasek on June 08, 2013, 05:32:03 AM
Forgive me my ignorance here.... Me simple, me think tif better than jpg.

The splitting normally extracts jpg's. With my script there are tifs. Do the jpg's have another color space as the tifs?

Tifs are then fused into png's. Do the tifs have another colorspace as the png's?

Is your conclusion that using jpg's throughout the process will give a better result as the tif-png version?


Title: Re: Lossless File Method for HDR Video in here.
Post by: Audionut on June 08, 2013, 05:54:12 AM
The color space problems have nothing to do with tiff vs jpg.

It's the scripts themselves.  My avisynth is a bit rusty but I've got no idea why it's using,

A = ConvertToRGB(A, matrix="PC.601", interlaced=false)

That seems totally redundant to me.

I'd delete those 2 lines (both the A and the B ones) and change the following line to,

A = Converttoyv12(A, matrix="PC.601")           # convert to yv12 needed for Interframe

Again, both the top A and bottom B ones.

That gets rid of 2 color space conversions.

That script is a pretty ugly way to do things IMO.  Mainly as a result of having to extract frames from the MOV.
Raw video will be much better in this regard.  You can enfuse the raw images without all these color space conversions and only have 1 converttoyv12() at the last step before encoding. 
Title: Re: Lossless File Method for HDR Video in here.
Post by: GedePasek on June 08, 2013, 06:10:48 AM
Ow, I thought it had something to do with my changes to the script.... this colorspace thing... but not.. they were already there...

I removed those lines, and things seem just working fine, and indeed a bit faster.

I will add these changes to my changed script and reupload...


(Ok, an ugly way, and raw would be better, sure, but i will have to work with the equipment (cf cards)  i have, so the avisynth/virtualdub will have to do. And the script got a bit better now ;) )
Title: Re: Updated script for HDR Video in here.
Post by: Audionut on June 08, 2013, 06:21:09 AM
Yep, I understand not everyone can work with raw.  Glad it worked out better for you.  If I get time I'll try and update it further, but at a quick glance it looks about as good as it's going to get.

I've updated your thread title, as honestly, there's isn't anything lossless about it (I understand where you were coming from though :) ).  Good to see people working on the old features to make them better ;)
Title: Re: Updated script for HDR Video in here.
Post by: GedePasek on June 08, 2013, 06:29:23 AM
Ok, maybe not lossless, but I just compared the difference with the "old" and "new" scripts, and there is a real huge difference in quality in favor of the new... huge.
Title: Re: Updated script for HDR Video in here.
Post by: deleted.account on June 08, 2013, 09:22:41 AM
re Ugly way to work, I used to use Vdub and Avisynth but now prefer AVSPMod + Avisynth.

Here's some posts off my blog that may help you, about using Canon MOV's with Avisynth, Imagemagick and getting 16bit image sequences out of the process, something VDub can't do.

From two years ago, so plugin links may have expired or plugins improved:

http://blendervse.wordpress.com/2011/12/24/canon-magic-lantern-hdr-feature-to-10bit-lossless-h264/

http://blendervse.wordpress.com/2011/09/16/8bit-video-to-16bit-scene-referred-linear-exrs/

http://blendervse.wordpress.com/2013/03/12/pipe-to-rgb-avspmod-to-imagemagick/

http://blendervse.wordpress.com/2013/03/23/pipe-to-rgb-quick-how-to/
Title: Re: Updated script for HDR Video in here.
Post by: jerrykil on June 15, 2013, 08:09:57 AM
using enfuse and ffmpeg
enfuse frames get distributed to 7 threads

#!/bin/bash
rm -rf audio.mp3
rm -rf video.avi
rm -rf png1/
rm -rf png2/
rm -rf output/
mkdir png1/
mkdir png2/
set -e
ffmpeg -i RAW.MOV -filter:v select="not(mod(n\,2))",setpts="N/(59.94*TB)" png2/%06d.png
ffmpeg -i RAW.MOV -filter:v select="not(mod(n-1\,2))",setpts="N/(59.94*TB)" png1/%06d.png
mkdir output/
cnt=0;
cd png1/
for f in $(ls *.png); do
    echo $f "(+)" ../png2/${f} "=>" ../merged${f}; 
      let cnt=cnt+1;
  if [ $cnt -eq 1 ];
    then echo One $f;
    enfuse -m 6000 -o ../output/merged${f} $f ../png2/${f}>>log1&                                 
  fi;
    if [ $cnt -eq 2 ];
    then echo Two $f;
    enfuse -m 6000 -o ../output/merged${f} $f ../png2/${f}>>log2&               
  fi;
    if [ $cnt -eq 3 ];
    then echo Three $f;
    enfuse -m 6000 -o ../output/merged${f} $f ../png2/${f}>>log3&                                   
  fi;
    if [ $cnt -eq 4 ];
    then echo Four $f;
    enfuse -m 6000 -o ../output/merged${f} $f ../png2/${f}>>log4&                                   
  fi;
    if [ $cnt -eq 5 ];
    then echo Five $f;
    enfuse -m 6000 -o ../output/merged${f} $f ../png2/${f}>>log5&                                   
  fi;
      if [ $cnt -eq 6 ];
    then echo Six $f;
    enfuse -m 6000 -o ../output/merged${f} $f ../png2/${f}>>log6&                                   
  fi;
    if [ $cnt -eq 7 ];
    then echo Seven $f;
     enfuse -m 6000 -o ../output/merged${f} $f ../png2/${f}>>log7             
    cnt=0;                         
  fi;
done
ffmpeg -i ../RAW.MOV -vn -ar 44100 -ac 2 -ab 192k -f mp3 ../audio.mp3
ffmpeg  -i ../audio.mp3 -r 29.97 -f image2 -i ../output/merged%06d.png -r 29.97 -acodec ac3 -vcodec mpeg4 -vb 80M  ../video.avi