ML RAW-tools for Linux

[revast]

ML RAW-tools & more how to deal with .raw/mlv & cDNG files in Linux

News
Updated on 05-2017

MLFVS (MLV files can now be simply mounted thanks to MLFVS. 64bit binary; thread; homepage; It's fast and efficient, real-time playback possible with fast storage. I made a Python-based MLVFS-AppIndicator, which allows to control MLVFS from systray)

Installation & Usage

A MLV "converter" that provides a virtual file system of converted data 'on the fly' using Filesystem in Userspace (FUSE). It allows you to "mount" an MLV file(s) which shows up as a directory of converted CDNGs. The data for these CDNGs is provided 'on the fly' by MLFVS as the data is being requested by whatever raw editor or post processing software you are using. It's fast and efficient, and allows for real-time playback.

Advantages
- You don't have to convert first, you more or less "instantly" have "converted" DNGs
- You don't have to choose between keeping the original MLVs or converted DNGs or doubling disk usage to keep both.
- There's no need for a GUI, there already is one provided by the OS (i.e. the OS's file browser).
- Possible to vary the quality, pre-processing, bit depth, etc, of the DNGs on the fly, via webinterface.

convmlv (convmlv + required tools; thread; homepage, written in Bash. A featurefull converter from ML formats (RAW, MLV, DNG sequences) to many formats (EXR, DPX, 10bit Prores). You have to install libc6:i386 to get its dependencies run on 64bit systems.)

Features
-Bash script, based on mlv_dump, raw2dng, mlv2badpixels.sh
-Uses mlv/raw files as arguments
-Create image sequences/movies in EXR, DPX, Prores
-Develop specific frame range easily, making MLRawViewer an ideal companion
-Complete control over RAW development: Highlight reconstruction, demosaicing, color space, chroma smoothing, white balance...
-100% colormanaged, output gamma/gamut (including Cineon, S-Log3, DCI-P3, Rec709...), no quality loss from sensor --> output
-Create/use darkframe subtraction, temporal denoising, wavelet denoising for noise reduction
-Multithreaded
-HDR Dual ISO processing
-Easy bad pixel removal
-Autom. applies lossless compression
-Option to create jpg/mp4 proxies
-Apply ffmpeg-supported filters
-Optional Auto White Balance
-Sound is available as a .wav file
-Metadata dumped from MLV or DNG files to settings.txt or terminal
-Config file interface lets you automate all, saving time

documentation: convmlv -h, pdf

Usage (excerpt)
    -i: output full-quality EXR sequence
    -m: output high-quality Prores 4444 file
    -p 3: generate proxies both JPG seq, H.264 video.
    -C: local config file
    -s %: denotes proxy scale, in %
    -b: seamlessly removes focus pixels, mlv2badpixels.sh.
    -k: keep DNGs
    -d 3: hq AHD demosaicing
    -g 0: Standard Gamma
    -G 2: Rec.709 Gamut
    -o ./output: One folder per converted file, each containing all materials
   
All options must go before the MLV/RAW files.

MLVApp (MLVApp+static FFMpeg, 64bit; thread; homepage, written in qt/C++. A RAW player/converter for MLV files with ProRes export.)

Features
-Free and Open Source!
-Fast (The main goal), Fast playback as well.
-Develop your raw video files like in Adobe Camera Raw (Exposure, Temperature, Tint, Saturation, Dark Strength, Dark Range, Light Strength, Light range, Lighten, Highlight reconstruction)
-Gamma curves: Tonemapping, Alexa Log-C, Cineon Log, Sony S-Log3
-Correct ML Raw files issues (Focus dots, Bad Pixel, Chroma Smooth, Pattern noise, Vertical Stripes, Deflicker target)
-Histogram/WaveformMonitor/Zebras
-Support for Lossless compressed MLV
-Support for 10/12 bit
-Batch export
-ProRes (and other formats) export (Prores 4444, 422 Proxy, 422 LT, 422 HQ, 422, uncompressed AVI so far)

Linux compile howto

ubuntu 16.04 with kubuntu packports ppa activated, so qt 5.6.1:

Code Select Expand

sudo apt-get install libpng16-dev qtmultimedia5-dev qt5-qmake qtbase5-dev libz-dev

I had to change line 22 of MainWindow.cpp to: 

#include <libpng16/png.h>

and line 55 of MLVApp.pro to:   

linux-g++*: LIBS += -L/usr/local/lib/ -lz -lpng16

then the ususal:

Code Select Expand

qt5-qmake MLVApp.pro
make

ffmpeg binary must be in the same directory as MLVApp for working export.

[UNMAINTAINED]MlRawViewer ( 64bitBinary2016-08-15; thread; homepage) .mlv/raw Player (no 10/12 bit .mlv support); converter; GUI & command-line; written in Python. Needs python-opengl python-numpy python-tk python-pyaudio libglfw 3.1+ python-scandir; While unmaintaind, works, particularly for previewing/playing footage, also has powerful LUT support. Prores export 10bit only (ffmpeg). Great for quickly going through a card full of MLVs as you can preview and export only what you need very easily. mostly keyboard controlled

supports RAW MLV in both single and multi-file versions, sound playback from MLV_SND audio, external WAV, playback of directories containing sequences of "CinemaDNG" .DNG files; BSD-style license.

Features
    RAW, MLV and CinemaDNG playback in real-time (if you have fast file read and modest GPU), single and multi-file
    Progress bar and timestamp overlay. Click to jump in file
    Pause playback: SPACE key or play/pause icon.
    Toggle fullscreen and windowed mode: TAB key or fullscreen icon
    Jump to next/previous RAW or MLV files in same directory as current file using O/P keys.
    Drag a RAW/MLV/DNG file onto the window to view that file. Drag a WAV file to play with the current video.
    Step forward and backward through frames by one second (LEFT/RIGHT cursor keys) or one frame (,/. keys) while playing or paused
    Brightness control while playing: UP/DOWN cursor keys. Or click/drag in brightness box.
    Fast GPU Bilinear demosaicing for playback
    High-quality (but slow) CPU demosaicing (AMaZE) when paused or enabled (Q key)
    Export current marked range with audio: E key or red button.
    Choose export folder: W key.
    Choose MOV (ProRes) or DNG sequence as export type using D key.
    White Balance adjustments. Keys: 1=WhiteFluoro,2=Daylight,3=Cloudy,0=None, 4/7=Red down/up, 5/8=Green down/up, 6/9=Blue down/up. Or click/drag in Red/Blue box.
    sRGB colour output
    sRGB or Rec.709 gamma, global (HDR) tone mapping, Linear mapping, Log mapping toggle "T" key or mapping icon
    Drop-frame+audio and non-drop-frame modes. Toggle "F" key
    Slide audio sync -0.5,-0.05,+0.05,+0.5 seconds with V/B/N/M keys.
    Mark IN/OUT, Jump to PREV/NEXT mark with J/K and H/L keys.
    Loop or one play mode: I key or loop icon.
    Exit viewer: ESC key

1D and 3D LUTs can be used.

Keyboard shortcuts

Compilation

Usage:
             ./mlrawviewer.py  [<path_to_ML_RAW_or_MLV_base_file>] [<wav_file_to_play_along_with_video>] [<output_file_name_for_encoding>]  (if no argument given, a file-open dialog pops up)
             

raw2cdng (download latest on website) ML .raw to cinemaDNG ConverterGUI; (10bit LOG, 12bit & 16bit), batch convert, 15% white level adjust; dotnet40 based, runs via Wine; .mlv support, supports audio from mlv

Installation

drag&drop works flawlessly, converting too.

now GPL open source, native MLV and RAW Support, AudioSupport, multithreaded, code at https://bitbucket.org/chmee/raw2cdng




MLVProducer (download, choose amd or intel optimized;  thread; Converter; GUI (Visual Basic); runs via Wine. currently crashes at export, but export is fine)

Installation

Features:
-open Source, Visual Basic
-Correction, Curves, Filters
-Encode to  h264, ProRes, DNxHD, FFVH, JPEG, PNG, TIFF, CinemaDNG, DNG
-HDR Processing
-Dual-ISO(in progress)
-Focus Dots suppression
-Vertical Stripes suppression
-Horizontal noise reducing
-Apply 3DLut: .cube, .3dl
-Color Spaces: Linear, sRGB, AdobeRGB, ClogC, CanonLogC, AlexaLogC, REDLog, BMDFilm, Cineon, REC709, SLog, SLog2, SLog3, SLogF35, Panasonic V-Log,...
-Multithreading
-Demosaic with different algorithms.
-Group\Global editing
-Cache frames almost like AE. 1.5GB Limit.
-Process MLV and RAW Files
-.M00,M01,Mxx.. Files
-.R00,R01,Rxx.. Files
-Process Huge files.
-Process some broken files.
-Video Import
-Audio Support
-16bit precessing
-measuring tools: Histogram, Waveform, RGBParade, RGB Balance

raw2dng (raw2dng2016-08-14; thread) Converter; reference implementation, commandline; 32bit; compiling for 64bit does NOT work, so you have to install libc6:i386 to get it run on 64bit systems; NO MLV SUPPORT

usage:

./raw2dng file.raw [prefix]

=> will create prefix000000.dng, prefix0000001.dng and so on.

mlv_dump (mlv_dump.zip; thread) Converter; reference implementation, commandline; 32bit; libc6:i386 on 64bit. Unzip, rename mlv_dump.linux > mlv_dump, make it executeable (chmod +x ./mlv_dump)

mlv_dump
- used for debugging and converting .mlv files
- can dump .mlv to legacy .raw files  + .wav files
- can dump .mlv to CinemaDNG  + .wav
- can compress and decompress frames using LZMA
- convert bit depth (any depth in range from 1 to 16 bits)

compression:
you can get a data reduction of ~60% with 12 bit files.
down-converting to 8 bits gives you about 90% data reduction.
this feature is for archiving your footage.
converting back to e.g. legacy raw doesn't need any parameters - it will decompress and convert transparently without any additional parameter.

[UNMAINTAINED]mlv2dng (mlv2dng2014-04-30; thread) ML .raw to DNG converter; commandline; 64bit; maybe still useful. NO MLV SUPPORT



[UNMAINTAINED]MLVViewSharp (MLVViewSharp.zip; thread) ML .mlv Player; GUI written in c#, runs with mono-runtime, libmono-corlib2.0-cil libmono-winforms2.0-cil installed

a _very_ simple and hackish MLV viewer to check your footage
will read uncompressed MLV files and display the frames with a few frames per second.

[UNMAINTAINED]Magic Lantern RAW (MagicLanternRAW-latest.zip; thread) Converter GUI; written in QT; 32bit, so needs libc6:i386 and 32bit qt libs to run on 64bit systems;  NO MLV SUPPORT




cr2hdr / Dual ISO(cr2hdr2014-04-29; thread). commandline tool; 32bit; libc6:i386 @ 64bit systems.

Increases dynamic range by sampling the sensor at two different ISOs, switching ISO for every other line pair. This trick cleans up shadow noise, resulting in a dynamic range improvement of around 3 stops, at the cost of reduced vertical resolution, aliasing and moire.

Works for both raw photos (CR2) and raw videos (DNG sequences). You need to postprocess these files with a tool called cr2hdr.

After postprocessing, you will get a DNG that looks like a dark ISO 100 shot, but you can bring the exposure back up and be delighted by how little noise is present in those recovered shadows.

Compilation

Usage:

LC_ALL=C cr2hdr *.CR2

Linux Applications known to work on/with dng, cinemaDNG, respectively .raw/.mlv files support:

Digikam raw photo developer; open source; (ML .raw support via their kipi-plugins; tutorial)https://extragear.kde.org/apps/kipi/

Darktable raw photo developer; open source; (ML .raw import should already work ; also powerful commandline interface for processing frames; thread ; proof of concept tutorial of working with raw image sequences in Darktable )

Rawtherapee, Lightzone, Photivio, Rawstudio are open source raw (photo) developers which are also dealing with .dng, with Rawtherapee having the by far most feature-rich demosaicing possibilities and settings.

Natron opensource compositor modeled after Nuke; node-based; .dng support via openimageIO , exr, tiff dpx etc. supported, ofx-host;

Digital Fusion compositor; freeware(slightly less features)/commercial; now owned by BlackMagicDesign, native Linux version, .dng support; ofx-host;

The Foundry Nuke/NukeX compositor; crossplatform; node-based; free for noncommercial use Edition (watermarked, special file format, features disabled); from $4213; You can use J_Ops , tutorial which includes J_rawReader and allows direct reading of .dng (as well as other raw formats) into nuke. Widely used industry standard compositor, if big Shops do not use inhouse tools, they use this. ofx-host;

Mistika grading/compositing, editing/finishing; commercial; only available as a turnkey system, bundled with hardware $100000+; originally Runs on Suse/Novell Enterprise Linux; ofx-host;

MambaFX grading/compositing; commercial, crossplatform; cinemaDNG only; FREE FOR 1080p! MambaFX is derived from Mistika, without timeline and editing capabilities, so a single Shot based compositor, with most of the powerful compositing/grading features of the full Mistika, for ~ $300!!, with cheap and expensive plugin options ( codecs, 4k, DCP, 3D-rig=$9000) available; recent Nvidia card & proprietary drivers needed.

Ifxsoftware Piranha grading/compositing/editing; crossplatform; commercial, but free for 720p; $1000; EDL; supports cinemaDNG via gpu based plugin (FAST), or dng via DCRAW plugin (slow) screenshot

Davinci Resolve for Linux grading/compositing, editing/finishing; freeware(slightly less features)/commercial; $20000+;commercial version only available bundled with hardware console;  FCPX xml, EDL; Supports CinemaDNG and DNG; ofx-host;

Autodesk Flame Premium editing/conforming/finishing. compositing/grading; $10000+; originally runs on RedHat / CentOS
overview Training edition available. 16 GB ram & recent Nvidia (also geforce 8xx+ works now) card & proprietary drivers needed., Supports DNG, DPX, OpenEXR

Apple Shake compositing; commercial, abandonware; was $20,000 before EOL in 2006; supports DNGs through this openSource dcraw plugin: smshplugins; Very capable  RGBa-only node-based piece of Software from Apple, which is still used in some VFX shops; training material still available; ofx-host;

Speedgrade NX grading/compositing; commercial, abandonware; was $19,999; Abandonware on Linux (as now owned by Adobe, they dropped Linux Version);  cinemaDNG only;



The apertusProject is also dealing with DNG for their axiom, they also want to build an openSource DNGraw processing tool, Open Cine. This software is in the development. You can help out!



common errors:
- "command not found": you are on a 64bit system, but no 32bit C library installed:  install libc6:i386;
- NO you do NOT need wine, except for MLVProducer, raw2cdng

[scurrvy2020]

Thanks for the tips on linux software available to extract the raw files.  What is the best way to put the sequences into a movie file on Linux?

[ItsMeLenny]

Thanks for the tips on linux software available to extract the raw files.  What is the best way to put the sequences into a movie file on Linux?

Probably piping to ffmpeg/avconv.

[ItsMeLenny]

What about tools for dual iso?

[baldand]

See also the thread about MlRawViewer, which is cross-platform (python), but quite focused on Linux:

http://www.magiclantern.fm/forum/index.php?topic=9560.msg91165#msg91165

[a1ex]

What about tools for dual iso?

cr2hdr should work out of the box:

Code Select Expand


hg clone -r unified https://bitbucket.org/hudson/magic-lantern
cd magic-lantern/modules/dual_iso
make cr2hdr
sudo cp cr2hdr /usr/local/bin


then, cr2hdr *.CR2

See also my ufraw tweaks (especially the modified soft-film curve) and my automatic color grading script built on top of ufraw+enfuse. Both of them should be very helpful with dual ISO on Linux (this is what I use).

[ItsMeLenny]

cr2hdr should work out of the box:

Yeah I managed to find the .exe files. I just couldn't find them originally after searching for quite a bit.
I ran them with wine and they worked no problems what so ever.

[a1ex]

On my machine, the exe is twice as slow as the native one, so you may want to build it from source.

[ItsMeLenny]

On my machine, the exe is twice as slow as the native one, so you may want to build it from source.

Ah! I only tried it on 2 pics, haven't gotten completely into it yet, but if that is the case I will do.

[rkonrad]

I used the ffmpeg command suggested to convert tiffs to final movie.  Unfortunately I wasn't successful - I did substitute jpg for tiff.  Here is the printout plus my command:

ffmpeg -f image2 -i ~/FFMPEG_IN/*.jpg -vcodec prores -profile 3  -qscale: v 1 ~\FFMPEG_OUT\proresttest1.mov
ffmpeg version 0.8.10-6:0.8.10-0ubuntu0.13.10.1, Copyright (c) 2000-2013 the Libav developers
  built on Feb  6 2014 20:53:28 with gcc 4.8.1
*** THIS PROGRAM IS DEPRECATED ***
This program is only provided for compatibility and will be removed in a future release. Please use avconv instead.
Input #0, image2, from '/home/rkonrad/FFMPEG_IN/000000.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0.0: Video: mjpeg, yuvj444p, 960x544 [PAR 1:1 DAR 30:17], 25 tbr, 25 tbn, 25 tbc
Requested output format ' ��4�' is not a suitable output format
Segmentation fault
Wow, this is a lot of work for a 24 sec video:)

Thanks in advance!

[ItsMeLenny]

On mlrawview, this "./mlrawgui.py   (for GUI)", doesn't actually work.
It was something I was working on but now the GUI has been implemented into the opengl window,
and the small amount of file browser code was ported from that, Which now means:
mlrawviewer can just be run by ./mlrawviewer.py
without specifying extra info, and it will pop up a file browser instead to select the file.

[escho]

Wow, this is a lot of work for a 24 sec video:)

ffmpeg did it´s work correct: found the wrong formatted output dir (backslash).

Edgar

[nachordez]

I'm unable to find why cr2hdr (both old and new 20-bit) builds properly (or at least without no error message) in my OpenSuse 13.1 desktop computer, but does not work when used. In my Ubuntu netbook builds and works perfectly with the same dependencies.

I've tried dcraw 9.19 and 9.20, but no difference. when i try to pocess a CR2 it starts with it usual lines, read properly the camera model (600D), and then says 'dcraw could not open this file'

Code Select Expand

>cr2hdr LaPalma_1488.CR2
cr2hdr: a post processing tool for Dual ISO images

Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
cr2hdr: print a message when overwriting the output file

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (default)

Input file      : LaPalma_1488.CR2
Camera          : Canon EOS 600D
Camera model    : Canon EOS 600D
dcraw could not open this file

I've tried to open it manually with dcraw, and it works perfect (as expected, 'cause all raw editing software that uses dcraw as backend works).

So, I've tried to dig on source into dcraw_connector files and related  (though I'm not programmer), but cannot see nothing wrong. And the same sources compile in my ubuntu box, so, even more confused... anyway, cr2hdr is not calling properly dcraw in my box (which is in path and properly working)...

The only thing I haven't tried yet (I'll do it now, but with no good expectations) is to use my ubuntu compiled one in my opensuse box, just in case...

¿Any idea?

Thanks

[a1ex]

What happens if you do this change?

Code Select Expand


         if (raw_width == 0)
         {
             printf("dcraw could not open this file\n");
+            system(dcraw_cmd);
             continue;
         }


[nachordez]

Then the output is so:

Code Select Expand

>./cr2hdr LaPalma_1490.CR2
cr2hdr: a post processing tool for Dual ISO images

Last update: 30a5132 on 2014-05-07 10:34:25 UTC by a1ex:
cr2hdr: print a message when overwriting the output file

Active options:
--amaze-edge    : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default)
--cs2x2         : apply 2x2 chroma smoothing in noisy and aliased areas (default)
--wb=graymax    : set AsShotNeutral by maximizing the number of gray pixels (default)

Input file      : LaPalma_1490.CR2
Camera          : Canon EOS 600D
Camera model    : Canon EOS 600D
dcraw could not open this file

Nombre de archivo: LaPalma_1490.CR2
Marca de fecha: Sat May  3 12:45:19 2014
Cámara: Canon EOS 600D
Velocidad ISO: 100
Disparador: 1/1328.0 seg
Obertura: f/4.6
Distancia focal: 18.0 mm
Perfil ICC asociado: no
Número de imágenes crudas: 1
Tamaño de la miniatura: 5184 x 3456
Tamaño real:            5344 x 3516
Tamaño de la imagen:    5202 x 3465
Tamaño de salida:       5202 x 3465
Colores crudos: 3
Patrón de filtrado: RGGBRGGBRGGBRGGB
Multiplicadores de luz de día: 2.339939 0.935795 1.311823
Multiplicadores da cámara: 2305.000000 1024.000000 1567.000000 1024.000000


dcraw output is in spanish, but I think that does not disturb you... :-)
thanks!

[a1ex]

See http://www.magiclantern.fm/forum/index.php?topic=7693.msg69865#msg69865

[nachordez]

thanks!

Such a stupid question... At the same time you were answering, i was thinking about that, and tried the dcraw -v -i -t 0 in my ubuntu box, which is also in spanish, but... dcraw output was in english (don't know why...). So, it's clear and easily patchable!

thanks again!

PS: working great, of course!

[revast]

working pad - maybe useful

dcraw -a -c -H 0 -6 -W -q 3 *.dng | ffmpeg -f image2pipe -vcodec ppm -r "$FPS" -i pipe:0 -vcodec prores_ks -profile:v 3 -vendor ap10 -pix_fmt yuv444p10 -y -r "$FPS" output.mov

When run in the same folder with MLV and RAW files it will convert and apply exiftool information including correct fps to ffmpeg to transcode to ProRes. Many of the script calls needs the binares from cr2hdr-r but should work with just Mlv_dump, Raw2dng, exiftool, dcraw and ffmpeg.

cr2hdr-r
http://www.magiclantern.fm/forum/index.php?topic=13512.0

I run an old ffmpeg called ffmpegx86 in the script so if running a newer release rename accordingly.

Part of the code. When filming slowmotion ffmpeg will scale the file to correct aspect ratio.

raw2dng *.RAW ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_ | awk '/FPS/ { print $3; }' > /tmp/fps

FPS=$(cat /tmp/fps) ;

exiftool -tagsfromfile ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_000000.dng -FrameRate="$FPS" *.dng -FrameRate="$FPS" -overwrite_original ;
01_C0000_000000.DNG "-FrameRate="$FPS"" -overwrite_original ;


if grep -q '40.000\|48.000\|50.000\|60.000' /tmp/fps
then
dcrawx86 -a -c -H 0 -6 -W -q 3 *.dng | ffmpeg -f image2pipe -vcodec ppm -r "$FPS" -i pipe:0 -vcodec prores_ks -profile:v 3 -vendor ap10 -pix_fmt yuv444p10 -y -r "$FPS" -vf scale=iw:ih*1.61 ../${BASE}_1_$(date +%Y-%m-%d)_0001_C0000.mov ;

else

dcrawx86 -a -c -H 0 -6 -W -q 3 *.dng | ffmpeg -f image2pipe -vcodec ppm -r "$FPS" -i pipe:0 -vcodec prores_ks -profile:v 3 -vendor ap10 -pix_fmt yuv444p10 -y -r "$FPS" ../${BASE}_1_$(date +%Y-%m-%d)_0001_C0000.mov ;
fi


For MLV (Part of the code)

exiftool and awk grabbing the framerate

exiftool ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_000000.dng | awk '/Frame Rate/ { print $4; }' > /tmp/fps

   FPS=$(cat /tmp/fps)


When I take those DNGs and pipe them through the script, the ProRes result is very Rec709-ish. Means, colors and contrast look already pretty normal which doesn't leave awfully much space to work with. Is it possible to get a result more towards the direction of C-Log or Cinestyle?
Report to moderator   Logged
Danne


You could work with gamma settings but more efficient would be to implement a lut. I just got a log lut from Andy600 that fits really good here but with dual iso files.

I,d also recommend using find when piping bigger amounts of files since ls won,t list endless amount of files. I changed my code to this.

find . -name '*.DNG' -print0 | xargs -0 dcraw -c -H 0 -6 -W -q 3 -r $Wtemp | ffmpeg -f image2pipe -vcodec ppm -r "$FPS" -i pipe:0 -vcodec prores_ks -pix_fmt yuv444p10 -n -r "$FPS" -vf lut3d=/usr/bin/bmcc_ixml_luts/Andy600_cr2hdr-r_luts/Andy600_lin_to_log.cube ../${BASE}_1_$(date +%Y-%m-%d)_0001_C0000.mov ;

-r $Wtemp (white balance) and $FPS" (frames per second). THese are settings used with awk and not to be used as is.

Log lut implementation

-vf lut3d=/usr/bin/bmcc_ixml_luts/Andy600_cr2hdr-r_luts/Andy600_lin_to_log.cube



Create the fuse.conf file and add the 'user_allow_other' value into it

Code: [Select]

echo "user_allow_other" | sudo tee -a /etc/fuse.conf


If everything is ok, you can create 2 folders (eg. mlv-in and mlv-out)

Code: [Select]

mkdir /home/<YOUR_NAME_HERE>/mlv-in
mkdir /home/<YOUR_NAME_HERE>/mlv-out


and now, you can launch mlvfs !

Code: [Select]

mlvfs -f /home/<YOUR_NAME_HERE>/mlv-out --mlv_dir=/home/<YOUR_NAME_HERE>/mlv-in -o allow_other


Don't close the terminal, open a browser and type : 'http://127.0.0.1:8000'

Magic as a lantern ! 




#!/bin/bash

# Roba da convertire
roba=/home/gigi/Videos/Mjpeg/Convertire/*

# Inizio del ciclo
for file in /home/gigi/Videos/Mjpeg/Convertire/*; do `ffmpeg -i $file -r 30 -s hd1080 -vcodec mjpeg  -qmax 1 -qmin 1 -acodec copy $file.mov` ;done


exit 0;#!/bin/bash

# Roba da convertire
roba=/home/gigi/Videos/Mjpeg/Convertire/*

# Inizio del ciclo
for file in /home/gigi/Videos/Mjpeg/Convertire/*; do `ffmpeg -i $file -r 30 -s hd1080 -vcodec mjpeg  -qmax 1 -qmin 1 -acodec copy $file.mov` ;done


exit 0;

[Danne]

Hi Revast! I see you grabbed a lot of the cr2hdr-r scriptbase. Also check these new lines. For matching audio with mlv-files.

mlv_dump --dng -o ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_ $FILE | awk '/Processed/ { print $2; }' > /tmp/magic_l/framecount_5 ;

    frct_5=$(cat /tmp/magic_l/framecount_5) ;
    rm /tmp/magic_l/framecount_5
   
exiftool ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_000000.dng | awk '/Frame Rate/ { print $4; }' > /tmp/magic_l/framecount_fps_5 
    FPS=$(cat /tmp/magic_l/framecount_fps_5)
    rm /tmp/magic_l/framecount_fps_5
   
echo $frct_5/$FPS | bc -l | awk 'FNR == 1 {print}' > /tmp/magic_l/framecount_result_5
    frct_result_5=$(cat /tmp/magic_l/framecount_result_5) ;
    rm /tmp/magic_l/framecount_result_5

   
/usr/bin/sox_1 ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_.wav ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000.wav trim 0 $frct_result_5 ;
rm ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_.wav


And also very important. To get the correct same multiplier level white balance the script will choose the auto white balance from the first file and apply this to the whole file opposed to running auto on every single dng which can lead to unwanted variations in wb.

dcraw -T -a -v -c ${BASE}_1_$(date +%Y-%m-%d)_0001_C0000_000000.dng 2>&1 | awk '/multipliers/ { print $2,$3,$4,$5 }' > /tmp/magic_l_MLV/mltpl_1

Wtemp=$(cat /tmp/magic_l_MLV/mltpl_1)


(example)

if ls *.wav
then
find . -maxdepth 1 -mindepth 1 -name '*.DNG' -print0 | xargs -0 dcraw -c -H 0 -6 -W -q 3 -r $Wtemp | ffmpeg -i *.wav -f image2pipe -vcodec ppm -r "$FPS" -i pipe:0 -c:v copy -c:a aac -strict experimental -vcodec prores_ks -pix_fmt yuv444p10 -n -r "$FPS" -vf lut3d=/usr/bin/bmcc_ixml_luts/Andy600_cr2hdr-r_luts/Andy600_lin_to_log.cube,lut3d=$lut3d_1_MLV_a,lut3d=$lut3d_2_MLV_a,lut3d=$lut3d_3_MLV_a ../${BASE}_1_$(date +%Y-%m-%d)_0001_C0000.mov

[revast]

thanks! - I am trying to stick together some loose parts - I made an indicator applet for mlvfs, its just a dropdown menu with menu items like mount/unmount, start webinterface, configure mount path, mlvfs command; open mount folder in filebrowser and terminal... and want to make it possible to run scripts from there - custom ones (like a please enter script popup, which autosaves last versions entered), and something like yours, as fixed menu item. Seems to work fine so far -it's using python and is completely self-explanatory - all actual functions are external and are written in shellscript. If you're interested I can post it.
Something to check out is how to handle the folders - I mount all .mlv files contained in a folder right now - so I would have to stick the scripts into a for loop working through the Folders..

[Danne]

Sounds super interesting. Would be really nice to collaborate and maybe implement some things to cr2hdr-r. Right now I,m checking out some other stuff, like color matrice handling and possibilities to alternative transcoding engine so time is an issue. 
What I,m curious about right now is how to get mlvfs transcode to ProRes. Seems you got something going in that direction?

[Renovatio]

Hi guys, can you please share your workflow on linux?
I'm going to install lubuntu, would like to see if it's possible to make something good. See applications here but would like to know the process. Thanks.

[fotorebelion]

Hello everybody am new here to this for and using ml..I decided to try it on my 650d.. im having some problems and questions after reading through the forum.
For legacy raw I using raw2dng and pinkdotremover this works fine..to make a usable video I was tinkering around with the dng2avi script of using uraw-batch to convert to jpg, and  then ffmpg. To make the fim.. the problem with this is that the jpgs. That come are pink / magenta.. so I am using dark table to color grade before exporting... a question to this is is there a way to make a command line script for dark table ?
Also next problem is mlv! The mlv_dump doesn't compile for me in Linux neither the mlvrawviewer..so I must use wine versions..the mlv_dump with wine functions to make a raw and then I use the same raw workflow as above... but the wine mlvrawviewer doesn't take away pinkdots and the dngs they produced can't be opened by dark table..also Linux version crashes with GL error.
Any advice?

[Welles]

@fotorebelion,

MlRawViewer is supposed to work natively I think, you should not have to use wine. Anyway, MlRawViewer is not maintained anymore. Did you have a look to MLVFS?

[Danne]

If on linux I would definitely try out this workflow from so-rose
http://magiclantern.fm/forum/index.php?topic=16799.0