Show Posts
1
Raw Video Postprocessing / ML RAW-tools for Linux
« on: November 15, 2013, 03:05:58 PM » ML RAW-tools & more how to deal with .raw/mlv & cDNG files in Linux
News
Updated on 05-2017
Updated on 05-2017
MLFVS (MLV files can now be simply mounted thanks to MLFVS. 64bit binary; thread; homepage; It's fast and efficient, real-time playback possible with fast storage. I made a Python-based MLVFS-AppIndicator, which allows to control MLVFS from systray)
Installation & Usage
Quote
A MLV "converter" that provides a virtual file system of converted data 'on the fly' using Filesystem in Userspace (FUSE). It allows you to "mount" an MLV file(s) which shows up as a directory of converted CDNGs. The data for these CDNGs is provided 'on the fly' by MLFVS as the data is being requested by whatever raw editor or post processing software you are using. It's fast and efficient, and allows for real-time playback.
Quote
Advantages
- You don't have to convert first, you more or less "instantly" have "converted" DNGs
- You don't have to choose between keeping the original MLVs or converted DNGs or doubling disk usage to keep both.
- There's no need for a GUI, there already is one provided by the OS (i.e. the OS's file browser).
- Possible to vary the quality, pre-processing, bit depth, etc, of the DNGs on the fly, via webinterface.
convmlv (convmlv + required tools; thread; homepage, written in Bash. A featurefull converter from ML formats (RAW, MLV, DNG sequences) to many formats (EXR, DPX, 10bit Prores). You have to install libc6:i386 to get its dependencies run on 64bit systems.)
Quote
Features
-Bash script, based on mlv_dump, raw2dng, mlv2badpixels.sh
-Uses mlv/raw files as arguments
-Create image sequences/movies in EXR, DPX, Prores
-Develop specific frame range easily, making MLRawViewer an ideal companion
-Complete control over RAW development: Highlight reconstruction, demosaicing, color space, chroma smoothing, white balance...
-100% colormanaged, output gamma/gamut (including Cineon, S-Log3, DCI-P3, Rec709...), no quality loss from sensor --> output
-Create/use darkframe subtraction, temporal denoising, wavelet denoising for noise reduction
-Multithreaded
-HDR Dual ISO processing
-Easy bad pixel removal
-Autom. applies lossless compression
-Option to create jpg/mp4 proxies
-Apply ffmpeg-supported filters
-Optional Auto White Balance
-Sound is available as a .wav file
-Metadata dumped from MLV or DNG files to settings.txt or terminal
-Config file interface lets you automate all, saving time
documentation: convmlv -h, pdf
Usage (excerpt)
-i: output full-quality EXR sequence
-m: output high-quality Prores 4444 file
-p 3: generate proxies both JPG seq, H.264 video.
-C: local config file
-s %: denotes proxy scale, in %
-b: seamlessly removes focus pixels, mlv2badpixels.sh.
-k: keep DNGs
-d 3: hq AHD demosaicing
-g 0: Standard Gamma
-G 2: Rec.709 Gamut
-o ./output: One folder per converted file, each containing all materials
All options must go before the MLV/RAW files.
MLVApp (MLVApp+static FFMpeg, 64bit; thread; homepage, written in qt/C++. A RAW player/converter for MLV files with ProRes export.)
Quote
Features
-Free and Open Source!
-Fast (The main goal), Fast playback as well.
-Develop your raw video files like in Adobe Camera Raw (Exposure, Temperature, Tint, Saturation, Dark Strength, Dark Range, Light Strength, Light range, Lighten, Highlight reconstruction)
-Gamma curves: Tonemapping, Alexa Log-C, Cineon Log, Sony S-Log3
-Correct ML Raw files issues (Focus dots, Bad Pixel, Chroma Smooth, Pattern noise, Vertical Stripes, Deflicker target)
-Histogram/WaveformMonitor/Zebras
-Support for Lossless compressed MLV
-Support for 10/12 bit
-Batch export
-ProRes (and other formats) export (Prores 4444, 422 Proxy, 422 LT, 422 HQ, 422, uncompressed AVI so far)
Linux compile howto
ubuntu 16.04 with kubuntu packports ppa activated, so qt 5.6.1:Code: [Select]sudo apt-get install libpng16-dev qtmultimedia5-dev qt5-qmake qtbase5-dev libz-dev
I had to change line 22 of MainWindow.cpp to:
#include <libpng16/png.h>
and line 55 of MLVApp.pro to:
linux-g++*: LIBS += -L/usr/local/lib/ -lz -lpng16
then the ususal:Code: [Select]qt5-qmake MLVApp.pro
make
ffmpeg binary must be in the same directory as MLVApp for working export.
[UNMAINTAINED]MlRawViewer ( 64bitBinary2016-08-15; thread; homepage) .mlv/raw Player (no 10/12 bit .mlv support); converter; GUI & command-line; written in Python. Needs python-opengl python-numpy python-tk python-pyaudio libglfw 3.1+ python-scandir; While unmaintaind, works, particularly for previewing/playing footage, also has powerful LUT support. Prores export 10bit only (ffmpeg). Great for quickly going through a card full of MLVs as you can preview and export only what you need very easily. mostly keyboard controlled
Quote
supports RAW MLV in both single and multi-file versions, sound playback from MLV_SND audio, external WAV, playback of directories containing sequences of "CinemaDNG" .DNG files; BSD-style license.
Features
RAW, MLV and CinemaDNG playback in real-time (if you have fast file read and modest GPU), single and multi-file
Progress bar and timestamp overlay. Click to jump in file
Pause playback: SPACE key or play/pause icon.
Toggle fullscreen and windowed mode: TAB key or fullscreen icon
Jump to next/previous RAW or MLV files in same directory as current file using O/P keys.
Drag a RAW/MLV/DNG file onto the window to view that file. Drag a WAV file to play with the current video.
Step forward and backward through frames by one second (LEFT/RIGHT cursor keys) or one frame (,/. keys) while playing or paused
Brightness control while playing: UP/DOWN cursor keys. Or click/drag in brightness box.
Fast GPU Bilinear demosaicing for playback
High-quality (but slow) CPU demosaicing (AMaZE) when paused or enabled (Q key)
Export current marked range with audio: E key or red button.
Choose export folder: W key.
Choose MOV (ProRes) or DNG sequence as export type using D key.
White Balance adjustments. Keys: 1=WhiteFluoro,2=Daylight,3=Cloudy,0=None, 4/7=Red down/up, 5/8=Green down/up, 6/9=Blue down/up. Or click/drag in Red/Blue box.
sRGB colour output
sRGB or Rec.709 gamma, global (HDR) tone mapping, Linear mapping, Log mapping toggle "T" key or mapping icon
Drop-frame+audio and non-drop-frame modes. Toggle "F" key
Slide audio sync -0.5,-0.05,+0.05,+0.5 seconds with V/B/N/M keys.
Mark IN/OUT, Jump to PREV/NEXT mark with J/K and H/L keys.
Loop or one play mode: I key or loop icon.
Exit viewer: ESC key
1D and 3D LUTs can be used.
Keyboard shortcuts
Compilation
Quote
Usage:
./mlrawviewer.py [<path_to_ML_RAW_or_MLV_base_file>] [<wav_file_to_play_along_with_video>] [<output_file_name_for_encoding>] (if no argument given, a file-open dialog pops up)
raw2cdng (download latest on website) ML .raw to cinemaDNG ConverterGUI; (10bit LOG, 12bit & 16bit), batch convert, 15% white level adjust; dotnet40 based, runs via Wine; .mlv support, supports audio from mlv
Installation
drag&drop works flawlessly, converting too.
now GPL open source, native MLV and RAW Support, AudioSupport, multithreaded, code at https://bitbucket.org/chmee/raw2cdng
MLVProducer (download, choose amd or intel optimized; thread; Converter; GUI (Visual Basic); runs via Wine. currently crashes at export, but export is fine)
Installation
Quote
Features:
-open Source, Visual Basic
-Correction, Curves, Filters
-Encode to h264, ProRes, DNxHD, FFVH, JPEG, PNG, TIFF, CinemaDNG, DNG
-HDR Processing
-Dual-ISO(in progress)
-Focus Dots suppression
-Vertical Stripes suppression
-Horizontal noise reducing
-Apply 3DLut: .cube, .3dl
-Color Spaces: Linear, sRGB, AdobeRGB, ClogC, CanonLogC, AlexaLogC, REDLog, BMDFilm, Cineon, REC709, SLog, SLog2, SLog3, SLogF35, Panasonic V-Log,...
-Multithreading
-Demosaic with different algorithms.
-Group\Global editing
-Cache frames almost like AE. 1.5GB Limit.
-Process MLV and RAW Files
-.M00,M01,Mxx.. Files
-.R00,R01,Rxx.. Files
-Process Huge files.
-Process some broken files.
-Video Import
-Audio Support
-16bit precessing
-measuring tools: Histogram, Waveform, RGBParade, RGB Balance
raw2dng (raw2dng2016-08-14; thread) Converter; reference implementation, commandline; 32bit; compiling for 64bit does NOT work, so you have to install libc6:i386 to get it run on 64bit systems; NO MLV SUPPORT
Quote
usage:
./raw2dng file.raw [prefix]
=> will create prefix000000.dng, prefix0000001.dng and so on.
mlv_dump (mlv_dump.zip; thread) Converter; reference implementation, commandline; 32bit; libc6:i386 on 64bit. Unzip, rename mlv_dump.linux > mlv_dump, make it executeable (chmod +x ./mlv_dump)
Quote
mlv_dump
- used for debugging and converting .mlv files
- can dump .mlv to legacy .raw files + .wav files
- can dump .mlv to CinemaDNG + .wav
- can compress and decompress frames using LZMA
- convert bit depth (any depth in range from 1 to 16 bits)
Quote
compression:
you can get a data reduction of ~60% with 12 bit files.
down-converting to 8 bits gives you about 90% data reduction.
this feature is for archiving your footage.
converting back to e.g. legacy raw doesn’t need any parameters - it will decompress and convert transparently without any additional parameter.
[UNMAINTAINED]mlv2dng (mlv2dng2014-04-30; thread) ML .raw to DNG converter; commandline; 64bit; maybe still useful. NO MLV SUPPORT
[UNMAINTAINED]MLVViewSharp (MLVViewSharp.zip; thread) ML .mlv Player; GUI written in c#, runs with mono-runtime, libmono-corlib2.0-cil libmono-winforms2.0-cil installed
Quote
a _very_ simple and hackish MLV viewer to check your footage
will read uncompressed MLV files and display the frames with a few frames per second.
[UNMAINTAINED]Magic Lantern RAW (MagicLanternRAW-latest.zip; thread) Converter GUI; written in QT; 32bit, so needs libc6:i386 and 32bit qt libs to run on 64bit systems; NO MLV SUPPORT
cr2hdr / Dual ISO(cr2hdr2014-04-29; thread). commandline tool; 32bit; libc6:i386 @ 64bit systems.
Quote
Increases dynamic range by sampling the sensor at two different ISOs, switching ISO for every other line pair. This trick cleans up shadow noise, resulting in a dynamic range improvement of around 3 stops, at the cost of reduced vertical resolution, aliasing and moire.
Works for both raw photos (CR2) and raw videos (DNG sequences). You need to postprocess these files with a tool called cr2hdr.
After postprocessing, you will get a DNG that looks like a dark ISO 100 shot, but you can bring the exposure back up and be delighted by how little noise is present in those recovered shadows.
Compilation
Quote
Usage:
LC_ALL=C cr2hdr *.CR2
Linux Applications known to work on/with dng, cinemaDNG, respectively .raw/.mlv files support:
Digikam raw photo developer; open source; (ML .raw support via their kipi-plugins; tutorial)https://extragear.kde.org/apps/kipi/
Darktable raw photo developer; open source; (ML .raw import should already work ; also powerful commandline interface for processing frames; thread ; proof of concept tutorial of working with raw image sequences in Darktable )
Rawtherapee, Lightzone, Photivio, Rawstudio are open source raw (photo) developers which are also dealing with .dng, with Rawtherapee having the by far most feature-rich demosaicing possibilities and settings.
Natron opensource compositor modeled after Nuke; node-based; .dng support via openimageIO , exr, tiff dpx etc. supported, ofx-host;
Digital Fusion compositor; freeware(slightly less features)/commercial; now owned by BlackMagicDesign, native Linux version, .dng support; ofx-host;
The Foundry Nuke/NukeX compositor; crossplatform; node-based; free for noncommercial use Edition (watermarked, special file format, features disabled); from $4213; You can use J_Ops , tutorial which includes J_rawReader and allows direct reading of .dng (as well as other raw formats) into nuke. Widely used industry standard compositor, if big Shops do not use inhouse tools, they use this. ofx-host;
Mistika grading/compositing, editing/finishing; commercial; only available as a turnkey system, bundled with hardware $100000+; originally Runs on Suse/Novell Enterprise Linux; ofx-host;
MambaFX grading/compositing; commercial, crossplatform; cinemaDNG only; FREE FOR 1080p! MambaFX is derived from Mistika, without timeline and editing capabilities, so a single Shot based compositor, with most of the powerful compositing/grading features of the full Mistika, for ~ $300!!, with cheap and expensive plugin options ( codecs, 4k, DCP, 3D-rig=$9000) available; recent Nvidia card & proprietary drivers needed.
Ifxsoftware Piranha grading/compositing/editing; crossplatform; commercial, but free for 720p; $1000; EDL; supports cinemaDNG via gpu based plugin (FAST), or dng via DCRAW plugin (slow) screenshot
Davinci Resolve for Linux grading/compositing, editing/finishing; freeware(slightly less features)/commercial; $20000+;commercial version only available bundled with hardware console; FCPX xml, EDL; Supports CinemaDNG and DNG; ofx-host;
Autodesk Flame Premium editing/conforming/finishing. compositing/grading; $10000+; originally runs on RedHat / CentOS
overview Training edition available. 16 GB ram & recent Nvidia (also geforce 8xx+ works now) card & proprietary drivers needed., Supports DNG, DPX, OpenEXR
Apple Shake compositing; commercial, abandonware; was $20,000 before EOL in 2006; supports DNGs through this openSource dcraw plugin: smshplugins; Very capable RGBa-only node-based piece of Software from Apple, which is still used in some VFX shops; training material still available; ofx-host;
Speedgrade NX grading/compositing; commercial, abandonware; was $19,999; Abandonware on Linux (as now owned by Adobe, they dropped Linux Version); cinemaDNG only;
The apertusProject is also dealing with DNG for their axiom, they also want to build an openSource DNGraw processing tool, Open Cine. This software is in the development. You can help out!
common errors:
- "command not found": you are on a 64bit system, but no 32bit C library installed: install libc6:i386;
- NO you do NOT need wine, except for MLVProducer, raw2cdng