Magic Lantern Forum

Using Magic Lantern => Post-processing Workflow => Topic started by: katrikura on February 22, 2017, 12:17:51 PM

Title: fastcinemadng
Post by: katrikura on February 22, 2017, 12:17:51 PM
Hello:
          Any member of the magic lantern community has used Fast CinemaDNG?

http://www.fastcinemadng.com/

On the website of the manufacturer says: The software is also compatible with Canon 5D Mark III camera with Magic Lantern firmware after MLV to CinemaDNG transform.
Title: Re: fastcinemadng
Post by: viikossi on February 22, 2017, 02:31:34 PM
Working well with canon 7d using cDNG

Sent from my Xperia SP using Tapatalk

Title: Re: fastcinemadng
Post by: Walter Schulz on February 22, 2017, 04:22:02 PM
Means it can read CinemaDNG. So what?
Title: Re: fastcinemadng
Post by: reddeercity on February 23, 2017, 05:26:58 AM
Quote
Fast CinemaDNG Processor application allows to view DNG image series recorded with video and photo cameras.
The software can do standard image processing of RAW DNG and CinemaDNG files on NVIDIA GPU in realtime.
We currently support the following DNG (RAW) formats and compression options:
 Uncompressed DNG.
 DNG with lossless compression. It has zero loss of detail, but quite small compression, just around
35% file size reduction for very blurry / underexposed / overexposed images ranging to around 25%
file size reduction for well focused, well exposed, detailed images.
 BMD RAW 3:1 compressed lossy DNG. Some image detail are lost, but not much. The compression is
usually rate controlled to 3:1, so recording capacities of media are very predictable.
 BMD RAW 4:1 compressed lossy DNG. This is the same as 3:1, but with better compression. The
quality is still good, but not as good as 3:1.

The following 3 modes (three DNG formats) are currently offered with BlackMagic URSA:
 Losslessly Compressed RAW
 Visually Lossless 3:1 RAW (lossy)
 Visually Lossless 4:1 RAW (lossy)


Very interesting App , Wins only looks like and in beta so free , Like Beer  :P
It's basically a Grading app for dallies or what ever on Raw Cdng Uses Cuda real time noise reduction !
Very nice   8)  you can grade your MLV Raw and export as Cdng's !!

Quote
DNG export tab contains parameter, used to render project into DNG. Currently only compression and crop are supported.
Supported compression algorythms are:program directory.
Use original. In this mode project files are simply copyed despite of crop settings
Lossless JPEG. In this mode raw data are compressed with losseless jpeg algorythm.
BMD RAW 3:1. In this mode raw data are compressed with lossy BMD RAW 3:1 algorythm, designed byBlackmagic Design.
BMD RAW 4:1. In this mode raw data are compressed with lossy BMD RAW 4:1 algorythm, designed byBlackmagic Design
BMD RAW 5:1. In this mode raw data are compressed with lossy BMD RAW 5:1 algorythm, designed byBlackmagic Design

I did a quick test with 12 bit mlv+audio from MLVFS quick mount with fuse . Seem to work OK and debayed correctly.
Lots of camera raw adjusts to work with , Does take a fairly beefy machine to run
Tested on my over clock FX8350 (4.9 ghz) on dual GTX 580 cards 3GB Vram with 8TB Raid "0" OS on SSD
It seems to labour a little thou my Vid card are really out dated , there recommend at least "GTX 980TI, 1080 or Tesla K40 GPU memory 4-12 GB"
You also have a option for Working Color Space , e.g. sRGB , Adobe , etc... lots of professional stuff
So it seems it's was intended for Blackmagic Camera up to 8K , I highly recommend trying this out even if it's to convert your ML Cdng's to BDM raw3:1 to save space
plus you can have a grade applied or just leave in film log space , exports with FFmpeg , Cdng's , Tiff  . I Can see this app costing big $$$$ later down the road .
So don't loose out .

Edit:
I tested the compressed .dng export , original cdng from mlvfs was 1856x1044 3.75MB & the compressed dng is 1.29MB used the lossless jpeg compression
I can't tell the difference , here a link to each dng frame from my dropbox .
lossless_jpeg.dng (https://www.dropbox.com/s/09a1wif0oymfc25/M18-0120_000000_lossless_jpeg.dng?dl=0)    original ml dng (https://www.dropbox.com/s/1fmwdz8k500mlcl/M18-0120_000000.dng?dl=0)
Also tried 3:1 which gave me a frame size of 133KB
The only thing I see there's no frame rate tag but all other tag are pass though ,
exiftool tell me the 3:1 is 12bit with jpeg compression and the lossless is 16bits with jpeg compression and there again no frame rate tag .
Title: Re: fastcinemadng
Post by: megapolis on February 23, 2017, 09:05:19 AM
Quote
So it seems it's was intended for Blackmagic Camera up to 8K, I highly recommend trying this out even if it's to convert your ML Cdng's to BDM raw 3:1 to save space
We don't have any 8K source CinemaDNG images yet, currently it's working well with 4.6K BMD. The software is intended to work not only with image series from Blackmagic cameras but also with any CinemaDNG images. If something goes wrong with your CinemaDNG, please send these images to me.
It's not correct that the software is always converting your ML CDNG images to BMD RAW 3:1. You can choose lossless compression for any data source, even if it was originally BMD RAW 3:1 or 4:1.

NVIDIA GeForce GTX 580 is ok for image resolutions up to Full HD. For higher resolutions you will need more powerful GPU. If the player is working smoothly, it means that your CPU, GPU and SSD are ok.

There are some important issues which you have not mentioned:
1. One can play your CinemaDNG series from Windows Explorer. Just click right button on the folder and choose Fast CinemaDNG in the list. By pressing Tab you can run viewer at full screen mode.
2. Project culling by choosing starting and finishing points at the timeline.
3. One can crop and save DNG to reduce image size of RAW data.

Quote
Also tried 3:1 which gave me a frame size of 133KB
I agree that the image could be too much compressed and it could be a good idea to add a function for arbitrary compression ratio instead.

Quote
The only thing I see there's no frame rate tag
Yes, we've missed that, thanks for pointing out. This is our mistake and we will fix that soon. We are also going to add some more CDNG editing to the software. Our users are mostly asking us to add 1D LUT and denoising preprocessing features (both before debayering) with intention to save DNG for further processing.

Please have a look at Debayer options. Our latest debayer with MG algorithm is a recent improvement and we would be interested to get your feedback concerning debayer quality. You can zoom still image or video to see all details.

P.S. This year we are going to add support of MLV format to work directly with MLV files, without doing any preliminary MLV-CDNG conversion. Thanks to g3gg0, we've got necessary info.
Title: Re: fastcinemadng
Post by: katrikura on February 23, 2017, 11:50:49 AM
Thank you for sharing this information, proceed to test the software.


Katrikura
Title: Re: fastcinemadng
Post by: andy kh on February 23, 2017, 11:54:07 AM

P.S. This year we are going to add support of MLV format to work directly with MLV files, without doing any preliminary MLV-CDNG conversion. Thanks to g3gg0, we've got necessary info.
[/quote]

Wow!! Sounds great
Title: Re: fastcinemadng
Post by: DeafEyeJedi on February 23, 2017, 04:48:21 PM
...This year we are going to add support of MLV format to work directly with MLV files, without doing any preliminary MLV-CDNG conversion. Thanks to g3gg0, we've got necessary info.

+10 and big thanks to @g3gg0!
Title: Re: fastcinemadng
Post by: Lars Steenhoff on February 23, 2017, 05:52:42 PM
Cool stuff!  Mac version possible?
Title: Re: fastcinemadng
Post by: megapolis on February 23, 2017, 06:39:04 PM
Quote
Mac version possible?
In the core of Fast CinemaDNG software we have our own image processing SDK which is working on NVIDIA CUDA. Apart from DNG/CinemaDNG decoding we do almost everything on GPU. We believe that GPU could offer the fastest way of image processing and we do need that to insure realtime performance for 4K and higher resolutions. You can check the content of that SDK and have a look at performance benchmarks: http://www.fastcompression.com/products/sdk/sdk.htm

So we are working with NVIDIA GPUs only and unfortunately at the moment Apple doesn't manufacture any PC/laptop with NVIDIA GPUs, they install AMD/Intel only. This is the main obstacle. Our image processing SDK can work with Windows and Linux, and we can port it to MacOS as well, but our users don't have right hardware to work with. So we are waiting for good news from Apple.
Title: Re: fastcinemadng
Post by: Lars Steenhoff on February 23, 2017, 08:08:50 PM
(http://www.larssteenhoff.com/forum_pics/mac.png)
My mac is quite capable :) and with CUDA
but yea I understand apple has left nvida behind. 
that the main reason I don't want to upgrade to a newer mac.

I could install windows and try out the app. it seems really nice.
Title: Re: fastcinemadng
Post by: DeafEyeJedi on February 24, 2017, 05:57:56 AM
Tried running this on OS X 10.12.3 under Wine and got up to the point where it's 80% installed this error pops up...

(https://c1.staticflickr.com/3/2911/33040932516_41aa4aa730.jpg) (https://flic.kr/p/SkHq7h)

Here are some logs in case they give out hints? Maybe I need to install one more tweak within Wine under Tools advance settings?

Code: [Select]
fixme:module:load_library unsupported flag(s) used (flags: 0x00000800)
fixme:system:SetProcessDPIAware stub!
fixme:ver:GetCurrentPackageId (0x33f934 0x0): stub
fixme:toolhelp:CreateToolhelp32Snapshot Unimplemented: heap list snapshot
fixme:toolhelp:Heap32ListFirst : stub
fixme:nls:get_dummy_preferred_ui_language (0x8 0x33f9c0 0x33f9e4 0x33f9b8) returning a dummy value (current locale)
fixme:file:FindFirstFileExW flags not implemented 0x00000002
fixme:explorerframe:taskbar_list_SetProgressValue iface 0x1802cf8, hwnd 0x1006e, ullCompleted 64, ullTotal 64 stub!
fixme:explorerframe:taskbar_list_SetProgressState iface 0x1802cf8, hwnd 0x1006e, flags 0 stub!
fixme:explorerframe:taskbar_list_SetOverlayIcon iface 0x1802cf8, hwnd 0x1006e, hIcon 0x0, pszDescription (null) stub!
fixme:nls:get_dummy_preferred_ui_language (0x8 0x33c80c 0x33c830 0x33c804) returning a dummy value (current locale)
fixme:nls:get_dummy_preferred_ui_language (0x8 0x33c80c 0x33c830 0x33c804) returning a dummy value (current locale)
fixme:file:FindFirstFileExW flags not implemented 0x00000002
fixme:nls:get_dummy_preferred_ui_language (0x8 0x33c988 0x33c9ac 0x33c980) returning a dummy value (current locale)
fixme:nls:get_dummy_preferred_ui_language (0x8 0x33c948 0x33c96c 0x33c940) returning a dummy value (current locale)
fixme:commdlg:IServiceProvider_fnQueryService Interface {e07010ec-bc17-44c0-97b0-46c7c95b9edc} requested from unknown service {e07010ec-bc17-44c0-97b0-46c7c95b9edc}
fixme:shell:ViewModeToListStyle ViewMode 0 not implemented
fixme:shell:IShellBrowser_fnSendControlMsg stub, 0x193e2a0 (2, 1026, a003, 0, 0x33ba4c)
fixme:shell:IShellBrowser_fnSendControlMsg stub, 0x193e2a0 (2, 1026, a004, 0, 0x33ba4c)
fixme:shell:IShellBrowser_fnSendControlMsg stub, 0x193e2a0 (2, 1025, a003, 1, 0x33ba4c)
fixme:shell:IShellBrowser_fnSendControlMsg stub, 0x193e2a0 (2, 1025, a004, 1, 0x33ba4c)
fixme:nstc:NSTC2_fnSetControlStyle2 mask & style (0x00000004) contains unsupported style(s): 0x00000004
fixme:win:FlashWindowEx 0x33b6bc - semi-stub
fixme:win:FlashWindowEx 0x33c05c - semi-stub
fixme:explorerframe:taskbar_list_SetOverlayIcon iface 0x1823068, hwnd 0x1006e, hIcon 0x0, pszDescription (null) stub!
err:ole:CoCreateInstanceEx apartment not initialised
err:ole:CoCreateInstanceEx apartment not initialised
err:ole:CoCreateInstanceEx apartment not initialised
err:ole:CoCreateInstanceEx apartment not initialised
err:ole:CoCreateInstanceEx apartment not initialised
err:ole:CoCreateInstanceEx apartment not initialised
fixme:wscript:set_host_properties ignored L"nologo" switch
fixme:vbscript:VBScript_SetScriptState unimplemented SCRIPTSTATE_INITIALIZED
fixme:scrrun:filesys_MoveFile 0x45f33cb0 L"C:\\Program Files\\FastCinemaDNG\\maintenancetool.dat.new" L"C:\\Program Files\\FastCinemaDNG\\maintenancetool.dat"
fixme:wscript:set_host_properties ignored L"nologo" switch
fixme:vbscript:VBScript_SetScriptState unimplemented SCRIPTSTATE_INITIALIZED
fixme:scrrun:filesys_MoveFile 0x455d4cb0 L"C:\\Program Files\\FastCinemaDNG\\maintenancetool.exe.new" L"C:\\Program Files\\FastCinemaDNG\\maintenancetool.exe"
winedevice.exe(3731,0x401fb000) malloc: *** error for object 0x79be8dff: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

Code: [Select]
WineskinX11: main(): argc=12
Waiting for startup parameters via Mach IPC.
WineskinX11: do_start_x11_server(): argc=12
Attempting to use pixel depth of 24
[1853071.006] WineskinX11 starting:
[1853071.006] X.Org X Server 1.13.0
[1853071.006] Build Date: 20120921
[1853071.006] _XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.
[1853071.008] Initializing built-in extension Generic Event Extension
[1853071.008] Initializing built-in extension SHAPE
[1853071.008] Initializing built-in extension MIT-SHM
[1853071.008] Initializing built-in extension XInputExtension
[1853071.008] Initializing built-in extension XTEST
[1853071.008] Initializing built-in extension BIG-REQUESTS
[1853071.008] Initializing built-in extension SYNC
[1853071.008] Initializing built-in extension XKEYBOARD
[1853071.008] Initializing built-in extension XC-MISC
[1853071.008] Initializing built-in extension XINERAMA
[1853071.008] Initializing built-in extension PseudoramiX
[1853071.008] Initializing built-in extension XFIXES
[1853071.008] Initializing built-in extension RENDER
[1853071.008] Initializing built-in extension RANDR
[1853071.008] Initializing built-in extension DAMAGE
[1853071.008] Initializing built-in extension MIT-SCREEN-SAVER
[1853071.008] Initializing built-in extension DOUBLE-BUFFER
[1853071.008] Initializing built-in extension RECORD
[1853071.008] Initializing built-in extension X-Resource
[1853071.008] Initializing built-in extension XVideo
[1853071.008] Initializing built-in extension XVideo-MotionCompensation
[1853071.008] Initializing built-in extension GLX
[1853071.008] x: 0, y: 0, w: 2560, h: 1417
[1853071.021] (II) GLX: Initialized Core OpenGL GL provider for screen 0
[1853071.021] [dix] Could not init font path element /opt/X11/share/fonts/75dpi, removing from list!
[1853071.021] [dix] Could not init font path element /opt/X11/share/fonts/100dpi, removing from list!
[1853071.021] [dix] Could not init font path element /opt/X11/share/fonts/cyrillic, removing from list!
[1853071.021] [dix] Could not init font path element /opt/X11/share/fonts/misc, removing from list!
[1853071.021] [dix] Could not init font path element /opt/X11/share/fonts/OTF, removing from list!
[1853071.021] [dix] Could not init font path element /opt/X11/share/fonts/Speedo, removing from list!
[1853071.022] [dix] Could not init font path element /opt/X11/share/fonts/TTF, removing from list!
[1853071.022] [dix] Could not init font path element /opt/X11/share/fonts/Type1, removing from list!
[1853071.022] [dix] Could not init font path element /opt/X11/share/fonts/util, removing from list!
[1853071.123] noPseudoramiXExtension=0, pseudoramiXNumScreens=1
Engine Used: WS9Wine2.2
Hardware:

    Hardware Overview:

      Model Name: Mac mini
      Model Identifier: Macmini6,2
      Processor Name: Intel Core i7
      Processor Speed: 2.3 GHz
      Number of Processors: 1
      Total Number of Cores: 4
      L2 Cache (per Core): 256 KB
      L3 Cache: 6 MB
      Memory: 16 GB
      Boot ROM Version: MM61.0106.B0B
      SMC Version (system): 2.8f1

Graphics/Displays:

    Intel HD Graphics 4000:

      Chipset Model: Intel HD Graphics 4000
      Type: GPU
      Bus: Built-In
      VRAM (Dynamic, Max): 1536 MB
      Vendor: Intel (0x8086)
      Device ID: 0x0166
      Revision ID: 0x0009
      Metal: Supported
      Displays:
        DELL U2713HM:
          Resolution: 2560 x 1440 @ 59 Hz
          Pixel Depth: 32-Bit Color (ARGB8888)
          Display Serial Number: 7JNY5396175L
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Rotation: Supported
          Automatically Adjust Brightness: No
          Connection Type: DisplayPort
          Television: Yes

Title: Re: fastcinemadng
Post by: reddeercity on February 24, 2017, 07:16:27 AM
You need 
Quote
NVIDIA GPU (Fermi, Kepler, Maxwell or Pascal) installed to run the software on GPU.
Title: Re: fastcinemadng
Post by: megapolis on February 24, 2017, 07:53:22 AM
Quote
Tried running this on OS X 10.12.3 under Wine and got up to the point where it's 80% installed this error pops up...

Unfortunately we haven't tested the setup on OS X under Wine yet. We will definitely check that soon. The message says that one can't register dll which is responsible for context menu in Windows Explorer. We've already seen such a situation and in all cases Windows was corrupted.

Haven't seen any NVIDIA GPU in your hardware. The software can't work with Intel built-in GPU (Intel HD Graphics 4000).
Title: Re: fastcinemadng
Post by: masc on February 24, 2017, 11:34:16 AM
@DeafEyeJedi: I also tried to run it on Wine. When did the message came up? On install? Or on run? Since the new Wine 2.0 64bit on OSX is supported (with some bugs).
I got it installed 100%. But on run I got an error which tells me about a 64bit problem... maybe some settings were wrong?!
Title: fastcinemadng
Post by: DeafEyeJedi on February 24, 2017, 08:04:27 PM
@masc -- I ran Wineskin-2.6.2 (WS9Wine2.2) and the only tweak I added was vb6run under Winetrick within Advance settings.

The error (screenshot above) showed up as I was about 80% installed (then I clicked on ignore error) which then finished the installation but then it won't run for some reason. Did I miss a step for this particular app to work under Wine besides the 64-bit bug?
Title: Re: fastcinemadng
Post by: reddeercity on February 24, 2017, 10:18:55 PM
 @megapolis can Fast CinemaDng use multiply GPU's ,  2 or more (I have 2 gtx580 installed)
It seem I can choice either but not both .
Also do you or will you have support for a Capture/Playback card/device for monitoring on a external calibrated Grading HDMI & or HDSDI monitor ?
Since to have already integrated some blackmagic stuff maybe have support for there USB 3.0 device "UltraStudio SDI" has HDMI & HDSDI playback/output
and or any of the  blackmagic PCIe capture cards
I use my USB 3.0 device cross platform PC/MAC would be icing on top of the cake  ;)
Title: Re: fastcinemadng
Post by: megapolis on February 25, 2017, 09:40:47 AM
@reddeercity
Quote
@megapolis can Fast CinemaDng use multiply GPU's, 2 or more (I have 2 gtx580 installed)
Current version of Fast CinemaDNG is working with just one GPU, though our core SDK can work with multiple NVIDIA GPUs as well. We think that for resolutions like 2K and 2.5K, one GPU should be enough for full image processing in realtime. 4K and 4.6K resolutions could be processed on just one GPU GeForce GTX 1070 or 1080. We are still working on optimization issues for CPU/GPU, and final solution should be faster.

We plan to use the second GPU for more complicated denoising or for JPEG2000 encoding and decoding (instead of ProRes or DNxHD/HR). These algorithms need a lot of computations and GPU memory, and it's difficult to incorporate them into existing pipeline on the first GPU in realtime.

Quote
Also do you or will you have support for a Capture/Playback card/device for monitoring on a external calibrated Grading HDMI & or HDSDI monitor?
At the moment we support just one monitor which is connected to the same GPU. We take into account ICC-profile of the monitor and you can define corresponding profile in the software settings. We do that color correction on CUDA and send processed data to the monitor.
You can work with multiple monitors, which are connected to the same GPU, but the software will take into account just one ICC profile.
We are also considering a task of 10-bit monitor support, but at the moment NVIDIA is offering that feature only for the latest Quadro GPUs, which are quite expensive.

Quote
Since to have already integrated some blackmagic stuff maybe have support for there USB 3.0 device "UltraStudio SDI" has HDMI & HDSDI playback/output and or any of the blackmagic PCIe capture cards
In order to support any external USB3 device, we need developer kit and API to work with such a device. At the moment we don't have that.
As far as concerns Blackmagic PCIe capture cards, we have such an experience and we can use them as input to offer realtime image acquisition and processing. That feature is not yet released and if you need it, please specify the task in more detail.
Title: Re: fastcinemadng
Post by: masc on February 25, 2017, 11:14:36 AM
@DeafEyeJedi: Ok, I used exactly the same. I only did not try the vb6run. But I had no error durring installation.
Bug maybe the wrong word:
https://www.winehq.org/wwn/364#Wine64%20on%20Mac%20OS%20X (https://www.winehq.org/wwn/364#Wine64%20on%20Mac%20OS%20X)
and
https://wiki.winehq.org/FAQ (https://wiki.winehq.org/FAQ), point 2.6
Title: Re: fastcinemadng
Post by: megapolis on March 02, 2017, 03:17:42 PM
@reddeercity: We've fixed the issue with framerate tag, now it's working. Please check that once more.

In the latest release we've added two significant improvements for CinemaDNG processing:

1. For all demosaicing algorithms we've added “Enhance level” option and this improvement removed many debayer artifacts that we used to have. To compare with our previous debayer versions, you can set “Enhance level” = 1. That value is individual for each image and it's up to you what to choose. Actually we've added some more image processing stages before and after debayering to get the result.

Please test that software and share your experience with us. We would be interested to find out your results of comparison with demosaicing algorithms from ML, Davinci Resolve, Adobe Premiere Pro, ACR, Lightroom, etc.

2. We've improved Lossless JPEG decoder on CPU. This is one of the most important bottlenecks to overcome, before we could get smooth output in Player window. As soon as we don't use proxies and do all image processing for full-frame images, this is not simple, especially for 4K and 4.6K resolutions.

With new decoder on PC with NVIDIA GeForce GTX 1080 and CPU Intel Core i7 5930 we can process CinemaDNG footage with 4.6К resolution smoothly, with full image processing pipeline, including MG debayer, denoiser and unsharp mask.
Title: Re: fastcinemadng
Post by: reddeercity on March 03, 2017, 02:34:32 AM
@megapolis thanks for the update , Yes I can confirm the frame rate tag is back
I did notice the compressed Cdng from fastcdng changed the exposure tag , from 1/53  to 1/50
plus the light value was changed from 7.1 to 7.0 , I don't think that would cause any problems .
I haven't had time to fully check out all the changes yet , will do a.s.a.p.
below is the orginal Cdng from MLVFS and the exiftool info
https://www.dropbox.com/s/sqmgd2sif8vwyw4/M18-0120_000000_O.dng?dl=0
Code: [Select]
ExifTool Version Number         : 10.33
File Name                       : M18-0120_000000.dng
Directory                       : D:/Window7_Downloads Saves/exiftool-10.33/New
folder
File Size                       : 3.8 MB
File Modification Date/Time     : 2017:02:18 01:19:46-07:00
File Access Date/Time           : 2017:03:02 17:13:26-07:00
File Creation Date/Time         : 2017:03:02 17:13:26-07:00
File Permissions                : rw-rw-rw-
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Subfile Type                    : Full-resolution Image
Image Width                     : 1856
Image Height                    : 1044
Bits Per Sample                 : 16
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Fill Order                      : Normal
Make                            : Canon
Camera Model Name               : Canon EOS 5D Mark II
Strip Offsets                   : 65536
Orientation                     : Horizontal (normal)
Samples Per Pixel               : 1
Rows Per Strip                  : 1044
Strip Byte Counts               : 3875328
Planar Configuration            : Chunky
Software                        : MLVFS
Modify Date                     : 2017:01:18 01:19:46
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Exposure Time                   : 1/53
F Number                        : 4.5
ISO                             : 800
Sensitivity Type                : ISO Speed
Exif Version                    : 0230
Subject Distance                : 107 m
Focal Length                    : 57.0 mm
Lens Model                      : EF24-70mm f/2.8L USM
DNG Version                     : 1.4.0.0
Unique Camera Model             : Canon EOS 5D Mark II
Linearization Table             : (Binary data 19364 bytes, use -b option to ext
ract)
Black Level                     : 448
White Level                     : 3750
Default Crop Origin             : 0 0
Default Crop Size               : 1856 1044
Color Matrix 1                  : 0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1
496 0.1937 0.6651
As Shot Neutral                 : 0.4642547087 1 0.5671705788
Baseline Exposure               : 0
Calibration Illuminant 1        : D65
Active Area                     : 0 0 1044 1856
Frame Rate                      : 23.976
Baseline Exposure Offset        : 0
Aperture                        : 4.5
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 1856x1044
Megapixels                      : 1.9
Shutter Speed                   : 1/53
Focal Length                    : 57.0 mm
Light Value                     : 7.1

Compressed Cdng from FastCdng with exiftool info

https://www.dropbox.com/s/25p5oshysad5cwj/M18-0120_000000%20%282%29.dng?dl=0

Code: [Select]
ExifTool Version Number         : 10.33
File Name                       : M18-0120_000000 (2).dng
Directory                       : D:/Window7_Downloads Saves/exiftool-10.33/New
folder
File Size                       : 1328 kB
File Modification Date/Time     : 2017:03:02 17:24:16-07:00
File Access Date/Time           : 2017:03:02 17:26:18-07:00
File Creation Date/Time         : 2017:03:02 17:26:18-07:00
File Permissions                : rw-rw-rw-
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Subfile Type                    : Full-resolution Image
Image Width                     : 1856
Image Height                    : 1044
Bits Per Sample                 : 16
Compression                     : JPEG
Photometric Interpretation      : Color Filter Array
Make                            : Canon
Camera Model Name               : Canon EOS 5D Mark II
Orientation                     : Horizontal (normal)
Samples Per Pixel               : 1
Planar Configuration            : Chunky
Software                        : MLVFS
Modify Date                     : 2017:01:18 01:19:46
Tile Width                      : 464
Tile Length                     : 522
Tile Offsets                    : (Binary data 55 bytes, use -b option to extrac
t)
Tile Byte Counts                : (Binary data 55 bytes, use -b option to extrac
t)
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Exposure Time                   : 1/50
F Number                        : 4.5
ISO                             : 800
Sensitivity Type                : ISO Speed
Exif Version                    : 0230
Shutter Speed Value             : 1/50
Aperture Value                  : 4.5
Subject Distance                : 107 m
Focal Length                    : 57.0 mm
Lens Model                      : EF24-70mm f/2.8L USM
DNG Version                     : 1.4.0.0
DNG Backward Version            : 1.1.0.0
Unique Camera Model             : Canon EOS 5D Mark II
CFA Plane Color                 : Red,Green,Blue
CFA Layout                      : Rectangular
Linearization Table             : (Binary data 19364 bytes, use -b option to ext
ract)
Black Level Repeat Dim          : 1 1
Black Level                     : 448
White Level                     : 3750
Default Scale                   : 1 1
Default Crop Origin             : 0 0
Default Crop Size               : 1856 1044
Color Matrix 1                  : 0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1
496 0.1937 0.6651
Analog Balance                  : 1 1 1
As Shot Neutral                 : 0.464255 1 0.567171
Baseline Exposure               : 0
Baseline Noise                  : 1
Baseline Sharpness              : 1
Bayer Green Split               : 0
Linear Response Limit           : 1
Anti Alias Strength             : 1
Shadow Scale                    : 1
Calibration Illuminant 1        : D65
Best Quality Scale              : 1
Raw Data Unique ID              : 41D88194533B26B6F0D77FDC99B5C141
Active Area                     : 0 0 1044 1856
Profile Name                    : Embedded
Profile Embed Policy            : Allow Copying
Frame Rate                      : 23.976
New Raw Image Digest            : b2735ab08d8d3d9ead368696f0fb7d6f
Aperture                        : 4.5
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 1856x1044
Megapixels                      : 1.9
Shutter Speed                   : 1/50
Focal Length                    : 57.0 mm
Light Value                     : 7.0
-- press RETURN --




Title: Re: fastcinemadng
Post by: megapolis on March 03, 2017, 12:45:01 PM
@reddeercity: We've checked the issue with Exposure Time and have found that it's changed by a function from Adobe DNG SDK. Actually the software has found right value for Exposure Time in the original DNG and then compared it with predefined list of values. The closest one to 53 is 50, so 1/53 was switched to 1/50. Sure, we can fix that function, but there is a question. Does exist Exposure Time 1/53 s at your camera?

P.S. Please have a look at debayer quality for MG algorithm with enhance option.
Title: Re: fastcinemadng
Post by: hyalinejim on March 03, 2017, 07:16:30 PM
It seems good from what I briefly saw of it, "enhance" seems to reduce aliasing, but not as good as ACR with Cinelog, nor is the highlight reconstruction as good:

(https://s23.postimg.org/4456yom2j/image.jpg)

Sharpening and noise reduction is not working for me in the new version. I cannot change the sliders, even with the demo project. And I cannot find the export window.

(https://s17.postimg.org/m4fczdl8f/image.jpg)
Title: Re: fastcinemadng
Post by: megapolis on March 03, 2017, 08:04:19 PM
@hyalinejim: In order to be able to work with weak GPUs with 1-1.5 GB memory, we've implemented image processing pipeline manager. This is the way to exclude some image processing stages from the pipeline. You need to run the software without loading any project (or you can close current project) and click on Options button, then go to “Output and Extensions” tab. You will see there denoising, sharpening and other options for image processing which are excluded from the pipeline. Check necessary modules and try to open new project. If you have enough GPU memory, it should work. To get the same colors as ACR, please try both RGB and HSV LUTs as well. Manual in PDF you can find at the same folder as the software.
Usually we set "Enhance level" around 2.5-3.0 or even less. I think that 5 is often too much.
Could you please supply a link to DNG source image?
Title: Re: fastcinemadng
Post by: hyalinejim on March 03, 2017, 10:30:10 PM
Thanks megapolis for your quick reply! Your suggestions worked to re-enable parts of the interface. Here is a link to the DNG file:

https://drive.google.com/open?id=0B1exEpCRAfgFQ1VOV1BCazBrVVk

It was shot in anamorphic, so it is laterally distorted. It's an interesting image as the candle flame will show aliasing artifacts typical of the 5D as sharpness is increased. Otherwise, the sharpening is very good indeed.

However, panel docking is not working well.
Title: Re: fastcinemadng
Post by: megapolis on March 04, 2017, 04:07:36 PM
@hyalinejim: Thanks for your image and for your comparison.
I agree with you, but please note that for that specific image the job was done due to Cinelog camera profile which removed the problem with highlights in red on arms and candle flame. We see the only solution here – to process red channel with a curve before debayering.

Here you can see an example of processing with Fast CinemaDNG:
https://yadi.sk/i/z3VXLSw13F2Uzr (https://yadi.sk/i/z3VXLSw13F2Uzr)
Everything was set to its default value, we just reduced Red channel down (it's applied before debayer) and changed R curve to avoid highlights crashing in R channel. RGB Parade would help you in making proper settings.

We expect to release some features for solving such problems (3D LUT support, highlights recovery, DCP profiles support, LUTs for RAW data) within 2-3 months and we will check your image after that once more.

I also agree that panel docking is not working well. We got that module from QT and it seems to be not the best choice.
Title: Re: fastcinemadng
Post by: hyalinejim on March 04, 2017, 09:43:35 PM
That sounds awesome - particularly DCP support!
Title: Re: fastcinemadng
Post by: megapolis on March 16, 2017, 11:37:31 AM
@hyalinejim: Thanks again for your image and your question. I've checked your image and your solution once more and I can see the following:

You've solved the problem by applying Cinelog DCP profile for Canon 5D. Inside that profile one can see ToneCurve (log-like gamma) and HSV 3DLUT. It means that in Adobe ACR the task of Highlights Recovery was solved after debayering by applying gamma to RGB and 3D LUT to HSV. That solution makes sense and we see good result. We will be able to utilize that method, as soon as we add DCP support to our software in the near future. Anyway, there is also another possible solution.

One can do Highlights Recovery before debayering and it seems to be a better approach. If we remove highlights before debayer, then image quality after debayering will be better because this is the way to avoid interpolations with clipped pixels. We've added to Fast CinemaDNG software widget “Raw Curve” to be able to apply curves to RAW data before debayering. There is a master curve which is always applied to all three channels of RAW data, and individual curves for each color channel.

Here you can have a look at your image which is processed with curves before debayering:
(http://www.fastcinemadng.com/pic/MLV/candle_1000.jpg)

What Fast CinemaDNG software is doing before demosaicing:
1. Multithreaded reading and parcing of all DNGs in the current folder
2. Multithreaded DNG decoding for compressed data
3. Data copy from CPU to GPU (host to device transfer)
4. DNG crop
5. Data linearization according to 1D LUT from DNG
6. Black and white points from DNG
7. WB coefficients for R, G, B
8. Exposure correction
9. Composite raw curve (the same curve for all three channels)
10.  Individual curve for each raw channel
11.  Raw bayer denoiser
12.  Debayer HQLI, DFPD or MG with Enhance option

All these stages from image processing pipeline one can see at Benchmarks widget to check timing on GPU for each stage. To get fast result, we need to have powerful CPU, GPU and SSD.

Please let me know your opinion about suggested pipeline and about the results of our software for DNG image processing both in terms of quality and performance. You can download the latest release of Fast CinemaDNG software with “Raw Curve” module from www.fastcinemadng.com
Title: Re: fastcinemadng
Post by: hyalinejim on March 16, 2017, 11:20:25 PM
@megapolis

I'm away for a few weeks but will check it out on my return looks good from the image you posted!
Title: Re: fastcinemadng
Post by: megapolis on April 27, 2017, 01:03:29 PM
@hyalinejim: In the latest release of Fast CinemaDNG Processor software we've implemented support for external DCP profiles. Now it's possible to utilize any DCP profile from Adobe ACR, RawTherapee, Cinelog, etc. User can create his own DCP profile with any profiler and then add it to the current project. All computations for 1D and 3D LUTs from DCP are done on GPU in real time. Timing for each stage of DNG image processing on GPU is shown in the Benchmark window.

We've also added check boxes for GUI to switch on/off  Hue/Saturation Map, LookTable and Tone Curve to see what we can really get from DCP profile. Default path for a folder with DCP profiles are defined at Options section.

The latest release of Fast CinemaDNG Processor can be downloaded here:
http://www.fastcinemadng.com/download/FastCDNG_x64_Setup.exe
Title: Re: fastcinemadng
Post by: Andy600 on April 27, 2017, 02:05:40 PM
@megapolis - I'm not sure where you got your Cinelog profile(s) from? - but Cinelog-C profiles don't contain HSV luts and they will only work correctly in Adobe Camera Raw.
Title: Re: fastcinemadng
Post by: hyalinejim on April 27, 2017, 02:27:00 PM
Hi megapolis, I've added my Cinelog DCP profile to the folder as you suggest, but all it seems to do is change the saturation:

(https://s2.postimg.org/8n2lpo1a1/new_01.jpg)

Here is what it looks like in ACR:

(https://s2.postimg.org/5u9e5n0xl/new_02.jpg)
Title: Re: fastcinemadng
Post by: Andy600 on April 27, 2017, 02:58:44 PM
Even though you can read/apply Cinelog profiles in RawTherapee or other apps they will only produce Cinelog-C colorspace in Adobe Camera Raw because the profiles contain compensation for a limitation that is unique to ACR when it is used in conjunction with After Effects. Using the profiles in any other raw app (i.e. raw apps without that limitation i.e. any raw app that isn't ACR) you will be introducing a new issue.

I'm not sure why you would even want to use a fixed colorspace management DCP anyway as your app is built on GPU accelerated shaders!?
Title: Re: fastcinemadng
Post by: Lars Steenhoff on April 27, 2017, 06:10:55 PM
Its free and gpu accelerated, will have to give it a try
Title: Re: fastcinemadng
Post by: DeafEyeJedi on April 27, 2017, 06:24:27 PM
@megapolis - I'm not sure where you got your Cinelog profile(s) from?

I question this as well?
Title: Re: fastcinemadng
Post by: megapolis on April 28, 2017, 09:38:18 AM
Quote
I'm not sure where you got your Cinelog profile(s) from? - but Cinelog-C profiles don't contain HSV luts and they will only work correctly in Adobe Camera Raw.
Thanks, I will check that. I tested profiles from ACR and RT. Cinelog profile was tested by our customer. If it doesn't work, I will definitely remove it from the list of supported profiles.

Quote
Even though you can read/apply Cinelog profiles in RawTherapee or other apps they will only produce Cinelog-C colorspace in Adobe Camera Raw because the profiles contain compensation for a limitation that is unique to ACR when it is used in conjunction with After Effects. Using the profiles in any other raw app (i.e. raw apps without that limitation i.e. any raw app that isn't ACR) you will be introducing a new issue.
Thanks for the info.

Quote
I'm not sure why you would even want to use a fixed colorspace management DCP anyway as your app is built on GPU accelerated shaders!?
That application is built on CUDA and we can utilize any colorspace management in realtime. Currently we are working on DCP support. What approach would you suggest to implement?
Title: Re: fastcinemadng
Post by: megapolis on May 31, 2017, 11:00:41 AM
Fast CinemaDNG Processor is currently used in the Aeon Motion Scanning System for 3D scanning and 4D capture with 20 MPix industrial cameras:
http://ir-ltd.net/introducing-the-aeon-motion-scanning-system/  (http://ir-ltd.net/introducing-the-aeon-motion-scanning-system/)

This is comparison with Adobe Lightroom for DNG image processing performance:
«FastVideo is lightning fast. It’s still in early development but it can process sequence data on the fly, on the GPU in real-time in ms, rather than in minutes per frame. This means we can now process a 28,800 image sequence set in under 10 minutes, instead of 10 hours. Literally a game changer.»
Title: Re: fastcinemadng
Post by: jankrueck on June 01, 2017, 11:10:51 PM
hey Im not home yet, but is the software suporting the "new" 4k raw?
(there where some changes on mlv_dump made to fix linedropping)

looking forward to give it a try! soundds great so far
Title: Re: fastcinemadng
Post by: megapolis on June 02, 2017, 12:03:35 PM
Could you please share a link to "new" 4k raw?
At the moment we support DNG/CinemaDNG only. MLV support is expected soon.
Title: Re: fastcinemadng
Post by: andy kh on June 02, 2017, 11:03:56 PM
supporting mlv would be interesting
Title: Re: fastcinemadng
Post by: jankrueck on June 03, 2017, 02:14:41 PM
Could you please share a link to "new" 4k raw?
At the moment we support DNG/CinemaDNG only. MLV support is expected soon.

Ah ok. I thought you'd allready support it. nvm. I'll wait :D
Title: Re: fastcinemadng
Post by: DeafEyeJedi on June 03, 2017, 11:18:08 PM
Could you please share a link to "new" 4k raw?
At the moment we support DNG/CinemaDNG only. MLV support is expected soon.

http://www.magiclantern.fm/forum/index.php?topic=19300.0
Title: Re: fastcinemadng
Post by: jankrueck on July 09, 2017, 11:22:54 PM
any news?
Title: Re: fastcinemadng
Post by: megapolis on August 11, 2017, 02:56:04 PM
In the latest release of Fast CinemaDNG Processor we've fully redesigned CUDA code for Histogram/Parade module and Denoiser (which is working before debayer). We have also implemented some more features for image processing on GPU:
1. Rotation to arbitrary angle in realtime.
2. Now we can work with LCP (lens profiles) to do undistortion and CA removal. We utilize ready LCP or prepare them from Adobe Lens Profile Creator.
You can download the latest version from www.fastcinemadng.com (http://www.fastcinemadng.com)
Title: Re: fastcinemadng
Post by: DeafEyeJedi on August 11, 2017, 06:49:00 PM
Thanks for the heads up @megapolis -- any chance that this will eventually run under Wine? I just tried again and unfortunately the same errors came up from when I first reported (http://www.magiclantern.fm/forum/index.php?topic=19021.msg180458#msg180458first reported). Is it really due to the lack of not having required NVIDIA CUDA installed on this MBP?

(https://farm5.staticflickr.com/4375/35695302413_8ecb63e404.jpg) (https://flic.kr/p/WogKNv)

Seems I'll be better off running an emulator if one wants to try this app on their OS X?
Title: Re: fastcinemadng
Post by: megapolis on August 15, 2017, 11:14:58 AM
Current version is working with Windows only. At the moment we need to design and to implement on CUDA quite a lot of algorithms for DNG image processing. Other OS will be a subject for implementation in the future, but not now.
Title: Re: fastcinemadng
Post by: 3Dto5D on September 19, 2017, 04:51:23 PM
Thats a really nice GUI and looks good too. Too bad I'm in Mac OS. I really would Love to use this on the Mac natively though If there was any chance. Thanks.
Title: Re: fastcinemadng
Post by: megapolis on November 09, 2017, 08:50:39 AM
New features in the latest release:
1. Project templates to save current set of parameters and settings. Now one can utilize that template to process next DNG series with the same parameters. Have a look at Project templates widget.
2. DNG player is working with audio from wav-file.
3. This is the first release with accelerated lossless jpeg decoder on CPU. We will publish more info about that soon.

Download link to Fast CinemaDNG Processor:
http://www.fastcinemadng.com/download/FastCDNG_x64_Setup.exe (http://www.fastcinemadng.com/download/FastCDNG_x64_Setup.exe)
Title: Re: fastcinemadng
Post by: jankrueck on December 02, 2017, 11:56:17 AM
Hey!

I've been playing around with this. again.
And sorry to say, but I dont get it. All of this is available in Resolve or in ACR as well.
To me it feels kinda bulky to import and export files.

I don't want to spread heate, just saying, I dont get it :D
Initianly I was following this Thread for your mlv inApp support.


Anyways, I', still looking for a descent way to handle mlv/4kraw without debayering to 1238234 dng files ;)


keep it up, Jan
Title: Re: fastcinemadng
Post by: megapolis on December 06, 2017, 10:58:25 AM
It's difficult to agree that
Quote
All of this is available in Resolve or in ACR as well.
Just try to play DNG series with 4.6K resolution and Denoiser option on, you could hardly get smooth video output with ACR. It's much better with Resolve, but still this is a problem.
The most frequent usage of that software is preview of DNG series. You just need to choose a folder with DNG in Windows Explorer to see realtime video. This is fast and simple. The latest release is working with our new CPU-based Lossless JPEG decoder which is faster than decoders from dcraw, libraw, lj92, Adobe DNG SDK.
P.S. Unfortunately MLV support is not yet ready.
Title: Re: fastcinemadng
Post by: IDA_ML on December 06, 2017, 12:39:38 PM
Megapolis,

I was wondering if it might be possible to just add MLVFS to your software.  Although I agree that the preview quality of the DNGs is great, it simply takes to long to open a DNG folder and this really makes it too heavy to use unless you really have a very beefy videocard.  Opening MLV files directly would be a way better option, especially if this could be done fast.  MLVFS would help in that respect if it could be implemented.  Just a thought.
Title: Re: fastcinemadng
Post by: megapolis on December 07, 2017, 10:46:53 AM
As I know, usually there is no need to have beefy video card for image processing of DNG images with resolutions less than 2K. If I haven't understood you correctly, please advise. If our software is working slow, please send me a link to your MLV file for evaluation.

You can test your MLV files by running MLVFS and Fast CinemaDNG Processor. What you need in Windows-7/10 (64-bit):
1. Download Fast CinemaDNG Processor: http://www.fastcinemadng.com/download/FastCDNG_x64_Setup.exe
2. Download Dokany: https://github.com/dokan-dev/dokany/releases/download/v1.0.3/DokanSetup_redist.exe
3. Download MLVFS: http://www.dropbox.com/s/4ymbt8xxfrwtn1h/MLVFS.zip?dl=0
4. Run the following command: mlvfs -f Z: --mlv-dir=\mlvdir

Then from Windows Explorer you can click with right button on your MLV file and choose in the context menu “Preview with Fast CinemaDNG”. Actually we haven't done anything here. Transform from MLV to DNG is done by MLVFS.
Title: Re: fastcinemadng
Post by: megapolis on December 11, 2017, 11:00:28 AM
Sorry, the latest release of Dokany could be downloaded here:
https://github.com/dokan-dev/dokany/releases/download/v1.1.0/DokanSetup_redist.exe

This is a benchmark for GPU-based image processing of MLV which was loaded via MLVFS, processing is done with Fast CinemaDNG Processor on NVIDIA GeForce GTX 580 (PCI-Express x16 Gen2):

Total memory 1536 MB, free 179 MB, allocated 109 MB
Input image: 1280x720 pixels
Host-to-device transfer = 0.47 ms
Linearization LUT = 0.35 ms
White balance = 0.03 ms
Raw curve = 1.06 ms
Debayer = 1.15 ms
ProPhoto space transform = 0.09 ms
RGB Lut = 0.21 ms
Output color space transform = 0.09 ms
Crop time: 0.00 ms
Resize = 0.00 ms
16 to 8 bit transform = 0.06 ms
Histogram = 1.20 ms
RGB parade = 2.31 ms
Viewport crop = 0.00 ms
Viewport resize = 0.88 ms
Total GPU = 7.90 ms
Total GPU + CPU = 10.95 ms

P.S. Could you please give me a link to 4K MLV footage for testing?
Title: Re: fastcinemadng
Post by: IDA_ML on December 11, 2017, 06:05:12 PM
P.S. Could you please give me a link to 4K MLV footage for testing?

Megapolis,

I have prepared a short (84 frames) 3K MLV file (3072x1920) for you to test.  It is about 590 MB and you can download it from here:

https://we.tl/mdeBlxLtvx

This download link will be active for 7 more days.

It takes about 20s. to open this file with FastCinemaDNG from Explorer with MLVFS installed as you suggested above.  If I want to view the next MLV file, I have to close the first one and if I forget to do this, the software crashes.  The computer that I used for this test and for editing my videos has a Q6600 CPU, 8GB of RAM and a GTX 750Ti GPU (2GB DDR5 RAM).  I am on Win7x64. An opening time of  20 s. for such a small file is way too long.  MLV Producer opens the same file in about 3 s. Moreover, in Producer I can open a whole batch of MLV files in just a few seconds.  I have just tested it with 15 MLV files (a total of 8 GB) and it created thumbnails of all of them within 9 s. after dragging them from Explorer onto the MLVProducer icon.  Once the thumbnails are in the preview window and I click on each of them, the MLV file opens in full size almost instantaneously.  So, if I want to browse between lots of MLV files, I prefer to use Producer since it is so fast and easy to use.  I use FastCinemaDNG only if I want to view a particular MLV file at a high quality and real-time playback.

Conclusion:  FastCinemaDNG is great in terms of preview quality and fast playback, as well as editing options but slow and not convenient to work with.  Converting files using the FFMPEG option is a pain.  This would be a great software if it would have been faster, better organized and would offer convenient and fast export/converting options.

I hope, this feedback is useful and wish you great success in making your software more user friendly to work with. 
Title: Re: fastcinemadng
Post by: megapolis on December 12, 2017, 11:56:19 AM
Thanks a lot for your MLV file and for your feedback. On my PC (Core i7-3820 3.6 GHz, RAM 16 GB, NVIDIA GeForce GTX 580), which is quite old, it takes 5-6 seconds to load your project. I agree that 20 s is not acceptable, but generally, for DNG image processing we need to have good CPU, GPU and SSD, so the PC should be quite powerful. We always load new project without offloading previous one. We will check why it causes crash.

The first run of each software takes around 2-4 seconds, so it's not surprising. For GPU-based image processing we also have to initialize CUDA and allocate GPU memory, which are not very fast procedures. That's why 5-6 seconds for such a task at the first run should be reasonable. To insure fast loading for the next project we need to optimize our software and I hope to cope with that in a couple on weeks.

Interface to work with FFmpeg is quite simple, you just need to put there right command line. We have prepared a list of frequently used command lines, so it should be not very difficult, you can find that info in the manual.

The main idea of our software is to insure fast image processing and smooth output for 4K with maximum possible options like high quality debayering, denoising, undistortion, geometry transforms, 3D LUTs, etc. We think that it could be done on GPU in realtime with GeForce GTX 1060 or better.
Title: Re: fastcinemadng
Post by: IDA_ML on December 12, 2017, 05:50:33 PM
Megapolis,

Maybe, I should share another thought about FastCinemaDNG with you.  Working only with folders full of cDNG sequences is not a good idea at all.  This means that you have to drag all your MLV files that you wish to view, to a MLVFS folder and apply MLVFS to them.  This additional step is impractical and takes time and a lot of disk space.  In most cases, your MLV-files are spread in hundreds of directories on different internal and external drives.  What if you want to quickly view a specific MLV file or find it among 100 others?  Do you want to convert all of them into cDNG folders first?  This is very impractical.  It would be much easier if you could just click on the MLV-file and open it immediately.  This is how Producer works.  All you need to do is to set the file association .MLV to open with Producer. 

If you could add MLV support to FastCinemaDNG, this would make it much more user friendly.  Think about it.
Title: Re: fastcinemadng
Post by: megapolis on December 13, 2017, 08:25:56 AM
IDA_ML,
Thanks for sharing your thoughts. This is exactly what we are doing now and we hope to release direct support of MLV format soon. We are going to open MLV instantly from Explorer without running MLVFS and without convertion to cDNG.
Title: Re: fastcinemadng
Post by: IDA_ML on December 13, 2017, 04:09:48 PM
This is great, Megapolis!  May I make another, in my opinion, useful suggestion?  Could you possibly modify Fastcinemadng in such manner that the user could navigate to a folder with MLV files in Explorer and by clicking on that folder, Fastcinemadng would create inside Explorer a window with thumbnails of all MLVs inside the folder.  This thumbnail creation should happen very fast, as in Producer.  Once the user finds a  thumbnail of interest, he clicks on it and the MLV file opens in the full sized software window, ready for playback, adjustments, quick export, etc.  If the user wants to repeat the same thing with another MLV, he just clicks on its thumbnail and the file opens in the full sized window.  In other words, Fastcinemadng should work as any photo viewer (such as IrfanView for example), but with MLV-files, instead of JPEGs or CR2 files.  You don't even have to animate the thumbnails as DaVinci Resolve does.  All you need to do is to use a single frame from somewhere in the middle of MLV clip to create its thumbnail.  It should just give a hint to the user what that clip is all about and I think this would be very helpful and user friendly already.  Do you think, this would be possible?
Title: Re: fastcinemadng
Post by: megapolis on December 14, 2017, 11:21:53 AM
Thanks for your suggestions. We do have such plans, but we are going to implement them in a way of Windows Explorer shell extension, that would generate thumbnails for MLV files while viewing folder content. It shouldn't be time consuming procedure as soon as image processing pipeline for such a task is very simple. The same feature is implemented in Adobe DNG Codec for Windows. It will be done later, because currently we are working on MLV support and on fast DNG decoding.
Title: Re: fastcinemadng
Post by: megapolis on December 19, 2017, 09:25:48 AM
When we do GPU-based image processing with Fast CinemaDNG Processor software, we have to bear in mind that performance of CPU and SSD also play an important role in this process, since for any image we have to read, to download, to parse, to decode DNG, and only then we can upload uncompressed raw data into GPU memory for image processing and display.

The main bottleneck for CPU is fast DNG decoding. Typically DNG images are encoded with lossless compression algorithm and usually this is Lossless JPEG.
Lossless JPEG algorithm is essentially serial, so GPU can't help at decoding. To speed up decompression on CPU, it is possible to do decoding of each tile or of the entire image in a separate thread, and one can accelerate Huffman decoding algorithm on CPU.

We have implemented both methods, so we can do lossless jpeg decoding at multi-threaded mode, and we have also optimized the process of DNG decoding on CPU. It's difficult to say how fast that new DNG decoder, because decoding performance strongly depends on image content. Here you can see some benchmarks which correspond to the best and the worst cases of Lossless JPEG decoding for multithreaded applications. These examples illustrate the idea of multithreading performance for lossless jpeg decoding on multicore CPU.

16-bit image, compression ratio 10.4 bpp (lossless compression)
   LJ92 (library liblj92): 266 MPix/s
   Fastvideo LJ Decoder: 407 MPix/s

12-bit image, compression ratio 5.6 bpp (lossless compression)
   LJ92 (library liblj92): 284 MPix/s
   Fastvideo LJ Decoder: 475 MPix/s

These results show that fast DNG decoding on CPU is possible in realtime for DNG series with 4K resolution and more. Decoding optimization, vectorization and multithreading are key factors to achieve high performance decoding.

At the following link there is more detail concerning benchmarks and other info about lossless jpeg decoding on CPU:
http://www.fastcinemadng.com/info/jpeg/lossless-jpeg-decoder.html

Our new DNG decoder helps to reduce CPU load. Earlier, due to lack DNG decoding speed on CPU, we ran into problems with smooth video playback for 4.6K footages from BMD URSA, and now we don't have these jerks on good PC even in the case if we switch on gpu-based denoiser.