Kinochrome - GPU accelerated raw video processing

[doktorkrek]

Kinochrome is an app written in Rust using cross-platform Vulkan API and the fast lossless JPEG decoder to process raw video on the GPU.

Here, it plays a 2560x1440 losslessly compressed MLV video on a 2012 ThinkPad 430 with integrated GPU in real time:

Features

• MLV and CinemaDNG import (uncompressed and lossless)
• Debayering and color grading done on GPU via Vulkan compute shaders, rendered to the UI from the Vulkan image with no overhead
• Supports ffmpeg for export, with customizable recipes for video and audio encoding
• Fast multi-threaded CinemaDNG (losslessly compressed) export

Download
Builds for Linux, macOS and Windows

The source code is available on SourceHut (with a GitHub mirror) under the free and open source AGPL license. The project is currently in an alpha state, with no builds available, but the interested people can build it from the source following the instructions.

Feedback and reporting bugs is encouraged!

[names_are_hard]

Sounds interesting!

Note for devs: AGPL is a difficult license to work with.  Do not use any of this code without being well informed about how this license functions.  Using the code may force your code to be re-licensed in ways you don't want.  All companies I have worked for ban use of AGPL licensed code.

[doktorkrek]

I'm aware of this. I'm statically linking with a DNG library that uses AGPL, so I don't have much choice.

[names_are_hard]

Hah, well, you're choosing to get screwed by it, I wanted to give other people a chance to avoid it.  It's a weird license and surprises many people.

[doktorkrek]

Come on, it's unnecessary to say that I'm "choosing to get screwed by it".
The only significant difference from the GPL is the (somehow controversial) clause 13 about remote network interaction, which doesn't apply, as the program doesn't have a network functionality. I don't mind that it makes it difficult for companies to use my code to build some SaaS.

The more reusable parts of the code, like the MLV parser/writer and lossless JPEG encoder/decoder are and will be published as standalone libraries under more permissive licenses.

Using the code may force your code to be re-licensed in ways you don't want.

Do you mean something else besides the clause 13?

[names_are_hard]

Well, how screwed or not you are is up to you to decide, my main point was the fact you chose it.

It's quite a curious license.  I don't think it does what the FSF want it to do, SaaS like providers can easily use AGPL software and avoid following the obligations.  It's a very unpopular license.  There's a lot of legal uncertainty around it.  It's just not very well written, regarding what "remote network interaction" means, and what the effects should be.

If you did this video processing on a server, and provided an API, do you have to provide the source?  Probably yes.  How do you send the required notification about source availability to the user?  With every file processed?  What if you wrote an AGPL TCP/IP stack?  Every packet?  Now what happens if I put a proxy in front of your service, and don't pass on the notification?  Presumably I'm not bound by the copyright license, so I don't have to provide any notification, or source, and in any case I can't provide the source, since I don't have access to your server.  But I'm allowing users to interact with AGPL licensed code over a network.  The license would like my code to become AGPL and be released, but I'm not making a copy of any AGPL code, so why should I be bound by it?  And so on.  This is all very clumsy and difficult to reason about.

Re "using the code may force your code to be re-licensed in ways you don't want." - this is standard GPL virality / license incompatibility.  E.g. if you choose to use (significant) portions of AGPL code in your code, you're now supposed to license all of that code as AGPL.  So now you get all the rest of the terms that you might not have had before.

Anyway, didn't want to sidetrack - I just wanted to point out to people with some familiarity with software licensing that AGPL is not a "normal" GPL license, and you need to think hard if you really want to use it.

[iaburn]

Can't wait to try! Another MLV tool is always welcome 

[vastunghia]

Super cool! A few questions:

Does the fact that it relies on Vulkan mean that it should support AMD GPUs and not only nVidia? I think that would make it the first ever GPU-accelerated MLV viewer / editor for AMD owners.

What is the performance with lossless compressed raw? Isn't LJ92 decompression of a single frame difficult (if not impossible) to parallelize?

Any ETA?

Cross-OS?

Thank you!

[doktorkrek]

Does the fact that it relies on Vulkan mean that it should support AMD GPUs and not only nVidia? I think that would make it the first ever GPU-accelerated MLV viewer / editor for AMD owners.

Yes, AMD, Intel (including integrated) and Apple (with MoltenVK or on Asahi Linux) are all supported.

What is the performance with lossless compressed raw? Isn't LJ92 decompression of a single frame difficult (if not impossible) to parallelize?

Yes, the Huffman decoding is notoriously sequential. Nevertheless, it is fast enough to allow real-time playback of 1440p MLV videos even on old Intel Ivy Bridge CPUs and 4K on more modern ones.

Any ETA?

I'm polishing things for the first alpha release, hopefully this week.

Cross-OS?

Tested on Linux so far, but there should be nothing preventing it to run on Windows and macOS (with MoltenVK), potentially even on Android and iOS. Testing will be welcome.

[vastunghia]

Testing will be welcome.

Whoah, count me in! Intel iMac with AMD Radeon RX 6800 eGPU (or AMD Radeon Pro 560X built-in discrete GPU if you prefer), running both MacOS and Win11. Can't wait!

Must confess I'm a bit skeptical on performance with the typical format I use (14-bit lossless compressed raw, 25 fps, 3.7K @1:2.39 aspect ratio). Afraid CPU (i5 Coffee Lake in my case) will be a bottleneck. But anxious to see!

[Jip-Hop]

Looking forward to try this on my MBP M2 Max!

[doktorkrek]

Added a link to the source code to the main post. The interested people are encouraged to build it and report bugs.

In particular, I currently have no Windows nor mac machine to test on, but it should be working there. On macOS, MoltenVK is needed to run Vulkan programs; I will add the instructions once it runs somewhere.

[Skinny]

Wow, this looks really promising. I hope we will see a release soon

[doktorkrek]

I've created a GitHub action to generate builds for Linux, macOS and Windows, both Intel/AMD and ARM architectures.
link
Feel free to give it a dry and report issues.

Note on a Mac, one needs to install Vulkan SDK in order to run this app. I don't have a Mac to test it on, so please report if this doesn't work.

[onlykhaz]

It works!, But when you export it doesn't have the fix focus dots or the dead pixels, It would be cool to import the receipts like on mlv. Apart from that, its very fast   


Edit: I was missing the .fpm file for my pixel_maps folder, once there it worked perfect.
Also, you need to download the recipe folder, in the zip is missing.

[doktorkrek]

Thanks, I added the missing pixel maps and the recipes folder into the release archives

[ML700D]

Thanks, I added the missing pixel maps and the recipes folder into the release archives

how to install or use in windows? I didn't see anything when run kinochrome.exe

[doktorkrek]

how to install or use in windows? I didn't see anything when run kinochrome.exe

Have you first extracted the archive and then run it?

[ML700D]

Have first extracted the archive and then run it?

yes..

[doktorkrek]

yes..

I see. Can you run the program in the command line and post the output?

[iaburn]

For me it worked very well (nvidia rtx 3070), very smooth playback and export is reasonably fast
It would need more color grading options to be an alternative to MLVApp, but it's very promissing already 

[ML700D]

I see. Can you run the program in the command line and post the output?

https://i.imgur.com/mPNSjAP.jpeg

[masc]

Note on a Mac, one needs to install Vulkan SDK in order to run this app. I don't have a Mac to test it on, so please report if this doesn't work.

Why that? Isn't there a way to include/deploy the needed libs in the package, as usual? Looks like "libvulkan.dylib" has to be deployed somehow.

What debayer algorithm(s) do you use on GPU?
Edit: Ah, I found out: MHC? We tried MHC in a GPU based MLVApp many years ago, too.

[doktorkrek]

Why that? Isn't there a way to include/deploy the needed libs in the package, as usual? Looks like "libvulkan.dylib" has to be deployed somehow.

Indeed there is, it should be straightforward to package it in an .app bundle, I just currently don't have a Mac machine to test it on. Contributions are welcome

What debayer algorithm(s) do you use on GPU?
Edit: Ah, I found out: MHC? We tried MHC in a GPU based MLVApp many years ago, too.

Yes, currently it's MHC. I plan to add an option for RCD, which is the default algorithm in Darktable and for which there already is a Vulkan shader written for the vkdt project. RCD should be the default for exporting videos, I need to test whether it's suitable for preview, but at least on more contemporary GPUs, it shouldn't be a problem.

[doktorkrek]

https://i.imgur.com/mPNSjAP.jpeg

Maybe try in PowerShell:

Code Select Expand

./kinochrome.exe | echoAlso what GPU do you have? I suspect Vulkan drivers are missing.