Need help/looking for anyone interested in writing a script/app

[Milk and Coffee]

Hey all!

I've recently purchased a Sigma FP and am shooting UHD 12bit RAW cDNG. I use my Canon 5DIII with Magic Lantern shooting RAW as a B-Cam. The Sigma FP does not apply lossless LJ92 compression out of the camera, and I love that MLV APP allows me to export Lossless DNG for my Canon MLV RAW. I have processed the Sigma FP files with Adobe DNG Converter to apply lossless compression, but I lose all cinemaDNG-associated metadata in the process (timecode, color matrix, profile hue sat map, and bit depth.) Also, the audio must have embedded timecode, and thus does not sync with my Adobe-DNG-converted sequence. TLDR, Adobe DNG Converter does the trick to apply lossless compression, but comes with many caveats for cDNG.

Thus I'm determined to learn enough coding to write a script that can import cDNG, apply lossless LJ92 compression, and thats it. Leaving all metadata intact, and no conversion to higher bit depths.

I found this to be helpful in understanding the compression: https://thndl.com/how-dng-compresses-raw-data-with-lossless-jpeg92.html

I've spoken with a member of this forum on tips how to do this. The following was their instructions:

"First you would need to make a python extension allowing you to use the LJ92 library: https://github.com/ilia3101/MLV-App/tree/09a726ef66f693f29dd551534f7340cadb1c1955/src/mlv/liblj92. MlRawViewer had code for this, but it is for Python2 (now deprecated) and includes extra functions for bayer->RGB conversion. You would need to start from this file: https://bitbucket.org/baldand/mlrawviewer/src/master/bitunpack.c and cut it down to just the LJPEG parts + update for Python3 as required.

Python setup.py file: https://bitbucket.org/baldand/mlrawviewer/src/master/setup.py At this point I would also suggest updating the LJ92 to the version in mlv-app, but it may have some slight interface changes. Then you will need to update the DNG.py file for Python3: https://bitbucket.org/baldand/mlrawviewer/src/master/DNG.py After that you should have the needed parts to make a script that can read existing cDNG files, compress them, and write out a copy."

I have started a repo here: https://github.com/devonstanczyk/Digital-Negative-Lossless-Compression.git

I am very new to this, so please any help/collaboration is appreciated! Also, if I'm stepping on any toes, please know it's unintentional and let me know!

Thanks all!

[a.sintes]

why are you specifically searching for LJ92 compression?
does any alternative lossless compression algorithms supporting 12bits depth fits your initial need?
do you really need to work with lossless media or a professional lossy compression enough for you post-production requirements?

depending of these answers, I guess you can maybe avoid a lot of struggle just by using standard tools like ffmpeg/Resolve, alongside JPEG2000/x265/AppleProRes4444 algorithms?

[Danne]

This is also working:
https://www.slimraw.com/

[Milk and Coffee]

why are you specifically searching for LJ92 compression?
does any alternative lossless compression algorithms supporting 12bits depth fits your initial need?
do you really need to work with lossless media or a professional lossy compression enough for you post-production requirements?

depending of these answers, I guess you can maybe avoid a lot of struggle just by using standard tools like ffmpeg/Resolve, alongside JPEG2000/x265/AppleProRes4444 algorithms?

I want to preserve RAW data as after lossless compression, it's file size is not much larger than ProRes422.

This is also working:
https://www.slimraw.com/

Danne, this is extremely helpful! But was hoping to have a free solution.

[Milk and Coffee]

To help clarify why i'm so set on coding this myself: I'm wanting to learn python so that I can start creating DCTL's in DaVinci Resolve. I thought this would be a good challenge to help with that process.

[names_are_hard]

It can be good to have an end goal in mind when learning to program, but from experience teaching people, you don't want to start with a challenge.  Learning to program is enough challenge on its own, initially.

Start with standard boring tutorials (text, not video).  Learn how to use python first, write a range of simple but functioning programs, then consider starting on your actual goal.  If you're confused by python *and* your goal, it's much harder to learn.  Keep the problems simple so you're sure you understand them, while you're still getting over the confusion around programming itself.

[Danne]

Chatgpt is a good friend explaining functions and what is going on in syntax and code in general.

[names_are_hard]

Sadly, ChatGPT is also drunk, and a liar.

[Danne]

😂.
Still very useful imo. Especially in company with other sources. I learnt Lua to the extent I could build a lightoom app more or less entirely cooperating with chatgpt.

[Milk and Coffee]

It can be good to have an end goal in mind when learning to program, but from experience teaching people, you don't want to start with a challenge.  Learning to program is enough challenge on its own, initially.

Start with standard boring tutorials (text, not video).  Learn how to use python first, write a range of simple but functioning programs, then consider starting on your actual goal.  If you're confused by python *and* your goal, it's much harder to learn.  Keep the problems simple so you're sure you understand them, while you're still getting over the confusion around programming itself.

I appreciate all the advice! And I agree.

Chatgpt is a good friend explaining functions and what is going on in syntax and code in general.

I thought about doing this to help! - What did you prompt ChatGPT with? A question, then pasted the code?