Current state of MLV-or-cDNG compression utilities?

Started by adrjork, August 07, 2019, 05:56:41 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

adrjork

Hi everyone, newbie here.
I'm facing the common problem of storage limits for MLV and cDNG files.
I read some topics in this forum about different methods/softwares, but there are a lot of informations all together and I can't understand which one matches my workflow better (that is strictly based on Davinci for color grading).
Up to now I converted all my 14-bit MLVs to 16-bit cDNGs with Switch, and then Davinci. I storaged MLVs for safety reason, and also cDNG for Davinci projects, and this means a LOT of storage... Too much for me.

Now, I've seen at least 3 different solutions for my storage problem:

1. MLVs to compressed MLVs via Switch (LJ92 option), then deleting original MLVs, then compressed MLVs to Davinci via MLVFS.

  • Q1: What do I actually lose compressing MLVs? Grading quality in Davinci somehow? Speed performance in Davinci?
  • Q2: Is it "safe" keeping only compressed MLVs? Could it happen that Switch introduces some error or similar?
2. MLVs to compressed DNGs via SlimRAW, then deleting original MLV, then Davinci.

  • Q3: What do I lose with "lossless compressed" DNGs against "full" DNGs? Again, grading quality or speed performance in Davinci?
  • Q4: Nobody in this forum recommend keeping only DNGs and deleting original MLVs, why?
3. Simply keeping original MLVs, then on-the-fly to Davinci via MLVFS.

  • Q5: MLVFS is on-the-fly, does it mean a slowdown in Davinci? And the grading is really as accurate as with rendered DNGs?
  • Q6: Is there some feature in MLVFS that is unaccurate in comparison with Switch?
Which is in your opinion the most "safe" and "high quality" solution? (Because – as someone else wrote in this forum – I have OCD for quality loss ;D and the word "lossless" is not reassuring to me...)

Thanks a lot for your help.

Walter Schulz

All file compression utilities (for ZIP, RAR, and all other compressed file formats) are using lossless compression. Ever heard of those utilities generating volatile output? Nope, because their behaviour is deterministic, reproducable and reversible.
Lossless means lossless.

adrjork

Thanks Walter for your reply.
Quote from: Walter Schulz on August 07, 2019, 06:40:09 AM
Lossless means lossless.
Well, sometimes lossless could mean "perceived quality lossless" that's not exactly reversible. This is one of my doubts.
Moreover, in the SlimRAW dedicated thread, yobarry asked "I read in this thread many users advising to keep the original MLV files, is this because later advances/improvements to programs such as MLVFS will increase the quality of the original CDNGs?"... And I share this doubt too.

Anyway, for compressing DNG files there is SlimRAW but also Switch has an option to compress DNG files with LJ92. Which is the difference between these two type of compression?
And the same Switch has a MLV compression option too! So the question is: in which case you recommend MLV compression over DNG compression with Switch over DNG compression with SlimRAW? Which one you prefer and why?

So, there are many similar workflows to obtain the same result: saving space in the RAID. I'd strongly like to know which are PROs and CONs of each one.

Thanks for your help.

masc

Compressed MLV uses LJ92.
Compressed DNG uses LJ92.
For a MLV you can easily do RAW Corrections (with some ML tools), for DNG you can't (yet, because of the lack of tools).
5D3.113 | EOSM.202

adrjork

Thanks a lot masc!
Quote from: masc on August 07, 2019, 10:18:58 AM
For a MLV you can easily do RAW Corrections (with some ML tools), for DNG you can't (yet, because of the lack of tools).
This is a very good reason to keep the MLV files. One doubt gone forever :)
Now, just few doubts remain to me:

1. LJ92 does "visually lossless" or "actually data lossless" compression of MLV and DNG? (Forgive me for inaccurate terms.)

2. Is safe storaging compress MLVs and deleting original MLV? (Can I always recover the original MLV from a compressed one?)

3. Is it better doing 14-bit DNG lossless compression with SlimRAW or with Switch-LJ92?

4. Lossless compressed 14-bit DNGs can somehow slowdown Davinci (in comparison with regular 14-bit DNG)?

5. Is it possible that MLVFS' on-the-fly DNGs slowdown Davinci (in comparison with rendered DNGs)?

Thanks really a lot.

masc

1. Lossless in the meaning you can 100% recover original data.
2. If no error (e.g. by a bug) happens on transcoding, yes.
3. If slimRAW is lossy, LJ92 is better in terms of quality. But I don't use SlimRAW, as it is a commercial product, while LJ92 is free.
4. Depends on your system, if it is noticable. Compressed data must be decoded by some processor... this mostly needs time. Therefor you safe disk space.
5. Depends on your system, if it is noticable. Transcoding data must be done by some processor... this mostly needs time. Therefor you safe disk space.
5D3.113 | EOSM.202

adrjork

...sorry masc... I repeated the same question 2 times... I need some sleep...
Quote from: masc on August 07, 2019, 10:47:56 AM
2. If no error (e.g. by a bug) happens on transcoding, yes.
And it should be directly visible? (MlRawViewer can see compressed MLV, right?)

Anyway, which MLV workflow do you personally recommend?

masc

Quote from: adrjork on August 07, 2019, 12:00:16 PM
And it should be directly visible? (MlRawViewer can see compressed MLV, right?)
Nobody knows... this question is very theoretic.

Quote from: adrjork on August 07, 2019, 12:00:16 PM
Anyway, which MLV workflow do you personally recommend?
When working with them I use uncompressed MLV clips because of speed reasons, for archiving I compress my clips to lossless MLV and I am happy with it.
5D3.113 | EOSM.202

adrjork

Quote from: masc on August 08, 2019, 03:35:40 PM
When working with them I use uncompressed MLV clips because of speed reasons
You use uncompressed MLV: in the sense that you use MLVFS to "virtualize" DNG on-the-fly, or in the sense that you don't convert to DNG at all?

masc

Mostly I don't use DNG at all. Sometimes I do and work with MLVFS on the fly & Resolve.
5D3.113 | EOSM.202