Author Topic: Compressing raw video files to save hard drive space  (Read 3595 times)


Doyle4

  • Hero Member
  • *****
  • Posts: 690
  • Canon 5DRAWii & 600Dual-Iso
Re: Compressing raw video files to save hard drive space
« Reply #1 on: August 03, 2013, 06:23:50 PM »
Nice find.

Also, on MAC, after compressing the dng's into a folder you could also put them into a compressed DMG :)

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3157
Re: Compressing raw video files to save hard drive space
« Reply #2 on: August 03, 2013, 07:39:40 PM »
for the new .mlv video format there is also a concept for data compression on pc-side.
its not implemented yet, just a pure concept.

1) remove any NULL frames that were needed for maximum speed and alignment in write process (its just 1%)
2) crop the area specified in active_area of raw_info (i.e. remove borders)
    -> this makes black, white level and wb harder? (alex?)

3) compress the frames using a LZMA compression (they will take only 75% then)
    -> option: align the pixels in 16 bit words instead of 14 to improve compression?
4) reduce bit depth to 13, 12, 10, ... bits
5) re-order the frames to be in consecutive order
    -> compression improvement? at least post processing is easier


this way i expect the file sizes to go down by at least 30%, maybe even 50%
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Doyle4

  • Hero Member
  • *****
  • Posts: 690
  • Canon 5DRAWii & 600Dual-Iso
Re: Compressing raw video files to save hard drive space
« Reply #3 on: August 03, 2013, 07:41:57 PM »
@g3gg0

Thanks for that, i also use windows now n again so good to know.

mageye

  • Senior
  • ****
  • Posts: 296
  • Atheist
Re: Compressing raw video files to save hard drive space
« Reply #4 on: August 03, 2013, 09:07:07 PM »
I have already been through the process of using the Adobe DNG Converter to convert cinema DNG back to just DNG. It does indeed save a lots of space. That is certainly welcome.

However I would NOT use 'lossy'. AT ALL

The comment in the guys post: Part 1: http://thejohnha.blogspot.com/2013/07/compressing-canon-5d-raw-video-to-save.html

"In the world of moving pictures, the use of lossy DNG compression will not affect the end result at all"

I disagree and so do others. Check here:

http://forums.adobe.com/message/5425932

His comments are at the very least debatable and basically it does have issues. I would be careful about using lossy as you will never get it back. After all it is lossy which means you will loose something.

Of course this is just my opinion and it is totally down to your own judgement if you want to use lossy. It's a trade off I suppose
5DMKII | 500D | KOMPUTERBAY 32GB Professional 1000x |Canon EF 50mm f/1.8 II | Samyang 35mm f/1.4 ED AS UMC | Canon EF 75-300mm f/4-5.6 III | Zoom H2 (4CH. audio recorder) | Mac OS X 10.9.2 | Photoshop CC | After Effects CC | Final Cut Pro 7

Sganzerla

  • New to the forum
  • *
  • Posts: 43
Re: Compressing raw video files to save hard drive space
« Reply #5 on: August 04, 2013, 01:03:37 AM »
I'm storing the .raw files, should I store and backup the .dng ones instead?

I may be very wrong but if I remember well there were differences in how the versions of RAW2DNG were dealing with pink spots and banding.