Author Topic: ML Raw vs. Log  (Read 6661 times)

franciscolobo

  • New to the forum
  • *
  • Posts: 2
ML Raw vs. Log
« on: July 10, 2013, 07:36:06 PM »
I was wondering. Raw video at 14bit takes up and extreme amount of disk space.
Technicaly, Log can get about 95% of the information and about 10% disk space.
So, would it make sense to capture instead of a stream of .png 14bit images from the camera, something like a .jpeg 12bit log (cinestyle for example) stream of images?
That should have some pretty nice results with a lot less disk space.


To what point would that be possible and not only reduce disk space usage but also allow slower cards and possibly faster framerates.


Has anyone thought of that or tried to develop it?
Do you guys think it would make sense?

I know nothing about programming and the insides of ML but if someone would be interested in trying/developing this with me I think it could go somewhere.

Thanks

NedB

  • Freshman
  • **
  • Posts: 85
Re: ML Raw vs. Log
« Reply #1 on: July 10, 2013, 08:47:24 PM »
@franciscolobo: What I have gleaned from reading this forum somewhat comprehensively for the past few months is that, while this and other similar ideas could work theoretically, none of the cameras has a CPU (or DIGIC processor) which is powerful enough to record raw footage and simultaneously convert it into another form. It is not a question of development, there is simply not enough remaining hardware capacity to do the job. Although at the same time I must say that all the devs laughed themselves silly a year or so ago at anyone who suggested we could one day have raw video. I believe the difference is that the raw video was in a sense always there, just waiting for A1ex, g3ggo, 1%, nanomad, etc., to figure out how to grab it and write it to the SD and/or CF cards. There is no extant stream (as far as we know, but....) of raw video which has already been converted to some other, smaller form, other than the H.264 output from the dedicated chip which is the result when shooting normally (i.e., non-raw). So far, the only possibility which seems doable to the devs (but I'm not sure if anyone is still working on it) is to reduce the bit-depth of the raw video from 14-bit to 12-bit or 10-bit, which reductions would bring only 14-29% in bitrate reduction, not enough to make the difference we need, especially in the older cameras like the 550D. Cheers!
550D - Kit Lens | EF 50mm f/1.8 | Zacuto Z-Finder Pro 2.5x | SanDisk ExtremePro 95mb/s | Tascam DR-100MkII

eatstoomuchjam

  • New to the forum
  • *
  • Posts: 41
Re: ML Raw vs. Log
« Reply #2 on: July 11, 2013, 12:45:45 AM »
I strongly suggest reading the forum and finding the several posts with similar ideas and their reception before creating another one with the same idea.

I was wondering. Raw video at 14bit takes up and extreme amount of disk space.
Technicaly, Log can get about 95% of the information and about 10% disk space.
So, would it make sense to capture instead of a stream of .png 14bit images from the camera, something like a .jpeg 12bit log (cinestyle for example) stream of images?
That should have some pretty nice results with a lot less disk space.


To what point would that be possible and not only reduce disk space usage but also allow slower cards and possibly faster framerates.


Has anyone thought of that or tried to develop it?
Do you guys think it would make sense?

I know nothing about programming and the insides of ML but if someone would be interested in trying/developing this with me I think it could go somewhere.

Thanks

Toffifee

  • Member
  • ***
  • Posts: 104
  • 5D Mark III Happy tester
Re: ML Raw vs. Log
« Reply #3 on: July 11, 2013, 01:26:48 AM »
The thing is with the RAW module is that the camera was already doing it.
Magic Lantern simply extracts data earlier in the process before it is processed and compressed.

Midphase

  • Hero Member
  • *****
  • Posts: 712
Re: ML Raw vs. Log
« Reply #4 on: July 11, 2013, 09:39:31 AM »
Hacking into Canon Firmware, adding features that weren't even considered feasible, effectively making the impossible possible takes a great deal of smarts.

There is no doubt in my mind that all of the ML developers are geniuses, that their IQ's far surpass mine and that they spend every available hour trying to figure out how to raise the bar even higher...

...which is why it always blows my mind that someone would have such little confidence in them to suggest that they somehow must have overlooked some "duh" idea.

As it's been posted above, this subject has been amply discussed. The ML team has gone out of their way to explain why it's not an obtainable goal (hint, CPU isn't fast enough).

If anything should change I'm sure they'll figure it out first, but in the meantime could you maybe give them a bit more credit?

Stedda

  • Hero Member
  • *****
  • Posts: 504
Re: ML Raw vs. Log
« Reply #5 on: July 11, 2013, 09:45:35 AM »
Magic Lantern simply extracts data earlier in the process before it is processed and compressed.

That simple huh? 

Ever take a look at the code they wrote for RAW alone... doesn't look that simple. The RAW you're getting is a hijacked Live View display image.
5D Mark III -- 7D   SOLD -- EOS M 22mm 18-55mm STM -- Fuji X-T1 18-55 F2.8-F4 & 35 F1.4
Canon Glass   100L F2.8 IS -- 70-200L F4 -- 135L F2 -- 85 F1.8 -- 17-40L --  40 F2.8 -- 35 F2 IS  Sigma Glass  120-300 F2.8 OS -- 50 F1.4 -- 85 F1.4  Tamron Glass   24-70 2.8 VC   600EX-RT X3

franciscolobo

  • New to the forum
  • *
  • Posts: 2
Re: ML Raw vs. Log
« Reply #6 on: July 11, 2013, 12:10:55 PM »
Thanks for all the replies. I tried to look before I posted but skimming the threads I didn't see anyone discussing this.

In no way was I trying to disrespect the ML team. I appreciate their huge effort.
I asked about this because I didn't know if anyone had tried it.

I'll try to get some more into it and if I find something usefull I'll try to help!

Thanks again