Blackmagic RAW support?

[Eugenia]

I wonder if it would be possible to implement the new BRAW format in ML (it's not open source, but it's an open standard apparently, they provide an SDK). The advantages are that it creates smaller files (so Canon cameras with old SD speeds would handle better), and the fact that it would be a one-stop solution for editing, via the free DaVinci Resolve video editor. No transcoding to massive DNGs, no messy post processing to recover highlights etc.

[masc]

Blackmagic only provided an Decoder SDK - not Encoder. As long as they don't tell how to encode, it won't be possible. On the other side I don't think the processor in our EOS cams is fast enough to handle that.

[7DUser]

I thought the same.

Because of the datatrates.

I saw 15 sec. of RAW are possible with the EOS 7D. But in FullHD+, 2.5K or so, it would take about 512 to 1024 MB, I guess.

So what option do we have?

- MP4 all-I, maybe. Something like Small-JPEGs, 24 a second. Because the MOV-Standard-Codec is not the best.
- BRAW? we would have to encode with an 9 year old ARM chip.

What about ower own RAW compression?

Let's think about that. BRAW could have about 12 Bits per Dot. FullHD would be

1920 * 1080 * 12 (Bits) * 24 (Frames)

2,073,600 pixels. ~ 2 Megapixel.

The 7D can do: RAW: 18M, M-RAW: 10M, S-RAW: 4,5M
And at the same time JPEG in L:18, M:10, S: 4,5

Would it be possible, to do JPEG and RAW in tandem?

The 7D can take 8 pictures at a second. So that could be 1 RAW + 1 JPEG, but the JPEG not at the same time. We maybe could get 16 Frames.


One S-RAW file is about 10 MB for 4,5 MegaPixel. So a Pixel is about 16 Bit or more.

So what we want is, to reduce the file-size and memory usage in the Canon EOS 7D's RAM and on the Compact Flash Card for some reasons: Able to store constant / continuous saving of the data stream and ... be able to save more data.

So maybe we can do a compromise, not having 16 Bit per Pixel, but only 12 Bits. Maybe we can reduce the resolution a little bit, maybe we could save not every Pixel, maybe we could combine Pixel's luminance.


What we don't want is, to have more artifacts.


Bayer RGBG array per 2x2 Pixels.  So ... what can we do? Target?

2K 16:9, 2048 x 1152

Could we do 4K?


The data rate from the Canon EOS 7D is fast enough for 18 MegaPixel 8 Frames a second. For film, we would need 24 Frames, so 3 times as fast.
18 Megapixel / 3 ~ 6. So 6 Megapixel would be the limit, I guess. That are about 3000 x 2000 (like the EOS 300D full sensor could do)

But, this datarate could not be done continuous.   

Canon says
"*Figure based on updated firmware, ISO 100, Standard Picture Style and with UDMA 7 CF memory cards. Note: UDMA 7 CF memory card read/write speeds are not fully supported with the EOS 7D Digital SLR camera, if using UDMA 7 memory cards, the read/write speeds will be equivalent to UDMA 6." https://www.usa.canon.com/internet/portal/us/home/products/details/cameras/eos-dslr-and-mirrorless-cameras/dslr/eos-7d

UDMA 6 is about: 133,3 MB.

So ~ 130 MB let's say. Max datarate would be 130 MB / s. 130 MB / 24 ~ 5.4 MB.
So one picture could be 5.4 MB in size.

2048 x 1152 *2 ~ 4.8 MB ~ 5 MB

24 * 5 ~ 120 MB. So we would target 120 MB per sec.

That could be about 8 min. on a 64 GB card.
Card-Size, DataRate, sec. per min.
64000/120/60

SanDisk Extreme Pro 256 GB are about 284,- Euro and have a 150 MB / s WriteSpeed.

Puh. At these days the card is nearly about the price of a used 7D on ebay.

So we would have to reduce the RAW size maybe more. And that would be with 12 Bit instead of 16 Bit per Pixel.
4.8 MB per Picture from 16 -> 12 is about 3.6 MB, that would be only ~ 90 MB per sec.

Now we could use also cheaper cards, with lower bandwith, and smaller in size. On a 32 GB Card that would be about 6 min.

BlackMagic BRAW can do that


12:1 with 46 MB, only half the size we could "easily" do. And BRAW has more frames and 4.6K and not only 2K.
BRAW: 4608*2592*30 ~ 46 MB (BRAW compression 12:1)
7DCR: 2048 x 1152*24 ~ 90 MB (with 16to12 Bit reduction, not yet implemented)

Resolution per Frame is about 5 times
(4608*2592) / (2048*1152) ~ 5
and more frames 30 / 24 ~ 1.25

factor 6.25. So if we would have BRAW 12:1 on 2K24 we would have about 7.36 MB per sec.
Or our "90 MB" / 12 ~ 7.5

1 Sec. is smaller in BRAW than 1 picture in CR2 at this moment.

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

Sony can compress RAW files 2:1

But how could we implement an easy for the  DIGIC-4 processor compression, without losing to much information?

I guess, we only could do something like DXF, ASTC, but that would be very lossy and more like a JPEG in some kind.
https://developer.nvidia.com/astc-texture-compression-for-game-assets

So if we would use the Bayer-Array 2x2 and look at maybe 8 x 8 Pixels. (4 x 4 x (2x2))

64 Pixels. Where half of them is Green.

We don't have RGB-Pixels, we have 4 Pixels stored in one 2x2 Bayer-Array. In RGB we would have 30 Bits (10 Bits per Channel), And with Bayer we only have 14 Bits per Pixel. So that is already a "compression". And with 14->12 we could do it a bit more.
http://cybercom.net/~dcoffin/dcraw/

assumed we could re-create on the computer later from our new Canon-Magic-Lantern-Compressed-RAW the Canon CR2. We don't need 12:1 compression, but everything could help, if the De-Bayering and or Unfolding would not create a heavy break in the workflow.

A test with 7ZIP to compress a CR2 file from 10 MB was only about to generate a 9 MB file and it was slow. (and not in-camera, but on a computer)

So we only could do lossy compressing, I guess.

So if we use that 4 x 4 Bayer-Patterns, so with our 64 Pixels, maybe we could calculate something where we could easily guess some of the Green dots.
Maybe one out of 4 Blue, one out of 4 Red and one out of 4 Green dots could be stored with changing values of the others.

So with that, we could save about 25%. Maybe we could reconstruct the sharp edges.
So in our 64 Foto-Pixels we would guess 16 Pixels. And they are so structed, that it doesn't look like a pattern, more random.

That could reduce the file size also from ~ 90 MByte / s to ~ 67.5 ~ 70 MByte.

That would be about 15 min. of 2K video on a 64 GByte card. not only 15 sec. at the time.

Datarate would be about 560 Mbit/s


So the question would be: How to implement that in Magic Lantern?


There are papers out there for Near-lossless compression algorithm for Bayer pattern color filter arrays
https://www.researchgate.net/publication/252546887_Near-lossless_compression_algorithm_for_Bayer_pattern_color_filter_arrays

With that compression it would be possible to have 3.68:1 ~ 3:1

That would get us ~ 30 min. "RAW" video on a 64 GByte card. The implementation of that, would be nearly the Holy Grail. If the DIGIC-4 (or both together) are fast enough to encode that frame in about under 40 milliseconds.