ProcessTwoInTwoOutLosslessPath

[janvkem]

Can you share your settings directory, so I can try to reproduce?

Sorry, my bad. Checked again with a completely fresh card (other test was just with deleting the ML folder and autoexec.bin file) and issue appears to be gone. I had 2 additional files on the card but cannot recall the names (removed them before I saw your post unfortunately)... My apologies for the incorrect information.

[hyalinejim]

Output of "mlv_dump file.mlv -v" ?

Is this what you need?

https://www.dropbox.com/s/d5op627w2seulc2/output.txt?dl=0

This is 1.1.3

[a1ex]

Yes; unfortunately, it doesn't give me a clue about what happened...

Would be best if you can find a way to reproduce.

[hyalinejim]

I should note that on my first recording I had modules mlv_rec, _play and _lite loaded: I got the grayscale preview for a very brief time and then the LCD switched to some kind of report log that I haven't seen before.

2nd recording I switched off all modules except mlv_lite: I got greyscale preview continuously but recording stopped automagically after around 4 minutes.

3rd recording lowered resolution to 2880 x 1320 and very quickly got the onscreen log again:

https://s17.postimg.org/6fokginq7/20170401_170314.jpg

Here are my original settings (first recording) if it helps:

https://www.dropbox.com/s/d07hnipovcsrpf4/SETTINGS.zip?dl=0

[marekk]

+1 Please!

mlv_dump for mac (compressed_raw branch)

https://drive.google.com/file/d/0B-7nnVbwzGVkUmpHLW5kbEU3OWc/view?usp=sharing

[eNnvi]

So how can we help to Port this on other cameras?

[a1ex]

[ERROR] Invalid block size at position 0x1981e600

Diagnosed the above issue (it was related to variable frame sizes) and hopefully fixed it. Tested by filling the card a couple of times and checking the validity of the recorded clip with mlv_play (it doesn't play the video yet, but from the error message it's possible to tell whether the file structure is valid or not).

New experimental build available.

Will explain how it works later.

[hyalinejim]

Nice one!

EDIT: Checked out new build and I'm happily recording at 3072 x 1286. Around 37 minutes into that take now and temperature is showing as 55 degrees. Previously I'd never really seen it go above around 45.

[lostfeliz]

37 minutes at 3072 x 1286?! Wow. How many GB was the whole clip?

[ilia3101]

hyalinejim is that losslessly compressed video you were able to record for 37 minutes?

[hyalinejim]

37 minutes at 3072 x 1286?! Wow. How many GB was the whole clip?

Just checked the card: 208GB.

hyalinejim is that losslessly compressed video you were able to record for 37 minutes?

Yes, indeed. By comparison, at the same resolution on my card I got around seven seconds in 10 bit. To be clear, I stopped the lossless 14bit recording at 37 minutes. It's only limited by the size of your recording media.

[Danne]

Lossless 14bit files 3k
I need my camera already.

[beauchampy]

Hey a1ex! Really nice job! I'm still wrapping my head around all these incredible developments.

Questions about the lossless compression - it says compressed with Canon lossless JPEG, but this is still raw, right? How does the information differ to the normal 14-bit raw files?

[Markus]

Still can't believe all this incredible progress isn't just a big april fools joke.

[a1ex]

Questions about the lossless compression - it says compressed with Canon lossless JPEG, but this is still raw, right? How does the information differ to the normal 14-bit raw files?

Lossless JPEG is what you have in regular CR2 files and compressed DNGs. It's not the 8-bit JPEG you are used with.

I was also fooled by this for a long time - the routines used to compress the CR2, on most camera models, had JPEG in their name, so I didn't look into them, thinking they were for regular 8-bit JPEG compression...

https://en.wikipedia.org/wiki/Lossless_JPEG
http://lclevy.free.fr/cr2/#lossless
https://bitbucket.org/baldand/mlrawviewer/src/e7abaaf4cf9be66f46e0c8844297be0e7d88c288/LJ92.py
https://bitbucket.org/hudson/magic-lantern/commits/83ced42a1009db90ff33ceae5e4e198fe5943b47

[nikfreak]

Have to ask it:only 14/15bit?

[bouncyball]

@a1ex: Happiest April 1st ever!

Just checked the card: 208GB.

Yes, indeed. By comparison, at the same resolution on my card I got around seven seconds in 10 bit. To be clear, I stopped the lossless 14bit recording at 37 minutes. It's only limited by the size of your recording media.

@hyalinejim: Hmm... it's about 93mb/sec. Which CF card did you use?

[Levas]

YEs, Canon standard CR2 compression for RAW video 

How hard is this to port to other cams, is it just a few lines of routines to be altered or is it a pain in the to port  ?

I hear compression ratios of 58%, is that true ? Because in that case it would be even better then 10 bit (about 70% smaller compared to 14 bit)

[hyalinejim]

@hyalinejim: Hmm... it's about 93mb/sec. Which CF card did you use?

OK, so the long file I recorded is 218,855,342 KB and it's 55,874 frames at 3072 x 1286 resolution.

Card is Komputerbay 256GB 1066x

[Levas]

That's about 57% compression ratio 

[eNnvi]

@levas no, as Alex tested a few weeks ago the compression would be the same as a 10bit stream would give the same results of a 14 bit as it's used as a 10bit with leading zeros. So it's not worth to use a 10bit stream with this method.

Just I'd like to know how I can help to Port this on other cameras and if this would get rid of focus pixels on many cams like 700D. Would be nice to know if it's the same routine to compress and delete focus pixels

[Levas]

OK, so the long file I recorded is 218,855,342 KB and it's 55,874 frames at 3072 x 1286 resolution.

The above gives about 4Mb frames which should be about 7Mb uncompressed, that's 57% isn't it?

Edit: I know that it doesn't work with 10 bit stream, which would be even more great.
But to me it looks like 14 bit compressed is more efficient  then 10 bit.
57% vs 71%

[bouncyball]

Just I'd like to know how I can help to Port this on other cameras and if this would get rid of focus pixels on many cams like 700D. Would be nice to know if it's the same routine to compress and delete focus pixels

The focus pixel issue has no relation to canon HW compression at all.

[eyeland]

This is amazing
Has anyone played around with cranking frame-rates up instead of resolution?

[COMMANDES]

How soon will it be possible to test a compressed RAW at 650D?