Author Topic: ProcessTwoInTwoOutLosslessPath  (Read 73975 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10453
  • 5D Mark Free
ProcessTwoInTwoOutLosslessPath
« on: December 18, 2016, 09:06:41 PM »
Managed to call ProcessTwoInTwoOutLosslessPath, which appears to perform the compression for RAW, MRAW and SRAW formats. The output looks like some sort of lossless JPEG, but we don't know how to decode it yet (this should help).

Proof of concept code (photo mode only for now): http://bitbucket.org/hudson/magic-lantern/commits/0e87b86

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10453
  • 5D Mark Free
Re: ProcessTwoInTwoOutLosslessPath
« Reply #1 on: December 18, 2016, 10:33:23 PM »
Proof of concept code works! Managed to decode a valid image with a tiny patch to dcraw.c.

https://bitbucket.org/hudson/magic-lantern/commits/06aedb0

Processing time: 120ms for 5920x3950 (14bpp), 11ms for 1920x1080 (not decoded).

Levas

  • Hero Member
  • *****
  • Posts: 940
  • 6d - Nightly build user
Re: ProcessTwoInTwoOutLosslessPath
« Reply #2 on: December 18, 2016, 11:06:57 PM »
What do you mean with the 1920 x 1080 not decoded ?
Do you mean you can make the file but can't decode it ?
Isn't this standard canon cr2 files what you're making here, which should be read by almost all raw editors out there.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10453
  • 5D Mark Free
Re: ProcessTwoInTwoOutLosslessPath
« Reply #3 on: December 18, 2016, 11:32:22 PM »
I simply changed the buffer size to 1920x1080 to check the processing speed, and I did not attempt to decode it. That test was done outside LiveView.

Seconds ago, I've also managed to run the test in LiveView, and the output appears valid (though distorted - wrong pitch). Should be fixable.

It is standard CR2, without header, embedded jpeg and whatever else it might contain.

reddeercity

  • Hero Member
  • *****
  • Posts: 1407
Re: ProcessTwoInTwoOutLosslessPath
« Reply #4 on: December 19, 2016, 02:29:20 AM »
Great news!
From your link a1ex
Quote
(5D Mark II sraw1) 
JPEG_SOF3: length=17  bits=15, wide=3872, high=2574
(5D Mark II sraw2) 
JPEG_SOF3: length=17 bits=15, wide=2808, high=1872
So does that mean 15bit Jpeg compression in raw(cr2) ? or is that referring to something else

Edit: after reading more it appears sraw is YCbCr 4:2:2.
interesting !
Quote
sRaw and sRaw2 (and surely sRaw1) are encoded in YCbCr format, and not as CFA RGB data like full RAW
•sRaw and sRaw2 use a YCbCr 4:2:2 chroma subsampling encoding : 2 Chroma values on the 1st Row of 4 pixels and 2 Chroma values on the 2nd row of 4 pixels,
•and sRaw1 use a 4:2:0 encoding : 2 Chroma values on the 1st Row of 4 pixels and 0 Chroma value on the 2nd row of 4 pixels

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10453
  • 5D Mark Free
Re: ProcessTwoInTwoOutLosslessPath
« Reply #5 on: December 19, 2016, 08:43:30 AM »
I've only tested regular RAW lossless compression, but other methods are available.

Code: [Select]
run_test:ff3d3f34:16:03: [TTL][0,0,0] RAW(5920,3950,0,14)
run_test:ff28df90:15:02: [JPCORE] SetJp57EncodeLosslessType 1
run_test:ff28f7d4:15:03: [JPCORE] SetJp57EncodeLosslessParam 2960,3950           <- note: 2 slices

run_test:ff3d3fe0:16:03: [TTL][0,0,0] SRAW(2880,1920,0,15)
run_test:ff28df90:15:02: [JPCORE] SetJp57EncodeLosslessType 2
run_test:ff28f8a8:15:02: [JPCORE] SetJp57EncodeYuv422LosslessParam 2880,1920

run_test:ff3d3f88:16:03: [TTL][0,0,0] MRAW(3960,2640,0,16)
run_test:ff28df90:15:02: [JPCORE] SetJp57EncodeLosslessType 3
run_test:ff28f97c:15:02: [JPCORE] SetJp57EncodeYuv420LosslessParam 7920,1320

The SRAW format appears to use 15-bit YUV422 internally.

Levas

  • Hero Member
  • *****
  • Posts: 940
  • 6d - Nightly build user
Re: ProcessTwoInTwoOutLosslessPath
« Reply #6 on: December 19, 2016, 08:55:41 AM »
Now the question most of us are thinking off   ???
Does get some chip(probably the digic) fried inside when this functions is used 25 frames per second  :P

chmee

  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: ProcessTwoInTwoOutLosslessPath
« Reply #7 on: December 19, 2016, 10:22:07 AM »
imho: i didnt like the mraw/sraw-Files because of the poor quality. Its using some kind of demosaicing inside the body to render these files. a lot of antialiasing and a noticeable hue-shift.

but in terms of video it should get a better output than the original video-processing-chain.

benefit: using the inbody demosaicing/rendering to yuv-(video)files
drawback: quality

regards chmee

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3052
Re: ProcessTwoInTwoOutLosslessPath
« Reply #8 on: December 19, 2016, 01:08:45 PM »
i played a bit with it yesterday.
in theory it would mean that we can compress frames to 50%-60% of the original size.
for raw video this would give us a higher video resolution for the same card speed.
compression speed is - at least in photo mode - good enough. LV measurement while card is writing etc has to be done.

*if* we can somewhen manage to run it in our raw recording environment.
the JPEG core is a bit picky with accesses and can even lock up the whole camera just by reading or writing its registers at the wrong point in time..

@chmee: we are not talking about mRaw/sRaw but the original cr2 raw (cRaw). this engine handles all three of them and a1ex' numbers were the cRaw ones.
we didn't try the other ones properly yet. maybe its good for raw video. maybe not. all to be tested.


theres a good chance that it wont work nicely all together (LV, CFDMA, JPEG) and causes hickups, crashes etc...
a1ex opened the door to in-camera compression. the next step is to check if that door is usable for e.g. raw video.
and if not, for what else we could use it :)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Levas

  • Hero Member
  • *****
  • Posts: 940
  • 6d - Nightly build user
Re: ProcessTwoInTwoOutLosslessPath
« Reply #9 on: December 19, 2016, 02:02:52 PM »
Can't resist too ask  ;D
Does it handle 10 bit files ?





a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10453
  • 5D Mark Free
Re: ProcessTwoInTwoOutLosslessPath
« Reply #10 on: December 19, 2016, 09:17:31 PM »
I've just got a valid LiveView image (decoded successfully). 2080x1080 14bpp, processing time 23ms, file size about 1.9MB (uncompressed 3.75MB).

You can feed 10-bit data by faking the input dimensions, but the predictors won't do a good job in this case. A quick test resulted in a compressed size of... 4.2MB.

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3052
Re: ProcessTwoInTwoOutLosslessPath
« Reply #11 on: December 19, 2016, 09:39:30 PM »
instead of 3.75MiB?
a really good compression ;D
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10453
  • 5D Mark Free
Re: ProcessTwoInTwoOutLosslessPath
« Reply #12 on: December 19, 2016, 09:42:03 PM »
... instead of 2080x1080x10bpp = 2.68MB :D

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3204
  • 60Da / 1100D / EOSM
Re: ProcessTwoInTwoOutLosslessPath
« Reply #13 on: December 19, 2016, 09:53:33 PM »
 haha

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3052
Re: ProcessTwoInTwoOutLosslessPath
« Reply #14 on: December 19, 2016, 10:13:31 PM »
haha overwhelming :)

but i am sure this is configurable. somehow.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Levas

  • Hero Member
  • *****
  • Posts: 940
  • 6d - Nightly build user
Re: ProcessTwoInTwoOutLosslessPath
« Reply #15 on: December 19, 2016, 10:26:24 PM »
Wow, nice compression -> 3.75MB to 1.9 MB  :o
Didn't expect that much, most of CR2 files from the 6d are 25MB in size, uncompressed should be about 35MB...



a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10453
  • 5D Mark Free
Re: ProcessTwoInTwoOutLosslessPath
« Reply #16 on: December 20, 2016, 10:11:19 PM »
0xC0F371FC is the unpacking mode (bit depth) register for the input module that transfers the image data from memory to the lossless encoder. Same values as PACK(16|32)_MODE. Other input modules with similar functionality are DSUNPACK, ADUNPACK, UNPACK24.

However, the encoder still treats the input data as 14-bit, as the input module pads the data with zeros when unpacking. The decompressed output has 14-bit metadata and levels, but the lower 2 or 4 bits are zero if the input data is 12- or 10-bit. Unfortunately, it doesn't help improving the compression ratio (output is still around 2M).



0xC0E20084 appears to configure the output bit depth and the number of slices. For bit depth, the only valid values appear to be 14 and 15. For 15-bit output, JPEGsnoop reports 15-bit data, but I didn't get numbers higher than 16383 when decoding (higher bits from a 16-bit input stream were trimmed). Changing the number of slices doesn't seem to work, probably some more registers have to be changed.

Ilia3101

  • Member
  • ***
  • Posts: 236
Re: ProcessTwoInTwoOutLosslessPath
« Reply #17 on: December 21, 2016, 12:09:32 AM »
I think noobs don't belong here, but is it possible to try this compression on other cameras? Or do they have different registers that need finding? I managed to compile the dm_whateveritscalled branch but don't know what to do with it. Could I be any help testing on the 5D2?
5D2

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10453
  • 5D Mark Free
Re: ProcessTwoInTwoOutLosslessPath
« Reply #18 on: December 21, 2016, 10:57:04 AM »
@Ilia: will reply as soon as I'll have a complete answer.

Meanwhile, I've tried to slap a DNG header - it almost gets decoded by both dcraw and dng_validate.exe.

There is an issue: Canon divides the image in two slices (link from first post). In DNG, that would require either two strips or two tiles, but the DNG spec requires that metadata should contain individual offset for each strip or tile. Canon output renders cleanly in dcraw as a single image with half width and double height.

So, I've reconfigured the encoder to use two vertical slices (top half and bottom half), instead of default configuration (left half and right half); this happens to be considered as a single logical slice, so it gives valid lossless DNG! Tested with vanilla dcraw and dng_validate.

All that's needed to change in the DNG header is:
- Compression (tag 0x103) set to 7 (JPEG)
- StripByteCounts set to compressed size (otherwise dng_validate will complain)

Therefore, we have all the info required to create valid lossless DNGs directly from camera!

Danne

  • Hero Member
  • *****
  • Posts: 3478
Re: ProcessTwoInTwoOutLosslessPath
« Reply #19 on: December 21, 2016, 11:02:58 AM »
Quote
Therefore, we have all the info required to create valid lossless DNGs directly from camera!
Ingenious.

chmee

  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: ProcessTwoInTwoOutLosslessPath
« Reply #20 on: December 21, 2016, 01:28:23 PM »
Quote
Therefore, we have all the info required to create valid lossless DNGs directly from camera!
marvelous.

Levas

  • Hero Member
  • *****
  • Posts: 940
  • 6d - Nightly build user
Re: ProcessTwoInTwoOutLosslessPath
« Reply #21 on: December 21, 2016, 04:10:22 PM »
  :D

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10453
  • 5D Mark Free
Re: ProcessTwoInTwoOutLosslessPath
« Reply #22 on: December 21, 2016, 10:27:04 PM »
Proof of concept code available in the compressed_raw branch (silent pictures with lossless DNG output).

Some more details about how I've got valid DNGs:

Take a look at this picture and look for TTL (TwoInTwoOutLossless) channels. The green one with two ramps (channel #8) has a pretty strange configuration:

Code: [Select]
(5180, skip 5208) x 3949, 5180, skip -41022212, (5180, skip 5208) x 3950

5180 + 5208 = 10388 = 5936 * 14/8 (horizontal raw resolution, including black bars and a few extra columns present in the raw buffer, but - for some unknown reason - not saved in the CR2)
5180 * 2 = 10360 = 5920 (horizontal CR2 resolution, with dcraw -4 -E)

(5180 + 5208) * 3949 + 5180 - 41022212 = 5180.

Now, the meaning of the above code is clear: read the left half of the image (5180 bytes = 2960 pixels), rewind and read the right half (also 2960). These are the two cr2_slice's (see dcraw code at lossless_jpeg_load_raw and lclevy's docs from first post).

A lossless DNG is decoded with a somewhat similar loop, in lossless_dng_load_raw, which treats the compressed payload as a single slice. With default encoder configuration, dcraw would render a 2960x7900 image (left and right halves stacked vertically).

Now, my encoder mod becomes obvious: double the slice width (so it becomes equal to image width) and halve the height. The encoder doesn't really care what the image layout is - it knows there are two slices (with given dimensions). It uses the data coming from the read EDMAC to fill the first slice, then the second one; therefore, the EDMAC transfer can be configured to just read the image as usual (the first half will be the top of the image, the second half will be the bottom).

In the DNG, the decoder sees a lossless jpg payload; it doesn't really care about the dimensions, as long as the total pixel count matches.

Result: the trick may be a little hackish, but Adobe's dng_validate accepts it :)

Danne

  • Hero Member
  • *****
  • Posts: 3478
Re: ProcessTwoInTwoOutLosslessPath
« Reply #23 on: December 22, 2016, 12:05:02 AM »
My 5D mark III just gave birth to some lossless DNG files. Here,s a compiled version for testing from a1ex compressed_raw branch. Awesome stuff.

5D mark III
https://bitbucket.org/Dannephoto/magic-lantern/downloads/magiclantern-Nightly.2016Dec21.5D3113_lossless_DNG.zip

Two lossless DNG files
https://bitbucket.org/Dannephoto/magic-lantern/downloads/65260001.DNG
https://bitbucket.org/Dannephoto/magic-lantern/downloads/65260002.DNG

goldenchild9to5

  • Senior
  • ****
  • Posts: 497
Re: ProcessTwoInTwoOutLosslessPath
« Reply #24 on: December 23, 2016, 04:19:43 PM »
@a1ex Are you serious, that's awesome man Team Lantern is pushing things to the limit WOW..  :D  @Danne Images looks great.