new Canon CR3 raw format from M50 camera

[lorenzo353]

Hi,

Please find a tentative to document the new CR3 format here:
https://github.com/lclevy/canon_cr3

Contributions are welcome to open this format with me, as DCraw from Dave Coffin seems frozen...

Laurent
(from http://lclevy.free.fr/cr2/)

[Walter Schulz]

as DCraw from Dave Coffin seems frozen...

Ouch! Not the news I wanted to read. Totally slipped under my radar!

[lorenzo353]

Hi,

I have now published a python tool to parse the CR3 structure and extract jpeg, crx pictures...

Help is welcome to understand crx compression.

kind regard,

Laurent

[lorenzo353]

Code: [Select]

>python parse_cr3.py -v2 canon_eos_m50_02.cr3|more
filesize 0x256fbe8
00000:ftyp: major_brand=b'crx ', minor_version=1, [b'crx ', b'isom'] (0x18)
00018:moov: (0x6b70)
00020:  uuid: b'85c0b687820f11e08111f4ce462b6a48' (0x62c0)
00038:    CNCV: b'CanonCR3_001/00.09.00/00.00.00' (0x26)
0005e:    b'CCTP' b'000000000000000100000003000000184343445400000000' (0x5c)
0003a:      b'CCDT' b'00000000000000100000000000000001' (0x18)
00052:      b'CCDT' b'00000000000000010000000000000002' (0x18)
0006a:      b'CCDT' b'00000000000000000000000000000003' (0x18)
000ba:    CTBO: (0x5c)
            1    6b88   10018
            2   16ba0   56d90
            3   6d930 25022b8
            4       0       0
00116:    b'free' b'0000' (0xa)
00120:    b'CMT1' b'49492a00080000000d000001030001000000701700000101' (0x188)
002a8:    b'CMT2' b'49492a000800000027009a82050001000000e20100009d82' (0x428)
006d0:    b'CMT3' b'49492a00080000002f000100030031000000420200000200' (0x1438)
01b08:    b'CMT4' b'49492a000800000001000000010004000000020300000000' (0x718)
02220:    THMB: width=160, height=120, jpeg_size=0x40a3 (0x40c0)
062e0:  b'mvhd' b'00000000d6b31018d6b31018000000010000000100010000' (0x6c)
0634c:  b'trak' b'0000005c746b686400000007d6b31018d6b3101800000001' (0x1e4)
06354:    b'tkhd' b'00000007d6b31018d6b31018000000010000000000000001' (0x5c)
063b0:    b'mdia' b'000000206d64686400000000d6b31018d6b3101800000001' (0x180)
063b8:      b'mdhd' b'00000000d6b31018d6b31018000000010000000115c70000' (0x20)
063d8:      b'hdlr' b'000000000000000076696465000000000000000000000000' (0x21)
063f9:      b'minf' b'00000014766d686400000001000000000000000000000024' (0x137)
06401:        b'vmhd' b'000000010000000000000000' (0x14)
06415:        b'dinf' b'0000001c6472656600000000000000010000000c75726c20' (0x24)
0641d:          b'dref' b'00000000000000010000000c75726c2000000001' (0x1c)
00010:            b'url ' b'00000001' (0xc)
06439:        b'stbl' b'000000807374736400000000000000010000007043524157' (0xf7)
06441:          b'stsd' b'000000000000000100000070435241570000000000000001' (0x80)
00010:            CRAW: (0x70)
                    width=6000, height=4000
0005a:              b'JPEG' b'00000000' (0xc)
00066:              b'free' b'0000' (0xa)
064c1:          b'stts' b'00000000000000010000000100000001' (0x18)
064d9:          b'stsc' b'0000000000000001000000010000000100000001' (0x1c)
064f5:          stsz: version=0, size=0x30d6ef, count=1 (0x14)
06509:          b'free' b'00000000000000' (0xf)
06518:          co64: version=0, size=0x6d940, count=1 (0x18)
06530:  b'trak' b'0000005c746b686400000007d6b31018d6b3101800000002' (0x248)
06538:    b'tkhd' b'00000007d6b31018d6b31018000000020000000000000001' (0x5c)
06594:    b'mdia' b'000000206d64686400000000d6b31018d6b3101800000001' (0x1e4)
0659c:      b'mdhd' b'00000000d6b31018d6b31018000000010000000115c70000' (0x20)
065bc:      b'hdlr' b'000000000000000076696465000000000000000000000000' (0x21)
065dd:      b'minf' b'00000014766d686400000001000000000000000000000024' (0x19b)
065e5:        b'vmhd' b'000000010000000000000000' (0x14)
065f9:        b'dinf' b'0000001c6472656600000000000000010000000c75726c20' (0x24)
06601:          b'dref' b'00000000000000010000000c75726c2000000001' (0x1c)
00010:            b'url ' b'00000001' (0xc)
0661d:        b'stbl' b'000000e4737473640000000000000001000000d443524157' (0x15b)
06625:          b'stsd' b'0000000000000001000000d4435241570000000000000001' (0xe4)
00010:            CRAW: (0xd4)
                    width=1624, height=1080
0005a:              CMP1: (0x3c)
                      65280,48,256,0,0,1624,0,1080,0,1624,0,1080,3649,0,0,112,0,0,257,0,257,0,257,0,257,0,
00096:              CDI1: (0x34)
                      0,0,0,40,18753,17457,0,0,1624,1080,1,0,1,0,1,0,1620,1079,0,0,1623,1079,
00048:                IAD1: (0x28)
                        0,0,1624,1080,1,0,1,0,1,0,1620,1079,0,0,1623,1079,
000ca:              b'free' b'0000' (0xa)
06709:          b'stts' b'00000000000000010000000100000001' (0x18)
06721:          b'stsc' b'0000000000000001000000010000000100000001' (0x1c)
0673d:          stsz: version=0, size=0x1cbc40, count=1 (0x14)
06751:          b'free' b'00000000000000' (0xf)
06760:          co64: version=0, size=0x37b030, count=1 (0x18)
06778:  b'trak' b'0000005c746b686400000007d6b31018d6b3101800000003' (0x258)
06780:    b'tkhd' b'00000007d6b31018d6b31018000000030000000000000001' (0x5c)
067dc:    b'mdia' b'000000206d64686400000000d6b31018d6b3101800000001' (0x1f4)
067e4:      b'mdhd' b'00000000d6b31018d6b31018000000010000000115c70000' (0x20)
06804:      b'hdlr' b'000000000000000076696465000000000000000000000000' (0x21)
06825:      b'minf' b'00000014766d686400000001000000000000000000000024' (0x1ab)
0682d:        b'vmhd' b'000000010000000000000000' (0x14)
06841:        b'dinf' b'0000001c6472656600000000000000010000000c75726c20' (0x24)
06849:          b'dref' b'00000000000000010000000c75726c2000000001' (0x1c)
00010:            b'url ' b'00000001' (0xc)
06865:        b'stbl' b'000000f4737473640000000000000001000000e443524157' (0x16b)
0686d:          b'stsd' b'0000000000000001000000e4435241570000000000000001' (0xf4)
00010:            CRAW: (0xe4)
                    width=6288, height=4056
0005a:              CMP1: (0x3c)
                      65280,48,256,0,0,6288,0,4056,0,3144,0,4056,3648,0,0,216,0,0,257,0,257,0,257,0,257,0,
00096:              CDI1: (0x44)
                      0,0,0,56,18753,17457,0,0,6288,4056,1,2,1,0,276,48,6275,4047,0,0,263,4055,264,0,6287,35,264,36,6287,4055,
00048:                IAD1: (0x38)
                        0,0,6288,4056,1,2,1,0,276,48,6275,4047,0,0,263,4055,264,0,6287,35,264,36,6287,4055,
000da:              b'free' b'0000' (0xa)
06961:          b'stts' b'00000000000000010000000100000001' (0x18)
06979:          b'stsc' b'0000000000000001000000010000000100000001' (0x1c)
06995:          stsz: version=0, size=0x201ef28, count=1 (0x14)
069a9:          b'free' b'00000000000000' (0xf)
069b8:          co64: version=0, size=0x546c70, count=1 (0x18)
069d0:  b'trak' b'0000005c746b686400000007d6b31018d6b3101800000004' (0x1b8)
069d8:    b'tkhd' b'00000007d6b31018d6b31018000000040000000000000001' (0x5c)
06a34:    b'mdia' b'000000206d64686400000000d6b31018d6b3101800000001' (0x154)
06a3c:      b'mdhd' b'00000000d6b31018d6b31018000000010000000115c70000' (0x20)
06a5c:      b'hdlr' b'00000000000000006d657461000000000000000000000000' (0x21)
06a7d:      b'minf' b'0000000c6e6d6864000000000000002464696e660000001c' (0x10b)
06a85:        b'nmhd' b'00000000' (0xc)
06a91:        b'dinf' b'0000001c6472656600000000000000010000000c75726c20' (0x24)
06a99:          b'dref' b'00000000000000010000000c75726c2000000001' (0x1c)
00010:            b'url ' b'00000001' (0xc)
06ab5:        b'stbl' b'0000005c7374736400000000000000010000004c43544d44' (0xd3)
06abd:          b'stsd' b'00000000000000010000004c43544d440000000000000001' (0x5c)
00010:            b'CTMD' b'000000000000000100000007000000010000001800000003' (0x4c)
06b19:          b'stts' b'00000000000000010000000100000001' (0x18)
06b31:          b'stsc' b'0000000000000001000000010000000100000001' (0x1c)
06b4d:          stsz: version=0, size=0xa04c, count=1 (0x14)
06b61:          b'free' b'00000000000000' (0xf)
06b70:          co64: version=0, size=0x2565b98, count=1 (0x18)
06b88:uuid: b'be7acfcb97a942e89c71999491e3afac' (0x10018)
16ba0:uuid: b'eaf42b5e1c984b88b9fbb7dc406e4d16' (0x56d90)
16bc0:  PRVW: width=1620, height=1080, jpeg_size=0x56d58 (0x56d70)
6d930:b'mdat' b'ffd8ffdb008400060404060404060604' (0x25022b8)
256fbe8:
{b'THMB': (160, 120, 16547, 8760), 'trak0': {b'CRAW': (6000, 4000), b'stsz': 3200751, b'co64': 448832}, 'trak1': {b'CRAW
': (1624, 1080), b'stsz': 1883200, b'co64': 3649584}, 'trak2': {b'CRAW': (6288, 4056), b'stsz': 33681192, b'co64': 55327
84}, 'trak3': {b'stsz': 41036, b'co64': 39213976}, b'PRVW': (1620, 1080, 355672, 93144)}
extracting jpeg (trak0) 6000x4000 from mdat... offset=0x6d940, size=0x30d6ef
extracting SD crx (trak1) 1624x1080 from mdat... offset=0x37b030, size=0x1cbc40
ff010008 001cbbd0 00000000
  ff020008 0007b5c0 08000000
  ff030008 0007b5c0 00200001
    b'00000000002027a5000004000f03e0347565417b810ded0ef68019d59085af6a'
  ff020008 00070600 18000000
  ff030008 00070600 00200002
    b'00000000002028ff00000a6000680ccecfdd76905615eb87c07047a8e10bb5a4'
  ff020008 00070640 28000000
  ff030008 00070640 00200006
    b'00000000002028d500000000004001800880baa0035a513e5a91891b50050ad5'
  ff020008 0006f9d0 38000000
  ff030008 0006f9d0 00200006
    b'0000000000202cab0000020002a2b7747063b83a27ff1625fb4d52b4c41823e5'
extracting HD crx (trak2) 6288x4056 from mdat... offset=0x546c70, size=0x201ef28
ff010008 00ff40b8 00000000
  ff020008 00405528 08000000
  ff030008 00405528 00200006
    b'0000000000202e45000000000040039226003b15c982d276151ca7cef3aa0b22'
  ff020008 003fc8a8 18000000
  ff030008 003fc8a8 00200003
    b'0000000000202fbd000000000040016000000000e801b88ac3590cd6c022df4d'
  ff020008 003fc6e8 28000000
  ff030008 003fc6e8 00200005
    b'0000000000202f9d0000000002000000000a80110bf884163afc8d3d28f76fe1'
  ff020008 003f5c00 38000000
  ff030008 003f5c00 00200000
    b'0000000000202f6100000000004003ae0000000062a9c1c8002b0471075d0a2d'
ff010008 0102ad98 00010000
  ff020008 0040cb88 08000000
  ff030008 0040cb88 00200006
    b'0000000000202f6b0000000000a0064e819b8854c64481e72f454f50a3242ab2'
  ff020008 0040eb50 18000000
  ff030008 0040eb50 00200006
    b'0000000000202feb00000000004003c20000000000800b68026fcbbd264dfba5'
  ff020008 0040ed48 28000000
  ff030008 0040ed48 00200002
    b'0000000000202fa10001c00001b0034367370ac4dec63b510ad0e2415a17c15f'
  ff020008 00400978 38000000
  ff030008 00400978 00200007
    b'00000000002031db000000000040062400000001b0005e43835fa07a05efb670'

Hi,

I have now published a python tool to parse the CR3 structure and extract jpeg, crx pictures...

Help is welcome to understand crx compression.

kind regard,

Laurent

[dfort]

So ML for Digic 8 is in the works?

Is the EOS M50 CHDK only or mostly or is there a chance for ML in the distant future? The specs look interesting for such a small camera. It is priced about what the original EOSM was priced when it was introduced 6 years ago so maybe they will be selling for $150 in 2024?

Just kidding of course. Nice work on the CR3 format!

[chris_overseas]

There's something of a suggestion here that the cr3 compression might be lossy: https://www.dpreview.com/articles/0483301097/quick-look-canons-new-compressed-raw-format

[a1ex]

Is the EOS M50 CHDK only?

It runs EOS firmware.

[lorenzo353]

Dcraw 9.28 has been published in june 2018, so Dave Coffin is going on!

Ouch! Not the news I wanted to read. Totally slipped under my radar!

[Walter Schulz]

Thanks!
Sergio Namias's Windows build (with wildcard support) is one version behind (9.27). Has anyone tried if wildcard problem for Windows binaries still exists with GNU Compiler?
https://gcc.gnu.org/ml/gcc-patches/2017-01/msg01299.html
 Or anyone compiled it with Visual Studio?

[digiboy]

@Walter Schulz
I try build in Visual Studio 2017 with NO_JASPER option (use for Decode Red camera movies)
I'm not sure if everything works.
Would you like to try ?

[Walter Schulz]

The only feature I can thoroughly test is wildcard handling on Windows. Link, please! For other Dcraw features there are some more competent people around. Dual-ISO picture processing is not my strong point.

EDIT: As long as GCC issue with wildcards is not solved (haven't heard anything about it, yet) ... If you have too much time to spend: Can you try to compile cr2hdr.exe, too?

[digiboy]

I build Windows version x86.
It should support wildcards.

I compiled without JASPER library.
So only raw from RED camera shouldn't work

Link to exe file:
https://drive.google.com/open?id=1jULQntHJr_ddz_HDOImepXS8LG80Oyra

If something doesn't work, please report an error.

[Danne]

Dcraw 9.28 has been published in june 2018, so Dave Coffin is going on!

That is really good news.

[lorenzo353]

Hi,

FYI, the CRX codec of the CR3 format has been reverse engineered and open sourced: https://github.com/LibRaw/LibRaw/blob/master/src/decoders/crx.cpp

Kind regards,

Laurent

[Danne]

Nice!
@Ili3101
Maybe something to consider for raw2mlv?

[DeafEyeJedi]

@Ili3101
Maybe something to consider for raw2mlv?

+1

[masc]

@Ili3101
Maybe something to consider for raw2mlv?

Should work out of the box. Just update sources of libraw and compile.

[Chronic]

Ouch! Not the news I wanted to read. Totally slipped under my radar!

If anyone is on Linux, there is a command line program called dnglab that converts like a champ with several different output conversion formats including layers [channels].