new Canon CR3 raw format from M50 camera

Started by lorenzo353, March 12, 2018, 10:48:00 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

lorenzo353

Hi,

Please find a tentative to document the new CR3 format here:
https://github.com/lclevy/canon_cr3

Contributions are welcome to open this format with me, as DCraw from Dave Coffin seems frozen...

Laurent
(from http://lclevy.free.fr/cr2/)

Walter Schulz

Quote from: lorenzo353 on March 12, 2018, 10:48:00 PM
as DCraw from Dave Coffin seems frozen...

Ouch! Not the news I wanted to read. Totally slipped under my radar!

lorenzo353

Hi,

I have now published a python tool to parse the CR3 structure and extract jpeg, crx pictures...

Help is welcome to understand crx compression.

kind regard,

Laurent

lorenzo353

>python parse_cr3.py -v2 canon_eos_m50_02.cr3|more
filesize 0x256fbe8
00000:ftyp: major_brand=b'crx ', minor_version=1, [b'crx ', b'isom'] (0x18)
00018:moov: (0x6b70)
00020:  uuid: b'85c0b687820f11e08111f4ce462b6a48' (0x62c0)
00038:    CNCV: b'CanonCR3_001/00.09.00/00.00.00' (0x26)
0005e:    b'CCTP' b'000000000000000100000003000000184343445400000000' (0x5c)
0003a:      b'CCDT' b'00000000000000100000000000000001' (0x18)
00052:      b'CCDT' b'00000000000000010000000000000002' (0x18)
0006a:      b'CCDT' b'00000000000000000000000000000003' (0x18)
000ba:    CTBO: (0x5c)
            1    6b88   10018
            2   16ba0   56d90
            3   6d930 25022b8
            4       0       0
00116:    b'free' b'0000' (0xa)
00120:    b'CMT1' b'49492a00080000000d000001030001000000701700000101' (0x188)
002a8:    b'CMT2' b'49492a000800000027009a82050001000000e20100009d82' (0x428)
006d0:    b'CMT3' b'49492a00080000002f000100030031000000420200000200' (0x1438)
01b08:    b'CMT4' b'49492a000800000001000000010004000000020300000000' (0x718)
02220:    THMB: width=160, height=120, jpeg_size=0x40a3 (0x40c0)
062e0:  b'mvhd' b'00000000d6b31018d6b31018000000010000000100010000' (0x6c)
0634c:  b'trak' b'0000005c746b686400000007d6b31018d6b3101800000001' (0x1e4)
06354:    b'tkhd' b'00000007d6b31018d6b31018000000010000000000000001' (0x5c)
063b0:    b'mdia' b'000000206d64686400000000d6b31018d6b3101800000001' (0x180)
063b8:      b'mdhd' b'00000000d6b31018d6b31018000000010000000115c70000' (0x20)
063d8:      b'hdlr' b'000000000000000076696465000000000000000000000000' (0x21)
063f9:      b'minf' b'00000014766d686400000001000000000000000000000024' (0x137)
06401:        b'vmhd' b'000000010000000000000000' (0x14)
06415:        b'dinf' b'0000001c6472656600000000000000010000000c75726c20' (0x24)
0641d:          b'dref' b'00000000000000010000000c75726c2000000001' (0x1c)
00010:            b'url ' b'00000001' (0xc)
06439:        b'stbl' b'000000807374736400000000000000010000007043524157' (0xf7)
06441:          b'stsd' b'000000000000000100000070435241570000000000000001' (0x80)
00010:            CRAW: (0x70)
                    width=6000, height=4000
0005a:              b'JPEG' b'00000000' (0xc)
00066:              b'free' b'0000' (0xa)
064c1:          b'stts' b'00000000000000010000000100000001' (0x18)
064d9:          b'stsc' b'0000000000000001000000010000000100000001' (0x1c)
064f5:          stsz: version=0, size=0x30d6ef, count=1 (0x14)
06509:          b'free' b'00000000000000' (0xf)
06518:          co64: version=0, size=0x6d940, count=1 (0x18)
06530:  b'trak' b'0000005c746b686400000007d6b31018d6b3101800000002' (0x248)
06538:    b'tkhd' b'00000007d6b31018d6b31018000000020000000000000001' (0x5c)
06594:    b'mdia' b'000000206d64686400000000d6b31018d6b3101800000001' (0x1e4)
0659c:      b'mdhd' b'00000000d6b31018d6b31018000000010000000115c70000' (0x20)
065bc:      b'hdlr' b'000000000000000076696465000000000000000000000000' (0x21)
065dd:      b'minf' b'00000014766d686400000001000000000000000000000024' (0x19b)
065e5:        b'vmhd' b'000000010000000000000000' (0x14)
065f9:        b'dinf' b'0000001c6472656600000000000000010000000c75726c20' (0x24)
06601:          b'dref' b'00000000000000010000000c75726c2000000001' (0x1c)
00010:            b'url ' b'00000001' (0xc)
0661d:        b'stbl' b'000000e4737473640000000000000001000000d443524157' (0x15b)
06625:          b'stsd' b'0000000000000001000000d4435241570000000000000001' (0xe4)
00010:            CRAW: (0xd4)
                    width=1624, height=1080
0005a:              CMP1: (0x3c)
                      65280,48,256,0,0,1624,0,1080,0,1624,0,1080,3649,0,0,112,0,0,257,0,257,0,257,0,257,0,
00096:              CDI1: (0x34)
                      0,0,0,40,18753,17457,0,0,1624,1080,1,0,1,0,1,0,1620,1079,0,0,1623,1079,
00048:                IAD1: (0x28)
                        0,0,1624,1080,1,0,1,0,1,0,1620,1079,0,0,1623,1079,
000ca:              b'free' b'0000' (0xa)
06709:          b'stts' b'00000000000000010000000100000001' (0x18)
06721:          b'stsc' b'0000000000000001000000010000000100000001' (0x1c)
0673d:          stsz: version=0, size=0x1cbc40, count=1 (0x14)
06751:          b'free' b'00000000000000' (0xf)
06760:          co64: version=0, size=0x37b030, count=1 (0x18)
06778:  b'trak' b'0000005c746b686400000007d6b31018d6b3101800000003' (0x258)
06780:    b'tkhd' b'00000007d6b31018d6b31018000000030000000000000001' (0x5c)
067dc:    b'mdia' b'000000206d64686400000000d6b31018d6b3101800000001' (0x1f4)
067e4:      b'mdhd' b'00000000d6b31018d6b31018000000010000000115c70000' (0x20)
06804:      b'hdlr' b'000000000000000076696465000000000000000000000000' (0x21)
06825:      b'minf' b'00000014766d686400000001000000000000000000000024' (0x1ab)
0682d:        b'vmhd' b'000000010000000000000000' (0x14)
06841:        b'dinf' b'0000001c6472656600000000000000010000000c75726c20' (0x24)
06849:          b'dref' b'00000000000000010000000c75726c2000000001' (0x1c)
00010:            b'url ' b'00000001' (0xc)
06865:        b'stbl' b'000000f4737473640000000000000001000000e443524157' (0x16b)
0686d:          b'stsd' b'0000000000000001000000e4435241570000000000000001' (0xf4)
00010:            CRAW: (0xe4)
                    width=6288, height=4056
0005a:              CMP1: (0x3c)
                      65280,48,256,0,0,6288,0,4056,0,3144,0,4056,3648,0,0,216,0,0,257,0,257,0,257,0,257,0,
00096:              CDI1: (0x44)
                      0,0,0,56,18753,17457,0,0,6288,4056,1,2,1,0,276,48,6275,4047,0,0,263,4055,264,0,6287,35,264,36,6287,4055,
00048:                IAD1: (0x38)
                        0,0,6288,4056,1,2,1,0,276,48,6275,4047,0,0,263,4055,264,0,6287,35,264,36,6287,4055,
000da:              b'free' b'0000' (0xa)
06961:          b'stts' b'00000000000000010000000100000001' (0x18)
06979:          b'stsc' b'0000000000000001000000010000000100000001' (0x1c)
06995:          stsz: version=0, size=0x201ef28, count=1 (0x14)
069a9:          b'free' b'00000000000000' (0xf)
069b8:          co64: version=0, size=0x546c70, count=1 (0x18)
069d0:  b'trak' b'0000005c746b686400000007d6b31018d6b3101800000004' (0x1b8)
069d8:    b'tkhd' b'00000007d6b31018d6b31018000000040000000000000001' (0x5c)
06a34:    b'mdia' b'000000206d64686400000000d6b31018d6b3101800000001' (0x154)
06a3c:      b'mdhd' b'00000000d6b31018d6b31018000000010000000115c70000' (0x20)
06a5c:      b'hdlr' b'00000000000000006d657461000000000000000000000000' (0x21)
06a7d:      b'minf' b'0000000c6e6d6864000000000000002464696e660000001c' (0x10b)
06a85:        b'nmhd' b'00000000' (0xc)
06a91:        b'dinf' b'0000001c6472656600000000000000010000000c75726c20' (0x24)
06a99:          b'dref' b'00000000000000010000000c75726c2000000001' (0x1c)
00010:            b'url ' b'00000001' (0xc)
06ab5:        b'stbl' b'0000005c7374736400000000000000010000004c43544d44' (0xd3)
06abd:          b'stsd' b'00000000000000010000004c43544d440000000000000001' (0x5c)
00010:            b'CTMD' b'000000000000000100000007000000010000001800000003' (0x4c)
06b19:          b'stts' b'00000000000000010000000100000001' (0x18)
06b31:          b'stsc' b'0000000000000001000000010000000100000001' (0x1c)
06b4d:          stsz: version=0, size=0xa04c, count=1 (0x14)
06b61:          b'free' b'00000000000000' (0xf)
06b70:          co64: version=0, size=0x2565b98, count=1 (0x18)
06b88:uuid: b'be7acfcb97a942e89c71999491e3afac' (0x10018)
16ba0:uuid: b'eaf42b5e1c984b88b9fbb7dc406e4d16' (0x56d90)
16bc0:  PRVW: width=1620, height=1080, jpeg_size=0x56d58 (0x56d70)
6d930:b'mdat' b'ffd8ffdb008400060404060404060604' (0x25022b8)
256fbe8:
{b'THMB': (160, 120, 16547, 8760), 'trak0': {b'CRAW': (6000, 4000), b'stsz': 3200751, b'co64': 448832}, 'trak1': {b'CRAW
': (1624, 1080), b'stsz': 1883200, b'co64': 3649584}, 'trak2': {b'CRAW': (6288, 4056), b'stsz': 33681192, b'co64': 55327
84}, 'trak3': {b'stsz': 41036, b'co64': 39213976}, b'PRVW': (1620, 1080, 355672, 93144)}
extracting jpeg (trak0) 6000x4000 from mdat... offset=0x6d940, size=0x30d6ef
extracting SD crx (trak1) 1624x1080 from mdat... offset=0x37b030, size=0x1cbc40
ff010008 001cbbd0 00000000
  ff020008 0007b5c0 08000000
  ff030008 0007b5c0 00200001
    b'00000000002027a5000004000f03e0347565417b810ded0ef68019d59085af6a'
  ff020008 00070600 18000000
  ff030008 00070600 00200002
    b'00000000002028ff00000a6000680ccecfdd76905615eb87c07047a8e10bb5a4'
  ff020008 00070640 28000000
  ff030008 00070640 00200006
    b'00000000002028d500000000004001800880baa0035a513e5a91891b50050ad5'
  ff020008 0006f9d0 38000000
  ff030008 0006f9d0 00200006
    b'0000000000202cab0000020002a2b7747063b83a27ff1625fb4d52b4c41823e5'
extracting HD crx (trak2) 6288x4056 from mdat... offset=0x546c70, size=0x201ef28
ff010008 00ff40b8 00000000
  ff020008 00405528 08000000
  ff030008 00405528 00200006
    b'0000000000202e45000000000040039226003b15c982d276151ca7cef3aa0b22'
  ff020008 003fc8a8 18000000
  ff030008 003fc8a8 00200003
    b'0000000000202fbd000000000040016000000000e801b88ac3590cd6c022df4d'
  ff020008 003fc6e8 28000000
  ff030008 003fc6e8 00200005
    b'0000000000202f9d0000000002000000000a80110bf884163afc8d3d28f76fe1'
  ff020008 003f5c00 38000000
  ff030008 003f5c00 00200000
    b'0000000000202f6100000000004003ae0000000062a9c1c8002b0471075d0a2d'
ff010008 0102ad98 00010000
  ff020008 0040cb88 08000000
  ff030008 0040cb88 00200006
    b'0000000000202f6b0000000000a0064e819b8854c64481e72f454f50a3242ab2'
  ff020008 0040eb50 18000000
  ff030008 0040eb50 00200006
    b'0000000000202feb00000000004003c20000000000800b68026fcbbd264dfba5'
  ff020008 0040ed48 28000000
  ff030008 0040ed48 00200002
    b'0000000000202fa10001c00001b0034367370ac4dec63b510ad0e2415a17c15f'
  ff020008 00400978 38000000
  ff030008 00400978 00200007
    b'00000000002031db000000000040062400000001b0005e43835fa07a05efb670'


Quote from: lorenzo353 on March 29, 2018, 08:29:23 PM
Hi,

I have now published a python tool to parse the CR3 structure and extract jpeg, crx pictures...

Help is welcome to understand crx compression.

kind regard,

Laurent

dfort

So ML for Digic 8 is in the works?

Is the EOS M50 CHDK only or mostly or is there a chance for ML in the distant future? The specs look interesting for such a small camera. It is priced about what the original EOSM was priced when it was introduced 6 years ago so maybe they will be selling for $150 in 2024?

Just kidding of course. Nice work on the CR3 format!

chris_overseas

EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II


lorenzo353

Dcraw 9.28 has been published in june 2018, so Dave Coffin is going on!

Quote from: Walter Schulz on March 12, 2018, 11:00:56 PM
Ouch! Not the news I wanted to read. Totally slipped under my radar!

Walter Schulz

Thanks!
Sergio Namias's Windows build (with wildcard support) is one version behind (9.27). Has anyone tried if wildcard problem for Windows binaries still exists with GNU Compiler?
https://gcc.gnu.org/ml/gcc-patches/2017-01/msg01299.html
Or anyone compiled it with Visual Studio?

digiboy

@Walter Schulz
I try build in Visual Studio 2017 with NO_JASPER option (use for Decode Red camera movies)
I'm not sure if everything works.
Would you like to try ?

Walter Schulz

The only feature I can thoroughly test is wildcard handling on Windows. Link, please! For other Dcraw features there are some more competent people around. Dual-ISO picture processing is not my strong point.

EDIT: As long as GCC issue with wildcards is not solved (haven't heard anything about it, yet) ... If you have too much time to spend: Can you try to compile cr2hdr.exe, too?

digiboy

I build Windows version x86.
It should support wildcards.

I compiled without JASPER library.
So only raw from RED camera shouldn't work

Link to exe file:
https://drive.google.com/open?id=1jULQntHJr_ddz_HDOImepXS8LG80Oyra

If something doesn't work, please report an error.

Danne

Quote from: lorenzo353 on October 20, 2018, 08:50:07 PM
Dcraw 9.28 has been published in june 2018, so Dave Coffin is going on!
That is really good news.

lorenzo353

Hi,

FYI, the CRX codec of the CR3 format has been reverse engineered and open sourced: https://github.com/LibRaw/LibRaw/blob/master/src/decoders/crx.cpp

Kind regards,

Laurent

Danne

Nice!
@Ili3101
Maybe something to consider for raw2mlv?

DeafEyeJedi

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

masc

Quote from: Danne on November 09, 2019, 03:36:23 PM
@Ili3101
Maybe something to consider for raw2mlv?
Should work out of the box. Just update sources of libraw and compile.
5D3.113 | EOSM.202

Chronic

Quote from: Walter Schulz on March 12, 2018, 11:00:56 PM
Ouch! Not the news I wanted to read. Totally slipped under my radar!

If anyone is on Linux, there is a command line program called dnglab that converts like a champ with several different output conversion formats including layers [channels].