Limits be gone.

[jonas18z]

Did some search on google and found this perl script, maybe it could help for patching the file?

Update (2012-02-18): Vladimir Ivanov has contributed an updated version of the script – it now handles 64-bit QuickTime files properly and can also extend 32-bit files to 64-bit if they grow above the limit because of the hidden container. There's also a new option to temporarily modify the hybrid file in a way that makes it possible to change the TrueCrypt password of the encrypted volume.

http://keyj.emphy.de/files/tcsteg2.py
http://keyj.emphy.de/real-steganography-with-truecrypt/

[g3gg0]

okay it seems just to do one part.

the .mov file has so called "atoms" which have a 4 byte size field in front of the field name.
that tool patches these size fields.
but we have to patch stco to co64 atoms. the script 'just' uses the atom that is required.
we have to fix stuff in the atoms.


i have one problem with patching container. tried to patch the size of 'mdat' to use 64 bit sizes.
that didnt work - the file could not be played back.
then i just made the section before (some filler) by 8 byte smaller  (the 64 bit length i added) and the file played.
so there might be an absolute reference into the mdat section somewhere, but didnt find anything
did you see anything that might be a absolute pointer into 'mdat'?

[1%]

I agree the file repairer is much much faster. If one is made we're pretty much complete on 600d.


Did you look at the chunk offset table? That has a bunch of absolute addresses for the chunks and they are ~32bit

0x00000000000106CF stco <Size : 0x00000000000003B0> Chunk Offset Atom/Box
                             

[g3gg0]

yep.

the absolute addresses you see in the dumper, are not in the file. these are the adresses, the dumper uses.
only the lengths are in the atom structure.

edit: oh sorry, misunderstood your post.
true, thats the table we must patch.
but the offsets there are "from the start of mdat section", so they are no absolut file positions (at least the QT docs said that iirc)

[1%]

So it still didn't play after you patched chunk table and mdat? When I tweaked mdat file size there were still 2 more "fake" mdats in the dumper. (i was trying to make it into 1) But I never patched the chunks.


My first chunk is at ~0x38008

my mdat is at 0x0000000000038000 mdat <Size : 0x000000001BE38A40> Movie Data Atom/Box

[g3gg0]

i just patched the size of mdat by setting size to 1 and adding 8 byte extended length.
video didnt play anymore.

stco positions are relative to mdat data payload start, so i dont have to patch them (except of 32->64 bit conversion).

[1%]

Do the mdat sections from the corrupt file collapse into 1 in the atom dumper?

0x38008... with a 64bit size this start isn't affected? Or are all of these populated automatically?

Also, sync samples have a length in them and chunks have an address... the numbers seem to match up so each sync sample = 1 chunk except there is 1 less sync sample?


Sample-To-Chunk Table
                                          (      0) First Chunk :       1, Samples Per Chunk :       1, Sample Description ID :       1
                              0x000000000001198B stsz <Size : 0x00000000000110F8> Sample Size Atom/Box
                                   


                                          (      0) Sample Size :   99788
                                          (      1) Sample Size :   78896
                                   

                                          (  17463) Sample Size :  246228
                                          (  17464) Sample Size :  337912
                             

                                          (      0) Chunk Offset :  0x00038008
                                          (      1) Chunk Offset :  0x000505D4
                                          (  17462) Chunk Offset :  0x1BD900B0
                                          (  17463) Chunk Offset :  0x1BDE2074
                                          (  17464) Chunk Offset :  0x1BE1E248

Also... maybe the players don't understand large sizes for stco/mdat? Need to look at how a real 64bit mov file is laid out.

[g3gg0]

my wrong:

Note that offsets are file offsets, not the offset into any atom within the file (for example, a 'mdat'
atom). This permits referring to media data in files without any atom structure. However , be careful
when constructing a self-contained QuickTime file with its metadata (movie atom) at the front because
the size of the movie atom affects the chunk offsets to the media data.

the offsets ARE absolute. then the mdat patching problem makes sense.

[1%]

If anybody has a over 4gb AVC/h.264 .mov that plays, installing the atom dumper and exporting the headers would help us.

[Kenjee]

I`m rendering one with 20 minutes, I`ll dump the header.

[ilguercio]

What can be done for cameras which don't support exFAT?

[g3gg0]

task:
allow recordings >4GiB with all cameras

problems:
a) camera limits recording to 4GiB
b) FAT does not support files > 4GiB, some cameras support exFAT, some not
d) camera uses .mov with 32-bit tags only, which woll overflow and for sure cause trouble

solutions:
a) we can patch that "max length" field (success on 600D, others will follow)
b) we can either use exFAT or hook the writer routines to split the huge container into separate files (first works, second not yet)
c) we can patch camera firmware (a lot of work and very model-specific) or build a fixer that corrects the file afterwards (preferred)

already explained

[ilguercio]

Ok, looks like you're trying your best. I wish you some success.

[Kenjee]

Ok here it is.

It is around 15 minutes with the size of 7.6 GB

I`m not a developer of any kind so this is educated guess.

From AtomBoxStudio I`ve exported structure

http://www.megafileupload.com/en/file/363853/Exported-Structure-zip.html


And this is under Movie Header Atom/Box:

Code Select Expand

0x00000001E98DBDD0   74 00 00 00 00 00 05 21 93 6D 6F 6F 76 00 00 00   t......!.moov...
0x00000001E98DBDE0   6C 6D 76 68 64 00 00 00 00 CC 49 B6 84 CC 49 C6   lmvhd....ÌI¶.ÌIÆ
0x00000001E98DBDF0   B2 00 00 75 30 01 9D 60 78 00 01 00 00 01 00 00   ²..u0..`x.......
0x00000001E98DBE00   00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00   ................
0x00000001E98DBE10   00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00   ................
0x00000001E98DBE20   00 00 00 00 00 00 00 00 00 40 00 00 00 00 00 00   .........@......
0x00000001E98DBE30   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0x00000001E98DBE40   00 00 00 00 00 00 00 00 04 00 04 F7 48 74 72 61   ...........÷Htra
0x00000001E98DBE50   6B 00 00 00 5C 74 6B 68 64 00 00 00 0F CC 49 B6   k...\tkhd....ÌI¶
0x00000001E98DBE60   84 CC 49 C6 B2 00 00 00 01 00 00 00 00 01 9D 60   .ÌIƲ..........`
0x00000001E98DBE70   78 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   x...............
0x00000001E98DBE80   00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0x00000001E98DBE90   00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0x00000001E98DBEA0   00 40 00 00 00 07 80 00 00 04 38 00 00 00 00 00   [email protected].....
0x00000001E98DBEB0   18 6C 6F 61 64 00 00 00 00 00 00 00 00 00 00 00   .load...........
0x00000001E98DBEC0   00 00 00 01 00 00 00 00 24 65 64 74 73 00 00 00   ........$edts...
0x00000001E98DBED0   1C 65 6C 73 74 00 00 00 00 00 00 00 01 01 9D 60   .elst..........`
0x00000001E98DBEE0   78 00 00 00 00 00 01 00 00 00 00 00 14 74 72 65   x............tre
0x00000001E98DBEF0   66 00 00 00 0C 74 6D 63 64 00 00 00 03 00 04 F6   f....tmcd......ö
0x00000001E98DBF00   94 6D 64 69 61 00 00 00 20 6D 64 68 64 00 00 00   .mdia... mdhd...
0x00000001E98DBF10   00 CC 49 B6 84 CC 49 C6 B2 00 00 75 30 01 9D 60   .ÌI¶.ÌIƲ..u0..`
0x00000001E98DBF20   78 00 00 00 00 00 00 00 3A 68 64 6C 72 00 00 00   x.......:hdlr...
0x00000001E98DBF30   00 6D 68 6C 72 76 69 64 65 61 70 70 6C 00 00 00   .mhlrvideappl...
0x00000001E98DBF40   00 00 01 00 23 19 41 70 70 6C 65 20 56 69 64 65   ....#.Apple Vide
0x00000001E98DBF50   6F 20 4D 65 64 69 61 20 48 61 6E 64 6C 65 72 00   o Media Handler.
0x00000001E98DBF60   04 F6 32 6D 69 6E 66 00 00 00 14 76 6D 68 64 00   .ö2minf....vmhd.
0x00000001E98DBF70   00 00 01 00 40 80 00 80 00 80 00 00 00 00 39 68   [email protected]
0x00000001E98DBF80   64 6C 72 00 00 00 00 64 68 6C 72 61 6C 69 73 61   dlr....dhlralisa
0x00000001E98DBF90   70 70 6C 00 00 00 01 00 01 00 2E 18 41 70 70 6C   ppl.........Appl
0x00000001E98DBFA0   65 20 41 6C 69 61 73 20 44 61 74 61 20 48 61 6E   e Alias Data Han
0x00000001E98DBFB0   64 6C 65 72 00 00 00 24 64 69 6E 66 00 00 00 1C   dler...$dinf....
0x00000001E98DBFC0   64 72 65 66 00 00 00 00 00 00 00 01 00 00 00 0C   dref............
0x00000001E98DBFD0   61 6C 69 73 00 00 00 01 00 04 F5 B9 73 74 62 6C   alis......õ¹stbl
0x00000001E98DBFE0   00 00 00 B9 73 74 73 64 00 00 00 00 00 00 00 01   ...¹stsd........
0x00000001E98DBFF0   00 00 00 A9 61 76 63 31 00 00 00 00 00 00 00 01   ...©avc1........
0x00000001E98DC000   00 01 00 01 61 70 70 6C 00 00 00 00 00 00 04 00   ....appl........
0x00000001E98DC010   07 80 04 38 00 48 00 00 00 48 00 00 00 00 00 00   ...8.H...H......
0x00000001E98DC020   00 01 05 48 2E 32 36 34 00 00 00 00 00 00 00 00   ...H.264........
0x00000001E98DC030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0x00000001E98DC040   00 00 00 18 FF FF 00 00 00 12 63 6F 6C 72 6E 63   ....ÿÿ....colrnc
0x00000001E98DC050   6C 63 00 01 00 01 00 01 00 00 00 10 70 61 73 70   lc..........pasp
0x00000001E98DC060   00 00 07 80 00 00 07 80 00 00 00 2D 61 76 63 43   ...........-avcC
0x00000001E98DC070   01 4D 40 33 FF 01 00 16 27 4D 40 33 A9 18 0F 00   .M@3ÿ...'M@3©...
0x00000001E98DC080   44 FC B8 03 50 10 10 1B 6C 2B 5E F7 C0 40 01 00   Dü¸.P...l+^÷À@..
0x00000001E98DC090   04 28 DE 09 C8 00 00 00 00 00 00 00 18 73 74 74   .(Þ.È........stt
0x00000001E98DC0A0   73 00 00 00 00 00 00 00 01 00 00 69 B8 00 00 03   s..........i¸...
0x00000001E98DC0B0   E9 00 00 00 1C 73 74 73 63 00 00 00 00 00 00 00   é....stsc.......

I hope I`ve dumped it ok, if not tell me how.

Cheerz

[g3gg0]

i am building a chunk fixer right now.

[g3gg0]

600D users who have files larger than 4GiB: try this!
http://upload.g3gg0.de/pub_files/6ed398407e34c3fb2b443923f6f9295e/EOSMovieFixer.exe

(updated)

linux/mac osx users: run it with mono. should work.

[jonas18z]

Good work, I. Don't have the 600d so can't try. How did you solve the patching?

[1%]

Not running under wine. it just quits out. What about patch in place so you don't have to re-save? Too risky?


Also patch for ML so I can combine this with 600D audio. Although I have not tried running over 4gb with sound yet. So far 4.5gb and 5gb movies fixed perfectly. I need to try that 7gb movie next. Now I can record a ~64gb file

[g3gg0]

you do not need wine, use mono. its a .NET program.
didnt try, but its as simple as hello world. it will work for sure.

as explained above, patching flash routines would be possible but
a) highly model/version dependent
b) a lot of work
c) not clear if it will cause severe side effects

so i chose the "safe" way of fixing the file afterwards.

[1%]

No, I mean patch the file in place instead of saving new copy. Or are you pretty much rewriting the file anyway when you make the addresses 64bit?

[g3gg0]

1. i am reading all atoms, their positions and the reference into source file.
2. if encountering a 'mdat' section, i try to find its end by adding 0x0100000000 (32 bit overflow bug)
3. then i read the 'stco' section content, extract the offsets relative to 'mdat' start. (original is absolute file position)
4. in the next step i patch 'stco' to 'co64' which contains 64 bit pointers
5. then i calculate the new size of all atoms
6. after that i update the newly created 'co64' section with pointers not relative to 'mdat', but the absolute file position
7. as last step the whole file is written to disk using the new atoms

so yes, i am rewriting the *whole* .mov file. not just patching some bits.

[g3gg0]

reason for rewriting:
canon kinda reserves the first 0x30000 bytes in the file for headers, pointers etc.
but i am not sure if it this is constant or if i can be sure that all models have that.

i could rely on that offset and try to patch the header contents only.
but the main work is reading/parsing/extending header atoms, and thats something i dont want to do in C.
i could do it in C, but that would be either ugly as hell or it would be really a lot of work.
nothing i would do for fun

so the solution to patch it later on computer is my favorite, because we can merge split files there too.
(other models without exFAT would require it anyway)

as the program is simplest .NET, it will run on linux/osx as well.
i am using windows forms (buttons, treeview and textbox) BinaryReader, BinaryWriter and Open/SaveFileDialog.
i do not expect that it wont work in mono maybe the .NET version can be lowered, but i think mono can run .NET 3.x
can you try it?

[1%]

I was missing a bunch of stuff from mono like the forms, it runs now. Its easy enough to fix the files after. The other part sounds like too much work. I wonder what sound will do, I have not tried a movie with audio, thats up tomorrow since its 320am here. Or did you patch for that already?

[g3gg0]

uuhm patching audio?
no need. everthing is getting fixed.
the file should be perfect afterwards. (and even smaller since i clean up the unnecessary atoms)

[1%]

There was a separate atom for audio I believe.

I'm missing Open/SaveFileDialog. from mono, not sure which package its in.

Lulz, it pops under.


It was:

0x0000000000010B34 smhd <Size : 0x0000000000000010> Sound Media Information Header Atom/Box

and sowt data + its chunks.