Unknown Block errors PLEASE HELP

sax100 · September 20, 2018, 05:19:55 PM

Ok, so I tried @Danne's mlv_dump and it's an improvement!

I don't know how to run the patch, which I probably need.

Rendered frames 1-1343 and 1582-2090 (frame 2090 is the end, so missing 238 continuous frames)

These are the errors I got:

Code Select

 MLV Dumper
-----------------

Mode of operation:
   - Input MLV file: 'M15-0904.MLV'
   - Using output path 'M15-0904_' for DNGs
   - Convert to DNG frames
   - Output into 'M15-0904_'
File M15-0904.MLV opened
File M15-0904.M00 opened
File M15-0904.M01 not existing.
Processing...
Unknown Block: %%, skipping
Next known block: NULL at 0x7db66ae0+0x280 = 0x7db66d60
[ERROR] Invalid block size at position 0xd95feae0
Next known block: NULL at 0xe15feaf0+0x270 = 0xe15feaf0
[ERROR] Invalid block size at position 0xe15feaf0
Next known block: NULL at 0xe95feb00+0x11a260 = 0xe95feb00
[ERROR] Invalid block size at position 0xe95feb00
Next known block: NULL at 0xf15feb10+0x233fd0 = 0xf15feb10
[ERROR] Invalid block size at position 0xf15feb10
Next known block: NULL at 0xf95feb20+0xb9240 = 0xf95feb20
[ERROR] Invalid block size at position 0xf95feb20
Next known block: NULL at 0xfff1a260+0x1d2fb0 = 0xfff1a260

Reached end of chunk 1/2 after 3512 blocks
[ERROR] Invalid block size at position 0x00a538b4
Next known block: NULL at 0xa538c4+0x270 = 0xa53b34
[ERROR] Invalid block size at position 0x012128b4
Next known block: NULL at 0x12128c4+0x270 = 0x1212b34
[ERROR] Invalid block size at position 0x0173c8b4

Code Select

Next known block: NULL at 0x512c28c4+0x270 = 0x512c2b34
[ERROR] Invalid block size at position 0x517ec8b4
Next known block: NULL at 0x517ec8c4+0x270 = 0x517ecb34
[ERROR] Invalid block size at position 0x51d168b4
Next known block: NULL at 0x51d168c4+0x270 = 0x51d16b34
[ERROR] Invalid block size at position 0x522408b4
Next known block: NULL at 0x522408c4+0x270 = 0x52240b34

Reached end of chunk 2/2 after 1532 blocks
Processed 1853 video frames at 24.00 FPS (77.21 s)
Done

Danne · September 20, 2018, 05:29:43 PM

What you got already is the patch in mlv_dump. You don't patch the mlv.
If interested in digging into code and applying patches download the source and dependencies. Search 'compiling macintosh' to get into some useful guidance.

dfort · September 20, 2018, 05:50:07 PM

@sax100

Long story longer -- Like Danne said, the source code needs to be patched then compiled. You can't just run the patch on the binary or the file you're trying to process.

I ran Danne's build on the problem MLV and it extracted what seems to be all the frames from that file. The audio cuts out early. Note that this is only the first 4GB chunk of that shot (1344 frames), there is also a second file named M15-0904.M00 that needs to be tacked onto the end of this.

[EDIT] Saw you ran it on both M15-0904.MLV and M15-0904.M00. Curious if the audio runs the full length of the clip.

Quote from: a1ex on September 20, 2018, 12:09:10 PM
Pretty sure it was this bug. You should have mentioned all the relevant info in this thread.

That was a double posting issue. The relevant info was in the other topic:

Quote from: sax100 on September 17, 2018, 06:50:43 PM
Canon 5D Mark III / KomputerBay 1000x 128GB CF
ML for 5d3 2014.01.08 Build
Firmware Version 1.1.3-ml-v2.3

Though this is in your signature, not all forum readers see this.

ML is in a constant state of development and the bug that caused the problems you're experiencing has been fixed. That's why users are encouraged to use the latest builds.

a1ex · September 20, 2018, 05:57:58 PM

Maybe some Mac compatibility issue, possibly at fseek? On Linux, with the same MLV file I get valid DNGs until frame 1581.

edit: updated the patch and confirmed it's working on the Mac VM:

mlv_dump_bruteforce-v2.patch

Code Select


mlv_dump M15-0904.MLV --dng

 MLV Dumper
-----------------

Mode of operation:
   - Input MLV file: 'M15-0904.MLV'
   - Using output path 'M15-0904_' for DNGs
   - Convert to DNG frames
   - Output into 'M15-0904_'
File M15-0904.MLV opened
File M15-0904.M00 not existing.
Processing...
Unknown Block: %%, skipping
Next known block: NULL at 0x7db66ae0+0x280 = 0x7db66d60
[ERROR] Invalid block size at position 0xd95feae0
Next known block: NULL at 0xd95feaf0+0x270 = 0xd95fed60
[ERROR] Invalid block size at position 0xda052ae0
Next known block: NULL at 0xda052af0+0x270 = 0xda052d60
[ERROR] Invalid block size at position 0xda57cae0
Next known block: NULL at 0xda57caf0+0x270 = 0xda57cd60
[ERROR] Invalid block size at position 0xdaaa6ae0
Next known block: NULL at 0xdaaa6af0+0x270 = 0xdaaa6d60
[ERROR] Invalid block size at position 0xdafd0ae0
Next known block: NULL at 0xdafd0af0+0x270 = 0xdafd0d60
[ERROR] Invalid block size at position 0xdb4faae0
Next known block: NULL at 0xdb4faaf0+0x270 = 0xdb4fad60
[ERROR] Invalid block size at position 0xdba24ae0
Next known block: NULL at 0xdba24af0+0x270 = 0xdba24d60
[ERROR] Invalid block size at position 0xdbf4eae0
Next known block: NULL at 0xdbf4eaf0+0x270 = 0xdbf4ed60
[ERROR] Invalid block size at position 0xdc478ae0
Next known block: NULL at 0xdc478af0+0x270 = 0xdc478d60
[ERROR] Invalid block size at position 0xdc9a2ae0
Next known block: NULL at 0xdc9a2af0+0x270 = 0xdc9a2d60
[ERROR] Invalid block size at position 0xdd161ae0
Next known block: NULL at 0xdd161af0+0x270 = 0xdd161d60
[ERROR] Invalid block size at position 0xdd68bae0
Next known block: NULL at 0xdd68baf0+0x270 = 0xdd68bd60
[ERROR] Invalid block size at position 0xddbb5ae0
Next known block: NULL at 0xddbb5af0+0x270 = 0xddbb5d60
[ERROR] Invalid block size at position 0xde0dfae0
Next known block: NULL at 0xde0dfaf0+0x270 = 0xde0dfd60
[ERROR] Invalid block size at position 0xde89eae0
Next known block: NULL at 0xde89eaf0+0x270 = 0xde89ed60
[ERROR] Invalid block size at position 0xdedc8ae0
Next known block: NULL at 0xdedc8af0+0x270 = 0xdedc8d60
[ERROR] Invalid block size at position 0xdf2f2ae0
Next known block: NULL at 0xdf2f2af0+0x270 = 0xdf2f2d60
[ERROR] Invalid block size at position 0xdf81cae0
Next known block: NULL at 0xdf81caf0+0x270 = 0xdf81cd60
[ERROR] Invalid block size at position 0xdfd46ae0
Next known block: NULL at 0xdfd46af0+0x270 = 0xdfd46d60
[ERROR] Invalid block size at position 0xe0270ae0
Next known block: NULL at 0xe0270af0+0x270 = 0xe0270d60
[ERROR] Invalid block size at position 0xe079aae0
Next known block: NULL at 0xe079aaf0+0x270 = 0xe079ad60
[ERROR] Invalid block size at position 0xe0f59ae0
Next known block: NULL at 0xe0f59af0+0x270 = 0xe0f59d60
[ERROR] Invalid block size at position 0xe1718ae0
Next known block: NULL at 0xe1718af0+0x270 = 0xe1718d60
[ERROR] Invalid block size at position 0xe1ed7ae0
Next known block: NULL at 0xe1ed7af0+0x270 = 0xe1ed7d60
[ERROR] Invalid block size at position 0xe2696ae0
Next known block: NULL at 0xe2696af0+0x270 = 0xe2696d60
[ERROR] Invalid block size at position 0xe2e55ae0
Next known block: NULL at 0xe2e55af0+0x270 = 0xe2e55d60
[ERROR] Invalid block size at position 0xe337fae0
Next known block: NULL at 0xe337faf0+0x270 = 0xe337fd60
[ERROR] Invalid block size at position 0xe38a9ae0
Next known block: NULL at 0xe38a9af0+0x270 = 0xe38a9d60
[ERROR] Invalid block size at position 0xe3dd3ae0
Next known block: NULL at 0xe3dd3af0+0x270 = 0xe3dd3d60
[ERROR] Invalid block size at position 0xe4592ae0
Next known block: NULL at 0xe4592af0+0x270 = 0xe4592d60
[ERROR] Invalid block size at position 0xe4abcae0
Next known block: NULL at 0xe4abcaf0+0x270 = 0xe4abcd60
[ERROR] Invalid block size at position 0xe4fe6ae0
Next known block: NULL at 0xe4fe6af0+0x270 = 0xe4fe6d60
[ERROR] Invalid block size at position 0xe5510ae0
Next known block: NULL at 0xe5510af0+0x270 = 0xe5510d60
[ERROR] Invalid block size at position 0xe5ccfae0
Next known block: NULL at 0xe5ccfaf0+0x270 = 0xe5ccfd60
[ERROR] Invalid block size at position 0xe648eae0
Next known block: NULL at 0xe648eaf0+0x270 = 0xe648ed60
[ERROR] Invalid block size at position 0xe6c4dae0
Next known block: NULL at 0xe6c4daf0+0x270 = 0xe6c4dd60
[ERROR] Invalid block size at position 0xe7177ae0
Next known block: NULL at 0xe7177af0+0x270 = 0xe7177d60
[ERROR] Invalid block size at position 0xe76a1ae0
Next known block: NULL at 0xe76a1af0+0x270 = 0xe76a1d60
[ERROR] Invalid block size at position 0xe7bcbae0
Next known block: NULL at 0xe7bcbaf0+0x270 = 0xe7bcbd60
[ERROR] Invalid block size at position 0xe838aae0
Next known block: NULL at 0xe838aaf0+0x270 = 0xe838ad60
[ERROR] Invalid block size at position 0xe88b4ae0
Next known block: NULL at 0xe88b4af0+0x270 = 0xe88b4d60
[ERROR] Invalid block size at position 0xe8ddeae0
Next known block: NULL at 0xe8ddeaf0+0x270 = 0xe8dded60
[ERROR] Invalid block size at position 0xe9308ae0
Next known block: NULL at 0xe9308af0+0x270 = 0xe9308d60
[ERROR] Invalid block size at position 0xe9ac7ae0
Next known block: NULL at 0xe9ac7af0+0x270 = 0xe9ac7d60
[ERROR] Invalid block size at position 0xea286ae0
Next known block: NULL at 0xea286af0+0x270 = 0xea286d60
[ERROR] Invalid block size at position 0xea7b0ae0
Next known block: NULL at 0xea7b0af0+0x270 = 0xea7b0d60
[ERROR] Invalid block size at position 0xeacdaae0
Next known block: NULL at 0xeacdaaf0+0x270 = 0xeacdad60
[ERROR] Invalid block size at position 0xeb204ae0
Next known block: NULL at 0xeb204af0+0x270 = 0xeb204d60
[ERROR] Invalid block size at position 0xeb9c3ae0
Next known block: NULL at 0xeb9c3af0+0x270 = 0xeb9c3d60
[ERROR] Invalid block size at position 0xec182ae0
Next known block: NULL at 0xec182af0+0x270 = 0xec182d60
[ERROR] Invalid block size at position 0xec941ae0
Next known block: NULL at 0xec941af0+0x270 = 0xec941d60
[ERROR] Invalid block size at position 0xed100ae0
Next known block: NULL at 0xed100af0+0x270 = 0xed100d60
[ERROR] Invalid block size at position 0xed8bfae0
Next known block: NULL at 0xed8bfaf0+0x270 = 0xed8bfd60
[ERROR] Invalid block size at position 0xee07eae0
Next known block: NULL at 0xee07eaf0+0x270 = 0xee07ed60
[ERROR] Invalid block size at position 0xee5a8ae0
Next known block: NULL at 0xee5a8af0+0x270 = 0xee5a8d60
[ERROR] Invalid block size at position 0xeead2ae0
Next known block: NULL at 0xeead2af0+0x270 = 0xeead2d60
[ERROR] Invalid block size at position 0xeeffcae0
Next known block: NULL at 0xeeffcaf0+0x270 = 0xeeffcd60
[ERROR] Invalid block size at position 0xef7bbae0
Next known block: NULL at 0xef7bbaf0+0x270 = 0xef7bbd60
[ERROR] Invalid block size at position 0xeff7aae0
Next known block: NULL at 0xeff7aaf0+0x270 = 0xeff7ad60
[ERROR] Invalid block size at position 0xf0739ae0
Next known block: NULL at 0xf0739af0+0x270 = 0xf0739d60
[ERROR] Invalid block size at position 0xf0ef8ae0
Next known block: NULL at 0xf0ef8af0+0x270 = 0xf0ef8d60
[ERROR] Invalid block size at position 0xf16b7ae0
Next known block: NULL at 0xf16b7af0+0x270 = 0xf16b7d60
[ERROR] Invalid block size at position 0xf1e76ae0
Next known block: NULL at 0xf1e76af0+0x270 = 0xf1e76d60
[ERROR] Invalid block size at position 0xf23a0ae0
Next known block: NULL at 0xf23a0af0+0x270 = 0xf23a0d60
[ERROR] Invalid block size at position 0xf28caae0
Next known block: NULL at 0xf28caaf0+0x270 = 0xf28cad60
[ERROR] Invalid block size at position 0xf2df4ae0
Next known block: NULL at 0xf2df4af0+0x270 = 0xf2df4d60
[ERROR] Invalid block size at position 0xf331eae0
Next known block: NULL at 0xf331eaf0+0x270 = 0xf331ed60
[ERROR] Invalid block size at position 0xf3848ae0
Next known block: NULL at 0xf3848af0+0x270 = 0xf3848d60
[ERROR] Invalid block size at position 0xf3d72ae0
Next known block: NULL at 0xf3d72af0+0x270 = 0xf3d72d60
[ERROR] Invalid block size at position 0xf429cae0
Next known block: NULL at 0xf429caf0+0x270 = 0xf429cd60
[ERROR] Invalid block size at position 0xf47c6ae0
Next known block: NULL at 0xf47c6af0+0x270 = 0xf47c6d60
[ERROR] Invalid block size at position 0xf4cf0ae0
Next known block: NULL at 0xf4cf0af0+0x270 = 0xf4cf0d60
[ERROR] Invalid block size at position 0xf521aae0
Next known block: NULL at 0xf521aaf0+0x270 = 0xf521ad60
[ERROR] Invalid block size at position 0xf5744ae0
Next known block: NULL at 0xf5744af0+0x270 = 0xf5744d60
[ERROR] Invalid block size at position 0xf5c6eae0
Next known block: NULL at 0xf5c6eaf0+0x270 = 0xf5c6ed60
[ERROR] Invalid block size at position 0xf6198ae0
Next known block: NULL at 0xf6198af0+0x270 = 0xf6198d60
[ERROR] Invalid block size at position 0xf66c2ae0
Next known block: NULL at 0xf66c2af0+0x270 = 0xf66c2d60
[ERROR] Invalid block size at position 0xf6becae0
Next known block: NULL at 0xf6becaf0+0x270 = 0xf6becd60
[ERROR] Invalid block size at position 0xf7116ae0
Next known block: NULL at 0xf7116af0+0x270 = 0xf7116d60
[ERROR] Invalid block size at position 0xf7640ae0
Next known block: NULL at 0xf7640af0+0x270 = 0xf7640d60
[ERROR] Invalid block size at position 0xf7b6aae0
Next known block: NULL at 0xf7b6aaf0+0x270 = 0xf7b6ad60
[ERROR] Invalid block size at position 0xf8094ae0
Next known block: NULL at 0xf8094af0+0x270 = 0xf8094d60
[ERROR] Invalid block size at position 0xf85beae0
Next known block: NULL at 0xf85beaf0+0x270 = 0xf85bed60
[ERROR] Invalid block size at position 0xf8ae8ae0
Next known block: NULL at 0xf8ae8af0+0x270 = 0xf8ae8d60
[ERROR] Invalid block size at position 0xf92a7ae0
Next known block: NULL at 0xf92a7af0+0x270 = 0xf92a7d60
[ERROR] Invalid block size at position 0xf9a66ae0
Next known block: NULL at 0xf9a66af0+0x270 = 0xf9a66d60
[ERROR] Invalid block size at position 0xfa225ae0
Next known block: NULL at 0xfa225af0+0x270 = 0xfa225d60
[ERROR] Invalid block size at position 0xfa9e4ae0
Next known block: NULL at 0xfa9e4af0+0x270 = 0xfa9e4d60
[ERROR] Invalid block size at position 0xfaf0eae0
Next known block: NULL at 0xfaf0eaf0+0x270 = 0xfaf0ed60
[ERROR] Invalid block size at position 0xfb438ae0
Next known block: NULL at 0xfb438af0+0x270 = 0xfb438d60
[ERROR] Invalid block size at position 0xfb962ae0
Next known block: NULL at 0xfb962af0+0x270 = 0xfb962d60
[ERROR] Invalid block size at position 0xfbe8cae0
Next known block: NULL at 0xfbe8caf0+0x270 = 0xfbe8cd60
[ERROR] Invalid block size at position 0xfc3b6ae0
Next known block: NULL at 0xfc3b6af0+0x270 = 0xfc3b6d60
[ERROR] Invalid block size at position 0xfc8e0ae0
Next known block: NULL at 0xfc8e0af0+0x270 = 0xfc8e0d60
[ERROR] Invalid block size at position 0xfce0aae0
Next known block: NULL at 0xfce0aaf0+0x270 = 0xfce0ad60
[ERROR] Invalid block size at position 0xfd334ae0
Next known block: NULL at 0xfd334af0+0x270 = 0xfd334d60
[ERROR] Invalid block size at position 0xfd85eae0
Next known block: NULL at 0xfd85eaf0+0x270 = 0xfd85ed60
[ERROR] Invalid block size at position 0xfdd88ae0
Next known block: NULL at 0xfdd88af0+0x270 = 0xfdd88d60
[ERROR] Invalid block size at position 0xfe2b2ae0
Next known block: NULL at 0xfe2b2af0+0x270 = 0xfe2b2d60
[ERROR] Invalid block size at position 0xfe7dcae0
Next known block: NULL at 0xfe7dcaf0+0x270 = 0xfe7dcd60
[ERROR] Invalid block size at position 0xfef9bae0
Next known block: NULL at 0xfef9baf0+0x270 = 0xfef9bd60
[ERROR] Invalid block size at position 0xff4c5ae0
Next known block: NULL at 0xff4c5af0+0x270 = 0xff4c5d60
[ERROR] Invalid block size at position 0xff9efae0
Next known block: NULL at 0xff9efaf0+0x270 = 0xff9efd60
[ERROR] Invalid block size at position 0xfff19ae0
Next known block: NULL at 0xfff19af0+0x270 = 0xfff19d60

Reached end of chunk 1/1 after 4169 blocks
Processed 1582 video frames at 24.00 FPS (65.92 s)
Done

dfort · September 20, 2018, 06:15:03 PM

I get this on my Mac PowerBook using Danne's patched mlv_dump:

Code Select

./mlv_dump --dng M15-0904.MLV 

 MLV Dumper
-----------------

Mode of operation:
   - Input MLV file: 'M15-0904.MLV'
   - Using output path 'M15-0904_' for DNGs
   - Convert to DNG frames
   - Output into 'M15-0904_'
File M15-0904.MLV opened
File M15-0904.M00 not existing.
Processing...
Unknown Block: %%, skipping
Next known block: NULL at 0x7db66ae0+0x280 = 0x7db66d60
[ERROR] Invalid block size at position 0xd95feae0
Next known block: NULL at 0xe15feaf0+0x270 = 0xe15feaf0
[ERROR] Invalid block size at position 0xe15feaf0
Next known block: NULL at 0xe95feb00+0x11a260 = 0xe95feb00
[ERROR] Invalid block size at position 0xe95feb00
Next known block: NULL at 0xf15feb10+0x233fd0 = 0xf15feb10
[ERROR] Invalid block size at position 0xf15feb10
Next known block: NULL at 0xf95feb20+0xb9240 = 0xf95feb20
[ERROR] Invalid block size at position 0xf95feb20
Next known block: NULL at 0xfff1a260+0x1d2fb0 = 0xfff1a260

Reached end of chunk 1/1 after 3512 blocks
Processed 1344 video frames at 24.00 FPS (56.00 s)
Done

So yeah, the Mac is not processing those 238 missing frames.

sax100 · September 20, 2018, 06:26:43 PM

@Danne
Thanks for that, I'll study compiling macintosh docs to get a better understanding of this stuff.

@dfort
I missed that! I'm uploading M15-0904.M00 now (it's 1.4 GB) and will post link shortly.

@a1ex
I haven't had this problem for 3 years using the build I have with same camera and settings, CF cards and Mac OSX 10.85. Could I have some virus or a conflicting software that's affecting this?

@ bouncyball

QuoteBTW audio level is very low)

I need to update to latest build - but what I have now only allows me to adjust speaker level (which is set to 3). I'll need to test with level 4 or 5 once I get this issue settled. Latest build may be updated, so I'll look into that.

I recorded with short boom mic into Zoom H1 which was routed into my Canon 5D3 via mic port. Levels looked normal on screen, but your right, the audio is a little too soft.

a1ex · September 20, 2018, 06:29:22 PM

Quote from: sax100 on September 20, 2018, 06:26:43 PM
@a1ex
I haven't had this problem for 3 years using the build I have with same camera and settings, CF cards and Mac OSX 10.85. Could I have some virus or a conflicting software that's affecting this?

Quote from: a1ex on September 20, 2018, 12:09:10 PM
Pretty sure it was this bug.

Click on the link; it has the answer to your question.

TLDR: edge case, non-deterministic bug, hard to narrow down.

dfort · September 20, 2018, 07:07:55 PM

Quote from: a1ex on September 20, 2018, 05:57:58 PM
Maybe some Mac compatibility issue, possibly at fseek?

Been in fseek hell before. Maybe the Windows exception also applies to the Mac?

Code Select

/* platform/target specific fseek/ftell functions go here */
uint64_t file_get_pos(FILE *stream)
{
#if defined(__WIN32)
    return ftello64(stream);
#else
    return ftello(stream);
#endif
}

uint32_t file_set_pos(FILE *stream, uint64_t offset, int whence)
{
#if defined(__WIN32)
    return fseeko64(stream, offset, whence);
#else
    return fseeko(stream, offset, whence);
#endif
}

How is the Mac defined?

a1ex · September 20, 2018, 07:39:24 PM

Quote from: dfort on September 20, 2018, 07:07:55 PM
How is the Mac defined?

#else, in that snippet, means "anything but __WIN32", so that includes Mac.

Check the updated patch above.

sax100 · September 20, 2018, 08:25:47 PM

Again, I'm still learning... I'm sure I'm doing something wrong.

I downloaded mlv_dump.osx under Last Successful Artifact provided by g3gg0 here.

And @a1ex patch - mlv_dump_bruteforce-v2.patch

Tried to run patch, but failed:

patch < mlv_dump_bruteforce-v2.patch

File to patch: mlv_dump.osx
patching file mlv_dump.osx
Hunk #1 FAILED at 448.
Hunk #2 FAILED at 1239.
Hunk #3 FAILED at 2155.
Hunk #4 FAILED at 4112.
4 out of 4 hunks FAILED -- saving rejects to file mlv_dump.osx.rej

dfort · September 20, 2018, 09:50:35 PM

Quote from: sax100 on September 20, 2018, 08:25:47 PM
patch < mlv_dump_bruteforce-v2.patch

That's not the way to apply the patch. It needs to be applied to the source code. Compiling mlv_dump is a more advanced topic that we probably don't need to go into at this time but I'll be glad to discuss it with you when we meet--we're practically neighbors.

Quote from: a1ex on September 20, 2018, 07:39:24 PM
#else, in that snippet, means "anything but __WIN32", so that includes Mac.

Check the updated patch above.

Right, I know that. Just thought I'd experiment with different ftello/fseeko options for the Mac. No luck so far.

The patch applied cleanly but I'm not getting the 1582 frames you're getting on Linux.

a1ex · September 20, 2018, 09:51:57 PM

I'm getting 1582 frames on Mac, too...

dfort · September 20, 2018, 10:07:24 PM

Quote from: a1ex on September 20, 2018, 05:57:58 PM
edit: updated the patch and confirmed it's working on the Mac VM:

mlv_dump_bruteforce-v2.patch

Doh! Missed that link to the v2 patch. Yes, it works on the Mac!

@sax100 - Try the new mlv_dump I compiled for you and please report back.

Danne · September 20, 2018, 10:10:06 PM

Yup, version 2 works, kudos a1ex:

mlv_dump version here:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump

Code Select

Last login: Thu Sep 20 22:12:37 on ttys003
dans-MacBook-Pro:corruption dan$ /Users/dan/Desktop/Timer_A/corruption/mlv_dump --dng /Users/dan/Desktop/Timer_A/corruption/M15-0904.MLV 

 MLV Dumper
-----------------

Mode of operation:
   - Input MLV file: '/Users/dan/Desktop/Timer_A/corruption/M15-0904.MLV'
   - Using output path '/Users/dan/Desktop/Timer_A/corruption/M15-0904_' for DNGs
   - Convert to DNG frames
   - Output into '/Users/dan/Desktop/Timer_A/corruption/M15-0904_'
File /Users/dan/Desktop/Timer_A/corruption/M15-0904.MLV opened
File /Users/dan/Desktop/Timer_A/corruption/M15-0904.M00 not existing.
Processing...
Unknown Block: %%, skipping
Next known block: NULL at 0x7db66ae0+0x280 = 0x7db66d60
[ERROR] Invalid block size at position 0xd95feae0
Next known block: NULL at 0xd95feaf0+0x270 = 0xd95fed60
[ERROR] Invalid block size at position 0xda052ae0
Next known block: NULL at 0xda052af0+0x270 = 0xda052d60
[ERROR] Invalid block size at position 0xda57cae0
Next known block: NULL at 0xda57caf0+0x270 = 0xda57cd60
[ERROR] Invalid block size at position 0xdaaa6ae0
Next known block: NULL at 0xdaaa6af0+0x270 = 0xdaaa6d60
[ERROR] Invalid block size at position 0xdafd0ae0
Next known block: NULL at 0xdafd0af0+0x270 = 0xdafd0d60
[ERROR] Invalid block size at position 0xdb4faae0
Next known block: NULL at 0xdb4faaf0+0x270 = 0xdb4fad60
[ERROR] Invalid block size at position 0xdba24ae0
Next known block: NULL at 0xdba24af0+0x270 = 0xdba24d60
[ERROR] Invalid block size at position 0xdbf4eae0
Next known block: NULL at 0xdbf4eaf0+0x270 = 0xdbf4ed60
[ERROR] Invalid block size at position 0xdc478ae0
Next known block: NULL at 0xdc478af0+0x270 = 0xdc478d60
[ERROR] Invalid block size at position 0xdc9a2ae0
Next known block: NULL at 0xdc9a2af0+0x270 = 0xdc9a2d60
[ERROR] Invalid block size at position 0xdd161ae0
Next known block: NULL at 0xdd161af0+0x270 = 0xdd161d60
[ERROR] Invalid block size at position 0xdd68bae0
Next known block: NULL at 0xdd68baf0+0x270 = 0xdd68bd60
[ERROR] Invalid block size at position 0xddbb5ae0
Next known block: NULL at 0xddbb5af0+0x270 = 0xddbb5d60
[ERROR] Invalid block size at position 0xde0dfae0
Next known block: NULL at 0xde0dfaf0+0x270 = 0xde0dfd60
[ERROR] Invalid block size at position 0xde89eae0
Next known block: NULL at 0xde89eaf0+0x270 = 0xde89ed60
[ERROR] Invalid block size at position 0xdedc8ae0
Next known block: NULL at 0xdedc8af0+0x270 = 0xdedc8d60
[ERROR] Invalid block size at position 0xdf2f2ae0
Next known block: NULL at 0xdf2f2af0+0x270 = 0xdf2f2d60
[ERROR] Invalid block size at position 0xdf81cae0
Next known block: NULL at 0xdf81caf0+0x270 = 0xdf81cd60
[ERROR] Invalid block size at position 0xdfd46ae0
Next known block: NULL at 0xdfd46af0+0x270 = 0xdfd46d60
[ERROR] Invalid block size at position 0xe0270ae0
Next known block: NULL at 0xe0270af0+0x270 = 0xe0270d60
[ERROR] Invalid block size at position 0xe079aae0
Next known block: NULL at 0xe079aaf0+0x270 = 0xe079ad60
[ERROR] Invalid block size at position 0xe0f59ae0
Next known block: NULL at 0xe0f59af0+0x270 = 0xe0f59d60
[ERROR] Invalid block size at position 0xe1718ae0
Next known block: NULL at 0xe1718af0+0x270 = 0xe1718d60
[ERROR] Invalid block size at position 0xe1ed7ae0
Next known block: NULL at 0xe1ed7af0+0x270 = 0xe1ed7d60
[ERROR] Invalid block size at position 0xe2696ae0
Next known block: NULL at 0xe2696af0+0x270 = 0xe2696d60
[ERROR] Invalid block size at position 0xe2e55ae0
Next known block: NULL at 0xe2e55af0+0x270 = 0xe2e55d60
[ERROR] Invalid block size at position 0xe337fae0
Next known block: NULL at 0xe337faf0+0x270 = 0xe337fd60
[ERROR] Invalid block size at position 0xe38a9ae0
Next known block: NULL at 0xe38a9af0+0x270 = 0xe38a9d60
[ERROR] Invalid block size at position 0xe3dd3ae0
Next known block: NULL at 0xe3dd3af0+0x270 = 0xe3dd3d60
[ERROR] Invalid block size at position 0xe4592ae0
Next known block: NULL at 0xe4592af0+0x270 = 0xe4592d60
[ERROR] Invalid block size at position 0xe4abcae0
Next known block: NULL at 0xe4abcaf0+0x270 = 0xe4abcd60
[ERROR] Invalid block size at position 0xe4fe6ae0
Next known block: NULL at 0xe4fe6af0+0x270 = 0xe4fe6d60
[ERROR] Invalid block size at position 0xe5510ae0
Next known block: NULL at 0xe5510af0+0x270 = 0xe5510d60
[ERROR] Invalid block size at position 0xe5ccfae0
Next known block: NULL at 0xe5ccfaf0+0x270 = 0xe5ccfd60
[ERROR] Invalid block size at position 0xe648eae0
Next known block: NULL at 0xe648eaf0+0x270 = 0xe648ed60
[ERROR] Invalid block size at position 0xe6c4dae0
Next known block: NULL at 0xe6c4daf0+0x270 = 0xe6c4dd60
[ERROR] Invalid block size at position 0xe7177ae0
Next known block: NULL at 0xe7177af0+0x270 = 0xe7177d60
[ERROR] Invalid block size at position 0xe76a1ae0
Next known block: NULL at 0xe76a1af0+0x270 = 0xe76a1d60
[ERROR] Invalid block size at position 0xe7bcbae0
Next known block: NULL at 0xe7bcbaf0+0x270 = 0xe7bcbd60
[ERROR] Invalid block size at position 0xe838aae0
Next known block: NULL at 0xe838aaf0+0x270 = 0xe838ad60
[ERROR] Invalid block size at position 0xe88b4ae0
Next known block: NULL at 0xe88b4af0+0x270 = 0xe88b4d60
[ERROR] Invalid block size at position 0xe8ddeae0
Next known block: NULL at 0xe8ddeaf0+0x270 = 0xe8dded60
[ERROR] Invalid block size at position 0xe9308ae0
Next known block: NULL at 0xe9308af0+0x270 = 0xe9308d60
[ERROR] Invalid block size at position 0xe9ac7ae0
Next known block: NULL at 0xe9ac7af0+0x270 = 0xe9ac7d60
[ERROR] Invalid block size at position 0xea286ae0
Next known block: NULL at 0xea286af0+0x270 = 0xea286d60
[ERROR] Invalid block size at position 0xea7b0ae0
Next known block: NULL at 0xea7b0af0+0x270 = 0xea7b0d60
[ERROR] Invalid block size at position 0xeacdaae0
Next known block: NULL at 0xeacdaaf0+0x270 = 0xeacdad60
[ERROR] Invalid block size at position 0xeb204ae0
Next known block: NULL at 0xeb204af0+0x270 = 0xeb204d60
[ERROR] Invalid block size at position 0xeb9c3ae0
Next known block: NULL at 0xeb9c3af0+0x270 = 0xeb9c3d60
[ERROR] Invalid block size at position 0xec182ae0
Next known block: NULL at 0xec182af0+0x270 = 0xec182d60
[ERROR] Invalid block size at position 0xec941ae0
Next known block: NULL at 0xec941af0+0x270 = 0xec941d60
[ERROR] Invalid block size at position 0xed100ae0
Next known block: NULL at 0xed100af0+0x270 = 0xed100d60
[ERROR] Invalid block size at position 0xed8bfae0
Next known block: NULL at 0xed8bfaf0+0x270 = 0xed8bfd60
[ERROR] Invalid block size at position 0xee07eae0
Next known block: NULL at 0xee07eaf0+0x270 = 0xee07ed60
[ERROR] Invalid block size at position 0xee5a8ae0
Next known block: NULL at 0xee5a8af0+0x270 = 0xee5a8d60
[ERROR] Invalid block size at position 0xeead2ae0
Next known block: NULL at 0xeead2af0+0x270 = 0xeead2d60
[ERROR] Invalid block size at position 0xeeffcae0
Next known block: NULL at 0xeeffcaf0+0x270 = 0xeeffcd60
[ERROR] Invalid block size at position 0xef7bbae0
Next known block: NULL at 0xef7bbaf0+0x270 = 0xef7bbd60
[ERROR] Invalid block size at position 0xeff7aae0
Next known block: NULL at 0xeff7aaf0+0x270 = 0xeff7ad60
[ERROR] Invalid block size at position 0xf0739ae0
Next known block: NULL at 0xf0739af0+0x270 = 0xf0739d60
[ERROR] Invalid block size at position 0xf0ef8ae0
Next known block: NULL at 0xf0ef8af0+0x270 = 0xf0ef8d60
[ERROR] Invalid block size at position 0xf16b7ae0
Next known block: NULL at 0xf16b7af0+0x270 = 0xf16b7d60
[ERROR] Invalid block size at position 0xf1e76ae0
Next known block: NULL at 0xf1e76af0+0x270 = 0xf1e76d60
[ERROR] Invalid block size at position 0xf23a0ae0
Next known block: NULL at 0xf23a0af0+0x270 = 0xf23a0d60
[ERROR] Invalid block size at position 0xf28caae0
Next known block: NULL at 0xf28caaf0+0x270 = 0xf28cad60
[ERROR] Invalid block size at position 0xf2df4ae0
Next known block: NULL at 0xf2df4af0+0x270 = 0xf2df4d60
[ERROR] Invalid block size at position 0xf331eae0
Next known block: NULL at 0xf331eaf0+0x270 = 0xf331ed60
[ERROR] Invalid block size at position 0xf3848ae0
Next known block: NULL at 0xf3848af0+0x270 = 0xf3848d60
[ERROR] Invalid block size at position 0xf3d72ae0
Next known block: NULL at 0xf3d72af0+0x270 = 0xf3d72d60
[ERROR] Invalid block size at position 0xf429cae0
Next known block: NULL at 0xf429caf0+0x270 = 0xf429cd60
[ERROR] Invalid block size at position 0xf47c6ae0
Next known block: NULL at 0xf47c6af0+0x270 = 0xf47c6d60
[ERROR] Invalid block size at position 0xf4cf0ae0
Next known block: NULL at 0xf4cf0af0+0x270 = 0xf4cf0d60
[ERROR] Invalid block size at position 0xf521aae0
Next known block: NULL at 0xf521aaf0+0x270 = 0xf521ad60
[ERROR] Invalid block size at position 0xf5744ae0
Next known block: NULL at 0xf5744af0+0x270 = 0xf5744d60
[ERROR] Invalid block size at position 0xf5c6eae0
Next known block: NULL at 0xf5c6eaf0+0x270 = 0xf5c6ed60
[ERROR] Invalid block size at position 0xf6198ae0
Next known block: NULL at 0xf6198af0+0x270 = 0xf6198d60
[ERROR] Invalid block size at position 0xf66c2ae0
Next known block: NULL at 0xf66c2af0+0x270 = 0xf66c2d60
[ERROR] Invalid block size at position 0xf6becae0
Next known block: NULL at 0xf6becaf0+0x270 = 0xf6becd60
[ERROR] Invalid block size at position 0xf7116ae0
Next known block: NULL at 0xf7116af0+0x270 = 0xf7116d60
[ERROR] Invalid block size at position 0xf7640ae0
Next known block: NULL at 0xf7640af0+0x270 = 0xf7640d60
[ERROR] Invalid block size at position 0xf7b6aae0
Next known block: NULL at 0xf7b6aaf0+0x270 = 0xf7b6ad60
[ERROR] Invalid block size at position 0xf8094ae0
Next known block: NULL at 0xf8094af0+0x270 = 0xf8094d60
[ERROR] Invalid block size at position 0xf85beae0
Next known block: NULL at 0xf85beaf0+0x270 = 0xf85bed60
[ERROR] Invalid block size at position 0xf8ae8ae0
Next known block: NULL at 0xf8ae8af0+0x270 = 0xf8ae8d60
[ERROR] Invalid block size at position 0xf92a7ae0
Next known block: NULL at 0xf92a7af0+0x270 = 0xf92a7d60
[ERROR] Invalid block size at position 0xf9a66ae0
Next known block: NULL at 0xf9a66af0+0x270 = 0xf9a66d60
[ERROR] Invalid block size at position 0xfa225ae0
Next known block: NULL at 0xfa225af0+0x270 = 0xfa225d60
[ERROR] Invalid block size at position 0xfa9e4ae0
Next known block: NULL at 0xfa9e4af0+0x270 = 0xfa9e4d60
[ERROR] Invalid block size at position 0xfaf0eae0
Next known block: NULL at 0xfaf0eaf0+0x270 = 0xfaf0ed60
[ERROR] Invalid block size at position 0xfb438ae0
Next known block: NULL at 0xfb438af0+0x270 = 0xfb438d60
[ERROR] Invalid block size at position 0xfb962ae0
Next known block: NULL at 0xfb962af0+0x270 = 0xfb962d60
[ERROR] Invalid block size at position 0xfbe8cae0
Next known block: NULL at 0xfbe8caf0+0x270 = 0xfbe8cd60
[ERROR] Invalid block size at position 0xfc3b6ae0
Next known block: NULL at 0xfc3b6af0+0x270 = 0xfc3b6d60
[ERROR] Invalid block size at position 0xfc8e0ae0
Next known block: NULL at 0xfc8e0af0+0x270 = 0xfc8e0d60
[ERROR] Invalid block size at position 0xfce0aae0
Next known block: NULL at 0xfce0aaf0+0x270 = 0xfce0ad60
[ERROR] Invalid block size at position 0xfd334ae0
Next known block: NULL at 0xfd334af0+0x270 = 0xfd334d60
[ERROR] Invalid block size at position 0xfd85eae0
Next known block: NULL at 0xfd85eaf0+0x270 = 0xfd85ed60
[ERROR] Invalid block size at position 0xfdd88ae0
Next known block: NULL at 0xfdd88af0+0x270 = 0xfdd88d60
[ERROR] Invalid block size at position 0xfe2b2ae0
Next known block: NULL at 0xfe2b2af0+0x270 = 0xfe2b2d60
[ERROR] Invalid block size at position 0xfe7dcae0
Next known block: NULL at 0xfe7dcaf0+0x270 = 0xfe7dcd60
[ERROR] Invalid block size at position 0xfef9bae0
Next known block: NULL at 0xfef9baf0+0x270 = 0xfef9bd60
[ERROR] Invalid block size at position 0xff4c5ae0
Next known block: NULL at 0xff4c5af0+0x270 = 0xff4c5d60
[ERROR] Invalid block size at position 0xff9efae0
Next known block: NULL at 0xff9efaf0+0x270 = 0xff9efd60
[ERROR] Invalid block size at position 0xfff19ae0
Next known block: NULL at 0xfff19af0+0x270 = 0xfff19d60

Reached end of chunk 1/1 after 4169 blocks
Processed 1582 video frames at 24.00 FPS (65.92 s)
Done

Checking dng amount:

Code Select

dans-MacBook-Pro:corruption dan$ find . -name '*.dng' | wc -l
    1582

All ok.

sax100 · September 20, 2018, 11:10:38 PM

IT WORKS!!! Thank you so much @a1ex, @Danne and @dfort!!!

All frames are rendered, audio too, from beginning to end.

Is there a command to create a folder (M15-0904) to put all .dng from (M15-0904.MLV + M15-0904.M00) into it automatically and do the same for all other clips?

I've been using the script below, which made it easier to convert 20-30 files at once and keep them organized.

I tried it, but the conversion stops at frame 780.

Code Select

workingDir=`dirname "$0"`
cd "${workingDir}"

sudo mv ./mlv_dump.osx /usr/bin/mlv_dump
sudo chmod +x /usr/bin/mlv_dump

for FILE in `ls -A1 *.MLV *.mlv 2>/dev/null`; do
    BASE=`echo $FILE | cut -d "." -f1`;
    mkdir $BASE;
    mv ./"$BASE".M* ./$BASE
    cd ./$BASE
    /usr/bin/mlv_dump -x $FILE
    /usr/bin/mlv_dump --dng -o ${BASE}_frame_ $FILE
    /usr/bin/mlv_dump -m -o meta.mlv $FILE
    /usr/bin/mlv_dump -v meta.mlv >$FILE.txt
    rm ./meta.mlv
    rm ./meta.mlv.wav
    cd ..
done

sax100 · September 20, 2018, 11:15:25 PM

@dfort
Thank you very much for compiling the new and working mlv_dump! I hope to increase my IQ to be able to do that.

dfort · September 21, 2018, 12:58:23 AM

For that script to work you need to change this line:

Code Select

sudo mv ./mlv_dump.osx /usr/bin/mlv_dump

To this:

Code Select

sudo mv ./mlv_dump /usr/bin/mlv_dump

Note that those sudo commands are only needed once. Once mlv_dump is living in /usr/bin if you remove those two lines from the script you won't need to enter your password to run the script.

Another option is to use Danne's switch app. Ask him to prepare a build for you with the version 2 patched mlv_dump.

[EDIT] What's the purpose of these lines? Make a log file?

Code Select

    /usr/bin/mlv_dump -m -o meta.mlv $FILE
    /usr/bin/mlv_dump -v meta.mlv >$FILE.txt
    rm ./meta.mlv
    rm ./meta.mlv.wav

I believe that the -m option is no longer valid.

sax100 · September 21, 2018, 02:43:34 AM

Thanks again to EVERYONE for all your great help!

I took out the .osx as you instructed and it worked! Thank you @dfort!
Also took out the -m option as well.

How can I make this script recursive?

Code Select

workingDir=`dirname "$0"`
cd "${workingDir}"

sudo mv ./mlv_dump /usr/bin/mlv_dump
sudo chmod +x /usr/bin/mlv_dump

for FILE in `ls -A1 *.MLV *.mlv 2>/dev/null`; do
    BASE=`echo $FILE | cut -d "." -f1`;
    mkdir $BASE;
    mv ./"$BASE".M* ./$BASE
    cd ./$BASE
    /usr/bin/mlv_dump -x $FILE
    /usr/bin/mlv_dump --dng -o ${BASE}_frame_ $FILE
    /usr/bin/mlv_dump -o meta.mlv $FILE
    /usr/bin/mlv_dump -v meta.mlv >$FILE.txt
    rm ./meta.mlv
    rm ./meta.mlv.wav
    cd ..
done

sax100 · September 21, 2018, 02:45:12 AM

@Danne
Thank you for your mlv_dump also - I downloaded and tried yours as well and it works great!

Danne · September 21, 2018, 08:48:33 AM

That script is problematic for several reasons as dfort points out. I will put this into Switch when when I can soon.

Danne · September 21, 2018, 10:27:06 AM

Updated Switch:
https://www.magiclantern.fm/forum/index.php?topic=15108.0

commit:
https://bitbucket.org/Dannephoto/switch/commits/d1f3b9cf31616376c10fcd69cb0ef035804fdce6

If you still want to use your script you could simply do this with the new version of mlv_dump. Scriptwise there´s a lot more to wish for but hey, whatever works works:

Code Select

workingDir=`dirname "$0"`
cd "${workingDir}"

sudo mv ./mlv_dump /usr/bin/mlv_dump
sudo chmod +x /usr/bin/mlv_dump

for FILE in `ls -A1 *.MLV *.mlv 2>/dev/null`; do
    BASE=`echo $FILE | cut -d "." -f1`;
    mkdir $BASE;
    mv ./"$BASE".M* ./$BASE
    cd ./$BASE
    /usr/bin/mlv_dump --dng -o ${BASE}_frame_ $FILE
    cd ..
done

I assume you are using an older OS version since /usr/bin is a no go zone in Sierra.
Have you checked Mlv App by the way? Still needs the patch but I am sure Bouncyball will fix it.

bouncyball · September 21, 2018, 11:08:06 AM

@Danne:
BTW when I posted recovered 1st MLV part above I did not see a1ex's patch yet, so it was done manually with hexedit, I wish I had refreshed the last page of this thread

@a1ex:
Thank you for simple and 100% working code, I wanted to implement brute force method into mlv app long time ago but been lazy and slow (as usual) lately

.

regards
bb

Edit: From the beginning it was obvious that recording of that clip's been done with some old build of mlv_rec, but I could not imagine it was that old

, so issues like this are not expected in the nearest future, but anyway would be nice to update mlv app's loading section.

g3gg0 · September 21, 2018, 12:16:33 PM

https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/36/

does this version work?

from: https://www.magiclantern.fm/forum/index.php?topic=19761.msg186010#msg186010
"you can specify a bugfix option "-F 3" to fix that issue"

dfort · September 21, 2018, 12:28:17 PM

@g3gg0 - No, that build doesn't work with this file:

Code Select

./mlv_dump.osx -F 3 --dng M15-0904.MLV 
FIX BUG #3 [active]

 MLV Dumper
-----------------

Mode of operation:
   - Input MLV file: 'M15-0904.MLV'
   - Decompressing before writing DNG
   - Enforcing 14bpp for DNG output
   - Using output path 'M15-0904_' for DNGs
   - Convert to DNG frames
   - Output into 'M15-0904_'
File M15-0904.MLV opened
File M15-0904.M00 not existing.
Processing...


Vertical stripes correction:
  1.00000  1.00290  0.99698  1.00259  0.99626  1.00102  0.99782  0.99646
Cold pixels : 0                             
Unknown Block: %%, skipping
[ERROR] Invalid block size at position 0x7dd46afe
Processed 777 video frames at 24.00 FPS (32.38 s)
Done

bouncyball · September 21, 2018, 02:52:21 PM

@Danne

Quote from: Danne on September 21, 2018, 10:27:06 AM
Have you checked Mlv App by the way? Still needs the patch but I am sure Bouncyball will fix it.

Done

News:

Unknown Block errors PLEASE HELP

dfort

dfort

dfort

dfort

dfort

dfort

dfort