I used 3x different KomputerBay CF cards 128GB 1000x cards and also tested problem with 3x different 64GB 1000x cards. Nearly every clip would only convert up to around 45-55 seconds and spit out 1-4 frames at the end of clip.
Is there something wrong with my CF cards?
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: 'M15-0904.MLV'
- Verify file structure
- Output .idx file for faster processing
File M15-0904.MLV opened
File M15-0904.M00 opened
Processing...
Unknown Block: %%, skipping
Unknown Block: ;?H, skipping
Reached end of chunk 1/2 after 2059 blocks
Unknown Block: ????, skipping
Reached end of chunk 2/2 after 10 blocks
Processed 781 video frames
XREF table contains 1286 entries
File M15-0904.IDX opened for writing
Done
Please upload a file that won't convert.
Uploading 4GB affected file to Google Drive - another 1.5 hrs left. I'll provide link shortly.
Here it is...
M15-0904.MLV (https://drive.google.com/file/d/1ZsuLpIpQe6VioRssQQvAFqmvwE65gV_F/view?usp=sharing)
Seeing the same issue with your file:
mlv_dump --dng --relaxed M15-0904.MLV
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: 'M15-0904.MLV'
- Using output path 'M15-0904_' for DNGs
- Convert to DNG frames
- Output into 'M15-0904_'
File M15-0904.MLV opened
File M15-0904.M00 not existing.
Processing...
Unknown Block: %%, skipping
[ERROR] Invalid block size at position 0x7dd46afe
Processed 777 video frames at 24.00 FPS (32.38 s)
Done
Let's keep the help discussion going here and leave the intro topic for--well, introductions.
Taking a closer look:
Block: VIDF
Offset: 0x7d8d2290
Number: 2056
Size: 2705488
Time: 32341.457000 ms
Frame: #0764
Crop: 152x272
Pan: 146x271
Space: 4016
Block: %%
Offset: 0x7db66ae0
Number: 2057
Size: 1966110
Time: 6192543978356.757812 ms
Unknown Block: %%, skipping
[ERROR] Invalid block size at position 0x7dd46afe
Processed 777 video frames at 24.00 FPS (32.38 s)
Done
0x7dd46afe (hex) is 2111073022 (dec) so does that mean that the problems start here?
(https://farm2.staticflickr.com/1889/42982307570_b791cabc62.jpg) (https://flic.kr/p/28ucyys)
I'm not very knowledgable on saving corrupted MLV files so maybe someone else could give it a shot?
BTW--the video and audio frames don't match, that might be a problem.
File Header (MLVI)
...
Frames Video: 1586
Frames Audio: 330
Can´t download the file over here, too big but what happen if ou select no audio and relaxed setting in mlv_dump?
You can probably test these settings in Switch(mac), batch_mlv(windows) and/or Mlv App.
Same issue:
mlv_dump --dng --relaxed --no-audio M15-0904.MLV
...
Skipping 'AUDF' block
Skipping 'AUDF' block
Skipping 'AUDF' block
Skipping '%%' block
[ERROR] Invalid block size at position 0x7dd46afe
Processed 777 video frames at 24.00 FPS (32.38 s)
Done
I'm downloading now , I'll have a look -- I've repaired a few file before with HxD hex editor & mlvdiag.1.0.1 (creator of Raw2Cdng)
Ok , so what's the problem with the file ?
I see no problem with the mlv I downloaded , Audio & Cdng export fine here with MLVProducer .
1920x804 , 777 Cdng frames & 32 second of audio
(https://image.ibb.co/mXeXZz/MLVProducer_info_tab.png) (https://imgbb.com/)
I quickly ran the file thought MLVProducer with a lite grad in sRGB color space and saved a ffmpeg prores 4444 (https://drive.google.com/file/d/1LaSB_TQAl646LJsmtxqnCVe82sgxk403/view?usp=sharing)
No problem extracting audio also , same length as video file .
Just tested MLVFS on quick mount in wins 7 all fine here , every thing exports as it should .
Edit: Tried the file with mlv_dump , did get a error message
I:\2018-9-19_problem_mlv_5d3\cdngs>mlv_dump --dng M15-0904.mlv
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: 'M15-0904.mlv'
- Using output path 'M15-0904_' for DNGs
- Convert to DNG frames
- Output into 'M15-0904_'
File M15-0904.mlv opened
File M15-0904.m00 not existing.
Processing...
Unknown Block: % % , skipping
[ERROR] Invalid block size at position 0x7dd46afe
Processed 777 video frames at 24.00 FPS (32.38 s)
Done
cdng's extracted normally with audio , thou it was 4 second shorter then mlvfs & mlvproducer those where 32 second .
Can't see any other problems , maybe a bad block on card or the trim (https://en.wikipedia.org/wiki/Trim_(computing)) function on the CF card didn't erase the block fully .
https://en.wikipedia.org/wiki/CompactFlash
QuoteCompactFlash Revision 6.0 (November 2010) added UltraDMA Mode 7 (167 MByte/s), ATA-8/ACS-2 sanitize command, TRIM and an optional card capability to report the operating temperature range of the card
QuoteThe TRIM command enables an operating system to notify the SSD of pages which no longer contain valid data. For a file deletion operation, the operating system will mark the file's sectors as free for new data ....
The problem is that there are 1586 frames in that file.
ok I see it now ,
Edit: in MLV autopsy when I run this
mlv_dump --dng --skip-block 0x7dd46afe M15-0904.mlv
I get this
Processing...
[ERROR] Missing file header
Processed 0 video frames at -1.#J FPS (-1.#J s)
Done
with out skip block it converts the 777 frames , strange
Damage file header ?
I don't have a 5d3 to make mlv file for the header & replace it with a different header .
Here are two versions of mlv_dump which I removed the "goto abort" code. Could you test and process the file with one of these?
Mac:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump
Windows:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump.exe
No , tried windows version still same issue -- There one thing that puzzle me there a Alignment fill after every frame block .
(https://image.ibb.co/iJ2d7K/MLVdiag_101_small.png) (https://imgbb.com/)
(https://image.ibb.co/mERVnK/MLVdiag_101_med.png) (https://imgbb.com/)
Is this one of those null bugs I remember hearing about about half year ago ?
I don't see this on 5d2
Sorry, missed one code part. Please try once more, should work now:
Mac:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump
Windows:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump.exe
Tried some more, downloaded the file but seems corrupted. Checking output in MlRawViewer shows black image after dng 777:
(https://i.postimg.cc/9F6Gq7tH/Screen_Shot_2018-09-20_at_10.20.28.png_scaled.jpg)
When all else fails, brute force prevails. Put this at the end of this script (https://www.magiclantern.fm/forum/index.php?topic=21694.msg197960#msg197960):
def read_block_bf(f):
data = ""
count = 0
while True:
chunk = f.read(1024 * 1024);
if len(chunk) == 0: return
data += chunk
next_vidf = data.find("VIDF")
if next_vidf >= 0:
f.seek(next_vidf - len(data), 1)
data = ""
block_header = f.read(mlv_block_hdr.size)
blocktype, blocksize, timestamp = mlv_block_hdr.unpack_from(block_header)
f.seek(-mlv_block_hdr.size, 1);
count += 1
print hex(f.tell()) + (" [%d]:" % count),
block = f.read(blocksize)
yield block
with open(sys.argv[1], 'rb') as f:
[parse_block(block) for block in read_block_bf(f)]
0x7c6bf260 [770]: Block VIDF (2705536 bytes)
0x7c954260 [771]: Block VIDF (2705536 bytes)
0x7cbe9280 [772]: Block VIDF (2705504 bytes)
0x7ce7e260 [773]: Block VIDF (2705536 bytes)
0x7d113280 [774]: Block VIDF (2705504 bytes)
0x7d3a8260 [775]: Block VIDF (2705536 bytes)
0x7d63d260 [776]: Block VIDF (2705536 bytes)
0x7d8d2290 [777]: Block VIDF (2705488 bytes)
0x7db67280 [778]: Block VIDF (2705504 bytes)
0x7ddfc260 [779]: Block VIDF (2705536 bytes)
0x7e091260 [780]: Block VIDF (2705536 bytes)
0x7e326280 [781]: Block VIDF (2705504 bytes)
0x7e5bb260 [782]: Block VIDF (2705536 bytes)
0x7e850280 [783]: Block VIDF (2705504 bytes)
0x7eae5260 [784]: Block VIDF (2705536 bytes)
0x7ed7a280 [785]: Block VIDF (2705504 bytes)
...
0xffc85280 [1582]: Block VIDF (2705504 bytes)
So, it looks like the video frames are there. Adding this into mlv_dump (diff from crop_rec_4k branch):
diff -r 0d39d995cb65 modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c Tue Sep 18 07:54:06 2018 +0200
+++ b/modules/mlv_rec/mlv_dump.c Thu Sep 20 10:41:34 2018 +0200
@@ -1239,6 +1239,33 @@
}
}
+static int seek_to_next_known_block(FILE * in_file)
+{
+ int read_ahead_size = 128 * 1024 * 1024;
+ uint8_t * ahead = malloc(read_ahead_size);
+ assert(ahead);
+
+ int read = fread(ahead, 1, read_ahead_size, in_file);
+ /* fixme: file_set_pos doesn't work */
+ fseek(in_file, -read, SEEK_CUR);
+ for (int i = 0; i < read; i++)
+ {
+ if (memcmp(ahead + i, "VIDF", 4) == 0 ||
+ memcmp(ahead + i, "AUDF", 4) == 0 ||
+ memcmp(ahead + i, "NULL", 4) == 0)
+ {
+ print_msg(MSG_INFO, "Next known block: %c%c%c%c at 0x%"PRIx64"+0x%x = ", ahead[i], ahead[i+1], ahead[i+2], ahead[i+3], file_get_pos(in_file), i);
+ fseek(in_file, i, SEEK_CUR);
+ print_msg(MSG_INFO, "0x%"PRIx64"\n", file_get_pos(in_file));
+ free(ahead);
+ return 1;
+ }
+ }
+
+ print_msg(MSG_ERROR, "Could not find any known block from 0x"PRIx64".\n", file_get_pos(in_file));
+ free(ahead);
+ return 0;
+}
int main (int argc, char *argv[])
{
@@ -2129,7 +2156,13 @@
else
{
print_msg(MSG_ERROR, "Invalid block size at position 0x%08" PRIx64 "\n", position);
- goto abort;
+
+ /* attempt to seek to next valid block */
+ if (seek_to_next_known_block(in_file)) {
+ goto read_headers;
+ } else {
+ goto abort;
+ }
}
}
@@ -4080,6 +4113,11 @@
else
{
print_msg(MSG_INFO, "Unknown Block: %c%c%c%c, skipping\n", mlv_block->blockType[0], mlv_block->blockType[1], mlv_block->blockType[2], mlv_block->blockType[3]);
+
+ /* this block might be bad; seek to next valid one */
+ fseek(in_file, -mlv_block->blockSize, SEEK_CUR);
+ seek_to_next_known_block(in_file);
+ goto read_headers;
}
lua_handle_hdr(lua_state, mlv_block->blockType, "", 0);
Appears to work, but have to go, didn't check the DNGs.
cc @g3gg0 for cleanup and commit, possibly as an option.
Cool. Tried hg import but fail. Need to go now but a1ex provided the needed fix so...
Also pm:d g3gg0.
hg import /Users/dan/Desktop/patch.diff
applying /Users/dan/Desktop/patch.diff
patching file modules/mlv_rec/mlv_dump.c
Hunk #1 FAILED at 1238
Hunk #2 FAILED at 2128
Hunk #3 FAILED at 4079
3 out of 3 hunks FAILED -- saving rejects to file modules/mlv_rec/mlv_dump.c.rej
abort: patch failed to apply
dans-MacBook-Pro:magic-lantern dan$
@Danne: "hg up crop_rec_4k -C" (or crop_rec_4k_mlv_snd) before "hg import". Or, for temporary patching:
patch -p1 < foo.patch
I've got your error message (with your line numbers minus 1) after trying to apply the patch twice. Guess: it either worked the first time, or you've got some incompatible changes in your repository.
Quote from: sax100 on September 17, 2018, 06:50:43 PM
ML for 5d3 2014.01.08 Build
Pretty sure it was this bug (https://www.magiclantern.fm/forum/index.php?topic=19761.msg186060#msg186060). You should have mentioned all the relevant info in this thread.
The DNGs near frame 777 look OK to me. The file seems to contain some more bad blocks, but this brute-forcing trick appears to jump over them.
Block: VIDF
Offset: 0x7d8d2290
Number: 2056
Size: 2705488
Time: 32341.457000 ms
Frame: #0764
Crop: 152x272
Pan: 146x271
Space: 4016
Block: %%
Offset: 0x7db66ae0
Number: 2057
Size: 1966110
Time: 6192543978356.757812 ms
Unknown Block: %%, skipping
Next known block: NULL at 0x7db66ae0+0x280 = 0x7db66d60
Block: NULL
Offset: 0x7db66d60
Number: 2057
Size: 1280
Time: 18446744073709552.000000 ms
Block: ELVL
Offset: 0x7db67260
Number: 2058
Size: 32
Time: 32884.321000 ms
Roll: 1.00
Pitch: 3.50
Block: VIDF
Offset: 0x7db67280
Number: 2059
Size: 2705504
Time: 32883.162000 ms
Frame: #0777
Crop: 152x272
Pan: 146x271
Space: 4032
...
[ERROR] Invalid block size at position 0xfff19ae0
Next known block: NULL at 0xfff19af0+0x270 = 0xfff19d60
Block: NULL
Offset: 0xfff19d60
Number: 4168
Size: 1280
Time: 18446744073709552.000000 ms
Reached end of chunk 1/1 after 4169 blocks
Processed 1582 video frames at 24.00 FPS (65.92 s)
Done
Here (https://drive.google.com/open?id=1o7XlKnhqeCQDm6r7LhLTNnmlaWL4qDi9) is repaired MLV. More than 100 errors showed up. Mostly VIDF block size mismatches (VIDF blocks had ~600bytes more at the end). One NULL block size mismatch.
Plays fine in MLV App. Some async/glitches with audio at the end of the clip (BTW audio level is very low).
Unfortunately MLV has no VERS info block. Hence can not identify which ML build and versions of modules were used.
ELVL blocks are all over there (after each VIDF/NULL pair).
Cool to see solutions.
@a1ex
No idea what´s up with hg here. Redownloaded magic-lantern repo and got into the mentioned branch:
dans-MacBook-Pro:magic-lantern dan$ hg branch
crop_rec_4k
First try:
patch -p1 < /Users/dan/Desktop/patch.diff
patching file modules/mlv_rec/mlv_dump.c
Hunk #1 FAILED at 1239.
Hunk #2 FAILED at 2156.
patch unexpectedly ends in middle of line
Hunk #3 FAILED at 4113.
3 out of 3 hunks FAILED -- saving rejects to file modules/mlv_rec/mlv_dump.c.rej
Erasing, importing vanilla magic-lantern repo, entering crop_rec_4k branch:
Second try:
dans-MacBook-Pro:magic-lantern dan$ hg import /Users/dan/Desktop/patch.diff
applying /Users/dan/Desktop/patch.diff
patching file modules/mlv_rec/mlv_dump.c
Hunk #1 FAILED at 1238
Hunk #2 FAILED at 2128
Hunk #3 FAILED at 4079
3 out of 3 hunks FAILED -- saving rejects to file modules/mlv_rec/mlv_dump.c.rej
abort: patch failed to apply
dans-MacBook-Pro:magic-lantern dan$
Will check and see if I can find an answer to this...
Just to exclude copy/paste errors:
mlv_dump_bruteforce.patch (https://a1ex.magiclantern.fm/bleeding-edge/mlv-recover/mlv_dump_bruteforce.patch) (buggy, updated below)
Your patch file works! Successfully patched and processed the file.
Tried copy pasting from the forum post but gives errors:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/patch.diff
Copy pasting your content into the above file works. Some formatting issue? No clean answers checking mercurial forums.
Ok, here we go. Formatting issue. Run this in terminal:
unix2dos /Users/dan/Desktop/Timer_A/Mercurial/patch.diff
unix2dos: converting file /Users/dan/Desktop/Timer_A/Mercurial/patch.diff to DOS format...
Now it´s possible to add the patch on mac...sigh
hg import /Users/dan/Desktop/Timer_A/Mercurial/patch.diff --edit danne
At least kind of worked. I probably missed a line at the bottom:
dans-MacBook-Pro:magic-lantern dan$ hg import /Users/dan/Desktop/Timer_A/Mercurial/patch.diff --edit danne
applying /Users/dan/Desktop/Timer_A/Mercurial/patch.diff
patching file modules/mlv_rec/mlv_dump.c
Hunk #3 succeeded at 4114 with fuzz 1 (offset 0 lines).
abort: no username supplied
(use 'hg config --edit' to set your username)
dans-MacBook-Pro:magic-lantern dan$
Thanks for all of your diligent investigating of this problem!
Bare with me, I'm still learning my ABC's...
I tried to run the patches @a1ex and @Danne provided. I'm unsure if this the correct command?
mlv_dump patch < patch.patch M15-0904.MLV
Cause I got this error...
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: 'patch'
- Verify file structure
[ERROR] Failed to open file 'patch'
Then got message from @Danne and since I'm on a MAC, I ran unix2dos command, but...
My mac (OSX 10.85) doesn't recognize unix2dos. Researched online and found [url https://brew.sh/]Homebrew[/url] to allow me to use unix2dos, but wouldn't install. Does anyone know why? Or is there another way of doing this?
==> Downloading and installing Homebrew...
curl: (35) error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
Failed during: /bin/bash -o pipefail -c '/usr/bin/curl -fsSL https://github.com/Homebrew/brew/tarball/master | /usr/bin/tar xz -m --strip 1'
Long story short. Use this version of mlv_dump:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump
Ok, so I tried @Danne's mlv_dump and it's an improvement!
I don't know how to run the patch, which I probably need.
Rendered frames 1-1343 and 1582-2090 (frame 2090 is the end, so missing 238 continuous frames)
These are the errors I got:
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: 'M15-0904.MLV'
- Using output path 'M15-0904_' for DNGs
- Convert to DNG frames
- Output into 'M15-0904_'
File M15-0904.MLV opened
File M15-0904.M00 opened
File M15-0904.M01 not existing.
Processing...
Unknown Block: %%, skipping
Next known block: NULL at 0x7db66ae0+0x280 = 0x7db66d60
[ERROR] Invalid block size at position 0xd95feae0
Next known block: NULL at 0xe15feaf0+0x270 = 0xe15feaf0
[ERROR] Invalid block size at position 0xe15feaf0
Next known block: NULL at 0xe95feb00+0x11a260 = 0xe95feb00
[ERROR] Invalid block size at position 0xe95feb00
Next known block: NULL at 0xf15feb10+0x233fd0 = 0xf15feb10
[ERROR] Invalid block size at position 0xf15feb10
Next known block: NULL at 0xf95feb20+0xb9240 = 0xf95feb20
[ERROR] Invalid block size at position 0xf95feb20
Next known block: NULL at 0xfff1a260+0x1d2fb0 = 0xfff1a260
Reached end of chunk 1/2 after 3512 blocks
[ERROR] Invalid block size at position 0x00a538b4
Next known block: NULL at 0xa538c4+0x270 = 0xa53b34
[ERROR] Invalid block size at position 0x012128b4
Next known block: NULL at 0x12128c4+0x270 = 0x1212b34
[ERROR] Invalid block size at position 0x0173c8b4
Next known block: NULL at 0x512c28c4+0x270 = 0x512c2b34
[ERROR] Invalid block size at position 0x517ec8b4
Next known block: NULL at 0x517ec8c4+0x270 = 0x517ecb34
[ERROR] Invalid block size at position 0x51d168b4
Next known block: NULL at 0x51d168c4+0x270 = 0x51d16b34
[ERROR] Invalid block size at position 0x522408b4
Next known block: NULL at 0x522408c4+0x270 = 0x52240b34
Reached end of chunk 2/2 after 1532 blocks
Processed 1853 video frames at 24.00 FPS (77.21 s)
Done
What you got already is the patch in mlv_dump. You don't patch the mlv.
If interested in digging into code and applying patches download the source and dependencies. Search 'compiling macintosh' to get into some useful guidance.
@sax100
Long story longer -- Like Danne said, the source code needs to be patched then compiled. You can't just run the patch on the binary or the file you're trying to process.
I ran Danne's build on the problem MLV and it extracted what seems to be all the frames from that file. The audio cuts out early. Note that this is only the first 4GB chunk of that shot (1344 frames), there is also a second file named M15-0904.M00 that needs to be tacked onto the end of this.
[EDIT] Saw you ran it on both M15-0904.MLV and M15-0904.M00. Curious if the audio runs the full length of the clip.
Quote from: a1ex on September 20, 2018, 12:09:10 PM
Pretty sure it was this bug (https://www.magiclantern.fm/forum/index.php?topic=19761.msg186060#msg186060). You should have mentioned all the relevant info in this thread.
That was a double posting issue. The relevant info was in the other topic:
Quote from: sax100 on September 17, 2018, 06:50:43 PM
Canon 5D Mark III / KomputerBay 1000x 128GB CF
ML for 5d3 2014.01.08 Build
Firmware Version 1.1.3-ml-v2.3
Though this is in your signature, not all forum readers see this.
ML is in a constant state of development and the bug that caused the problems you're experiencing has been fixed. That's why users are encouraged to use the latest builds.
Maybe some Mac compatibility issue, possibly at fseek? On Linux, with the same MLV file I get valid DNGs until frame 1581.
edit: updated the patch and confirmed it's working on the Mac VM:
mlv_dump_bruteforce-v2.patch (https://a1ex.magiclantern.fm/bleeding-edge/mlv-recover/mlv_dump_bruteforce-v2.patch)
mlv_dump M15-0904.MLV --dng
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: 'M15-0904.MLV'
- Using output path 'M15-0904_' for DNGs
- Convert to DNG frames
- Output into 'M15-0904_'
File M15-0904.MLV opened
File M15-0904.M00 not existing.
Processing...
Unknown Block: %%, skipping
Next known block: NULL at 0x7db66ae0+0x280 = 0x7db66d60
[ERROR] Invalid block size at position 0xd95feae0
Next known block: NULL at 0xd95feaf0+0x270 = 0xd95fed60
[ERROR] Invalid block size at position 0xda052ae0
Next known block: NULL at 0xda052af0+0x270 = 0xda052d60
[ERROR] Invalid block size at position 0xda57cae0
Next known block: NULL at 0xda57caf0+0x270 = 0xda57cd60
[ERROR] Invalid block size at position 0xdaaa6ae0
Next known block: NULL at 0xdaaa6af0+0x270 = 0xdaaa6d60
[ERROR] Invalid block size at position 0xdafd0ae0
Next known block: NULL at 0xdafd0af0+0x270 = 0xdafd0d60
[ERROR] Invalid block size at position 0xdb4faae0
Next known block: NULL at 0xdb4faaf0+0x270 = 0xdb4fad60
[ERROR] Invalid block size at position 0xdba24ae0
Next known block: NULL at 0xdba24af0+0x270 = 0xdba24d60
[ERROR] Invalid block size at position 0xdbf4eae0
Next known block: NULL at 0xdbf4eaf0+0x270 = 0xdbf4ed60
[ERROR] Invalid block size at position 0xdc478ae0
Next known block: NULL at 0xdc478af0+0x270 = 0xdc478d60
[ERROR] Invalid block size at position 0xdc9a2ae0
Next known block: NULL at 0xdc9a2af0+0x270 = 0xdc9a2d60
[ERROR] Invalid block size at position 0xdd161ae0
Next known block: NULL at 0xdd161af0+0x270 = 0xdd161d60
[ERROR] Invalid block size at position 0xdd68bae0
Next known block: NULL at 0xdd68baf0+0x270 = 0xdd68bd60
[ERROR] Invalid block size at position 0xddbb5ae0
Next known block: NULL at 0xddbb5af0+0x270 = 0xddbb5d60
[ERROR] Invalid block size at position 0xde0dfae0
Next known block: NULL at 0xde0dfaf0+0x270 = 0xde0dfd60
[ERROR] Invalid block size at position 0xde89eae0
Next known block: NULL at 0xde89eaf0+0x270 = 0xde89ed60
[ERROR] Invalid block size at position 0xdedc8ae0
Next known block: NULL at 0xdedc8af0+0x270 = 0xdedc8d60
[ERROR] Invalid block size at position 0xdf2f2ae0
Next known block: NULL at 0xdf2f2af0+0x270 = 0xdf2f2d60
[ERROR] Invalid block size at position 0xdf81cae0
Next known block: NULL at 0xdf81caf0+0x270 = 0xdf81cd60
[ERROR] Invalid block size at position 0xdfd46ae0
Next known block: NULL at 0xdfd46af0+0x270 = 0xdfd46d60
[ERROR] Invalid block size at position 0xe0270ae0
Next known block: NULL at 0xe0270af0+0x270 = 0xe0270d60
[ERROR] Invalid block size at position 0xe079aae0
Next known block: NULL at 0xe079aaf0+0x270 = 0xe079ad60
[ERROR] Invalid block size at position 0xe0f59ae0
Next known block: NULL at 0xe0f59af0+0x270 = 0xe0f59d60
[ERROR] Invalid block size at position 0xe1718ae0
Next known block: NULL at 0xe1718af0+0x270 = 0xe1718d60
[ERROR] Invalid block size at position 0xe1ed7ae0
Next known block: NULL at 0xe1ed7af0+0x270 = 0xe1ed7d60
[ERROR] Invalid block size at position 0xe2696ae0
Next known block: NULL at 0xe2696af0+0x270 = 0xe2696d60
[ERROR] Invalid block size at position 0xe2e55ae0
Next known block: NULL at 0xe2e55af0+0x270 = 0xe2e55d60
[ERROR] Invalid block size at position 0xe337fae0
Next known block: NULL at 0xe337faf0+0x270 = 0xe337fd60
[ERROR] Invalid block size at position 0xe38a9ae0
Next known block: NULL at 0xe38a9af0+0x270 = 0xe38a9d60
[ERROR] Invalid block size at position 0xe3dd3ae0
Next known block: NULL at 0xe3dd3af0+0x270 = 0xe3dd3d60
[ERROR] Invalid block size at position 0xe4592ae0
Next known block: NULL at 0xe4592af0+0x270 = 0xe4592d60
[ERROR] Invalid block size at position 0xe4abcae0
Next known block: NULL at 0xe4abcaf0+0x270 = 0xe4abcd60
[ERROR] Invalid block size at position 0xe4fe6ae0
Next known block: NULL at 0xe4fe6af0+0x270 = 0xe4fe6d60
[ERROR] Invalid block size at position 0xe5510ae0
Next known block: NULL at 0xe5510af0+0x270 = 0xe5510d60
[ERROR] Invalid block size at position 0xe5ccfae0
Next known block: NULL at 0xe5ccfaf0+0x270 = 0xe5ccfd60
[ERROR] Invalid block size at position 0xe648eae0
Next known block: NULL at 0xe648eaf0+0x270 = 0xe648ed60
[ERROR] Invalid block size at position 0xe6c4dae0
Next known block: NULL at 0xe6c4daf0+0x270 = 0xe6c4dd60
[ERROR] Invalid block size at position 0xe7177ae0
Next known block: NULL at 0xe7177af0+0x270 = 0xe7177d60
[ERROR] Invalid block size at position 0xe76a1ae0
Next known block: NULL at 0xe76a1af0+0x270 = 0xe76a1d60
[ERROR] Invalid block size at position 0xe7bcbae0
Next known block: NULL at 0xe7bcbaf0+0x270 = 0xe7bcbd60
[ERROR] Invalid block size at position 0xe838aae0
Next known block: NULL at 0xe838aaf0+0x270 = 0xe838ad60
[ERROR] Invalid block size at position 0xe88b4ae0
Next known block: NULL at 0xe88b4af0+0x270 = 0xe88b4d60
[ERROR] Invalid block size at position 0xe8ddeae0
Next known block: NULL at 0xe8ddeaf0+0x270 = 0xe8dded60
[ERROR] Invalid block size at position 0xe9308ae0
Next known block: NULL at 0xe9308af0+0x270 = 0xe9308d60
[ERROR] Invalid block size at position 0xe9ac7ae0
Next known block: NULL at 0xe9ac7af0+0x270 = 0xe9ac7d60
[ERROR] Invalid block size at position 0xea286ae0
Next known block: NULL at 0xea286af0+0x270 = 0xea286d60
[ERROR] Invalid block size at position 0xea7b0ae0
Next known block: NULL at 0xea7b0af0+0x270 = 0xea7b0d60
[ERROR] Invalid block size at position 0xeacdaae0
Next known block: NULL at 0xeacdaaf0+0x270 = 0xeacdad60
[ERROR] Invalid block size at position 0xeb204ae0
Next known block: NULL at 0xeb204af0+0x270 = 0xeb204d60
[ERROR] Invalid block size at position 0xeb9c3ae0
Next known block: NULL at 0xeb9c3af0+0x270 = 0xeb9c3d60
[ERROR] Invalid block size at position 0xec182ae0
Next known block: NULL at 0xec182af0+0x270 = 0xec182d60
[ERROR] Invalid block size at position 0xec941ae0
Next known block: NULL at 0xec941af0+0x270 = 0xec941d60
[ERROR] Invalid block size at position 0xed100ae0
Next known block: NULL at 0xed100af0+0x270 = 0xed100d60
[ERROR] Invalid block size at position 0xed8bfae0
Next known block: NULL at 0xed8bfaf0+0x270 = 0xed8bfd60
[ERROR] Invalid block size at position 0xee07eae0
Next known block: NULL at 0xee07eaf0+0x270 = 0xee07ed60
[ERROR] Invalid block size at position 0xee5a8ae0
Next known block: NULL at 0xee5a8af0+0x270 = 0xee5a8d60
[ERROR] Invalid block size at position 0xeead2ae0
Next known block: NULL at 0xeead2af0+0x270 = 0xeead2d60
[ERROR] Invalid block size at position 0xeeffcae0
Next known block: NULL at 0xeeffcaf0+0x270 = 0xeeffcd60
[ERROR] Invalid block size at position 0xef7bbae0
Next known block: NULL at 0xef7bbaf0+0x270 = 0xef7bbd60
[ERROR] Invalid block size at position 0xeff7aae0
Next known block: NULL at 0xeff7aaf0+0x270 = 0xeff7ad60
[ERROR] Invalid block size at position 0xf0739ae0
Next known block: NULL at 0xf0739af0+0x270 = 0xf0739d60
[ERROR] Invalid block size at position 0xf0ef8ae0
Next known block: NULL at 0xf0ef8af0+0x270 = 0xf0ef8d60
[ERROR] Invalid block size at position 0xf16b7ae0
Next known block: NULL at 0xf16b7af0+0x270 = 0xf16b7d60
[ERROR] Invalid block size at position 0xf1e76ae0
Next known block: NULL at 0xf1e76af0+0x270 = 0xf1e76d60
[ERROR] Invalid block size at position 0xf23a0ae0
Next known block: NULL at 0xf23a0af0+0x270 = 0xf23a0d60
[ERROR] Invalid block size at position 0xf28caae0
Next known block: NULL at 0xf28caaf0+0x270 = 0xf28cad60
[ERROR] Invalid block size at position 0xf2df4ae0
Next known block: NULL at 0xf2df4af0+0x270 = 0xf2df4d60
[ERROR] Invalid block size at position 0xf331eae0
Next known block: NULL at 0xf331eaf0+0x270 = 0xf331ed60
[ERROR] Invalid block size at position 0xf3848ae0
Next known block: NULL at 0xf3848af0+0x270 = 0xf3848d60
[ERROR] Invalid block size at position 0xf3d72ae0
Next known block: NULL at 0xf3d72af0+0x270 = 0xf3d72d60
[ERROR] Invalid block size at position 0xf429cae0
Next known block: NULL at 0xf429caf0+0x270 = 0xf429cd60
[ERROR] Invalid block size at position 0xf47c6ae0
Next known block: NULL at 0xf47c6af0+0x270 = 0xf47c6d60
[ERROR] Invalid block size at position 0xf4cf0ae0
Next known block: NULL at 0xf4cf0af0+0x270 = 0xf4cf0d60
[ERROR] Invalid block size at position 0xf521aae0
Next known block: NULL at 0xf521aaf0+0x270 = 0xf521ad60
[ERROR] Invalid block size at position 0xf5744ae0
Next known block: NULL at 0xf5744af0+0x270 = 0xf5744d60
[ERROR] Invalid block size at position 0xf5c6eae0
Next known block: NULL at 0xf5c6eaf0+0x270 = 0xf5c6ed60
[ERROR] Invalid block size at position 0xf6198ae0
Next known block: NULL at 0xf6198af0+0x270 = 0xf6198d60
[ERROR] Invalid block size at position 0xf66c2ae0
Next known block: NULL at 0xf66c2af0+0x270 = 0xf66c2d60
[ERROR] Invalid block size at position 0xf6becae0
Next known block: NULL at 0xf6becaf0+0x270 = 0xf6becd60
[ERROR] Invalid block size at position 0xf7116ae0
Next known block: NULL at 0xf7116af0+0x270 = 0xf7116d60
[ERROR] Invalid block size at position 0xf7640ae0
Next known block: NULL at 0xf7640af0+0x270 = 0xf7640d60
[ERROR] Invalid block size at position 0xf7b6aae0
Next known block: NULL at 0xf7b6aaf0+0x270 = 0xf7b6ad60
[ERROR] Invalid block size at position 0xf8094ae0
Next known block: NULL at 0xf8094af0+0x270 = 0xf8094d60
[ERROR] Invalid block size at position 0xf85beae0
Next known block: NULL at 0xf85beaf0+0x270 = 0xf85bed60
[ERROR] Invalid block size at position 0xf8ae8ae0
Next known block: NULL at 0xf8ae8af0+0x270 = 0xf8ae8d60
[ERROR] Invalid block size at position 0xf92a7ae0
Next known block: NULL at 0xf92a7af0+0x270 = 0xf92a7d60
[ERROR] Invalid block size at position 0xf9a66ae0
Next known block: NULL at 0xf9a66af0+0x270 = 0xf9a66d60
[ERROR] Invalid block size at position 0xfa225ae0
Next known block: NULL at 0xfa225af0+0x270 = 0xfa225d60
[ERROR] Invalid block size at position 0xfa9e4ae0
Next known block: NULL at 0xfa9e4af0+0x270 = 0xfa9e4d60
[ERROR] Invalid block size at position 0xfaf0eae0
Next known block: NULL at 0xfaf0eaf0+0x270 = 0xfaf0ed60
[ERROR] Invalid block size at position 0xfb438ae0
Next known block: NULL at 0xfb438af0+0x270 = 0xfb438d60
[ERROR] Invalid block size at position 0xfb962ae0
Next known block: NULL at 0xfb962af0+0x270 = 0xfb962d60
[ERROR] Invalid block size at position 0xfbe8cae0
Next known block: NULL at 0xfbe8caf0+0x270 = 0xfbe8cd60
[ERROR] Invalid block size at position 0xfc3b6ae0
Next known block: NULL at 0xfc3b6af0+0x270 = 0xfc3b6d60
[ERROR] Invalid block size at position 0xfc8e0ae0
Next known block: NULL at 0xfc8e0af0+0x270 = 0xfc8e0d60
[ERROR] Invalid block size at position 0xfce0aae0
Next known block: NULL at 0xfce0aaf0+0x270 = 0xfce0ad60
[ERROR] Invalid block size at position 0xfd334ae0
Next known block: NULL at 0xfd334af0+0x270 = 0xfd334d60
[ERROR] Invalid block size at position 0xfd85eae0
Next known block: NULL at 0xfd85eaf0+0x270 = 0xfd85ed60
[ERROR] Invalid block size at position 0xfdd88ae0
Next known block: NULL at 0xfdd88af0+0x270 = 0xfdd88d60
[ERROR] Invalid block size at position 0xfe2b2ae0
Next known block: NULL at 0xfe2b2af0+0x270 = 0xfe2b2d60
[ERROR] Invalid block size at position 0xfe7dcae0
Next known block: NULL at 0xfe7dcaf0+0x270 = 0xfe7dcd60
[ERROR] Invalid block size at position 0xfef9bae0
Next known block: NULL at 0xfef9baf0+0x270 = 0xfef9bd60
[ERROR] Invalid block size at position 0xff4c5ae0
Next known block: NULL at 0xff4c5af0+0x270 = 0xff4c5d60
[ERROR] Invalid block size at position 0xff9efae0
Next known block: NULL at 0xff9efaf0+0x270 = 0xff9efd60
[ERROR] Invalid block size at position 0xfff19ae0
Next known block: NULL at 0xfff19af0+0x270 = 0xfff19d60
Reached end of chunk 1/1 after 4169 blocks
Processed 1582 video frames at 24.00 FPS (65.92 s)
Done
I get this on my Mac PowerBook using Danne's patched mlv_dump:
./mlv_dump --dng M15-0904.MLV
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: 'M15-0904.MLV'
- Using output path 'M15-0904_' for DNGs
- Convert to DNG frames
- Output into 'M15-0904_'
File M15-0904.MLV opened
File M15-0904.M00 not existing.
Processing...
Unknown Block: %%, skipping
Next known block: NULL at 0x7db66ae0+0x280 = 0x7db66d60
[ERROR] Invalid block size at position 0xd95feae0
Next known block: NULL at 0xe15feaf0+0x270 = 0xe15feaf0
[ERROR] Invalid block size at position 0xe15feaf0
Next known block: NULL at 0xe95feb00+0x11a260 = 0xe95feb00
[ERROR] Invalid block size at position 0xe95feb00
Next known block: NULL at 0xf15feb10+0x233fd0 = 0xf15feb10
[ERROR] Invalid block size at position 0xf15feb10
Next known block: NULL at 0xf95feb20+0xb9240 = 0xf95feb20
[ERROR] Invalid block size at position 0xf95feb20
Next known block: NULL at 0xfff1a260+0x1d2fb0 = 0xfff1a260
Reached end of chunk 1/1 after 3512 blocks
Processed 1344 video frames at 24.00 FPS (56.00 s)
Done
So yeah, the Mac is not processing those 238 missing frames.
@Danne
Thanks for that, I'll study compiling macintosh docs to get a better understanding of this stuff.
@dfort
I missed that! I'm uploading M15-0904.M00 now (it's 1.4 GB) and will post link shortly.
@a1ex
I haven't had this problem for 3 years using the build I have with same camera and settings, CF cards and Mac OSX 10.85. Could I have some virus or a conflicting software that's affecting this?
@ bouncyball
QuoteBTW audio level is very low)
I need to update to latest build - but what I have now only allows me to adjust speaker level (which is set to 3). I'll need to test with level 4 or 5 once I get this issue settled. Latest build may be updated, so I'll look into that.
I recorded with short boom mic into Zoom H1 which was routed into my Canon 5D3 via mic port. Levels looked normal on screen, but your right, the audio is a little too soft.
Quote from: sax100 on September 20, 2018, 06:26:43 PM
@a1ex
I haven't had this problem for 3 years using the build I have with same camera and settings, CF cards and Mac OSX 10.85. Could I have some virus or a conflicting software that's affecting this?
Quote from: a1ex on September 20, 2018, 12:09:10 PM
Pretty sure it was this bug (https://www.magiclantern.fm/forum/index.php?topic=19761.msg186060#msg186060).
Click on the link; it has the answer to your question.
TLDR: edge case, non-deterministic bug, hard to narrow down.
Quote from: a1ex on September 20, 2018, 05:57:58 PM
Maybe some Mac compatibility issue, possibly at fseek?
Been in fseek hell before (https://www.magiclantern.fm/forum/index.php?topic=15705.msg153446#msg153446). Maybe the Windows exception also applies to the Mac?
/* platform/target specific fseek/ftell functions go here */
uint64_t file_get_pos(FILE *stream)
{
#if defined(__WIN32)
return ftello64(stream);
#else
return ftello(stream);
#endif
}
uint32_t file_set_pos(FILE *stream, uint64_t offset, int whence)
{
#if defined(__WIN32)
return fseeko64(stream, offset, whence);
#else
return fseeko(stream, offset, whence);
#endif
}
How is the Mac defined?
Quote from: dfort on September 20, 2018, 07:07:55 PM
How is the Mac defined?
#else, in that snippet, means "anything but __WIN32", so that includes Mac.
Check the updated patch above.
Again, I'm still learning... I'm sure I'm doing something wrong.
I downloaded mlv_dump.osx under Last Successful Artifact provided by g3gg0 here (https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/).
And @a1ex patch - mlv_dump_bruteforce-v2.patch
Tried to run patch, but failed:
patch < mlv_dump_bruteforce-v2.patch
File to patch: mlv_dump.osx
patching file mlv_dump.osx
Hunk #1 FAILED at 448.
Hunk #2 FAILED at 1239.
Hunk #3 FAILED at 2155.
Hunk #4 FAILED at 4112.
4 out of 4 hunks FAILED -- saving rejects to file mlv_dump.osx.rej
Quote from: sax100 on September 20, 2018, 08:25:47 PM
patch < mlv_dump_bruteforce-v2.patch
That's not the way to apply the patch. It needs to be applied to the source code. Compiling mlv_dump is a more advanced topic that we probably don't need to go into at this time but I'll be glad to discuss it with you when we meet--we're practically neighbors.
Quote from: a1ex on September 20, 2018, 07:39:24 PM
#else, in that snippet, means "anything but __WIN32", so that includes Mac.
Check the updated patch above.
Right, I know that. Just thought I'd experiment with different ftello/fseeko options for the Mac. No luck so far.
The patch applied cleanly but I'm not getting the 1582 frames you're getting on Linux.
I'm getting 1582 frames on Mac, too...
(https://a1ex.magiclantern.fm/bleeding-edge/mlv-recover/macmac.png)
Quote from: a1ex on September 20, 2018, 05:57:58 PM
edit: updated the patch and confirmed it's working on the Mac VM:
mlv_dump_bruteforce-v2.patch (https://a1ex.magiclantern.fm/bleeding-edge/mlv-recover/mlv_dump_bruteforce-v2.patch)
Doh! Missed that link to the v2 patch. Yes, it works on the Mac!
@sax100 - Try the new mlv_dump I compiled for you and please report back.
Yup, version 2 works, kudos a1ex:
mlv_dump version here:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump
Last login: Thu Sep 20 22:12:37 on ttys003
dans-MacBook-Pro:corruption dan$ /Users/dan/Desktop/Timer_A/corruption/mlv_dump --dng /Users/dan/Desktop/Timer_A/corruption/M15-0904.MLV
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: '/Users/dan/Desktop/Timer_A/corruption/M15-0904.MLV'
- Using output path '/Users/dan/Desktop/Timer_A/corruption/M15-0904_' for DNGs
- Convert to DNG frames
- Output into '/Users/dan/Desktop/Timer_A/corruption/M15-0904_'
File /Users/dan/Desktop/Timer_A/corruption/M15-0904.MLV opened
File /Users/dan/Desktop/Timer_A/corruption/M15-0904.M00 not existing.
Processing...
Unknown Block: %%, skipping
Next known block: NULL at 0x7db66ae0+0x280 = 0x7db66d60
[ERROR] Invalid block size at position 0xd95feae0
Next known block: NULL at 0xd95feaf0+0x270 = 0xd95fed60
[ERROR] Invalid block size at position 0xda052ae0
Next known block: NULL at 0xda052af0+0x270 = 0xda052d60
[ERROR] Invalid block size at position 0xda57cae0
Next known block: NULL at 0xda57caf0+0x270 = 0xda57cd60
[ERROR] Invalid block size at position 0xdaaa6ae0
Next known block: NULL at 0xdaaa6af0+0x270 = 0xdaaa6d60
[ERROR] Invalid block size at position 0xdafd0ae0
Next known block: NULL at 0xdafd0af0+0x270 = 0xdafd0d60
[ERROR] Invalid block size at position 0xdb4faae0
Next known block: NULL at 0xdb4faaf0+0x270 = 0xdb4fad60
[ERROR] Invalid block size at position 0xdba24ae0
Next known block: NULL at 0xdba24af0+0x270 = 0xdba24d60
[ERROR] Invalid block size at position 0xdbf4eae0
Next known block: NULL at 0xdbf4eaf0+0x270 = 0xdbf4ed60
[ERROR] Invalid block size at position 0xdc478ae0
Next known block: NULL at 0xdc478af0+0x270 = 0xdc478d60
[ERROR] Invalid block size at position 0xdc9a2ae0
Next known block: NULL at 0xdc9a2af0+0x270 = 0xdc9a2d60
[ERROR] Invalid block size at position 0xdd161ae0
Next known block: NULL at 0xdd161af0+0x270 = 0xdd161d60
[ERROR] Invalid block size at position 0xdd68bae0
Next known block: NULL at 0xdd68baf0+0x270 = 0xdd68bd60
[ERROR] Invalid block size at position 0xddbb5ae0
Next known block: NULL at 0xddbb5af0+0x270 = 0xddbb5d60
[ERROR] Invalid block size at position 0xde0dfae0
Next known block: NULL at 0xde0dfaf0+0x270 = 0xde0dfd60
[ERROR] Invalid block size at position 0xde89eae0
Next known block: NULL at 0xde89eaf0+0x270 = 0xde89ed60
[ERROR] Invalid block size at position 0xdedc8ae0
Next known block: NULL at 0xdedc8af0+0x270 = 0xdedc8d60
[ERROR] Invalid block size at position 0xdf2f2ae0
Next known block: NULL at 0xdf2f2af0+0x270 = 0xdf2f2d60
[ERROR] Invalid block size at position 0xdf81cae0
Next known block: NULL at 0xdf81caf0+0x270 = 0xdf81cd60
[ERROR] Invalid block size at position 0xdfd46ae0
Next known block: NULL at 0xdfd46af0+0x270 = 0xdfd46d60
[ERROR] Invalid block size at position 0xe0270ae0
Next known block: NULL at 0xe0270af0+0x270 = 0xe0270d60
[ERROR] Invalid block size at position 0xe079aae0
Next known block: NULL at 0xe079aaf0+0x270 = 0xe079ad60
[ERROR] Invalid block size at position 0xe0f59ae0
Next known block: NULL at 0xe0f59af0+0x270 = 0xe0f59d60
[ERROR] Invalid block size at position 0xe1718ae0
Next known block: NULL at 0xe1718af0+0x270 = 0xe1718d60
[ERROR] Invalid block size at position 0xe1ed7ae0
Next known block: NULL at 0xe1ed7af0+0x270 = 0xe1ed7d60
[ERROR] Invalid block size at position 0xe2696ae0
Next known block: NULL at 0xe2696af0+0x270 = 0xe2696d60
[ERROR] Invalid block size at position 0xe2e55ae0
Next known block: NULL at 0xe2e55af0+0x270 = 0xe2e55d60
[ERROR] Invalid block size at position 0xe337fae0
Next known block: NULL at 0xe337faf0+0x270 = 0xe337fd60
[ERROR] Invalid block size at position 0xe38a9ae0
Next known block: NULL at 0xe38a9af0+0x270 = 0xe38a9d60
[ERROR] Invalid block size at position 0xe3dd3ae0
Next known block: NULL at 0xe3dd3af0+0x270 = 0xe3dd3d60
[ERROR] Invalid block size at position 0xe4592ae0
Next known block: NULL at 0xe4592af0+0x270 = 0xe4592d60
[ERROR] Invalid block size at position 0xe4abcae0
Next known block: NULL at 0xe4abcaf0+0x270 = 0xe4abcd60
[ERROR] Invalid block size at position 0xe4fe6ae0
Next known block: NULL at 0xe4fe6af0+0x270 = 0xe4fe6d60
[ERROR] Invalid block size at position 0xe5510ae0
Next known block: NULL at 0xe5510af0+0x270 = 0xe5510d60
[ERROR] Invalid block size at position 0xe5ccfae0
Next known block: NULL at 0xe5ccfaf0+0x270 = 0xe5ccfd60
[ERROR] Invalid block size at position 0xe648eae0
Next known block: NULL at 0xe648eaf0+0x270 = 0xe648ed60
[ERROR] Invalid block size at position 0xe6c4dae0
Next known block: NULL at 0xe6c4daf0+0x270 = 0xe6c4dd60
[ERROR] Invalid block size at position 0xe7177ae0
Next known block: NULL at 0xe7177af0+0x270 = 0xe7177d60
[ERROR] Invalid block size at position 0xe76a1ae0
Next known block: NULL at 0xe76a1af0+0x270 = 0xe76a1d60
[ERROR] Invalid block size at position 0xe7bcbae0
Next known block: NULL at 0xe7bcbaf0+0x270 = 0xe7bcbd60
[ERROR] Invalid block size at position 0xe838aae0
Next known block: NULL at 0xe838aaf0+0x270 = 0xe838ad60
[ERROR] Invalid block size at position 0xe88b4ae0
Next known block: NULL at 0xe88b4af0+0x270 = 0xe88b4d60
[ERROR] Invalid block size at position 0xe8ddeae0
Next known block: NULL at 0xe8ddeaf0+0x270 = 0xe8dded60
[ERROR] Invalid block size at position 0xe9308ae0
Next known block: NULL at 0xe9308af0+0x270 = 0xe9308d60
[ERROR] Invalid block size at position 0xe9ac7ae0
Next known block: NULL at 0xe9ac7af0+0x270 = 0xe9ac7d60
[ERROR] Invalid block size at position 0xea286ae0
Next known block: NULL at 0xea286af0+0x270 = 0xea286d60
[ERROR] Invalid block size at position 0xea7b0ae0
Next known block: NULL at 0xea7b0af0+0x270 = 0xea7b0d60
[ERROR] Invalid block size at position 0xeacdaae0
Next known block: NULL at 0xeacdaaf0+0x270 = 0xeacdad60
[ERROR] Invalid block size at position 0xeb204ae0
Next known block: NULL at 0xeb204af0+0x270 = 0xeb204d60
[ERROR] Invalid block size at position 0xeb9c3ae0
Next known block: NULL at 0xeb9c3af0+0x270 = 0xeb9c3d60
[ERROR] Invalid block size at position 0xec182ae0
Next known block: NULL at 0xec182af0+0x270 = 0xec182d60
[ERROR] Invalid block size at position 0xec941ae0
Next known block: NULL at 0xec941af0+0x270 = 0xec941d60
[ERROR] Invalid block size at position 0xed100ae0
Next known block: NULL at 0xed100af0+0x270 = 0xed100d60
[ERROR] Invalid block size at position 0xed8bfae0
Next known block: NULL at 0xed8bfaf0+0x270 = 0xed8bfd60
[ERROR] Invalid block size at position 0xee07eae0
Next known block: NULL at 0xee07eaf0+0x270 = 0xee07ed60
[ERROR] Invalid block size at position 0xee5a8ae0
Next known block: NULL at 0xee5a8af0+0x270 = 0xee5a8d60
[ERROR] Invalid block size at position 0xeead2ae0
Next known block: NULL at 0xeead2af0+0x270 = 0xeead2d60
[ERROR] Invalid block size at position 0xeeffcae0
Next known block: NULL at 0xeeffcaf0+0x270 = 0xeeffcd60
[ERROR] Invalid block size at position 0xef7bbae0
Next known block: NULL at 0xef7bbaf0+0x270 = 0xef7bbd60
[ERROR] Invalid block size at position 0xeff7aae0
Next known block: NULL at 0xeff7aaf0+0x270 = 0xeff7ad60
[ERROR] Invalid block size at position 0xf0739ae0
Next known block: NULL at 0xf0739af0+0x270 = 0xf0739d60
[ERROR] Invalid block size at position 0xf0ef8ae0
Next known block: NULL at 0xf0ef8af0+0x270 = 0xf0ef8d60
[ERROR] Invalid block size at position 0xf16b7ae0
Next known block: NULL at 0xf16b7af0+0x270 = 0xf16b7d60
[ERROR] Invalid block size at position 0xf1e76ae0
Next known block: NULL at 0xf1e76af0+0x270 = 0xf1e76d60
[ERROR] Invalid block size at position 0xf23a0ae0
Next known block: NULL at 0xf23a0af0+0x270 = 0xf23a0d60
[ERROR] Invalid block size at position 0xf28caae0
Next known block: NULL at 0xf28caaf0+0x270 = 0xf28cad60
[ERROR] Invalid block size at position 0xf2df4ae0
Next known block: NULL at 0xf2df4af0+0x270 = 0xf2df4d60
[ERROR] Invalid block size at position 0xf331eae0
Next known block: NULL at 0xf331eaf0+0x270 = 0xf331ed60
[ERROR] Invalid block size at position 0xf3848ae0
Next known block: NULL at 0xf3848af0+0x270 = 0xf3848d60
[ERROR] Invalid block size at position 0xf3d72ae0
Next known block: NULL at 0xf3d72af0+0x270 = 0xf3d72d60
[ERROR] Invalid block size at position 0xf429cae0
Next known block: NULL at 0xf429caf0+0x270 = 0xf429cd60
[ERROR] Invalid block size at position 0xf47c6ae0
Next known block: NULL at 0xf47c6af0+0x270 = 0xf47c6d60
[ERROR] Invalid block size at position 0xf4cf0ae0
Next known block: NULL at 0xf4cf0af0+0x270 = 0xf4cf0d60
[ERROR] Invalid block size at position 0xf521aae0
Next known block: NULL at 0xf521aaf0+0x270 = 0xf521ad60
[ERROR] Invalid block size at position 0xf5744ae0
Next known block: NULL at 0xf5744af0+0x270 = 0xf5744d60
[ERROR] Invalid block size at position 0xf5c6eae0
Next known block: NULL at 0xf5c6eaf0+0x270 = 0xf5c6ed60
[ERROR] Invalid block size at position 0xf6198ae0
Next known block: NULL at 0xf6198af0+0x270 = 0xf6198d60
[ERROR] Invalid block size at position 0xf66c2ae0
Next known block: NULL at 0xf66c2af0+0x270 = 0xf66c2d60
[ERROR] Invalid block size at position 0xf6becae0
Next known block: NULL at 0xf6becaf0+0x270 = 0xf6becd60
[ERROR] Invalid block size at position 0xf7116ae0
Next known block: NULL at 0xf7116af0+0x270 = 0xf7116d60
[ERROR] Invalid block size at position 0xf7640ae0
Next known block: NULL at 0xf7640af0+0x270 = 0xf7640d60
[ERROR] Invalid block size at position 0xf7b6aae0
Next known block: NULL at 0xf7b6aaf0+0x270 = 0xf7b6ad60
[ERROR] Invalid block size at position 0xf8094ae0
Next known block: NULL at 0xf8094af0+0x270 = 0xf8094d60
[ERROR] Invalid block size at position 0xf85beae0
Next known block: NULL at 0xf85beaf0+0x270 = 0xf85bed60
[ERROR] Invalid block size at position 0xf8ae8ae0
Next known block: NULL at 0xf8ae8af0+0x270 = 0xf8ae8d60
[ERROR] Invalid block size at position 0xf92a7ae0
Next known block: NULL at 0xf92a7af0+0x270 = 0xf92a7d60
[ERROR] Invalid block size at position 0xf9a66ae0
Next known block: NULL at 0xf9a66af0+0x270 = 0xf9a66d60
[ERROR] Invalid block size at position 0xfa225ae0
Next known block: NULL at 0xfa225af0+0x270 = 0xfa225d60
[ERROR] Invalid block size at position 0xfa9e4ae0
Next known block: NULL at 0xfa9e4af0+0x270 = 0xfa9e4d60
[ERROR] Invalid block size at position 0xfaf0eae0
Next known block: NULL at 0xfaf0eaf0+0x270 = 0xfaf0ed60
[ERROR] Invalid block size at position 0xfb438ae0
Next known block: NULL at 0xfb438af0+0x270 = 0xfb438d60
[ERROR] Invalid block size at position 0xfb962ae0
Next known block: NULL at 0xfb962af0+0x270 = 0xfb962d60
[ERROR] Invalid block size at position 0xfbe8cae0
Next known block: NULL at 0xfbe8caf0+0x270 = 0xfbe8cd60
[ERROR] Invalid block size at position 0xfc3b6ae0
Next known block: NULL at 0xfc3b6af0+0x270 = 0xfc3b6d60
[ERROR] Invalid block size at position 0xfc8e0ae0
Next known block: NULL at 0xfc8e0af0+0x270 = 0xfc8e0d60
[ERROR] Invalid block size at position 0xfce0aae0
Next known block: NULL at 0xfce0aaf0+0x270 = 0xfce0ad60
[ERROR] Invalid block size at position 0xfd334ae0
Next known block: NULL at 0xfd334af0+0x270 = 0xfd334d60
[ERROR] Invalid block size at position 0xfd85eae0
Next known block: NULL at 0xfd85eaf0+0x270 = 0xfd85ed60
[ERROR] Invalid block size at position 0xfdd88ae0
Next known block: NULL at 0xfdd88af0+0x270 = 0xfdd88d60
[ERROR] Invalid block size at position 0xfe2b2ae0
Next known block: NULL at 0xfe2b2af0+0x270 = 0xfe2b2d60
[ERROR] Invalid block size at position 0xfe7dcae0
Next known block: NULL at 0xfe7dcaf0+0x270 = 0xfe7dcd60
[ERROR] Invalid block size at position 0xfef9bae0
Next known block: NULL at 0xfef9baf0+0x270 = 0xfef9bd60
[ERROR] Invalid block size at position 0xff4c5ae0
Next known block: NULL at 0xff4c5af0+0x270 = 0xff4c5d60
[ERROR] Invalid block size at position 0xff9efae0
Next known block: NULL at 0xff9efaf0+0x270 = 0xff9efd60
[ERROR] Invalid block size at position 0xfff19ae0
Next known block: NULL at 0xfff19af0+0x270 = 0xfff19d60
Reached end of chunk 1/1 after 4169 blocks
Processed 1582 video frames at 24.00 FPS (65.92 s)
Done
Checking dng amount:
dans-MacBook-Pro:corruption dan$ find . -name '*.dng' | wc -l
1582
All ok.
IT WORKS!!! Thank you so much @a1ex, @Danne and @dfort!!!
All frames are rendered, audio too, from beginning to end.
Is there a command to create a folder (M15-0904) to put all .dng from (M15-0904.MLV + M15-0904.M00) into it automatically and do the same for all other clips?
I've been using the script below, which made it easier to convert 20-30 files at once and keep them organized.
I tried it, but the conversion stops at frame 780.
workingDir=`dirname "$0"`
cd "${workingDir}"
sudo mv ./mlv_dump.osx /usr/bin/mlv_dump
sudo chmod +x /usr/bin/mlv_dump
for FILE in `ls -A1 *.MLV *.mlv 2>/dev/null`; do
BASE=`echo $FILE | cut -d "." -f1`;
mkdir $BASE;
mv ./"$BASE".M* ./$BASE
cd ./$BASE
/usr/bin/mlv_dump -x $FILE
/usr/bin/mlv_dump --dng -o ${BASE}_frame_ $FILE
/usr/bin/mlv_dump -m -o meta.mlv $FILE
/usr/bin/mlv_dump -v meta.mlv >$FILE.txt
rm ./meta.mlv
rm ./meta.mlv.wav
cd ..
done
@dfort
Thank you very much for compiling the new and working mlv_dump! I hope to increase my IQ to be able to do that.
For that script to work you need to change this line:
sudo mv ./mlv_dump.osx /usr/bin/mlv_dump
To this:
sudo mv ./mlv_dump /usr/bin/mlv_dump
Note that those sudo commands are only needed once. Once mlv_dump is living in /usr/bin if you remove those two lines from the script you won't need to enter your password to run the script.
Another option is to use Danne's switch app. Ask him to prepare a build for you with the version 2 patched mlv_dump.
[EDIT] What's the purpose of these lines? Make a log file?
/usr/bin/mlv_dump -m -o meta.mlv $FILE
/usr/bin/mlv_dump -v meta.mlv >$FILE.txt
rm ./meta.mlv
rm ./meta.mlv.wav
I believe that the -m option is no longer valid.
Thanks again to EVERYONE for all your great help!
I took out the .osx as you instructed and it worked! Thank you @dfort!
Also took out the -m option as well.
How can I make this script recursive?
workingDir=`dirname "$0"`
cd "${workingDir}"
sudo mv ./mlv_dump /usr/bin/mlv_dump
sudo chmod +x /usr/bin/mlv_dump
for FILE in `ls -A1 *.MLV *.mlv 2>/dev/null`; do
BASE=`echo $FILE | cut -d "." -f1`;
mkdir $BASE;
mv ./"$BASE".M* ./$BASE
cd ./$BASE
/usr/bin/mlv_dump -x $FILE
/usr/bin/mlv_dump --dng -o ${BASE}_frame_ $FILE
/usr/bin/mlv_dump -o meta.mlv $FILE
/usr/bin/mlv_dump -v meta.mlv >$FILE.txt
rm ./meta.mlv
rm ./meta.mlv.wav
cd ..
done
@Danne
Thank you for your mlv_dump also - I downloaded and tried yours as well and it works great!
That script is problematic for several reasons as dfort points out. I will put this into Switch when when I can soon.
Updated Switch:
https://www.magiclantern.fm/forum/index.php?topic=15108.0
commit:
https://bitbucket.org/Dannephoto/switch/commits/d1f3b9cf31616376c10fcd69cb0ef035804fdce6
If you still want to use your script you could simply do this with the new version of mlv_dump. Scriptwise there´s a lot more to wish for but hey, whatever works works:
workingDir=`dirname "$0"`
cd "${workingDir}"
sudo mv ./mlv_dump /usr/bin/mlv_dump
sudo chmod +x /usr/bin/mlv_dump
for FILE in `ls -A1 *.MLV *.mlv 2>/dev/null`; do
BASE=`echo $FILE | cut -d "." -f1`;
mkdir $BASE;
mv ./"$BASE".M* ./$BASE
cd ./$BASE
/usr/bin/mlv_dump --dng -o ${BASE}_frame_ $FILE
cd ..
done
I assume you are using an older OS version since /usr/bin is a no go zone in Sierra.
Have you checked Mlv App by the way? Still needs the patch but I am sure Bouncyball will fix it.
@Danne:
BTW when I posted recovered 1st MLV part above I did not see a1ex's patch yet, so it was done manually with hexedit, I wish I had refreshed the last page of this thread ;)
@a1ex:
Thank you for simple and 100% working code, I wanted to implement brute force method into mlv app long time ago but been lazy and slow (as usual) lately ;D.
regards
bb
Edit: From the beginning it was obvious that recording of that clip's been done with some old build of mlv_rec, but I could not imagine it was that old ;), so issues like this are not expected in the nearest future, but anyway would be nice to update mlv app's loading section.
https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/36/
does this version work?
from: https://www.magiclantern.fm/forum/index.php?topic=19761.msg186010#msg186010
"you can specify a bugfix option "-F 3" to fix that issue"
@g3gg0 - No, that build doesn't work with this file:
./mlv_dump.osx -F 3 --dng M15-0904.MLV
FIX BUG #3 [active]
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: 'M15-0904.MLV'
- Decompressing before writing DNG
- Enforcing 14bpp for DNG output
- Using output path 'M15-0904_' for DNGs
- Convert to DNG frames
- Output into 'M15-0904_'
File M15-0904.MLV opened
File M15-0904.M00 not existing.
Processing...
Vertical stripes correction:
1.00000 1.00290 0.99698 1.00259 0.99626 1.00102 0.99782 0.99646
Cold pixels : 0
Unknown Block: %%, skipping
[ERROR] Invalid block size at position 0x7dd46afe
Processed 777 video frames at 24.00 FPS (32.38 s)
Done
@Danne
Quote from: Danne on September 21, 2018, 10:27:06 AM
Have you checked Mlv App by the way? Still needs the patch but I am sure Bouncyball will fix it.
Done
Thanks!
Quote from: dfort on September 21, 2018, 12:28:17 PM
@g3gg0 - No, that build doesn't work with this file:
./mlv_dump.osx -F 3 --dng M15-0904.MLV
FIX BUG #3 [active]
MLV Dumper
-----------------
Mode of operation:
- Input MLV file: 'M15-0904.MLV'
- Decompressing before writing DNG
- Enforcing 14bpp for DNG output
- Using output path 'M15-0904_' for DNGs
- Convert to DNG frames
- Output into 'M15-0904_'
File M15-0904.MLV opened
File M15-0904.M00 not existing.
Processing...
Vertical stripes correction:
1.00000 1.00290 0.99698 1.00259 0.99626 1.00102 0.99782 0.99646
Cold pixels : 0
Unknown Block: %%, skipping
[ERROR] Invalid block size at position 0x7dd46afe
Processed 777 video frames at 24.00 FPS (32.38 s)
Done
@Danne:
Tested mlv_dump_on_steriods with altered Script you provided and it works like a charm! Thank you for that!
I can run it once inside a folder with all my .mlv files and it converts and organizes without a problem.
I'm still using OSX Mountain Lion 10.8.5 (I know, it's really old and I'll have to upgrade soon).
It works well with Davinci Resolve 10 Lite and Adobe CS6. Tried the upgrade to El Capitan to use Resolve 11 and 12, but too many little issues with CS6 and not enough good reasons to use Resolve 11/12 turned me away. But when I upgrade to Sierra or above, I'm going to try the Mac Switch.
It's all a1ex patches and g3gg0 groundwork but some testing and compiling is good to be able to do.
Mlv App might work with your OS.
Yes, I hope so... I compile MLVApp on OSX 10.9.5 and my compiler and one ML user told it is compatible with 10.8.5. ;) Have a try!
My problem has already been fixed, but for future reference, I indicated I had the wrong build - this is my correct build.
Nightly.2014May22.5D3113
@bouncyball
Quote from: bouncyball on September 20, 2018, 12:28:54 PM
BTW audio level is very low
I indicated that I used a Zoom H1 and short boom mic connected to the mic jack of my Canon 5D3, but that was wrong. For this particular shot I recorded, I didn't use a mic, instead I used the built in mic of the camera. So I did the below to increase the audio level, which I will make sure to keep for future shoots.
I corrected this by entering my Canon 5D's menu (not MLV's) and going to the SHOOT4 window, to Sound recording, and setting Rec level to 75% (it was previously set to 25%). The audio level seems normal now based on audio meter levels.
I've also Enabled Wind Filter to see if it would help - I need to test extensively to verify.
And Sound Rec. is set to Manual, as it was previously.
@sax100
Quote from: sax100 on September 24, 2018, 07:24:49 PM
My problem has already been fixed, but for future reference, I indicated I had the wrong build - this is my correct build.
Nightly.2014May22.5D3113
You should try this one (https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/80/artifact/platform/5D3.113/magiclantern-crop_rec_4k.2018Jul22.5D3113.zip)
Turned out that @sax100 lives nearby so I stopped by his place the other day and got him setup with the latest crop_rec_4k build and showed him the H.264 proxy workflow using Switch.
Looking forward to his feature (https://www.imdb.com/title/tt4924656/), shot on Magic Lantern. So far all I saw were some test posters but the poster was made from a frame of raw video and it looked good.
a 83 minute movie shot with ml raw?
Quote from: g3gg0 on November 29, 2018, 08:05:50 PM
a 83 minute movie shot with ml raw?
That's right. Now streaming online!
https://www.debtofmaximillian.com/