Magic Lantern Forum

Using Magic Lantern => Raw Video => Topic started by: Toffifee on April 08, 2014, 08:26:45 AM

Title: 7/32 MLVs empty
Post by: Toffifee on April 08, 2014, 08:26:45 AM
Hi,

I apologise if there is already a post for this, however I did not see one.

I seem to have several .MLVs not being able to process at all. Out of 32 files processed I had 7 folders with no .DNGs only 1 .WAV file. The .WAV files are always exactly 44 bytes large. I tried different programs to process with and it's a problem I've had on two different nightly builds of which March 12 5D3113 on a 5D Mark III.

Has anybody encountered similar problems and if so, how can I fix it?


Best regards,
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 08, 2014, 09:48:18 AM
can you check those faulty .mlv files for their size?
also an output of "mlv_dump -v <file.mlv>" would help
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 08, 2014, 07:00:52 PM
1.29 Gb
232 Mb
915 Mb


I'm working on the mlv_dump, but I keep deleting the script to do it, so I'm just trying to find the terminal code to use the program.
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 08, 2014, 07:14:19 PM
This is what MLV_Dump says when I go mlv_dump <filename> in terminal:


MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'm23-0532.mlv'
   - Verify file structure
File m23-0532.mlv opened
Processing...
Reached end of chunk 1/1 after 726 blocks
Processed 250 video frames
Done



edit: Then nothing happens, no files are extracted.
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 08, 2014, 09:25:04 PM
i asked for the output of "mlv_dump -v <file>"  ;)

if you want to have frames dumped, use "mlv_dump --dng <file>"
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 09, 2014, 04:08:27 AM
Oops!
Didn't see that part  :)

The report exceeds the maximum allowed length of characters this forum allows (20k).

Here's the dropbox link to download the report.

https://dl.dropboxusercontent.com/u/21538108/mlv_dump%20-v.rtf
Title: Re: 7/32 MLVs empty
Post by: AndreaFox on April 09, 2014, 05:38:01 AM
Try to use MLV Mystic, for me it works very well

http://secondsparkproductions.com/index.php/blog/8-mlv-mystic-mlv-raw-to-dng-batch-converter
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 09, 2014, 06:48:57 AM
I have it installed, honestly it's probably the program that work the least for me.
At the moment I'm most using a windows program through parallells:

http://www.magiclantern.fm/forum/index.php?topic=10198.0
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 15, 2014, 06:19:15 AM
Did you see anything irregular g3gg0??
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 15, 2014, 12:27:20 PM
didnt notice you wrote.

a) cannot open the .rtf. can you post that on pastebin.com?
b) did you try the --dng parameter i wrote?
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 15, 2014, 07:39:43 PM
a)
http://pastebin.com/fxvYdDfT


b) Here's the result I get from --dng:


mlv_dump --dng M23-0532.MLV

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'M23-0532.MLV'
   - Convert to DNG frames
   - Output into 'M23-0532_frame_'
File M23-0532.MLV opened
Processing...
[ERROR] File ends in the middle of a block
Processed 0 video frames
Done


and it creates a 44 byte .wav file.
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 15, 2014, 07:42:02 PM
Also, I've noticed that the aperture values always reads as f/0.00.
That's a bug right?
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 15, 2014, 10:01:37 PM
interesting....
can you do a mlv_dump -v --dng M23-0532.MLV please?
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 15, 2014, 10:02:10 PM
Quote from: Toffifee on April 15, 2014, 07:42:02 PM
Also, I've noticed that the aperture values always reads as f/0.00.
That's a bug right?

should not happen. does it change when you select M mode?
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 16, 2014, 04:20:25 AM
I always shoot in M mode.

Here's the mlv_dump -v --dng M23-0532.MLV



MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'M23-0532.MLV'
   - Verbose messages
   - Convert to DNG frames
   - Output into 'M23-0532_frame_'
File M23-0532.MLV opened
Processing...
File Header (MLVI)
    Size        : 0x00000034
    Ver         : v2.0
    GUID        : 4449552110025059234
    FPS         : 23.976000
    File        : 0 / 1
    Frames Video: 252
    Frames Audio: 54
Block: RAWI
  Offset: 0x00000034
    Size: 180
    Time: 234.528000 ms
    Res:  1920x1080
    raw_info:
      api_version      0x00000001
      height           1318
      width            2080
      pitch            3640
      frame_size       0x00493450
      bits_per_pixel   14
      black_level      2045
      white_level      15000
      active_area.y1   28
      active_area.x1   146
      active_area.y2   1318
      active_area.x2   2078
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1
Block: INFO
  Offset: 0x000000e8
    Size: 16
    Time: 238.052000 ms
Block: RTCI
  Offset: 0x000000f8
    Size: 44
    Time: 0.001000 ms
     Date:        23.03.2014
     Time:        05:32:31 (GMT+0)
     Zone:        ''
     Day of week: 0
     Day of year: 81
     Daylight s.: 0
Block: EXPO
  Offset: 0x00000124
    Size: 40
    Time: 0.002000 ms
     ISO Mode:   0
     ISO:        100
     ISO Analog: 72
     ISO DGain:  0/1024 EV
     Shutter:    19983 µs (1/50.04)
Block: LENS
  Offset: 0x0000014c
    Size: 96
    Time: 0.003000 ms
     Name:        'EF24-105mm f/4L IS USM'
     Serial:      ''
     Focal Len:   24 mm
     Focus Dist:  386 mm
     Aperture:    f/0.00
     IS Mode:     14
     AF Mode:     0
     Lens ID:     0x000000ED
     Flags:       0x00000000
Block: IDNT
  Offset: 0x000001ac
    Size: 84
    Time: 0.004000 ms
     Camera Name:   'Canon EOS 5D Mark III'
     Camera Serial: '55C289CD8'
     Camera Model:  0x80000285
Block: WBAL
  Offset: 0x00000200
    Size: 44
    Time: 0.005000 ms
     Mode:   0
     Kelvin:   2500
     Gain R:   491
     Gain G:   1024
     Gain B:   638
     Shift GM:   0
     Shift BA:   0
Block: STYL
  Offset: 0x0000022c
    Size: 52
    Time: 0.006000 ms
     picStyle:   34
     contrast:   0
     sharpness:  5
     saturation: 0
     colortone:  0
Block: NULL
  Offset: 0x00000260
    Size: 36
Block: WAVI
  Offset: 0x00000284
    Size: 32
    Time: 280.990000 ms
    wav_info:
      format           1
      channels         2
      samplingRate     48000
      bytesPerSecond   192000
      blockAlign       4
      bitsPerSample    16
Block: EXPO
  Offset: 0x000002a4
    Size: 40
    Time: 307.326000 ms
     ISO Mode:   0
     ISO:        100
     ISO Analog: 72
     ISO DGain:  0/1024 EV
     Shutter:    19983 µs (1/50.04)
Block: LENS
  Offset: 0x000002cc
    Size: 96
    Time: 307.361000 ms
     Name:        'EF24-105mm f/4L IS USM'
     Serial:      ''
     Focal Len:   24 mm
     Focus Dist:  386 mm
     Aperture:    f/0.00
     IS Mode:     14
     AF Mode:     0
     Lens ID:     0x000000ED
     Flags:       0x00000000
Block: STYL
  Offset: 0x0000032c
    Size: 52
    Time: 307.551000 ms
     picStyle:   34
     contrast:   0
     sharpness:  5
     saturation: 0
     colortone:  0
Block: MARK
  Offset: 0x00000360
    Size: 20
    Time: 344.731000 ms
  Button: 0x00
Block: VIDF
  Offset: 0x00000374
    Size: 3632620
    Time: 0.000000 ms
   Frame: #0
    Crop: 18774x17988
     Pan: 28160x55
   Space: -20
[ERROR] File ends in the middle of a block
Processed 0 video frames
Done
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 16, 2014, 11:34:31 AM
yeah your file obviously is corrupted.
can you try to upload the first few megabytes of that file?
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 16, 2014, 09:31:12 PM
Well that sucks. Am I really the only one who has this problem?

How do I upload only a portion of a file?

The file is 915mb in total and I'd upload the first .dngs except as I said the mlv_dump or any other mlv program doesn't output any .dngs.
So do you want me to upload the whole thing?

Also, would it help to get the -v --dng report for more of the corrupted .MLVs?
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 16, 2014, 09:38:17 PM
well i could manually fix that problem if i can somehow can access it on your computer using an hex editor like HxD. ( http://mh-nexus.de/en/hxd/ )
maybe we could do this via remote desktop or whatever.

it seems you have used a very old version of mlv_rec.
there was one with a bug that could cause that kind of errors.
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 16, 2014, 09:42:30 PM
Do you have Screens?
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 16, 2014, 09:51:16 PM
eeeh screens?
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 17, 2014, 01:21:43 AM
http://edovia.com/screens/
Forgot that you're windows based.

You are looking to control my mac remotely, correct?
I use OS X, so it would have to be a client that is cross platform. TeamViewer?
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 17, 2014, 01:28:48 AM
yes, but if you are on a mac, you can try to dump the first few megabytes using 'dd'.

dd if=file.mlv of=partial.mlv bs=1024 count=8192
and you will get a 8 MiB file you can send me
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 17, 2014, 01:32:41 AM
"dd if=M23-0532.mlv of=partial.mlv bs=1024 count=8192
8192+0 records in
8192+0 records out
8388608 bytes transferred in 0.028051 secs (299049519 bytes/sec)"


Here's the file that I got, put it on dropbox:
https://dl.dropboxusercontent.com/u/21538108/partial.mlv


Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 17, 2014, 08:41:07 AM
Here's the whole file if it'd be interesting.

https://copy.com/igxTv3OTObV2
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 18, 2014, 01:14:51 AM
did you try the latest mlv_dump?

http://ml.g3gg0.de/modules/mlv_dump.zip/mlv_dump.zip
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 18, 2014, 07:34:47 AM
I downloaded the newest mlv_dump with your link and tried the --dng command, no luck

mlv_dump --dng m23-0532.mlv

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'm23-0532.mlv'
   - Convert to DNG frames
   - Output into 'm23-0532_frame_'
File m23-0532.mlv opened
Processing...
[ERROR] File ends in the middle of a block
Processed 0 video frames
Done
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on April 18, 2014, 10:38:21 AM
add -v
worked fine here.

maybe thats an older version. will udpate later.
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 18, 2014, 07:02:25 PM
Here's -v


http://pastebin.com/49J2xfPX


So using your mlv_dump you got to output real working .dngs?
Title: Re: 7/32 MLVs empty
Post by: Toffifee on April 26, 2014, 03:08:48 AM
Did you see anything with the new -v?

and also for clarification, did you manage to output .dngs? if so how?
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 11, 2014, 08:14:57 PM
bump
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on May 11, 2014, 11:51:04 PM
try with --fix-bug=1 and a recent mlv_dump
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 12, 2014, 08:07:21 AM
I downloaded the mlv_dump from your main MLV (RAW 2.0) thread and tried the --fix-bug=1, but it came out

mlv_dump --fix-bug=1 m23-0532.mlv
mlv_dump: unrecognized option `--fix-bug=1'
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on May 12, 2014, 12:15:10 PM
try the short version: "-F 1"
perhaps i mixed smth up, will check when at home.
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on May 12, 2014, 09:36:42 PM
yep, the long option didnt work, use -F 1
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 13, 2014, 07:39:33 AM
I tried the -f1, but it only gave me a 44 byte .wav file again.
Looks like this:

mlv_dump -f1 m23-0532.mlv

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'm23-0532.mlv'
   - Verify file structure
File m23-0532.mlv opened
Processing...
Reached end of chunk 1/1 after 726 blocks
Processed 250 video frames
Done


Also tried with 10 frames, but no change.

mlv_dump -f10 m23-0532.mlv

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'm23-0532.mlv'
   - Verify file structure
File m23-0532.mlv opened
Processing...
Reached end of chunk 1/1 after 726 blocks
Processed 250 video frames
Done
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on May 13, 2014, 08:16:15 AM
uppercase F :)
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 14, 2014, 10:43:20 PM
Uppercase F doesn't work apparently.

"
mlv_dump -F1 m23-0532.mlv
mlv_dump: invalid option -- F
"
Title: Re: 7/32 MLVs empty
Post by: Frank7D on May 14, 2014, 10:56:19 PM
Quote from: Toffifee on May 14, 2014, 10:43:20 PM
Uppercase F doesn't work apparently.

"
mlv_dump -F1 m23-0532.mlv
mlv_dump: invalid option -- F
"

I could be totally off base, but it looks like g3gg0 posted "-F 1" with a space between F and 1, but you left the space out.
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on May 14, 2014, 11:15:28 PM
are you sure you have a recent version of mlv_dump?
you should get that:


root@linux-dev:/media/sf_D_DRIVE/cygwin/home/g3gg0/EOS/unified/merge_7D/modules/mlv_rec# ./mlv_dump.linux -F 1 test.mlv
FIX BUG #1 [active]

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'test.mlv'
   - Verify file structure


please redownload the latest version (mlv_rec main post, there is a link to http://ml.g3gg0.de/modules/mlv_dump.zip/mlv_dump.zip)
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 15, 2014, 12:08:42 AM
Just redownloaded to the latest mlv_dump and retried with same results.

Am I the only one who has problems like this or do you know of anybody else in my situation?

Also, I uploaded the file to my public dropbox folder if you would like to give it a try. It's less than 1Gb, so it's not a huge file.
https://db.tt/8uIo65Uw
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 15, 2014, 12:15:38 AM
Also @Frank, I tried with and without space between the f/F and the numbers
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 15, 2014, 12:22:38 AM
Here's the codes I've tried:


Upper/ lower case
mlv_dump -f1 m23-0532.mlv
mlv_dump -F1 m23-0532.mlv

With / without space
mlv_dump -f 1 m23-0532.mlv
mlv_dump -F 1 m23-0532.mlv

And the results I've gotten in the same order:



mlv_dump -f1 m23-0532.mlv

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'm23-0532.mlv'
   - Verify file structure
File m23-0532.mlv opened
Processing...
Reached end of chunk 1/1 after 726 blocks
Processed 250 video frames
Done



mlv_dump -F1 m23-0532.mlv
mlv_dump: invalid option -- F



mlv_dump -f 1 m23-0532.mlv

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'm23-0532.mlv'
   - Verify file structure
File m23-0532.mlv opened
Processing...
Reached end of chunk 1/1 after 726 blocks
Processed 250 video frames
Done


mlv_dump -F 1 m23-0532.mlv
mlv_dump: invalid option -- F
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on May 15, 2014, 12:38:20 AM
which OS are you using?
did you install mlv_dump into PATH ?
type 'dir' or 'ls' and check the executable date/time.
if it is linux/mac, type './mlv_dump'
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 15, 2014, 02:44:38 AM
OS X 10.9 Mavericks

I just put the mlv_dump into the same folder as the file and type in terminal "cd desktop", "cd test", then "mlv_dump --dng (file name)", which worked for outputting .dngs for all the other files except these ones.

Did that answer your question?
Title: Re: 7/32 MLVs empty
Post by: g3gg0 on May 15, 2014, 09:32:13 AM
please enter ./mlv_dump not mlv_dump
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 17, 2014, 09:08:47 AM
Sorry for the late reply. Tried it, here's the result.

./mlv_dump -F1 m23-0532.mlv
-bash: ./mlv_dump: No such file or directory


./mlv_dump -f1 m23-0532.mlv
-bash: ./mlv_dump: No such file or directory
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 17, 2014, 09:10:28 AM
Did you get to take a look at the file I linked to btw?
and if so, did you have any luck with it?
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 17, 2014, 10:32:12 AM
Good news! Just tried out Thomas Worth's new RAWMagic app and it converted the file just fine  :D :D :D
Only problem is cr2hdr doesn't recognise the .DNGs as Dual ISO, so when processing the files, it just deletes all of them.
Title: Re: 7/32 MLVs empty
Post by: a1ex on May 17, 2014, 11:33:16 AM
Quote from: Toffifee on May 17, 2014, 10:32:12 AM
Only problem is cr2hdr doesn't recognise the .DNGs as Dual ISO, so when processing the files, it just deletes all of them.

Steps to reproduce? If the files are not recognized, they should be left untouched, not deleted or overwritten.
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 17, 2014, 07:33:18 PM
This is the result I get when trying to put the .dng into the app.
I'm using the 2.0 beta, but I will try later and see if there's an update and try with that.

cr2hdr: a post processing tool for Dual ISO images Last update: 2f4e2df on 2013-12-02 17:45:36 UTC by alex: cr2hdr: fix DNG compression under Windows Active options: --amaze-edge : use a temporary demosaic step (AMaZE) followed by edge-directed interpolation (default) --cs2x2 : apply 2x2 chroma smoothing in noisy and aliased areas (default) Input file : M15-1408_C0000_00002.CR2 Camera : Canon EOS 5D Mark III Full size : 1920 x 1080 Active area : 1920 x 1080 Doesn't look like interlaced ISO


I uploaded a single .dng if you would like to look at it. I copied and backed up the file and then threw it into cr2hdr to make sure that it also is being deleted, as was the result.

Here is the link to the 1 .dng
M15-1408_C0000_00003.dng
Title: Re: 7/32 MLVs empty
Post by: a1ex on May 18, 2014, 10:17:02 AM
Quote from: Toffifee on May 17, 2014, 07:33:18 PM
This is the result I get when trying to put the .dng into the app.
I'm using the 2.0 beta

I did not label any of them "2.0 beta" and it's not an app, but a command-line tool. If you are using a GUI frontend, you should report it to the maintainer of that GUI.
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 18, 2014, 11:33:58 AM
I do not know who runs the frontend GUI, it's the one you have linked to in your 1st post on the main Dual ISO thread. Number 4 out of the 5 GUI frontends "Mac GUI for cr2hdr: OSX_cr2hdr_app.zip and OSX_cr2hdr.zip".

Do you know who runs this app and where I could notify them?
Title: Re: 7/32 MLVs empty
Post by: a1ex on May 19, 2014, 11:29:36 PM
a.d. on this forum, and a_d_ on bitbucket ( https://bitbucket.org/a_d_/magic-lantern/downloads/ )

He didn't post much about this GUI though, and wasn't very active lately, so I'd say it's best to ask him in a PM.

The code is in the zip file, and looks simple enough. I suggest changing it so that it only removes the CR2 if a corresponding DNG already exists, then ping a.d. to update the zip with your change.

edit: the relevant discussion about this GUI seems to start here: http://www.magiclantern.fm/forum/index.php?topic=7139.msg78966#msg78966
Title: Re: 7/32 MLVs empty
Post by: Toffifee on May 20, 2014, 06:23:13 PM
Ok, thanks!
I've notified him about the issue, so we'll see if he gets around to it.