The first MLV chunk convert in post not the others filming longer sequences. Used MLP(mlv_dump), worked splendidly(the first MLV chunk).Yeah, I was worried chunk splitting would have issues. Didn't have time to test it (takes too long to record a sample that hits the 4GB barrier). Can you send the first ~20MB of the split file or post a screenshot of a hexdump of the start of the file?
Same goes for MLVFS, only first chunk of files pops up, M00 etc doesn,t.
Would be nice to have it behave like FULL-MLV.Well that's sort of missing the point. If you want something that behaves like the full MLV, then use the full MLV. This should behave exactly like raw_rec except give you MLV files. I don't have any plans to modify the feature set of raw_rec, that's outside the scope of this effort. I just want to switch out the format and make sure everything works and achieves the same level of performance as the old raw_rec. If we can achieve that goal and verify it with good test reports, then we can be done with the old raw format once and for all.
mlv_dump -o output.M00 -f 20 input.M00Or renamed to MLV extension gives a segmentation fault 11 but here,s a screen dump. Tell me if you need anything else.3. What happens when card runs out of space?
The first MLV chunk convert in post not the others filming longer sequences. Used MLP(mlv_dump), worked splendidly(the first MLV chunk).I think I fixed the issue. Binary updated, please retry.
Same goes for MLVFS, only first chunk of files pops up, M00 etc doesn,t.
Seems the mlv.dll with Pismo was the only app producing pink(wrong black level I think) frame from 3xCrop mode.Pismo is obsolete: http://www.magiclantern.fm/forum/index.php?topic=13152.msg161902#msg161902
Maybe the mlv.dll needs up dating , not sure.
When I first tried to load that module , I had problems here is the Log files it generated ASSERT00-18.zip (https://www.dropbox.com/s/e41xjuojydoygo6/ASSERT00-18.zip?dl=0) . It keep generating files until I reboot the Cam. then it was okThat error is actually coming from mlv_rec. You probably had both loaded, which is causing some sort of conflict.
- fileCount field in the header may get set to the number of total chunks in this recordingAnd files like this process just fine for me in mlv_dump and MLVFS.
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: '/Users/Daniel/Desktop/mlv/regular mlv/M17-0810.MLV'
- Convert to DNG frames
- Output into 'hej'
File /Users/Daniel/Desktop/mlv/regular mlv/M17-0810.MLV opened
File /Users/Daniel/Desktop/mlv/regular mlv/M17-0810.M00 opened
File /Users/Daniel/Desktop/mlv/regular mlv/M17-0810.M01 opened
File /Users/Daniel/Desktop/mlv/regular mlv/M17-0810.M02 not existing.
Processing...
Vertical stripes correction:
1.000 1.000 0.995 1.021 0.997 0.997 0.995 1.004
Reached end of chunk 1/3 after 2764 blocks
Reached end of chunk 2/3 after 2687 blocks
Reached end of chunk 3/3 after 1 blocks
Processed 2683 video frames
Done
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: '/Users/Daniel/Desktop/mlv/mlv_lite not working/M17-0807.MLV'
- Convert to DNG frames
- Output into 'hej'
File /Users/Daniel/Desktop/mlv/mlv_lite not working/M17-0807.MLV opened
File /Users/Daniel/Desktop/mlv/mlv_lite not working/M17-0807.M00 opened
File /Users/Daniel/Desktop/mlv/mlv_lite not working/M17-0807.M01 opened
File /Users/Daniel/Desktop/mlv/mlv_lite not working/M17-0807.M02 not existing.
Processing...
Vertical stripes correction:
1.000 1.000 0.995 1.016 0.999 0.997 0.998 1.003
[ERROR] VIDF: File ends in the middle of a block
Processed 1182 video frames
DoneIf you rename each .MXX chunk to .MLV will they process by themselves?Yes, working.
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: '/Users/Daniel/Desktop/mlv/mlv_lite not working/M17-0807.MLV'
- Verbose messages
- Convert to DNG frames
- Output into 'hej'
File /Users/Daniel/Desktop/mlv/mlv_lite not working/M17-0807.MLV opened
File /Users/Daniel/Desktop/mlv/mlv_lite not working/M17-0807.M00 opened
File /Users/Daniel/Desktop/mlv/mlv_lite not working/M17-0807.M01 opened
File /Users/Daniel/Desktop/mlv/mlv_lite not working/M17-0807.M02 not existing.
Processing...
File Header (MLVI)
Size : 0x00000034
Ver : v2.0
GUID : 7343463891422747826
FPS : 25.000000
File : 0 / 0
Frames Video: 0
Frames Audio: 0
Block: RAWI
Offset: 0x00000034
Size: 180
Time: 0.768000 ms
Res: 1920x1080
raw_info:
api_version 0x00000001Block: VIDF
Offset: 0xfff7221c
Size: 3633152
Time: 48853.163000 ms
Frame: #1182
Crop: 152x132
Pan: 146x133
Space: 4064
[ERROR] VIDF: File ends in the middle of a block
Processed 1182 video frames
Done[ERROR] VIDF: File ends in the middle of a blockSeems to be what is happening here.5) Via the Dokan v1rc mounts I can successfully import the DNG's into my Photo and video editing applications. A quick visual scan of the individual frames didn't spot
any bad or pink frames.
Interesting. So either solution then?I think the solution is probably 1 AND 2.
1 - reliable NULL clean up
2 - alter mlv_dump warning routine and keep on processing
3 - renaming routine which will process renamed spanned files with extension MLV(easily done with MLP)
Question. In MLVFS you managed to disregard this vidf issue since the show up fine in there but gives the problem with mlv_dump?The reason MLVFS works is that it is simply not as strict about validating all the data before trying to use it, and because the frame exists in the next chunk, it ends up never trying to load the VIDF chunk that is cut off. If you delete the .IDX and remove the .M00, M01, etc. leaving only the MLV, I bet MLVFS will either: crash, give you an error, or show part of the last frame as garbage.
| RAW Format | Mode (value that appears most often in a set of data) |
| MLV Lite | 63 frames |
| MLV | 66 frames |
| Raw | 71 frames |
Unique Camera Model : Canon EOS REBEL T5i
Make : Canon
Camera Model Name : Canon EOS REBEL T5i
Unique Camera Model : Canon EOS 650D
Make : Canon
Camera Model Name : Canikon
Unique Camera Model : Canon EOS M
Make : Canon
Camera Model Name : Canikon
Unique Camera Model : Canikonmlv_lite: also update the skip offsets every frame
Does that mean focus pixel fixing when using dynamic panning in 5x zoom mode is supported? I haven't run any tests on that yet.No. Not if the raw buffer itself moves. Only if the skip is moved around. There isn't anywhere stored what part of the sensor is being recorded when in zoom mode, and AFAIK, currently we can only approximately guess where it is anyway.
make 2000x1080 for 50D, its work in last Tragic Lantern :-\
It needs to be a multiple of 1632 ;)
sorry i got a noob question - other than the early bugs for now, are there other disadvantages to MLV Lite files vs full RAW files?
1. Metadata is only for what the settings were when recording was started (e.g. you get expo metadata, but if you change exposure, you don't get that new information).
2. No audio
3. No card spanning (recording to both CF and SD card)
Actually produced a lot of corrupt frames when using global draw and focus peaking. Global draw off yields no corrupted frames, everything works ok.Yes that been noted before with the 7D & 5D2 back a few years ago now.
70D?
Corrupt frame? Do you have the actual mlv file or a sample dng?
Thanks. Seems corrupted. Preview seems fine though. Only one bad fram out of 174 mlv files shot in 50fps? That seems to me like mlv_lite is a trustworthy format.
And what's wrong with figuring out why the files don't open in MlRawViewer?I keep recommending MSVFS to users that keep popping on the MLRAWViewer thread with issues, since it's not developed anymore maybe MLVFS already has their use case covered.
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: 'M20-1519.MLV'
- Verbose messages
- Verify file structure
- Dump all block information
File M20-1519.MLV opened
File M20-1519.M00 not existing.
Processing...
File Header (MLVI)
Size : 0x00000034
Ver : v2.0
GUID : 10587574007593374301
FPS : 29.970000
File : 0 / 0
Frames Video: 0
Frames Audio: 0
Block: RAWI
Offset: 0x00000034
Size: 180
Time: 0.791000 ms
Res: 1920x1080
raw_info:
api_version 0x00000001
height 1318
width 2080
pitch 3640
frame_size 0x00493450
bits_per_pixel 14
black_level 2046
white_level 15000
active_area.y1 28
active_area.x1 146
active_area.y2 1318
active_area.x2 2078
exposure_bias 0, 0
cfa_pattern 0x02010100
calibration_ill 1
Block: IDNT
Offset: 0x000000e8
Size: 84
Time: 0.804000 ms
Camera Name: 'Canon EOS 5D Mark III'
Camera Serial: 'B052G52X3'
Camera Model: 0x80000285
Block: EXPO
Offset: 0x0000013c
Size: 40
Time: 0.835000 ms
ISO Mode: 0
ISO: 800
ISO Analog: 96
ISO DGain: 0/1024 EV
Shutter: 29994 microseconds (1/33.34)
Block: LENS
Offset: 0x00000164
Size: 96
Time: 1.190000 ms
Name: 'EF-S24mm f/2.8 STM'
Serial: ''
Focal Len: 24 mm
Focus Dist: 65535 mm
Aperture: f/4.00
IS Mode: 0
AF Mode: 3
Lens ID: 0x0000003A
Flags: 0x00000000
Block: RTCI
Offset: 0x000001c4
Size: 44
Time: 1.207000 ms
Date: 20.04.2016
Time: 15:19:25 (GMT+0)
Zone: ''
Day of week: 3
Day of year: 110
Daylight s.: 0
Block: WBAL
Offset: 0x000001f0
Size: 44
Time: 4.029000 ms
Mode: 9
Kelvin: 6400
Gain R: 1024
Gain G: 1024
Gain B: 1024
Shift GM: 0
Shift BA: 0
Block: NULL
Offset: 0x0000021c
Size: 484
Time: 47594302144599.398438 ms
Block: VIDF
Offset: 0x00000400
Size: 3629056
Time: 1471.736000 ms
Frame: #0000
Crop: 152x132
Pan: 146x133
Space: 32
Block: VIDF
Offset: 0x00376400
Size: 3629056
Time: 1505.470000 ms
Frame: #0001
Crop: 152x132
Pan: 146x133
Space: 32
Block: VIDF
Offset: 0x006ec400
Size: 3629056
Time: 1538.806000 ms
Frame: #0002
Crop: 152x132
Pan: 146x133
Space: 32
[...]
Block: VIDF
Offset: 0xffad4400
Size: 3629056
Time: 40911.497000 ms
Frame: #1182
Crop: 152x132
Pan: 146x133
Space: 32
Block: VIDF
Offset: 0xffe4a400
Size: 3629056
Time: 40944.810000 ms
Frame: #1183
Crop: 152x132
Pan: 146x133
Space: 32
Block: [unreadable characters]
Offset: 0xffe4a420
Size: 32
Time: 4504149383184.391602 ms
Unknown Block: , skipping
[ERROR] Invalid block size at position 0xffe4a440
Processed 1184 video frames
Done
A:/DCIM/100EOS5D/M20-1512.MLV: 602 frames, 2.034653 GB
A:/DCIM/100EOS5D/M20-1513.MLV: 911 frames, 3.079018 GB
A:/DCIM/100EOS5D/M20-1514.MLV: 909 frames, 3.072258 GB
A:/DCIM/100EOS5D/M20-1515.MLV: 1160 frames, 3.920594 GB
A:/DCIM/100EOS5D/M20-1516.M00: 1257 frames, 4.260247 GB -- note: it prints the last chunk and the total size
A:/DCIM/100EOS5D/M20-1517.M00: 1442 frames, 4.873704 GB
A:/DCIM/100EOS5D/M20-1518.MLV: 744 frames, 2.514588 GB
And what's wrong with figuring out why the files don't open in MlRawViewer? I've tried both apps, and it doesn't look like MLVFS is a complete replacement for MlRawViewer.
The point of MLV Lite is not to break compatibility with existing apps. I don't know yet who is to blame - if it's MlRawViewer, we should fix that (hint: it's open source); if it's MLV Lite creating slightly incorrect files (not fully compliant with the spec), we should fix that.
Workaround: use MLVFS and MLRawViewer together (MLRawViewer is capable of playing back DNG files).
@DeafEye, I sent you a PM. Did you get it? Thanks
and what's wrong with MLVFS + DR12 for quick previews @platu?
Workaround: use MLVFS and MLRawViewer together (MLRawViewer is capable of playing back DNG files).
Could it be as trivial as mlv_lite doesn,t report amount of frames?
Processing...
[ERROR] Error: Frame sizes of footage and subtract frame differ (1494976, 1494720)Processed 0 video frames
Donemlv_dump -o a_M20-1342.MLV -s avg_EOSM_res_1728x692_iso_400_fps_24.002000.MLV M20-1342.MLVIt is black until you press play, then the movie will preview in camera.Yes, I knew that. I usually shoot hundreds videos files... all will have a black previews. This is not usable.
all will have a black previews. This is not usable.
Are you talking about previewing in the camera in the Canon "Play" mode?Yes. All thumbnails are black... This is problem.
ML ASSERT:
slots[slot_index].size < max_frame_size
at mlv_lite.c:3554 (raw_video_rec_task), task raw_rec_task
lv:0 mode:3
Block: AUDF
Offset: 0x087bad00
Number: 112
Size: 38656
Time: 28.069000 ms
Frame: #0000
Space: 232
Block: AUDF
Offset: 0x087c4400
Number: 113
Size: 38656
Time: 228.578000 ms
Frame: #0001
Space: 232
Block: AUDF
Offset: 0x087cdb00
Number: 114
Size: 38656
Time: 429.834000 ms
Frame: #0002
Space: 232
It works on the 100D :). And with latest branch of MlRawViewer you get audio and preview working with lossless footage:So now, no need for the H264 proxy method + switch for audio with lossless footage ? In any case, proxy H264 are still usefull for preview and direct offline/online.
http://www.magiclantern.fm/forum/index.php?topic=9560.msg191988#msg191988
Well as you say, proxy is still useful. Tested a few lossless 12bit files and worked fine.Danne, you mean lossless with sound ?
Not me IDA_ML, it´s all @ErwinH work:
Regarding the preview image, isn't it an option to sync based on black frames at the end of the recording? Might take a bit more time in post processing, but at least you get to keep the preview images.Beginning and end of the file. Start H.264, then RAW. When raw starts the H.264 blackframes stops. At the end raw stops then H.264. Blackframes starts again after raw stops and then H.264 stops. This is my understanding.
Out of four days of shooting that amounted to a few hundred clips and several hours (estimated about 300 shots, 4 hours) there were only two clips where the length of the raw footage didn't match up with the trimmed H.264 proxy. Being the perfectionist that he is Danne tried his best but the issue isn't with Switch--it turned out that sometimes the H.264 recording stops before the MLV. One short clip was off by 1 frame and the other, an 8 minute shot that spanned two H.264 files was off by 2 frames.
1. I only tested with 12bit Loss Less, with MLV Sound working, continuous recording
4. I also tried some crop mode recording in 1920x1080 @ 48p and 1920x800 @60p , and it works with sound as well. However, since I run it with realtime color LiveView, I get dropped frames after some time. I noticed when editing those clips that stopped due to dropped frames, the sound cuts out a couple seconds before the the video cuts out. I dunno if that could be related to the "audio failed to stop" message or not.
At what resolution did you test the 12bit Loss Less, with MLV Sound? Did you also try the Crop mode at 3K and 24fps? Could you also check the sound recording in the 14/12/10 bit uncompressed modes in the MLV-recording module?
I will ask my friend to repeat the tests on his 5D3, maybe with a faster CF card and see if he can get rid of the "sound failed to stop" message.
Thanks for your important feedback, JCut !
Can you try also h264 proxy recording on canon 5dmk3?
I did get some sound issue with that. ( static noise )
Thanks JCut,
I will send him your settings. The problem could be that he is using a slower card and loading to many modules on it. Did you test synchronisity between sound and video already? That would be helpful too.
static void mlv_snd_stop()
{
trace_write(trace_ctx, "mlv_snd_stop: stopping worker and audio");
mlv_snd_state = MLV_SND_STATE_SOUND_STOPPING;
/* wait until audio and task stopped */
uint32_t loops = 100;
while((mlv_snd_state != MLV_SND_STATE_SOUND_STOPPED) && (--loops > 0))
{
msleep(20);
}
if(mlv_snd_state != MLV_SND_STATE_SOUND_STOPPED)
{
bmp_printf(FONT(FONT_MED, COLOR_RED, COLOR_BLACK), 10, 130, "audio failed to stop, state %d", mlv_snd_state);
trace_write(trace_ctx, "mlv_snd_stop: failed to stop audio (state %d)", mlv_snd_state);
beep();
} /* some models may need this */
StopASIFDMAADC();
// SoundDevShutDownIn(); /* no model seems to need this */
audio_configure(1);That StopASIFDMAADC / SoundDevShutDownIn issue was resolved in the crop_rec_4k branch but the unified branch still has a hack where SoundDevShutDownIn is using the address for StopASIFDMAADC on some cameras. In order for audio recording to work on H.264 and MLV Lite simultaneously will probably require re-working mlv_snd and possibly requiring the SoundDevShutDownIn stub.Oh, I forgot about the black frames so you're totally right. One thing might be to have this proxy mode+snd available for all cameras that could handle it with mlv without sound (5d2, 7D, 700d, 6D, and 100D). I mean, when crop_rec_4k would work on both of them.
Recording sound in both the proxy and raw file may seem like a good thing in theory, in practice it doesn't really have much advantages. The black frames need to be trimmed off of the H.264 proxies in order for them to match the raw files. That step is necessary before editing starts or you will have issues later on. Switch can do this very quickly in the field as you're copying the files off the cards. Most people are probably processing their MLV files after the shoot so it should be no problem having Switch sync the audio automatically.
Speaking of theory and practice, it shouldn't be necessary to have the audio with the DNG files because you only need to conform the picture for color grading.
Most people are probably processing their MLV files after the shoot so it should be no problem having Switch sync the audio automatically. Switch is prepared for the day that mlv_lite has audio. It will simply not extract audio into the dng folder when a wav file already is in there. I'm getting glitches :(
You mean the 10bit/12bit build?
Delete the ML settings and try from a fresh start. On both the 7D and 500D I found that it works the first time but you get "earthquake" video on restart.
Hey, this is highly experimental stuff. Help figure it out.