Silent picture capture with raw video format

[brapodam]

Oh i see.Yes for the 60D i used the July 22 nghtly with the silent.mo you posted on post #153.So its ok to continue with 60D and July 22 nighly if i understand correctly?
I actually did the same with my 550D but with the silent.mo in mk11174 put in  latest 60D build only with full res silent pics (no mlv).
Should i stop using that as well?
I have no means to compile the build for the 60D you say though,if someone has already done that and wouldn't mind posting a link to it
i would continue shooting further tests or any kind of test you or anyone else would like for me to do.

I've used a self-compiled build from the main brach with dmilligan's silent.mo on my 60D without any problems, but I've compiled a build from his branch, and here's the link:
https://www.dropbox.com/s/yhevc9pkdkhuvsp/ML_60D.zip

Interestingly enough, I've had a weird bug with the build from his branch, but I think it's unrelated. After shooting a short test "time lapse", my camera refuses to take pictures, and the live view button has decided it would function as a AF mode toggle instead. A battery pull fixed the problem though.

[gcrook]

I've used a self-compiled build from the main brach with dmilligan's silent.mo on my 60D without any problems, but I've compiled a build from his branch, and here's the link:
https://www.dropbox.com/s/yhevc9pkdkhuvsp/ML_60D.zip

Interestingly enough, I've had a weird bug with the build from his branch, but I think it's unrelated. After shooting a short test "time lapse", my camera refuses to take pictures, and the live view button has decided it would function as a AF mode toggle instead. A battery pull fixed the problem though.

Thank you i will test this.
Cheers.

[dmilligan]

Went back and tested with your module saving in plain dng and the framing/centering are correct.  Switch to mlv and I get black borders.  Have two files if you want a look.

thanks, I may have left out some field in an mlv chunk header, I'll have a look.

[dubarry]

Just wanted to add:  I was getting err 70 & raw file error message because I had raw video enabled (on) in the ML menu.  Once I turned this off everything worked as previously mentioned.

[josepvm]

@dmilligan

done: https://bitbucket.org/dmilligan/magic-lantern/commits/dcc3c2f8304529ec0392f2f73f8b3a77872b31b6

I have tried to build the full res silent pics branch for 500D, replacing "silent.c" with your modified version, and building for the silent.mo fails:

Code: [Select]

Building module silent...
REBUILDING
[ README   ]   module_strings.h
[ CC       ]   silent.o
[ MODULE   ]   silent.mo
[ STRIP    ]   silent.mo
[ EXPORTS  ]   silent.sym
[ DEPENDS  ]   silent.dep
Will NOT load on:
    500D (mlv_fill_expo, mlv_fill_rtci, mlv_generate_guid, and 7 others)
Not checked (compile ML for these cameras first):
    1100D, 50D, 550D, 5D2, 5D3, 600D, 60D, 650D, 6D, 700D, 7D, EOSM
make[3]: *** [silent.dep] Error 1

********************************************************
WARNING: module silent failed to build, deleting
********************************************************


If I build the branch with the unmodified "silent.c", the module builds Ok, and full res silent pics work in camera.

Any thoughts, what could be the problem?

You need an autoexec.bin compiled from the full-res silent pic branch with the correct EDMAC channels and skip offsets for your camera.

I think this is already done for 500D:

https://bitbucket.org/hudson/magic-lantern/pull-request/530/500d-switched-raw_photo_edmac-to-the-one/diff

[dmilligan]

you need to compile mlv_rec

I need to make all the references to mlv_rec functions WEAK_FUNC so that the module will still work without mlv_rec (it just won't be able to save mlv)

[josepvm]

you need to compile mlv_rec

I have compiled all modules at the same time, I think, as I usually do:

magic-lantern$ make 500D
magic-lantern$ cd modules
magic-lantern/modules$ make
magic-lantern/modules$ cd ..
magic-lantern$ cd platform/500D.111
magic-lantern/platform/500D.111$ make
magic-lantern/platform/500D.111$ make install

 

When I build the unmodified fullres-silent-pics branch, and test it on camera, mlv_rec module loads without problems, and mlv video recording works. (with a maximum resolution of 860x486. due to SD bus bandwith limited to 21 MB/s)

[garry23]

Latest:

I have reloaded the 5DIII-123 (full) build from this thread, ie replaced all the other ML files, and downloaded the silent.mo from this thread that does MLV.

Strangely I still have the left & top black 'bars'/offsets.

Can someone see what am I doing wrong?

[Greg]

@josepvm

You need to compile mlv_rec before silent.


MLV full res not remove the black borders:

[josepvm]

@josepvm

You need to compile mlv_rec before silent.

Ok, thanks.

I have edited "Makefile.modules.default" and seen that "silent" was placed before mlv_rec.

Moving "silent" to the end of the line, the build works.

And the resulting module works in the camera. Saving many silent pics in a single .mlv file is a very useful feature, and works nicely.   I do not see any black borders.

[garry23]

I have spent hours trying every permutation of downloads and settings (on my 5DIII-123), but still have the dreaded off centre black bars.

I used the silent.mo from here: https://bitbucket.org/hudson/magic-lantern/downloads/silent.mo

And from here: https://bitbucket.org/mk11174/magic-lantern/downloads/magiclantern-Nightly.2014Jul18.5D3123.zip

Would one kind person using FF silent on a 5DIII-123 and not getting the black bars point me in the right direction.

As I say, it all looks OK other than the bars.

Cheers

Garry

[dmilligan]

for those getting black bars, please post the output of mlv_dump -v
you should see something like this:

Code: [Select]

File Header (MLVI)
    Size        : 0x00000034
    Ver         : v2.0
    GUID        : 17136314949046505859
    FPS         : 1.000000
    File        : 0 / 0
    Frames Video: 1
    Frames Audio: 0
Block: RAWI
  Offset: 0x00000034
    Size: 180
    Time: 9.281000 ms
    Res:  5344x3516
    raw_info:
      api_version      0x00000001
      height           3516
      width            5344
      pitch            9352
      frame_size       0x01F5BBE0
      bits_per_pixel   14
      black_level      2045
      white_level      14803
      active_area.y1   50
      active_area.x1   142
      active_area.y2   3516
      active_area.x2   5344
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1

Notice the OB areas are accounted for in the 'active_area' settings, 50 from the top, 142 from the left (on my 60D). When I use mlv_dump to convert this to DNG, I get a correct DNG with the OB removed (no black bars). So, if you have black bars it means one of two things:
1) your MLV converter program of choice is not respecting the active_area of the RAWI chunk and outputing the OB areas in the DNG,
2) the OB areas are coming in wrong from the ML raw backend and need to be re-calibrated
(the other possibility is that I was not writing them into the MLV file correctly, but that doesn't seem to be the case, I've checked)

You'll know it's #1 if you look at mlv_dump output and 'active_area' has some values that don't match the DNG you're getting (e.g. DNG width should be active_area.X2 - active_area.X1). You'll know it's #2 if the converted DNG appears to match 'active_area' but there are still black bars.

If you're using something besides mlv_dump then I'd say #1 is the most likely. Most normal raw videos don't have OB areas, so some converters might be ignoring the 'active_area' settings (you should submit a bug report to the author of the converter).

[sdesign]

im running the july 18 build on the 5d3 123 and keep getting Raw Error, no file saved, can somebody helm me?

[dubarry]

hey sdesign is you raw video set to 'off' ?

[josepvm]

for those getting black bars, please post the output of mlv_dump -v
...

When I use mlv_dump to convert this to DNG, I get a correct DNG with the OB removed (no black bars).

Me too, as I posted earlier, I do not see any black borders in my 500D tests. Intervalometer saving in mlv file, and using mlv_dump  (mlv_dump --dng *.MLV) to extract the DNGs.

The output for mlv_dump -v is very similar to the one you have posted. Here is the output:

Code: [Select]

File Header (MLVI)
    Size        : 0x00000034
    Ver         : v2.0
    GUID        : 5214970244131067897
    FPS         : 0.066667
    File        : 0 / 0
    Frames Video: 6
    Frames Audio: 0
Block: RAWI
  Offset: 0x00000034
    Size: 180
    Time: 23.117000 ms
    Res:  4832x3201
    raw_info:
      api_version      0x00000001
      height           3201
      width            4832
      pitch            8456
      frame_size       0x019D0508
      bits_per_pixel   14
      black_level      1023
      white_level      12932
      active_area.y1   24
      active_area.x1   62
      active_area.y2   3201
      active_area.x2   4832
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1



The size obtained substracting "active_area" values here, are:

active_area.x2 - active_area.x1  => 4770
active_area.y2 - active_area.y1  => 3177

Opening the DNGs with Darktable, the size of the images I get is exactly this value, 4770x3177
And these are the same values reported by ML on the camera screen when saving every fullres silent pic.


But Rawtherapee reports a sligthly smaller size for the images, 4762x3169, I don't know why 

[TrueIndigo]

It's just Raw Therapy - from my limited experiments with it last year I found that it converts raw images a few pixels less than expected; whereas the same files brought into Adobe Camera Raw produce image frame sizes exactly as expected. Strange.

[Greg]

with mlv_dump work well, thanks! 
before use mlv2dng   


When use the intervalometer -> Take pics -> like crazy
Each picture is a different file MLV

When use the intervalometer -> Take a pic every -> 1s
Each photo is in the same file MLV (work well)

[barepixels]

Last night I asked josepvm to compile a fresh build for my 5D2.  He did it by using build from the current "fullres-silent-pics" branch including the modified "silent.c" from dmilligan, for saving silent pics in mlv or raw files.

With his version I was able to create this MLV

https://dl.dropboxusercontent.com/u/63842491/00000001.MLV  (188mb 5 pics)

on Windows 7 64  I was able to extract the 5 dngs using either

a) MlRawViewer_1_1_7
b) mlv2dng.exe   found in MLV Converter 1.9.2/Tools/mlv2dng.exe

the DNGs from the above shows black bars

-------------------------------------------------------------
-------------------------------------------------------------

josepvm asked me to send him the https://dl.dropboxusercontent.com/u/63842491/00000001.MLV  so he can "try to process it with mlv_dump, to see if this way the black borders are seen also."

here is his reply

I have processed the MLV, got 5 DNG images, and they do not show the black borders seen in your DNG.

This is one of the images I have processed from th mlv file:

https://www.dropbox.com/s/cdlu54kbtrlorub/00000001_frame_000000.dng


But there is an issue. When processing your mlv with mlv_dump I got this error:

Code: [Select]

mlv_dump --dng *.MLV

 MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: '00000001.MLV'
   - Convert to DNG frames
   - Output into '00000001_frame_'
File 00000001.MLV opened
File 00000001.M00 not existing.
Processing...

Vertical stripes correction:
  1.000  1.000  1.006  1.007  1.002  1.002  1.006  1.004
Reached end of chunk 1/1 after 37 blocks
Processed 5 video frames
*** Error in `mlv_dump': munmap_chunk(): invalid pointer: 0xf55ad008 ***

Aborting ( kernel image dump)

The 5 files have been already extracted before the error occurs, and they look all fine. But I do not see this errors when processing my own mlv files. The output I get is:

Code: [Select]

mlv_dump --dng *.MLV

 MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: '00000000.MLV'
   - Convert to DNG frames
   - Output into '00000000_frame_'
File 00000000.MLV opened
File 00000000.M00 not existing.
Processing...
Reached end of chunk 1/1 after 30 blocks
Processed 4 video frames
Done

josepvm explain that his copy of mlv_dump is build by himself from source, ML unified branch.

But is a Linux version, he use Linux exclusively.  It will not work in Windows.

-------------------------------------------------------------
-------------------------------------------------------------

When I try to use the mlv_dump that came with MLV Converter 1.9.2 for windows  OR from
http://www.magiclantern.fm/forum/index.php?topic=7122.0  this is what I get

Code: [Select]

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'D:\2014-07-23-Sillence-Full-MLV\00000001.MLV'
   - Convert to DNG frames
   - Output into 'D:\2014-07-23-Sillence-Full-MLV\00000001_frame_'
File D:\2014-07-23-Sillence-Full-MLV\00000001.MLV opened
File D:\2014-07-23-Sillence-Full-MLV\00000001.M00 not existing.
Processing...
[b][ERROR] File ends in the middle of a block[/b]
Processed 0 video frames
Done

-------------------------------------------------------------
-------------------------------------------------------------

Am asking other developer to look at my
https://dl.dropboxusercontent.com/u/63842491/00000001.MLV  (188mb 5 pics)

and see if you can extract DNGs without black bar like josepvm did with his Linux version of mlv_dump
Please state which OS and software you used.  Thnx

[josepvm]

@ barepixels

I think there is something wrong in your MLV file.

I succedeed to extract the DNGs with my linux "mlv_dump" build, but getting an error, as you have already posted.

Now I've tried to process your mlv file again with the windows version found here:

http://ml.g3gg0.de/modules/mlv_dump.zip/mlv_dump.zip

I've used it from linux, with Wine emulator, and I get the same error you have found, I can't extract the DNGs:

Code: [Select]

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: '00000001.MLV'
   - Convert to DNG frames
   - Output into '00000001_frame_'
File 00000001.MLV opened
File 00000001.M00 not existing.
Processing...
[ERROR] File ends in the middle of a block
Processed 0 video frames
Done



The same version of mlv_dump.exe for Windows works fine with my own mlv files  (on linux using wine, but should be the same in Windows)

[Attero]

HHmm... i tryed to use mlv_dump to extract my own .mlv . It worked, he sad "Processed 3 video frames" but he did not created anything. What did i wrong?
If i extract my .mlv with mlv2dng i got the black borders...

When i change the format inside the Camera to .DNG i got nice Images without black bars.
Using the 600D with the Nightlybuild from 19July together with the silent.mo from dmilligan.

[barepixels]

Attero, what camera model do you have?  what OS do you use?  If your MLV is small enough, maybe you can put it on dropbox and someone who using Linux can try to mlv_dump and see if black border goes away

[Attero]

Windows 7 64bit, Camera like i wrote in the last Post.

This .MLV contains just 1 image. I have a very slow internet conection so the less MB the faster it goes
https://www.dropbox.com/s/97ki94gildw70cx/37830000.mlv

Tell me if u can extract the .DNG without black bars.

[barepixels]

Just an update everyone.

I asked josepvm to process an older mlv file that were made with mk11174  build  (see post 17 on first page) and guess what

josepvm's linux version of mlv_dump produce DNGs with no black border

Here is what Josep wrote

Done.

Exactly the same behaviour.

With my Linux build of mlv_dump I can extract the 5 images and they look fine, no black borders. But I get the same error message:

Code: [Select]


Processing...

Vertical stripes correction:
  1.000  1.000  1.008  1.010  1.000  1.003  1.005  1.005
Reached end of chunk 1/1 after 37 blocks
Processed 5 video frames
*** Error in `mlv_dump': munmap_chunk(): invalid pointer: 0xf5530008 ***

With the Windows version of mlv_dump, same behaviour also as with previous mlv file. I can't extract the images:

Code: [Select]


File 00000004.MLV opened
File 00000004.M00 not existing.
Processing...
[ERROR] File ends in the middle of a block
Processed 0 video frames
Done


[josepvm]

This is the result, from the second mlv file I have processed for barepixels.

Using mlv_dump, Linux build.  No black borders:

https://www.dropbox.com/s/i7l5ps0hy22pumk/00000004_frame_000000.jpg



And @barepixels:  my name is "Josep".    Is a Catalan name. A little longer than the Spanish "José", but a little shorter than the English "Joseph" 

[josepvm]

Attero's file is correctly processed by mlv_dump for Linux.

No errors, no black borders in the resulting DNG:

https://www.dropbox.com/s/tw8yhg4u16687o9/37830000_frame_000000.jpg

But this one can be processed also with mlv_dump.exe through wine, no errors, same results.