Silent picture capture with raw video format

Started by dmilligan, July 20, 2014, 01:48:25 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

MA Visuals

Tested (5d3) the updated mlv_dump on a much larger spanned set of files and all works well and no black borders.   Thank you.

josepvm

Has anyone succeeded to get a fullres silent timelapse in MLV format using ETTR ?

If I set ETTR as "always on", apparently the capture works, the ETTR autoadjusts the exposure to changes in scene lighting... but I cannot process the resulting MLV files.

With mlv_dump I can extract only one frame, and it is completely black.   And I get an error:


Vertical stripes correction:
  1.000  1.000    1      1      1      1      1      1 
[ERROR] Invalid block size at position 0x019d06c4
Processed 1 video frames
Done


But "mlv_dump -v " says that all the frames are there.

This is the complete output of "mlv_dump -v --dng *.MLV" for a file with 101 pics:


MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: '00000000.MLV'
   - Verbose messages
   - Convert to DNG frames
   - Output into '00000000_frame_'
File 00000000.MLV opened
File 00000000.M00 not existing.
Processing...
File Header (MLVI)
    Size        : 0x00000034
    Ver         : v2.0
    GUID        : 10537559721678771798
    FPS         : 1.000000
    File        : 0 / 0
    Frames Video: 101
    Frames Audio: 0
Block: RTCI
  Offset: 0x00000034
    Size: 44
    Time: 843519.905000 ms
     Date:        25.07.2014
     Time:        10:01:14 (GMT+0)
     Zone:        ''
     Day of week: 5
     Day of year: 0
     Daylight s.: 0
Block: EXPO
  Offset: 0x00000060
    Size: 40
    Time: 843520.475000 ms
     ISO Mode:   0
     ISO:        100
     ISO Analog: 72
     ISO DGain:  0/1024 EV
     Shutter:    500000 µs (1/2.00)
Block: LENS
  Offset: 0x00000088
    Size: 96
    Time: 843520.534000 ms
     Name:        '1-65535mm'
     Serial:      ''
     Focal Len:   300 mm
     Focus Dist:  0 mm
     Aperture:    f/16.00
     IS Mode:     0
     AF Mode:     3
     Lens ID:     0x0000001B
     Flags:       0x00000000
Block: IDNT
  Offset: 0x000000e8
    Size: 84
    Time: 843520.550000 ms
     Camera Name:   'ERR:1 md:0x       0 ml:0'
     Camera Serial: ''
     Camera Model:  0x00000000
Block: WBAL
  Offset: 0x0000013c
    Size: 44
    Time: 843520.599000 ms
     Mode:   0
     Kelvin:   5000
     Gain R:   1024
     Gain G:   1024
     Gain B:   1024
     Shift GM:   0
     Shift BA:   0
Block: STYL
  Offset: 0x00000168
    Size: 52
    Time: 843520.611000 ms
     picStyle:   133
     contrast:   0
     sharpness:  0
     saturation: 0
     colortone:  0
Block: VIDF
  Offset: 0x0000019c
    Size: 27067688
    Time: 845473.718000 ms
   Frame: #100
    Crop: 0x0
     Pan: 0x0
   Space: 0

Vertical stripes correction:
  1.000  1.000    1      1      1      1      1      1 
[ERROR] Invalid block size at position 0x019d06c4
Processed 1 video frames
Done


If I set the "Silent pics" module to save individual DNGs for each frame, the timelapse series looks fine, ETTR does his job of continuously autoadjust exposure to changing light.

brapodam

Quote from: josepvm on July 25, 2014, 11:57:21 AM
Has anyone succeeded to get a fullres silent timelapse in MLV format using ETTR ?
Yes, I have succeeded in shooting a simple test while using ETTR. I haven't used mlvdump though. I used the MLV batch converter from this thread http://www.magiclantern.fm/forum/index.php?topic=10198.0

It worked without errors for me

surami

May I have the latest silent.mo module to test on the 550D, I only have to previous one without the file format options.
550D + nightly ML

josepvm

Quote from: brapodam on July 25, 2014, 03:05:51 PM
Yes, I have succeeded in shooting a simple test while using ETTR. I haven't used mlvdump though. I used the MLV batch converter from this thread http://www.magiclantern.fm/forum/index.php?topic=10198.0

It worked without errors for me

Thanks, I have not tried MLV Batch converter (I use Linux) but reading the thread you linked, It seems it is a GUI frontend for "mlv_dump" ?

Quote
I've been making this simple program in Visual Basic for the last few days. It is, like the title says, the first MLV to DNG batch converter for windows, which basically converts all your MLVs into different DNG folders. It has a lot of features, it is very simple and efficient. It runs from mlv_dump.

If this is the case, I suspect it will give me the same error I get with mlv_dump alone  :-\

I will search for the available mlv converters, and give them a try.

---------
update:  mlv2dng does not work either, with my mlv files when I use ETTR

josepvm

Quote from: surami on July 25, 2014, 03:31:23 PM
May I have the latest silent.mo module to test on the 550D, I only have to previous one without the file format options.

You have the module in this previous dmilligan post: http://www.magiclantern.fm/forum/index.php?topic=12733.msg122887#msg122887

But be aware that you need also a complete ML build for your camera from the fullres-silent-pics branch. You cannot put the silent.mo in a regular Nightly Build, it will not work.

Ask for a fullres-silent-pics build for your camera here: 
http://www.magiclantern.fm/forum/index.php?topic=12608.0

surami

Quote from: josepvm on July 25, 2014, 04:28:54 PM
You have the module in this previous dmilligan post: http://www.magiclantern.fm/forum/index.php?topic=12733.msg122887#msg122887

But be aware than you need also a complete ML build for your camera from the fullres-silent-pics branch. You cannot put the silent.mo in a regular Nightly Build, it will not work.

Ask for a fullres-silent-pics build for your camera here: 
http://www.magiclantern.fm/forum/index.php?topic=12608.0

Thanks for the link to the dmilligan post and for the warning.

I have this 550D nightly build created by mk11174
(Mercurial changeset: 25735947ce04+ (fullres-silent-pics), built on 2014.07.18.

Is this ok for testing purpose?

Yes I know about that topic, I asked there too for the proper build.
550D + nightly ML

josepvm


barepixels

Hi Josep

I just tried ETTR and Full-res Silent MLV

the new mlv_dump were able to extract the DNGs fine

here are each DNGs EXIF info

1) 1/15sec  f11.3  ISO 400
2) 1/15sec  f11.3  ISO 200
3) 1/15sec  f11.3  ISO 100
4) 1/15sec  f11.3  ISO 100
5) 1/15sec  f11.3  ISO 100
6) 1/15sec  f11.3  ISO 100
7) 1/15sec  f11.3  ISO 100
8) 1/15sec  f11.3  ISO 100
9) 1/15sec  f11.3  ISO 100
10) 1/15sec  f11.3  ISO 100

So I guess ETTR is working.

These are the module loaded at the time of shooting
dual_iso
ettr
mlv_play
mlv_rec
silent

Note: am using the build you made for me.  Which doesn't have dmilligan latest update:
http://www.magiclantern.fm/forum/index.php?topic=12733.50.msg123088#msg123088
update: EXIF shutter speed should now be the time of FA_CreateTestImage (the time you see displayed on the LV screen), not the value set in the Canon menu (which is wrong). This value may be slightly longer than the true exposure time, but it's more correct than Canon menu shutter speed.

If you need I can upload the mlv file for you.  It's 10  shot 376mb
Or if you want  I can try to extract your on my machine
5D2 + nightly ML

barepixels

I just tried again but this time ETTR and DUAL ISO is activate

Once again the new mlv_dump extracted all 5 DNGs fine

Here are EXIF info

1) 1/15 sec   F11.3  ISO 100
2) 1/1000 sec   F11.3  ISO 100   this is the ONLY file  have horizontal stripes aka dual iso (why is that?)
3) 1/1000 sec   F11.3  ISO 100
4) 1/1000 sec   F11.3  ISO 100
5) 1/166 sec   F11.3  ISO 100

Note: am hand held the camera  so the camera is constant moving, so sometime the camera is closer to the screen or has more bright white blank area in the frame
5D2 + nightly ML

josepvm

----------- EDIT: this explanation is not true, see the posts below ..............................................

Mistery solved, thanks, I can process now my MLV timelapses with ETTR on.


The problem was the lens.

In my previous tests, I was using a manual lens, following the advice in ML user guide, section "Shoot - Intervalometer". The guide suggests to use a manual lens, or an EF lens with aperture locked (I don't know how to do that  ???), and unscrewed, to avoid flicker.

But I have seen that ETTR with the manual lens fools the "silent" module when writing the MLV file. The lens I used was a m42 Pentax lens coupled to the 500D through a ring adapter with an EMF programmable AF confirm chip.

With a Canon EF lens, it works fine. That might be the first thing I should have tested.

I have shot the timelapse indoors usiing a dimmed lamp. I have altered the light intensity several times during the test, and ETTR continuosly adjusts the exposure time, everything works fine.


Next test will be outdoors during a sunset, or perhaps a moonrise nightshot beside the sea  ;)
This will be a much more interesting timelapse than my living room lighted with a dimmed lamp  :D

------------------------------------------------Edit -----------------------------

barepixels

Today is one of the rare days we have clouds in Sunny Diego, California.  Currently am test shooting full-size silent mlv with ettr time-lapse

I decided not to do the lens twist to test the power of ETTR module.  I hope I made the right decision.  Beside I like to see aperture in EXIF
5D2 + nightly ML

a1ex

ETTR does not adjust, or care about, the aperture. If you notice different behavior with manual vs Canon lens, that's a bug.

Can you describe the issue in detail? It's not clear to me what the problem is.

josepvm

Quote from: barepixels on July 25, 2014, 06:35:29 PM
I just tried again but this time ETTR and DUAL ISO is activate

It is possible to get a Dual-ISO fullres silent timelapse?  ??? cr2hdr can process correctly the resulting DNGs? This will be great!



a1ex

If the skip offsets are correct, it should work.

josepvm

Quote from: a1ex on July 25, 2014, 08:30:52 PM
ETTR does not adjust, or care about, the aperture. If you notice different behavior with manual vs Canon lens, that's a bug.

Can you describe the issue in detail? It's not clear to me what the problem is.

Using the manual lens I can get a fullres silent pics timelapse, using ETTR to autoadjust the exposure, and works fine if I set "silent.mo" options to save the pics in individual DNG files. This way there is no problem at all.

But If I choose to save the pics alltogether in a single MLV file, then I cannot extract the pictures during postprocessing, mlv_dump reports an error.

I have posted mlv_dump output when this happens in a previous post:

http://www.magiclantern.fm/forum/index.php?topic=12733.msg123271#msg123271


With a Canon EF lens, both ways work fine. No errors when processing the MLV file with mlv_dump






barepixels

5D2 + nightly ML

barepixels

crap, am in trouble now



it produce one file number 109  and that dng cant be open with ACR.

Grrr 471 pics over 9 gigs

Using Canon 24-70mm 2.8 and not twist.
======================================
EDIT  just tried second time, same settings but only 19 pics.  Extracted OK.  Going for my third try .... longer ... aiming for 100 pics
5D2 + nightly ML

josepvm

Quote from: barepixels on July 25, 2014, 08:53:26 PM
crap, am in trouble now



it produce one file number 109  and that dng cant be open with ACR.

Grrr 471 pics over 9 gigs

Using Canon 24-70mm 2.8 and not twist.

This is the error I observed this morning.

I'm doing a more complete series of tests now, and I see I was wrong, the error is not related to the lens (manual or EF). Tried several manual lenses, with adapters with or without AF confirm chip, and they all work well.

I suspect the problem is related to the interval between shots defined in the intervalometer.  But I need another test to confirm it.

barepixels

I notice each chunk is 2 gigs.  I was expecting 4 gigs for each

5D2 + nightly ML

josepvm

Quote from: barepixels on July 25, 2014, 11:02:55 PM
I notice each chunk is 2 gigs.  I was expecting 4 gigs for each

The same happened to me with the problematic mlv files this morning.

josepvm

Still no success to reproduce the error  :(

It seems I need a long test to reproduce it. It happened to me with timelapses of 101 images.

Now I have done several tests of 20 images, and they always work, regardless of the lens or the delay choosed in the intervalometer.

barepixels

maybe it has to do with splitting in chunks.
5D2 + nightly ML

josepvm


g3gg0

Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!