Silent picture capture with raw video format

[ayshih]

the seek call
> FIO_SeekFile(save_file, 0, SEEK_END);
seems not to seek to the end correctly. i know it wont handle positions > 2GiB if passed as parameter.

dmilligan could use the one i use for mlv_play to seek correctly:


/* as soon all cameras provide FIO_SeekSkipFile, we can remove that temporary code */
static uint64_t FIO_SeekSkipFile_emulate(FILE* stream, uint64_t position, int whence)

I've just recently come across this issue also, and it doesn't look like FIO_SeekSkipFile_emulate works properly across the 2 GB boundary, at least on the 50D.  FIO_SeekFile returns -1 when seeking across that boundary, and FIO_SeekSkipFile_emulate just calls that.  Since the stub for FIO_SeekSkipFile has been found for nearly all cameras, I think it's time to ditch the emulated version.

[dmilligan]

here's a version using FIO_SeekSkipFile: silent.mo

[g3gg0]

I've just recently come across this issue also, and it doesn't look like FIO_SeekSkipFile_emulate works properly across the 2 GB boundary, at least on the 50D.  FIO_SeekFile returns -1 when seeking across that boundary, and FIO_SeekSkipFile_emulate just calls that.  Since the stub for FIO_SeekSkipFile has been found for nearly all cameras, I think it's time to ditch the emulated version.

uhm wait, calling FIO_SeekFile to incrementally position beyond the 2GiB border doesnt work on 50D only?
didnt know that.

[barepixels]

Success!!



extracted 200 Full-res Silent MLV ETTR pics

Thank you

[ayshih]

uhm wait, calling FIO_SeekFile to incrementally position beyond the 2GiB border doesnt work on 50D only?
didnt know that.

Yup, I first encountered it when trying to skip frames in a RAW video longer than 2GiB, and that's when I realized it was the same reason that building a MLV index was crapping out at the 2GiB point too.

Also see my mlv_play pull request

[josepvm]

here's a version using FIO_SeekSkipFile: silent.mo

Thanks!  It works 

I have repeated the indoor test with 101 pics, EOS 500D, same settings and conditions, and mlv_dumps extracts now correctly all 101 frames.

The MLV now is a 2.55 GB file. Previously, for 101 pics, it was always a 2 GB File.

--------------------------------------------------------------------------------
Edit:  I have completed now a longer test, with 301 pictures.  Now I get 2 chunks, a 4 GB ".MLV" file and a 3.6 GB ".M00" file.

All the pictures are correctly extracted by mlv_dump, no errors.

The time value in exif data increases in every frame at a much higher rate than the real time. I mean, the last frames in the series show a time stamp that is more than an hour later than the real time when the series finished.  I know, that's another story, I will measure this time offset better next week.

But the problem with files larger than 2GB seems solved. The transition to another chunk when the 4 GB limit is reached works well now.

[barepixels]

Confirm on the time.  My interval was 10 seconds apart but the files creation date reported as  20 seconds apart

tried it again with interval at 30 seconds.. file creation date reported as 60 seconds apart... with final few files almost 1 hour ahead of current time.

with that said, it didn't effect my workflow

[dmilligan]

lets try to keep this thread on topic, ETTR questions and issues should not be posted here

[marekk]

I've got one question regarding power saving feature and silent pics. Is there a possibility to change state of camera to "recording" when ML is saving silent pics with intervalometer ? When mlv_rec is recording movie we can dim or turn of display to save battery power. With silent pics camera is in standby mode. When silent pics saving is triggered by intervalometer, display is turned on for a while or brightness is reconfigured from minimal to standard value.. I would be nice to turn off display completely during long timelapse recordings..

[Attero]

Yea, i fully agree with marekk. To turn off the Display would be great trough timelapse! Or at least to auto decrees the brightniss of the Display.

[reddeercity]

Trying out full res silent pictures, very nice. 
I saved the pictures as .RAW not MLV so what would the commanded line syntax (windows 7) for raw2dng to convert All (240) full res silent pictures to DNG's.
Or do I have to write a batch file to do this ?

[dmilligan]

You'll probably have to write a batch file. I have an example of a bash script (linux/mac) that will do this on the first page. I'm not familiar enough with the batch file scripting language to give an example, but it should be fairly straight forward.

[g3gg0]

maybe you should consider adding INFO tag using IME editing.
so users can enter what scenery/take etc this take was?

[dmilligan]

good idea, will do

[dmilligan]

oh and btw g3gg0,

It very much sounds - though I can't confirm for sure without the files in my hands - as though the silent picture MLV code is not writing the MLVI block at the start of the subsequent files (.M00, .M01, etc) in the series.

From the MLV format thread (http://www.magiclantern.fm/forum/index.php?topic=7122.0):
"- MLVI block is the first block in every .mlv file"
and
"- using the GUID a tool can detect which partial or spanning files belong together, no matter how they are named"

While mlv_dump may be handling/decoding these file sets ok, MlRawViewer expects that block at the start of each file in the series. I think you should raise this as a possible bug against the Magic Lantern silent picture MLV code.

I was wondering what your thoughts were on this. I see Andrew's point but I think the spec is slightly ambiguous on this point. I didn't originally read it this way, and just assumed I only needed the MLVI once, not in the chunks, it only becomes clear reading the two points together. The spec should probably explicitly say the MLVI block should start every file including spanned chunks, I took, "every .mlv file" to mean only the ".mlv" and not the chunks, ".m00", ".m01" etc.

This requirement would also add a bit of complication to the code as you have to go back and re-write the MLVI header in each file (rather than just re-writing it in the first file).

[g3gg0]

reasons:

- files on disk should have predictable headers that declare which content they have (MLVI)
- this way a stray .m00 file can be uniquely identified, using its UUID (which is the same in main .mlv) no matter which name it has
- next to the UUID, the logical file number is coded in this header, so all files could be named randomly, you will find the ones that belong together
(imagine someone recovers files from a card named CHK00000.DAT, CHK00001.DAT etc, this must not be a problem for MLV)

especially the first reason is my strongest concern.
i dont like files whose content can only be determined by inspecting its payload. i want to check the first few bytes and know what it is, if its
part of a group of files and which others file belong to it.

hint:
mlv_rec does not rewrite the total file count - it just wirtes zero. its a deviation to the things i've defined, i know.
you can do it the same
this is documented in the spreadsheed btw:
> fileCount: when set to other than zero: how many files belong to this group (splitting or parallel), else: autodetect this

[reddeercity]

I write a quick batch file for Windows7 to process multiply  ".raw" saved full res silent pictures with raw2dng to .dng
Its not prefect , it make the file name to long but it works. It anyone cares to fine tune the batch file please go ahead .
I haven write a batch file in years  so I'm rusty on the syntax.

Code: [Select]

set raw2dng="D:\FullRes_Bat\raw2dng.exe"
for %%f in (*.raw) do %raw2dng% "%%f" "%%f"

Make sure raw2dng is with all the .raw files in same folder , it will exact to the same folder.
Set your path where raw2dng is , in my case drive "d:"
Just open windows notepad save as a "something.bat"
double click to run .

[dmilligan]

I've (hopefully) fixed the issue with MLVI headers. Testers needed: https://bitbucket.org/hudson/magic-lantern/pull-request/588/silent-pics-mlv-timelapse-write-the-mlvi/diff

[g3gg0]

should do, but i recommdend setting the fileNum to the actual file number.
depends on how strict tools parse this field.
>> mlv_file_hdr.fileNum = fileNumber;

MLV = 0, M00 = 1, M01 = 2

you can keep the fileCount zero for autodetection.

[Attero]

Just for Info. 600D 19July Nightly + silent.mo from dmiligan ist still working great for me. Taking 500 pictures without any dropframes and saves same into seperat .mlv/.m00 files. Extracting with mlv_dump.exe and works perfectly.

Here is one framegrab from a little "ML Full Res Silent Pic" promo video im putting together.



Just some Ideas. It would be great to see the (RAW?) Histogram during saving or when the pic is saved. For Day to Night(or Night to Day) TimeLapse this is very important.

[gcrook]

Today i started doing some more silent/mlv tests with my 60D using a late July nightly using the same silent.mo -dmilligan?- posted a few weeks back.
For the life of me i can't understand why after a photo is taken, camera stays in preview after file is being copied to the mlv.It doesnt revert back to live view,if only for a split second before next photo is being taken, and i have tried every combo in the camera menus under the
image review.I didnt have this problem when i did the initial tests a couple of months back.Any thoughts?

[Attero]

Yeha. I have the same problem... Would be nice if it jumps back to LV earlyer to study the histogram. Now i have only 1/2 of a sec for this.

[Attero]

So, like i sad. I put together some sequenzes i've shoot. I hope u enjoy this short clip.
A High-Five to A1ex, Dmilligan and the whole Magic Lantern Team for this modul!

[bookemdano]

@dmilligan - thanks for this.

I just compiled and am testing this out on my 6D. For my purposes though I'm actually using silent.mo Simple mode because I need speed (going to try to use it to telecine some old 8mm home movies by "projecting" onto the camera sensor at a low frame rate and triggering the camera to photograph each frame of the film.

I was hoping with your addition that my pics would all be saved into a single MLV container. But when I take a bunch of silent pics it's presently saving each one to its own MLV. When I came back to this thread and re-read the first page I see now that you only implemented saving a single MLV for the intervalometer function.

How difficult would it be to enable this capability (single MLV) for silent pics taken via multiple half-presses? Any chance you could implement that? Maybe as a separate option in the settings since I guess for others it would be undesirable to have all pics from a "session" in a single MLV.

I'm sure you're not looking for more work, so I understand if this isn't something you want to do. Just thought I'd check in case it's a fairly simple change.

Thank you again!

[ferrocium]

So I'm totally new to this forum and pretty new to ML.  I just want to say to all the ML developers...YOU GUYS ROCK... ML makes the Canon cameras soooo much better.  I've been using it on my 60D for over a month now and I'm loving all the features (I can basically trash my plug-in intervalometer beast .
As I said, I'm a total noob to this forum so I don't know how to find info very well on here yet, or how to put new build modules onto my card.
I'm a timelapse junkie but I don't want to wear my camera out and this silent shooting thing in full res raw looks like the perfect solution. 
I see that you guys have it working for the 60D, but at the beginning of this post is says that this module will not work with the regular nightly build install. And then I saw some technical talk about having to "change the whole branch file".
Are you guys able to help a programming-ignorant dummy out and give me a step by step how-to guide so I can get silent shooting to work and save files as raw (CR2 im guessing) on my camera.
Also, I tried shooting in silent mode and it's saving pictures as DNG, but when I open the DNG file in PS Elements 10 all I see is a black image with red letters on top that says the file name (ex. 001.dng).
Just a few more quick questions:
What size raw file can it capture in silent mode? Largest?
Whats the longest exposure time in silent mode?
Will silent raw capture work with bracketing and ettr?

Can it damage the camera to have the mirro/shutter open for that long? Bad to point it at bright light?

I apologize if all of this is redundant for what may have already been covered somewhere.

Any help would be appreciated.  Thank you all in advance.

Edit: I fixed the black image issue in PS Elements for DNG but I'm still very interested in getting the pictures to save in full res / raw format in silent mode.