Messages

An update: I've been using FRSP + external flash (for extreme macro stacking; ~200-300 shots in a stack) successfully with Canon 50D and my DIY Arduino stacking rail Fast Stacker, for some time now. The good timing parameters (using Lexar 1000 card) are:  camera exposure 1/4s, flash delay (after initiating the shot) 1.1s, minimum delay between shots 5.5s. The FRSP is fully supported in my most recent rail software version, s1.14. For this to work you need an adapter like this:

FRSP adapter by First Last, on Flickr

I just finished writing a detailed tutorial on how to find proper values of the three timing parameters, for any  camera model:

http://pulsar124.wikia.com/wiki/User_guide#Electronic_shutter

FYI: I just released the newest public version (s1.14) of my Arduino focus stacking macro rail software, "Fast Stacker", which (among other things) adds support for FRSP + external flash. The feature seems to be stable and fully usable now - I used it to generate a few 300-shots focus stacks (using an external flash), for example (ugly bug alert!): https://www.flickr.com/photos/99745838@N03/25384340795 . More details here:

http://pulsar124.wikia.com/wiki/Whats_new

I'll probably make a short youtube video demonstrating FRSP + flash + Fast Stacker in action.

a1ex and other contributors: thanks a lot for this very useful feature!

My speculation is that extending half-shutter separates well the two points in time when camera's cpu gets very busy: around the time when the capture takes place, and when the captured image gets processed and written to the card. Because they are well separated, there is enough of time for the cpu to "settle down" between the events, which is perhaps why timings become more predictable.

I agree that trying to make timings much better than 1/4s is probably not worth the efforts - no advantage in terms of noise, and in terms of how dark the studio should be (because FRSP cannot  have effective exposure shorter than ~0.14ms - capture time for 50D). I think it is already very good as is.

[exposure]

I am happy to report here that after doing more testing of FRSP + flash (for macro focus stacking purposes) I got much better timings. Specifically, I managed to reduce the camera exposure from 0.8s down to 0.25s, with the same level of reliability (only one black frame out of 284 shots - similar to my previous best case scenario with 0.8s exposure). I actually did a few changes compared to my previous setup, not sure which of them was the main culprit for much better timings. In particular, I switched to MLV from DNG; I put to "Auto" long exposure noise reduction in my camera (it was "Always on"); and I significantly extended the shutter half-press time (from ~1s to ~2s). I am still using the same 1 shot every 6s frame rate. (Will try to reduce it in future tests.) By the way, I didn't realize that the timing of when you depress half-press also matters (before I assumed only the timing of engaging the half-press matters).

The new setup has the following sequence of events (camera: 50D; exposure: 1/4s; MLV format; Lexar 1000x card):

1) 0.0s: the half-press is enabled
2) 1.1s: external flash is triggered
3) 2.0s: the half-press is disengaged
4) 6.0s: going back to step (1)

A small caveat: the first shot is always black with the above settings, if you start from the Live View screen. There is a simple workaround for that: before starting focus stacking, briefly press the camera's half-shutter: the camera will take one shot (dark; one can use it later to fix hot pixels in your stack) and will exit LV (black screen). Now if you initiate focus stacking with the above timings, all the frames - including the first one - will be flash exposed.

Second important finding: this morning I made a number of dark frame shots at different exposures (from 1/160s to 0.8s), using both regular shutter and FRSP. When I processed them in the same fashion (using command line dcraw utility:  dcraw  -T -6 -W -b 500 file_name ; this command increases the brightness of dark shots by a factor of 500x, to make noise much more obvious), I didn't see a significant difference in the noise levels of all these shots, made both with shutter and FRSP.

Specifically, I wanted to compare the situation when I am doing focus stacking using shutter with 1/160s + external flash versus FRSP at 1/4s + flash. Be the judge: here are the two corresponding dark frames, brightened by the same factor (500x), and the noise seems to be pretty much at the same level in both cases:

Noise test: FRSP at 1/4s by First Last, on Flickr
Noise test: shutter at 1/160s by First Last, on Flickr

This alleviates the biggest concern I had with using FRSP + flash for focus stacking - the noise level (because of the longer exposure).

So now, with significantly shorter exposures (1/4s - meaning your studio doesn't have to be very dark) and the quality of the shots being the same or even better (same noise levels; mirror/shutter caused vibrations are eliminated), FRSP seems to be a great tool for macro focus stacking with an external flash - if you have the proper gadget (like my Fast Stacker) to trigger both camera and flash with specified time intervals, ans also to move the camera between shots. I will be releasing very soon the next public version of my rail's software (v1.14) which adds the FRSP support for Canon cameras.

a1ex: Thanks, lots of useful info!

I was actually hoping that with the exposures I need for focus stacking + flash (~0.8s at ISO100) noise will not be an important issue. I will do some testing.

So here is a summary of pluses and minuses of FRSP + flash used for macro focus stacking:

+ Mechanical parts (shutter and mirror) are not wearing off (with 300+ shots per single macro photo this could be huge cost saving);
+ Much less vibrations (=sharper photos), especially for extreme macro stacking (10:1 and up), when camera movements are small (<10 um), so shutter and mirror movements become the dominant source of vibrations;
- Shots are noisier (to be tested how seriously)
- Frame rate is limited to one shot per 6s (on my 50D) - so not suitable for live insects
- Requires a dark studio (because of the 0.8s exposures; probably not so bad with typical extreme macro effective apertures f/20 or less and ISO 100)

I didn't have issues reading MLV files on a PC (I used Mystic soft), except for the fact that different focus stacks were merged in a single huge MLV file.

I think this post by g3gg0 is relevant:

http://www.magiclantern.fm/forum/index.php?topic=12523.msg138135#msg138135

In particular,

updated silent.mo with this stuff:
- allow user to choose between DNG, Single-Frame-MLV and Multi-Frame-MLV (disable and re-enable "Silent Pic" from menu to start a new file in Multi-Frame-MLV mode)

So the way to start a new MLV file is to disable and re-enable "Silent Pic" from menu - I wonder if it is still the case?

Also, it is interesting that there used to be a Single-Frame-MLV option - I think that would work better for macro focus stacking. Any chance it could be resurrected?

But I don't use intervalometer! It is fully externally operated (from my macro rail). So it looks like I need to enable and then disable intervalometer to start a new MLV file?

Another question (sorry could find the answer in this thread): do silent pics get any long exposure noise reduction inside camera? I suspect noise will be the biggest issue, when comparing regular shots with a flash (exposure 1/160s) vs silent pics with a flash (exposure 0.8s).

a1ex: I need to see a few last shots I took, not just the last one, and not just for a second. When a flash dies e.g., by the time I paused the rail could already 3-5 shots behind.

mothaibaphoto: The only modules I loaded were mlv_play, mlv_rec, pic_view, silent. So you are suggesting I also need to load file_man, and the look for the recent photos in the Debug/Folder section?

My other grip with MLV is that it seems to always adding new pics to the same MLV file, even after I leave LV and re-enable it, so it ends up merging totally unrelated focus stacking sequences. Is there a way to start new MLV file when I want?

Yes I do see preview. I only take silent pics manually (from my Arduino macro rail, via camera shutter  cable).

^ Yes I do have image review enabled - is it a problem? I actually need it to see live if the shot I just made was properly exposed by the flash (when testing), and to see if my stacking is progressing well (when doing focus stacking).

I am puzzled myself about MLV not working with flash - I need to do more testing to figure out what's going on.

I think I just didn't spend much time trying to find good timings for MLV. I tried to use the DNG optimal timings with MLV, and all but first frames were black, so some more tinkering is needed.

Too bad that I cannot see DNG files inside the camera. For extreme macro focus stacking (typically 300 shots or more for a single stacked photo) I need a solution where I can pause stacking if something goes wrong (e.g. flash gets overheated, or battery dies), check inside the camera how many recent shots are bad (black), and then rewind the corresponding number of frame positions on my rail, and restart stacking.

I don't really have experience with MLV format - will it allow me to see recent individual shots inside camera easily? If yes, how to do that?

This is the picture I have about FRSP, which suggests the best way to trigger the flash. Please let me know if you find serious flaws with this picture.

FRSP + flash by First Last, on Flickr

Horizontal line is the time axis. Large red rectangular curve is the half-press button - zero when it is not pressed, at a high position when pressed. The duration of the press is dt_AF - it should be long enough for ML to interpret this as a silent picture (not as a command to switch to Live View). In my tests dt_AF=0.8s or more for Canon 50D.

I am assuming that the time from initiating a shot (the rise of the red curve) to the time when light capturing starts, simultaneously by all pixel rows, is variable, with sort of normal (Gaussian) probability distribution (the blue bell curve). dt_exp after the light gathering started, the very top pixel row stops gathering light (followed later by the rows below). dt_exp=top_row_exposure_time (measured to be ~1/30s for my 50D by people in this thread) + the camera exposure time, and should have a fixed (not variable) value. (Or perhaps it is the maximum of the two values, not their sum?) So the end of light capture will have the same bell shape as the blue line, just shifted to the right by dt_exp.

The top panel shows the situation when camera exposure is short (say 0.1s or less), resulting in the blue and green bell curves partially overlapping. Now you can see there is no perfect spot to trigger the flash; the best spot (thick black vertical line) is half-way between the bell curves, but even there there will be some shots (represented by the green filled area) where the light capture of the top row ended before the flash - resulting in some (or all) of the top rows being dark. For a similar fraction of shots (represented by solid blue area) flash goes off even before the capture started, which will always result in all rows being black.

So let's increase the camera exposure! (Bottom panel). Now the two bell curves are well separated, so one can choose such a time to trigger the flash (at a later time, compared to the top panel) when neither "capture ended before the flash" nor "flash goes off before the capture" problems ever happen. (Well, if the distributions are truly Gaussian - which I doubt - there still will be a very small chance of the either problem happening; but this will affect only a tiny fraction of all shots).

I think these plots (if they indeed capture the essence of what's going on) give a good idea regarding how to choose the three time delay parameters involved - dt_AF, dt_exp, and dt_flash.  In particular, camera exposure should be comparable or larger than the total width of either of the bell curves. (From my measurements, it is ~0.8s for 50D).  It is also obvious that dt_flash is a function of exposure - the larger the exposure is, the larger dt_flash becomes (it always tracks the half-way point between the two bell curves.)

I hope this will help other people interested in using a flash with FRSP, for macro focus stacking.

I've spent half a day today trying to figure out shortest timings which would enable flash use with FRSP. I used my modified Arduino rail for that, with two circuits controlling separately camera's half-press and flash. As expected, flash worked reliably once you make exposure very long (say 5s), and trigger the flash half way through exposure. Then I tried to reduce exposure as much as possible, and arrived at the following timings which seem to work well on my 50D (using DNG format; my card is Lexar 1000x):

Exposure: 0.8s;
1) t=0s: enable half-press;
2) t=0.9s: trigger flash
3) t=1s: release flash and half-press;
4) t=6s: go back to 1st step.

The 0.8s exposure happens somewhere between steps (1) and (4). The exposure is long enough to absorb any variations in the delays between the half-press and the actual capture. It needs at least 6 seconds between shots for reliability, which is fine as this is a normal delay used in focus stacking to let the camera vibrations from the rail motions to settle down.

I ran a long test with the above timings - 283 shots (almost 30 minutes), and all the shots where perfectly exposed by the flash except for a single black shot. The timings will be different for different cameras, and likely different for different memory cards.

But I think overall this is a good news - it looks like the full resolution silent picture feature is totally usable for macro focus stacking photography, with and without a flash. One needs fairly long exposures for this to work reliably (0.8s for my 50D), which will require a fairly dark studio environment (extreme macro is done at effective apertures f/20 or smaller, which helps). Also I need to check if noise becomes a problem - in the past I saw quite a bit more noise with exposures of a few seconds, compared to exposures 1/100s or shorter.

I plan to release soon the new version of my Arduino macro rail software with the support for the FRSP feature. (It already works, just needs to be thoroughly tested.)

This was all done with DNG format. For some reason MLV didn't work for me - perhaps I just need to spend more time on that.

For some reason I cannot see DNG shots inside my camera (they are totally invisible) - can anyone tell me if I am doing something wrong here? These are the modules I loaded: mlv_play, mlv_rec, pic_view, silent. The ML is the nightly from a few days ago.

I modified my macro rail code to use half-press instead of full press, and did some testing. When camera exposure is set to 0.1s, half-press should be at least 0.8s long to reliably take a silent picture. And I can reliably take a long sequence (at least 100) silent pics if the interval between pics is 4s (MLV format). My camera is 50D, the card is Lexar Pro 1000x.

So up to now it seems like it might work with my focus stacking rail. The critical test will be (after a bit of soldering) to connect both camera and external flash to my rail, and to see if I can come up with a fixed delay (plus play with different exposures) for the flash to work reliably.

It is a good timing, as I just (finally) received the RMS-EOS adapter for my new microscope objective, which will give me 10:1 macro magnification.

Thanks for the info! I did see LED feedback suggested as a means to use silent pictures with remote triggers, but I'm still hoping that under certain (well controlled) circumstances fixed delays will do the job. For example, using a fast card + shooting at low enough fps should reduce or eliminate write buffer related uncertainty. And it sounds like I'll need a lot of hacking to my rail to enable an LED feedback. I'll try my simple idea first. For studio focus stacking, everything can be controlled very well - I can always use the same exposure, the same fps; my rail would allow me to fix precisely all the timings - how long the half-press is etc. We'll see.

I noticed there are two ways to trigger a sequence of silent pictures - one by using one long half-press per photo, the other one using two short presses per photo. (The first one would be easier to implement in my rail). Do you know which approach results in faster framerate for actual photos?

I wouldn't use ML as a focus stacking controller - I tried it initially, before building my Arduino rail, and didn't like it (mostly because of the need to use LV - meaning very slow frame rate). With my Arduino rail I can reliably do 4 fps (RAW images) for at least 30 shots, with external flash - which is perfect for focus stacking of live insects. This is not possible with ML.

Indeed, my numbers were not right. I redid the test today, and with 0.3s exposure and DNG format I got two timings - 550ms for capture, then 3200ms (I guess total). With MLV the capture timing is not shown (or shown too briefly to be read) - that was the source of my confusion before. The total timing with MLV is not fixed - it floats around 1300-1900ms. So now it does make sense.

I also tested triggering silent picture using a wired remote (the AF switch) - it does work, for multiple shots.

Finally, I set exposure to 1s and then manually triggered an external flash, after half-pressing the shutter button. It does work, but I have to wait until the blue LED lights up (takes ~1s), and then a bit more (~0.5s). If I press the flash earlier it doesn't show in the shot, but if I do it ~1.5s after half-press, the flash works as expected - no artifacts (gradients etc.).

So it does look like the silent picture feature can be used with my Arduino macro rail together with an external flash. The sequence of events should be like this:

- the rail triggers the AF (half-press) relay for a specified time (~1s);
- after that it waits for another fixed delay (~0.5s ?) and then triggers the second relay, connected to the flash
- then it waits for a third specified delay (~2s?), and goes back to the step 1.

The big question is whether  three fixed delays will result in a reliable (no skipped shots) long (~300 shots) focus stacking sequences. I think I am motivated enough at this point to start tinkering with my rail's software to test this out.

Thanks for the reply! The timings I am talking about are those displayed on the camera screen right after the shot is taken. I understand that these timings are not just the capture time (which I know from a post here to be 140ms for 50D), but also include writing time.

I am pretty positive that these timings were ~350ms for DNG and much longer for MLV, but I will double check that.

I have a pretty decent CF card - Lexar Pro 1000x, and from RAW video experiments I know I can have a sustained write speed around 84 MB/s. So perhaps the ~350ms result is not so surprising?

Do you know anything about the usability of a flash with FRSP? Will it work in principle (e.g. are there any rolling shutter etc. artifacts when using a flash)?

Hello,

I just discovered that ML now has silent full-resolution pictures ability (thanks a1ex for the great work!). These days I am into extreme macro focus stacking - I designed and built an Arduino based automated focus stacking rail, "Fast Stacker", and with my new macro "lens" - Nikon M Plan 10x/0.25 microscope objective - I will be routinely doing 300-1000 shots focus stacks, so the limited shutter life of my 50D is becoming a serious concern.

I tried the newest nightly yesterday on my 50D, and the full-rez silent pic feature seems to work as it should. One observation is that apparently DNG is much faster than MLV for silent pics - I get 320ms vs. ~1500 ms respectively, so I think I'll stick with DNGs. But is the difference normal? Perhaps 50D's ancient CPU struggles with MLV packing?

But flash doesn't work with the silent pic feature, as my testing and google searching showed. Flash lighting is very critical for extreme macro (magnifications 10:1 and up), as it helps to freeze the unavoidable environmental vibrations. I couldn't quite understand why it is not possible to make flash work with silent pictures - I think it was something about not knowing when exactly to trigger flash.

My Arduino rail can control two separate relays - for now they are used for AF (half-press) and shutter (full press). I can quite easily add a new mode, when the rail would first trigger a half-press with one relay, and then after some delay - trigger an off-camera flash using the other relay. I'd also need to modify the cable connecting the rail to the camera accordingly.

Will this work? Is it possible to reliably take silent pics with a flash when it is triggered a constant delay after camera's half-press? I'd like to understand this better before I start messing up with my rail. It looks like using a flash would solve the gradient issue, as all rows would get exactly the same (very short: <1ms) exposure from the flash.

How often can I take silent pics with my 50D? The first post here mentions "at least every 10s", which is kind of slow - I normally take shots every 2s for studio macro stacking. From the data I saw here it looks like my 50D has the shortest readout time - at 0.14s - so perhaps I can achieve more usable frame rate (ideally 0.5 fps)?

You know that you can always do the dot-tune manually, by clicking the half-shutter button and changing MFA value manually? You can do that to detect which ML method (if any) produces the correct result on your 6D.

I agree that your approximate method does work reasonably well in my case (though intervals are not always +-8 - I have +-10, and for some finicky lenses that might make it or break it in terms if pinpointing the only reasonable MFA value which would satisfy all the constraints), but why settle for approximate if one could easily obtain the real thing - the actual good MFA intervals? My thinking is that the good MFA intervals are the primary data, while the "optimal" MFA is a derivative. I'd feel more confident analyzing the primary data when dealing with particularly finicky zoom lenses.

I've been doing MFA of my lenses for a few years now, and tried most of the approaches (e.g., I wasted my money on the commercial product Reican FoCal - terrible customer support, never worked on my Windows XP laptop), and found the dot-tune method (either manual, or with ML) to be the best. There is only one feature I wish the ML's implementation of dot-tune had: at the end of the calibration, I'd want to see not just the median MFA value, but also the good MFA interval (the lower and higher values). The reason: MFA values are different at different distances to the target, and at different focal lengths of zoom lenses. And the only way to find the optimal (good for any distance and/or focal length) MFA value is to determine the good MFA min...max intervals at all distance/FL combinations, and then overlap these intervals, and find the more narrow interval (if it exists) which would be present in all of the individual intervals.

A point in case. A couple of days ago I re-calibrated all my 4 lenses. My fast zoom lens (Sigma 17-50mm f2. was the most tricky as usual. I measured the good MFA intervals by observing the output of the ML on my camera screen. This is not very reliable, but this is the best I could do. I did three distances (1, 3, 8m) and three FLs (17, 30, 50mm). I obtained the following good MFA intervals:

Sigma 17-50 OS: -3 [-3.5,-2]

    1m: [-3.5,-2]
        50mm: -10, [-18,-2]
        28mm: +1, [-7,+9]
        17mm: +4, [-3.5,+11.5]
    3m: [-8,+0.5]
        50mm: -7, [-14.5,+0.5]
        28mm: +0, [-9.5,+9.5]
        17mm: +0, [-8,+8]
    8m: [-7.5,+5.5]
        50mm: -2, [-10,+6]
        28mm: -3, [-11.5,+5.5]
        17mm: +0, [-7.5,+7.5]

As you can see, the biggest tension was at the shortest distance - 1m. If all I had was the "optimal" (median) MFA values reported by ML, then I'd be in a difficult situation trying to pick the best number from -10, +1 and +4. But everything is much more straightforward when you compute the overlapping part of the three intervals, which was pretty narrow: -3.5...-2. All other distances and FLs were consistent with this interval. So I went ahead and chose one value from that interval, -3, and it indeed works very well at all distances and FLs.

If instead I chose to simply average all the 9 median MFA values, I'd get -2. This would still probably be okay, but I feel the interval approach gives much more reliable result.

Can printing the good MFA interval on the camera screen be implemented in ML? I think it'd be a very useful addition, and suspect quite easy to implement.

Thanks for the great work!

I just tried the MLV module, in the newest TL build. I unpacked it with http://www.magiclantern.fm/forum/index.php?topic=7802.0 - is this the proper tool? It did complain about some keywords not recognized (MARK, ...). My newest ACR does read the proper aperture and FL from the DNG files (great!), but doesn't detect the lens automatically. I checked with exiftools, and the CR2 files have these lines:

Lens Type                       : Canon EF 85mm f/1.2L or Sigma or Tamron Lens
Lens Model                      : 17-50mm
Lens                            : 17.0 - 50.0 mm
Lens ID                         : Sigma 17-50mm f/2.8 OS HSM or Tamron SP 17-50mm f/2.8 XR Di II VC
Lens                            : 17.0 - 50.0 mm (35 mm equivalent: 26.9 - 79.2 mm)

Whereas the DNG files only had this line:
DNG Lens Info                   : 17-50mm

But this is not big deal - I can set the lens type manually.

I am not sure if lens correction (vignetting, distortion) works correctly though - the ACR profiles were designed for specific full censor resolution of the camera (I think), and also for a different aspect ratio (3:2). Does anyone know how to modify ACR lens profiles to make them work perfectly with MLV DNG files?

So from what I understand, the Digital Dolly feature doesn't work in the most recent build, right? It doesn't do anything for me. But even if it worked as intended (I guess by operating the joystick while shooting), I am not sure it'd be very useful - I can't see how one could use it to implement a slow, steady and smooth parallel shift of the window.

How about my original suggestion - would it be too hard to implement a pre-programmed Dolly - you specify the shift (pixels per frame; can be negative), the starting position of the window will be at the opposite end, and when you start shooting the window will start smoothly shifting towards the opposite end (when you reach the opposite end, it can just continue recording)? In simplest case, only one additional input parameter would be required - delta_x (pixels per frame). Ideally, the shift could be made in any or both directions, by using the second parameter, delta_y.

I think such a feature would be really useful - steady, adjustable and very smooth panning, with no jerks and camera shakes from using the joystick.

What do you think?

Hope I don't sound dumb, but I'm confused on the ACR.  You mean Adobe Camera Raw, right?  And if so, why would you run DNGs through it again, isn't that what raw2dng is for?  What do you mean "ACR-tweaked." 

I generally used raw2dng, then if I want tiffs, UFRAW with options.

ACR is an equivalent of UFRAW. It can output processed RAW files as JPEG, TIFF (8/16 bit), and as DNG.

I think I am converging to use 30p for my attempts at footage stocks with 50D. It looks like stocks prefer 30p over 24p, and they strongly prefer 16:9 full HD. I checked yesterday, and Oct. 11 TL has no problem recording uncropped video at 16:9 and 30p to my Lexar 32GB 1000x. I upgraded my ACR to the newest one (8.2), and now it does have the arbitrary resizing option at the end - meaning I can do almost everything now inside ACR (with resizing and final sharpening). (Still it doesn't output PNG, so I have to rely on JPEGs to deliver frames from ACR to Virtualdub). The workflow now is

DNGs->ACR->JPEGs->Virtualdub->video

The only filter I need in Virtualdub is Crop: upscaled uncropped frames in ACR have the size 1920x1081, so I tell Virtualdub to shave off that one extra row of pixels.

My research also shows that footage stocks frowns upon color grading (it should be left for the buyer to do that, to match their whole sequence of clips), which simplifies life. I just plan to use ACR to fit the dynamic range into 8 bits (lift shadows, recover highlights etc), correct WB, and perhaps increase saturation (especially for landscape and abstract videos).