Messages

Is this possible? Say, saving only the central 50% crop when shooting with full resolution silent picture? This could be used for example for 8mm film scanning (frame by frame, with a macro lens), where the full sensor resolution is an overkill, plus one can only fit ~2 minutes of 8mm film on a 64GB card.

@a1ex: I just tried your focus stacking (not  the aligning) Octave script on my recent stack. Specifically, it is ~3.5:1 macro with a very good quality macro lens (Componon Schneider 28mm f/4 enlarger lens), 14 um steps, 120 raw images in a stack. First image was processed with enfuse, second - with your script. I did not apply any sharpening. Overall, your script produces more clear results (less halo around bright objects). But it does leave some artifacts (instead of halo, now it's a ring), which will cause issues if you still plan to do some fairly significant PP (sharpening, 2D deconvolution etc). In particular, see the left bottom corner of the image, with the bright piece of copper.

I suspect your script will be the best when used with fairly low magnifications (like in your original example) - less than 1. For magnifications lager than 1, a significant PP sharpening is always involved, and that works better with the enfuse approach.

And of course, another obvious advantage of your script - it doesn't use much memory (~1.5GB). With the ~100 16-bit TIFFs stack, enfuse used almost 20GB of RAM. One can use my Macro-Scripts instead (two-phase enfuse application, which reduces RAM consumption by a square root of N), but it produces even more prominent halos.

Again, 3D deconvolution would be the ultimate solution for halos and hazyness in macro shots...

enfuse by First Last, on Flickr

focus_stack by First Last, on Flickr

Very nice. Scripting around align_image_stack and enfuse, with additional processing

The reason I've asked for a test image sequence: I was recently asked by the Apertus folks to help them with a focus stack of a PCB, where align_image_stack was failing (and, as it turned out later, enfuse had suboptimal output on that image sequence). The main problem was the image sequence itself, which apparently was captured wide-open with an old manual lens; I've recommended them some new settings (along the lines of stopping down the aperture and using ISO 100, even if that meant a slightly long exposure), but re-shooting that particular PCB wasn't an option at the time (don't ask me why). So, I ended up rewriting align_image_stack and the focus stacking part of enfuse, from scratch, as Octave scripts, for that particular image sequence. Yes, in that case it was easier - for me - to reinvent the wheel, rather than fine-tuning the existing software.

When seeing your images, I thought my modified align_image_stack / enfuse versions might give slightly better output, so I was tempted to give it a try. Now that I see you are using an workflow based on align_image_stack and enfuse, you might be able to try them yourself (there are minimal modifications on the command line). Please find my scripts here:

http://files.apertus.org/align_image_stack/

The first script is align_image_stack_finetune.m. It uses an initial brute force search, followed by coordinate descent (parameters: X and Y offsets, zoom aka focus breathing, optional rotation), first on low-resolution copies of the original images, then, increasing the resolution until 1/8 x 1/8 of the original (you can change that by editing the script). It aligns all the images in the sequence, to the middle image (as specified on the command line). It's significantly slower than the original align_image_stack, but it worked out of the box on the PCB test sequence (a few images available on that link, but not the entire sequence). It should handle images with tiny depth of field, where align_image_stack is unable to find its control points.

The second one is focus_stack.m, which outputs a weighted average of a pre-aligned image sequence. Weight is computed by looking at how much each image differs from a blurred copy of itself (computed as grayscale, for speed reasons). The output contains less halo artifacting, compared to enfuse, although I didn't use any deconvolution. I haven't tried it on other images.

Before: PB5-enfuse.jpg
After: PB5-focus_stack.jpg

If you decide to try these scripts, please share the results. If this focus stacking algorithm can be useful to a wider audience, I'm happy to contribute it upstream (to be included in the official enfuse). With align_image_stack, it's a bit more complicated, as my approach is fundamentally different; besides, it would take significant work to convert it to another programming language (haven't checked the upstream sources yet).

Please explain quoted part in detail!
Are you using radio triggers whch you can adjust / program a delay or do you do this with ML specific code. I am very interested in this. And I thought the shutter limit was already solved by a1ex for FRPS - at least on 5D3 !?!?!? So you could probably use more flash power. Pleas tell me I can use some kind of "tail sync hack" with frsp. I need this adjustable delay for flash trigger function

My trick does work (I used it with 50D and 6D), but it requires an external Arduino gadget which controls separately the camera shutter and the flash (in my case, it also controls the focus stacking rail of my design - https://pulsar124.fandom.com/wiki/Fast_Stacker). All the details are here:

https://pulsar124.fandom.com/wiki/User_guide#Electronic_shutter

@a1ex: Thanks, I'll give your scripts a try. Actually, ever since I switched to always run align_image_stack from the middle image (this is what my Align.sh script does), I never had issues with this program. So I am more interested in your focus stacking script, as bright halos bother me a lot in my stacked images. As I mentioned, I am looking into 3D deconvolution of bright field microscopy (which is more or less what macro focus stacking deals with), which would be the best tool for the task. One can try to use theoretical PSF (e.g. this recent Science paper https://www.sciencedirect.com/science/article/pii/S0006349517309840), but they are complicated to compute, and I am sure are not exact match to real life macro photography. I'll try to measure PSF myself - one needs a tiny (smaller than the resolution limit) light object against dark background. I am thinking perhaps a reflection of a single bright LED (placed at a large distance) in a small metal ball from a bearing might do the trick. Or perhaps a few tiny particles of flour on a clean glass plate.

Have you seen this software: https://github.com/PetteriAimonen/focus-stack  ? It is super fast (both aligning and stacking are using GPU), but many times the alignment part failed on my, due to some "failed to converge" issue.

Hi,

Some of these photos have lots of details on Flickr (most of them are old, so I won't even remember the details if they are not there!). Indeed, a few of them are microscopic (in the sense that I use a pro microscope objective with some extension tubes and/or telephoto lens). The top 3 shots are made with 20:1 magnification, using Olympus ULWD Neo Splan 20 0.40 8/- f=180 microscope objective (I was lucky to get one on ebay for ~150$; normally ULWD ones sell for $400+). I also have a 10:1 Nikon microscope objective, for 10:1 macro. For lower magnifications, I use a very decent "macro lens" - old film enlarger lens Schneider Componon 28mm f4, with and without extension tubes (gives me 1.5-5:1 macro).

A lot of postprocessing goes into shots like this, especially extreme macro like 20:1 which suffer significantly from diffraction. (At 20:1, my 6D pixel size is only 0.3 um - much smaller than the light wavelength). It typically involves multiple sharpening steps, at different scales. I wrote my own scripts and programs for the whole workflow, from raw images processing, bad pixels removal and white balance, to alignment, to focus stacking, and multi-scale sharpening:

https://github.com/pulsar123/Macro-scripts

I think I am getting better with PP, that's why my more recent shots (sanding paper) look sharper than earlier ones (like CPU pin).

I am still doing research into proper PP for focus stacks. My holy grail is to be able to apply 3D deconvolution to the stack before focus stacking - this should remove all those glowing blobs around bright parts of the stacked image. Right now, I am playing with ImageJ with DeconvolutionLab2 plugin (medical software, used for microscopy).

In terms of light - it is not that extreme. For many shots I use a flash with a softbox (I figured out how to use flash with FRSP - the trick is to find a delay, specific to your camera, between triggering FRSP and triggering the flash), typically at 1:8 power. The FRSP exposure is so long (0.4s for my 6D), that one can just use interior ceiling lights for proper exposure (but I think sharpness suffers, because of long exposure - from microvibrations). E.g., this wire spiral from a wire nut:

Macro mountain by SyamAstro (750,000 views - thank you!), on Flickr

More examples:

https://www.flickr.com/photos/syamastro/albums/72157627584126318

In particular (10:1 macro),

Macro 10:1 by SyamAstro (750,000 views - thank you!), on Flickr

and (6:1)

Peppercorn by SyamAstro (750,000 views - thank you!), on Flickr

All done with ML using FRSP and https://pulsar124.fandom.com/wiki/Fast_Stacker (focusing rail of my design). Typically between 50 and 200 shots per stack.

Sanding paper:
Sanding paper 20:1 macro by SyamAstro (750,000 views - thank you!), on Flickr

CPU pin:
Extreme macro at 20x by SyamAstro (750,000 views - thank you!), on Flickr

Ant's head:
Extreme macro at 20x by SyamAstro (750,000 views - thank you!), on Flickr

Lady's watch:
Time machine by SyamAstro (750,000 views - thank you!), on Flickr

Untitled by SyamAstro (750,000 views - thank you!), on Flickr

 Good news - with the Lua script for half-shutter and a recent nightly lua_fix build of ML, the FRSP + focus stacking works without issues even when Wait=ON in Focus settings. (That was critical, as with Wait=off focus stacking was all wrong). My 3D scanning (photogrammetry) gadget now works for small objects requiring focus stacking for each angle of the object. I wrote a bash script which on a Windows PC (under Cygwin) automatically performs focus stacking for all groups of N shots using Zerene stacking software.

One remaining issue is the extra duplicate shot at the beginning of focus stacking sequences. Unfortunately it doesn't happen all the time - probably ~50% of the time, so I cannot write script which would get rid of the duplicates automatically. Time codes are of no use - duplicates are spaced in time with the same interval as normal shots.  It is not a huge issue, as I can find and delete the duplicates visually.

I just shot ~1000 shots of a small object (~4cm high), resulted in 105 shots after focus stacking (which worked perfectly). I am now trying to make a 3d model out of it...

It's Canon 6D. I will do the testing later today. The focus stacking + FRSP issue I had was on the mainstream ML builds - first on 1 year old build, but also on the up-to-date one. I haven't tested this on the lua_fix build.

I don't see why this would change anything regarding FRSP; this option only affects the delay between focus commands.

I am not quite sure how, but with the Wait ON (default 10, I also tried much larger values) focus stacking didn't work at all with FRSP (both DNG and MLV). I struggled for a while until I discovered that setting Wait to OFF resolves this issue. It's almost like with the Wait ON the focusing move took too long time, and something in the focus stacking algorithm timed out.

I figured this out - I had to place the earlier scriplet inside the second one. I also had to reduce the sleep time to 1s (it didn't work reliably with 2s). Here is the complete script:

Code Select Expand


--Stacking
function on_keypress(key)
    if key == KEY.HALFSHUTTER then
        menu.close()
        sleep(1)
        menu.set("Focus Stacking", "Run focus stack", 1)
    end
end

event.keypress = on_keypress


It seems to work reliably. It does produce an extra shot at the beginning, but this is easy to take care of during post-processing. Using DNG format FRSP on my Canon 6D + 135L with an extension tube, the time it takes to run the full focus stacking sequence (including the rewinding part) is

dt_seconds = 11 + 7.77*N

Here N is the number of shots taken (including the first one; I am not counting the extra shot at the beginning). With N=10 it takes 88s; with N=20 it takes 167s. I used focus stacking step=10, 0s Start Delay, and the default Step size=2 in Focus settings. The exposure is 0.4s.

The real test will be running focus stacking 40 times in a row, automatically (controlled by my 3D Scanner gadget). I am concerned about the possibility that the starting focus point will be drifting - because I set Step Wait to OFF in Focus settings (that was the only way to make focus stacking work with FRSP).

Any further pointers? I am stuck trying to figure out how to fix the script.

Alex: I tried your second script - copied it as is to a *.lua file, ran it from my camera (latest lua_fix ML build), and when in Live View I point the cursor in ML menu to Run focus stack , then briefly press half-shutter, the focus stacking sequence is executed. At the end I see the Live View on the screen. So far so good! But when I try to repeat the sequence by again pressing the half-shutter, it only takes a single FRSP shot, then displays it on the screen with wording "Waiting for long half-press". So the second focus stacking sequence is not taken. To make it work again, I have to use camera's controls to bring the ML menu back, and point the cursor at Run focus stack.

What am I doing wrong here? It feels like I'm almost there, just a small piece is missing.

Here are the details: exposure 1/4s, Expo Override On, ExpSim On, FRSP with DNG.

To tell you the truth, this is so far the only good scan I had - likely because it's fairly large (~25cm high), so DoF wasn't an issue, plus it has lots of texture, which really helps in photogrammetry. I also tried smaller items (5-10cm high), but narrow DoF would result in poor quality models. Now that I modified my Arduino code - so it can do non-continuous shooting, making focus stacking possible - I am hoping to get much better results with smaller items.

I don't have any raw images (it's been a while), but I do have the 3D model as a PDF file (you'd need a recent Adobe PDF reader to see it as a 3D model):

https://www.physics.mcmaster.ca/~syam/statue3d.pdf

There is an option there to choose what to see (solid, wireframe etc).

Thanks a bunch! This is a Lua script, correct? How do I use it? Just enable Lua module, and place the script in ML/SCRIPTS? And what exactly you mean by "you just need to run this on half-shutter press"? Do I still need to add more lines to the script?

Regarding photogrammetry - one can start from the wikipedia article, https://en.wikipedia.org/wiki/Photogrammetry . I have some experience using an open source software, VisualSFM , but got better results with PhotoScan. Here is my test scan of a 20cm high statue, using ~100 photos (made with my 3D Scanner gadget - FRSP + flash):



This doesn't work with smaller objects (say 5cm high) because of the limited DoF - hence my need for automating focus stacking with ML. I want to 3D scan some of the clay figures made by my daughter, and then to 3D print them.

I am trying to use the ML's focus stacking feature (along with full resolution silent picture, FRSP) to automate photogrammetry (making 3D models from lots of photos of the object from different angles) for small objects (where the DoF is too narrow, and one has to use focus stacking for each angle of the camera). This is operated by my own 3D Scanner gadget based on Arduino:

https://pulsar124.wikia.com/wiki/3D_Scanner

The official code for the 3D Scanner only supports continuous shooting mode (the platform with the object continuously rotates, while camera is taking pictures with the flash used to freeze the motion). I recently modified the code, and now it can be used in a non-continuous fashion: it turn the platform a bit, then stops, and half-presses the camera's shutter button to do the shot, then it tuns again and so on.

This works fine with regular shooting (with no focus stacking). It would also work with focus stacking - if only I could figure out how to trigger ML's focus stacking from my Arduino gadget. Right now the gadget can operate half-press (AF pin) and full press (shutter pin), via opto-couplers.

Now I am stuck. I already discovered that focus stacking works great with FRSP. (One twist was that I had to set Step Wait to "off" in Focus Settings to make it work.) But I can't figure out how to trigger focus stacking from Arduino. The stacking can be triggered with the Set, Play, and Left Arrow buttons, but none of them can be controlled remotely. If I half-press the shutter button while the cursor is pointing at the Run Focus Stack option, the camera takes the regular FSRP photo, no focus stacking happens.

Is there a trick or special setting (either in ML, or camera's menu) I can use to enable remote triggering of focus stacking? If not, perhaps there is an easy ML code fix to make this happen? (For example, when the cursor is pointing at the Run Focus Stack option, Half-Press button is reassigned from triggering FRSP to starting a focus stack.) Or may be a simple Lua script can do this?

BTW my camera is Canon 6D mk1. I am using the latest stable ML version (from July of this year).

Thanks for any advise!

One more reason to love the FRSP feature - I recently designed and built a 3D scanning assistant (for photogrammetry processing). It rotates a platform while triggering FRSP shots with external flash(es). The full description is here:

http://pulsar124.wikia.com/wiki/3D_Scanner

It works great. FRSP really saves the shutter life of my Canon 6D, as one needs hundreds of shots for a quality scan.

Here is my first test scan, made out of 150 FRSP shots from 3 different camera angles (Canon 6D + 50mm f1.8 STM lens + two YN560III flashes inside softboxes), of a 20cm tall statue - no artifacts, high 3d resolution. (But it took many hours to process on my PC):

Thanks for the details, things are quite a bit more complicated than I imagined!

In my case, silent pictures are triggered by an external switch (optocoupler connected to Arduino). I thought CUSTOM_PICTURE_TAKING can help me to detect these events - was I wrong? Perhaps key() function is the one I need to use here?

Pressing full-shutter while taking FRSP still results in a "normal" picture taken, as it used to be? It just occurred to me that I could use one more optocoupler (connected to the SHUTTER pin of the camera) to let my arduino rig communicate to camera when it has to move focus. But I suspect I'd have to be careful with timings - perhaps moving focus can only be done at certain times (e.g. not when FRSP image is recorded).

My setup for 3D scanning using FRSP (see above) seems to work well, but one important feature is missing. For this kind of shooting (object size between 2-20 cm), DoF is fairly narrow, so one has to gradually shift the focus while shooting, and have a few full rotations of the object, to have between 2 to 10 focus positions for each full rotation. (So the 3D reconstruction software will be effectively doing focus stacking while reconstructing). To make it repeatable and automatic, a ML script would be needed which (a) detects when a FRSP picture has been taken, and (b) after each N such shots, it moves the focus by a predefined number of steps.

I've never done ML scripting (I am looking into it now), but it looks like a combination of custom_picture_taking() and focus() functions should do the trick - am I right? I can't find much documentation on custom_picture_taking() - can you point to any, or perhaps to some example scripts? Thanks!

Indeed - I didn't realize it wasn't done yet. I just tried the newest build, and it worked. Thanks!

Success! I managed to use an external flash with FRSP on my 6D, using pretty much the same setup as when I used my Arduino focus stacking macro rail with 50D. I just ran a 100-shots test, and there were no issues (black or partially exposed frames). This will be used to do photogrammetry (3D scanning from multiple images).

Here is my setup:

Untitled by First Last, on Flickr

The core is Arduino UNO running my simple code. It drives both the stepper motor via the stepper driver (which rotates the scanning platform, not shown here), and the small breadboard I just put together, with two optocouplers (PC817 - got 50 of them for 1$ from China), and two limiting resistors (220 Ohm). Also on the photo you can see one of my RF flash remote triggers Yongnuo (I have a bunch of Yongnuo 560III flashes in my home studio which can be triggered simultaneously with this trigger). The two connectors are to the stepper motor (white) and camera (black). The whole thing (including  the stepper motor) is powered by a single 12V power adapter.

As an initial guess I tried exactly the same timing settings which worked well on my old camera (50D),  and surprizingly they worked perfectly on 6D, despite 6D having twice slower write speed (40 MB/s vs 50D's 80 MB/s), and slower sensor read speed (0.3s vs 0.15s -ish). I used shutter speed 0.3s.

t=0.0s: shutter half-press initiated
t=1.1s: flash is  triggered (for 0.1s)
t=2.0s: shutter half-press released
t=6.0s: end of the cycle

As I speculated before, perhaps it is the fact that I am (half)pressing the shutter for quite a while (2s) that stabilizes the system, and results in highly accurate timings for the images. (SO that a constant delay works perfectly fine for the flash.)

Now back to the interesting stuff - making 3D scanning work...

Danne: your mlv_dump.exe worked, thanks a bunch! I just copied your mlv_dump.exe inside my images folder, run it from there, and it worked without issues, for both small (5 shots) and large (144 shots) MLV files I made with 6D.

Thanks, I'll try this now. I wonder why I got the odd number of pixels - perhaps it's normal for 6D?

EDIT: g3gg0: I am still getting the same behaviour, with your latest build. BTW the program doesn't crash on me - it just runs too fast, produces no error messages, but also no DNG files. I am using Windows 7 64 bit.

I  just upgraded to 6D (from 50D), and noticed that I cannot open MLV files made with 6D, using the tools which worked fine with 50D (a year ago). I am using a recent nightly build (magiclantern-Nightly.2017Aug18.6D116), and load module mlv_rec and silent. I take full resolution silent pictures. There seem to be no issues taking the photos, the file size seem to be right, but neither mlv_dump nor MLV Mystic work for me. The Mystic would run way too fast, and there would be no DNG files produced. mlv_dump -v seems to report no errors, runs a bit longer (prints the exif info), but then again no files are generated.

Is it a known issue? How could I troubleshoot this?

Here is one of these files (5 shots; 177 MB):

http://syam.no-ip.org/pub/11370000.MLV

Perhaps I don't understand how to use mlv_dump? My command is

mlv_dump --dng 11370000.MLV

Thanks!