Depth map generation using focus stacking

Started by Karma Lies, December 17, 2019, 08:36:14 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Karma Lies

Hi! This is my first post, but I've been a longtime user of ML. I just graduated from Purdue with a degree in computer science, where I researched the concept of generating depth maps for a scene using a DSLR. My vision is to be able to take an input stack of images as DNGs which are then processed into a single, all-in-focus DNG with an embedded depth map. That could open up the creative post-processing potential for focus stacks in RAW editors like Photoshop/Lightroom since those programs already support the use of embedded depth masks.

At the project's current state, there's quite a bit of room for improvement in polishing up the generated depth maps. Although, just a few improvements (i.e. image segmentation, Homography/ECC alignment) to the core pipeline of the project could improve the quality of these depth maps to near perfection. The algorithm is currently fed by JPEG inputs for ease of fleshing out the core of the program, but I want to eventually integrate DNG processing for the above reasons.

The work I've done so far is standalone to ML, but I would be interested in involving my project in some ML development (if deemed justifiable or necessary beyond the current focus stacking module).

Down the road, I would even like to integrate depth by defocus algorithms to generate a depth map from 2 or 3 shallow depth-of-field shots. That would unlock some potentially game-changing portrait post-processing capabilities.

Feel free to check out the project's GitHub repository: https://github.com/tonyketcham/Depth-Mapper

I would love some more contributors in manifesting this vision. To my knowledge, I haven't seen any other open source projects like this for creatives -- only a couple proprietary depth-map generators and then some algorithms described in academic papers without any source code.

yokashin

70D.112 [main cam] | M.202 | S110 [CHDK]

histor

Welcome and... you have some real giants to compete with. Zerene Stacker is definitely the first one to mention.

names_are_hard

Cool project, and congratulations on graduating :)

I skimmed your code, it looks approachable.  It's clearly producing usable depth maps!  I like macro work a lot so I could use a nice automated focus stacking workflow, I might have a play.  The demo image I would guess has some artifacts from Jpeg compression, especially obvious around the head of the panther figurine.  The white aura means it thinks there's pixels close to the viewer, right?  The shapes there look like Jpeg spatial quantisation to me.  The random noise in the lower-right corner also looks funky though, and that's probably unrelated.

What were you hoping to gain from ML integration?  I can see some angles, ML can control focus and customise info embedded in the image.  Focus peaking info might be useful to embed.

You may well enjoy the work of Levon Biss:
https://levonbiss.com/macros/macro-information
http://microsculpture.net/

ilia3101

Amazing!!!

I know this is not strictly ML related, but I am glad you came here. This forum is a great place for open source image processing.

chris_overseas

EOS R5 1.1.0 | Canon 16-35mm f4.0L | Tamron SP 24-70mm f/2.8 Di VC USD G2 | Canon 70-200mm f2.8L IS II | Canon 100-400mm f4.5-5.6L II | Canon 800mm f5.6L | Canon 100mm f2.8L macro | Sigma 14mm f/1.8 DG HSM Art | Yongnuo YN600EX-RT II