Magic Lantern Forum

Developing Magic Lantern => Feature Requests => Topic started by: Taynkbot on October 25, 2018, 06:29:50 PM

Title: Utilizing Dual-Pixel Sensors for Depth Map Generation
Post by: Taynkbot on October 25, 2018, 06:29:50 PM
I would like advice on using my dual-pixel (dp) 5D4 to generate a depth map. Here is a paper I found discussing what I believe to be a compatible idea (though I'm not entirely sure if Canon's dp is capable of the differential phase detection outlined here):

https://www.osapublishing.org/DirectPDFAccess/9F568B2B-F2DE-C75B-3D2341DC73BD0F38_344306/oe-24-12-12868.pdf?da=1&id=344306&seq=0&mobile=no

I would like to attempt to generate a depth map manually, in post, from a dp RAW image before getting more ambitious and trying to incorporate it into ML. If it is possible in post, I would like to attempt to automate the process in-camera (though I admit I'm fuzzy on how computationally heavy are the algorithms needed to generate a depth map) by creating an option similar to the option that records both RAW and JPEG versions of an image, but in this case I would like the JPEG to be a generated depth map.
The in-camera option will probably be too intensive for my camera to perform acceptably, but I would appreciate advice and more information.

Do you know of any programs that already exist that can take two images, half a pixel apart, and combine them into a stereoscopic image? Any open source code that might aid me would also be very welcome.


Finally, for the stickler sticklebacks who want to describe how inferior this kind of approach is compared to a properly generated depth map..... I'm not asking for perfection. Also, I don't want to have to buy a Light L16, but I am up for donations of this sort:
https://www.adorama.com/hsh6d400cmp.html?gclid=Cj0KCQjwsMDeBRDMARIsAKrOP7GOGlPCfKPSXxRzODAXCVGLeo77uBuzOx4IZH3zoMB3rk7ZQxo8lS4aAp1_EALw_wcB
;)

Here are some similar posts that didn't explicitly discuss the possibility of exploiting dp.
https://www.magiclantern.fm/forum/index.php?topic=12172.0
https://www.magiclantern.fm/forum/index.php?topic=9094
https://www.magiclantern.fm/forum/index.php?topic=11886
Title: Re: Utilizing Dual-Pixel Sensors for Depth Map Generation
Post by: eduperez on October 26, 2018, 01:16:17 PM
Your idea seems quite interesting, I think I posted an anaglyph made from a dual-pixed photo some time ago in this forum. You just need to get your hands on some dual-pixel RAWs and start experimenting on your computer. However, I do not see any advantage (and I see many disadvantages) on doing this on camera...
Title: Re: Utilizing Dual-Pixel Sensors for Depth Map Generation
Post by: a1ex on October 26, 2018, 04:08:59 PM
Have you guys followed the 5D4 thread? A proof of concept was posted ~ 2 years ago.

https://www.magiclantern.fm/forum/index.php?topic=17695.msg172097#msg172097
https://a1ex.magiclantern.fm/bleeding-edge/5D4/5d4-dual-pixel.html
https://a1ex.magiclantern.fm/bleeding-edge/5D4/dual-pixel/5D4-lv-daf-raw.html

Link to the paper is not working; you mean, this (https://www.osapublishing.org/oe/abstract.cfm?uri=oe-24-12-12868)?

Some source code would be nice, too.