Light Trails mode

[pursehouse]

I'm not sure if this falls under the category of "real time video processing" or not... but hopefully not

I used the Sony A7S recently, and it has a "light trails" feature.

Basically what it does, is go into bulb mode and make it so that during each scan of the sensor, only pixels that are brighter than the previous scan are added to the photo, instead of globally adding brightness to each pixel.
So it starts as black (obviously), and the first scan gets the scene with your ISO/AP settings. Then each sequential scan will check for pixels that are brighter than the current compiled pixel set has, and brighten those pixels accordingly.
The photos I did with it are amazing and impossible without this sort of new-age tickery. Canon is always so far behind with the present day features in their prosumer lines. Hopefully you geniuses at ML can make it happen

Here is Sony's video that demonstrates the feature

https://www.youtube.com/watch?v=COAb3II1PSs

[dmilligan]

I would say this falls under the category of "things that can be done in post". Simply take individual exposures and then stack them in post with a maximum function. You can do this in Photoshop by loading all images into layers and then setting the blending mode to "Lighten". The results will be identical.


Features that are NOT possible - please don't request them

Things that can be done in post
Why spending development time on things like in-camera HDR? Magic Lantern is not a replacement for Photoshop

[jose_ugs]

dmilligan just stole my whole response away

[dmilligan]

I might add, you can do even better in post by doing something like this:

Stack the frames with an average() function (many astrophotography programs can do this) to reduce noise and get more DR, then blend that with a maxium() stack (as described above) using a mask based on the luminance. This will add back in the brighter star trails you get from using the maximum technique, while reducing noise in the rest of the frame. Now you get the best of both techniques: less noise from averaging and more prominent star trails from maximum

[pursehouse]

Sorry I guess I should have clarified... this feature is not related to star photos. It's more for similar things to car-light streaks that were demo'd in the Sony video.

As far as comparing this feature to post-compiling a series of photos...

Imagine you have a light trails (/painting) situation going on. Which is of uncontrollable fast moving items ( for instance cars, people, trains ). Let's assume time span needed to get the photo is 6 minutes ( as was the case in a photo I did with the Sony ).
So these fast moving items are racing across the screen. In the case of the normal series of photos, the camera would take a picture, delay a moment, then another picture and repeat.
So in the repeated photos style if you chose to take slow shutter photos for each shot.
You could take a 10 second photo, then the camera will shutter, buffer the picture, and start the next picture. In that moment you just missed movement of the scene. So there will be a gap of missed movement between each of those shots, even though they are long.
If you took fast shutter photos instead to try and make it so that there are even gaps.
You would have to shoot extremely fast in order to get any sort of steady (although still gaped) movement. let's say.. 30 frames per second, at least. And the last I checked... there are very few cameras able to get anywhere near that, let alone sustain that speed for 6 minutes.

The closest thing you could do to come even slightly close to this, is to film raw video ( extremely limited in resolution and a temperamental feature ), then adjust those frames, then compile.

As far as the noise reduction is concerned... that is already handled implicitly by the maximum(a,b) happening on every sensor scan cycle. So (without confirming) I'd guess most of the noise would already be taken care of, if not more, because the sensor would be getting far more data than any compilation of photos could ever get.

Trust me guys, I thought about it a lot The in-camera processing of each scan frame and controlling how it is added to the compiled final image is a huge feature that unlocks an unreal difference in abilities. Specifically this style

Sony does have a star trails mode too, but I haven't looked into it or used it yet.

So does anyone know if this feature is even possible with what Magic Lantern is able to access/accomplish?

thanks

[g3gg0]

remove the tag "invalid" please.

its probably something for FRSP - if we could reduce the lag. (yet this is the only problem atm)
keeping that in mind when working on FRSP

[Audionut]

Plenty of people take images of car light streaks currently with current functionality.  So I'm not really sure how this is useful.

[g3gg0]

sometimes you cannot go up with f/ and down with iso or cant afford a ND filter to expose 1 hour of bright stars.

[chris_overseas]

I can certainly see the use of this, though if consecutive shots can be taken without any lag between the two then I don't think there's anything further for the camera to do that couldn't be done in post instead. Note that the 5D3 has some blending modes built in to the stock firmware already, though seamless light-trails currently can't be done because of the lag between each photo.

If blending was to be done in camera, presumably this would be the way to go: http://www.magiclantern.fm/forum/index.php?topic=13408.msg129483#msg129483

Audionut: It's useful because you don't overexpose the static parts of the image when blending in this way, compared to a single long exposure.

[g3gg0]

if you want to take 10 hours of 5-sec per frame footage in FRSP images into a MLV, that one picture will take a lot of space.....
so i can think of making it a good addition to "merge" frames

[DeafEyeJedi]

Interesting points guys... Definitely would be a nice addition to FRSP regarding "merge" frames!

I'll definitely follow this thread... Thanks @pursehouse!

[pursehouse]

Great, I'm glad you guys are interested in the feature

Yeah @chris_overseas , that EekoAddRawPath/TwoAdd using max(a,b) does seem to be exactly what I was talking about. I am assuming the image a/b they reference are single scans of the censor during the shutter being open?

And like @g3gg0 said, even if the frames were shot fast enough (which is impossible as far as I can tell), the space it would take to store them would be astronomical. Doing it in-camera before output to the file system opens up a whole new world of possibilities.

Perhaps if this feature was simply referenced as a "frame scan mode" (or something like that) with the user-available options listed on the page Chris linked ( additive, subtraction, max, min ). And if that existed, it could even open the door for having more complicated things like a time based mathematical curve option added to it to hide end points of motion... stuff like that. Endless possibilities

Also, this feature would make photos of lightning MUCH easier. I've used the motion trigger feature for lightning, and while being a great feature, it is still extremely difficult to get the right settings/setup for the changing and limited situation fast enough

Thanks everyone!

[pursehouse]

Oh, here is the train trails photo I took. This is an unedited export of the raw.

The shutter was open about 6 minutes, because that's how long of a time span there was between the two left/right trains passing me.

details: 15mm, f/4, ISO 100

http://garandy.com/t/japantrain1

As you can see, the area was pretty bright and extremely contrasted with the boarding platform lights on the right, the dark sky, and the building lights.. etc... And of course, the trains are moving pretty fast in the frame when they are passing near the camera.

I'm going to try a few light painting projects using this Sony feature as well. In semi-lit city areas, where hopefully the person-shadows will be a non-issue because of how the max() processing style

[DeafEyeJedi]

cool pic. did you use an ND filter?

If not, then how would you able to manage to keep it from getting overexposed during 6 minutes of open shutter?

especially @ f/4...

Thanks for sharing @pursehouse!

[pursehouse]

@DeafEyeJedi
no filters or anything. That's the beauty of the max(a,b) feature. It only brightens pixels that change between sensor scans, not brightening all pixels over time just because the shutter is still open. So, you could light paint in direct sunlight if you were able to darken your input enough ( probably need an ND for that ) and then had a light source bright enough. Now I have to try that out, light painting in the sun

[DeafEyeJedi]

Sounds marvelous!

Regarding the Sun painting -- You should use a ND or at least a UV filter otherwise...

[emoji6]

[pursehouse]

So what does it take to get this feature further into the likelihood of actually getting worked on by the people that know how to do it?

[dmilligan]

Nothing.

It's actually already been discussed before you even requested it:
http://www.magiclantern.fm/forum/index.php?topic=13408.msg129520#msg129520
and here:
http://www.magiclantern.fm/forum/index.php?topic=10176.msg98049#msg98049

Basically all this is going to do for you is save some storage space by not needing to save every frame. That's all we're really talking about here.

Right now if you want to do something like this, you need to capture every frame and do the blending in post.

Here's an example I did using raw video:
http://www.magiclantern.fm/forum/index.php?topic=12392.0

That was before FRSP, so I suppose you could do something similar at full resolution with FRSP, if you can make your shutter speed long enough, then the previous image should be done saving by the time the next image is ready, so there will be as small as possible time between frames (though it will still be significant, I think when I implemented a sort of FRSP 'live-view' for the luminance trigger, the best I could was about 90ms between frames IIRC).

I would love to play with the EekoAddRawPath stuff alas, I only have DIGIC 4 cameras:

Problem: these functions are only available on DIGIC 5 cameras.

[pursehouse]

@dmilligan

The delay between pics (any delay at all) makes normal merging photos style an unusable option for what I am talking about.
Things like waterfalls which repeat over again are simple to deal with, just like stars are easy because of their slow motion. This is for full motion scenarios. Even video (I assume 60fps) would be far to slow for this.

And although space is an issue, because  a single raw file would be around 40mb, and 6 minutes of HD RAW 1920x1080x24fps video would be 29,880gb ( 74,700 at 60fps ). And that is a very low resolution for quality photos. Then you just spent your day setting up to take one photo per expensive card you have capable of handling that speed. And last I checked, 60fps RAW at 1080p was stretched up from a lower height and barely doable on my 5dmark3 + 1000xCF card.

And longer shutter speed without the maximum() processing goes against the effect of this feature.

But yes, the Eeko multiple features is exactly what I'm looking for, as we were talking about earlier in this thread

[dmilligan]

The delay between pics (any delay at all) makes normal merging photos style an unusable option for what I am talking about.

Then what you are talking about is impossible. There's always going to be some finite delay between photos. However, I disagree that "any delay at all" makes what you are trying to do an "unusable option".

But yes, the Eeko multiple features is exactly what I'm looking for

No it's not. I think you have a misunderstanding about what it does. All the EekoAddRawPath stuff does is merge two already captured frames (the main advantage is that it is hardware accelerated so it can do the merge faster than we could with the CPU).

Even video (I assume 60fps) would be far to slow for this.

Why don't you stop making assumptions and actually try it.

[pursehouse]

It's weird that you are the only one arguing against this feature. There is a reason Sony has it in all their new cameras. And the feature concept is to add to sensor scans, not photos. And storage speed/size limitations make mimicking this feature in post impossible. I'm fairly certain I explained that multiple times.

[a1ex]

@pursehouse: I totally agree with what dmilligan said here, and I bet the other devs do the same.

And he's not arguing against this feature, FYI.

[pursehouse]

ok so, you guys are the experts with this code base/exposed canon functionality.
Is it possible to override the way that the sensor adds to the image inbetween scans?
So that instead of the ( pixelResult += newScanPixel ) with I assume an internal maximum set during the process as opposed to when the buffer is output as a frame, instead it could be set to do ( pixelResult += max (pixelResult,newScanPixel) ) ?
If it's not possible to do that, then that answers my original question. Although I still consider the same ability in final frames to be worth it for many situations. Particularly the fact that the storage required for some projects makes them a nightmare to do out in the wild or while traveling. But, please, original question, can the sensor scan addition logic be altered?

thanks

[a1ex]

There's no addition logic used when taking regular images (just hardware integration). So, there's nothing to alter.

[g3gg0]

take a bucket
hold it under a water-tap
let water flow into the bucket

now explain me how to do max(water, tap) of that constellation