Newbie: scripting for astrophoto, o

[gerlos]

Hello everyone!
Going to do astrophotography in the next weeks, using my Canon 550D. Last time Magic Lantern helped me a lot: I used time lapse feature to setup an automatic shot sequence and leave it go while I was doing something else (like sleeping).

This time I'm planning to shoot M31, the Andromeda Galaxy, and need several different exposures to get it right.

I remember that months ago someone here talked about a scripting feature, not present in the current stable, that could help me in this project. I actually need to do something like this: 10 shots at 600 sec with 30 sec of pause between each photo, 10 shots at 300 sec, and so on. Perfect for a script!

Is this possible with recent nightly builds? Is there any reference for the scripting language used?

If scripting isn't possible at the moment, is it possible to change the minimum value for the "stop after" option of the time lapse tool? 100 frames are really too many for my needs, I actually would be happy with 10, 15 or 20 frames.

thanks in advance
gerlos

[dmilligan]

I have created a module that you could potentially do something like this with, but I haven't implemented the bulb timer on it yet so you can only go to 30". I'm working on it though and hopefully have bulb timer working soon.

http://www.magiclantern.fm/forum/index.php?topic=8431.0

[SpcCb]

gerlos, you can write a script with the PicoC scripting API to do this.
-> http://www.magiclantern.fm/forum/index.php?topic=4362.0

But I'm sure it's the most simple way to do what you want.
The 30s between shoots because of dithering?
And what about darks in this sequence?


PS. 300s looks to much for the core IMHO, face to 600s. Make a try with 180s and I'm sure you find it better.

[gerlos]

gerlos, you can write a script with the PicoC scripting API to do this.
-> http://www.magiclantern.fm/forum/index.php?topic=4362.0

But I'm sure it's the most simple way to do what you want.

Thanks both for the fast answer! I'm looking at PicoC, and it seems to fit perfectly my needs.
Just to be sure: to use it, I can just pick latest nightly, is it right?

The 30s between shoots because of dithering?
And what about darks in this sequence?

Right. During the pauses, I'll move slightly the mount to do dithering. I'm also looking for a way to do it automatically with my SkyWatcher Synguider, with no luck at the moment, but I don't give up hope (also sent a mail to SkyWatcher asking for this feature, low probability of success, but it worth a try).
Pauses help also with sensor temperature, since during the idle time it cools a little.

PS. 300s looks to much for the core IMHO, face to 600s. Make a try with 180s and I'm sure you find it better.

Actually, I'm going to do some experiments, since the 250 lens I'll use is not very good. I'm still unsure if I could use it at f/4, f/5.6 or f/8, that seems my best option for sharpness, but it's really too dark.
I'm planning also several sequences at different exposures, to span 3 or 4 stops of exposure (maybe 60s, 150s, 300s, 600s) so I can pick the best settings for my setup.

Now I'm going to do some experiments, and wait for a clear new moon night!

thanks again
gerlos

[gerlos]

Just downloaded latest nightly and put it on my sdcard.

Unfortunately, there not seems to be any scripting menu Is this feature been removed? Or I need to load some module (first time I see them on my camera).

Looked also at dmilligan's post about adv_int.mo module, but didn't understand how could I add this module to magic lantern (I don't see it in the modules menu). Anyway, the lack of bulb mode makes it less useful to me.

But, good news: latest nighly's intervalometer lets me set any number I want in the "Stop after" option! Not a perfect solution, but better than before.

Also: kudos for the latest nightly, its look is a lot more polished than my old stable!

[wolf]

I think scripting is still broken.   
PicoC will be replaced by tcc.
You could try a older (3 or 4 month?) ML version.

[SpcCb]

(...)
Right. During the pauses, I'll move slightly the mount to do dithering. I'm also looking for a way to do it automatically with my SkyWatcher Synguider, with no luck at the moment, but I don't give up hope (also sent a mail to SkyWatcher asking for this feature, low probability of success, but it worth a try).
(...)

Hoosh... Very hard.
It was why I ask about dithering.

You will have to sync dithering between shoots, so you will need a special firmware for the SW SG (Synguider) and add a plug in the SW SG to drive the bulb of your camera.
Other option, set the SW SG as a slave device, with a mod FW too, and use a master device like Arduino to drive all the setup.

And in all options, I don't see how make a script in the camera to drive something outside with in/out informations (sync)... Maybe with flash on second curtain + timming (?). It looks hard.

Note; I do dithering without guiding on a EQ6 for short focal length, the most simple and reliable solution I found is to drive mount and camera with a small Arduino. But for dithering and auto-guiding I use a laptop (EEEPC), it's the most simple (+ cheap) way.

[JCBEos]

what you wanna do (composite different exposure to get the galaxy core not burn for ex.) is quite similar to HDR... have you simply tried with advanced bracketing?

[SpcCb]

what you wanna do (composite different exposure to get the galaxy core not burn for ex.) is quite similar to HDR... have you simply tried with advanced bracketing?

Indeed, it's like a HDR.
But in astro-photography, regular photographic bracketing is not relevant because we need a very high S/N ratio (many shoots of the same bracket).

[gerlos]

You will have to sync dithering between shoots, so you will need a special firmware for the SW SG (Synguider) and add a plug in the SW SG to drive the bulb of your camera.
Other option, set the SW SG as a slave device, with a mod FW too, and use a master device like Arduino to drive all the setup.

And in all options, I don't see how make a script in the camera to drive something outside with in/out informations (sync)... Maybe with flash on second curtain + timming (?). It looks hard.

It's true, it's hard, so it's not my priority at the moment.

At the moment, I just start autoguiding, take a shot, manually stop auto guiding, move slightly the mount, so the star on the screen of the guider moves from one side of the frame to the opposite side, start guiding again, and take another shot, and so on. Sometimes I don't need to even manually hit buttons, since when I stop guiding the star starts to oscillate around (maybe because of periodic error).

It would be wonderful to automate dithering, but how? Maybe I could just start guiding and acquisition sequence on the camera, and set a timer somehow to stop autoguiding every, say, 6 minutes, during the pauses of the acquisition (commands GUIDE -> STOP and GUIDE -> RESUME).

Note; I do dithering without guiding on a EQ6 for short focal length, the most simple and reliable solution I found is to drive mount and camera with a small Arduino. But for dithering and auto-guiding I use a laptop (EEEPC), it's the most simple (+ cheap) way.

Interesting. Can you give me more details about it, or a link to read?

regards,
gerlos

[dmilligan]

To use my module simply download the adv_int.mo file linked to in that thread and add it to the ml/modules folder on your card.

Personally I think it's better to spend as much time as possible "collecting photons" rather than stopping and waiting for the camera to cool or taking darks, and then just making sure I get darks later for all the various temps my camera was at and properly calibrate my frames. Usually the next day after a night of shooting I look at the temperature in the exif data and shoot darks for every unique temp and shutter length that I see. I get my camera to the temps it was at by putting it in the fridge to cool it down or running liveview to warm it up. This way I don't waste any time I could have spent exposing (or sleeping). But if you have some evidence that giving the camera a break to cool is actually better, I'd certainly amend my current process.

It's possible for ML to send out signals through various means. You can send out data over the USB with PTP, also for something more simple, turning on/off the audio preamp causes a detectable signal in the audio line in. Don't know if that would be helpful for your dithering. I cannot afford an autoguide setup, so my cheap eq mount's periodic error does this for me :p The draw back is that I could never hope to get 600s like you mention, 120s is my practical limit and that is usually only for small declinations, and if I spend a lot of time getting the polar alignment perfect.

[JCBEos]

Indeed, it's like a HDR.
But in astro-photography, regular photographic bracketing is not relevant because we need a very high S/N ratio (many shoots of the same bracket).

I know that 

(
http://starshoot.over-blog.com/article-le-gso-rc-8-sort-du-bois-71292056.html
http://jcbphoto.free.fr/astro/M51_110411_version_01.jpg with my RC and NEQ6
)

Just thought that it was possible to combine intervallometer and bracketting, by this I mean take -1/0/+1 shots at "T" time, then the same at "T+1" etc

 

[SpcCb]

(...)
At the moment, I just start autoguiding, take a shot, manually stop auto guiding, move slightly the mount, so the star on the screen of the guider moves from one side of the frame to the opposite side, start guiding again, and take another shot, and so on. Sometimes I don't need to even manually hit buttons, since when I stop guiding the star starts to oscillate around (maybe because of periodic error).

I see, you do manual dithering. It's nice
But don't forget what if you want a good dithering, displacement have to be 'randomized' in both two dimensions and not only in the RA axe (because PE).

It would be wonderful to automate dithering, but how? Maybe I could just start guiding and acquisition sequence on the camera, and set a timer somehow to stop autoguiding every, say, 6 minutes, during the pauses of the acquisition (commands GUIDE -> STOP and GUIDE -> RESUME).

With AG (auto-guiding) the scenario should be:
1 start >> 2 AG is running >> 3 start shoot >> 4 stop shoot >> 5 stop AG >> 6 random displacement >> 7 start AG >> 8 AG stabilized >> GoTo 3
It needs communication to sync all events because sometimes step 6 or 8 takes 10s, but sometimes it takes 30s. Plus, set intervals in different devices is hard to be sync after 1 or 2h runing; it could be out of sync for 1 or 2s so you ruin your shoot.

Interesting. Can you give me more details about it, or a link to read?
(...)

For AG + dithering with a laptop you can find a couple of software what can drive mount, imaging camera, guiding camera and more (focuser, etc.) via a standard protocol: ASCOM. With that you can automate all your night.
Example, with a light mobile setup for wild field you can use software like PHD Guiding or Guidemaster after installed the ASCOM platform. Many astronomers use this kind of setup.
Sorry to not be prolix but I'm not sure what it's the right place here to detail all parts.

(...)
Personally I think it's better to spend as much time as possible "collecting photons" rather than stopping and waiting for the camera to cool or taking darks, and then just making sure I get darks later for all the various temps my camera was at and properly calibrate my frames. Usually the next day after a night of shooting I look at the temperature in the exif data and shoot darks for every unique temp and shutter length that I see. I get my camera to the temps it was at by putting it in the fridge to cool it down or running liveview to warm it up. This way I don't waste any time I could have spent exposing (or sleeping). But if you have some evidence that giving the camera a break to cool is actually better, I'd certainly amend my current process.

I'm totally agree with you: signal is the most important.
But where signal is, noise is. So we need to process images with darks, bias, etc. if we want to get 'only' the signal ('only' is a dream).

Make a dark library is very interesting, specially for 600 or 1200s shoots (in some case night is too short to get lights + darks).
But (I'm sorry to say 'but' all times), its very hard to get 'similar' darks (you can do the test with two darks at the same temp, one from 3 months ago and one just right now). The only thing what is a bite 'stable' is bias.
About temp rising during shoots, it's important to use the dithering time to cool the CMOS, specially with big 24x36 CMOS. Take a look in this interesting page, on the bottom

It's possible for ML to send out signals through various means. You can send out data over the USB with PTP, also for something more simple, turning on/off the audio preamp causes a detectable signal in the audio line in. Don't know if that would be helpful for your dithering. I cannot afford an autoguide setup, so my cheap eq mount's periodic error does this for me :p The draw back is that I could never hope to get 600s like you mention, 120s is my practical limit and that is usually only for small declinations, and if I spend a lot of time getting the polar alignment perfect.

It would be amazing to use ML to drive dithering, bulb shoots, etc. !
I think it's possible (if we could send orders on ST4 guiding protocol etc.), but wahoo... It's a big project.

(...)
(
http://starshoot.over-blog.com/article-le-gso-rc-8-sort-du-bois-71292056.html
http://jcbphoto.free.fr/astro/M51_110411_version_01.jpg with my RC and NEQ6
)
(...)

Woho, I see we are some French astronomers here


Edit: corrected links

[gerlos]

Usually the next day after a night of shooting I look at the temperature in the exif data and shoot darks for every unique temp and shutter length that I see. I get my camera to the temps it was at by putting it in the fridge to cool it down or running liveview to warm it up.

Really smart. Thanks for sharing.
So far I used a simpler workflow, since I take dark frames during twilight and during dawn, when the sky is brighter, and sometimes a couple of them in the middle of the night, when my subject changes side of the meridian and I have to rotate the mount.
I'll try your approach, maybe this way I'd get better calibration.

But if you have some evidence that giving the camera a break to cool is actually better, I'd certainly amend my current process.

It works. I've somewhere the results of an experiment done during a cloudy night, when I find them I'll post them.

You can send out data over the USB with PTP, also for something more simple, turning on/off the audio preamp causes a detectable signal in the audio line in.

Good to know. Maybe we could program and arduino board to capture such signal and do something cool. ;-) Need to tinker a little with it...

I cannot afford an autoguide setup, so my cheap eq mount's periodic error does this for me :p The draw back is that I could never hope to get 600s like you mention, 120s is my practical limit and that is usually only for small declinations, and if I spend a lot of time getting the polar alignment perfect.

Similar performance on my CG5 before I plugged in the auto guider... My limits were between 200 and 300s. Now I do polar alignment using a polar finder and then using goto software in less than half an hour (made some marks on the mount for easy weight balancing), and the Synguider do the rest.
The problem is that it's "too perfect" ;-) But once I tell it to stop guiding, periodic error and everything else do their job, moving my frame in unpredictable way.

[RenatoPhoto]

But if you have some evidence that giving the camera a break to cool is actually better, I'd certainly amend my current process.

http://www.astrosurf.com/comolli/strum124h.jpg

http://stargazerslounge.com/topic/167076-the-noise-produced-by-a-canon-1100d-at-various-iso-settings-and-temperatures/

[dmilligan]

okay, I didn't really state clearly exactly what I meant, obviously a cooler camera works better, I know that, what I really meant was:

spending time letting the camera cool vs. using that time to collect more signal

lets say you have a total of an hour to spend: is it better to give the camera 15s breaks every 5 mins and end up with 3 fewer minutes of total exposure time, or is it better to just spend the whole hour exposing? Seems to me like it's better to just expose the whole time. Additionally, is there any evidence that giving the camera a break for short periods of time actually lowers the temperature?

[SpcCb]

By experience, with 30s breaks between 600s subs I can see CMOS temp becomes stable after 3~4 shoots. And without breaks CMOS temp does not stop to rise, even after 1h.

[gerlos]

By experience, with 30s breaks between 600s subs I can see CMOS temp becomes stable after 3~4 shoots. And without breaks CMOS temp does not stop to rise, even after 1h.

I confirm this. Here's my figures:

• 6 shots, 300s, no pauses: T1=37°C, T2=39°C, T3=40°C, T4=41°C, T5=42°C, T6=43°C
• 6 shots, 300s, 60s pauses: T1=39°C, T2=38°C, T3=36°C, T4=39°C, T5=39°C, T6=39°C

Did this experiment at home, with higher temps compared to usual astro photography conditions.
For the first group, I set the intervalometer to 5 min and bulb exposure to 5 min. For the second group, I set the intervalometer to 6 min and bulb exposure to 5 min.
Between first and second group I did a pause of ~15 min.

Later I downloaded the shots, and read the temperatures using exiftool.
60 seconds seems to me a reasonable interval also for (manual) dithering and occasional checks to see if everything is running fine.

[dmilligan]

okay, so the camera does cool some if you give it a break, the question now remains, does the improved performance from giving it a break to cool acutally outweight the time you loose collecting signal. In terms of your specific example, here's what I'm asking: which would give the best SNR, 360s @43 C or 300s @ 39 C?

I imagine the answer may actually vary based on a number of factors including ambient temperature, total exposure time, etc. Depending on the situation it may be better to to do pauses, or it may not, or the optimum pause time may vary.

btw, I'm really enjoying this conversation, this is something I had not considered this carefully before.

[SpcCb]

okay, so the camera does cool some if you give it a break, the question now remains, does the improved performance from giving it a break to cool acutally outweight the time you loose collecting signal. In terms of your specific example, here's what I'm asking: which would give the best SNR, 360s @43 C or 300s @ 39 C?

I imagine the answer may actually vary based on a number of factors including ambient temperature, total exposure time, etc. Depending on the situation it may be better to to do pauses, or it may not, or the optimum pause time may vary.

btw, I'm really enjoying this conversation, this is something I had not considered this carefully before.

Indeed, it depends of the local temperature, wind-child, camera model (CMOS size & type), optic & ring type on the camera, etc.

What I can say if it can help is, with a 5D2 on a big refractor with a massive 3" ring, thermal signal rise with factor ~5 during 1h without breaks between subs of 300s. And with 30s breaks between each subs thermal signal rise factor ~2 during the same time.

For light signal during 1h, we can get 12 subs without breaks and 11 subs with, so 1/12th of light signal loose between both.
Synthetic abstract [Signal time / Noise rise]: St/Nr(no breaks) -> 12/5 and St/Nr(w/ breaks) -> 11/2

Ratio with breaks is outright better. More of 2x less of noise for just 1/12x less of signal.
And this is for 300s subs; I get better ratio with 600 and 1200s subs.

[gerlos]

okay, so the camera does cool some if you give it a break, the question now remains, does the improved performance from giving it a break to cool acutally outweight the time you loose collecting signal. In terms of your specific example, here's what I'm asking: which would give the best SNR, 360s @43 C or 300s @ 39 C?

Good question. I don't know the answer, I'd make an experiment.
 
To be picky, teoretically it may be problematic calibrating and stacking several light frames with different temperatures. Anyway, modern processing software, like Pixinsight, adopt approaches that mitigate a lot this problem.

Pragmatically, I'd do pauses between frames anyway, since they fit in my workflow, and help me avoiding errors and problems, or at least spot them easily (I would be afraid to stop a long sequence to check the setup). 
I consider the fact that pauses give me more stable temperatures as a positive side effect ;-)

btw, I'm really enjoying this conversation, this is something I had not considered this carefully before.

I'm enjoying it too, but maybe this OT is going for too long, don't know if moderators could complain about it.

Regards
Gerlos