Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - xaint

#1
Quote from: dmilligan on January 28, 2017, 04:53:52 PM
We could easily, but not until the author posts his source code (as required by the terms of the GPL).


Hey @dmilligan, 'thanks' for remembering..

You can find the source code here: source
#2
The functions are in the Focus menu now, so maybe it'll work on other cameras.
(It worked for me with the latest nightly...)

Added the link above.
#3
Updated the link, forgot to add the autoexec.bin  ::) (550D only)
It should work now...
#4
Hello!

I've made a module called astro a couple of years ago, which I never published because of three reasons.

1: the source code is messy
2: it has never been tested thoroughly
3: it stayed in a highly experimental state

But finding it on a VM a couple of years later, I thought it may worth to share (in spite of the fact that the reasons listed above still apply :) )
Someone may find it useful or may improve it, who knows. :)

You can find the source code at the bottom somewhere.






Basically it's got three functions, which can help you setting up your EQ mount for astrophotography:

- camera alignment (by drawing a horizontal line :), really)
- polar alignment (D.A.R.V. or Drift Alignment by Robert Vice)
- star focusing using the half flux diameter formula
- bonus: you can also change the color scheme to dark red

The basic idea comes from a video on youtube (link is below), where the guy uses a laptop, and a software called BackyardEOS to do the drift alignment & star focusing, but carrying around a laptop with all the other gear can be a bit how ya doin'. Since we already have a nice intervalometer built into magic lantern, I thought it would be nice to have everything built into the camera.

Here's the video:
https://youtu.be/zQB6UnrTEEM?list=PLdSieE6m2qgtXhHODxebbiRg5INaL5IpF&t=1693

In order to do the drift alignment, basically you'd have to burn a star's trail into the image using a long exposure, which of course requires the shutter.

Using this module you don't have to release the shutter, it recognises the star (which is used for the alignment) at given points of the screen, then it draws some lines to show it's path.

Something like this:









As you can see, the star arrived a bit above the starting point, so you need to do some adjustments...

(But doing it in BackyardEOS is more precise though, cause you have a high res picture which can be analyzed much better)



And to get an idea, here's how the star focusing works:



Basically, the sharper the star is, the lower the HFD value and the higher the Max pixel value should be.

(by doing some quick tests I've noticed that sometimes the HFD value continues to go down even when we're going out of focus  :-[)

As I've said, it was a highly experimental thing...
Maybe I could fix the bugs, if I had some spare time, or I could just share the source..

If you're brave enough, you can give it a try, here's the module, compiled using the latest build.
I've only tried it on a 550D so be aware. Oh and do it on your own risk! :D

550D only: astro.rar


Here's the independent module (no custom menu, no telescope icon):
You'll find the functions inside the Focus menu.
(According to the compiler: Will load on:  500D, 50D, 550D, 5D2, 5D3, 600D, 60D, 650D, 700D, 7D, EOSM)

Should run on "all" cameras: astro.mo
Never tested though...


You can find the source code here: source


What do you think? Share your opinion!
#5
Homework :D

(The cam overheated, sorry. Today was the hottest day of the summer)

Mode: movie 1920x1080 25p, Global Draw: OFF
SanDisk ExtremePro 16GB



10312704 207
15885312 208
80896 86
8949760 207
28661760 208
16264192 206
24521728 209
16424960 207
13148160 208
15619072 206
1658880 195
21001216 208
8548352 206
19760128 208
5965824 204
24906752 207
13968384 207
16770048 207
4542464 202
13063168 206
33263616 208
30312448 208
16713728 208
3887104 203
21841920 206
12846080 207
16192512 207
3333120 202
6719488 204
17313792 207
23374848 206
2865152 198
32488448 208
18588672 207
4186112 203
821248 185
3661824 204
18359296 208
19875840 208
22754304 208
13665280 206
24957952 208
28555264 206
33442816 208
10708992 204
20242432 207
8774656 206
11216896 204
23156736 208
12750848 205
12369920 209
2157568 195
15279104 205
29943808 209
30321664 206
19004416 203
268288 145
23778304 209
467968 165
1905664 198
32368640 209
19829760 206
26281984 207
24153088 207
11364352 205
32160768 207
5944320 206
24118272 207
28511232 209
31526912 207
15956992 208
1732608 196
13881344 208
6106112 204
17367040 209
21693440 207
30835712 209
1139712 188
13698048 203
19388416 208
1532928 195
16333824 205
15963136 208
23540736 207
23770112 208
7317504 206
6019072 204
13303808 206
32039936 207
13137920 206
24311808 207
25442304 208
8986624 206
25421824 208
2720768 200
18526208 209
12448768 206
6514688 203
6632448 205
17897472 206
26569728 206
22568960 208
11912192 206
22007808 206
30651392 209
29615104 208
18367488 207
21049344 206
15778816 206
24951808 208
5359616 200
23476224 202
17125376 208
3318784 201
12782592 205
30965760 209
1974272 198
32021504 209
24463360 205
28246016 208
23352320 203
27161600 208
25191424 206
10841088 206
22846464 208
19111936 207
26234880 207
32610304 209
14385152 207
18029568 208
11080704 205
3933184 201
22414336 208
8314880 204
27988992 209
20073472 207
23487488 208
30233600 205
25299968 208
2406400 196
28059648 208
2145280 196
27503616 207
20705280 208
28731392 207
2458624 198
19662848 208
6198272 204
21655552 208
32225280 206
21788672 207
16669696 206
12007424 205
4321280 201
14878720 207
10118144 205
2836480 200
21092352 207
17086464 205
1707008 194
17509376 205
28161024 208
18457600 206
16290816 207
10758144 204
17766400 204
28011520 208
19835904 202
11879424 205
18191360 208
12581888 203
15906816 208
6525952 203
30545920 206
934912 186
6738944 204
18920448 204
27171840 208
19916800 207
28598272 207
23926784 207
10470400 206
20392960 206
7282688 201
4824064 202
11820032 207
30988288 205
26221568 204
13527040 207
30140416 204
6785024 202
25762816 207
23324672 205
26011648 205
4475904 200
33500160 205
8732672 206
29461504 207
12411904 203
12294144 207
11481088 205
12786688 205
15858688 206
16142336 203
29415424 206
32114688 208
18460672 205
15672320 206
12832768 207
23447552 206
18734080 206
25832448 206
20690944 207
27086848 206
24894464 208
32919552 207
30015488 208
2870272 197
27643904 208
25778176 206
4189184 202
13642752 204
7861248 203
21999616 202
4232192 202
12140544 206
3162112 201
27608064 208
18791424 204
5019648 203
15958016 208
20426752 207
685056 174
18709504 206
6188032 204
6668288 205
30684160 207
9438208 204
20264960 206
1707008 192
4702208 204
27417600 208
31681536 206
1326080 192
9743360 207
29618176 206
28645376 208
21629952 206
21515264 206
12170240 205
29555712 206
2771968 198
30593024 209
23996416 206
31484928 208

#6
Quote from: 1% on June 19, 2013, 11:58:02 PM
I did this to 50D and write speed fell... but idle time did too.

Well thing is size/2 makes ~13mb buffers so maybe having them at 16MB would be ideal.

What do you mean that size/2 makes ~13Mb buffers? How big the size then? I don't know the 50D, in 550D we have 2x31Mb, and the remaining 7-8Mb from the 75Mb shoot_malloc.
#7
Quote from: a1ex on June 19, 2013, 10:24:22 PM
Okay guys, got some homework for you.

Take a look here: http://www.magiclantern.fm/forum/index.php?topic=5471

and run the same test (Debug - Card buffer benchmark), but this time in movie mode. The camera will shutdown after half an hour, so touch the half-shutter every now and then. Use any of the autoexecs from this thread.

Then post the logs.

As you wish sir!  :)

I think there is a lot of factors, that affects the write speed, including the card itself, the buffer distribution, the order of the buffers, the frame_size, (the varying cpu load maybe?) and so on.. So good luck a1ex to find the best solution, what works best in all cases! :)
I'll try to help if I can! :)
#8
Quote from: marekk on June 19, 2013, 09:57:02 PM
what should I change in raw_rec.c to allocate 16MB memory chunks instead of 32 ?

I don't know whether I'm doing it well or not, a1ex will tell, but:

       
                buffers[buffer_count].ptr = (void*)(((intptr_t)ptr + 4095) & ~4095);
                buffers[buffer_count].size = size/2 - 8192;
                buffers[buffer_count].used = 0;

                buffers[buffer_count+1].ptr = (void*)(((intptr_t)ptr+size/2 + 4095) & ~4095);
                buffers[buffer_count+1].size = size/2 - 8192;
                buffers[buffer_count+1].used = 0;


                buffer_count+=2;
                total += size;
#9
Quote from: a1ex on June 19, 2013, 07:36:11 PM
The 9x8 looks close to optimal for this case. I've got only 1910 frames with the variable buffer method in simulation (compared to 1565 for 9x8MB).

A question for you: what write speed are you getting if you only write one frame at a time? (that is, split the buffers in very small chunks equal to frame_size - and as many as you can - should be 81 stars for the numbers above). This will be helpful in fine-tuning the speed curve.

Edit: current simulation results (1280x426, 23.976, speed tuned to 21.16MB/s to match 1565 frames):

Warning: this is pure theory, I didn't try this on any camera. But you can see the mathematical background behind raw recording, study the algorithms and suggest improvements.

Note: if write speed wouldn't vary with buffer size, the optimal algorithm would be trivial: just write one frame at a time.


I think it's not that easy :D

With frame_size sized buffers, it says 19.8Mb/sec  (~1100 frames). So I think we can rule this theory out :)

As I can see it now, it not just depends on the buffer size, but also on the resolution.   ??? Here's how:

With 1152x460 the 1x8Mb + 4x16Mb setup gives better results:









SortingBuffers                    FramesSeconds
asc9x8Mb4371~182
asc1x8Mb + 4x16Mb5168~215

But in 1280x426, inversely, the 9x8Mb gives the better results:









SortingBuffers                    FramesSeconds
asc9x8Mb1524~63
asc1x8Mb + 4x16Mb1395~58

The write speeds are better with the 1x8Mb + 4x16Mb setup, in BOTH cases! It says constant 21.0Mb/sec...
With 9x8Mb however it says 20.8-20.9Mb/sec in both cases!

All tests were performed twice, gives almost the same results! (but there may be some randomness... :D )


Maybe I'm stupid, but isn't it possible, that the 16Mb buffers are faster than the 8Mb ones?
If I believe in this diagram:


I've lost track, help me out!!!  ;D  ;D ;D ;D ;D

EDIT: I think the write speed vary. A lot. :D
#10
Quote from: a1ex on June 19, 2013, 05:06:38 PM
I'm investigating a writing strategy with variable buffer sizes, stay tuned.

Me too :D
#11
Quote from: a1ex on June 19, 2013, 02:44:17 PM
Updated the simulation formula, now it's matches the real numbers perfectly for all test cases I've tried.

https://bitbucket.org/hudson/magic-lantern/commits/7eee493f6fc9

There's no best strategy for sorting; in some cases it's best to use biggest buffers first, in others, smallest buffers first. I think you will get better overall results if you call sim_frames and decide the sorting direction on the fly, right after allocating the buffers.


To be honest :), I also thought, that the more buffers are better, even if the overall size isn't bigger, I don't know why I haven't tried so far: D, but here it is:

Some really quick test results here, on 550D with SanDisk ExtremePro 16GB, 1280x426 @ 24fps:












SortingBuffers                                               FramesSeconds
asc8Mb+31Mb+31Mb317~13
desc16Mb+16Mb+15Mb+15Mb+8Mb1326~55
asc8Mb+15Mb+15Mb+16Mb+16Mb1395~58
none15Mb+15Mb+16Mb+16Mb+8Mb1326~55

Try and let me know:
http://www.mediafire.com/download/rkq9kh63po8xxg4/ML_550D_Xaint.rar

Great findings by the way a1ex, I'm still experimenting with this, and with your speedsym.py script.. looks like it worth the try :) thanks!



EDIT (9 x 8Mb):







SortingBuffers                                                                         FramesSeconds
-8Mb+8Mb+8Mb+8Mb+8Mb+8Mb+8Mb+8Mb+8Mb1565~65
#12
Quote from: a1ex on June 18, 2013, 06:08:24 PM
Can you implement this buffering strategy in raw_rec and check if my theory is valid?

I'll check.
#13
Quote from: marekk on June 19, 2013, 12:17:45 PM
Tragic Lantern raw_rec.c has  "if(buffers[j].size < buffers[j + 1].size)" .. It's better to sort it the other way ?


Quote from: xaint on June 06, 2013, 05:33:29 AM

And I think it's better if the buffers are in ascending order (start writing ASAP):
/* Sort Buffers */
    static struct buff temp[10];
    for (int i = 0; i < buffer_count; i++)
        for(int j = 0; j < buffer_count- i - 1; j++)
           if(buffers[j].size > buffers[j + 1].size)
           {
                temp[j] = buffers[j];
                buffers[j] = buffers[j + 1];
                buffers[j + 1] = temp[j];
           }


(It seems it's no longer in the unified?)


Some test results here, on 550D with SanDisk ExtremePro 16GB, at 1280x426 @ 24fps without HaCKeD mode:






Buffers                 FramesSeconds
2x31Mb68<3
2x31Mb + 9Mb26511
9Mb + 2x31Mb27411.4

My version is probably not up to date, so take this into account!
And I don't know how it may affect the other models btw, need reviews...
#14
Quote from: a1ex on June 06, 2013, 08:39:29 AM
Sorting might help with very small buffers when you have to write 100-200 frames or so (e.g. a particular write speed may work better with a particular buffer order), but will not help at all for continuous recording. Use my python script and run some simulations.

You can't ifdef in raw_rec, and 1MB squeeze at the cost of maybe not working at all... I wouldn't recommend.

Yeah I know this isn't ,,The Solution" for continuous recording, it only delays the ,,buffer overflow", but it is at least a few seconds plus recording time.

Until we figure out how to get more memory (if it is possible at all on 550D), this is something at least. :)

I didn't try that speedsym script, but I will!

dlrpgmsvc: about 1mb squeezing in exmem.c: I agree with a1ex: 1MB squeeze at the cost of maybe not working at all... I wouldn't recommend.
#15
Quote from: dlrpgmsvc on June 06, 2013, 12:59:11 AM
- Modifying the "backup_size" (by upsizing or downsizing it), doesn't sort any effect at all.
- Changing the probe size from 4 to 2 and then to 1 (just to dig out more memory chunks), doesn't sort any effect at all.
- Forcing "size" to zero into the shoot_malloc allocation function, just to force it to allocate the maximum size, takes the camera to stack overflows and freezes that need to pull out the battery: no luck...


I think we can gain a few more extra frames (on 550D at least), if we are a little bit more permissive with the current waste recycling mechanism:

/* try to recycle the waste */
    if (waste >= 8*1024*1024 + 8192)
    {
        buffers[buffer_count].ptr = (void*)(((intptr_t)(fullsize_buffers[0] + buf_size) + 4095) & ~4095);
        buffers[buffer_count].size = waste - 8192;
        buffers[buffer_count].used = 0;
        buffer_count++;
        total += waste;
    }


Let's say we can use those chunks from the remaining, which are larger than 8Mb. Because on the 550D we have ~75Mb shoot_malloc, and with the two ~31Mb contiguous buffers only,we left unused around 10Mb.

And I think it's better if the buffers are in ascending order (start writing ASAP):
/* Sort Buffers */
    static struct buff temp[10];
    for (int i = 0; i < buffer_count; i++)
        for(int j = 0; j < buffer_count- i - 1; j++)
           if(buffers[j].size > buffers[j + 1].size)
           {
                temp[j] = buffers[j];
                buffers[j] = buffers[j + 1];
                buffers[j + 1] = temp[j];
           }


(It seems it's no longer in the unified?)


Some test results here, on 550D with SanDisk ExtremePro 16GB, at 1280x426 @ 24fps without HaCKeD mode:






Buffers                 FramesSeconds
2x31Mb68<3
2x31Mb + 9Mb26511
9Mb + 2x31Mb27411.4

My version is probably not up to date, so take this into account!
And I don't know how it may affect the other models btw, need reviews...

dlrpgmsvc: Just to make you happy  :D ;D

a1ex please confirm these!

Forgive me if I wrote stupid things :D and apologize for my english :D
#16
Quote from: pavelpp on June 03, 2013, 05:53:54 PM
One more sample from today 1408x480 @18fps

http://youtu.be/bleWVuDiqtE

So.. several bugs to report.
1) Deleting raw files does not work properly and I can't switch from one raw file to another for in camera playback
2) 10x zoom kills camera, so would be nice to have it disabled by default.
3) Something weird going on with aperture and shutter settings. I can't get the logic here, but they seem to be changing when I switch to 5x. Also exposure visually seems to be different after camera restart, though the aperture and shutter settings remain the same. Why are they shown in red btw? Shutter setting seems to have almost no effect, when I change it from 200 to anything up to 4000, only aperture affects the amount of light..Maybe I am missing something?
4) When I extract dng's they seem to be at a smaller resolution compared to settings in camera, why is it so? For example for 1408x480 video dng's are 1392×464.

3) FPS override maybe?
#17
That's why I needed a low-level formatting:  :-\

#18
Quote from: y3llow on June 02, 2013, 01:10:39 AM
xaint, looks good but was that shot in Photo mode? The metadata for the .mp4 says 1920x640 at 30fps? mediainfo and VLC concurr.

Regarding photo mode, I seem to get more resolution, without failing, but guess something else is going on. I have fps override on at 24.000 exact and manual lens.

Nope, i think it was in movie mode (1920 24fps), fps override on at 24.000. I think it is just wrong comp settings in After Effects. I hurried..
(so actually you see the things a little bit faster than it was :D)
#19
Quick test shot (1200x400 23.976)
SanDisk Extreme Pro 16GB 95Mb/s

#20
Quote from: a1ex on June 01, 2013, 07:10:25 PM
You must update *addr_AllocMem_end and ml_reserved_mem; otherwise, Canon code will overwrite ML sooner or later and you'll wonder what's going on.

Ok, i did it, i updated  *addr_AllocMem_end and ml_reserved_mem, but it shows ~2500 K free mem for AllocateMemory.

I think this is not good :)



What should I do?
#21
Quote from: a1ex on June 01, 2013, 05:57:41 PM
I recommend keeping the classic boot method (I prefer to avoid cache hacks if possible); in theory, all you have to do is to adjust this:

BTW, my plan is to keep ML slim and load things on demand, rather than just making it bigger and bigger.

Oh a1ex, just in time with the simpler solution! :D

Loading things on demand is a good idea.

As i understand, you mean that almost every in-camera functions is just a module?  Like: RAW video feature will always be in the Movie menu, and when the user enabe it, then ML loads the raw_rec module in the background automatically. Right?
(Sorry for my english :| )

changing back to the classical mode btw..
#22
Quote from: dlrpgmsvc on June 01, 2013, 05:28:58 PM
Have you checked all the hints in the head post and the last I written to mk11174, so to be in phase with the mods ?  ;) They are 100% certified from... 1% !  ;D

Yes i've checked the hints in the head post, and i've used it really often, so thank you!

Quote from: dlrpgmsvc on June 01, 2013, 05:01:31 PM
Also, very important :

don't forget to replace ifdef 6D with
#ifdef HIJACK_CACHE_HACK_ALLOCMEM_SIZE_ADDR in boot-hack.c and get rid of allocate mem in internals.h

I'm not sure if this is needed!


Quote from: mk11174 on June 01, 2013, 05:38:16 PM
You too man! Without your posts things would have gone much much slower I am sure!

And that's very true! So thanks again :)
#23
Quote from: mk11174 on June 01, 2013, 05:13:41 PM
Ahh, there we go, cool, thanks man!

Update: Yep, 640k cool!! Thanks again xaint

If the changes what i've made are working, how can i push it to the main repo? Only the three affected files, without having to spoil something? :) Pull request?)
Thanks in advance ;)
#24
Hi guys, you can find the cache_hack i've made, in my repo, if you wanna try:
I think it is here: https://bitbucket.org/xaint/ml/commits/f3631da68349f1dda86ee136bbddc5a3c4efc9bc
Try it on your own risk (for me it's pretty stable) :D:D




(I'm just now editing the test shots, what i've made)
#25
Ok, now where the hell can i find a magic.sym? :D :D

no matter, problem solved!