550D/T2i raw video recording port official thread

[funkysensation]

Here is another Test video i did last Weekend in Copenhagen Denmark.

http://550draw.blogspot.de/

[Ivoprod]

Here is another Test video i did last Weekend in Copenhagen Denmark.

http://550draw.blogspot.de/

Which build you use?
And What SD card ?
Also Interesting , moire is in very usable range, which lens you use?

[funkysensation]

I´m impressed too that moire is not that much visible. Try to shoot stuff like the shot at 01:45 you will get crazy with the intern H264. 
Lenses:
Zenitar Fisheye  16mm f2.8
Sigma 28-300mm

latest built I think
ML_RAW_MEM_FIX 

Sandisk Extreme Pro 95MB/s 32GB

[a1ex]

I've added a small buffer optimization to the raw rec simulator:

https://bitbucket.org/hudson/magic-lantern/commits/0db037b3b960

With the numbers from the example above:

1152x460 23.976fps 20.0MB/s => with current ML method, the simulation says 198 frames.
After optimization (only reordering and splitting the buffers differently) => 880 frames;

At 21.0 MB/s => 729 frames with current method, 2880 optimized.

Notice there's a slight difference from the previous numbers, because the latest script also models the slight speed loss from using small buffers.

Can you implement this buffering strategy in raw_rec and check if my theory is valid?

[MaKsOZ]

Here is another Test video i did last Weekend in Copenhagen Denmark.

http://550draw.blogspot.de/

Very nice quality, congratz

[Kim.dh]

Here is another Test video i did last Weekend in Copenhagen Denmark.
http://vimeo.com/68607691

http://550draw.blogspot.de/

Nice! How do you avoid temporal aliasing in small details like leaves and roof tiles?

I got some footage of skerries covered with dried lichen (tiny details) which shows bad noise from temporal aliasing (I assume) in bright areas, which is driving me crazy trying to correct.

[mk11174]

This is the patch I am using in both Magic and Tragic module, nothing has changed since the first time we found out about the mem fix.

Code Select Expand

     /* try to recycle the waste */

-    if (waste >= 16*1024*1024 + 8192)

+    if (waste >= 8*1024*1024 + 8192)

     {

         buffers[buffer_count].ptr = (void*)(((intptr_t)(fullsize_buffers[0] + buf_size) + 4095) & ~4095);

         buffers[buffer_count].size = waste - 8192;

@@ -625,6 +625,17 @@
         buffer_count++;

         total += waste;

     }

+

+/* Sort Buffers */

+    static struct buff temp[10];

+    for (int i = 0; i < buffer_count; i++)

+ for(int j = 0; j < buffer_count- i - 1; j++)

+ if(buffers[j].size > buffers[j + 1].size) // > kisbuffer elöl

+            {

+                temp[j] = buffers[j];

+                buffers[j] = buffers[j + 1];

+                buffers[j + 1] = temp[j];

+            }

Tragic version is not from the main repo, that is from his own repo. So maybe something differ in the code that makes it work differ. Not sure.

I just use Magic version and it has not let me down yet, very happy with how it works considering our controller limitation, sometimes I get good frames, sometimes bad, I think it depends on the card condition at the time, seems to random to say it is in the code that causes it.

Maybe if you try without loading other modules to see if that makes a differ in performance, in this version I added the picview module too.

Go to the module menu and go to each module except Raw_Rec and press set on each and choose not to autoload them, then set the main autoload to on and restart camera so it only loads raw_rec.

[JuanIrache]

A1ex, if that patch works you'll have to tell us where to build you a statue.

In another order of things, would it be possible to keep the 10x crop for focusing purposes and avoid the camera crash? Something like ML turns Raw off the moment you go 10x and then turns it on when you're back to normal or 5x.

[mk11174]

not exactly sure if or what I should change unless the mem patch I mentioned in last post is already the best method which is the one we have been using.

Or am I suppose to change something else to try Alex's find?

I would be glad to test it if I knew what to change.

Do you simply want me to change if (waste >= 8*1024*1024 + 8192)  to if (waste >= 2 + 8*1024*1024)

[mk11174]

This version seems to run good for me, things I do with this one that helps is.
Turn Global Draw Off
Use FPS to set fps to 23.976 but 23 seems like it will go on forever
I was testing 1152x460
I use Canon Preview mode
And I used Magic Raw

But first try it with your normal settings you are use to and see if this version does better or worse.

If first record does not do good, try again and see how good next record does.

http://www.mediafire.com/download/25py137awp0s7x9/ML_550D_June_19_2013_Update.zip

UPDATE: After more tests I found this one was no better and recorded corrupted frames.

[Rewind]

These two last records doesn't work in photo mode.
But previous one was able to get as much as twice more frames compared to movie mode. What is the reason for that?

[marekk]

This is the patch I am using in both Magic and Tragic module, nothing has changed since the first time we found out about the mem fix.

Code Select Expand

 
+ if(buffers[j].size > buffers[j + 1].size) // > kisbuffer elöl


Tragic Lantern raw_rec.c has  "if(buffers[j].size < buffers[j + 1].size)" .. It's better to sort it the other way ?

[a1ex]

Use the simulation script to find out.

Does that sorting algorithm actually work?! I can't understand it...

[mk11174]

These two last records doesn't work in photo mode.
But previous one was able to get as much as twice more frames compared to movie mode. What is the reason for that?

Raw record is disabled in photo mode, has nothing to do with the new versions here, it is just like that now.

[mk11174]

Use the simulation script to find out.

Does that sorting algorithm actually work?! I can't understand it...

Sorting code makes a huge differ on this camera. I added the sorting code to Magic Raw, it was already on the Tragic Raw

[mk11174]

Tragic Lantern raw_rec.c has  "if(buffers[j].size < buffers[j + 1].size)" .. It's better to sort it the other way ?

To find out compare your results between Tragic and Magic, Magic uses it the opposite of Tragic.

[xaint]

Tragic Lantern raw_rec.c has  "if(buffers[j].size < buffers[j + 1].size)" .. It's better to sort it the other way ?

And I think it's better if the buffers are in ascending order (start writing ASAP):

Code Select Expand

/* Sort Buffers */
    static struct buff temp[10];
    for (int i = 0; i < buffer_count; i++)
        for(int j = 0; j < buffer_count- i - 1; j++)
           if(buffers[j].size > buffers[j + 1].size)
           {
                temp[j] = buffers[j];
                buffers[j] = buffers[j + 1];
                buffers[j + 1] = temp[j];
           }


(It seems it's no longer in the unified?)


Some test results here, on 550D with SanDisk ExtremePro 16GB, at 1280x426 @ 24fps without HaCKeD mode:

Buffers	Frames	Seconds
2x31Mb	68	<3
2x31Mb + 9Mb	265	11
9Mb + 2x31Mb	274	11.4

My version is probably not up to date, so take this into account!
And I don't know how it may affect the other models btw, need reviews...

[xaint]

Can you implement this buffering strategy in raw_rec and check if my theory is valid?

I'll check.

[JuanIrache]

This version seems to run good for me, things I do with this one that helps is.
Turn Global Draw Off
Use FPS to set fps to 23.976 but 23 seems like it will go on forever
I was testing 1152x460
I use Canon Preview mode
And I used Magic Raw

But first try it with your normal settings you are use to and see if this version does better or worse.

If first record does not do good, try again and see how good next record does.

http://www.mediafire.com/download/25py137awp0s7x9/ML_550D_June_19_2013_Update.zip

UPDATE: After more tests I found this one was no better and recorded corrupted frames.

I'm getting pink frames with this one (*too). Never experienced this before, I think.

[mk11174]

I'm getting pink frames with this one. Never experienced this before, I think.

Same here, that is why I put an update about it having corrupt frames.

[dlrpgmsvc]

Sorting code makes a huge differ on this camera. I added the sorting code to Magic Raw, it was already on the Tragic Raw

So, do this can be inserted in the raw rec module with a conditional compile statement only for 550d? If the conditionals are not implemented for this module, do we can implement them? So, we can take advantage of different strategies on different cameras! Let me know!

[a1ex]

Updated the simulation formula, now it's matches the real numbers perfectly for all test cases I've tried.

https://bitbucket.org/hudson/magic-lantern/commits/7eee493f6fc9

There's no best strategy for sorting; in some cases it's best to use biggest buffers first, in others, smallest buffers first. I think you will get better overall results if you call sim_frames and decide the sorting direction on the fly, right after allocating the buffers.

[xaint]

Updated the simulation formula, now it's matches the real numbers perfectly for all test cases I've tried.

https://bitbucket.org/hudson/magic-lantern/commits/7eee493f6fc9

There's no best strategy for sorting; in some cases it's best to use biggest buffers first, in others, smallest buffers first. I think you will get better overall results if you call sim_frames and decide the sorting direction on the fly, right after allocating the buffers.

To be honest , I also thought, that the more buffers are better, even if the overall size isn't bigger, I don't know why I haven't tried so far: D, but here it is:

Some really quick test results here, on 550D with SanDisk ExtremePro 16GB, 1280x426 @ 24fps:

Sorting	Buffers	Frames	Seconds
asc	8Mb+31Mb+31Mb	317	~13
desc	16Mb+16Mb+15Mb+15Mb+8Mb	1326	~55
asc	8Mb+15Mb+15Mb+16Mb+16Mb	1395	~58
none	15Mb+15Mb+16Mb+16Mb+8Mb	1326	~55

Try and let me know:
http://www.mediafire.com/download/rkq9kh63po8xxg4/ML_550D_Xaint.rar

Great findings by the way a1ex, I'm still experimenting with this, and with your speedsym.py script.. looks like it worth the try thanks!



EDIT (9 x 8Mb):

Sorting	Buffers	Frames	Seconds
-	8Mb+8Mb+8Mb+8Mb+8Mb+8Mb+8Mb+8Mb+8Mb	1565	~65

[a1ex]

Not always. Smaller buffers will reduce the write speed; the speed profile looks like this: http://www.magiclantern.fm/forum/index.php?topic=5471

Of course, the speed drop from 32M to 16M is very small, and the smaller granularity is useful here. But if the card is fast enough for continuous recording, it may work better with only the two 32MB buffers; the 8MB one may slow it down.

At startup and when the buffer is about to overflow, you want small buffers. In the middle of recording, when things are rolling smooth, you want large buffers. I'm investigating a writing strategy with variable buffer sizes, stay tuned.

[xaint]

I'm investigating a writing strategy with variable buffer sizes, stay tuned.

Me too