uncompressed 14-bit RAW video recording

[squig]

Cool! Have you tried filling up the card and if so does the MK3 start to smoke?

[lourenco]

Cool! Have you tried filling up the card and if so does the MK3 start to smoke?

I did a test to see what would happen if I filled the memory card. It records the first file fine.  It keeps recording until the memory card is full, but I am having strange results after the first file. I just sent a1ex a message about it. 

[squig]

Big congrats and thx to all the devs, this is huge for digital indie cinema. Beware the Zaibatsu Ninja.

[1%]

Figured out how to get pic sizes... 6D its 1920 exactly. Weird, eh? Sped up to 22MB/s too when the size was corrected.

[squig]

So is 1280x720 60p RAW now doable too? Or 48p?

[kgv5]

Figured out how to get pic sizes... 6D its 1920 exactly. Weird, eh? Sped up to 22MB/s too when the size was corrected.

You mean 1920 instead of 1840 in RAW mode? Sped up to 22 MB/s so what was the speed before correction?

[noisyboy]

Big congrats and thx to all the devs, this is huge for digital indie cinema. Beware the Zaibatsu Ninja.

Yup! Seriously well done! Amazing effort the ML wizards! 

[Kabuto1138]

Nice guys!  great job!

This is only working in the 6D, right?

[N.Mendes]

Ok, Merry Christmas all..

It keeps recording until the memory card is full, but I am having strange results after the first file.

Did you try to use CF+SD ? In this case, maybe the second file will be as clean as the first?
About the aspect ratio, again, maybe "CF+SD" will allow to shoot 1920x1080 (or more?)

ps: your test is crazy, in term of detail and dynamic range, really really crazy, thanks dude..

[Africashot]

Any chance of getting anywhere near 400 frames at 24 fps with the good old 5D2? Sorry for asking, I know you guys are working on it and I don't want to be pushy but man this is so exciting I just couldn't hold myself and if there is only the slightest chance it would be way beyond what I had ever dared to dream of!

[a1ex]

Run the benchmarks and post screenshots. I don't remember seeing any benchmarks from 1000x cards with 5D2.

[Africashot]

Run the benchmarks and post screenshots. I don't remember seeing any benchmarks from 1000x cards with 5D2.

Thanks Alex! Will try when my 1000x card gets here, shipping to Africa takes eternities...

[arrinkiiii]

!!! AMAZING !!!

Anybody thinks that this will run on the 7D?

[Andy600]

Super impressive stuff. Seriously well done guys!

@1% have you tried this on the 600d yet? I know it probably wont handle what the 6d and Mk3 can but I'm eager to know what, if any chance there is of pushing the shot length up... even if only by a few seconds.

[RenatoPhoto]

Congratulations!
You are the Nobel prize winners for arm development period!
Thanks for your amazing work.
Renato

[g3gg0]

please dont ask about too many models. we are aware that 550d, 50d, 60d, 6d, 600d, 5d2, 7d, etc are also widely used.
before we start analyzing these models, we want prove our theory by making a reference implementation.

as soon this reference implementation is proven stable, we will advance to these models.

current state: trying to make the code as performant, stable and portable as possible.

[minimimi]

Just added a RAW to DNG converter: https://bitbucket.org/hudson/magic-lantern/commits/5bc3488c7a78

It's just a small command-line tool. Feel free to build a nice GUI on top of it.

For some reason, I couldn't compile it in 64-bit mode, so you may need the 32-bit compatibility libraries. Fixes are welcome

With a little luck, this program may work under Windows too.

Alex, Are you forget to add this ?(when the file size reached 2Gig , then rotation the file)
I'm tried to concatinate by cat command, but something wrong. (4gig of file not supported??)
And cat need a lot of time and disk spaces. So I think each file must have footer info.

Code: [Select]

diff -r 5bc3488c7a78 modules/lv_rec/lv_rec.c
--- a/modules/lv_rec/lv_rec.c   Sat May 11 14:35:52 2013 +0300
+++ b/modules/lv_rec/lv_rec.c   Sun May 12 21:51:27 2013 +0900
@@ -649,6 +649,8 @@
                     {
                         yPos++;
                         bmp_printf( FONT(FONT_MED, COLOR_WHITE, COLOR_BLACK), 30, 20 * yPos++, "Creating next file");
+                       save_data.frameCount = 1;
+                       lv_rec_save_footer(save_data.handle, &save_data);
                         FIO_CloseFile(save_data.handle);
                         save_data.handle = NULL;
                         lv_rec_update_suffix(&save_data);


[g3gg0]

the lv_rec code basically is my work, alex updated it to the latest research results.
so i will answer to your question.

i wanted the split file to be really "split" just as if they were splitted by common split and merge utilities.
this means the payload is split over several files and at the end we have one footer that contains information for all frames.

simply concatenate the files to have a big movie file.

the reason for a footer is simple: in YUV mode you can process the files with the existing tools, you dont have to cut the footer away.
if i had added a header, it wont be that simple - you have to remove the header before you can use old tools.

when the tools are finished, it is as simple as specifying the first file M000000.RAW and it will autodetect that there is a .R00, R01 etc
and will virtually address them as one file.

[1%]

@1% have you tried this on the 600d yet?

Almost... I got a size of 1652 but the YUV buffer is 1728... when I open the files they are still messed up. Maybe I need to pull a bigger buffer than 2k and run image analysys again... that or sync is fucked?

Have to check on 720P mode, the size might be different. If it is, should pick based on YUV buffer size... like if buffer is 1280 then raw video is XXX, etc.

the lv_rec code basically is my work,

This in the new memecpy is freaking awesome

[a1ex]

Heh, people got excited without even knowing the big news: g3gg0 just discovered how to use the DMA cropping routines, which just made possible RAW video recording at 1920x1080 at 24fps on 1000x cards.

Technical: we now know how to copy a cropped version of some image buffer at very high speeds (over 700MB/s), and with this trick we can save the video data the card at full speed, without being slowed down by image borders, for example.

1920x1080 RAW video now requires 83MB/s at 24fps, so it should work just fine on 1000x cards. I didn't try it.

So, I've lost my patience and rewritten the lv_rec module from scratch, to use these new routines and to experiment with different buffering algorithms. The new module is called raw_rec and outputs the same file format (RAW files).

Main changes:

- The ring buffer only uses 32MB memory blocks (maximum we can get). Reason: card benchmarks showed higher data rates for large buffers.
- Frame copying is done outside the LiveView task (not sure if it has any effect).
- When the buffer gets full, it skips some frames, rather than stopping.
- Fewer hardcoded things: should be easier to port.
- Resolution presets, from 640x320 to 3592x1320.

Just like lv_rec, this is in very early stages, so you have to compile it yourself.

Source code: https://bitbucket.org/hudson/magic-lantern/commits/54537cb85d7d

If you try it, I'd like you to look for any signs of image tearing. The source raw data is single-buffered, but it's possible to make it double-buffered if the vertical sync is less than ideal.

All credits go to g3gg0 - without his reverse engineering work on understanding the image processor, this would have been impossible.

[minimimi]

Wrote reply but alex is making a code from scratch,,,,, ignore this.........

i wanted the split file to be really "split" just as if they were splitted by common split and merge utilities.
this means the payload is split over several files and at the end we have one footer that contains information for all frames.

simply concatenate the files to have a big movie file.

I already read and understand your codes.
But , concatenate time is really long.
So I think if the all of files has footer info, we don't need time to concatenate files and tempolary disk spaces.
Also , I'm tried to concat by cat command, I don't know why, but fopen failed (3.6Gig file on 32bit linux).
(?? cat command broken files???)

the reason for a footer is simple: in YUV mode you can process the files with the existing tools, you dont have to cut the footer away.
if i had added a header, it wont be that simple - you have to remove the header before you can use old tools.

when the tools are finished, it is as simple as specifying the first file M000000.RAW and it will autodetect that there is a .R00, R01 etc
and will virtually address them as one file.

If we can cut footer , we have no problem to use old tools when each files has footer info.

It's my thinking.

[Stedda]

Heh, people got excited without even knowing the big news: g3gg0 just discovered how to use the DMA cropping routines, which just made possible RAW video recording at 1920x1080 at 24fps on 1000x cards.

True wizards! Nice work guys!

[1%]

Source code: https://bitbucket.org/hudson/magic-lantern/commits/54537cb85d7d

Going to try it... old LV_REC would crash in 720P raw because of fixed size.

Ok, just tested on 600D, thats the repo I have open ... 32mb is too big to allocate here, tried 16, going to try 24. Wish the skipping was optional, sometimes you want continous frames for less time. Stopping hangs up the ML menu/camera for some reason, LV still moves and I think you can hit canon stuff. Once you switch modes LV won't go away. Have to check this out. Going to see if files are readable with the 1740 preset and try on 6D.

[a1ex]

Don't forget to try "don't click me" - g3gg0's demo is really cool.

[1%]

On 600D the demo lagged, on 6D it was perfectly fluid. Image would move back and forth... just there was no way to stop it

Oh man,,, the raw file is 640MB... I just recorded a few sec. 296 frames... 15MB/s @16MB malloc.. will try 24MB. Its funny but 600D/6D sd rates are very very close, maybe with the larger writing it will finally crack 30MB/s

1740 its like the files are off by 1 or 2 pixels, slightly skewed to the side. Doing img.py

600D size is 1738.

Ok, more testing: LV_REC will not record the 1738 files, they come out all pink and screwed up, why?
Raw_rec does a good job but ending is screwed up and for some reason the resolution is detected as 1740 and when manually fixed the files say unexpected end of file. Gimp can open them, they look like this: