7D Raw Thread

[feureau]

Yea, I posted a compiled one, if it used MLV before then this should run. Does the cache hack work?

You mean the small hacks? I don't see any difference though.

On my sandisk 60mb/s - I can get 55mb/s out of g3gg0's raw_rec.mo, while the build posted by britom clocks in at 50mb/s

On g3gg0's, I'm using buffer fill method 3 and CF-only buffers 8. (buffer fill method seems to be missing on your build)

[1%]

Buffer fill is there, I thought CF only is for splitting CF/SD

[feureau]

Buffer fill is there, I thought CF only is for splitting CF/SD

Sorry I got that mixed up. Buffer fill is there. It's the CF only buffer that is missing.

Not sure what CF-only is for, but on one of my previous tests, increasing that seems to make it write faster, but it slowed down significantly at CF only 9.

I just did a quick test, and it doesn't seem to effect anything on g3gg0's latest from nick.

[blackjack102]

I decided to use 30/08/2013 britom's build with raw files and went to an anime convention next day! I used 1280x514, 16:9 and small hacks to make this video. I don't use card warm-up, and the build can push up to 1600 frames. I surprised to find fewer pink corrupted files than previous builds. Some scenes did not generate any corrupted files. I faced two problems.

First, I tried to use upsidedown setting on my monitor, so I can upsidedown my camera and shoot. I noticed that live view was chopped and hoped it doesn't affect the scene video. When I went to home, I discovered that the scene was affected by upsidedown setting. The scene generated 200 pink corrupted out of  600 frames. I should do the test before I go.

Second problem, I turned off global draw and did not know that my memory was about to full. I stopped recording scene and found that my camera was so busy to write for while that I had to wait cosplayer for next scene. After writing, I started the next record, and I turned off/on camera after the instance and found the message: “full memory”. I swapped memory cards. At my computer, I discovered that the next scene did not record at all.

Everything is experiment to me! I am so amazed to see the build did not use a lot energy from battery; in fact, my 7D wrote 22 GB of raw files and used ONE bar of battery. If I use canon's standard video process, I am sure my battery will dead at end of day. I am sure my video can improve more but am doing it as hobby. I am happy with the result.

[a1ex]

Great find about upside down vs pink frames (I was looking for a way to reproduce them)

[wheezl]

On the britom_1% build 1728x786 (2.20:1) is working quite well and 1728x864 (2:1) seems to push the card all the way to 57.5MB/s at peak with the mlv_rec module. After giving that a go ML seems to think my card is good for 57.3MB/s using whatever math is does (avg?) from the last run. Trying 1728x934(1.85:1) *appears* to push the card to 58.2 MB/s but I don't really know how accurate the numbers are during recording.

Because of issues I am having loading all of the modules I only have the ime_* modules and mlv_rec loaded at the moment.

It may be interesting to note that loading the file manager module later seemed to impact write speed for some reason.  Obviously this is just me eyeballing it and not a proper test but I routinely get 55-58MB/sec with it off and 50-55MB/sec with it on.  It might be nothing since obviously I am not doing scientific testing.  I'll see in a bit if it affects anything in playback benchmark.

Sandisk Extreme Pro 16GB (60MB/s)
128MB warmup (Not sure this matters)
Buffer Fill Method: 4

 

[1%]

Ditto, about the pink frames, now I want to try upside down on cameras that support it and see if it makes things worse.

On 50D it takes ~3 recordings to get up to 80Mb/s. The numbers should be mostly accurate. You can also compare between, ie, run a normal raw module and compare to MLV.

with faster benchmark speeds, hopefully higher resolutions above 1728 x 972 in 1X mode will be possible.

In 1x its probably not going to get any bigger... but a CF camera should record 1x no sweat.

Priority is FPS + display filters to get it up to speed with the rest of the ML stuff. I think only EOSM is missing display filters because of the sync and 7D because of ? (dual digic or sync?) Still waiting on camera in hand tho. First first thing will be to check edmacs/reslock if you're still getting pink frames.

[dmk]

Running make from the modules/raw_rec directory fails for me the same reason from both- "/usr/include/features.h:324:26: fatal error: bits/predefs.h: No such file or directory"

Using the newest toolchain on ubuntu 12.04 vm 64bit with libc6-i386 installed, in case that matters

So it was a 64-bit vs. 32-bit issue, running sudo apt-get install libc6-dev-i386 fixed it to work on the main hudson branch

Ran into another issue with 64v32bit on g3gg0's repository in the lzma stuff though, I think I'll just fire up a 32-bit VM to make life more predictable.

Noting this here in case it helps anyone else...

[thoma]

@ 1%

Sorry if I'm beating a dead horse but why will the 7D never make it to 1920x1080 at 1x? You said:

'In 1x its probably not going to get any bigger... but a CF camera should record 1x no sweat.'

How do you mean 'a CF camera'?

Thanks!

[britom]

@thoma

Because the sensor size in pixels is 5,184 × 3,456

In 1x non crop mode the procesor does line skipping using 1 line every 3 lines (it's designed like this so the camera can supposedly output 1920x1080 h264 video without a lot of processing power)
In 5x cropped mode the processor takes a small part of the sensor without line skipping, therefore is possible to record at a higher resolution but not with the whole sensor (2512x1200)

Just divide 5184/3=1728 (max horizontal resolution in non cropped mode).

Now this comunity has acheived unbeliable things, so it's ok to dream that this barrier can be overcome.

[wheezl]

I think he was referring to the possibility of 1728x1156 in 1x no?  Possibly useful for anamorphic etc...

EDIT: I actually went back and read that properly... never mind me

[1%]

It can't even record full 1080P H264.... that YUV size in the debug menu is what its upscaled from... let me guess, close to or smaller than 1728.

The best/most realistic thing would probably be to add line skipping to 5x mode so that its not so cropped. I.e, instead of 3 do 2. I guess we'll see what comes from the ADTG gui/thread.

1728x1156

So a 1000x card can't handle 1156 or close to it like 1038?

[wheezl]

It claims a 79.9MB/s for 1728x1156 @ 23.976.  I'd be interested to see @ted ramasola test with the latest builds on the cards he has.

[britom]

@1% Wow, you mean 1056x704!?!? i thought it was atleast 1500xsomething upscaled to 1080p :/



Talking about the 5x cropped mode, doing line skipping every 2 lines means we can cover the equivalent area of 5024x2400 or 5024x1200 pixels (the full sensor is 5184x3456) but still recording as 2512x1200. Am i right?

[feureau]

@1% Wow, you mean 1056x704!?!? i thought it was atleast 1500xsomething upscaled to 1080p :/

Hmm.. I'm seeing 720x480, 720x480 on my Image buffers in the Debug menu. Is there some settings that effects this?

EDIT: If there's some settings that effects the  YUV buffer thingy, any chance of increasing that and getting higher-res h.264s?

[thoma]

@ 1%

Ahhhhhh....it's so (deceptively) simple.  Is that also why you always hear that the 1920 H.264 files are only really resolving ≤ ~1k horizontal lines?  Because the final output is simply an upscaling of the line skipped video feed? 

Somewhat related - I'm assuming that magic lantern is grabbing the raw feed after the line skipping/binning process and the possibility of getting further upstream and doing a cleaner downsample is not possible right?

thanks again

[fadase]

Please i don't find modules on my 7d
 

[1%]

1056x704

Thats a zoom window or idle size. This blows up to the input size when recording.

720x480 is the LV yuv for the LCD.

Its like mm1 -> yuv - > h264. the camera can do a couple YUV buffers at once in diff sizes

[feureau]

Thats a zoom window or idle size. This blows up to the input size when recording.

720x480 is the LV yuv for the LCD.

Its like mm1 -> yuv - > h264. the camera can do a couple YUV buffers at once in diff sizes

Has anyone tried investigating/experimenting with changing YUV sizes for the h.264? (and maybe get a hi-res 2.64?)

[1%]

The H264 hardware encoder will only output the sizes it has. You can maybe feed it something else at best.

[feureau]

The H264 hardware encoder will only output the sizes it has. You can maybe feed it something else at best.

Would feeding it the liveview raw feed let it encode something un-upscaled? That would yield a 1080p H.264 that is ML-Raw-like in terms of resolution...

Basically eliminating that softness in the H.264 video that people have been complaining about...

Can it be done via modules?

[mucher]

A bit away from the current topic. 7D's write speed is only around 63% of its theoretical speed. I am wondering if it is due to the fact that the ML is booting from the slave CPU, and I think that it might be writing faster from the master CPU saving the penalty of overhead from communicating between the CPU?

[ted ramasola]

So a 1000x card can't handle 1156 or close to it like 1038?

Aug 30 build 7D

Just tested on
KomputerBay 256gig 1200X (formatted as 128gig)

1728 x 1038 = indicated continuous recording, I stopped at past 51 gig to test another card. Buffer barely budged. Card needed some warm up.

in crop mode @ 30p
2048 x 682 = indicated continuous recording, I stopped at past 30 gig, as temp warning showed up. buffer barely moved.


KomputerBay 64gig 1000X

1728 x 1038 = indicated continuous recording, I stopped at past 31 gig. Buffer at 20-30%. Card needed to warm up.

in crop mode @ 30p
2048 x 682 = indicated continuous recording, I stopped at past 24 gig. buffer barely moved.

Lexar 32gig 1000X

1728 x 1038 = filled up 32 gig card. Needs to warm up card first.

in crop mode @ 30p
2048 x 682 = 7649 averaged from several takes.

[feureau]

Aug 30 build 7D

Just tested on
KomputerBay 256gig 1200X (formatted as 128gig)

1728 x 1038 = indicated continuous recording, I stopped at past 51 gig to test another card. Buffer barely budged. Card needed some warm up.

in crop mode @ 30p
2048 x 682 = indicated continuous recording, I stopped at past 30 gig, as temp warning showed up. buffer barely moved.


KomputerBay 64gig 1000X

1728 x 1038 = indicated continuous recording, I stopped at past 31 gig. Buffer at 20-30%. Card needed to warm up.

in crop mode @ 30p
2048 x 682 = indicated continuous recording, I stopped at past 24 gig. buffer barely moved.

Lexar 32gig 1000X

1728 x 1038 = filled up 32 gig card. Needs to warm up card first.

in crop mode @ 30p
2048 x 682 = 7649 averaged from several takes.

That is amazing. Did you write down the write speed for each of these?

Have you tried shooting at 2.5k and how long did it last?

[ted ramasola]

That is amazing. Did you write down the write speed for each of these?

Have you tried shooting at 2.5k and how long did it last?

I don't know about write speeds, I only did the benchmarks which I posted here:

http://www.magiclantern.fm/forum/index.php?topic=7503.msg71926#msg71926

for 2.5K using aug 30 build
Highest possible in crop mode 2512 x 1200 @30P

KB 256gig 1200X = 102 frames average

KB 64gig 1000X = 102 frames average

KB 128gig 1000X = 100 frames average

LX 32gig 1000X = 100 frames average