50D Raw video

Started by Andy600, May 22, 2013, 03:40:57 PM

Previous topic - Next topic

0 Members and 4 Guests are viewing this topic.

GregoryOfManhattan

@KahL what i have is less stable than currently available.





JulianH

I'm surprised about your stability issues with the build from the 2nd post Gregory. Ok, granted, I didn't shoot much long sequences, just tried the 4GB limit once, but recording is quite stable. When I shot the anamorphic video I made about 30 sequences in the course of one hour (each 10-20 seconds), without any complications.

If I try 5x or 10x zoom the camera crashes almost instantly. When do you experience problems?

P337

Quote from: Digital Corpus on May 29, 2013, 09:49:35 AM
Ok, call me ignorant. I've followed the 5D3 thread, and I've searched this thread. Why is the resolution limited to 1592 x 1062? I noted another user stating that the resolution is bumped down to this when trying to select 1920x1080. Also, why is the default fps at 30?

In terms of pixels a second:
1592x1062*30 = 50,721,120
1920x1080*24 = 49,766,400

In other words, in terms of bandwidth, 1080p @ 24 fps is "easier". Again, call me ignorant :)

Aside from that, congrats to the achievement.

Unfortunately 1920x1080 14bit at 24fps is about 88 MB/s which may be more than the 50D and 5D2 could ever handle, in fact I don't think 1592x1062 at 30p even works (you have to framerate override to 24 and that requires about 71 MB/s which doesn't last long)

How big is the frame buffer on the 50D?  I think I heard 60MB

Andy600

The raw_rec buffer on the 50d (as all cameras that are recordings raw video) is split into 31mb chunks and is derived from shoot_malloc memory which in the 50d is 220MB. The early build for the 50d is not yet fine tuned and there is more buffer to be had which, in theory, means longer shooting times at larger frame sizes.

The CF controller is UDMA6 'compatible' which, again in theory, is capably of recording data up to 133MB/s. I doubt it will get anywhere close to this but I am pretty sure it can and will match the speed of the 5d2 as it has essential the same hardware controlling the CF card. 'UDMA6 Compatible' is an important phrase in this sense as we don't (yet) know the specifics of the controller chipset and what it's maximum transfer rate is.

The maximum recordable frame sizes are derived from the LV display feed that comes from the sensor. In normal shooting this is a maximum of 1592x1062 and in 1:1 crop (x5 zoom) this goes up to 1992x1080. When everything is tuned and the bugs are ironed out I believe we can achieve continual recording at maximum frame size (normal mode) and very close to it in crop mode. Restricting the frame size to specific aspect ratios reduces file sizes still further so my guess is that, on the right card, we can push the 50d to ITS maximums while the 5d2, having larger frame/file sizes may remain slightly under its maximums limited by hardware.

As it states in the first post of this topic "The raw video recording features on the 50d are still at a very early stage of development so don't expect everything to be working flawlessly." so just give it time to mature ;)
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

GregoryOfManhattan

@Andy600 yes agree very early and experimental

@JulianH with the 2013May23 build - i'm getting crashes or lock ups on 3 out of 4 takes. the good news is that most of the .raw files can be processed. this is on a lexar 1000x 32G card - at various resolutions and resulting bit rates 1592x720, 1592x540, 1320x720
like your anamorphic video - do you reset the camera pull battery after every shot?

GregoryOfManhattan

code notes on src/raw.c - posting these so the community can confirm values
@Smeangol - great job figuring this out, getting the values and adjusting the code to make it possible to try .raw on 50D

there are 6 conditionals for each camera type.
Smeangol has inserted values for 5 of these 6 - leaving out the QR_mode (only needed for playback i think).

the comment on the color matrix is "CONFIG_50D // not found yet"
are these values are correct ?


1. LiveView raw buffer address
same as 5D2
#if defined(CONFIG_5D2) || defined(CONFIG_50D)
#define RAW_LV_EDMAC 0xC0F04508

2. Photo-mode raw buffer address
same as 5D2
#define RAW_PHOTO_EDMAC 0xc0f04A08

3. Color matrix should be copied from DCRAW.
#ifdef CONFIG_50D // not found yet
    //~{ "Canon EOS 50D", 0, 0x3d93,
   //~{ 4920,616,-593,-6493,13964,2784,-1774,3178,7005 } },
    #define CAM_COLORMATRIX1                       \
     4920, 10000,      616, 10000,    -593, 10000, \
    -6493, 10000,    12964, 10000,    2784, 10000, \
    -1774, 10000,     3178, 10000,    7005, 10000
#endif

4. raw_update_params - adjust EDMAC RAW file sizes for each camera which have "unused areas, usually black"
#ifdef CONFIG_50D
skip_top    = 22;
skip_left   = 72;
skip_right  = 0;
skip_bottom = 0;
#endif

5. (QR_MODE) // image review after taking pics-  Raw buffer size for photos (using FFT to guess the period of the image stream.)
/*        #if defined(CONFIG_50D) NEED Raw dump to get correct values
        width = 5344;
        height = 3516;
        skip_left = 142;
        skip_right = 0;
        skip_top = 50;
        #endif */

6. Dynamic range, from DxO
#ifdef CONFIG_50D
int dynamic_ranges[] = {1140, 1106, 1051, 965, 872, 772, 679};
#endif

P337

Thanks Andy!

So if we have 220MBs and UDMA6 CF cards then what's the bottleneck currently preventing us from 88MB/s footage (14bit RAW 1080p24)?  I assume it's the bandwidth of that 220MB memory buffer.

Anyone know how many of those 31MB chunks that 220MB buffer is currently passing per second? (I assume about 60MB/s worth) 

@Andy600 What do you mean 'there's more buffer to be had' (size or speed?) and how do you know there's more; have the other cameras gone through fine tuning to improve their buffer that the 50D hasn't yet?  (Aren't they all using the same "shoot memory buffer" and therefore limited to the specific size and speed of each camera's "shoot_malloc"?)  I'm not trying to call you out on anything, just trying to figure out potential for the 50D (maybe look at how fast the 5D2 shoot memory is running)

Also in the 5x zoom mode, is it changing the field of view or is it still recording the same image but in 1992x1080?

Thai

Hey everyone, found out about ML yesterday from Engadget. Installed it last night on my 50D. LOVE IT!

But a quick question for you guys, as there is no audio, do you just buy a good audio recorder to sync up?

I'd like something that's somewhat lowprofile to mount on the 50D's hotshoe. Any suggestions?

What about?

Tascam DR-07mkII Portable Digital Audio Recorder
http://www.bhphotovideo.com/c/product/760375-REG/Tascam_DR_07MKII_DR_07mkII_Portable_Digital_Audio.html



and:

Revo Hot Shoe to 1/4"-20 Male Post Adapter
http://www.bhphotovideo.com/c/product/860269-REG/Revo_sa_cs_14m_Hot_Shoe_to_1_4.html


Is that typically what you guys are doing? Any way to have recording start and stop at the same time as video recording?

Thanks.

Andy600

Quote from: P337 on May 29, 2013, 04:02:44 PM
Thanks Andy!

So if we have 220MBs and UDMA6 CF cards then what's the bottleneck currently preventing us from 88MB/s footage (14bit RAW 1080p24)?  I assume it's the bandwidth of that 220MB memory buffer.

Anyone know how many of those 31MB chunks that 220MB buffer is currently passing per second? (I assume about 60MB/s worth) 

@Andy600 What do you mean 'there's more buffer to be had' (size or speed?) and how do you know there's more; have the other cameras gone through fine tuning to improve their buffer that the 50D hasn't yet?  (Aren't they all using the same "shoot memory buffer" and therefore limited to the specific size and speed of each camera's "shoot_malloc"?)  I'm not trying to call you out on anything, just trying to figure out potential for the 50D (maybe look at how fast the 5D2 shoot memory is running)

Also in the 5x zoom mode, is it changing the field of view or is it still recording the same image but in 1992x1080?

By 'more buffer to be had' I mean that the 50d port is still missing a few ML features and some of the features it has can be refined further. 1% demonstrated this with his Tragic lantern ports for the 600d. The 50d currently shows 4 buffer stars = @120MB so I think there is more. The bigger the buffer allocation = the more time you can record at larger frame sizes BUT it then needs to be written to the card and that is where the bottleneck is. I think this bottleneck (although hardware) is, I believe also being restricted in software atm and might be better calculated better to allow faster card writing.

5x zoom will do exactly what it sounds like. The field of view recorded will be zoomed and exactly what you see in LV. The actual recorded area is denoted by the white box. At 1992x1080 the white box covers the whole sensor.

The 50d runs the unified build of Magic Lantern with module support added by @Smeangol (i.e. raw rec) but there is a lot of 50d specific adjustments that can be made to improve it.

The 600d did not have sraw and mraw but 1% enabled those features and 600d. The 50d currently needs to be in jpeg mode for raw video recording but has sraw. I'm not sure if using those settings will improve things when the code is worked on. Who knows?

@Smeangol has done some amazing work so far. He was also responsible (with help from A1ex) for adding 1080p video when the camera had no video capability but he has very limited time to devote to the project ATM. We are hoping to have 1% onboard soon but need to get him a 50d to work on ;)
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

Andy600

Quote from: Thai on May 29, 2013, 04:27:11 PM
Hey everyone, found out about ML yesterday from Engadget. Installed it last night on my 50D. LOVE IT!

But a quick question for you guys, as there is no audio, do you just buy a good audio recorder to sync up?

I'd like something that's somewhat lowprofile to mount on the 50D's hotshoe. Any suggestions?

Thanks.

Check out the latest RODE video mics with built-in recording. You can also output to a HDMI monitor and feed the monitor signal to an external recorder (one with with sound recording capability, inputs a mic etc) to get a proxy video (in whatever format the recorder uses) with sound. This will help with sync. If you just want sound recording you'll need to go old skool and use a clapper board. The 50d can't record the audio itself but it will show the clapper strike as a visual cue for audio syncing.


Just saw your updated post: The Tascam will work great. You will need to start audio recording before hitting video record. You can't currently control recording stop/start on an external device with the 50d.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

JulianH

Quote from: GregoryOfManhattan on May 29, 2013, 03:42:39 PM
like your anamorphic video - do you reset the camera pull battery after every shot?
I basically left the camera on for about an hour while walking around taking the shots from time to time. No problems whatsoever. Switched it off sometimes, then I have to reload raw rec and put it on the right settings again. But I never had raw files that were not readable. Sometimes when I'm experimenting at home the camera won't power up and I have to pull out the battery first and put it back in. But overall, for a 'very experimental early build', this has worked pretty well for me :)

@Andy: great ideas on the hdmi+audio. If anamorphic live view is possible, I will get a cheap external LCD and still be able to get a corrected anamorphic fov... that would be great as lcd's with anamorphic options are pretty expensive.

Andy600

Just read a couple of tech blogs (there are dozens now) that have picked up on the 50d's new trick and most are getting it completely wrong so for any bloggers reading this without reading the front page:-

1) The 50d did not have any video modes when shipped.

2) Magic Lantern enabled compressed H.264 1080p at 30fps over a year ago. This is NOT the news ;)

3) The new 'raw video' ability captures 14bit uncompressed data from the sensor and writes this as a .raw file to a CF card. THIS is the news :D
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

goldenchild9to5

Quote from: blackroom on May 29, 2013, 08:31:38 AM
yes thanks.
i read somewhere (not sure where now) how write performance improves, like the camera needs to warm up or something? seems to be the case, the first few shots when i start filming are all 3-4 stars and end within a few hundred frames.  a few minutes later i turn it back on, format in camera and try again and its solid 1 star for 4 gigs. i dunno! i'll test my old Lexar 600X next:)

i benched with Crystal Disk Mark and got this:
komputerbay 64gig 1000x
CrystalDiskMark 3.0.2

           Sequential Read :   140.503 MB/s
          Sequential Write :   113.864 MB/s
         Random Read 512KB :    63.013 MB/s
        Random Write 512KB :    23.002 MB/s

Thanks for reporting back, great findings now gotta let the camera warm-up and card I guess where you shooting @ 1592 x 720?

goldenchild9to5

Quote from: Andy600 on May 29, 2013, 05:12:37 PM
Just read a couple of tech blogs (there are dozens now) that have picked up on the 50d's new trick and most are getting it completely wrong so for any bloggers reading this without reading the front page:-

1) The 50d did not have any video modes when shipped.

2) Magic Lantern enabled compressed H.264 1080p at 30fps over a year ago. This is NOT the news ;)

3) The new 'raw video' ability captures 14bit uncompressed data from the sensor and writes this as a .raw file to a CF card. THIS is the news :D

Say it like it is Andy, let them know... 50D no video + M.L = Raw 14bit recording incredible

goldenchild9to5

What I don't get is how come some people are having problems with their 50D's crashing and other ok from the same build.  Does it have to do with the way you load ML on CF cards, or is it a something missing in the folders.  Don't have my 50D yet it's on his way, just trying to get a ball park answer so when I get mines it can run as smooth as @Julian.H's 50D.  @Andy600 hope you get that card soon man so you can start testing. 

GregoryOfManhattan

had a chance to test a little bit more the unified merge build and see that it records well up to around 1437 frame - 60 seconds.
this merged code can record with graphics overlay enabled at a wide variety of bit rates, just crashes if recording goes past 60 seconds.

the interface is different now and to record,
-live view on
-modules enable raw_rec
-frame rate 24
-Raw Rec - turn off sound adjust width and aspect ratio as desired
- hit live view
- half press shutter to start a regular video (may not be required)
-stop regular video
- now hit live view again to record raw
- turn off before frame 1437 or the camera crashes.

http://pic.twitter.com/rrxmC8OupA

Andy600

6 buffer stars :)  ::)

@GregoryOfManhattan - Did you try crop mode (5x zoom)?
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

simulacro

Quote from: JulianH on May 29, 2013, 01:48:07 PM

If I try 5x or 10x zoom the camera crashes almost instantly. When do you experience problems?

That happens also in the 550d. When i tried the 10x zoom the camera crashes.

i found a trick: I zoom 5x, push the menu button twice, then the camera goes to liveview again

goldenchild9to5

Quote from: GregoryOfManhattan on May 29, 2013, 07:29:11 PM
had a chance to test a little bit more the unified merge build and see that it records well up to around 1437 frame - 60 seconds.
this merged code can record with graphics overlay enabled at a wide variety of bit rates, just crashes if recording goes past 60 seconds.

the interface is different now and to record,
-live view on
-modules enable raw_rec
-frame rate 24
-Raw Rec - turn off sound adjust width and aspect ratio as desired
- hit live view
- half press shutter to start a regular video (may not be required)
-stop regular video
- now hit live view again to record raw
- turn off before frame 1437 or the camera crashes.

http://pic.twitter.com/rrxmC8OupA

Great findings.. so that means without the graphic overlay's it should record flawlessly than.  Also this unified build you talking about download link is on the first page right? 

goldenchild9to5

Quote from: Andy600 on May 29, 2013, 07:33:38 PM
6 buffer stars :)  ::)

@GregoryOfManhattan - Did you try crop mode (5x zoom)?

Andy your right it has 6 Stars (*.....)  What happened there 2 more stars added in this new build?

JulianH

Quote from: goldenchild9to5 on May 29, 2013, 07:54:57 PM
Great findings.. so that means without the graphic overlay's it should record flawlessly than.  Also this unified build you talking about download link is on the first page right?
You're making conclusions a bit too fast here ;) The built by Gregory is custom, but another modification than the one that is on the first page here.

Things are looking good though and it sure looks like the 50D has more headroom then what's possible right now. Don't rush things. I'm sure it should be possible to get it up and running as smooth as it has been for me, with the right card. CF cards seem to be a bit of a guess. Not all cards (even same manufacturer and specifications) are equal.



goldenchild9to5

I understand still early, is Gregory part of the ML coding team?   

Andy600

Quote from: JulianH on May 29, 2013, 08:01:24 PM
....CF cards seem to be a bit of a guess. Not all cards (even same manufacturer and specifications) are equal.

I wouldn't know about such things thanks to the local post office >:( I'm sure it would be faster for me to just make one LOL
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

JulianH

/Edit: forget what I wrote here.. premature conclusions, have to check how/if the footage actually works.

GregoryOfManhattan

@JulianH if footage is looking good, we will need a few more 50D testers to review build.