uncompressed 14-bit RAW video recording

[bumkicho]

The tearing fix is eating quite a bit of RAM. Exmem is not yet fully understood.

That explains! Thanks.

[1%]

But there is the 18MB at the end.. .only needs 10... so if we were to do 30-30-30-30 and then the 18 is left over there is enough for the double buffer. Right now its 30 getting taken for the buffer and 18 left over at the end?

*my write speed takes a dive when it gets into that last buffer but with only 3 it was doing much worse. It would hit that 3rd one and then start skipping.

[a1ex]

Then alloc the fullsize at the end. On 5D3 this doesn't work, because it has 4x32M and a bunch of tiny ones.

[squig]

macgregor wrote "since there's no metadata on the raw files, there are no color profiles asigned to the images. Camera raw I suspect is using the standard adobe color profile, which in my opinion sucks. Canon profiles are much better. Adobe over saturates blues and skintones are less nice. So I wonder if the ML guys could apply the canon profiles to the dng so we could fix this. We could even use VSCO film raw picture profiles, for some extra fun."

[1%]

When I try placing
fullsize_buffers[0] = shoot_malloc(10*1024*1024); /* any image size bigger than that? */
   
after allocating all of the buffers I get a memory error.

[eatstoomuchjam]

Has anyone tried this raw shooting with a hdmi monitor? I assume it won't affect anything, but want to confirm it if anyone tried.

I just shot a 4G clip with an external monitor plugged in.  ML reported 8 skipped frames at 1080p on my Lexar 1000x card which is the same as what it reports when I'm on the internal screen.  I haven't looked at the footage, but I'd expect it to be fine.

[bumkicho]

I just shot a 4G clip with an external monitor plugged in.  ML reported 8 skipped frames at 1080p on my Lexar 1000x card which is the same as what it reports when I'm on the internal screen.  I haven't looked at the footage, but I'd expect it to be fine.

I just tried it myself and yes, it seems to work fine except for one. Did you notice oddity on frame overlay when shooting 1080p? It looks way off center. Although footage looks good.

[Jonas]

Would it be silly to think we could make a CF to CF cable and then use an apple camera connection kit to transfer the files directly to an ipod/iPhone/ipad?

[1%]

I think would be better if it used shoot malloc contig for all buffers and just use the last smallest one for the double buffer.

1/2 working.. now get 4x32 buffers then freeze. Error handling is why it wasn't working... doh

Hmm... I think it has to do allocate contiguous for all buffers and then use the last smallest one. I can't get it to use shoot_malloc after running the allocation loop.

[Dash]

You have RAW files. You can apply (create) any color profile you want on your favourite RAW software after all.
I think there are more important things to do right now.

macgregor wrote "since there's no metadata on the raw files, there are no color profiles asigned to the images. Camera raw I suspect is using the standard adobe color profile, which in my opinion sucks. Canon profiles are much better. Adobe over saturates blues and skintones are less nice. So I wonder if the ML guys could apply the canon profiles to the dng so we could fix this. We could even use VSCO film raw picture profiles, for some extra fun."

[JohnBarlow]

My first post, so my thanks and congratulations go out to all the devs involved in bringing this home.

I saw this post

I don't think the UHS-I pins on the cards are supported by the Canon DSLRs, only the normal interface. So you're still going to get the normal SD card speeds whether it's UHS or not. I have been designing a device that plugs into the SD card slot of the DSLR and accesses the SPI pins. An ARM chip then emulates an SD card with data storage on a FAT32 partitioned SSD. This would allow very high data speeds. That will erase the problems of the SD card being too slow, but I have no idea how fast the SPI Bus on the canon cameras can go. Does anybody know of the max mhz the bus can run at?

Thanks

and I also would like to vent an idea with all concerned.

For starters check out this vid--->
http://www.youtube.com/watch?feature=endscreen&NR=1&v=qgwTCn_X8Wg

This geeza managed to replace the SD Slot controller board on a 450D with complete instructions. At time 3:16 you can see the board, which by the way only costs $30 from Canon spares.

This got me thinking - what if the board from a 700D has a faster transfer rate and can be used in the 550D/600D etc OR what if a cable can be hooked up to an exterior device like a CF or SSD?

I dont have the skills but a hardware guru could in principle cook something up.

Share your thoughts......

[deleted.account]

macgregor wrote "since there's no metadata on the raw files, there are no color profiles asigned to the images. Camera raw I suspect is using the standard adobe color profile, which in my opinion sucks. Canon profiles are much better. Adobe over saturates blues and skintones are less nice. So I wonder if the ML guys could apply the canon profiles to the dng so we could fix this. We could even use VSCO film raw picture profiles, for some extra fun."

There's no color profile assigned because its raw, color primaries and therefore gamut are not even defined, when raw is 'developed' in whatever app used you define the color space, dcraw for example lets you develop as XYZ, Prophoto, Adobe RGB or sRGB, lets you choose any gamma to be applied, so use a color managed app, define your working color space, choose different working spaces, import dng's, compare histograms and channel clipping, go from there.

I believe for example Apple products choose ProPhoto by default if no color space is chosen.

Another thing dcraw will do is let you set sensor saturation default which varies with camera model and manufacturer to avoid the magenta tint.

[Kuky]

macgregor wrote "since there's no metadata on the raw files, there are no color profiles asigned to the images. Camera raw I suspect is using the standard adobe color profile, which in my opinion sucks. Canon profiles are much better. Adobe over saturates blues and skintones are less nice. So I wonder if the ML guys could apply the canon profiles to the dng so we could fix this. We could even use VSCO film raw picture profiles, for some extra fun."

Colour profiles have nothing to do with raw formats. When you transcode to an RGB format you can assign a colour profile to it.

But what can you do is make camera calibration (dcp file) by shooting a colourchecker passport card (macbeth). But soon you'll find out that LR doesn't let you assign the dcp file to the dng files created by the raw module. That is happening because LR doesn't recognise the DNG as being created with a Canon 5D (or whatever). LR thinks it is clever and doesn't let you apply custom camera calibrations.

But RawTherapee will let you do that (much clever program to decode raw) and you end up with a colour managed shooting with the Canon RAW.

Regards,
Cristian

[VectorZ]

The community (Or Canon) should buy you guys a C300.

This would actually turn the C300 into a professional Tool.

[tackyattack]

My first post, so my thanks and congratulations go out to all the devs involved in bringing this home.

I saw this post


and I also would like to vent an idea with all concerned.

For starters check out this vid--->
http://www.youtube.com/watch?feature=endscreen&NR=1&v=qgwTCn_X8Wg

This geeza managed to replace the SD Slot controller board on a 450D with complete instructions. At time 3:16 you can see the board, which by the way only costs $30 from Canon spares.

This got me thinking - what if the board from a 700D has a faster transfer rate and can be used in the 550D/600D etc OR what if a cable can be hooked up to an exterior device like a CF or SSD?

I dont have the skills but a hardware guru could in principle cook something up.

Share your thoughts......

Very interesting...

I'm planning on running a speed test on the 550d SPI bus to see what I can get. If it can sustain a very high mhz, I might have a few ideas. One being able to record using a computer as a storage device via a ethernet cable. So you'd plug this small SD card connector into the card slot of the camera, then it'd go to a small conversion board, then ethernet to the computer. Much like the SI-2k camera.

Also, I think CF cards communicate with an ATA interface. I think both parallel and serial. So if we could create a connector, it could be very simple to slip it into a CF card camera (5d/7d) and have it connect to an SSD with a PATA/SATA interface. Cheaper, faster, larger drives!

[payne]

Yes, partial. Full sensor mode is only achieved with 1740x1160, but transfer rate is too high for SD controller to record 24p continuously.

Two ideas:

1) Is there anyway to bypass the SD controller to get closer to HD resolutions out of these APS-C cameras that so many of us have (60D, T2i, 600D)? Could we send out data via HDMI? This would remove the speed bottlenecks for at least 1080p out.

2) Is it conceivable in a future update to apply a 2:1 or 3:1 compression of some kind in camera so that we can lower the data rate if the SD controller is our only way out?

I'm very worried that a mentality will set in to abandon work and hope on the vast majority of these Canon DSLRs that most of us can afford. I'm a 60D guy myself due to budget and when the camera needed to be purchased.

[eatstoomuchjam]

I just tried it myself and yes, it seems to work fine except for one. Did you notice oddity on frame overlay when shooting 1080p? It looks way off center. Although footage looks good.

I didn't notice anything weird at 1080p, but I set it down to 1920x720 and the overlay was way off center, yeah.

[coutts]

This got me thinking - what if the board from a 700D has a faster transfer rate and can be used in the 550D/600D etc OR what if a cable can be hooked up to an exterior device like a CF or SSD?

and who will write the new firmware for DryOS to communicate with the changed slot?

[1%]

The slot probably just has power stuff... I doubt it has the IO chip on it.. also IO lines are limited to the MHZ that the SD runs at.

Better off messing with the USB... it has UHS theoretical write, facilities are there for the WFT stuff. Just someone has to figure out if its possible thru camera port or grip.

[JohnBarlow]

and who will write the new firmware for DryOS to communicate with the changed slot?

I appreciate what your sayin, but maybe the existing code just needs a tweak? No one really know at this juncture.

[1%]

I know... I read the datasheet for the IO chip trying to overclock the SD slot as is done on some android. Data integrity be damned

You'd honestly be better off finding the traces for CF. The code for CF and SD is inside the FW for all cameras and that USB hard disk stuff too... These all seem like better avenues than trying to do something thats not possible. Even with UHS the max speed  we're getting is like 40MB/s.

[Peter Linov]

I also do experiments with RAW
This is very interesting, but I need a fast flash drive!
and some video:

[xaled]

@tackyattack
Look at the raw to ssd thread, there is some info on cf adapters.

Tho the wft usb path seams to be more promising.

[aaphotog]

Does anyone know why I continue getting this error when I boot the camera(5D3)
camera was not shut down cleanly. skipping module loading

I get that message after I boot the camera. but it only happens after I've recorded a raw video, then shut the camera down.
Only way I know to get rid of the message is to format the card and reinstall ML

[mattmvpmedia]

I'm using a SanDisk Extreme Pro 95MB/s 16GB card on my 600D (T3i). When shooting 960x540 raw I get a few dropped frames but not bad. I shot a comparison test of a scene with 1. Normal Canon 1080p h.264 quicktime movie settings and then 2. Magic Lantern cropped sensor 960x540 raw. I used raw2dng on Mac and then After Effects to process the dng files for the 960x540 and exported a ProResHQ movie.

There is currently too much tearing and blocking of magenta blocks for the raw footage to be usable, but the detail in the frames that don't have the tearing is an incredible improvement over the h.264 quicktime movie. Even when I blow up the 960x540 ProRes to 1920x1080 and compare it with the standard Canon 1920x1080 h.264 file the footage shot 960x540 raw blows the h.264 away in terms of detail and dynamic range. Love it.

I'm definitely looking to get a 5D mark iii now, but if the tearing and blocking can be fixed on the 600D it is a huge improvement to what it currently shoots in h.264, even when shooting 960x540 raw. Fantastic job by the ML team!