Magic Lantern Forum

Using Magic Lantern => Hardware and Accessories => Topic started by: PeteTomov on June 05, 2013, 01:32:21 AM

Title: Recording RAW to SSD
Post by: PeteTomov on June 05, 2013, 01:32:21 AM
A buddy and I had a crack at making a SATA to CF adapter over the weekend. The technology is nothing special and adapters to use CF cards as SATA drives already exist, but until now I guess  nobody's had reason to want one going the other way.

For the purposes of the test we hard wired our adapter to a Lexar USB 3.0 reader, since this was a DYI job and we couldn't make the adapter small enough to fit into a CF slot, and I didn't feel like hard wiring it to my 5d  ::). The important part is that the OCZ Agility SSD we had for the test benchmarked at exactly the same speed as when it was connected via the SATA cable(around 200MB/s write speed).

While this is as far as the actual test went, yesterday I broke out my mad design skills (not really) and designed a box that screws into the bottom of the camera like a battery grip and holds an SSD and a big enough battery to keep it going for at least 2 hours(around 6000mAh). Then I got my buddy to design the actual adapter so that it fits into a CF card body(that's sorta what he does for a living) that connects to the SSD box via a SATA cable and we've got a complete solution that record as fast as the 5d can write to a cheap-ish(cheaper than CF X1000 cards)memory.

We can't actually manufacture this our self so we contacted some manufacturers to get prices and I was just wondering if you guys are generally interested in that sort of a device. We're probably going to have to order at least 200, so I was thinking that if there's interest maybe I can start a kickstarter project. Obviously, the more we order the cheaper they are going to be so I was even thinking that if we can get a lot of orders we can donate some of the money to Magic Lantern guy to continue doing their magic  :).

Anyway, I just wanted to get your thoughts on the subject. I will have some renders of the design in a day or two, and I'll also update you on potential price as soon as we get one.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: Audionut on June 05, 2013, 01:47:50 AM
I'm sure you will get much more interest if you can post some video of the design in action, showing things like recording speed.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: yobarry on June 05, 2013, 01:56:31 AM
Quote from: Audionut on June 05, 2013, 01:47:50 AM
I'm sure you will get much more interest if you can post some video of the design in action, showing things like recording speed.

I agree, if you can show examples of what's capable and get yourself a nice battery grip mimicked design, you might find yourself  with a sweet kickstarter!
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: PeteTomov on June 05, 2013, 01:59:58 AM
Quote from: Audionut on June 05, 2013, 01:47:50 AM
I'm sure you will get much more interest if you can post some video of the design in action, showing things like recording speed.

That is a really good point.

If anyone else wants to see anything in particular, keeping in mind what we have for now(read the first post), post what you want see and I'll get to shooting some videos it in the next few days.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: hirethestache on June 05, 2013, 02:00:20 AM
I too would like to see proof-of-concept. Im working with a colleague that is an engineer at Makerbot on an interface, and it'd be great to see results.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: mogs on June 05, 2013, 03:25:37 AM
This CF extender may be helpful in your research.

http://store.calexium.com/en/316-cf-extender-290-mm-with-flat-ribbon.html

Using a CF reader as a lab rat is an excellent idea.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: Roman on June 05, 2013, 03:48:03 AM
So just to be clear though, it hasnt been tested connected to an actual camera?

As that's half the battle by the sounds, making sure it's all a format etc that the camera is willing or capable of interacting with.

I wouldnt be getting carried away with getting 100s manufactured etc at this stage, if it's not been proven to work with a camera yet.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: Cityeater on June 05, 2013, 09:11:45 AM
Good to see some progress in this area, thanks for your hard work. My concern is that for those of us that work with camera rigs the extended battery grip wouldn't be ideal with the equipment we've got and might make some gear (for instance follow focus) incompatible due to the new lens height.
For me, I like everything you've suggested except if I could choose I would prefer to have a simple external box that I can mount. I need a little more flexibility and I find the battery grip inconvenient.
Looks like a good project.

On a side note could I suggest a mounting option on the bottom (if you go that way) so its not a single screw but also has a registration pin like a normal film cam for base plates and so forth.

EDIT
It hasn't been attached to a camera yet? I think you'll find that's the tough bit, its more than just a physical interfacing problem. Looks like a good project though, best of luck.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: heavygrafix on June 05, 2013, 10:10:31 AM
i also thought a solution as battery grip isn´t optimal, so i designed some SSD housing that is directly connected to the Rig Rails. A Battery Pack (10Ah), that is powering both, camera and SSD (or some other stuff on the Rig) is (in my mind) the best solution.

if you did not saw my rendering yet:

(http://www.heavygrafix.de/ZZD2s.jpg)
(http://www.heavygrafix.de/ZZD3s.jpg)
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: PeteTomov on June 05, 2013, 11:06:24 AM
Quote from: mogs on June 05, 2013, 03:25:37 AM
This CF extender may be helpful in your research.

http://store.calexium.com/en/316-cf-extender-290-mm-with-flat-ribbon.html

Using a CF reader as a lab rat is an excellent idea.

I just ordered one of these, thanks for the link. The only problem is that we think it'll lower the transfer rate significantly due to the high frequency which is why we think the actual adapter should be built into a CF card body. But at the very least we can use it to do a proof of concept test with a camera.

...and sorry for getting the wrong forum, I was looking for a Hardware section but it was about 1am here and I must've missed it

Quote from: Roman on June 05, 2013, 03:48:03 AM
So just to be clear though, it hasnt been tested connected to an actual camera?

As that's half the battle by the sounds, making sure it's all a format etc that the camera is willing or capable of interacting with.

I wouldnt be getting carried away with getting 100s manufactured etc at this stage, if it's not been proven to work with a camera yet.

Not really, the SSD acts 100% like a CF card on the other side of the adapter. Also, as I mentioned earlier, the technology is nothing special and has been done before in other forms.

We will get it inside a camera somehow to prove the concept but it probably won't run at full SSD speed.

The thing about the manufacturing is the in order for adapter to work 100% at those speeds the components have to be very precise and that requires machines that we don't have. And ordering one to be manufactured cost pretty much the same as ordering 50.


Quote from: Cityeater on June 05, 2013, 09:11:45 AM
Good to see some progress in this area, thanks for your hard work. My concern is that for those of us that work with camera rigs the extended battery grip wouldn't be ideal with the equipment we've got and might make some gear (for instance follow focus) incompatible due to the new lens height.
For me, I like everything you've suggested except if I could choose I would prefer to have a simple external box that I can mount. I need a little more flexibility and I find the battery grip inconvenient.
Looks like a good project.

On a side note could I suggest a mounting option on the bottom (if you go that way) so its not a single screw but also has a registration pin like a normal film cam for base plates and so forth.

EDIT
It hasn't been attached to a camera yet? I think you'll find that's the tough bit, its more than just a physical interfacing problem. Looks like a good project though, best of luck.

The bottom mount is just so that you can easily use the box without a rig. You can put anywhere you like. The only limitation is that the cable has to be fairly short or it won't work properly.

Quote from: heavygrafix on June 05, 2013, 10:10:31 AM
i also thought a solution as battery grip isn´t optimal, so i designed some SSD housing that is directly connected to the Rig Rails. A Battery Pack (10Ah), that is powering both, camera and SSD (or some other stuff on the Rig) is (in my mind) the best solution.

if you did not saw my rendering yet:


I hadn't seen your design. Looks pretty cool.
We're going for a slightly different approach with a smaller all-in-one box with removable batteries.

Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: PeteTomov on June 05, 2013, 11:11:10 AM
I just scrolled through the forum and noticed how many similar projects there are...should have probably done that before we started work on it  ;D
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: heavygrafix on June 05, 2013, 11:15:52 AM
Quote from: PeteTomov on June 05, 2013, 11:06:24 AM
I hadn't seen your design. Looks pretty cool.
We're going for a slightly different approach with a smaller all-in-one box with removable batteries

so would you use 1,8" SSDs or 2.5" Ones?

i think 1.8" would be better with my design. but 2.5" are cheaper in fact of Gb/$
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: heavygrafix on June 05, 2013, 11:24:32 AM
i just got on funny idea. don't take it serious..... What if we can record directly to DDR3 RAM.. just kidding.


I real like what´s going on these days here. And if somebody needs some rendering, i can do this 4 you awesome guys. And at least because so many people working at similar projects, it would be fine to connect and swop some ideas and experiences. Together it would be awesome to get the kickstarted project done.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: PeteTomov on June 05, 2013, 11:32:00 AM
Quote from: heavygrafix on June 05, 2013, 11:15:52 AM
so would you use 1,8" SSDs or 2.5" Ones?

i think 1.8" would be better with my design. but 2.5" are cheaper in fact of Gb/$
We are actually hoping to be able to record higher the full hd resolution in crop mode with the speed we got (though we still don't know how fast the 5d can write), so 2.5" seems more appropriate considering file sizes.

Quote from: heavygrafix on June 05, 2013, 11:24:32 AM
i just got on funny idea. don't take it serious..... What if we can record directly to DDR3 RAM.. just kidding.


I real like what´s going on these days here. And if somebody needs some rendering, i can do this 4 you awesome guys. And at least because so many people working at similar projects, it would be fine to connect and swop some ideas and experiences. Together it would be awesome to get the kickstarted project done.

I actually considered a CF card raid so the whole thing can be smaller but I also wanted something cheaper.

Also, thanks for the offer. I consider myself fairly decent 3ds Max 'artist' but I realize I'm wrong and take up on that :)
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: Michael Zöller on June 05, 2013, 12:12:27 PM
Pete, can you please post some pictures and benchmarks? Thanks!
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: heavygrafix on June 05, 2013, 06:11:29 PM
Quote from: Michael Zöller on June 05, 2013, 12:12:27 PM
Pete, can you please post some pictures and benchmarks? Thanks!

I wanna see too
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: PeteTomov on June 06, 2013, 03:04:38 AM
The prototype is a my buddy's place where we worked on it. We'll get together over the weekend and take some pictures, maybe some videos and run some benchmarks. We should also have a response on price by then so we might think about setting up the kickstarter as well.

The thing about this project is that while we would love to share out project with world, probably get a small office and hire someone to "handle things", maybe get a couple free adapters for ourselves  in the process, we're not actually expecting to make any serious money of it. We'll keep the price as low as possible and share any extra money we get with the Magic Lantern team who I imagine do this for the for the same reasons we do.

During the week we both have other jobs,well he actually does electronics design which is why we're able to do this in the first place and I myself am trying to start a video game company(it's a long story), so expect the more substantial updates during the weekends.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: Steven on June 06, 2013, 08:28:43 PM
So, have you been able to actually record from the camera to the attached SSD yet?
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: AnotherDave on June 06, 2013, 08:33:15 PM
I'm rather excited to see this thing!  I'd make a video of yourselves using it so people could see it working.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: 5DanielMIII on June 10, 2013, 10:08:33 PM
 
Quote from: PeteTomov on June 06, 2013, 03:04:38 AM

with the Magic Lantern team who I imagine do this for the for the same reasons we do.

As in sharing info for free so more people can contribute to develop? Like they (ML-team) do;-)
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: PeteTomov on June 13, 2013, 11:17:15 AM
Just a quick update since I've been really busy and haven't been able to check forum as much as I'd like.

We finalized the design and got a response from the manufacturer. Initial manufacturing of the first 200 units is going to be a bit more expensive than we anticipated. With the additional cost of batteries,handling, shipping and extra units to cover warranty(we managed to negotiate warranty with the manufacturer but we can't keep people waiting while they manufacture some more) we're looking at a $50 000 kickstarter. This means the price of a shipped unit(removable rechargeable batteries included) will be $250. While obviously larger quantities will bring the per unit price down I was original hoping to keep the price at around $150.

I'll set up the kickstarter as soon we have some material to show but in the meantime what do you, guys, think of the price? I guess it's not too expensive for a 5d accessory but I was really hoping to keep it even more affordable. 

Expect pictures, final specs and stuff soon.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: PeteTomov on June 13, 2013, 11:20:45 AM
Quote from: 5DanielMIII on June 10, 2013, 10:08:33 PM

As in sharing info for free so more people can contribute to develop? Like they (ML-team) do;-)

As in giving people affordable (we obviously can't make hardware free but we are looking to make it as cheap as possible) tools to help them make their work better.  ;)
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: haemma on June 13, 2013, 11:34:46 AM
Sounds good! Hopefully the kickstarter project works well! I definetly will support this.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: 5D3shooter on June 13, 2013, 11:41:33 AM
I'm very interested also, I'm so sick of CF cards.

Just so you know, if you do kickstart this, you will definitely receive more than $50,000...probably $250,000
A LOT of people want this. 
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: squig on June 13, 2013, 12:18:50 PM
Quote from: PeteTomov on June 13, 2013, 11:17:15 AM

We finalized the design and got a response from the manufacturer. Initial manufacturing of the first 200 units is going to be a bit more expensive than we anticipated. With the additional cost of batteries,handling, shipping and extra units to cover warranty(we managed to negotiate warranty with the manufacturer but we can't keep people waiting while they manufacture some more) we're looking at a $50 000 kickstarter. This means the price of a shipped unit(removable rechargeable batteries included) will be $250. While obviously larger quantities will bring the per unit price down I was original hoping to keep the price at around $150.


$250 sounds reasonable and is about what I was expecting. Keep in mind that a Ninja costs $700 and produces an inferior recorded image. My advice is to run the specs and design by the peeps on the forum just to make sure you're on the right track. Personally I think a 2x LPE6 hot swappable battery mount is a good way to go. Some people have been asking for an under the camera mount but this is a bad place to put it, it raises the centre of gravity between the camera and rig/stabiliser and puts the camera too high over rods for a mattebox. The best option is to just put some 1/4" threads on it so people can mount it wherever they want.

EDIT: 6000mAh for 2 hours? Really? The Ninja can run longer with less power.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: Michael Zöller on June 13, 2013, 12:43:59 PM
One thing to think about is - would it technically work with any CF camera/equipment? That would potentially increase the audience.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: PeteTomov on June 13, 2013, 01:22:33 PM
Quote from: squig on June 13, 2013, 12:18:50 PM


EDIT: 6000mAh for 2 hours? Really? The Ninja can run longer with less power.

6000mAh ~2 hours at max SDD speed. At ~90MB/s the battery should last a lot longer, probably at least twice that. Also the SSD will be powered on and off by the camera so it shouldn't draw any power when not recording.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: PeteTomov on June 13, 2013, 01:29:55 PM
Quote from: Michael Zöller on June 13, 2013, 12:43:59 PM
One thing to think about is - would it technically work with any CF camera/equipment? That would potentially increase the audience.
Yes it will, it will act 100% like a fast CF card. The 5D should be able to do 5-6fps in photo mode without running out of buffer with a fast SSD. I'm not particularly sure if anyone needs that but I was planning on mentioning it in the kickstarter project. Somebody might come up with another use.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: albert-e on June 13, 2013, 04:07:39 PM
Quote from: PeteTomov on June 13, 2013, 01:29:55 PM
Yes it will, it will act 100% like a fast CF card. The 5D should be able to do 5-6fps in photo mode without running out of buffer with a fast SSD. I'm not particularly sure if anyone needs that but I was planning on mentioning it in the kickstarter project. Somebody might come up with another use.

We would like to see proof of concept: A CF-SSD adapter plugged into camera and functions. A Canon 5D has the software interface if not mistaken, to work with CF Type II (such as micro drive), so it should work. Pictures (even a video) makes a thousand words.

On the same subject: http://www.magiclantern.fm/forum/index.php?topic=5470.250
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: Walter Schulz on June 13, 2013, 04:15:44 PM
Quote from: albert-e on June 13, 2013, 04:07:39 PM
A Canon 5D has the software interface if not mistaken, to work with CF Type II (such as micro drive)

This doesn't make much sense! Read http://en.wikipedia.org/wiki/CompactFlash#Type_I_and_Type_II

Ciao, Walter
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: AnotherDave on June 13, 2013, 04:21:37 PM
I'm a little surprised that we haven't seen a proof-of-concept video yet...
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: Malcolm Debono on June 13, 2013, 05:21:52 PM
Very interesting project, and sounds like you're moving in the right direction. However, I would make sure that it actually works (using cameras like the 5D) first rather than basing everything on theory. Many others have tried similar stuff which didn't work.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: albert-e on June 13, 2013, 06:32:47 PM
Quote from: Walter Schulz on June 13, 2013, 04:15:44 PM
This doesn't make much sense! Read http://en.wikipedia.org/wiki/CompactFlash#Type_I_and_Type_II

Ciao, Walter

Read the CF Specifications,because it makes sense.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: vertigopix on June 13, 2013, 06:35:56 PM
Quote from: AnotherDave on June 13, 2013, 04:21:37 PM
I'm a little surprised that we haven't seen a proof-of-concept video yet...

Me too...
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: AnotherDave on June 13, 2013, 07:06:13 PM
I'm starting to believe that the cross-over from CF to SATA isn't practical.  In theory, yes, SSD can record at a very fast rate.  But you'd always be limited by the speed of the interface. 

Wouldn't it be more practical (but less cost effective) to construct a CF to CF-RAID to increase the speed?  CF is an IDE interface... there have been IDE raids in the past.  Seems like it'd be the easiest way to go... also, it'd be lighter than a SSD and use less power (probably).
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: pascalc on June 13, 2013, 07:31:40 PM
Quote from: PeteTomov on June 13, 2013, 01:22:33 PM
6000mAh ~2 hours at max SDD speed. At ~90MB/s the battery should last a lot longer, probably at least twice that. Also the SSD will be powered on and off by the camera so it shouldn't draw any power when not recording.

A bit strange.
@ 5v, a 6000mAh batterie gives you 30w during one hour.
As the writing consumption of most of the  SSD is around 3w, it's not 2 but 10 hours that this battery will last.
The converter borad might consume a little bit I guess but it's not significant and in the other hand the SSD won't write continuously.
Your manufacturer didn't advised you that you can save load and space at the battery level ?

Need to see serious proofs before investing money here.

Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: albert-e on June 13, 2013, 08:55:59 PM
Quote from: AnotherDave on June 13, 2013, 07:06:13 PM
I'm starting to believe that the cross-over from CF to SATA isn't practical.  In theory, yes, SSD can record at a very fast rate.  But you'd always be limited by the speed of the interface. 

Wouldn't it be more practical (but less cost effective) to construct a CF to CF-RAID to increase the speed?  CF is an IDE interface... there have been IDE raids in the past.  Seems like it'd be the easiest way to go... also, it'd be lighter than a SSD and use less power (probably).

RED cameras have SSD's...just to let you know. It's  not new to use SATA SSD's. Maximum transfer is >200.MB/s,enough to bite (byte) the dust!

Just get on with your project, it's worth it.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: AnotherDave on June 13, 2013, 10:36:12 PM
Quote from: albert-e on June 13, 2013, 08:55:59 PM
RED cameras have SSD's...just to let you know. It's  not new to use SATA SSD's. Maximum transfer is >200.MB/s,enough to bite (byte) the dust!

Just get on with your project, it's worth it.

Yes, but REDs don't use CF to SSD adapters.  I don't believe you'll get the maximum data rate from a CF (167MB/s) to a SSD.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: Steven on June 13, 2013, 10:45:58 PM
The right adapter will bring the right performance for the job. You just have to find the right adapter...
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: rlasson on June 13, 2013, 11:07:36 PM
My hunch is that the data rate out of the 5DIII will never reach over 167MB/sec as that's what UDMA 7 is rated at. You'd never expect a motherboard with 1.5Gb SATA connection to go over 150MB/sec, or a 3Gb SATA connection to go over 300MB/sec. You'd need a new motherboard to go over those speeds.

Still, if you could reliably record something close to 167MB/sec, that would certain help. Even just having storage capacity go higher than 128GB and also become more affordable would also help and make a CF to SSD adapter marketable.

I don't expect you to show us all the parts you're using or even picture of how you're currently using it, but right now you're just talk. Show us a screen grab of the benchmarks from the camera or upload a raw stream that breaks 120MB/sec and then you'll have our attention. You'll also likely get funded very, very quickly.

Russ
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: albert-e on June 13, 2013, 11:35:40 PM
Quote from: rlasson on June 13, 2013, 11:07:36 PM
My hunch is that the data rate out of the 5DIII will never reach over 167MB/sec as that's what UDMA 7 is rated at. You'd never expect a motherboard with 1.5Gb SATA connection to go over 150MB/sec, or a 3Gb SATA connection to go over 300MB/sec. You'd need a new motherboard to go over those speeds.

Still, if you could reliably record something close to 167MB/sec, that would certain help. Even just having storage capacity go higher than 128GB and also become more affordable would also help and make a CF to SSD adapter marketable.

I don't expect you to show us all the parts you're using or even picture of how you're currently using it, but right now you're just talk. Show us a screen grab of the benchmarks from the camera or upload a raw stream that breaks 120MB/sec and then you'll have our attention. You'll also likely get funded very, very quickly.

Russ

Maximum Transfer Rate Of Early & Late ATA/ATAPI:

Ata-4 Ultra DMA 0,1,2 aka UDMA/33
Ata-5 Ultra DMA 3,4 aka UDMA/66
Ata-6 UDMA 5 aka UDMA 100 48-bit addressing
Ata-7 UDMA 6 aka UDMA/133 SATA
Ata-8 UDMA 7 hybrid drive

Maximum Transfer Rates: Ultra DMA ATA 4-7, 66.7,100,133,167MB/s

The architecture between CFA and ATA devices are pretty similar.

A CF device in TrueIDE mode is exactly identical to any ATA/ATAPI-x hard
disk drive device. In fact some CF devices really are hard disk drives
(and not flash memory devices that look like a hard disk drive). A CF
device has all the same ATA registers, the same ATA command protocols and
mostly the same ATA command set (R/W SECTORS/MULTIPLE/DMA, SET FEATURES,IDENTIFY, etc).

...
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: albert-e on June 13, 2013, 11:49:13 PM
Quote from: AnotherDave on June 13, 2013, 10:36:12 PM
Yes, but REDs don't use CF to SSD adapters.  I don't believe you'll get the maximum data rate from a CF (167MB/s) to a SSD.


RED uses SSD's,I didn't say they use CF-to-SATA SSD's. It is the interface were are talking about: The goal is to:
1. Replace the CF with the ATA or SATA drives, meaning the CF external adapter connected to a ATA drive.The CF-ATA adapter is just a test tool to prove the concept. Red camera's and high end cameras are using the technology. It's been done, it's nothing new.

The CF and ATA/ATAPI specifications are out there, read it. And I thought you knew what you are talking about the "Kickstarter" project.

...
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: AnotherDave on June 13, 2013, 11:54:10 PM
Quote from: albert-e on June 13, 2013, 11:49:13 PM
Red camera's and high end cameras are using the technology. It's been done, it's nothing new.
...

Dude, what I am saying is that I think you're going to lose so much in the conversion to SSD that the data rate isn't going to improve or that it isn't going to be significant enough to use to record raw.

The Red uses a proprietary SSD, BMCC uses SSDs... but SSDs aren't the issue.  It is converting the CF to an SSD which is what I am saying is probably impossible or at very least going to require a lot to do it.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: 5D3shooter on June 14, 2013, 12:09:02 AM
Even if it is capped at 160MB/S won't that get us close to hitting 4K?  1920x1080 only requires ~80MB/S
Then if Magic Lantern team can optimize the code even more.  That might be just enough speed.

We're all setting ourselves up to not be able to live without RAW video, so I hope it works. 

Kickstart a motherboard mod lol, anything to not have to buy a new cam.

http://www.bhphotovideo.com/bnh/controller/home?O=&sku=964119&is=REG&Q=&A=details (http://www.bhphotovideo.com/bnh/controller/home?O=&sku=964119&is=REG&Q=&A=details)
Those sons of *******...lol
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: albert-e on June 14, 2013, 12:31:07 AM
Quote from: AnotherDave on June 13, 2013, 11:54:10 PM
Dude, what I am saying is that I think you're going to lose so much in the conversion to SSD that the data rate isn't going to improve or that it isn't going to be significant enough to use to record raw.

The Red uses a proprietary SSD, BMCC uses SSDs... but SSDs aren't the issue.  It is converting the CF to an SSD which is what I am saying is probably impossible or at very least going to require a lot to do it.

"It is converting the CF to an SSD which is what I am saying is probably impossible or at very least going to require a lot to do it."
How did you know that? And that is why we wish you success in your goal. Your dreams are ours too in this matter, dude.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: rlasson on June 14, 2013, 12:34:42 AM
Quote from: 5D3shooter on June 14, 2013, 12:09:02 AM
Even if it is capped at 160MB/S won't that get us close to hitting 4K?  1920x1080 only requires ~80MB/S
Then if Magic Lantern team can optimize the code even more.  That might be just enough speed.

I like your optimism! Would be really great if they could figure out a way to compress the RAW files while it's in the buffer and then write it out. As it stands, 80MB/sec for 1080P would be 320MB/sec for 4K. 19GB/minute. 1.1TB/hour. That's a lot of data.

Russ
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: squig on June 14, 2013, 12:37:12 AM
Quote from: AnotherDave on June 13, 2013, 10:36:12 PM
Yes, but REDs don't use CF to SSD adapters.  I don't believe you'll get the maximum data rate from a CF (167MB/s) to a SSD.

I have an expresscard CFreader and I can write over 150MB/s to an SSD in my macbook.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: albert-e on June 14, 2013, 12:43:10 AM
Quote from: squig on June 14, 2013, 12:37:12 AM
I have an expresscard CFreader and I can write over 150MB/s to an SSD in my macbook.

There's your proof. "I've been there, done that"
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: vertigopix on June 19, 2013, 08:05:30 AM
So... Where is your proof of concept ?    :-X
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: PeteTomov on June 20, 2013, 12:09:43 PM
Wow, I should really check the forum more often  ::).

To clarify the current situation:

The adapter has not been tested in camera. It has only been tested hardwired into an USB 3.0 reader. The version we put together over a weekend in simply not small enough to fit inside a camera and we're not looking at enough profit to justify taking apart my 5D for testing.

The main problem is the precision required when working with such high frequencies. You need machines to manufacture the components, you can't just solder some wires.I'm guessing that's why the other projects fail since everything else is pretty simple and based on existing technology.

The reason we are 100% sure the adapter will work in camera is the way CF cards work in general. They are basically faster IDE drives and can work with an IDE to SATA adapter although I haven't seen one small enough or fast enough since IDE drives back in the day were much slower. We're simply adding a processor fast enough for SATA3 speeds,batteries for the SSD and packing the actual adapter into a small enough size to fit inside a CF card slot...also the processor is fast enough to emulate anything it has to in case we're wrong  ::).

As I've mentioned earlier,we're not really doing this for profit(we're looking at about 5-10%), we both have other jobs. We're doing it because we want one ourselves. We understand if don't want to take that chance and in that case we simply won't even bother with a kickstarter. Hopefully someone else will get make it so we can buy a couple  ;D

Best we can offer is a video of the adapter working with the reader(in case you're even interested in that), only 3-4 months of production time after the kickstarter and I can personally offer a money back guarantee if it doesn't work.


Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: 5D3shooter on June 20, 2013, 12:50:49 PM
You could've bricked your camera installing ML, cmon man, bust her open and start soldering!

LOL, jk...

Seriously though, I think the profit is there.. Do the kick starter (after u can confirm it works in camera), you will probably pledge $10,000 and receive $200,000.. I see that everyday on there, for stupid shit too.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: g3gg0 on June 20, 2013, 12:58:16 PM
you are sure that it will work in camera?
how does your CF adapter emulate the CIS data that is obligatory for canon?
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: pascalc on June 20, 2013, 12:59:53 PM
Yes please, post a video of the adapter working in the CF reader.
Title: Re: Recording RAW to SSD
Post by: mageye on June 20, 2013, 02:57:37 PM
Why does the word 'bottleneck' come to mind? :'(
Title: Re: Recording RAW to SSD
Post by: PeteTomov on June 21, 2013, 09:43:03 AM
The camera is definitely the bottleneck but we know that's fast enough :)
On another note, I'm releasing two games for mobile devices over the next couple of months and putting together a team to work on a bigger PC game. If the mobile games do well I might just fund the project myself. That is if the Black Magic 4k camera hasn't rendered the adapter useless by then. I'm waiting for footage to make my call on that one.
Title: Re: Recording RAW to SSD(and it actually works...sorta)
Post by: albert-e on June 22, 2013, 03:27:12 PM
Quote from: g3gg0 on June 20, 2013, 12:58:16 PM
you are sure that it will work in camera?
how does your CF adapter emulate the CIS data that is obligatory for canon?

CIS( Card Information Storage ) are basically data stored when (1) an Output Enable strobe is generated by the host interface. It is a read data from the CompactFlash Card in Memory Mode & PC Card I/O Mode, and are located in CIS and configuration registers. Per CF specification, it doesn't say it's from Canon, except there are data bits assigned for specific vendor. Correct me if I'm wrong here...just want to clarify my understanding of the CF specs.
Title: Re: Recording RAW to SSD
Post by: 5D3shooter on June 23, 2013, 04:47:25 AM
Quote from: PeteTomov on June 21, 2013, 09:43:03 AM
That is if the Black Magic 4k camera hasn't rendered the adapter useless by then. I'm waiting for footage to make my call on that one.

Most of us are not gonna buy that.. We want our damn SSD adapter!!!  lol
Title: Re: Recording RAW to SSD
Post by: boniac on June 23, 2013, 07:11:06 PM
Yes. Interested in buying a couple of them. No joke. GIVE IT TO ME NOW.
Title: Re: Recording RAW to SSD
Post by: 1% on June 23, 2013, 08:39:52 PM
I'm cool with just UDMA 6/5 and IDE drives.  Sata SSD combo is nice but IDE is realistic. Can't you emulate the CIS data in the adapter?

I have plenty of 40/80/+ IDE drives collecting dust... camera like the 50D has to be plugged in anyway or you'll be shooting for 5 minutes and drain the battery. Even 5 more MB over the CF card will net me 1080P continuous + cheap media.

Title: Re: Recording RAW to SSD
Post by: albert-e on June 23, 2013, 09:35:48 PM
Quote from: 1% on June 23, 2013, 08:39:52 PM
I'm cool with just UDMA 6/5 and IDE drives.  Sata SSD combo is nice but IDE is realistic. Can't you emulate the CIS data in the adapter?

I have plenty of 40/80/+ IDE drives collecting dust... camera like the 50D has to be plugged in anyway or you'll be shooting for 5 minutes and drain the battery. Even 5 more MB over the CF card will net me 1080P continuous + cheap media.

What is exactly we want to emulate (CIS data) on the interface adapter. What is the CIS (Card Information Structure) data that we need to "crack"?
Title: Re: Recording RAW to SSD
Post by: 1% on June 23, 2013, 09:37:42 PM
g3ggo would know... he made the module that lists some of it, but I doubt that is all.

So basically pretend to be a CF card facing the camera and a normal IDE/SATA/whatever you can muster on the other end.
Title: Re: Recording RAW to SSD
Post by: albert-e on June 23, 2013, 10:02:31 PM
Quote from: 1% on June 23, 2013, 09:37:42 PM
g3ggo would know... he made the module that lists some of it, but I doubt that is all.

So basically pretend to be a CF card facing the camera and a normal IDE/SATA/whatever you can muster on the other end.

Thanks 1%. You gave me an idea now of what these data are; it's a vendor type locked" feature of camera manufacturer. The CIS is a hidden area located outside of the emulated logical or physical drive. It contains information about the card manufacturer, many of the device characteristics and, is sometimes used to "hide" manufacturer specific features (such as the Olympus Panoramic mode, etc. for example).

So, g3gg0 cracked it for 5D3? Okay, so we're off to see the wizard then. "g3gg0", we need your assistance. If we can write to it (CIS), then that would be easy. Isn't it, g3gg0?

...
Title: Re: Recording RAW to SSD
Post by: 1% on June 23, 2013, 10:07:03 PM
Its not 5d3 specific.. it reads the data fine on 50D. Just camera uses the CIS data so if its missing the CF or "CF" won't detect/work.
Title: Re: Recording RAW to SSD
Post by: albert-e on June 23, 2013, 10:19:20 PM
Quote from: 1% on June 23, 2013, 10:07:03 PM
Its not 5d3 specific.. it reads the data fine on 50D. Just camera uses the CIS data so if its missing the CF or "CF" won't detect/work.

If the interface adapter emulates in TRUE IDE mode as specified in CF specifications then it should work, theoretically, just like a CF Type II microdrive. I should be able to connect a CF micro drive just to prove it and see if my Canon 50D recognizes it. Okay, I'll perform the elementary tests and see what happen. But first, let me get a micro drive.

To be continued...
Title: Re: Recording RAW to SSD
Post by: Core2k on October 16, 2013, 05:15:14 PM
If anyone can use it...

(http://s7.directupload.net/images/131016/temp/o6m3z54o.jpg)

http://www.ebay.com/itm/CF-IDE-SATA-SSD-RAW-FB4658-ADA-COMPACTFLASH-ATA-IDE40-/161129010587?
Title: Re: Recording RAW to SSD
Post by: g3gg0 on October 16, 2013, 05:32:35 PM
Quote from: albert-e on June 23, 2013, 10:19:20 PM
If the interface adapter emulates in TRUE IDE mode as specified in CF specifications then it should work, theoretically, just like a CF Type II microdrive.

on 5D3 it wont - microdrives are banned and the code absolutely relies on the CIS being readable.
"True IDE mode" is just a subset of CF and thus no CIS is emulated.
Title: Re: Recording RAW to SSD
Post by: 1% on October 16, 2013, 09:14:54 PM
Holy shit... I hope the above adapter works.
Title: Re: Recording RAW to SSD
Post by: g3gg0 on October 16, 2013, 09:50:28 PM
i guess it will not
Title: Re: Recording RAW to SSD
Post by: 1% on October 16, 2013, 09:53:36 PM
No ic inside the CF case?, just a dumb adapter?

So it would need CIS emulation.
Title: Re: Recording RAW to SSD
Post by: PressureFM on October 17, 2013, 12:29:41 AM
http://www.rcpcomputer.com/automation/FA/CF%20Adapter%20Boards/IDE%20to%20CF/FB4658.pdf

Not sure that helps any.
Title: Re: Recording RAW to SSD
Post by: 5D3shooter on October 18, 2013, 12:51:44 PM
Quote from: PressureFM on October 17, 2013, 12:29:41 AM
http://www.rcpcomputer.com/automation/FA/CF%20Adapter%20Boards/IDE%20to%20CF/FB4658.pdf

Not sure that helps any.

We've all seen this.. old news.

I don't know if it will ever come true =(
Title: Re: Recording RAW to SSD
Post by: PressureFM on October 18, 2013, 04:01:15 PM
Quote from: 5D3shooter on October 18, 2013, 12:51:44 PM
We've all seen this.. old news.

I don't know if it will ever come true =(

It was simply the PDF for the listed product on eBay a few posts up.
Title: Re: Recording RAW to SSD
Post by: Vlad on November 07, 2013, 01:06:54 AM
I am familiar with group of developers who can make the device that the flow of video was written down on SSD. but in their opinion this device will have a small demand. the cost of development is in limits of 20-30 thousand dollars. May we arrange vote to learn real need for such device?
Title: Re: Recording RAW to SSD
Post by: 5D3shooter on November 07, 2013, 02:40:54 AM
Quote from: Vlad on November 07, 2013, 01:06:54 AM
I am familiar with group of developers who can make the device that the flow of video was written down on SSD. but in their opinion this device will have a small demand. the cost of development is in limits of 20-30 thousand dollars. May we arrange vote to learn real need for such device?

Kickstarter...  If your guys can really do this and request 30,000$ ...They will probably raise over 100,000$