Magic Lantern Forum

Using Magic Lantern => Raw Video => Raw Video Postprocessing => Topic started by: mixmastermike on June 06, 2013, 07:05:53 AM

Title: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: mixmastermike on June 06, 2013, 07:05:53 AM
Having Magic Lantern allow me to shoot Raw video has been a christmas present come early. You never really understand until you actually experience it yourself.  The post production possibilities with raw lead me to believe I have an opportunity at creating much more professional work. Weather or not I can pull it off remains to be seen ;) I do know however that recording time is very limited and big, fast, reliable CF cards that record 1920 x1088/1080 are expensive.

In the event that some of the lovely people who are working on an SSD solution get it to work this might not be too important, but for now I thought I would share my workflow.

I have a professional shoot that Im going to be working on and that gives me a lot of headaches.

CF Cards, a Lexar 128Gb x1000 only holds around 30 mins (I am lucky I have two for this project)

The time to dump the CF card to my Macbook pro via USB2 is over 1 hour

The card's cost a fortune, so buying a third or fourth is out of the question

So how can I recycle the cards quicker?

I'm not taking any credit for this build as it wasn't mine, however It does work and it cuts the transfer time by over 1/3rd.

It's late here and I really need to spend some of the evening with my wife, so please hold tight for pictures of proof of concept, I will upload some screenshots and pictures in the morning (PST) and edit the post.

Using this set up (below) I was able to dump 128Gb from the CF card to my HD in just under 20 minutes, thats nice compared to over 1 hour for USB2.

http://texassoundguy.blogspot.com/2013/04/creating-thunderbolt-compact-flash-card.html (http://texassoundguy.blogspot.com/2013/04/creating-thunderbolt-compact-flash-card.html)

I'm guessing that this is probably the fastest available CF card reader on the market and I'm happy with it! I can confirm it works, I ordered it all from a retailer with a 30 Day guarantee!

Next Projects....

Highpoint Esata 6Gb/s hub and PCI-E card to test some more streamlining options of the workflow, now wouldn't that be nice if we get SSD's to be the native format.

Side Note,

Does anyone know if ACR is using Multi core processing power? Please let me know. My next thought is how do we improve the time it takes to convert the DNG's to TIFF's. Right now, it takes the same time for my quad core MacBook Pro and my i7-990X Hackintosh 6 core running at 4Ghz whats up with that?
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: Pelican on June 06, 2013, 08:40:04 AM
The USB 3.0 card readers (even the cheap ones) can read the card's maximum speed so how the thunderbolt can be faster than that?
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: mixmastermike on June 06, 2013, 09:39:28 AM
In all honesty I'm not too sure, but I know I don't have USB3 on my MacBook Pro.
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: scarluuk on June 14, 2013, 05:06:01 PM
If i'm not mistaken, USB 3.0 builds the dataflow up to the maximum speed while Thunderbolt is always at maximum data speed.
At least that's what they told me at the apple shop
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: mageye on June 14, 2013, 05:10:37 PM
Thunderbolt = 10 Gbit/s × 2

USB 3.0 = 4,800 Mbit/s

UltraDMA Mode 7 = 1,336 Mbit/s (167 MByte/s)

Source: https://en.wikipedia.org/wiki/List_of_device_bandwidths (https://en.wikipedia.org/wiki/List_of_device_bandwidths)

;)
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: scarluuk on June 14, 2013, 05:12:55 PM
Lovely numbers, now if only our cards would reach up to those speeds then it would be a lot easier to copy files :p
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: snefferdy on February 11, 2015, 02:47:29 PM
Just to add a bit more information to what Mageye was saying...

Here are the data transfer speeds:

1000x CompactFlash (UDMA 7) = 167 MBytes/sec

USB 2.0...................60 MBytes/sec
Firewire 800.............98 MBytes/sec
SATA 2.0................300 MBytes/sec
USB 3.0.................625 MBytes/sec
Thunderbolt.........1,250 x 2 MBytes/sec

So, it makes complete sense that mixmastermike found a dramatic increase in speed when he switched from USB 2.0 to Thunderbolt. That meant that he could then transfer data at the CF card's maximum speed of 167 MBytes/sec rather than being limited by USB 2.0's maximum speed of 60 MBytes/sec. That's nearly tripling his transfer rate.

For many mac users, we sadly don't have the option of USB 3.0. We have USB 2.0, Firewire 800, and Thunderbolt. The only one of these three options that can take full advantage of the speed of a 1000x (UDMA 7) CF Card is Thunderbolt. Not even Firewire 800 could handle the 1000x CF card speed.

So there you go.
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: snefferdy on February 11, 2015, 04:40:16 PM
Given what I previously wrote, there is a better (or at least cheaper) solution to getting the full transfer speed out of a 1000x (UDMA 7) CompactFlash card into a Mac that isn't equipped with USB 3.0.

Instead of using the $187 Thunderbolt -> SATA 2 -> CompactFlash method that mixmastermike used (http://blog.texassoundguy.com/2013/04/creating-thunderbolt-compact-flash-card.html), it should be just as fast to go with a $98 Thunderbolt -> USB 3.0 -> CompactFlash. It would require these 3 items:

http://amzn.com/B0056TYRMW
http://amzn.com/B00LOLBBQQ
http://amzn.com/B00BCY1AA2

This is what I am considering doing. Any thoughts?
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: DeafEyeJedi on February 11, 2015, 06:03:22 PM
Hmmm... This is quite thoughtful!
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: baldavenger on February 11, 2015, 06:20:48 PM
I use a Belkin Thunderbolt Express Dock with my iMac to give me USB3 options (along with a Lexar CF card reader), but on a recent on location shoot I used the Kanex Thunderbolt to USB3 adapter and it did a great job.  If what you're after is just the most cost effective way of maximum CF card transfer speed then I would recommend the Kanex and a USB3 card reader (ideally Lexar as that guarantees compatibility with Lexar and Komputerbay cards).
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: DeafEyeJedi on February 11, 2015, 07:07:06 PM
Well I use Hoodman STEEL UDMA Reader w USB 3.0 which is fine.

Though admittedly at times I wish I could do faster... IMO!

@baldavenger: So would I need to try the Kanex w my Hoodman reader to get the most out of it?
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: baldavenger on February 12, 2015, 03:23:04 AM
@ DeafEyeJedi:  How are you using your card reader?  If you already have a USB 3 input on your computer then it probably won't make a difference.  If you only have Thunderbolt then it will.
Title: Re: DIY Thunderbolt CF Reader - Recycling CF cards quicker!
Post by: DeafEyeJedi on February 12, 2015, 06:46:17 AM
I have both USB 3.0 and Thunderbolt inputs on my late 2012 Mac Mini...