usb hard drive raid tera storage

[nightracer]

Hi all,

I was thinking about some cheap way to set up a few TB in a storage center with raid and an encrypted filesystem, and I would like some reactions. Like whether it is feasable, any good, really as cheap as I think, ...

So here are the parts:

10 * 400GB IDE drives (Western digital sells around 260 a piece)
10 * usb hard drive enclosure (30 something a piece)
3 usb hubs (also some 30 a piece)

=> total cost = 10 * 260 + 10 * 30 + 3 * 30 ~ 3000

The set up is as follows:

2 usb hubs are connected to the third. That should leave 10 usb ports free on the hubs. Connect 2 * 4 drives to the 2 completely free hubs, and connect the last 2 drives to the third hub.
Finally, connect the third hub to the PC. For raid, I would use the software raid in the linux kernel.
In raid 5 according to the kernel, this should give you the following storage capacity: 400 * (10 - 1) = 3600 GB of storage.
I would choose raid 5 for the benefit of redundancy.
On top of the raid array, I thought I would place LVM and dm-crypt.

Perhaps instead of 1 connection to the PC, I could use 3 usb ports on the PC, leaving 12 usb ports free and the possibility to add 2 extra drives. The price would be about 3500 and the storage capacity would be about 4,4 TB, and also increasing the throughput, from 1 * 40 MB to 3 * 40MB

It would just be to store data, logs and output. So I don't really think very fast access is needed.
So, how about it, does anyone think it is a good idea?

Regards,

Nightracer

[Dlareh]

No, in general multi-USB-device raids are a very bad idea. USB is not very robust, both in hardware and software. Once broken connection to one drive, for whatever reason, and you've got to rebuild the whole raid which can take days. 2 failures at the same time, and you lose everything.

Instead of spending $400 on USB stuff, assemble a proper NAS. For instance:

http://www.newegg.com/Product/Product.asp?Item=N82E16811152017

Or, cheaper, buy a non-redundant PSU for one of these, which are not hot swappable.

I personally have a PC-V2000A with eight assorted 200GB and 300GB drives... it's the greatest ATX tower ever made.

But if you really want headless external storage, how about a raid 5 with four of these: http://www.zipzoomfly.com/jsp/ProductDetail.jsp?ProductCode=102670 . Less connections to worry about, and those can do firewire.

Additionally, I don't think dm-crypt would be a good idea on an array of this size. Too many things can go wrong. It's not production-ready in my experience.
_________________
"Mr Thomas Edison has been up on the two previous nights discovering 'a bug' in his phonograph." --Pall Mall Gazette (1889)
Are we THERE yet?

[Genone]

Also with your USB solution you'd need a lot of power plugs (one per disc and one per hub), I'd really recommend you build a proper fileserver instead, for the cost of the usb equipment you can already get a nice (though cheap) barebone. Or just get one or more Terastations

[Dlareh]

[nightracer]

I looked out after a serial ata raid controller, it does raid 5 and supports 8 SATA sockets.
Made by adaptec. So, while I was already thinking about a new PC, perhaps one of these and a lot of drives can also do the trick.
At the website it says, that linux is supported, so that's great. (pretty cheap also)

For the encryption, I'm going to look around a bit and see what other people experiences are.
Thanks for the suggestions,


Nightracer