LVM on Raid5 on AoE (Storage Cluster)

[skceb]

Hi Folks,

for a hosting projekt which will need a lot of space and might grow quite fast i did some research how to build an affordable storage cluster.
Now i came up with:

Setting up a buch of boxes, serving as AoE devices.
Now putting them on the so called AoE-Client in a software raid5 to ensure data security. (raid5 accross the network, e.g. with 3 partitions on seperate aoe-nodes)
but eh- it must be scalable, so now i would put LVM on top of the Raid5('s).

Now my misery is that i dont have the servers yet, so i cant test this solution.

Anybody have set a storage cluster up like described, or has a solution similar to the needed features scalability, security (network AND data itself) affordability?

I highly appreaciate any comments from ppl using those techniques, and - of course - i will post my setup as soon as finished (and working eh ;) )


regards

[avendesora]

Any reason why you're not using "standard" storage stuff (direct attached SCSI shelves or small SAN solutions?)
Depends on the volumes & price range, but what you describe sounds very complex, and unless you can use the
CPU cycles on those other servers, you're wasting money (and power).

[skceb]

well it has to be cost effective, i cant afford to use "ready to use" storage systems like you describe.
i dont know how much cpu power will be comsumed by AoE, so thats a question hopefully will be answered in this thread.
i could use very cheap cpus for the aoe-nodes, like sempron/celeron if aoe does not use a lot of power.
and with a second nic, i could use the aoe-nodes as webserver nodes too. even if the data is so transfered twice, within a GBit Network that should not be a problem at all.

[MrUlterior]

I played with AOE to mount several TB of storage; ditched it in favour of clusters of NFS servers backed by LVM & RAID1 (software/mdadm) for performance reasons. AOE performed badly and generated high load & memory usage on the server leaving it little for actual application use.
_________________

Misanthropy 2.0 - enough hate to go around

[skceb]

for myself i thought about the following sheme:

5 aoe-nodes, each with 5 hds (maybe a 6th hd for the system, or ill boot the system via lan) so a raid5 doesnt waste that much space.

on the "aoe-client" ill create 5 raid5 arrays, each with 5 drives from all 5 aoe-nodes, so no data loss occurs even if a whole node brakes down.

did u have the high mem-load on the aoe-nodes, or on the client?

[MrUlterior]