| View previous topic :: View next topic |
| Author |
Message |
AeroIllini
Tux's lil' helper
Joined: 01 Feb 2004
Posts: 94
|
 Posted: Sun Nov 20, 2005 12:40 am Post subject: Questions about proposed data server |
 |
|
Hi all.
I'm planning on building a data server for home use, and I had some questions to ask the general forum population before I get started, to be sure that I don't spend money on a stupid mistake.
This will be a fairly large data server, serving my CD collection as flac, all my MythTV programs, and supporting my Blender habit. Basically, it will be the central repository for every other computer on the network. It will NOT be doing any hardcore computation (i.e., compiling, rendering, etc.) ... data serving only.
The proposed layout:
- Motherboard/CPU
- PCI IDE controller card, with 2 IDE channels (Promise ULTRA133 TX2, likely)
- PCI gigabit ethernet card
- Four large identical disks: one in each of four IDE channels as master (two on motherboard and two on expansion card)
- Smaller bootable system disk (as slave) in primary IDE channel
- Gentoo running software RAID-5 and NFS
The questions I would like answered before I start purchasing components:
- What kind of power consumption should I be worrying about? Will 400W be sufficient?
- If all this server is doing is hosting data, and not actually working on it, what kind of processor/memory requirements do I have? I've never used a software RAID setup before and I'm not sure what the CPU overhead is like.
- Where are my data throughput bottlenecks (PCI, system bus, IDE channels, etc.)? My minimum requirement is to stream 12 MB/s MPEG-2 data realtime, but theoretically I would like to achieve the highest speeds possible without spending tons of money.
Please feel free to ask for clarification if I wasn't clear in my descriptions or questions.
And thanks for helping me with my project.
- Kevin |
|
| Back to top |
|
 |
eelke
Guru
Joined: 17 May 2004
Posts: 406
Location: Earth, Netherlands, Friesland
|
| Posted: Sun Nov 20, 2005 7:32 am Post subject: |
|
|
| Quote: | | - What kind of power consumption should I be worrying about? Will 400W be sufficient? |
Don't look at the Watts only a good 350 Watt is better then a bad 450 Watt. Depending on the CPU and the VGA card 400 Watt should be enough. Most harddisks use about 10 Watt so 4 harddisks won't use too much power.
| Quote: | | - If all this server is doing is hosting data, and not actually working on it, what kind of processor/memory requirements do I have? I've never used a software RAID setup before and I'm not sure what the CPU overhead is like. |
CPU overhead is mainly a concern during writing (or when one disk has failed) because it has todo XOR calculations while writing. When reading it doesn't have todo XOR calculations and the overhead will be small. So if you mainly read don't worry about CPU overhead. 512 MB of memory should be plenty.
| Quote: | - Where are my data throughput bottlenecks (PCI, system bus, IDE channels, etc.)? My minimum requirement is to stream 12 MB/s MPEG-2 data realtime, but theoretically I would like to achieve the highest speeds possible without spending tons of money.
|
With one drive per channel your IDE channels will be able to handle the load however a single PCI bus has a maximum of 132 MB/s. 4 drives could reach a sustained transferrate upto 200 MB/s which a single PCI bus isn't capable of handling. Ofcourse most of the time they won't reach it so you probably shouldn't worry about it.
You might wish to consider buying a second hand Athlon-MP system because for servers two slow CPU's are often better then one fast CPU and most Athlon-MP mobo's have 64bit 66MHz PCI-X slots which with the right harddisk controller card will remove the PCI bottleneck. |
|
| Back to top |
|
|
AeroIllini
Tux's lil' helper
Joined: 01 Feb 2004
Posts: 94
|
| Posted: Wed Nov 23, 2005 7:57 pm Post subject: |
|
|
| Quote: | | CPU overhead is mainly a concern during writing (or when one disk has failed) because it has todo XOR calculations while writing. When reading it doesn't have todo XOR calculations and the overhead will be small. So if you mainly read don't worry about CPU overhead. 512 MB of memory should be plenty. |
Writing will likely be a significant percentage of RAID usage. Since this will be handling all the data from my MythTV system, it will be writing large MPEG files rather regularly (not compressing, though-- that will be done on the MythTV box's video card), and as part of the Blender habit, the server could, in the future, be hooked up to a small render farm which is saving frame data as individual png files. The CD collection will obviously be mostly read, since I only have to write that data once. Just as a ballpark figure here, would a 1.0 GHz processor be sufficient, or do I need to go higher? How important is name brand in a processor (Intel or AMD vs. Via or other generic)? And what's the cheapest, most reliable dual-processor mobo/CPU combo around?
| Quote: | | With one drive per channel your IDE channels will be able to handle the load however a single PCI bus has a maximum of 132 MB/s. 4 drives could reach a sustained transferrate up to 200 MB/s which a single PCI bus isn't capable of handling. Of course most of the time they won't reach it so you probably shouldn't worry about it. |
Yeah, I think 132 MB/s will be fine; since no one will ever be working directly on the server, all data must be sent through the Gigabit network. Gigabit is 1,000 Mb/s, which is 125 MB/s. As long as the PCI is faster than the network, it's not a bottleneck. I was more worried about the IDE transfer rate, but if it hits 200 MB/s (one drive per channel) then it's all good.
One more question: will the system drive on the primary IDE channel as slave affect the data speeds to/from the RAID array? Since IDE controller cards are very cheap, it might be worth my while to put in dual PCI IDE controller cards for the raid array, and save the mobo IDE channels for the system drive.
Thanks for the info; this is all very helpful.
- Kevin |
|
| Back to top |
|
|
tgh
Apprentice
Joined: 05 Oct 2005
Posts: 222
|
| Posted: Thu Nov 24, 2005 12:05 am Post subject: |
|
|
I have an (8) drive box running on a Asus A8V motherboard at the moment. If you can find 5400rpm drives, you can assume 10W per drive. But 7200rpm drives you should probably weigh in at 12-15W (active) with overhead for startup. The power-supply that I'm using on the 8-drive unit is a 560W ThermalTake Silent Power. I've also built boxes with (4) drives that used 450W Antec power supplies. So if you have a *good* 400W PSU, you will probably be fine. I can't find my old PSU at the moment for when I was running (6) drives. I'm thinking it was either a 350W or 450W.
As for configuration... try to avoid putting (2) drives on the same cable with PATA. (It mostly works, except when it doesn't or causes issues.) Also, you could get away with only using (4) drives instead of the (5) that you are planning on with some creative use of software RAID and LVM2.
I'm pretty sure that you're going to find that RAID5 is slow. I've worked on a (6) drive RAID5 (7200rpm 72GB PATA drives) that could barely manage 6-8MB/s sustained. I've also seen 10krpm SCSI RAID5 arrays with a trio of drives that only manage 15MB/s sustained. YMMV, naturally. I've retired that 275GB 6x72GB RAID5 system and replaced it with a 2x300GB RAID1. Still, even 6MB/s is fast enough to keep up with MPEG2 data rates (which are around 1MB/s, and HDTV is 2.5MB/s roughly).
CPU/Memory - the more the merrier for disk cache. I went with an Athlon64 3200+ with 2GB of RAM for my (8) drive system. But I also plan on running half a dozen server applications on it. You could probably get away with only 1GB if you want to save a few bucks. I spent $450 for my CPU / RAM / motherboard bundle.
My (8) drive A8V system is configured as:
(1) 7200rpm 300GB PATA, motherboard primary IDE connector
(1) CD-ROM, motherboard secondary IDE connector
(1) 7200rpm 300GB PATA, motherboard Promise RAID connector
(1) 7200rpm 200GB SATA, motherboard VIA SATA connector
(3) 5400rpm 300GB PATA, connected to (3) Highpoint Rocket133SB PCI cards
(2) 5400rpm 300GB PATA, connected to (1) Promise Ulta133 TX2
I have (2) RAID1 arrays (the 7200rpm PATAs and a pair of the 5400rpm PATAs). The other (4) disks (1 SATA, 3 PATA) are used as scratch drives. It works out to 550GB of RAID1 space and around 1TB of no-RAID space. The non-RAID'd space is used as online backup targets for machines on the local network. Originally, I was going to use a single Promise SX6000 card with (6) PATA ports, but had troubles so I pulled the Highpoint/Promise cards out of my junk drawer and used them instead. While building the RAID1 arrays, mdstat was reporting throughput of 30-55MB/s during synchronization.
Now, for nitty gritty ideas:
If you're determined to go with RAID5, you could setup the (4) drives as follows. I've simply numbered them hda to hdd for ease. I'll assume 300GB drives (280GB net space), here are the partitions. All of these partitions get labeled as type "fd".:
hda - hda1 128MB / hda2 2GB / hda3 8GB / hda4 270GB
hdb - hdb1 128MB / hdb2 2GB / hdb2 8GB / hdb4 270GB
hdc - hdc1 10GB / hdc2 270GB
hdd - hdd1 10GB / hdd2 270GB
So you're going to lose about 10GB per disk, but that can be used as backup space to make backups of your configuration data. Overall, you'll only lose 30GB worth of space that could've been used for the RAID5 array. Now for software RAID configuration, md0 to md3 are RAID1, md4 is RAID5:
hda1+hdb1 = md0 -- /boot partition
hda2+hdb2 = md1 -- swap partition
hda3+hdb3 = md2 -- root partition (8GB should be large enough)
hdc1+hdd1 = md3 -- backup partition
hda4+hdb4+hdc2+hdd2 = md4 -- RAID5, should net out around 800GB of space. You may wish to install LVM2 on top of this raid device to make it easier to manage partitions and segregate content into separate filesystems.
So even if you lose a single disk, your system won't be affected at all. No need to rebuild the box from scratch and it will continue to keep chugging along until you take it offline to replace the disk. |
|
| Back to top |
|
|
tgh
Apprentice
Joined: 05 Oct 2005
Posts: 222
|
| Posted: Thu Nov 24, 2005 12:17 am Post subject: |
|
|
Two pieces of hardware that you may find useful when attempting to cram extra drives into a case:
1) A 3:2 cooling bay unit (MWave sells these currently, SKU BA21364). This allows you to put up to (3) 3.5" drives into (2) 5.25" drive bays. It comes with a front-mounted 80mm fan that pulls air in over the drives. Personally, for 7200rpm drives, I still only put (2) drives in the cooling unit (which gives a sizeable gap between the 2 drives and better airflow).
2) A 4:3 cooling bay unit. (CoolerMaster sells these.) Same concept as the 3:2 unit, except that you can fit up to (4) 3.5" drives into (3) 5.25" bays. This unit uses a 120mm fan, but does not come with a front cover/filter. Again, for 7200rpm drives, I wouldn't put more then (3) drives into this.
I use a 4:3 cooling bay unit in my (8) drive system. The Antec p160 case only has (4) internal drive bays, but (4) external 5.25" bays. So I have my 7200rpm drives down below and have packed (4) 5400rpm drives into the cooling bay unit. The 5400rpm drives don't get hot enough to care about being packed so close together. (They run cool to the touch, even when tightly packed.) |
|
| Back to top |
|
|
linuxtuxhellsinki
l33t
Joined: 15 Nov 2004
Posts: 700
Location: Hellsinki
|
| Posted: Thu Nov 24, 2005 1:42 am Post subject: |
|
|
& if U want to make it more cheaply, I'd prefer to buy some used server (you can get 'em quite cheaply).
1. There's usually good cooling with big fans (also to hard-drives).
2. Big power supplys & many have even dual hotplug (like mine proliant 2x725W)
3. Integrated scsibuses with hotpluggable SCSi drives for system, & many are with raid-cards (battery powered).
4. Every integrated components are working really well together, cause they're mentioned to work with bigger loads all day long. (even my other older 2x200Mhz Pentium Pro feels really fast on emerging etc.) 
Bad things....
- Usually there's not so good IDE-bus, cause it's mainly mentioned for CD-rom. But it doesn't matter if you're going to buy some IDE/RAID-cards anyway.
- Some are quite noisy with big fans, but if it's in closet then it's not so big problem.
I bought one Proliant3000 with two redundant hot-swappable 725W power supplies, there's about 6 big fans, Dual Xeon mbo (with only one cpu), 6x Hot-swappable scsi-drives (only 1x18GB when I bought it), 2x Integrated scsi-buses, ECC Ram (only 128MB when I bought it) and the price was just 60euros 
Even just for the big power supplys, fans and case for building your own system it would be wise to buy one 
_________________
1st use 'Search' & lastly add [Solved] to
the subject of your first post in the thread. |
|
| Back to top |
|
|
MrUlterior
Guru
Joined: 22 Mar 2005
Posts: 511
Location: Switzerland
|
| Posted: Thu Nov 24, 2005 10:57 am Post subject: |
|
|
| linuxtuxhellsinki wrote: |
- Some are quite noisy with big fans, but if it's in closet then it's not so big problem.
|
What on earth is the point of ventilation/fans if the box is then locked in a closet? It's just a bigger box, so more air to heat. Servers should be in an extremely well ventilated area for best reliability or performance. If noise is a concern, water cool it - the kits are hardly expensive or complicated these days.
_________________
Misanthropy 2.0 - enough hate to go around
|
|
| Back to top |
|
|
augury
l33t
Joined: 22 May 2004
Posts: 722
Location: philadelphia
|
| Posted: Thu Nov 24, 2005 4:34 pm Post subject: |
|
|
| Sata is more reliable than pata and many mobo's use dual or more channels on sata allowing for software raid. you say that most of the use will be from mythtv. do you really need a redundantcy for this kind of info concidering new sata drives have > 95% in some cases 99%+ reliablity over 3-5 years and should still be reliable long after that. Unless of course you have the disks and want to do something with them. Its just been my experience that pata is more expensive, uses more power, slower, tends to be old/leftovers. I've personally seen new disks break within 6 months that are the current pata on the market (one of two) where as I have 2 dozen sata disks that have never lost a bit, spinning for over two years and are quite as mice, never know they're there and cost me less than the pata for more space. and i can get more on a single system. |
|
| Back to top |
|
|
linuxtuxhellsinki
l33t
Joined: 15 Nov 2004
Posts: 700
Location: Hellsinki
|
| Posted: Thu Nov 24, 2005 9:58 pm Post subject: |
|
|
| MrUlterior wrote: | | linuxtuxhellsinki wrote: |
- Some are quite noisy with big fans, but if it's in closet then it's not so big problem.
|
What on earth is the point of ventilation/fans if the box is then locked in a closet? It's just a bigger box, so more air to heat. Servers should be in an extremely well ventilated area for best reliability or performance. If noise is a concern, water cool it - the kits are hardly expensive or complicated these days. |
We're usually having here air ventilation in closet (I don't mean the 0.5 square meter closet). 
And they're all not so big & noisy with many fans, that was an example of one model. I've worked in fixing some dual & quad cpu 466-600Mhz 64bit AlphaServers with 3 powers and those are not so noisy at all. And I've few other smaller proliants, which are also really silent.
So maybe you shouldn't buy one of those big (30-40kg) servers if the noise is the problem. Only if you want to buy it for some quality parts like big case, powers, 15 & 20cm fans (U don't have to use 'em all, 1 big one is good enough), case for hotpluggable scsi-drives, raid-cards etc.....
_________________
1st use 'Search' & lastly add [Solved] to
the subject of your first post in the thread. |
|
| Back to top |
|
|
eelke
Guru
Joined: 17 May 2004
Posts: 406
Location: Earth, Netherlands, Friesland
|
| Posted: Fri Nov 25, 2005 7:07 pm Post subject: |
|
|
| Quote: | Yeah, I think 132 MB/s will be fine; since no one will ever be working directly on the server, all data must be sent through the Gigabit network. Gigabit is 1,000 Mb/s, which is 125 MB/s. As long as the PCI is faster than the network, it's not a bottleneck. I was more worried about the IDE transfer rate, but if it hits 200 MB/s (one drive per channel) then it's all good.
|
You should note that the Gigabit network and the IDE controllers will be sharing the PCI bus. This will probably mean you maxout at 60 MB/s if this is enough you will have to go for PCI-X slots. With PCI-X slots notice that the frequency of the bus is determined by the slowest PCI-device.
| Quote: | | One more question: will the system drive on the primary IDE channel as slave affect the data speeds to/from the RAID array? Since IDE controller cards are very cheap, it might be worth my while to put in dual PCI IDE controller cards for the raid array, and save the mobo IDE channels for the system drive. |
The drives on a single cable cannot be accessed at the same time. As long as the system disk is not being accessed the raid array will not be affected.
| Quote: | I'm pretty sure that you're going to find that RAID5 is slow. I've worked on a (6) drive RAID5 (7200rpm 72GB PATA drives) that could barely manage 6-8MB/s sustained. I've also seen 10krpm SCSI RAID5 arrays with a trio of drives that only manage 15MB/s sustained. YMMV, naturally. I've retired that 275GB 6x72GB RAID5 system and replaced it with a 2x300GB RAID1. Still, even 6MB/s is fast enough to keep up with MPEG2 data rates (which are around 1MB/s, and HDTV is 2.5MB/s roughly).
|
What kind of controllers were these using? What was the readahead value, bad raid-5 performance is often caused by the readahead value being too small. I was sitting behind a Dual PIII 1 GHz with 3x 18 GB 10krpm SCSI drives on a LSI Megaraid controller with 32MB controller cache only yesterday and it did 80 MB/s. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Kernel & Hardware
