high-end database hardware spec

[dcneting]

Im going to setup a database server using the hardware specification listed below:

Dual-Core Intel(R) Xeon(R) 2.8GHz, 2x2MB,800MHz FSB
4GB (2x2048), DDR-2 400MHz ECC 1R Memory
146GB Ultra320 (10K RPM, 80-pin) SCSI Hard Drive
Integrated Dual Intel Gigabit Network Card
Integrated Dual Channel Ultra 320 SCSI Controller
C4: Raid 5 (PERC4e/Di with 256MB cache), ROMB


Im planning to use Gentoo as OS, Apache as webserver and PostgreSQL as my DBMS.

If any of you has any experience setting up high end database server under Gentoo,
mind give me some idea or comments about my hardware spec

Thanks a million.

[Moriah]

I take it you have nultiple hard drives, otherwise why the RAID controller.

My preference is software raid, as linux does that well, but ymmv. Software raid makes it easy to use drives that are not quite identical. I partition the drive so the raid space is a bit shy of a full disk. That way, if I have to stuff a substitute drive into the slot in the middle of the night, I can still get it to work even if the drive is from a different vendor and not exactly the same size or geometry.

I am presently running a server with software raid-1 as a 2-way mirror on a pair of 250 gb ide drives. I have /dev/md running software raid-1 twice on those drives. The first partition is 50 mb for .boot, and the second is the rest of the drive for /. The / is also under lvm2, primarily for the snapshot capability to insure clean backups. This box boots from either drive if the other one is offline, else it boots from the first drive. It will run with either drive in place. The root filesystem, /, boots directly into lvm2. I am using ext2 for /boot, and reiserfs for /.

You might also consider evms instead of lvm2; both work well. evms has clustering facilities, but lvm is more "stick shift", while evms is "automatic". Either would have worked on this box, but I have other configurations where evms is just too smart for its own good, so I use lvm2 there. Since I did not want to have to support both approaches, I use lvm2 on all the boxes now.
_________________
The MyWord KJV Bible tool is at http://www.elilabs.com/~myword

Foghorn Leghorn is a Warner Bros. cartoon character.

[dcneting]

Thanks for replying friend

Yes, Im planning to use multiple SCSI harddisks using hardware RAID. Actually, Im still confuse about which RAID is better for a high-end Gentoo database server: hardware RAID or software RAID?

I want to use up to 5 harddisks. Want is the optimum size for each harddisk? Is it ok 146Gb?

[Moriah]

Please remember that with software raid, only raid-1 can be a boot device, as all disks in a mirror look like a normal non-raid disk to the bios. The striping of other raid setups makes them unbootable from the bios. Hardware raid should not care, as the bios see the disks thru the hardware raid controller, but with software raid, the bios does not see thru the raid controller, since it is in software that has not been booted when the bios runs.

Number of disk in raid5 affects the number of stripes, which affects i/o throughput speed, assuming you have independent channels to each of the disks. With scsi, you can usually have a few disks on a single bus, but with 5 drives, you pobably wamt at least 2 controllers for maximum speed. This of course depends on the controller and the drives. Just add together the data transfer speeds off the disks (not just from the cache) for each of the drives, and if it goes over about 80% of what your scsi controller can handle, add another controller or suffer speed limitations caused by bus bandwidth saturation. With IDE you should always use only one drive per cable, or performance degrades severly, and reliability suffers too. With SATA, of course one drive per cable is your only option anyway.

Bandwidth saturation can be a limitation with hardware raid controllers as well. If you have a single hardware raid controller running 5 drives, then you may hit the same bandwidth limits with the hardware controller that you see with software raid and a single scsi controller. With software raid, it is easy to add more hardware busses to a single raid setup. With hardware raid, you usually do not have this flexibility.

Another consideration is that hardware controllers usually want identical disk drives in all the slots. While this is not a problem when building a new array, consider a few years later when a drive fails and the drives you built the array with are no longer available. You must use a substitute drive. If all drives must be identical, then you must change all the drives which means you must backup the array, take it down, reslot it with the new drives, bring it back up, then restore your backup. That kind of kills any advantage for hot swapping.

With software raid, you can always substitute a bigger drive without problems. I usually format drives in a software raid so they have a single partition that is a bit smaller than the full capacity of the drive. This lets me substitute a similar drive of a different manufacturer later on, even if the substitute drive is a bit smaller. For instance, compare the actual capacity of 80 GB drives from different vendors. They are slightly different. I would format these with say a 75 gb partition and build the raid with these partitions. Then if I have to substitute later on, I am not in trouble.

Size of disks is strictly up to you. You know your application and what you need. Just remember, if all the spindles are spinning at the same speed, then a disk with twice the capacity usually has the same number of cylinders, but twice as many sectors per cylinder, so that means bigger disks are faster disks. So a bigger disk can make your system faster even if you do not need all the capacity.

Regarding raid speed, I did a study about 1 1/2 years ago. The results are at:

http://www.elilabs.com/~rj/raid_timing.html

This study was mainly aimed at raid-1 mirrors running thru aes-256 crypto via /dev/loop, but the figures are still interesting.
_________________
The MyWord KJV Bible tool is at http://www.elilabs.com/~myword

Foghorn Leghorn is a Warner Bros. cartoon character.

[Moriah]

Furthermore, if you run lvm on top of the raid, you can hot swap a bigger drive in one slot, regen the raid, then hot swap another drivce, etc. until you have upgraded all the drives on the box without shutting down the machine. After all the drives are bigger, you can use lvm to expand the logical volume size to use the extra space, then resize the filesystem to fit the expanded logical volume size. Voila! You just upgraded to a larger storage capacity without shutting down the box! Coll, huh?
_________________
The MyWord KJV Bible tool is at http://www.elilabs.com/~myword

Foghorn Leghorn is a Warner Bros. cartoon character.

[Moriah]

Another approach I am considering is to only have raid5 on the box, and to boot from a cd, dvd, or usb flash drive. That way, the initrd in the boot can start the raid5 on the main box before the pivot_root, so after the pivot_root, the system will be running totally from the raid5 without any other mass storage needing to remain online.

This has particular advantages for systems that want to run evrything encrypted, as the boot drive is inserted only to boot the system, then removed after the boot is done. That way, the boot drive acts like an ignition key to start the machine. If a machine must recover from a power failure unattended, just leave the boot drive in it. If security is more important, remove the boot drive.

While I am interested in this approach for servers, it is also very useful for a totally encrypted laptop. Laptop theft is a serious security risk. If the laptop must have the usb key inserted to boot it, then a stolen laptop is just a lost peice of capital equipment, not an intellectual property loss or a security breach. Just make sure you remove the boot key after the boot.
_________________
The MyWord KJV Bible tool is at http://www.elilabs.com/~myword

Foghorn Leghorn is a Warner Bros. cartoon character.