| View previous topic :: View next topic |
| Author |
Message |
edboyce
n00b
Joined: 10 Nov 2003
Posts: 5
|
 Posted: Sat Jan 10, 2004 2:40 am Post subject: Seeking advice on installing Gentoo on a monster workstation |
 |
|
Me and my colleagues in a university high performance computing lab are soon to attempt installing Gentoo on a dual Opteron workstation that will run visualizations on a GeoWall (see http://www.geowall.org) and on an additional dozen Athlon64 workstations in the same lab. I'm posting the specs below because I'm interested whether anyone knows of problems I'm likely to run into (despite trying to avoid hardware known to have trouble with Gentoo). We will be glad to report on our installation experience after we've done it. We wonder if (and hope) that Gentoo 2004-1 with official AMD64 support is just around the corner.
One uber workstation to run an 8 ft x6 ft stereo rear projection wall:
Dual Opteron 244 workstation
Tyan K8W motherboard
4GB 333MzDDR EEC memory (4x1GB modules)
PNY Quadro FX1000 (to do quad buffered stereo)
80GB system disk (dual boot Gentoo and Windows 64 Beta)
3x250MB Seagate SATA drives in RAID 5 configuration controlled by a
3WARE 8506-8 8 port SATA RAID controller
DVD-ROM and floppy drive
12 "value" workstations (also capable of running the wall)
Athlon 64 3200
Asus K8V motherboard (I know I have to disable legacy USB, but are there any other troubles with this board?)
1GB 400Mz DDR ram (non ECC)
PNY Quadro 580XGL video card (quad buffered stereo capable)
120GB hard drives, standard CD-ROM and floppy drives.
Thanks for any feedback or heads-up the community may have to offer.
-- Ed |
|
| Back to top |
|
 |
Boris27
Guru
Joined: 05 Nov 2003
Posts: 562
Location: Almelo, The Netherlands
|
| Posted: Tue Jan 13, 2004 8:58 am Post subject: |
|
|
I don't know about the hardware, but I can give you this advice:
Distcc. Your compiles will be over in no time. |
|
| Back to top |
|
|
backebergd
n00b
Joined: 24 Nov 2003
Posts: 9
Location: Boston, MA
|
| Posted: Wed Jan 14, 2004 3:30 pm Post subject: Tyan boards? |
|
|
Which Tyan motherboard are you using? They have numbers, like 2885. Generally speaking, I've seen the Arima dual Opterons running better than the dual Opteron Tyan boards. Fewer problems in general. Also, if you're running DDR 333 or DDR 400, it pays to use vendor-approved memory. The cheap stuff can get you into trouble; it's worth the extra $20 /dimm or whatever the difference is.
Regarding the 3ware card, are you planning on RAID5? If you have the resources, do a benchmark between RAID5 through the 3ware card and JBOD with the 3ware card/software raid 5. My experience was that software RAID was almost twice as fast. We tried the 8506 with eight, six, and four 200GB SATA drives. That benchmark was actually done on a Xeon machine, but you should be able to see similar results on the dual Opteron.
And BTW, 4GB is a lot of ram, but it's not a monster machine anymore. The current quad opteron from Celestica supports 32GB if you use 2GB dimms. |
|
| Back to top |
|
|
haugboel
n00b
Joined: 30 Jul 2003
Posts: 11
Location: Denmark
|
| Posted: Fri Jan 16, 2004 4:17 am Post subject: Re: Tyan boards? |
|
|
| backebergd wrote: | Regarding the 3ware card, are you planning on RAID5? If you have the resources, do a benchmark between RAID5 through the 3ware card and JBOD with the 3ware card/software raid 5. My experience was that software RAID was almost twice as fast. We tried the 8506 with eight, six, and four 200GB SATA drives. That benchmark was actually done on a Xeon machine, but you should be able to see similar results on the dual Opteron.
|
Interesting. At my department we are buying one possibly two Dual 2GHz opterons with 6GB mem and 30 250GB Hard drives (6TB Raid5) attached using 4 8506 cards. They are going to be used for data analysis with large (100GB+) data sets, so we need every MB per sec we can get. This is a purely IO bound application.
Do you think it still would pay off to use a JBOD configuration or does it take up too many CPU cycles with 30 drives? I was wondering that for best performance a two layered structure would be optimal. Ie: Use hardware Raid0 and combine the raid sets of different controllers using software Raid5. Effectively giving raid50. In this way data read - typically a 2-4 GB chunk of data - would spread out over all the controllers and PCI busses.
Do you have any links to the test results 
What would be the optimal filesystem, XFS? ReiserFS? (Reiser4  )
thx |
|
| Back to top |
|
|
backebergd
n00b
Joined: 24 Nov 2003
Posts: 9
Location: Boston, MA
|
| Posted: Fri Jan 16, 2004 2:34 pm Post subject: SATA Raid |
|
|
| haugboel wrote: |
Interesting. At my department we are buying one possibly two Dual 2GHz opterons with 6GB mem and 30 250GB Hard drives (6TB Raid5) attached using 4 8506 cards. They are going to be used for data analysis with large (100GB+) data sets, so we need every MB per sec we can get. This is a purely IO bound application.
|
If this work can be spread across more than one machine your best performance might be to limit yourself to 15/16 disks per box. Is all this stuff going in a 6U chassis?!
2 8506-8s on one PCI bus is enough strain. I don't know what would happen with four. I know that dual channel-bonded Myrinet cards is enough to nearly chew up the bus capacity in a PC architecture.
| haugboel wrote: | | Do you think it still would pay off to use a JBOD configuration or does it take up too many CPU cycles with 30 drives? I was wondering that for best performance a two layered structure would be optimal. Ie: Use hardware Raid0 and combine the raid sets of different controllers using software Raid5. Effectively giving raid50. In this way data read - typically a 2-4 GB chunk of data - would spread out over all the controllers and PCI busses. |
I envision a few problems in this scenario. First of all, definitely put labels on both ends of every cable you use. Second, what about heat? How are you going to cool this monster set of drives? And then finally your question about cpu cycles. The real answer is I don't know your application. You say it's data bound, rather than cpu bound, so you'll have to do your own benchmarking to get the answer to that. Even if your performance is okay, think about the capacity you're losing:
30x250GB drives = 7500GB
for raid 5, each member has to be a least common denominator large. With 30 drives, some controllers would have 7 drives, others would have 8. If you then set those controllers with raid0, they report to linux as 7*250=1750GB or 8*250=2000GB
then with raid 5 you get N disks-1 worth of capacity, and you also lose the extra capacity of the 8-disk controllers (what I'm saying here is buy two more disks and get yourself 8 disks per controller).
You then lose a full controller's worth of capacity, due to parity.
7*250*(4-1)=5250GB which is not the same as 6TB.
Then you format this with your FS of choice, and lose even more.
| haugboel wrote: | Do you have any links to the test results
What would be the optimal filesystem, XFS? ReiserFS? (Reiser4 ) |
No links to the test results, but you can replicate everything I did yourself. Put the drives in, configure them how you want (press Alt-3 at boot, edit /etc/raidtab.conf, mkraid), then use multiple passes of hdparm, or even try out sample data with your actual applicaition.
I'm personally prejudiced towards reiser3. It's very solid, and how much trouble are you going to have if your data gets toasted? I don't think reiser4 is quite ready for production data. XFS is supposed to have some performance enhancements over reiserfs in certain tests. You should try both to see which gets better raw numbers on your data and the way you do your read/write patterns. One warning I've heard from others is that if you do choose XFS, have your box on a UPS, because it doesn't recover from disaster very well. Again, this is just second hand, I've never played with XFS. |
|
| Back to top |
|
|
haugboel
n00b
Joined: 30 Jul 2003
Posts: 11
Location: Denmark
|
| Posted: Fri Jan 16, 2004 4:21 pm Post subject: Re: SATA Raid |
|
|
| backebergd wrote: | | If this work can be spread across more than one machine your best performance might be to limit yourself to 15/16 disks per box. Is all this stuff going in a 6U chassis?! |
We are going to have it in a tower (or should I call it a cube  ), we found one rack mount chassis which where big enough, but it was from a different supplier, so we have to stick with a tower. Problem is we need 6GB memory to keep our data sets in memory, so not only does the hard disks cost, but actually the rest of the hardware weights in at around 1/3 the price.
Therefore we try to stick in as many hard disk in one chassis as possible. Also it is easier to administer one big box than two small.
| backebergd wrote: | | 2 8506-8s on one PCI bus is enough strain. I don't know what would happen with four. I know that dual channel-bonded Myrinet cards is enough to nearly chew up the bus capacity in a PC architecture. |
The TyanS2880GNR motherboard we will use has two PCI-X busses, there should be enough bandwith. Depending on how it goes we may opt for a quad opteron for the second system, if it comes with three or four pci busses.
| backebergd wrote: |
| haugboel wrote: | | Do you think it still would pay off to use a JBOD configuration or does it take up too many CPU cycles with 30 drives? I was wondering that for best performance a two layered structure would be optimal. Ie: Use hardware Raid0 and combine the raid sets of different controllers using software Raid5. Effectively giving raid50. In this way data read - typically a 2-4 GB chunk of data - would spread out over all the controllers and PCI busses. |
I envision a few problems in this scenario. First of all, definitely put labels on both ends of every cable you use. Second, what about heat? How are you going to cool this monster set of drives? And then finally your question about cpu cycles. The real answer is I don't know your application. You say it's data bound, rather than cpu bound, so you'll have to do your own benchmarking to get the answer to that. |
The case is designed for being a file server. Cooling should be ok. We get the system already assembled and stress tested. It is going to be in a server room which has a wind tunnel like ventilation
After doing some research on the topic - ie. googling - I found this link http://home.fnal.gov/~yocum/storageServerTechnicalNote.html which shows that
- It is not trivial to get full performance
- A combination of hard+soft raid is indeed the best
After thinking about it myself I have reached the conclusion that maybe the best way to partition the system is to make two Raid5 arrays with 4 disks on the two first controllers and one on the third and fourth and then make a raid0 scratch disk out of the remaining 6 disks (3 on controllers 3 and 4). Then use software Raid0 - which should be less stressing CPU wise than software raid5 - to combine the different arrays. We would then end up with 6x750GB Raid5 + 2x750GB Raid0 ie. 6TB raw drive space in 4.5TB Raid5 + 1.5TB Raid0 and redundancy isn't that bad either. We have a 42 disk SCSI array partitioned into 7 Raid5 sets of 6 disks and have never had a fatal error. Only thing that makes me nervous is the stability of software raid0. The system is on a UPS.
We are limited to 30 disks by the amount of hot-swap cages we can fit in the system. 10 cages of three disks per cage, but maybe we can work around that with a dirty hack.
| backebergd wrote: |
No links to the test results, but you can replicate everything I did yourself. Put the drives in, configure them how you want (press Alt-3 at boot, edit /etc/raidtab.conf, mkraid), then use multiple passes of hdparm, or even try out sample data with your actual applicaition.
I'm personally prejudiced towards reiser3. It's very solid, and how much trouble are you going to have if your data gets toasted? I don't think reiser4 is quite ready for production data. XFS is supposed to have some performance enhancements over reiserfs in certain tests. You should try both to see which gets better raw numbers on your data and the way you do your read/write patterns. One warning I've heard from others is that if you do choose XFS, have your box on a UPS, because it doesn't recover from disaster very well. Again, this is just second hand, I've never played with XFS. |
Mentioning Reiser4, was just for the hell of it ...After seeing the tests in the link above and other places I think we will try out both XFS and ReiserFS and see what works best. The box should be arriving in 3 weeks from now, but we want to be well prepared.
Thanks a lot for the prompt answer |
|
| Back to top |
|
|
edboyce
n00b
Joined: 10 Nov 2003
Posts: 5
|
| Posted: Sat Jan 17, 2004 12:10 am Post subject: Re: Tyan boards? |
|
|
| backebergd wrote: | Which Tyan motherboard are you using? They have numbers, like 2885. Generally speaking, I've seen the Arima dual Opterons running better than the dual Opteron Tyan boards. Fewer problems in general. Also, if you're running DDR 333 or DDR 400, it pays to use vendor-approved memory. The cheap stuff can get you into trouble; it's worth the extra $20 /dimm or whatever the difference is.
Regarding the 3ware card, are you planning on RAID5? If you have the resources, do a benchmark between RAID5 through the 3ware card and JBOD with the 3ware card/software raid 5. My experience was that software RAID was almost twice as fast. We tried the 8506 with eight, six, and four 200GB SATA drives. That benchmark was actually done on a Xeon machine, but you should be able to see similar results on the dual Opteron.
And BTW, 4GB is a lot of ram, but it's not a monster machine anymore. The current quad opteron from Celestica supports 32GB if you use 2GB dimms. |
Thanks for the feedback.
The board we're using is the Tyan 2885 "K8W"
Yes, we are intending to do RAID5. Our rationale in going with the 3ware card was that we didn't want to burden the Opterons with doing parity calculations and IO business (we want them to concentrate on live simulations) and 3ware seems to have a particularly good committment to Linux (and the Gentoo AMD64 tech notes says that it works). Also, RAID5 is providing us a sort of backup scheme, as we have no other. Our apps aren't mission critical or anything, but a half a terabyte of data is not easy to backup any other way without spending many extra $. Also the 8port 3ware card will let us grow our in-box data storage larger than the onboard software RAID would have let us. We'll be seeing if the 3ware card investment (about 8% of the entire system cost) was worth it.
And, I agree, 4GB isn't ridiculously huge these days (and we can upgrade to 8GB since we're using 1GB sticks. 2GB sticks are WAY too expensive to consider now). We do expect some of our visualization apps to make use of all of the memory we can throw at them. And yes we're using approved memory, which our vendor (www.reasonco.com) will stress test for us anyway (but we'll be doing the gentoo install).
-- Ed |
|
| Back to top |
|
|
doerrfleischfee
n00b
Joined: 11 Sep 2002
Posts: 27
Location: Bayreuth, Germany
|
| Posted: Sat Jan 17, 2004 12:35 am Post subject: |
|
|
Hello, may I ask you if you can tell me when you re encountering any problems regarding to 3ware escalade raid controllers an the tyan s2885 mobo?
My box has a 3ware 8506-8 attached to pci-x bus B on an K8W with four WD740GD discs in raid5.
On my way setting up this machine I am confronted with the following problems:
with kernel 2.6.0 (attached to pci-x bus 1 (133 MHz max)): decreased system stability, kernel module 3w-xxxx reporting fatal error "nothing to flush?", data loss
with kernel 2.6.1 (attached to pci-x bus 1):
system locks at boot when loading 3w-xxxx driver (compiled into kernel)
with kernel 2.6.0 (attached to pci-x bus 2):
3w-xxxx still producing "nothing to flush?"-error, but no data loss
with kernel 2.6.1 (attached to pci-x bus 2):
no big problems, but still bad performance:
| Code: |
# bonnie++ -u root -s 16g -n 100:50k -d /some-directory
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
machina 16G 24959 65 24993 10 17894 5 21846 71 86172 11 211.8 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
100:50:0 327 98 200883 100 41699 99 327 98 188339 100 390 49
machina,16G,24959,65,24993,10,17894,5,21846,71,86172,11,211.8,0,100:50:0,327,98,200883,100,41699,99,327,98,188339,100,390,49
|
I have contacted 3ware in this but still no answer... |
|
| Back to top |
|
|
edboyce
n00b
Joined: 10 Nov 2003
Posts: 5
|
| Posted: Sat Jan 17, 2004 3:01 am Post subject: re: 3ware 8506-8 on Tyan dual opteron board problems |
|
|
Oh no.... your 8506-8 configuration seems identical to what we've ordered... I must look into this immediately and see if I can have my vendor ask 3Ware about this. I made my storage subsystem choice partly based on the fact that in the the Gentoo AMD664 Tech Notes by Brad House at http://dev.gentoo.org/~brad_mssw/amd64-tech-notes.html said that the 3Ware 85xx controllers were known to work and I also took to heart what Mr. House said about software vs hardware RAID. But if there's a conflict between the 3Ware boards and Tyan's boards, then I need to solve it or find another solution.
Can I ask (being a RAID noob), how bad (qualitatively) is the bad performance you experience in the "working" configuration with the board attached to PCI-X bus 2?
I will certainly share any info I can track down on this.
-- Ed
| doerrfleischfee wrote: | Hello, may I ask you if you can tell me when you re encountering any problems regarding to 3ware escalade raid controllers an the tyan s2885 mobo?
My box has a 3ware 8506-8 attached to pci-x bus B on an K8W with four WD740GD discs in raid5.
On my way setting up this machine I am confronted with the following problems:
with kernel 2.6.0 (attached to pci-x bus 1 (133 MHz max)): decreased system stability, kernel module 3w-xxxx reporting fatal error "nothing to flush?", data loss
with kernel 2.6.1 (attached to pci-x bus 1):
system locks at boot when loading 3w-xxxx driver (compiled into kernel)
with kernel 2.6.0 (attached to pci-x bus 2):
3w-xxxx still producing "nothing to flush?"-error, but no data loss
with kernel 2.6.1 (attached to pci-x bus 2):
no big problems, but still bad performance:
| Code: |
# bonnie++ -u root -s 16g -n 100:50k -d /some-directory
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
machina 16G 24959 65 24993 10 17894 5 21846 71 86172 11 211.8 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
100:50:0 327 98 200883 100 41699 99 327 98 188339 100 390 49
machina,16G,24959,65,24993,10,17894,5,21846,71,86172,11,211.8,0,100:50:0,327,98,200883,100,41699,99,327,98,188339,100,390,49
|
I have contacted 3ware in this but still no answer... |
|
|
| Back to top |
|
|
doerrfleischfee
n00b
Joined: 11 Sep 2002
Posts: 27
Location: Bayreuth, Germany
|
|
| Back to top |
|
|
haugboel
n00b
Joined: 30 Jul 2003
Posts: 11
Location: Denmark
|
| Posted: Sat Jan 17, 2004 2:58 pm Post subject: |
|
|
If you check the whole archive(!) from November until now, the 3ware question comes up multiple times, and it seems they found a work around. Furthermore it should only be a problem for people with more than 3.5GB of memory. There are also problems with the AGP controller on the TYAN board. In the SUSE mailing list people upgrade to the latest (beta) bios. YMMV, but if I had known, I wouldn't have ordered a TYAN board.
Just my 2c - Troels |
|
| Back to top |
|
|
edboyce
n00b
Joined: 10 Nov 2003
Posts: 5
|
| Posted: Sat Jan 17, 2004 4:30 pm Post subject: |
|
|
I just did and I'm very concerned. I will also try to get answers from Tyan and 3Ware.
This leads me to another question: Is there another 8 port SATA controller out there that works with Gentoo under the 2.6x kernel on dual Opterons on a Tyan K8W? I wonder if the adaptec ones work without trouble. |
|
| Back to top |
|
|
Moos3man
n00b
Joined: 17 Jan 2004
Posts: 10
|
| Posted: Sat Jan 17, 2004 5:07 pm Post subject: |
|
|
Have you tried with the lateset patches from x84-64.org (dated 16 jan)
I am installing a Tyan with a 4 port 3ware controller atm and will do some benchmarks on it to
I use the 2.6.1-2 patch and it works fine for me
(ftp://ftp.x86-64.org/pub/linux-x86_64/v2.6/RELEASE-NOTES)
*edit
| Code: |
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
titan 16G 36201 99 94009 25 24543 7 21308 74 75273 11 250.8 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
100:50:0 2389 31 138209 99 1835 17 2391 32 129041 100 289 3
titan,16G,36201,99,94009,25,24543,7,21308,74,75273,11,250.8,0,100:50:0,2389,31,138209,99,1835,17,2391,32,129041,100,289,3
|
My 3ware is a 4 ports running raid 10 on 4 80 gig sata disks
*edit |
|
| Back to top |
|
|
Moos3man
n00b
Joined: 17 Jan 2004
Posts: 10
|
| Posted: Sat Jan 17, 2004 6:38 pm Post subject: lspci output |
|
|
| Code: | 00:06.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8111 PCI (rev 07)
00:07.0 ISA bridge: Advanced Micro Devices [AMD] AMD-8111 LPC (rev 05)
00:07.1 IDE interface: Advanced Micro Devices [AMD] AMD-8111 IDE (rev 03)
00:07.2 SMBus: Advanced Micro Devices [AMD] AMD-8111 SMBus 2.0 (rev 02)
00:07.3 Bridge: Advanced Micro Devices [AMD] AMD-8111 ACPI (rev 05)
00:07.5 Multimedia audio controller: Advanced Micro Devices [AMD] AMD-8111 AC97 Audio (rev 03)
00:0a.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 12)
00:0a.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X APIC (rev 01)
00:0b.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 12)
00:0b.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X APIC (rev 01)
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
00:19.0 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
00:19.1 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
00:19.2 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
00:19.3 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
01:06.0 RAID bus controller: 3ware Inc 3ware 7000-series ATA-RAID (rev 01)
02:07.0 VGA compatible controller: nVidia Corporation NV11 [GeForce2 MX/MX 400] (rev b2)
02:09.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5703 Gigabit Ethernet (rev 02)
03:00.0 USB Controller: Advanced Micro Devices [AMD] AMD-8111 USB (rev 0b)
03:00.1 USB Controller: Advanced Micro Devices [AMD] AMD-8111 USB (rev 0b)
03:0c.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link)
04:00.0 Host bridge: Advanced Micro Devices [AMD] AMD-8151 System Controller (rev 13)
04:01.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8151 AGP Bridge (rev 13)
|
[/quote][/code] |
|
| Back to top |
|
|
edboyce
n00b
Joined: 10 Nov 2003
Posts: 5
|
| Posted: Sun Jan 18, 2004 6:33 pm Post subject: 3Ware Knowledge Base ref. re: Tyan board compatibility probs |
|
|
Reading the 3Ware 8506 -- Tyan K8W incompatibility threads, I found reference to the following recent note in 3Ware's Knowledge Base! It seems to imply that the Escalade controllers won't work in Tyan dual Opteron workatation boards at all unless one uses a special riser card (not a good option for tower cases.
| Quote: | FROM 3Ware's Knowledge Base:
Q10964 - Software Configuration: I am having trouble with 2880 and 2885 Tyan motherboards that support AMD Opteron chipsets.
3ware recommends installing the Escalade 7506/8506 series controllers with a 3ware recommended riser card and only in slots 1 or 2 on the Tyan S2880 and the Tyan S2885. These slots are located closest to the memory modules (DIMM sockets) on the motherboards. A 3ware recommended riser card from Adex is required (see attached tech brief). Please review the linked technical brief that describes issues that are known that affect interoperability between 3ware 66 MHz RAID controllers, and certain riser card and Motherboard combinations.
Placing a 66MHz 3ware Escalade RAID controller in slots 1 or 2 without a 3ware recommended riser card, or in slots 3,4 or 5 on these Tyan platforms may impact data integrity. The attached document describes these issues in more detail.
|
The knowledge base entry (#10964) also points to a PDF short paper on the problem. https://www.3ware.com/kbadmin/attachments/TM900-0045-00%20Rev%20A_P.pdf
I'm not entirely sure whether the timing problem 3Ware is talking about here is the same as the IOMMU problems being discussed in threads like
http://lists.suse.com/archive/suse-amd64/2003-Nov/0104.html
I will try early this week to get answers from 3Ware and Tyan. Either I'm going to find indications soon that this problem will be solve imminently or I'm going to have to select another ATA RAID controller (or another dual Opteron motherboard). (In case I've got to do that, are there any recommendations in terms of ATA RAID controllers that ARE known to simply work without problems on K8W's running Gentoo AMD64 with 2.6x kernels? Is this configuration just too bleeding edge at the moment?)
-- Ed |
|
| Back to top |
|
|
doerrfleischfee
n00b
Joined: 11 Sep 2002
Posts: 27
Location: Bayreuth, Germany
|
| Posted: Tue Feb 03, 2004 9:37 am Post subject: |
|
|
Is there any news regarding this issue?
I was wondering if Tyan's Beta Bios, Kernel 2.6.2 or any 3ware solutions can help. |
|
| Back to top |
|
|
haugboel
n00b
Joined: 30 Jul 2003
Posts: 11
Location: Denmark
|
| Posted: Tue Feb 03, 2004 10:14 am Post subject: |
|
|
The place where we bought our box (see above for spec) is specialised in 3ware and only sells Tyan motherboards  So for the moment he doesn't offer any opteron solutions 
To me it sounds like a hardware problem (timing of PCI bus), so I don't see how you can fix that in software. Anyway we wanted to play it safe and he got a MSI mainboard for us. It is not the optimal solution - only has three PCI-X slots - so I would like to know if anybody knows of a mainboard there has 4 or more PCI-X slots on minimum 2 busses and is compatible with 3ware. It has to be opteron, because we need 64-bit adressing. It is ok with a quad CPU board (if they can be had somewhere). The reason for asking is that we probably - if everything works out fine - will build one box more. I will try to post some performance data here, whenever we get the box (approx 3 weeks from now) |
|
| Back to top |
|
|
doerrfleischfee
n00b
Joined: 11 Sep 2002
Posts: 27
Location: Bayreuth, Germany
|
| Posted: Thu Mar 18, 2004 9:44 am Post subject: |
|
|
| 3ware released their next generation of sata raid adapters. According to 3ware the new 9500 series are almost twice as fast as old ones, but I wasn't able to find any information about compatibility with tyan boards... |
|
| Back to top |
|
|
Rhysem
Tux's lil' helper
Joined: 18 Dec 2002
Posts: 76
|
| Posted: Thu Apr 01, 2004 4:08 pm Post subject: |
|
|
Someone asked before if mem=3000M would fix the problem.
The answer is no.
Grr. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Gentoo on Alternative Architectures
