Perc 6/i Raid Card poor performance

afterdani · n00b Joined: 10 May 2010 Posts: 1

Hi!

I'm installing a webserver on a Dell Poweredge R710 and I've chose Gentoo because I like its philosophy and customizing everything. I like very much Debian too, but it doesn't seem so customizable as Gentoo. A friend recommended it to me for its high performance scenarios.

Well, I installed and configured the environment and it was all ok. When I've finished, I have to explain why I've installed Gentoo instead of other distros as Ubuntu or Centos. Is said above: "I want to customize apache & company & other things with an easy method and I want the maximum performance", "We can make a bechmarking comparing Gentoo and Ubuntu Server performance, in equal hardware"

Hehe, Why I've said this?... Ubuntu have better results then Gentoo. The Ubuntu read disk speed was superior than Gentoo. But it doesn't matter! Measure performance is good. I will work it out until the performance will be ok. "Customize can be good or bad", I thought.
Then I've studied the Ubuntu kernel compile config and I chose the same filesystem, ext4. Before I was using reiserfs, the same IO scheduler, deadline instead "cfq", and more parameters (which I've been searching on google) that Ubuntu kernel config have and that are interesting... "Must be something of SCSI parameters", I thought.

Before doing the benchmarking, I was looking at the "sdparm -a /dev/sda" list of parameters in both OS and they were equal (can have a look to them at the end of page *1):
Then I've done "hdparm -tT /dev/sda" in both OS and I get:

Ubuntu results:
/dev/sda:
Timing cached reads: 17042 MB in 2.00 seconds = 8530.35 MB/sec
Timing buffered disk reads: 1322 MB in 3.00 seconds = 440.55 MB/sec

Gentoo results:
/dev/sda:
Timing cached reads: 17138 MB in 2.00 seconds = 8579.26 MB/sec
Timing buffered disk reads: 876 MB in 3.00 seconds = 291.98 MB/sec

The same happens with Apache Benchmarks Utility, obviously. Ubuntu takes 500ms aprox of advantage in response.

I've compared the source code of "megaraid_sas" driver included in the kernel of Ubuntu and Gentoo and "diff" says it's equal.

I found a difference when I do "lspci -vvnn" in both OS (I've attached to the end the full info *2)

I was reading about and I saw that there is a client to configure raid, called megacli, I've installed it and got many options to configure, but my doubt is: Has Ubuntu configured the RAID card with any parameters?

I reply to me: no, because I boot with Ubuntu LiveCD and the results are better. With Gentoo LiveCD installer the results are better too, but not better than Ubuntu.

I upgrade the Raid Card firmware too, with megacli client. I know, is the first thing to do. n.n

Anybody has any idea? I've googled this a lot, but maybe I'm missing something... I'm going astray?

Thank you for any suggestion, and if anybody would like a beer is invited!

PD: Sorry for my english, I'm working on it, hehe. Good morning!

*1:
/dev/sda: DELL PERC 6/i 1.22
Caching (SBC) mode page:
IC 0 [cha: n, def: 0, sav: 0]
ABPF 0 [cha: n, def: 0, sav: 0]
CAP 0 [cha: n, def: 0, sav: 0]
DISC 0 [cha: n, def: 0, sav: 0]
SIZE 0 [cha: n, def: 0, sav: 0]
WCE 0 [cha: n, def: 0, sav: 0]
MF 0 [cha: n, def: 0, sav: 0]
RCD 0 [cha: n, def: 0, sav: 0]
DRRP 0 [cha: n, def: 0, sav: 0]
WRP 0 [cha: n, def: 0, sav: 0]
DPTL 0 [cha: n, def: 0, sav: 0]
MIPF 0 [cha: n, def: 0, sav: 0]
MAPF 0 [cha: n, def: 0, sav: 0]
MAPFC 0 [cha: n, def: 0, sav: 0]
FSW 0 [cha: n, def: 0, sav: 0]
LBCSS 1 [cha: n, def: 1, sav: 1]
DRA 0 [cha: n, def: 0, sav: 0]
NV_DIS 0 [cha: n, def: 0, sav: 0]
NCS 0 [cha: n, def: 0, sav: 0]
CSS 128 [cha: n, def:128, sav:128]

*2:
In gentoo:
03:00.0 RAID bus controller [0104]: LSI Logic / Symbios Logic MegaRAID SAS 1078 [1000:0060] (rev 04)
Subsystem: Dell PERC 6/i Integrated RAID Controller [1028:1f0c]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 33
Region 0: Memory at df180000 (64-bit, non-prefetchable) [size=256K]
Region 2: I/O ports at fc00 [size=256]
Region 3: Memory at df1c0000 (64-bit, non-prefetchable) [size=256K]
Expansion ROM at df100000 [disabled] [size=128K]
Capabilities: [b0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 256 bytes, MaxReadReq 2048 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x8, ASPM L0s, Latency L0 <2us, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
Capabilities: [c4] MSI: Enable- Count=1/4 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [d4] MSI-X: Enable- Count=4 Masked-
Vector table: BAR=0 offset=0003e000
PBA: BAR=0 offset=0003f000
Capabilities: [e0] Power Management version 2
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [ec] Vital Product Data
pcilib: sysfs_read_vpd: read failed: Connection timed out
Not readable
Capabilities: [100] Power Budgeting <?>
Kernel driver in use: megaraid_sas

And in Ubuntu:

03:00.0 RAID bus controller [0104]: LSI Logic / Symbios Logic MegaRAID SAS 1078 [1000:0060] (rev 04)
Subsystem: Dell Device [1028:1f0c]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 33
Region 0: Memory at df180000 (64-bit, non-prefetchable) [size=256K]
Region 2: I/O ports at fc00 [size=256]
Region 3: Memory at df1c0000 (64-bit, non-prefetchable) [size=256K]
Expansion ROM at df100000 [disabled] [size=32K]
Capabilities: [b0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 256 bytes, MaxReadReq 2048 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x8, ASPM L0s, Latency L0 <2us, L1 unlimited
ClockPM- Suprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
Capabilities: [c4] Message Signalled Interrupts: Mask- 64bit+ Queue=0/2 Enable-
Address: 0000000000000000 Data: 0000
Capabilities: [d4] MSI-X: Enable- Mask- TabSize=4
Vector table: BAR=0 offset=0003e000
PBA: BAR=0 offset=00fff000
Capabilities: [e0] Power Management version 2
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [ec] Vital Product Data <?>
Capabilities: [100] Power Budgeting <?>
Kernel driver in use: megaraid_sas
Kernel modules: megaraid_sas

eccerr0r · Posted: Fri May 14, 2010 9:49 pm Post subject:

what versions of kernel and hdparm (hdparm -V) are you using on each test?
Using the same hdparm binary?

Latency != bandwidth, hdparm measures bandwidth, not sure how milliseconds fits into this as that's a measure of latency...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?