| View previous topic :: View next topic |
| Author |
Message |
MrUlterior
Guru
Joined: 22 Mar 2005
Posts: 511
Location: Switzerland
|
 Posted: Sat Dec 10, 2005 10:07 pm Post subject: Getting the best performance from your gigabit LAN |
 |
|
Okay, so I wanted to achieve the maximum performance from my gigabit LAN & I came across this site:
http://dsd.lbl.gov/TCP-tuning/linux.html
Following the instructions, on two machines I emerged iperf & netperf and started them on the one box, then I tested the performance:
| Code: | # netperf -f m -H xenophobe
TCP STREAM TEST to xenophobe
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
87380 16384 16384 10.01 303.45 |
Then I executed the following on both boxes:
| Code: |
echo 2500 > /proc/sys/net/core/netdev_max_backlog
echo "highspeed" > /proc/sys/net/ipv4/tcp_congestion_control
echo "16777216" > /proc/sys/net/core/rmem_max
echo "16777216" > /proc/sys/net/core/wmem_max
echo "4096 65536 16777216" > /proc/sys/net/ipv4/tcp_rmem
echo "4096 65536 16777216" > /proc/sys/net/ipv4/tcp_wmem
ifconfig eth0 txqueuelen 1000
|
And re-ran the test:
| Code: | # clear && netperf -f m -H xenophobe
TCP STREAM TEST to xenophobe
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
65536 65536 65536 10.06 395.82
# iperf -f M -c xenophobe
------------------------------------------------------------
Client connecting to xenophobe, TCP port 5001
TCP window size: 0.06 MByte (default)
------------------------------------------------------------
[ 3] local 10.1.0.11 port 39897 connected with 10.1.0.2 port 5001
[ 3] 0.0-10.0 sec 471 MBytes 47.0 MBytes/sec
|
As you can see, there seems to be quite a significant performance boost. Is there anyone else that'd care to do the same & post their results?
FYI
- the NICs in both PC's are set to full duplex
- there's no other traffic on the LAN
- both NICS run the skge driver & are identical:
# lspci | egrep -i ethernet
0000:00:0a.0 Ethernet controller: Marvell Technology Group Ltd. 88E8001 Gigabit Ethernet Controller (rev 13)
_________________
Misanthropy 2.0 - enough hate to go around
|
|
| Back to top |
|
 |
MrUlterior
Guru
Joined: 22 Mar 2005
Posts: 511
Location: Switzerland
|
| Posted: Mon Dec 12, 2005 10:18 am Post subject: |
|
|
Another question:
Is there any specific reason that Gentoo doesn't alter the default value of
/proc/sys/net/ipv4/tcp_congestion_control
?
If there's someone out there who'd care to benchmark their results on 2 x 100mbit NICs or 2 x 10mbit NICs with the full range of values, I'd be extremely interested in seeing the results.
_________________
Misanthropy 2.0 - enough hate to go around
|
|
| Back to top |
|
|
F.Ultra
Apprentice
Joined: 17 Mar 2004
Posts: 169
Location: Sweden
|
| Posted: Mon Dec 12, 2005 3:25 pm Post subject: |
|
|
| I am note sure what tcp_congestion_control does, but a word of advice is to not tune your system for situations that you will never ever see when in production. If we say that you do this tuning in order to maximise your performance over the Internet, then performing the tuning over a direct connection between two hosts is almost completely useless since there is a lot of things going on under the hood when sending/receiving data over a WAN instead of a LAN. Also tuning on an empty LAN could give worse performance when applied to a busy LAN and so on. |
|
| Back to top |
|
|
MrUlterior
Guru
Joined: 22 Mar 2005
Posts: 511
Location: Switzerland
|
| Posted: Mon Dec 12, 2005 3:34 pm Post subject: |
|
|
| F.Ultra wrote: | | I am note sure what tcp_congestion_control does, but a word of advice is to not tune your system for situations that you will never ever see when in production. If we say that you do this tuning in order to maximise your performance over the Internet, then performing the tuning over a direct connection between two hosts is almost completely useless since there is a lot of things going on under the hood when sending/receiving data over a WAN instead of a LAN. Also tuning on an empty LAN could give worse performance when applied to a busy LAN and so on. |
That's a fair point though I'm performance tuning the LAN only. I don't intend to alter MTUs so I don't see how this could impact my default route out to the WAN. However, your point is very valid about the difference between an idle LAN & busy LAN. I shall re-do my tests at a later stage with other hosts simulating a high load on the LAN and see what that does compared to the default settings.
So far though, things are looking good -- I'm getting much better throughput for backups.
_________________
Misanthropy 2.0 - enough hate to go around
|
|
| Back to top |
|
|
F.Ultra
Apprentice
Joined: 17 Mar 2004
Posts: 169
Location: Sweden
|
| Posted: Tue Dec 13, 2005 11:03 am Post subject: |
|
|
I found an interesting paper on TCP Congestion Control in Linux if any one would be interested  http://www.cs.helsinki.fi/research/iwtcp/papers/linuxtcp.pdf Something tells me that when you enter "highspeed" into the congestion control, then the tcp stack will be optimised for large tcp windows, something that would probably give better performance on a LAN, unsure about a WAN since the situation is far more complex there. |
|
| Back to top |
|
|
anidabi
Tux's lil' helper
Joined: 05 Feb 2005
Posts: 102
Location: Finland
|
| Posted: Fri Sep 22, 2006 7:08 pm Post subject: Re: Getting the best performance from your gigabit LAN |
|
|
Are those buffer sizes for 1GB only or do they apply also for 100Mbit network?
_________________
How do you do the things you do... |
|
| Back to top |
|
|
|
Networking & Security
