| View previous topic :: View next topic |
| Author |
Message |
spuniun
n00b
Joined: 14 Jan 2005
Posts: 72
|
 Posted: Sun Sep 23, 2007 3:07 am Post subject: after kernel upgrade WAN speeds slow to a crawl [SOLVED] |
 |
|
Due to some USB conflicts with nForce USB controller and my KVM provider, was urged to upgrade my kernel to test. Turned out the newer kernel resolved my USB problems but it created a new, more severe problem.
While running 2.6.16-gentoo-r13 all was well with my network. System is a desktop only using iptables for some internal port redirects so that users can run some services on reserved ports. WAN connection is asynchronous 30mbit/5mbit and speeds are perfect; gateway is a Cisco 871. Other linux, Windows & Mac workstations on the network have no network problems either.
Upgraded to 2.6.22-gentoo-r6 after rebuilding the config carefully; because so much has changed from the prior kernel version used, I really had no choice but to rebuild from scratch. I only made moderate alterations to fine tune the kernel and remove things that were obviously unneeded (like SMP support since this is a single-core CPU). After booting up to the new kernel, everything worked without issue or error, but WAN speeds dropped to a slow drool. LAN speeds remain consistent at gigabit rates. I disabled iptables completely while troubleshooting but it made no difference. While I'm not a stranger to TCP window tuning, this system has never benefited from any tweaks so I have never bothered altering the sysctl args and confirmed that all are the saem with both kernel versions. I'm at a loss as to what might have changed in the kernel that would cause this issue. Below are the pertinent pieces of data...
emerge --info | Code: | Portage 2.1.3.9 (default-linux/amd64/2007.0/desktop, gcc-4.2.0, glibc-2.6.1-r0, 2.6.16-gentoo-r13 x86_64)
=================================================================
System uname: 2.6.16-gentoo-r13 x86_64 AMD Athlon(tm) 64 Processor 3400+
Timestamp of tree: Thu, 20 Sep 2007 06:50:01 +0000
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
app-shells/bash: 3.2_p17-r1
dev-java/java-config: 1.3.7, 2.0.33-r1
dev-lang/python: 2.4.4-r4, 2.5.1-r2
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.10-r4
sys-apps/sandbox: 1.2.18.1
sys-devel/autoconf: 2.13, 2.61-r1
sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils: 2.18
sys-devel/gcc-config: 1.4.0-r1
sys-devel/libtool: 1.5.24
virtual/os-headers: 2.6.22-r2
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -fweb -frename-registers -ftracer -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/revdep-rebuild /etc/splash /etc/terminfo"
CXXFLAGS="-march=athlon64 -O2 -fweb -frename-registers -ftracer -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache distlocks metadata-transfer sandbox sfperms strict unmerge-orphans"
GENTOO_MIRRORS="http://mirrors.acm.cs.rpi.edu/gentoo/ http://gentoo.osuosl.org/ http://gentoo.mirrors.tds.net/gentoo http://mirror.mcs.anl.gov/pub/gentoo/ http://mirror.datapipe.net/gentoo"
LANG="en_US"
LC_ALL="en_US"
LINGUAS="en"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://totalmonkey.spuncookie.shacknet.nu/gentoo-portage"
USE="X a52 aac acl acpi aim alsa amd64 aotuv audiofile automount bash-completion bdf bitmap-fonts bittorrent bonobo browserplugin buttons bzip2 cairo cdda cddb cdparanoia cdr cdrom chroot clamav cli cpudetection cracklib crypt css cups curl curlwrappers dba dbus deprecated discouraged dri dts dv dvd dvdr dvdread ecc edl eds emboss emul-linux-x86 encode escreen etwin evo fam fame fbcon ffmpeg firefox flash font-server fontconfig fortran fpx ftp gaim gcj gd gdbm gif gimp glitz gmail gmailtimestamps gpm gstreamer gtalk gtk gtkhtml guile hbci iconv icq id3 idea idn ieee1394 imagemagick imap imlib insecure-savers irc ithreads jabber java java-external javascript jce jit joystick jpeg jpeg2k kerberos krb4 lame lcd lcms ldap libclamav live lm_sensors logitech-mouse logrotate lzo mad matroska matrox md5sum midi mikmod mjpeg mmx mng moznocompose moznoirc moznomail mozsvg mozxmlterm mp3 mp4 mpeg mplayer msn mudflap musepack ncurses network networking nls nolvm1 nomac nptl nptlonly nsplugin nvidia oav objc offensive ofx ogg oggvorbis on-the-fly-crypt openexr opengl openmp pccts pcre pda pdf perl png portaudio pppd python qt3 qt3support qt4 quicktime quotes readline reflection regex rtc samba sasl sdl sensord server session sftplogging shorten skey skins sndfile speex spell spl sse sse2 ssl svg svgz symlink syslog tcl tcltk tcpd tga theora threads tiff tk toolbar truetype truetype-fonts type1 type1-fonts userlocales v4l v4l2 vcd vim vim-with-x vnc vorbis wavpack winbind wma wmf wmp wxwindows x264 xanim xinerama xinetd xml xml2 xmlrpc xorg xpm xprint xscreensaver xv xvid yahoo yv12 zlib" ALSA_CARDS="intel8x0" ALSA_PCM_PLUGINS="plug ioplug dmix" ELIBC="glibc" INPUT_DEVICES="evdev joystick keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" USERLAND="GNU" VIDEO_CARDS="nv nvidia vesa"
Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LDFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS |
lspci | Code: | 00:00.0 Host bridge: nVidia Corporation nForce3 250Gb Host Bridge (rev a1)
00:01.0 ISA bridge: nVidia Corporation nForce3 250Gb LPC Bridge (rev a2)
00:01.1 SMBus: nVidia Corporation nForce 250Gb PCI System Management (rev a1)
00:02.0 USB Controller: nVidia Corporation CK8S USB Controller (rev a1)
00:02.1 USB Controller: nVidia Corporation CK8S USB Controller (rev a1)
00:02.2 USB Controller: nVidia Corporation nForce3 EHCI USB 2.0 Controller (rev a2)
00:05.0 Bridge: nVidia Corporation CK8S Ethernet Controller (rev a2)
00:06.0 Multimedia audio controller: nVidia Corporation nForce3 250Gb AC'97 Audio Controller (rev a1)
00:08.0 IDE interface: nVidia Corporation CK8S Parallel ATA Controller (v2.5) (rev a2)
00:0a.0 IDE interface: nVidia Corporation CK8S Serial ATA Controller (v2.5) (rev a2)
00:0b.0 PCI bridge: nVidia Corporation nForce3 250Gb AGP Host to PCI Bridge (rev a2)
00:0e.0 PCI bridge: nVidia Corporation nForce3 250Gb PCI-to-PCI Bridge (rev a2)
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
01:00.0 VGA compatible controller: nVidia Corporation NV35 [GeForce FX 5900XT] (rev a1)
02:07.0 USB Controller: OPTi Inc. 82C861 (rev 10)
02:09.0 Mass storage controller: Silicon Image, Inc. SiI 3512 [SATALink/SATARaid] Serial ATA Controller (rev 01) |
lsmod | Code: | Module Size Used by
cisco_ipsec 548744 0
w83627hf 25488 0
hwmon_vid 2752 1 w83627hf
i2c_isa 5504 1 w83627hf
snd_pcm_oss 46048 0
snd_mixer_oss 14976 1 snd_pcm_oss
snd_seq_oss 29504 0
snd_seq_midi_event 6912 1 snd_seq_oss
snd_seq 48736 4 snd_seq_oss,snd_seq_midi_event
snd_seq_device 7632 2 snd_seq_oss,snd_seq
adi 8768 0
gameport 12368 1 adi
joydev 9408 0
nvidia 8109944 24
snd_intel8x0 30632 1
snd_ac97_codec 97352 1 snd_intel8x0
snd_ac97_bus 2752 1 snd_ac97_codec
snd_pcm 78664 3 snd_pcm_oss,snd_intel8x0,snd_ac97_codec
snd_timer 20744 2 snd_seq,snd_pcm
snd 49144 11 snd_pcm_oss,snd_mixer_oss,snd_seq_oss,snd_seq,snd_seq_device,snd_intel8x0,snd_ac97_codec,snd_pcm,snd_timer
snd_page_alloc 8912 2 snd_intel8x0,snd_pcm
forcedeth 21380 0
dm_mod 45888 2
i2c_nforce2 6912 0 |
Typical WAN download on 2.6.16-gentoo-r13 | Code: | wget http://gentoo.osuosl.org/snapshots/portage-20070922.tar.bz2
--22:38:04-- http://gentoo.osuosl.org/snapshots/portage-20070922.tar.bz2
=> `portage-20070922.tar.bz2'
Resolving gentoo.osuosl.org... 64.50.236.52, 64.50.238.52
Connecting to gentoo.osuosl.org|64.50.236.52|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 38,745,995 (37M) [application/x-tar]
100%[============================================================================================================>] 38,745,995 1.69M/s ETA 00:00 |
Same download on same system with 2.6.22-gentoo-r6 | Code: | wget http://gentoo.osuosl.org/snapshots/portage-20070922.tar.bz2
--22:37:53-- http://gentoo.osuosl.org/snapshots/portage-20070922.tar.bz2
=> `portage-20070922.tar.bz2'
Resolving gentoo.osuosl.org... 64.50.238.52, 64.50.236.52
Connecting to gentoo.osuosl.org|64.50.238.52|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 38,745,995 (37M) [application/x-tar]
1% [=> ] 724,030 17.47K/s ETA 39:08 |
Kernel config for 2.6.16-gentoo-r13 can be seen at http://rafb.net/p/XNo6ny52.html
Kernel config for 2.6.22-gentoo-r6 can be seen at http://rafb.net/p/PxMc8n72.html
At this point I am stuck with 2.6.16-gentoo-r13 and no KVM so any assistance would be appreciated. I've been searching the forums for 2 days now and have yet to find any threads detailing my situation specifically.
Last edited by spuniun on Mon Sep 24, 2007 12:46 am; edited 1 time in total |
|
| Back to top |
|
 |
yuwy
n00b
Joined: 22 May 2006
Posts: 38
|
| Posted: Sun Sep 23, 2007 3:27 am Post subject: |
|
|
| if u can show us the 'iwconfig' command when your connected to your wireless hub |
|
| Back to top |
|
|
spuniun
n00b
Joined: 14 Jan 2005
Posts: 72
|
| Posted: Sun Sep 23, 2007 4:04 am Post subject: |
|
|
sorry, I meant to provide ifconfig | Code: | eth0 Link encap:Ethernet HWaddr 00:04:61:6B:57:D6
inet addr:223.192.168.5 Bcast:223.192.168.15 Mask:255.255.255.240
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:310460 errors:0 dropped:0 overruns:0 frame:0
TX packets:156233 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:431061510 (411.0 Mb) TX bytes:11352516 (10.8 Mb)
Interrupt:17 Base address:0x4000 |
ethtool | Code: | Settings for eth0:
Supported ports: [ MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 1
Transceiver: external
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Link detected: yes |
This is a completely hardwired network, no wireless involved. |
|
| Back to top |
|
|
spuniun
n00b
Joined: 14 Jan 2005
Posts: 72
|
| Posted: Sun Sep 23, 2007 11:44 pm Post subject: |
|
|
After doing a little more troubleshooting, I have ruled out a few things. I upgraded my local fileserver to a more recent kernel version and encountered the same loss of speed symptom. This rules out the issue being related to the forcedeth driver or x86_64. Below are the details for this fileserver.
emerge --info | Code: | Portage 2.1.3.4 (hardened/x86/2.6, gcc-4.1.1, glibc-2.6-r0, 2.6.21-hardened-r3 i686)
=================================================================
System uname: 2.6.21-hardened-r3 i686 Intel(R) Pentium(R) 4 CPU 3.00GHz
Gentoo Base System release 1.12.10
Timestamp of tree: Sun, 23 Sep 2007 06:50:01 +0000
distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.4 [enabled]
app-shells/bash: 3.2_p17-r1
dev-java/java-config: 1.3.7, 2.0.33-r1
dev-lang/python: 2.4.4-r4
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache: 2.4-r7
sys-apps/baselayout: 1.12.10-r4
sys-apps/sandbox: 1.2.18.1
sys-devel/autoconf: 2.13, 2.61-r1
sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils: 2.17-r1
sys-devel/gcc-config: 1.3.16
sys-devel/libtool: 1.5.24
virtual/os-headers: 2.6.22-r2
ACCEPT_KEYWORDS="x86 ~x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=prescott -O2 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/revdep-rebuild /etc/splash /etc/terminfo"
CXXFLAGS="-march=prescott -O2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache distlocks metadata-transfer sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LINGUAS="en en_US"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclu
de=/local --exclude=/packages --filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
USE="acl async authdaemond automount bash-completion big-tables bzip2 chroot cracklib crypt cscope fam gcj gdbm gmp gpm hardened ithreads java javascript jce jpeg jpeg2k jpgra
ph kerberos krb4 ldap logrotate md5sum midi mmx mng mp4 mp4live mpeg2 mysql ncurses nls noaudio nolvm1 nptl nptlonly oav odbc offensive pcre pdf perl pic png ppds python quota
s rar readline samba sasl skey sse sse2 ssl swat tcl tcltk tcpd threads tiff truetype unzip urandom userlocales vim-syntax vorbis win32codecs winbind wma x86 xinetd xvid zip z
lib" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm sof
tvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en en_US" USERLA
ND="GNU" VIDEO_CARDS="ati i810"
Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS |
lspci | Code: | 00:00.0 Host bridge: Intel Corporation 82865G/PE/P DRAM Controller/Host-Hub Interface (rev 02)
00:02.0 Display controller: Intel Corporation 82865G Integrated Graphics Controller (rev 02)
00:1d.0 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #1 (rev 02)
00:1d.1 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #2 (rev 02)
00:1d.3 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #4 (rev 02)
00:1d.7 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB2 EHCI Controller (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev c2)
00:1f.0 ISA bridge: Intel Corporation 82801EB/ER (ICH5/ICH5R) LPC Interface Bridge (rev 02)
00:1f.1 IDE interface: Intel Corporation 82801EB/ER (ICH5/ICH5R) IDE Controller (rev 02)
00:1f.3 SMBus: Intel Corporation 82801EB/ER (ICH5/ICH5R) SMBus Controller (rev 02)
00:1f.5 Multimedia audio controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) AC'97 Audio Controller (rev 02)
01:00.0 VGA compatible controller: ATI Technologies Inc RV280 [Radeon 9200 SE] (rev 01)
01:00.1 Display controller: ATI Technologies Inc RV280 [Radeon 9200 SE] (Secondary) (rev 01)
01:02.0 Mass storage controller: Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller (rev 02)
01:08.0 Ethernet controller: Intel Corporation 82562EZ 10/100 Ethernet Controller (rev 02) |
lsmod | Code: | Module Size Used by
uhci_hcd 20509 -
ehci_hcd 27002 -
snd_intel8x0 27576 -
snd_ac97_codec 89570 -
ac97_bus 1779 -
dm_mod 45523 -
e100 29556 -
i2c_i801 7287 -
rng_core 3820 -
intel_agp 20274 -
evdev 7455 - |
ifconfig | Code: | eth0 Link encap:Ethernet HWaddr 00:11:11:A1:A5:81
inet addr:223.192.168.4 Bcast:223.192.168.15 Mask:255.255.255.240
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:36050881 errors:0 dropped:0 overruns:0 frame:0
TX packets:115258388 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1500867184 (1431.3 Mb) TX bytes:250419645 (238.8 Mb) |
ethtool | Code: | Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: MII
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: g
Current message level: 0x00000007 (7)
Link detected: yes |
WAN download with 2.6.21-hardened-r3 | Code: | wget http://gentoo.osuosl.org/snapshots/portage-20070922.tar.bz2
--19:27:47-- http://gentoo.osuosl.org/snapshots/portage-20070922.tar.bz2
=> `portage-20070922.tar.bz2'
Resolving gentoo.osuosl.org... 64.50.238.52, 64.50.236.52
Connecting to gentoo.osuosl.org|64.50.238.52|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 38,745,995 (37M) [application/x-tar]
1% [> ] 414,430 14.57K/s ETA 44:02 |
Using older kernel versions like 2.6.16 provide expected speeds between 10mbit and 30mbit depending on source. I am completely stumped as to what might have changed that would cause such a major change to TCP window scaling. Again, these tests done with iptables disabled
Kernel config for 2.6.21-hardened-r3 can be seen at http://rafb.net/p/iDUQnU63.html |
|
| Back to top |
|
|
hanj
Veteran
Joined: 19 Aug 2003
Posts: 1500
|
|
| Back to top |
|
|
spuniun
n00b
Joined: 14 Jan 2005
Posts: 72
|
| Posted: Mon Sep 24, 2007 12:46 am Post subject: |
|
|
You get a Gold Star!
Thanks much hanj, that was the issue.
I guess the default was changed in >2.6.16 |
|
| Back to top |
|
|
hanj
Veteran
Joined: 19 Aug 2003
Posts: 1500
|
| Posted: Mon Sep 24, 2007 12:49 am Post subject: |
|
|
| spuniun wrote: | You get a Gold Star!
Thanks much hanj, that was the issue.
I guess the default was changed in >2.6.16 |
Sweet.. glad that worked for you.. that was a major major pain for me. I agree.. after 2.6.16.. also, it only applies to certain upstream routers, so it could be your initial modem, or router in the network upstream somewhere. I'm the only one personally that was affected by this when switching to 2.6.16.
hanji
_________________
Server Admin Blog - Uno-Code.com |
|
| Back to top |
|
|
spuniun
n00b
Joined: 14 Jan 2005
Posts: 72
|
| Posted: Mon Sep 24, 2007 1:52 am Post subject: |
|
|
| I never thought to look up the issue in Cisco TAC, but sure enough they have the symptom documented. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
