Strange transfer speed problem

Griffon26 · Posted: Mon Dec 15, 2003 1:30 pm Post subject: Strange transfer speed problem

I have two computers on my LAN, both running gentoo with 2.6.0-test11.
One of them (let's call it the server and the other one the client) is running vsftpd.

The problem is that when I download something from the server to the client with ncftp/ncftpget/wget, the transfer speed is about 50K/s. If I download the same file using Mozilla Firebird I get 18MB/s!! The server also is a NAT box and if I download something from the internet to the client with ncftp or wget I get 200K/s (which is the maximum my inet connection can handle).

I'm really confused what could have caused this. I just reinstalled the server and it has been like this ever since. Nothing changed in the hardware setup except that it now has another hard drive.

I'll list some possible causes for transfer speed problems and why they cannot be causing my problem.

Q) bad cable?
A) not possible; mozilla transfers at 18MB/s and wget from the internet at 200K/s

Q) configured ftp server limit or other traffic shaping?
A) not possible; mozilla transfers at 18MB/s

Q) difference between passive and active ftp?
A) not possible; I tried passive and active with ncftp, both get only 50K/s

Q) difference in user name/password?
A) nope, I logged in with ncftp as anonymous with password mozilla@example.com just like mozilla does

There is one thing I just noticed: I started a transfer with ncftp (50K/s) and while it was transferring I started a transfer with Mozilla (18MB/s) and it gave an error after a few seconds (cannot write to network stream or smth). I restarted the Mozilla transfer and that one did complete successfully. I don't know if this is relevant.

Any ideas for tracking down this problem will be highly appreciated.

Thanks,
Griffon26

neuron · Advocate Joined: 28 May 2002 Posts: 2371

is your mozilla set up to use a proxy? perhaps its following a diffrent route than your wget/ftp clients are?

Griffon26 · Posted: Mon Dec 15, 2003 1:41 pm Post subject:

No, it's not using a proxy.

If it were, it would be making a connection through my server (running the ftp) on to the internet to the proxy and back to my server, because I'm not running any proxies =]

Keep in mind that the transfer problem is seen in LAN-only transfer speeds.

neuron · Advocate Joined: 28 May 2002 Posts: 2371

yeah, I was thinking caching.

and.... if this is lan and not 1gb, then it's in mozilla cache. And not actually transfering it. 18mb/sec is over 100mbit speeds (100mbit max is 12.5mb/sec)

Griffon26 · Posted: Mon Dec 15, 2003 2:06 pm Post subject:

I have a Gbit LAN and it has always been 18MB/s, also the first transfer, so cache is not the problem. Btw, 50K/s is pretty lousy on any LAN =]

neuron · Advocate Joined: 28 May 2002 Posts: 2371

yeah, still, make sure mozilla isn't using it's cache when downloading your test file.

if everything goes at 50kb/sec it may be easier to debug the problem.
(network card broke and mozilla using it's cache or something perhaps?)

Griffon26 · Posted: Mon Dec 15, 2003 2:14 pm Post subject:

I downloaded another file and it also transferred at 18MB/s. Meanwhile I looked at disk activity on the server and it was reading at about 20MB/s, so it really was transferring at said speed.

neuron · Advocate Joined: 28 May 2002 Posts: 2371

you could try other browsers.. like opera for example.. worth a try.

Griffon26 · Posted: Mon Dec 15, 2003 2:41 pm Post subject:

Good idea.

An overview:

Mozilla (passive) -> fast
ncftp (passive) -> slow
ncftp (active) -> slow
ncftpget (passive) -> slow
wget (passive) -> slow
wget (active) -> slow
links (active) -> slow
links2 (active) -> slow
lynx (active) -> fast

neuron · Advocate Joined: 28 May 2002 Posts: 2371

hmm, sorry man, I have no idea :/

you could try a httpd or another ftpd to see if that'll change anything.

aridhol · Posted: Mon Dec 15, 2003 3:08 pm Post subject:

I have the same problem, although different speeds and not on a lan.

I have my computer directly connected to the Internet. A friend in Holland downloads something from my ftp (proftpd) and it goes at 4KBps. Upload (From her) is 16KBps. When she downloads via http she download at 80KBps.

I have not set up any speed-limits in my conf, and I can load fast ftp localhost....

My connection is 8/1mbps. Hers is 864/160kbps
_________________
72 of Pitcairn Islands 49 inhabitants use Seti@Home
"If you buy a DVD you have a copy. If you want a backup copy you buy another one."
"Anyone who is capable of getting themselves made President should on no account be allowed to do the job."

Griffon26 · Posted: Mon Dec 15, 2003 3:44 pm Post subject:

Are you also running 2.6.0 and using ip_conntrack?

It seems like we're not the only ones.
http://lists.netfilter.org/pipermail/netfilter/2003-September/047031.html
http://lists.netfilter.org/pipermail/netfilter-devel/2003-November/013259.html
http://lkml.org/lkml/2003/12/4/17

It's a kernel bug =/

aridhol · Posted: Mon Dec 15, 2003 5:07 pm Post subject:

oh crankey

Oh well... we'll just have to wait a bit then

_________________
72 of Pitcairn Islands 49 inhabitants use Seti@Home
"If you buy a DVD you have a copy. If you want a backup copy you buy another one."
"Anyone who is capable of getting themselves made President should on no account be allowed to do the job."