View previous topic :: View next topic |
Author |
Message |
tricolore n00b
Joined: 02 Apr 2009 Posts: 3
|
Posted: Thu Apr 02, 2009 9:32 am Post subject: [SOLVED] Large SFTP transfer - filesize stops increasing |
|
|
Hi!
I'm stuck with a weird problem:
SFTP PUT of a 50GB tarfile from Linux client A to Linux server B.
Everything starts fine, but after some hours (10MBIT Ethernet) the filesize on serverside stops increasing, while the FTP session appears to still transmit at full speed.
This happened once after transmitting 10GB, once after 22GB, no signs in syslog, the client reports 100% successful transmission afterwords.
Server has 1.8TB free disk space on a ReiserFS partition and a RT 8111C GBIT NIC
Could this be
- HDD going to sleep?
- Bug in the NIC driver?
- Bug in SFTP server?
- Bug in Linux SW RAID?
Any hint is appreciated.
Thanks in advance!
Last edited by tricolore on Mon Apr 06, 2009 9:09 am; edited 1 time in total |
|
Back to top |
|
|
Letharion Veteran
Joined: 13 Jun 2005 Posts: 1344 Location: Sweden
|
Posted: Thu Apr 02, 2009 10:19 am Post subject: |
|
|
I don't know really what happens, but, in order to "just fix it", I'd like to suggest using split and cat to transfer smaller files.
If it's a potential SFTP bug, maybe try scp instead?
File splitting/cating:
Code: | tar -cf /foo.bu.tar /fooSource &
split --line-bytes=500m foo.tar.gz foo_
cat foo_a* > foo_FULL.tar.gz |
Will take forever ofc
Last edited by Letharion on Thu Apr 02, 2009 11:53 am; edited 1 time in total |
|
Back to top |
|
|
tricolore n00b
Joined: 02 Apr 2009 Posts: 3
|
Posted: Thu Apr 02, 2009 10:36 am Post subject: |
|
|
Thank you for the workaround, learned a new tar trick!
However, this server shall become my backup machine and therefore should be able to handle such a trivial task.
I tried smb mount as an alternative, but this breaks with "file too large" (which is a smb bug as far as I found out)
I will
1) check out scp to rule out sftp
2) compile realteks driver and replace stock kernel driver
3) disable HDD spindown
Thinking about it, this must be either an application layer bug, or interface between network stack and application.
Everything up to TCP works obviously, so either SFTP becomes detached from the network stack and doesnt receive any data (while TCP happily maintains the connection), or just stops writing to disk. |
|
Back to top |
|
|
Letharion Veteran
Joined: 13 Jun 2005 Posts: 1344 Location: Sweden
|
Posted: Thu Apr 02, 2009 11:57 am Post subject: |
|
|
tricolore wrote: | Thank you for the workaround, learned a new tar trick!
However, this server shall become my backup machine and therefore should be able to handle such a trivial task.
I tried smb mount as an alternative, but this breaks with "file too large" (which is a smb bug as far as I found out)
I will
1) check out scp to rule out sftp
2) compile realteks driver and replace stock kernel driver
3) disable HDD spindown
Thinking about it, this must be either an application layer bug, or interface between network stack and application.
Everything up to TCP works obviously, so either SFTP becomes detached from the network stack and doesnt receive any data (while TCP happily maintains the connection), or just stops writing to disk. |
I hope you work it out
Maybe you'd figure yourself, but I see there's more than one mistake in the filenaming. |
|
Back to top |
|
|
tricolore n00b
Joined: 02 Apr 2009 Posts: 3
|
Posted: Mon Apr 06, 2009 9:08 am Post subject: Works |
|
|
In case anyone has the same issue:
SCP did the trick, worked flawlessly.
Thanks again for the hint. |
|
Back to top |
|
|
Letharion Veteran
Joined: 13 Jun 2005 Posts: 1344 Location: Sweden
|
Posted: Mon Apr 06, 2009 9:15 am Post subject: |
|
|
I absolutely <3 scp in favor of ftp |
|
Back to top |
|
|
|