Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] Large SFTP transfer - filesize stops increasing
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
tricolore
n00b
n00b


Joined: 02 Apr 2009
Posts: 3

PostPosted: Thu Apr 02, 2009 9:32 am    Post subject: [SOLVED] Large SFTP transfer - filesize stops increasing Reply with quote

Hi!

I'm stuck with a weird problem:
SFTP PUT of a 50GB tarfile from Linux client A to Linux server B.
Everything starts fine, but after some hours (10MBIT Ethernet) the filesize on serverside stops increasing, while the FTP session appears to still transmit at full speed.
This happened once after transmitting 10GB, once after 22GB, no signs in syslog, the client reports 100% successful transmission afterwords.
Server has 1.8TB free disk space on a ReiserFS partition and a RT 8111C GBIT NIC

Could this be
- HDD going to sleep?
- Bug in the NIC driver?
- Bug in SFTP server?
- Bug in Linux SW RAID?

Any hint is appreciated.

Thanks in advance!


Last edited by tricolore on Mon Apr 06, 2009 9:09 am; edited 1 time in total
Back to top
View user's profile Send private message
Letharion
Veteran
Veteran


Joined: 13 Jun 2005
Posts: 1344
Location: Sweden

PostPosted: Thu Apr 02, 2009 10:19 am    Post subject: Reply with quote

I don't know really what happens, but, in order to "just fix it", I'd like to suggest using split and cat to transfer smaller files.
If it's a potential SFTP bug, maybe try scp instead?

File splitting/cating:
Code:
tar -cf /foo.bu.tar /fooSource &
split --line-bytes=500m foo.tar.gz foo_
cat foo_a* > foo_FULL.tar.gz

Will take forever ofc


Last edited by Letharion on Thu Apr 02, 2009 11:53 am; edited 1 time in total
Back to top
View user's profile Send private message
tricolore
n00b
n00b


Joined: 02 Apr 2009
Posts: 3

PostPosted: Thu Apr 02, 2009 10:36 am    Post subject: Reply with quote

Thank you for the workaround, learned a new tar trick!
However, this server shall become my backup machine and therefore should be able to handle such a trivial task.

I tried smb mount as an alternative, but this breaks with "file too large" (which is a smb bug as far as I found out)

I will
1) check out scp to rule out sftp
2) compile realteks driver and replace stock kernel driver
3) disable HDD spindown

Thinking about it, this must be either an application layer bug, or interface between network stack and application.
Everything up to TCP works obviously, so either SFTP becomes detached from the network stack and doesnt receive any data (while TCP happily maintains the connection), or just stops writing to disk.
Back to top
View user's profile Send private message
Letharion
Veteran
Veteran


Joined: 13 Jun 2005
Posts: 1344
Location: Sweden

PostPosted: Thu Apr 02, 2009 11:57 am    Post subject: Reply with quote

tricolore wrote:
Thank you for the workaround, learned a new tar trick!
However, this server shall become my backup machine and therefore should be able to handle such a trivial task.

I tried smb mount as an alternative, but this breaks with "file too large" (which is a smb bug as far as I found out)

I will
1) check out scp to rule out sftp
2) compile realteks driver and replace stock kernel driver
3) disable HDD spindown

Thinking about it, this must be either an application layer bug, or interface between network stack and application.
Everything up to TCP works obviously, so either SFTP becomes detached from the network stack and doesnt receive any data (while TCP happily maintains the connection), or just stops writing to disk.


I hope you work it out :)
Maybe you'd figure yourself, but I see there's more than one mistake in the filenaming.
Back to top
View user's profile Send private message
tricolore
n00b
n00b


Joined: 02 Apr 2009
Posts: 3

PostPosted: Mon Apr 06, 2009 9:08 am    Post subject: Works Reply with quote

In case anyone has the same issue:
SCP did the trick, worked flawlessly.

Thanks again for the hint.
Back to top
View user's profile Send private message
Letharion
Veteran
Veteran


Joined: 13 Jun 2005
Posts: 1344
Location: Sweden

PostPosted: Mon Apr 06, 2009 9:15 am    Post subject: Reply with quote

I absolutely <3 scp in favor of ftp :)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum