Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
multithread downloader with http, ftp, sftp support [SOLVED]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
sobers_2002
Veteran
Veteran


Joined: 16 Mar 2004
Posts: 1128

PostPosted: Fri Feb 02, 2007 9:40 am    Post subject: multithread downloader with http, ftp, sftp support [SOLVED] Reply with quote

Hi,

Is there any client which can do all three ?? MultiGet does http and ftp but not sftp.
Kindly pour in suggestions.

thanks
Saurabh
PS: - It's cool even if the package is currently not in portage ;)
_________________
Pdict - dockable dictionary client for linux
FREE97WIN: Use this code on Dreamhost and you get $97 off !!


Last edited by sobers_2002 on Sat Feb 03, 2007 9:19 pm; edited 1 time in total
Back to top
View user's profile Send private message
derverstand
Guru
Guru


Joined: 15 Dec 2005
Posts: 511
Location: /dev/null

PostPosted: Fri Feb 02, 2007 6:48 pm    Post subject: Reply with quote

hi.
i do this with small bash scripts...
/br
Back to top
View user's profile Send private message
Dralnu
Veteran
Veteran


Joined: 24 May 2006
Posts: 1919

PostPosted: Fri Feb 02, 2007 8:29 pm    Post subject: Reply with quote

I know you can do it with Ruby. Bash could probably do it, or maybe, hey, use wget :?:
_________________
The day Microsoft makes a product that doesn't suck, is the day they make a vacuum cleaner.
Back to top
View user's profile Send private message
think4urs11
Bodhisattva
Bodhisattva


Joined: 25 Jun 2003
Posts: 6659
Location: above the cloud

PostPosted: Fri Feb 02, 2007 9:12 pm    Post subject: Reply with quote

cURL / libcurl (libcurl supports downloading multiple URLs simultaneously, curl doesn't)
_________________
Nothing is secure / Security is always a trade-off with usability / Do not assume anything / Trust no-one, nothing / Paranoia is your friend / Think for yourself
Back to top
View user's profile Send private message
derverstand
Guru
Guru


Joined: 15 Dec 2005
Posts: 511
Location: /dev/null

PostPosted: Sat Feb 03, 2007 12:16 am    Post subject: Reply with quote

More concrete. My hack scripts look somehow like this:
Code:

for file in abc def ...; do
  wget -p --convert-links $file &
  sleep 1 # if its goin too fast ;)
done

/BR
Back to top
View user's profile Send private message
Dralnu
Veteran
Veteran


Joined: 24 May 2006
Posts: 1919

PostPosted: Sat Feb 03, 2007 6:49 am    Post subject: Reply with quote

Think4UrS11 wrote:
cURL / libcurl (libcurl supports downloading multiple URLs simultaneously, curl doesn't)
OT: Who is the chick in the av?
_________________
The day Microsoft makes a product that doesn't suck, is the day they make a vacuum cleaner.
Back to top
View user's profile Send private message
sobers_2002
Veteran
Veteran


Joined: 16 Mar 2004
Posts: 1128

PostPosted: Sat Feb 03, 2007 7:00 am    Post subject: Reply with quote

thanks a lot for the options people, but I needed something that could do sftp also.

I don't know about curl, but wget can't do sftp.

thanks!
Saurabh
_________________
Pdict - dockable dictionary client for linux
FREE97WIN: Use this code on Dreamhost and you get $97 off !!
Back to top
View user's profile Send private message
think4urs11
Bodhisattva
Bodhisattva


Joined: 25 Jun 2003
Posts: 6659
Location: above the cloud

PostPosted: Sat Feb 03, 2007 9:12 am    Post subject: Reply with quote

http://curl.haxx.se/ wrote:
cURL supports FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, FILE and LDAP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling

_________________
Nothing is secure / Security is always a trade-off with usability / Do not assume anything / Trust no-one, nothing / Paranoia is your friend / Think for yourself
Back to top
View user's profile Send private message
sobers_2002
Veteran
Veteran


Joined: 16 Mar 2004
Posts: 1128

PostPosted: Sat Feb 03, 2007 9:17 pm    Post subject: Reply with quote

cool, thanks a lot !!
_________________
Pdict - dockable dictionary client for linux
FREE97WIN: Use this code on Dreamhost and you get $97 off !!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum