rsync/ftp mirroring [solved easily with wget]

eXess · Last edited by eXess on Sat Oct 22, 2005 8:03 pm; edited 1 time in total

Hi all,

I would like to setup RSYNC mirroring through a FTP connection. I don't even know if that is possible but I'd like to give it a try. Has anyone got experience with this ? I would really love to keep it to FTP because that's currently working and I'd hate to have to setup something else... The idea is to write a CRON task on the mirroring machine that would connect to the main server and RSYNC a directory (recursively, of course...)

Thanks a lot !
_________________
. . . . . . . .
eXess

kbielefe · n00b Joined: 08 Aug 2004 Posts: 35 Location: Gilbert, AZ

Technically, you could do it with a "wget -r" command, but the small amount of effort needed to set up rsyncd is well worth it. See "4. Setting up your own local rsync mirror" in http://www.gentoo.org/doc/en/rsync.xml. I had mine up and running in under 5 minutes.

eXess · Posted: Sat Oct 22, 2005 6:32 pm Post subject:

Thanks for the link and sorry to be a total n00b on this one but I need to know a little more about this. Do I need to install a special daemon for rsync (something like a rsync "server") on my main machine ? This would mean I'd have to open a port for NAT on my router... Then I guess RSYNC'ing through a FTP connection is not what I'd have to do then

Well, it's ok if rsyncd is the only (or best) way to do it, I'm admin of both machines, but then again... Really no way to do it with FTP? :oops:

_________________
. . . . . . . .
eXess

kbielefe · n00b Joined: 08 Aug 2004 Posts: 35 Location: Gilbert, AZ

As far as I know, there is no way to tunnel rsync over ftp. You could probably change the port to use the same port, but that would conflict with your ftp server. "wget -r ftp://my.server.com/directory" will recursively fetch everything under that directory into your current working directory, but you lose the bandwidth-saving benefits of rsync. Basically, it downloads the entire tree every time, where rsync will download only the differences.

You already have rsync installed in order to do "emerge sync". The same package provides the client and the server. You just need to open port 873 on your server's firewall, and set the server to start by default by running "rc-update add rsyncd default" as root. Edit your /etc/rsyncd.conf file to show something like the following:

eXess · Posted: Sat Oct 22, 2005 7:06 pm Post subject:

ok, I guess I'll have to do it that way then...

Nevertheless, applications like Dreamweaver and FTP clients (like... WS-FTP? FileZilla? can't remember) provide a facility to "synchronize" sites, by checking modification dates of the files and downloading only the needed one, thus achieving the same bandwidth savings as RSYNC (well, at least if I correstly understood what rsync was about)... The question was: is there another application out there ("ftpsync"?) which would do the same?
_________________
. . . . . . . .
eXess

kbielefe · n00b Joined: 08 Aug 2004 Posts: 35 Location: Gilbert, AZ

Okay, I understand the question better now. Sorry for the runaround. I did some more digging and found the -N option for wget. Check out http://www.sunsite.ualberta.ca/Documentation/Gnu/wget-1.7/html_chapter/wget_5.html for more information. Basically, you want to use the following command on the mirror::

eXess · Posted: Sat Oct 22, 2005 8:01 pm Post subject:

That's MAGIIIIIIIIC !!!