| View previous topic :: View next topic |
| Author |
Message |
ReeferMac
Guru
Joined: 30 May 2004
Posts: 389
|
 Posted: Sun Aug 22, 2004 12:28 pm Post subject: Moving apache? HTTP GET /* |
 |
|
I want to move my web server over from it's current location (my old windows box), to it's new location (this lil gentoo machine I'm on now). It's a big directory (~10GB's), lotsa.. crap in it. I've tried zipping up folders and transfering them, but am finding the process not only tedious, but problematic as well (limited disk space on windows machine, so I have to make small archives, of large directorys, lots of nesting and sub-dirs, and links getting broken).
I've looked for web whackers/mirror utils, but only found windows one's that want to charge you... Then I thought about it for a minute... can't I open a terminal, issue some http commands, set my local directory, and tell it "go"... downloading *.*?
Can it be that easy or am I getting a wee too wishful?
- Mac |
|
| Back to top |
|
 |
devon
l33t
Joined: 23 Jun 2003
Posts: 943
|
| Posted: Sun Aug 22, 2004 12:39 pm Post subject: |
|
|
You may want to try the --mirror option for wget.
| man wget wrote: | --mirror
Turn on options suitable for mirroring. This option turns on
recursion and time-stamping, sets infinite recursion depth and
keeps FTP directory listings. It is currently equivalent to -r -N
-l inf -nr. |
|
|
| Back to top |
|
|
ReeferMac
Guru
Joined: 30 May 2004
Posts: 389
|
| Posted: Sun Aug 22, 2004 12:59 pm Post subject: |
|
|
WoW! Thanks Devon, didn't even know about that command!
Looks like it will do the trick for me, thanks (I knew it would be easy under 'nix).
- Mac |
|
| Back to top |
|
|
ReeferMac
Guru
Joined: 30 May 2004
Posts: 389
|
| Posted: Sun Aug 22, 2004 1:18 pm Post subject: |
|
|
Hmm... OK, close.
wget grabbed everything from index.html on down the list.. of everything that's listed on one of the html pages.. But I've got gobs of file's just sitting in directory's (and nested sub-dir's....) that I want to get also. It's not linked, or in any html page.. just file's sitting in directory's, and I know the names of all the directory's, so it's easy for me to share the files (but if you don't know what dir. to search for..). How do I grab all THAT?
Literally, I want to cd into the wwwdir, and "cp *" but over http://. Am I going to have to setup file sharing between the two machines to do this? Do I have to append all the main dir's off the hostname? Will the -m option recursively go through all the dir's off the named dirs?
Do I have to tell it
| Code: |
wget -m http://10.0.1.2/foo
wget -m http://10.0.1.2/foo/foo1
wget -m http://10.0.1.2/foo/foo1/morefoo
...etc. |
or when I give it the path to /foo, will it find the sub-dirs (and all sub-dirs of foo1 and morefoo)?
Thanks a million.
- Mac |
|
| Back to top |
|
|
devon
l33t
Joined: 23 Jun 2003
Posts: 943
|
| Posted: Sun Aug 22, 2004 1:26 pm Post subject: |
|
|
| ReeferMac wrote: | | But I've got gobs of file's just sitting in directory's (and nested sub-dir's....) that I want to get also. It's not linked, or in any html page.. just file's sitting in directory's, and I know the names of all the directory's, so it's easy for me to share the files (but if you don't know what dir. to search for..). How do I grab all THAT? |
The --mirror option should only follow hyperlinks across the site. How could wget know that you have a /foo directory if there is no link to it?  You could setup wget to get the unlisted directories by hand, but that can get tedious.
| ReeferMac wrote: | | Am I going to have to setup file sharing between the two machines to do this? |
You could start sshd on your Gentoo machine and use WinSCP to copy from your Windows machine to your Gentoo machine. Or you could emerge a ftp server and copy the files over using that.
The sshd option should allow you to start the transfer quicker (sshd should already be installed and WinSCP is a small download), but the transfer rate will not be as fast as with ftp. |
|
| Back to top |
|
|
ReeferMac
Guru
Joined: 30 May 2004
Posts: 389
|
| Posted: Sun Aug 22, 2004 1:48 pm Post subject: |
|
|
Thanks Devon, that's one thing I'm thinking.... Much as I hate it (shudder), since I'm going to be shutting the windoze box down anyway, I could just slap IIS FTP server back on it and let things fly... it just didn't seem very gentoo-ish.  I've got a good sized wget -m directory coming over ATM, and it's been sucking up some serious bandwidth on the LAN.. so I think it's doing a good job of digging down through. I think I could prolly get a few of the larger dir's and sub-dir's off this way, delete them from the host, and then have the disk space to zip up the remaining contents of the server, and bring those over that way.
Thanks again!
- Mac |
|
| Back to top |
|
|
phypor
n00b
Joined: 25 Jun 2002
Posts: 68
Location: Texas
|
| Posted: Sun Aug 22, 2004 5:06 pm Post subject: samba + rsync |
|
|
On your old windoze box, share the directory with your files to move
On your linux box, mount the share
rsync --progress --modify-window=2 -rlt SRCDIR DESTDIR
I have it setup at work to mirror one of the secretaries document directories to my and my bosses laptops so we can have full access to those files even totally unconnected.
In that case, the linux box is glue between two windows machines
_________________
ALWAYS stay away from tanks enguled in fire.
- 2000 Emergency Response Guidebook, pg307 |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
