HOWTO:Download Cache for your LAN-Http-Replicator (ver 3.0)

dhurt · Posted: Thu Jun 03, 2004 5:13 am Post subject:

Use FTP and then have a different directory for each architecture. Then specify the location in /etc/make.conf and the bin host optoin:

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

I've submitted the updated ebuild, now it is up to the gentoo dev's to see if this makes it into the portage tree!!

I guess a couple of good words wouldn't hurt :-)

https://bugs.gentoo.org/show_bug.cgi?id=50872

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

Ymerej · n00b Joined: 17 Oct 2002 Posts: 10

I installed and set up http-replicator, I did an emerge of a package on a client box, and observed that it went through the proxy -- great.

I got here basically by the same path as some other posters: I didn't want to query the rsync servers too much, with "emerge rsync" on each of the three Gentoo machines on my LAN.

I am going to ask a really naive question here: Does this http-replicator also cache the "emerge rsync" so that I can do it once on the http-replicator server, and then on all my other gentoo boxes that use the http-replicator proxy, and not risk overloading rsync servers and getting banned?

Or do I still need to set up an rsync server on one machine on my LAN?

Thanks.
_________________
-Gentoo newbie

dhurt · Posted: Mon Jun 14, 2004 6:10 am Post subject:

You still need to set up the rsync mirror. It is not cached by the http proxy. Here is a link to the updated how-to:

https://forums.gentoo.org/viewtopic.php?t=180336&postdays=0&postorder=asc&start=0
_________________
"And isn't sanity really just a one-trick pony, anyway? I mean, all you get is one trick, rational thinking, but when you're good and crazy, ooh ooh ooh, the sky's the limit!" -- The Tick

soulwarrior · Guru Joined: 21 Oct 2002 Posts: 331

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

I recommend all users with a LAN setup http-replicator using this HOWTO and also setup a local rsync server using the HOWTO at https://forums.gentoo.org/viewtopic.php?t=180336

This will best serve your lan and gentoo!!

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

I made a small change to the title of this thread. I hope 'download' will make more people stop and look at what http-replicator can do for them!!

Work has started on the next version of http-replicator!!

dhurt · Posted: Mon Jun 21, 2004 3:07 pm Post subject:

Thanks for the hardwork

_________________
"And isn't sanity really just a one-trick pony, anyway? I mean, all you get is one trick, rational thinking, but when you're good and crazy, ooh ooh ooh, the sky's the limit!" -- The Tick

Gherald2 · Posted: Wed Jun 30, 2004 7:07 am Post subject:

Is there a way to configure the clients to fetch stuff themselves when the server is down?

I ask because I'd tell other people on our dorm network about my server, but I can only guarantee 90-95% uptime and I don't want their fetches to fail on my account...

deezee · n00b Joined: 19 Jul 2002 Posts: 7

This is a really good concept, but it seems to me that it would be a lot easier to mount /usr/portage/distfiles (and maybe even /usr/portage to solve the whole rsync thing as well) through NFS or samba, but of course this carries with it other problems. (Although none of them are unsolvable.)

Anywho, that's how I do it... :)

dhurt · Posted: Thu Jul 01, 2004 12:22 am Post subject:

One of the problems of NFS, Samba that this solves is when two clients start downloading the same file at the same time.
_________________
"And isn't sanity really just a one-trick pony, anyway? I mean, all you get is one trick, rational thinking, but when you're good and crazy, ooh ooh ooh, the sky's the limit!" -- The Tick

ejona · n00b Joined: 09 Jan 2004 Posts: 11 Location: Texas, USA

First of all, I would like to mention that I started reading on the board from the tsp program. I was happy with what that one offered (and was okay with the bugs), but this has no downfalls what-so-ever that I can see.

I can say that using the how-to worked perfectly. I am on a AMD64 processor and since it workes fine I would say the only thing I see that it needs is to add in amd64 to the KEYWORDS="x86" in the ebuild

.

Thanks for the work (I am especially thankful - for I am on a modem connection and have 4 gentoo boxes - needless to say, I am also using a local rsync mirror) and I hope the developers include the package in portage soon. I will also keep an eye out on ways to break it.

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

Thanks for the report ejona, I've updated the ebuild to include amd64 in the keywords.

Can anyone confirm any other arch's?

larand54 · l33t Joined: 20 Feb 2004 Posts: 695 Location: Sweden

I just tried to use this "http-mirror" but failed.
The server looks to be running but the client never tried to connect to it.

larand54 · l33t Joined: 20 Feb 2004 Posts: 695 Location: Sweden

I finally made it!

Monkeywrench · Posted: Fri Jul 02, 2004 12:43 am Post subject:

This is awesome! Both this and your rsyncd howto. Thank you flybynite. You should contact the people who do the gentoo weekly newsletter about both of these howtos. It would save the generous mirror providers some bandwidth =)
_________________
Folding@home -Join the Linux team (163)!

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

Thanks monkeywrench!!

I submitted the howto's for inclusion in GWN!!

The newsletter is probably the best way to get the word out!!

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

dmitrio · Posted: Sun Jul 04, 2004 12:23 pm Post subject: :. copied to gentoo-wiki.com

I have copied this HOWTO, with suggestion of flybynite, to gentoo-wiki.com
http://gentoo-wiki.com/HOWTO_Download_Cache_for_LAN-Http-Replicator

If you see anything that should be added or changed, feel free to do so.

Thank you for a great HOWTO.
_________________

... Leaving ground, destination is unknown,
into the darkness and far away from home,
Will your dream come true and what will you find,
when fate is your guide ...

meowsqueak · Posted: Mon Jul 05, 2004 12:01 am Post subject:

Ok, so http-replicator is set up as a proxy on my LAN for all my Gentoo boxes. But how do I set up the http-replicator itself to use an external proxy? I can't see anything in /etc/http-replicator.conf that might help.

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

Sorry for the delay, my net access has been a problem for almost a week.

Your right, support isn't there for a proxy.

It has been added to the next version already in testing....

xkb · n00b Joined: 13 Jun 2002 Posts: 20 Location: Netherlands

I get the following error on the machine running the proxy:

Gherald2 · Posted: Sun Jul 11, 2004 8:30 am Post subject:

Hmm, have you tried localhost:8080 instead?
_________________
Unregistered Linux User #17598363

meowsqueak · Posted: Sun Jul 11, 2004 10:51 am Post subject: