Is there a web-caching program that doesn't require proxies?

[kubark42]

You'll excuse me the question if the answer is patently obvious. I'm a newbie in this field. Anyway, on to the point...

I hate proxies. There's nothing more madenning for users to set up, and nothing more time consuming for me to deal with (mostly, I admit, because I don't know what I'm doing). I'm setting up a wireless network, so the users will most likely be very mobile and the last thing I want to do is mess with their browser settings.

I know that Squid can do caching through a web proxy. Are there any servers that don't? Is this even physically possible from a networking point of view? One idea I had was to set up my router/gateway and then put another server between the gateway and the internet, having the gateway server use the squid server as a proxy. Seems like an ugly hack, though. Any ideas?

P.S. If someone can just point me in the right direction, I'll use google from there.

[Spooky Ghost]

You have to use a proxy to get caching. At some point some program on your network has to inspect the requested URL to see if it is stored locally, there is no other way for this can be done. If you don't want user's to have to play with their browser settings then you should set the program up as a transparent proxy, all outgoing requests on port 80 get diverted at the gateway to your proxy software.

Most browsers now support "automatically detext proxy settings" anyway so you can create a wpad.dat file as the central proxy config information. If the user can't access this offsite then they will fall back to a direct connection without any intervention required. It also means if you change the proxy setup there is only one file to be edited to update all the users.

[Fulgore]

Don't know how you want to set up your net but if you are looking for a easy. O/S distro that can take the pain out the problem, check out www.ipcop org.
I've been using this as part of my home net now and find it great.

Low hardware req. (P1/64mb works for me)
2.4 kernel
Caching Web Proxy (can be transperent)
DNS cache
DHCP server
VPN
Stateful firewall with port fowarding
etc.,etc,

Don't mean to sound like a sales broucher

Its simple enough to join together your wireless A/P, home net & modem without getting paranoid whenever a H/D bursts into life!

[kubark42]

Fulgore--

I've checkedo out ipcop. It's great software, but since I'm setting up a network that I will leave to others to administrate, I'd rather only use the same system time and time again. I'm not building a home network, I'm building a campus wide dormitory network. Each building gets it's own server and ADSL connection. Yes, it's a waste of resources, but it's for the French administration, so what can I expect? Through their suppliers, I can get 300m of CAT5e for $180. In Paris, it can be bought on the open market for $90.

Spooky Ghost--

I figured that that'd be the problem. What do you think of the idea of putting the web-cache between the server and the Internet? The server would route all requests through the external WAN, and the users would only use the internal gateway that they get through DHCP.

If only DHCP could give out proxy settings. That would be really cool!

[Spooky Ghost]

Where you put the cache will probably depend on the architecture of your network and how many users there are on the network. If your gateway is also your firewall I would recommend that you put the proxy on the inside (or DMZ) of your network otherwise you will have to think about the security of another machine. On your gateway you should add firewall rules that redirect all traffic to port 80 to the proxy machine unless that traffic comes from the proxy.

To use the "automatically detect proxy settings" browser feature you need a webserver on your network. If you already have one you could use that, otherwise install a webserver on the Squid box. Once that is done add a CNAME entry in DNS for the machine using the name wpad. On the webserver you should create a proxy config file which is in the DocumentRoot called wpad.dat. See http://www.freeproxy.ru/en/free_proxy/faq/wpad.htm and http://wp.netscape.com/eng/mozilla/2.0/relnotes/demo/proxy-live.html for more on how to set this up.

[Houdini]

Look at transparent proxying with Squid. You get all the benefits of Squid (caching, content control, ACLs) with none of the hassles (client configuration). Your users just see that slashdot loads fast
_________________
^]:wq

[Fulgore]

I jacked a number of Ipcop boxes into bussiness network (none with wireless though) and find that have a standard web interface make admin a lot clearer, and also has good doc you can just give to the local admin.

I here you on the resource issue. Work of a campus over here with what seems to be the same moral fiber when it come to holding a monopoly on supply. Not good when you are working to a budget yourself.

If a 'plug'n'play' distro won't fit then totally agree with Spooky Ghost.
Transperent squid proxy, lock it up in a DMZ with a simple Apache web server to handle the auto-magic.

Depending on net topo should be simple & work fine.

Have fun

[tdb]

You can set up Squid and IPTables as a transparent web proxy. This way your users won't have to change a single setting. Iptables will intercept all outbound tcp port 80 web traffic and redirect it to squid. Get squid up and running. It will listen on port 3128 by default. Then, add this to your iptables script:

* iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

See: http://en.tldp.org/HOWTO/TransparentProxy.html
_________________
Do you know what a usufruct is?

[kubark42]

Okay, thanks for the replies, everybody. Now that I know that it's transparent caching that I want, I'll get right on it.

I still would like to know if it would be possible to install the proxy server OUTSIDE the intranet between the routing server and the ADSL line. Then I should be able to cache everything, not just webpages, right?

Also, I'm running Squid on the same machine (although on a different SCSI HD) that's the terminal server for five diskless terminals and the router for the wifi (student) network. Is this a bad idea from a security perspective? I plan on seperating the networks with a third ethernet card so that it will physically be a seperate network. Good idea/Bad idea?

--tdb

Thanks for the tip with iptables.

--Fulgore

It's especially bad when the administrators tack on expensive equipment to your budget because "you've got money." I'm trying to sell the project to the university as THE low cost solution, and that gets mighty hard when I don't have the budget numbers to prove it because some high up bureaucrat decided they needed something fancy.

[tdb]