need advise on clustering

sanrou · n00b Joined: 10 May 2004 Posts: 31

Hello all!

I run a web radio site on a AMD K-6 (500MHz + 192M) box, during peak times, the apache server load is dangerously high, since listeners are rushing to the chat room, browsing and posing on the forum, etc.

As I have a couple of similar machines (bandwidth is not a problem), I have been considering build a cluster to ease the situation. The information on clustering is overwhelming, so I guess I'd ask for some input as my initial step towards the project;-)

The objective is simple - to improve apache performance, response time....

My question is:

What would be the best thing I can do with my a couple of spared (low profile) machines? I have heard openMosix, MPICH, LVS, but I am really confused as what is my solution.

Thanks for any advice!

Jindan

PaveQ · Posted: Sun Feb 26, 2006 10:09 am Post subject:

It would be better to put mysql and apache to different machines and use 1G ethernet between them. Real clustering could be way too hard and inefficient. Just separate the services.

Janne Pikkarainen · Posted: Sun Feb 26, 2006 6:28 pm Post subject:

As a first advice I would also recommend separating the services to separate physical servers. Much easier than any kind of clustering.
_________________
Yes, I'm the man. Now it's your turn to decide if I meant "Yes, I'm the male." or "Yes, I am the Unix Manual Page.".

sanrou · n00b Joined: 10 May 2004 Posts: 31

hmm... sounds like clustering is not recommended...i guess i'll do separation of mysql and apache server for the first step.

still i need to clarify the point here: to improve the performance of apache + mysql + php, is it a good idea at all to cluster? if yes, what technique should i adopt? i've heard openMosix wouldn't do much good on this job, true?

thank you all

sanrou

Janne Pikkarainen · Posted: Mon Feb 27, 2006 7:43 am Post subject:

OpenMosix is good if you have to distribute lots of long-running tasks (such as computation tasks) around. But Apache needs to serve lots of requests, with a very short life-span each, so OpenMosix will not be very effective in that: migrating a process from one server to other takes more overhead than serving the actual http request.

If you really need to cluster, then you'll need to think your system.

- Is its filesystem layout mostly static? Is your radiostations playlist just a bunch of mp3 files where you add an album or to every now and then? Or do you generate some new content all the time?

If you don't need to generate new content very often, then you don't need any kind of clustering filesystem. Instead you could have a master server and from there copy everything to slave servers whenever something changes. Basically this can be done with simple shell scripts and rsync.

Then put some load-balancer in front of everything: most basic form would be a round-robin dns, but you could also put LVS or some iptables script to do the magic.

On the other hand, if you need to change content a lot, then you need some kind of cluster filesystem or a network filesystem. For many purposes NFS should be good enough: then your master server acts as a file server, and all the slave servers will mount an nfs mount to your master server. NFS has a tendency to cause problems in a very high traffic environment or with exotic applications, so test it with care.

Spreading MySQL across several physical servers in a multimaster manner is nowadays "simple": take a look at MySQL Cluster.

http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-overview.html

Using MySQL Cluster backend has some limitations, though: most notably you can't have very big databases, not at least with legacy hardware. You should read the manual for the other common pitfalls, too.
_________________
Yes, I'm the man. Now it's your turn to decide if I meant "Yes, I'm the male." or "Yes, I am the Unix Manual Page.".

sanrou · n00b Joined: 10 May 2004 Posts: 31

thank you! now i have a much clear idea!

i don't have much mp3s stored in the shoutcast server, i send the stream from different computer (home and office, windows box). about 2 hours a day, i do hosted program, take song requests from the listeners via chatroom, this is the time period when my server capability is challenged.

i'll need some time to digest the information you've provided.

thanks,