How to identify hack attempts?

[slackie]

Are there any "general" signs which I should look for if I think I'm having hack attempts? I've been running this gentoo box as a webserver for some while (132 days uptime) and yesterday I noticed that my mysqld crashed. The messages log says "out of memory... killing PID". I restarted the service and everything seems to be back at normal. The mysqld.err log doesn't show any signs (of what I can tell) either.

Sometimes the server harddrive has too much activity though I know that ain't running any other scripts (like torrents, or conversions...). Could this be a sign of something malicious or is some kind of memory/swap dumping?

How can i make this box bulletproof? How necessary is it to run firewall scripts like iptables?

Thanks in advance!

[NeddySeagoon]

slackie,

First, look in your logs. Look at everything in /var/log

Out of memory suggests you have something running that has a memory leak. Eventually, it fills all of memory and all of swap.
Under those conditions, processes that request more memory get told they can't have any ... all except one.
Normal proesses should deal with being denied dynamic memory gracefully.

The kernel is not a normal process - when it asks for dynamic memory and gets told there isn't any, it starts killing processes to free some.
I suspect thats what happened to your mysqld process. That doesn't mean that mysqld is the process with the memory leak, just that it got killed to free up some memory.

Keep an eye on memory and swap use in top. See if you can spot a process with a growing memory footprint.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[batistuta]

[NeddySeagoon]

batistuta,

I don't actually know how the kernel determines what to kill. It must do something simple, since there is no memory to use to make a sensible decision.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[Havin_it]

To go back to the question, I had to DMZ my home server recently (long story) and was concerned about this exact question. If you have any servers/services exposed to the Intarweb, then it's pretty much a given that people will check you out and some will have a stab at hacking you. When my SSH port was suddenly Internet-facing, and moved to the normal port number (22), my log immediately started recording pages and pages of login attempts.

Nobody got in, but who wants to keep giving these people infinite chances anyway, at a cost to your server's resources? So for sshd, I installed a Python app called denyhosts, which bans IP addresses after 3 (or whatever number you set) unsuccessful login attempts. That has kept the volume down a bit, but of course new ones keep trying...

General points: don't expose any more services to the Internet than you need to. Do run a firewall on the machine, and only open ports for the services you need to expose. Look at the logs for the exposed services, as already mentioned. For some services it's harder than for others -- Apache for instance has masses of options and masses of exploits if you don't secure it appropriately.

Something I noticed in my Apache logs was a lot of failed requests for pages that would've been the default page for various webapps, e.g. phpmyadmin or postnuke. More opportunistic script-kiddies wasting my damn bandwidth... <simmer, seethe> ... I'd be interested in some way of giving those people a "three-strikes-yer-out" treatment like denyhosts does with sshd, anyone know if such an app exists?

[uweklosa]

@Havin_it: Have a look at http://www.modsecurity.org/.

Don't know if it does what you want. But it can do much for you.

[Havin_it]

Hey, thanks! Will have some reading to work out what it does/doesn't do, but definitely looks interesting!