Have I been hacked??!!

[mattwood2000]

Hi guys, I run a Gentoo server on my home network - it acts as the bridge between my FiOS router and the rest of my network - it provides DHCP for the intranet and SSH access both on the WAN and LAN using public key authentication on a upper port.

I came home this afternoon and noticed my work computer had lost its exchange connection...which sometimes happens, so I didnt think much of it. I released and renewed the IP address, but the connection did not restore. I then did the usual debugging steps - ping and hostname, ping and IP, etc. Nothing was working so I tried to log into my server normally via SSH, but to my surprise, my key was refused!

I ran nmap on my external dnydns hostname and my internal server IP and got this strange output:

[cazort]

These results strike me as really weird and would raise some suspicions.

Did you run nmap with the exact same command line, in each case? I notice that the remainder of the ports are reported as "filtered" in one case but "closed" in the other. In some cases, though, this can be an artifact of running a "half-open scan", because this can't detect the difference between a filtered and closed port.

If you ran nmap the same way and got different results, and the ports were really being filtered on the box when it was not doing what you expected...and you had not set up the box to filter the ports, then...it's a sign someone may have set up their own firewall on the box (and was sloppy in that they set it up differently from yours which allowed you to see the difference you specified).

The response in the first case does not mean that the box was actually running those services...but it's weird that those ports are "closed" whereas the others are filtered.

---

What was the level of security on this machine? How many services were running on it? How diligent were you about security on it? I had a gentoo box on the open internet for years and it was constantly under attack by automated scripts, but I never had any trouble because I was moderately paranoid about how I set it up. If you compile with pie, the worst that will usually happen with buffer-overflow attacks is crashing your box or usually just some service on it.
_________________
Alex Zorach | Teacology | RateTea | Why This Way

[mattwood2000]

Hi cazort, I also found these strange addresses in my /var/log/messages log right after my reboot

[user124]

[mattwood2000]

I thought about an IP conflict, but I ruled that out when I nmap'ed my internal IP 192.168.0.1 which I have bind/dhcpd running on and still got the same strange port listing.

[mattwood2000]

Ok got an update on this. I found the strange ports open again when I nmaped my network this evening. So this time I controlled my fit of rage and didnt reset the server or anything. I brought the monitor down and logged in as root.

I ran "who" which showed only myself logged in. My lastlog did not show anything out of the ordinary either. I ran iptraf and logged what was going on and found a bunch of activity on 169.254.1.xxx addresses. I've attached the logfile to this post. Is this activity normal?

I tried a couple of the random addresses - especially ones close to the end of the log and there was some strange facebook crap. http://69.63.176.170/ showed "I'll find something to put here." but tracert traced it back to the facebook domain. As far as I know none of my computers were on facebook...I asked other people in the house and no one was on facebook at the time this went down...strange.

Also what I found strange was that if I ran "nmap -sS localhost" or "nmap -sS 192.168.0.1" from the server itself, the normal port listing showed up. Only from other computers on my network which had IP addresses assigned to them by the server did I see the strange port listing.

After rebooting the server, the same addresses were still there. I also rebooted the FiOS router and still the same addresses. Then just like that, I was able to log in again via ssh and the strange ports disappeared!

Something is definitely messed up.

Any suggestions?

Thanks, Matt.

[Jaglover]

Keeping a hacked box online is being accessory to a crime. You can investigate it by booting from some good CD but you have to take it offline immediately. In any case, in *NIX world there is no such thing as "fixing a hacked computer", only fresh clean install will do.

[mattwood2000]

Hi Jaglover...thats why I've already taken it off line...I wanted to see if it was a fluke or not. Now its offline....and yes there will be a fresh install happening along with a complete change of passwords, etc. My question, though is does the log file indicate a hacked box at all? What traffic should I look out for.

[Jaglover]

Well, there is literally nothing you can trust in this computer. For instance, ps command will not show pirate processes and so on. This is why you need to investigate it (or just the HDD) using some clean OS. Common places to look at are root command history and other log files. Basically, imagine you are a hacker and you just broke into a remote computer. You got root shell, what's your next task? Download your toolkit, naturally. So look for any traces of downloading, unpacking and executing such a toolkit. However, a highly qualified blackhat may leave no traces at all.
rkhunter may find something.
Checking files against a remote AIDE database will definitely tell the truth. Assuming you created one in the first place.
Another thing, how comes your firewall did not block opening those ports? Did this cracker hack your firewall too? Or you didn't have one? From description of your setup I take this FiOS box is not a router, it's just passing the public IP address to your Gentoo router. This means your Gentoo box is directly exposed to the internet. You cannot run a Linux computer without a well configured firewall when exposed directly to the internet. I'd consider using hardened Gentoo as well.

[Anarcho]

[Jaglover]

[Anarcho]

[Jaglover]

Agreed, if you have the luxury having different boxes. For a home network where one box has do do everything firewall is a good idea.

[Anarcho]