my deb server got hacked, little help?

Hello all.
I got debian server which was haxed today. (Wondered why it lagged and checked auth.log and noticed that german ip had logged in as root which should be impossible because no one else has access to server than me).
It´s Linux 2.2.20-compact.
I had run apt-get update and apt-get dist-upgrade yesterday so everything should be up to date.
So I ran chkrootkit. Nothing critical came up.
I checked roots bash history, nothing.
Then i unplugged it from internet.
I´d like to know:
1. How to see how that hacker got in/got my root pass?
2.How to see what he did when he was inside?
3.What i should do before reinstalling?

Thx for all who answer.
<Lorvija>

Wow.. old kernel. I'm not surprised you got compromised.

Sounds like the attacker removed/deleted logs. Have you checked syslog, dmesg and other logs in /var/log (I'm not familar with debian so I don't know the default structure)?

Look for hidden files/directories in root and elsewhere.

Before answering question 3, the question is what do you want to do? Legal action or not? If not, plan as to how you'll prevent this. If you want to do legal action, that box will be evidence. Alteration will make it hard to use.

What services are you running ? Are you running an IDS ? Do you keep a backup of your logs ? Do you have system integrity checkers ? And I think it's time to update your kernel...Check logs for suspicious activities...run a ps -ax just to be sure no malicious services have been started by the attacker..but beware, he could have trojaned a few commands on your system.....Just a little info for the future always secure a box before putting it on the internet, well if it isn't in a dmz that is...And for dealing with the kiddies, if he is one that is, a simple chattr +a /var/log/messages will do more wonders than you think....

I checked all logs in /var/log and found nothing.

Is ther any command to search new files made since ** and search for hidden files/dirs?
I havent thinked of legal actions. I just wanna know how he did it and how to prevent it. If he did anything serious then maybe legal actions.

Im running apache(1.3), php(4.3.2), ventrilo(voice communication server www.ventrilo.com), eggdrops, psybnc, sendmail, and maybe something else I cant remember now.

I dont have any backups of logs.
Im not sure what you mean with system integrity checkers but I think that chrootkit is one.

Thx for this fast help, any other ideas?

<Lorvija>

Just noticed that same guy had been in october 1. too. Perhaps he came back to check something?

<Lorvija>

The find command can do that.

Quote:

---

options for find command

-amin n
File was last accessed n minutes ago.

-anewer file
File was last accessed more recently than file was modified. -anewer is affected by
-follow only if -follow comes before -anewer on the command line.

-atime n
File was last accessed n*24 hours ago.

-cmin n
File's status was last changed n minutes ago.

-cnewer file
File's status was last changed more recently than file was modified. -cnewer is
affected by -follow only if -follow comes before -cnewer on the command line.

-ctime n
File's status was last changed n*24 hours ago.

---

You'd probably want something like find / -ctime 1 -exec ls -a {} \;

Chrootkit is more of a trojan detector. You'd want something like tripwire to do system integrity checks.

Your apache is quite old since it's 1.3 (newer version are up to 1.3.27 and 2.x). That might be one way in. Others include the eggdrops, php (again it's an old version) and sendmail.

Is BIND running? FTP?

And if he's come back that means the compromise was done far earlier and he's checking to see if you've fixed it or if he can continue using your server for whatever it is he's using it for (DoS, DDoS, sharing files, etc.)

BIND isnt running but I have FTP runnning. Its proftpd-1.2.8
Tried tripwire but couldnt understand how it works( im too noob apparently).

So how to check how he came in?Or does that old kernel have many security flaws?
So its just because its old?

<Lorvija>

There's a variety of things that you seem to be running that are old. The reality is that every release of kernel, application, etc. a new flaw will probably be found in a short period of time. So staying viligent and up-to-date can be helpful. Kernels older than 2.4.x are vulnerable to some serious ptrace() exploits amongst other things. I suspect however that the attacker used an application to get into the server.

By any chance was this running a live website on it? And if so, was the code tested to ensure that someone couldn't used it as a method in?

It had one php site which had one login field which php script parses and i didnt test it. So that is the only possibility what the hacker could have used, but I think it wouldnt be possible to get root pass with it

He doesn't need to get root access. Just access. Once able to setup an account, then he can continue. Have you checked your /etc/passwd for any "unusual" accounts?