Hackattack Response #2: STEP by STEP forensics for Linux and Windows

This has got potential to be a really good thread.

Let's imagine the following scenario.

One of your Red Hat servers has been compromised. The hacker has got root access. You want to know how did he get in.

What are the steps and methodology that you apply as soon as you are sitted in the console in order to get the big picture as fast as possible? Have you got this scripted?

EDIT: Let's assume this has to be a live analysis, we cannot afford to take it offline as it's a production server and every minute the server is down is worth several thousand dollars in cost.

My steps. Feel free to add yours.

[redhat@linux FC4]# w
[redhat@linux FC4]# who
[redhat@linux FC4]# ps -aux
[redhat@linux FC4]# lsof -i
[redhat@linux FC4]# netstat -an
[redhat@linux FC4]# dmesg
etc...

I would probably take the machine offline, pull the hdd, mount it in another machine with noexec, readonly, nosuid and nodev, then dd a backup off it and hash the backup to verify its contents later. Then I would copy the dd'd backup to another machine mount it noexec... and start your forensics with your favorite set of tools. If you had a network ids up somewhere on the same network you may be able to find out exactly how they got in. With the copy of the filesystem you should be able to find out a lot about what they did once on the machine.

Initially, you said the hacker has root; therefore, you are wasting your time typing those commands since you should operate under the assumption a rootkit has been applied/installed. You should have a CD on standby with statically linked versions of those commands, that way its a little harder for the commands to be hijaacked. Regardless, before I turned the system off, I'd use memdump to dump out everything thats running to removeable media. Browsing through memory can provide useful places for you to begin your search when you do go through the filesystem.

After you get the memdump, you can use something like helix to boot from to get your image of the hdd. After that, as h3r3tic said, mount the image (though I'd add to mount it read-only and to take a md5 hash of both the memory image and the hdd image and would also consider making two copies and only playing with the second). You can then go to town looking for the evidence (helix comes with several tools) but you can also use something like autopsy (if you don't have the funds for something like encase) and use it to browse the not only the file system but slack space and through deleted files and for looking at MAC times (modify/access/created), which can be very useful for backtracking what the hacker did while in the system.

Good tips there.

But I forgot to include an IMPORTANT detail here.

This is a production server, we are dealing with a critical server with a big database for an ecommerce application. Every minute that the server is offline costs the company several thousand dollars.

From a business perspective, we cannot simply afford to take it offline, even assuming root compromise or that a rootkit has been installed.

The first approach is to identify the incident, then contain it and then eradicate it. While always keeping it online, unless we detected that a major event has happened, such as credit numbers stolen.

So let's assume we cannot take it offline and we have to find out as fast as possible what has happened and how to contain it.

Let's see what are the steps and methodology we'd follow so that we can add detailed actions, tips, tricks and so on to the process.

Hi devantionline

Are you a former Enron employee?

Quote:

---

This is a production server, we are dealing with a critical server with a big database for an ecommerce application. Every minute that the server is offline costs the company several thousand dollars.

---

You have now transcended IT, and your decision making ambit..............time to call in the corporate lawyers and look at damage limitation exercises..............as well as your government law enforcement agencies?

You should have had a redundant back up server(S) shouldn't you?

If you are the type who believes in fighting fires on the fly, then I would FIRE you! :p

Your advice is verging on the criminally cretinous............you have obviously never had a proper job.

:D

Quote:

---

Your advice is verging on the criminally cretinous............you have obviously never had a proper job.

---

Judging by the scenario, most likely a student with an assignment due Monday....

:D

Quote:

---

Judging by the scenario, most likely a student with an assignment due Monday....

---

Could be?..............hell a "compromised " e-commerce server, and leave it online :eek:

Take it down immediately; or get sued by everyone who has their identity, credid card, bank account and other details stolen?

That is why you pay corporate taxation..................for the FEDs?

And if you p1$$ about like he is suggesting, you are compromising a CRIME SCENE!

Ack! phtt!

:D

Quote:

---

Originally posted here by nihil
Take it down immediately; or get sued by everyone who has their identity, credid card, bank account and other details stolen?

---

EXACTLY! Why in the world would you leave that timebomb online?

I think Relyt was right, it may be an assignment that is due soon. If this is a real e-commerce server, please let me know which company this is so I never use their site again! Cripes!

-Deeboe

Could someone do us all a favor and move this to tech humor.....

Obviously we have a young wo/man with a healthy imagination and the desire to be a hero....

Methinks, when s//herhe takes his high school computer science class s/he will see the error of his/her ways.... ;)

Wow, I think I've seen everything now. A newbie with a first post, posting way outta his/her depth. Getting a huffy replies from _nihil_, for crying out loud. AND, in serious danger of being eaten by Tiger Shark.

I can die in peace now.

;)