Webserver Messed up

Printable View

July 5th, 2005, 12:47 PM
gauravjulka

Webserver Messed up

Here I explain the activity which I have noticed recently in our organization's webserver recently.

The apache webserver is running on Windows 2003 protected by Symantec AV Corporate Edition. The virus definitions updated till 20th June.

For the past several days outward packet transmission is quite high. The virus spotted were w32.toxbot making entries in the registry and detected as files dhcpclnt.exe and netddeclnt.exe in windows system folder.

Though the virus have been cleaned after tweaking the registry but outward packet transmission is still quite high. The result a user sitting in Australia complaining our ISP that a machine from his IP range trying to port scan his computer.

Really Netstat revealed that unknowingly to us a large no. of ports are opened from our machine (numbered beyond 1024) and trying to scan the internet.. yeah its true... the packets are being sent outwards randomly to IPs say xxx.xx.xxx.12 then xxx.xx.xxx.13 then xxx.xx.xxx.14 and so on Infinite times ... tying to open port EPMAP at the remote end machines. Syn-Sent to all those machines and waiting for reply.

Has the webserver become netzombie, trying to amass an army of vulnerable machines over the internet creating a storm of unwanted packets over the internet.

What may be the reason for it...'cauz no virus is detected by Symantec AV now..

Help from the experienced minds at Antionline is solicited..
July 5th, 2005, 01:11 PM
Cemetric

Quote:

The apache webserver is running on Windows 2003 protected by Symantec AV Corporate Edition. The virus definitions updated till 20th June.

For the past several days outward packet transmission is quite high. The virus spotted were w32.toxbot making entries in the registry and detected as files dhcpclnt.exe and netddeclnt.exe in windows system folder.

Though the virus have been cleaned after tweaking the registry but outward packet transmission is still quite high.

So did you update your antivirus to the latest virus defs ... because the 20th of June seems a while back now doesn't it ??

If you look at this :

Quote:

Intelligent Updater:
Virus Definitions created July 4
Virus Definitions released July 4
Norton AntiVirus Corp. Edition:
Defs Version: 70704i
Sequence Number: 46379
Extended Version: 7/4/2005 rev. 9
Total Viruses Detected: 69962

LiveUpdate:
Virus Definitions created June 29
Virus Definitions released June 29
Norton AntiVirus Corp. Edition:
Defs Version: 70629h
Sequence Number: 46247
Extended Version: 6/29/2005 rev. 8
Total Viruses Detected: 69903

you can see that there are more recent virus defs out there.

I don't think by tweaking you registry alone you get rid of these viruses.

If I were you I would use an online antivirusscanner like TrendMicro and start from there ... then scan with antispyware tools and sorts ... most of the time adware and sorts can also be responsible for this kind of strange behaviour.

Best you take your system off-line too ... scan it in safe mode to be sure to get rid of all nasties.

And I think you better act quick , because if you look at this site you can see what this virus actually does and you should definetly update because this variant needs virus defs from:

Quote:

protection
# Virus Definitions (Intelligent Updater) :July 01, 2005
# Virus Definitions (LiveUpdate™) :July 06, 2005

I'm sure the smart minds on AO will say more or less the same and perhaps give you some more pointers.

[EDIT]

Sum Up:

- Update antivirus defs and rescan system (best offline)
- Use antispyware tools and check again.
- Be sure your system is completely clean and better protected before putting it back online.

[/EDIT]

C.[QUOTE]
July 5th, 2005, 01:50 PM
gauravjulka

Yeah the webserver is offline right now. Updated the Virus definitions too to 4th July. Found couple of new virus files netlib.exe and other files w32.spybot etc. AV cleaned all of them. Trendmicro has also been tried before while Online.
But anytime the system is put Online it sends out packets at extremely huge amount. THis is a complete mess, owing that the system is a webserver... we cant format it. Spybot Search and Destroy is also updated and tried along with McAfee's Stinger.
BUt the problem still persists. ANy other wayout of this mess plz... it will be highly appreciated.
July 5th, 2005, 03:07 PM
Cemetric

Quote:

THis is a complete mess, owing that the system is a webserver... we cant format it

So basicaly what your saying is that you don't have a backup incase things like this happen ??

That would be .... dumb ... sorry to say.

Anyway ...as it seems that these viruses/worms keep coming back they have to be coming from somewhere ...

Did you delete everything from the temp folders (documents and setting and under windows and maybe even under the root) best do this under safe mode?

Scanned it under safe mode as well?

Try a process explorer like this one to see which processes are active.
Also try to do a netstat -b to see which ports are opened by what executable or use a tool like this .

Maybe hijack this (found here ) can bring you more insight .

If you realy can't get rid of the pests there remains one solution... reinstall.

Good luck,

C.
July 5th, 2005, 07:41 PM
XTC46

Take the webserver offline. start it in safe mode and runa full system scan for viruses. The downtime sucks, but its better then getting blocked by your isp for constant port scanning. Make backups of all the important info, then nuke the box if you cant clean it.
July 6th, 2005, 06:10 AM
gauravjulka

Thanks cemetric and XTC for your help. I am yet to remove the temp files and folders from the system.. May be the spyware is hidden in somewhere there. Thanks for your links to useful sites to check ports and processes.
July 6th, 2005, 08:55 AM
SirDice

If you look at the definition w32.toxbot point 7..

It seems you "forgot" to update your server too..

Don't take any chances.. Take the server off-line, backup the important stuff, reinstall from original media, update it.. restore the backup..
July 6th, 2005, 09:17 AM
gauravjulka

After identifying and terminating some processes reported by Process Explorer as malware along with their thread trees. The activity subsided a lot.

Hey there are a lot of exes and batch files in Windows, system and Administrator/local settings folders having awkward names like fg01.exe, isupvsr.exe...... These too seem to be spyware stuff. But after terminating and deleting ISTSVC.exe and MediaCtrlK.exe along with associated process trees and all the temp files which I could find the activity is under control. ... but dont know for how much time.

Sirdice, I am trying to avoid all the executive formalities which will have to b fulfilled before proceeding towards backing up, formatting, and restoring the system. Because the site being hosted is crucial for the project and organization. Installing Service Pack 1 for window 2003 Server also. Maybe the luck will be on my side...

Also tools suggested by Cemetric for scanning Ports and Processes are extremely helpful.. thanx again for that ..
July 6th, 2005, 09:43 AM
SirDice

Quote:

Originally posted here by gauravjulka
Sirdice, I am trying to avoid all the executive formalities which will have to b fulfilled before proceeding towards backing up, formatting, and restoring the system.

You're screwed already.. Cut your losses.. start over and learn from the mistakes you've made...
There's no way to know what got changed, modified, backdoored whatever..

None of the programs on that server are to be trusted.. Netstat is a nice tool but if someone 0wn3s your system it might have been modified..

Quote:

Because the site being hosted is crucial for the project and organization.

It's probably also a good time to talk about backups, patch management etc....
You now have some "ammo" to get the executives to fork out some cash...
Invest it in a decent hardware firewall and backup equipment..
July 6th, 2005, 09:55 AM
Cemetric

Quote:

Also tools suggested by Cemetric for scanning Ports and Processes are extremely helpful.. thanx again for that ..

No problem glad to help...but

As I said before and as SirDice mentioned...

You should of had backups and you should of started to rebuild your server from scratch ...

That's a lot of should of's but again as SirDice mentioned also ... you can't trust any of those applications that runs on your system now since they could have been altered.

The tools I provided the links for were just to give you a clue as what it is your system is infected with and then to use that information to stop it in the future from happening again... preferably on a newly build system.. but that's up to you offcourse.

C.
July 6th, 2005, 11:22 AM
XTC46

Im sure your organization will be alot more upset when the server decides to take a nose dive without backups or when their crucial data disapears and shows up in the competitions hands, then they would be if you drop the server for a few hours to do this correctly.

You dont want to take chances on mission critical servers. A little downtime now WILL save you a lot of downtime later. spread the work out over a few days.

Run backups one night while no one is working, maybe a weekend. Gather all the software that you will need, find all CD keys, and create backups of your configs for each program.( I swaer there is nothing more frustrating then not being able to find a disk or a cd key when trying to meet a deadline) All of this can be done and the box is still operational. Write down all the IP information, all the little settings that may only take a few minutes to look up but will bite you in the ass when you are rushing to get it back online. Then one night do a differential backup of the box to go along with the full you did over the weekend. Order you a nice pizza, and some caffinated beverage of your choice and spend your night doing the following.

Format
Reinstall
Patch
Update
Restore
Test
Go Live.

then take the next day off.
July 6th, 2005, 12:22 PM
gauravjulka

Thank you all folks.. for your help.. Yeah XTC I suppose I will have to go for this procedure now. Still learnt a lot from all the recent R & D performed in order to clean. Got some positive results too. But am aware that the packet storm will start again if left alone.

Starting afresh is the only option .. . this week seems going to be verry busy for me now. Anywayz pizza with a bear is something I like most while working late at nights. THe procedure decided and the bash going to kick off. ..