Penetration Testing versus Vulnerability Scanning

[R0n1n]

It seems that more and more vulnerability scanning (i.e. running Nessus/ISS/Foundscan/Retina/etc...) against some boxes is being considered a penetration test by corporations (as well as various technical authors).

The reasons for Pen test loosing favor seems to be the requirements (or at least general consensus) that during a pen test identified holes be exploited wherever possible, and many people do now not want someone else poking around inside their network any more then they really have to allow. Also the time it takes for a pen test is typically much longer then what is required for a vuln scan and they are therefore more expensive.

With the amount of regulatory pressure increasing for Pen tests (FDIC, SEC, et al) are we getting into some dangerous waters with our move towards the easier/quicker/cheaper option?

It seems that in the current climate many organizations are being given a false sense of security by the fact that someone runs a scan against their network every once in a while. True, scans keep the skiddies out, but the determined attackers, the ones who actually might do some damage to your network, or steal something, do far more then this. (There was also a good talk at this years Blackhat briefings which higlighted some of the drawbacks of application scanning tools).

So, I am wondering what everyone thinks about this, are Pen Test and Vulnerability scanning the same (I believe that they are not, Vuln scanning is one piece of a pen test, and many times is only a very small piece), are Pen tests no longer needed? Do we need to redefine what a pen test is?

[hypronix]

I always thought a vulnerability scan is just the basic thing to do before attempting to enter a network. Just part of foot-printing, almost.

Now a penetration test is only an attempt. Because it all has to do with who's the smarter and innovative hacker, the guy pen testing or the guy wanting to get the files on some new cellphone technology [for example]. Within the pen test there is a limited time period, whereas a hacker generally has quite a lot more time to enter illegally into a network [no it's not always as in 'Hackers' ].

But I think pen testing is still a very good thing, and vuln scans don't come even close to what a real security audit should be about.

[The Grunt]

Personally I think you have to do a lot more than a vuln scan to count it as a pen test... I do not have much experience with most vuln scans as many cost money, but do they not only search for unpatched type vulns? What about stuff that is misconfigured? The program may think you want that to happen, when realistically the IT staff only wants internal access to the internal FTP server, but it's setup incorrectly to be open to all, anon connections. Will a vuln scanner pick that up? Granted that is a very basic and unlikely (I hope!) situation and is something not USUALLY overlooked but hey, it could happen. Basically I think if you aren't have a person pen test you are overlooking human error entirely. Not the error of the person doing the test, but rather the person who wrote the program. There is something that could possibly be open to attack that is not included in that program. It's the same as nothing can be secure. You have to test and configure from every possible angle, not just run some program that decides it's secure.

[thehorse13]

There is no such thing as a 100% hack proof environment. Pen testing isn't designed to achieve this and those who believe this are simply ignorant. Pen testing hedges your bets so to speak, or if you prefer, allows you to mitigate risk to an acceptable level. You can clean up low hanging fruit and such but if there is a particular system that you want to secure to the best of your ability, you better sit down and map out the entire box starting from the ground up. The same goes for network equipment too.

You will never see the end of pen testing tools, they are a part of many accepted best practices. The difference is you must understand how and when to use them. This is where their real value resides.

--TH13

[Tiger Shark]

Actually, I think there is a distinct difference between the two, their purposes and use. I'll throw my 2c into the ring..... Like you thought I wouldn't.....

Vulnerability Scan: This is generally an automated tool such as Nessus for use by systems administrators to check that they have properly patched and secured their boxes and applied appropriate firewall rules etc. In Hosses terms "the low hanging fruit". It's a confirmation check to ensure that everything on the hardening checklist was carried out and it all "took".

Penetration Test: This is a more intelligent "tool". It should be an independent person or persons who, within laid down and agreed guidelines, test such things as the security architecture, firewall rules that may leak information or allow more complex access than a simple Vulnerability Scan against services could show. It should also go further into looking for unprotected modem access, WAP's, the potential for social engineering, (are there stickies on monitors that shouldn't be there), maybe even a dumpster dive or two to see what is being given away. In short it's a much broader and more complex attempt to find holes in the network's security than scanning the obviously open services.

[slarty]

To me, pen testing might include the *intelligent* use of a vulnerability scanner, but if your report is JUST the output from a vuln scanner, consider firing your pen testers

Seriously though, pen testing involves a lot of other things, before and after, in the process which MAY include using a vulnerability scanner.

Also, it requires a professional to be able to interpret the output from modern vulnerability scanners. They tend to give rather a lot of false positives, i.e. flagging as a vulnerability something which in fact isn't.

Plus there is heaps of passive info gathering and intel which vulnerability scanners do not give you.

One of the points of a pen test is to find out how much information an attacker COULD obtain, perhaps with a view to restricting it. Even if they don't get anything important, they still might be able to obtain things which should be kept more restricted.

To me it seems that companies flogging vuln scanners are claiming either:
- It is a replacement for a pen tester, in a box, which is just as effective
- It will find every vulnerability which exists
- Any vulnerability it finds is real

None of which I believe to be true.

Slarty

[slarty]

Originally posted here by R0n1n
[B]The reasons for Pen test loosing favor seems to be the requirements (or at least general consensus) that during a pen test identified holes be exploited wherever possible...

In my quote document for pen testing work, I make it clear what my policy is on doing this. It won't be exploited "wherever possible". Only if I know I can do it in a fashion which probably won't cause any problems. And in any case, total precautions would be taken to ensure that any accidental DoS can be fixed very quickly. And I'd sign an NDA of course.

Slarty

[R0n1n]

Yes slarty I agree, my mistake putting "whenever possible" in there, I too only do when it isn`t going to cause a DoS, destroy a database etc...

So, it seems that everyone here is aware of the difference, which leads me to wonder why many of the companies I speak with don`t.......