Consumer Reports' Annual Test of Security Products

[Negative]

Consumer Reports just released its annual test of security software (both suites and stand-alone programs). Its hands-down overall winner: Trend Micro.

- For complete suites: Trend Micro’s PC-cillin Internet Security 2007 first place (with a score of 81), Check Point’s ZoneAlarm Internet Security Suite 7.0 second (75), McAfee’s Total Protection 2007 third (72). The biggest loser: Symantec (Norton 360 and Norton Internet Security 2007 both score 62).

- For stand-alone anti-virus software: Trend Micro first (81), Check Point second (79), MS’s OneCare (Vista) third (76). For those of us using free software: Avast and AVG score the worst of all 12 tested (55 for Avast and 53 for AVG).

- For stand-alone antispyware: Trend Micro first (82), Sunbelt’s CounterSpy 2 second (79). When it comes to free software: Windows Defender scores 69 (the #7 spot), Spybot scores 60 (#10 out of 12 candidates). The biggest loser here: Lavasoft’s Ad-Aware SE Plus with a score of 48.

[nihil]

Hi, with the "stand alones" was that test purely of retrospective detection, or of interactive and on demand scanning?

If it includes interactive then the two anti spyware free products will score low almost by definition? AdAware doesn't do interactive, and SpyBot's "teatimer" is pretty rudimentary.

Similarly with the free AVs.................... how did they test compressed files and text files?

Finally, what operating systems were used.

[Negative]

CR didn't release much details on its testing method - some snippets:

"Our tests of security software, done in our labs and those of an independent research laboratory, try as much as possible to emulate the conditions programs actually face on the Web. To do that, we use real malware, including viruses that are slightly modified version of those found online, and spyware that we actually harvest from the Web."

For anti-spyware programs:
"Detection indicates how well the product was able to detect and block installation of spyware. We used spyware determined to be "badware" by StopBadware.org, an organization of which CU is a member. We also created variants of the actual spyware, then we tried to install the spyware to see whether it could block the installation."

Keep in mind that, for the ratings, CR also looks at ease of use and features. Spybot scores a "Good" for Detection, Very Good for Features, and Very Good for Ease of Use. The top anti-spyware software (Trend Micro) scores Excellent for Detection, Excellent for Features, and Very Good for Ease of Use...

[nihil]

You have a good point there Negative.

It is after all, Consumer Reports Magazine, which is aimed at your average user rather than a computer buff with some knowledge of how security applications work.

I would expect that the ones that score highly would be the ones it would be best to give to the average home user?



EDIT: Here is an update: TrendMicro failed the latest VB100 test:

http://www.snpx.com/cgi-bin/news55.c...0858305?-11434

OK you need to watch VB tests to see if they are relevant to you..... this was on the 64bit Vista platform.