Consumer Reports: security tools test

Consumer Reports in its September edition has published a test of 12 anti-virus tools, 12 anti spyware tools, and 13 anti spam tools.

From "Our unique antivirus testing: how we did it"

Quote:

---

To see how quickly software makers update their signature lists, we gave all of the products Internet access. Then we spent weeks closely monitoring each product and noted how early, if at all, the manufacturer equipped it to detect newly discovered viruses.

[...]

To pit the software against novel threats not identified on signature lists, we created 5,500 new virus variants derived from six categories of known viruses, the kind you'd most likely encounter in real life.

[...]

Finally, to see how often the antivirus software raised false alarms by identifying benign files as viral, we scanned more than 100,000 clean files.

---

As for the results:

Both BitDefender Standard and Zone Labs ZoneAlarm Antivirus scored excellent (87% and 85%, respectively) on detection; along with Kaspersky's Anti-Virus Personal, they are recommended.

The overall scores (Detection, Features, Ease of Use, Speed):
1. BitDefender Standard - 87%
2. Zone Labs ZoneAlarm Antivirus - 85%
3. Kaspersky Labs Anti-Virus Personal - 82%
4. Norton Antivirus - 80%
5. Norton Antivirus for Macintosh - 80%
6. McAfee VirusScan - 77%
7. Trend Micro PC-cillin Internet Security - 75%
8. Alwil Avast! Antivirus - 68%
9. F-Secure Anti-Virus - 66%
10. Panda Software Titanium AV - 64%
11. CA/eTrust EZ Antivirus - 57%
12. PC Tools AntiVirus - 41%

For Antispyware:
1. F-Secure Anti-Spyware 2006 - 89%
2. Webroot Spy Sweeper 4.5 - 89%
3. PC Tools Spyware Doctor 3.8 - 88%
4. Trend Micro Anti-Spyware 3 - 85%
5. Lavasoft Ad-Aware SE Plus 1.06 - 79%
6. Spybot Search and Destroy 1.4 - 77%
7. Zone Labs Zone Alarm 6.5 - 76%
8. Sunbelt Software CounterSpy - 70%
9. CA/eTrust Pest Patrol - 67%
10. BitDefender Antispyware 9 - 51%
11. McAfee AntiSpyware 2006 - 44%
12. Microsoft Windows Defender (beta 1) - 43%

Hi Neg...hope you don't mind...thought I would provide another opinion piece..

Quote:

---

Looking for the Right Antivirus, Part V | Best Antivirus of 2006!
The series has reached the end of the road. I've made my decision about the Best Antivirus Product of 2006.

For those of you coming late to this party, over the last six months or so the newsletter has pursued an ongoing series on alternative antivirus packages. Back in December 2005 I wrote that I'd given up on Norton Antivirus and had been testing alternative antivirus utilities since the summer of 2005.

During the last year of testing, I've examined a wide range of antivirus product, and I've explored the features and options of many others. Products tested during this period include Avast 4.6 free and 4.7 Pro, AVG 7.1 Pro and Network Editions, BitDefender 9 Standard and 10 RC1, CA EZ Antivirus and eTrust Antivirus r8, F-Secure Anti-Virus 2006 and Internet Security 2006, Kaspersky 5 and 6, Nod32 2.5, Panda Titanium 2006 and Platinum 2006, and ZoneAlarm Antivirus. I've considered the features and specs of at least a dozen other products and rejected them because something didn't meet my ideal antivirus criteria.

---

http://www.scotsnewsletter.com/83.htm#avv

He took Bit Defender off of his review, so I guess a lot of these are subjective eh!

I knew someone would post something like that :D

I think we all know by now that everybody on the entire Internet has a differing opinion on what the best anti-virus product is... I'll go with the testing done by the leading independent consumer union in the world, rather than with Scot's, though - sorry, Scot :p

Hey, Neg! Was there a link to the article, or do you have to be a paid subscriber?

Or, was this from the printed mag?

It's in the September print issue. Last year's results are online ( http://www.consumerreports.org/cro/e...ware/index.htm ), but not the new ones (yet).

Wow. Trend has really dropped from last year (I have the corporate 2006 version), it's good to see the free Avast is holding it's own against the heavy weights, and it is surprising to see F-Secure at #9 for AV and #1 for Antispyware....

Seeing Windows Defender as last for Antispyware is surprising as their Antispyware Beta was fairly decent, maybe switching to Defender has become a low priority for them... :rolleyes:

I'm going to have to find the magazine and see just which versions were tested and what the criteria was. There are differenced in function and support between the consumer products and the enterprise products. And, some freebie versions are crippleware compared to the commercial versions. So, apples to apples comparison may not be possible.

AVG's not on there...

That's interesting that Zone Alarm did well. Considering I often see them downgraded by other leading mega magazines with deep pockets. I am with Negative, the Consumers Union and their laboratories are extremely independent and extremely kick ass. They don't mess around.

Another interesting note is that Spybot is down in the middle. I find it extremely useful. Windows defender? Well it's better than nothing.

Hmm thanks Neg, my enterprise license is comming due.

Hey, Road, that's why I want to see the article. CR does some pretty good comparisons, sometimes. Sometimes you wonder what they were thinking.

I remember a vaccuum comparison in the 80's. They put their stamp of approval on the cheap vaccuum that took out just 50% of the dirt, and diss'd the more expensive machine that got 80% of the dirt. If you made your purchase decision from that, you were condemned to eternal filth.

Why did some of those apps score lower than others. Spybot is the go-to (and free) tool, but they rank it in the middle. Does the automatic updates make the other better? I still run shy of commercial spyware tools, simply because I've heard too many stories, even now, of commercial tools caving to pressure to not treat some things as spyware.

CR rated them on blocking, features, and ease of use. The features rated are "Protects browser", "Protects start-up", and "Describes spy-ware".
Spybot scored "very good" on Blocking, "very good" on features", and "very good" on "ease of use." It's a Consumer Reports Quick Pick Recommendation.

F-Secure (#1) and Spy Sweeper (#2) scored "Excellent" on Blocking, "Excellent" on Features, and "Very good" on ease of use.
Spyware Doctor (#3) scored "Excellent" on Blocking, "Very good" on Features, and "Very good" on ease of use.
Trend Micro's product (#4) scored, respectively, Excellent, Excellent, and Good.
Ad-Aware (#5) scored Very good, Excellent, and Good.


Soooo... Spybot apparently doesn't pick up as much "dirt" as some of the other products; because it is free, though, it is recommended. What I like about CR is that they make it extremely easy to pick out either the absolute best performer (in this case, F-Secure) OR the best VALUE (in this case, Spybot). The vacuum test you're referring to, I assume, recommended a 50%-picker-upper over an 80%-picker-upper because the latter cost 10 times as much...

I use commercial tools to control a bunch of users who are clueless to what lurks among email and internet sites. I would have to compare the ability to see a snap shot at the click of a mouse for the entire enterprise before I would accept perhaps a quicker update. Especially when I can force stricter policies through McAfee in the interim before patch.

Not sure why they recomended a vacuum over an expensive model, unless you could by like 7 for the cost of 1. But they have a long and trusting track record with me. Hell my walmart POS vacuum is better than a 1000 dollar Kirby some dumb ass GF I had bought years ago.

Not all reviews are accurate, but CR does come out the best of the bunch, however they do rely on other's for their information as well.

Seems the worst one out there was PC World's recent reviews...

Quote:

---

Consumer Reports backs up CNET and PC Mag

The well-regarded U.S. product-testing magazine, Consumer Reports, hit the newsstands last week with its own ratings of PC security programs. The lab's testing separately rated the antivirus, antispyware, and antispam programs available from each vendor. In addition, the magazine contracted with security experts to generate 5,500 original virus variants to test behavior-based protection. CR also monitored how quickly the companies released updated signatures in real time over a period of weeks as new threats emerged on the Net.

Zone Alarm Security Suite received Consumer Reports' Quick Picks award — the magazine's version of Editors' Choice — for "the best all-around protection." Perhaps because it's well known that security suites haven't yet mastered the latest spyware, CR also gave Quick Picks awards to Webroot Spy Sweeper and PC Tools Spyware Doctor in the antispyware category (with the free Spybot as a complement).

These ratings make sense. They dovetail with CNET and PC Magazine's latest findings, both in the rankings and the award winners. Besides PC Magazine's Editors' Choice for the Zone Alarm Security Suite, for example, Editors' Choice awards also went to Webroot and PC Tools in the magazine's latest, July 2006 reviews of antispyware apps.

To be sure, it's not unusual for magazines to differ in their ratings of computer products. For one thing, PC World's tests were conducted in April using ZASS version 6.0 and the then-current versions of competing products. The other publications' latest awards are based on the newer ZASS 6.5.

But when a category is as important as security suites, and when one magazine's rankings deviate so much with no logical basis, I look for a reason.

I found the answer in personal interviews with principals at AV-Test, Symantec, McAfee, and Zone Labs. To solicit comments, I provided AV-Test and PC World with draft copies of this story. I then participated in a telephone conference call with PC World editor-in-chief Harry McCracken, test center director Ulrike Diehlmann, and senior associate editor Narasu Rebbapragada.

---

http://windowssecrets.com/comp/060810/