bit-tech.net

Microsoft Security Essentials fails AV-TEST again

Microsoft Security Essentials fails AV-TEST again

Microsoft's Security Essentials software has once again failed AV-TEST testing, with the company downplaying the results while also promising to learn its lessons.

Microsoft has committed to improving its Security Essentials and Forefront Endpoint Protection software packages, following the release of a second independent test that saw the company's anti-malware suites miss out on a passing grade.

Responding to the latest AV-TEST results, in which Security Essentials and its enterprise-grade equivalent missed out on certification for the second time, Microsoft's Joe Blackbird promised that the company would try to do better - but claimed that the test was possibly a little unfair.

'AV-TEST's test results indicate that our products detected 72 percent of all "0-day malware" using a sample size of 100 pieces of malware. We know from telemetry from hundreds of millions of systems around the world that 99.997 percent of our customers hit with any 0-day did not encounter the malware samples tested in this test,' Blackbird claims, pointing to AV-TEST skewing results by using malware samples which are not present in the wild in any quantity as one reason for his software's poor showing. 'AV-TEST's test results [also] indicate that our products missed 9 percent of "recent malware" using a sample size of 216,000 pieces of malware. We know from telemetry that 94 percent of these missed malware samples were never encountered by any of our customers. When we explicitly looked for these files, we could not find them on our customers' machines.'

Aside from the use of uncommon malware samples, Blackbird also took exception to how AV-TEST weights it scores. 'AV-TEST reports on samples hit/missed by category. We report - and prioritize our work - based on customer impact.' Despite this, Blackbird has promised that Microsoft will try to do better. 'We continually evaluate and look at ways to improve our processes. We know from feedback from customers that industry testing is valuable, and their tests do help us improve.

'We're committed to reducing our 0.0033 percent margin to zero,
' Blackbird concludes - somewhat sneakily reducing the 28 per cent of 0-day malware and 9 per cent of recent malware missed by Microsoft Security Essentials in AV-TEST's testing using figures received from customer telemetry, and reaching a much less scary figure.

27 Comments

Discuss in the forums Reply
Phalanx 17th January 2013, 12:19 Quote
I use MSE, and I must say I'm not worried. Their telemetry system is one of the best out there, and they actually... listen to this... CONTACT YOU BACK when you send in data and tick the box to allow them to contact you for more information. A revelation, I know!

I trust them.
Parge 17th January 2013, 12:31 Quote
Agree with Phalanx. Whats more, most other anti virus applications feel like malware themselves!
Snips 17th January 2013, 12:51 Quote
I agree with you both wholeheartedly. I've tried so many AV's and MSE has been the most unobtrusive and reliable than anything else I've used. It doesn't keep shouting at me like an attention seeking child or forever asking to be updated and would I like an upgrade or one of our partners would like to offer you something nonsense.

It sits there quietly and protects efficiently and by what it was saying above, it's doing it very well despite the scare tactics from a testing developer looking to attract attention by badmouthing Microsoft.
Gareth Halfacree 17th January 2013, 13:18 Quote
Quote:
Originally Posted by Snips
It sits there quietly and protects efficiently and by what it was saying above, it's doing it very well despite the scare tactics from a testing developer looking to attract attention by badmouthing Microsoft.
Eerm... AV-TEST is one of the most well-regarded independent test organisations around. It has 25 full-time employees. Every anti-virus vendor submits its products for testing, and the AV-TEST certification is proudly displayed when a product passes. They've been going for over 15 years now - I hardly think they need to 'attract attention' by deliberately failing Microsoft's products.

Microsoft Security Essentials was subjected to exactly the same testing procedure on the same malware corpus as every other package on test. Most passed with flying colours; Microsoft's did not. These are the facts, and no amount of arguing from Microsoft will alter them.
ZeDestructor 17th January 2013, 13:26 Quote
I've been seeing similar reports from av-comparatives as well, another well-regarded AV testing/review publication (yes, I call it a publication, each report is over 5 pages worth of pdfs, and the reviews are in excess of 50pages of deep analysis).

So while MSE has been taking a hit recently for coverage, I still use and recommend it since I'm careful enough to avoid viruses entirely, and the people I recommend it to are savvy enough to see through and and avoid nearly all. Most windows reinstalls I've done were just from corrupt system files and adware, not malware infections. In fact, I'm the one with the most malware cases (I'm excluding annoyance scripts because those do nothing truly harmful besides use some CPU time) in my entourage... with a total of 1 infection in something like 10 years...
dyzophoria 17th January 2013, 14:11 Quote
in defense of MS, they usually take/prodce their signatures from actual files taken from infected machines, so it really makes sense why MSE would fail on AV-TESTS, but as per MS, they are trying to improve on it though, my guess is without not much actual data regarding those missed viruses its plausible why MSE can't detect most of them,lol, especially they are quite new to anti virus tech compared to the veterans
GregTheRotter 17th January 2013, 14:14 Quote
I use malware bytes alongside MSE. Can't say I've had any issues so far. Then again, I'm careful with what I download/click on.
tigertop1 17th January 2013, 15:15 Quote
Not a problem as far as I can see. I have used MSE in conjunction with Malwarebytes and other software and never had a problem over 5 years . It is now the second most popular Anti Virus programme in the world now so others like it too! I have tried just about all of them right up to Norton 2013 and none of them are as simple, unobtrusive and as quiet to get on with things as MSE.
loftie 17th January 2013, 15:32 Quote
I like MSE. Not a fan of what MS have done to Windows Defender in 8 though. Not had any viruses since I switched to MSE and I imagine it'll continue that way, so not particularly worried if I'm honest.
Snips 17th January 2013, 15:41 Quote
Quote:
Originally Posted by Gareth Halfacree
Quote:
Originally Posted by Snips
It sits there quietly and protects efficiently and by what it was saying above, it's doing it very well despite the scare tactics from a testing developer looking to attract attention by badmouthing Microsoft.
Eerm... AV-TEST is one of the most well-regarded independent test organisations around. It has 25 full-time employees. Every anti-virus vendor submits its products for testing, and the AV-TEST certification is proudly displayed when a product passes. They've been going for over 15 years now - I hardly think they need to 'attract attention' by deliberately failing Microsoft's products.

Microsoft Security Essentials was subjected to exactly the same testing procedure on the same malware corpus as every other package on test. Most passed with flying colours; Microsoft's did not. These are the facts, and no amount of arguing from Microsoft will alter them.

'We're committed to reducing our 0.0033 percent margin to zero,'

Not bad then for REAL cases then is it?

I've never heard of AV whatever they're called until now Gareth. However, I'm not doubting their credentials if you aren't.
fdbh96 17th January 2013, 17:57 Quote
Quote:
Originally Posted by Gareth Halfacree
Eerm... AV-TEST is one of the most well-regarded independent test organisations around. It has 25 full-time employees. Every anti-virus vendor submits its products for testing, and the AV-TEST certification is proudly displayed when a product passes. They've been going for over 15 years now - I hardly think they need to 'attract attention' by deliberately failing Microsoft's products.

Microsoft Security Essentials was subjected to exactly the same testing procedure on the same malware corpus as every other package on test. Most passed with flying colours; Microsoft's did not. These are the facts, and no amount of arguing from Microsoft will alter them.

I use the test that if you don't get any viruses there isn't any need to change. If I ever get a virus, then Ill use something else.

As already mentioned, most other AV suites just have constant pop ups demanding updates/payment. MSE does none of this, and this is why I install it on all the family+relatives pc's because pop ups just confuse people.
SlowMotionSuicide 17th January 2013, 18:31 Quote
I've also been happily using MSE since it was published in 2009, and so far I've been nothing but happy with it, it's unobtrusive, simple and quiet little workhorse, and I haven't had a single infection all this time. I'm also installing it to any system I build to family, friends or relatives, with no issues whatsoever.

Our local computer publication tested several free and commercial internet security and antivirus products, and MSE was rated last in the test. Alerted by this, I decided to give a shot to a couple of other free programs mentioned (Avast, avira and comodo) and quickly went back to MSE. Even the installation of those programs were somewhat infuriating with them trying to sell me something I didn't want at every turn. And regarding unnecessary pop-ups, there's some serious attention whores out there, too.
dicobalt 17th January 2013, 20:16 Quote
This is a plot by John McAfee to sell more of his antivirus products. What? Don't tell me it's not the kind of thing he would do lol
Andy Mc 17th January 2013, 21:38 Quote
I find viruses/malware are not an issue at all. My daily driver is a Linux box and my gaming PC is a windows PC, that I use MSE on.

The main appeal of MSE is that it intergrates with the OS much better, due to being an MS product. Because of this I find it uses WAY less resources than other alternatives.

If I had to use another product then it'd have to be winClam. All other Commercial solutions are too resource hungry and all other free alternatives are just adware.
DragunovHUN 18th January 2013, 02:42 Quote
Eh, wouldn't that uncommon test malware be equally uncommon for the rest of the tested AV solutions too?
fluxtatic 18th January 2013, 07:03 Quote
Quote:
Originally Posted by Andy Mc
If I had to use another product then it'd have to be winClam. All other Commercial solutions are too resource hungry and all other free alternatives are just adware.

I used to use winClam in my BartPE builds, but it's no good for everyday AV, imo. Last I saw, no real-time protection and it was very strictly A/V - I'm way too lazy to scan like a responsible person. Just give me something with real-time protection and I'll scan files from torrents.

I've been using Avast for years (early on due to MS' stupid Genuine Advantage check on MSE) - other than the install being hideously annoying, it's great. Although last go-round, it would not leave me alone with wanting to install Chrome. Literally every few days it would pop up with a message saying I was 'nearly done' installing Avast, the last step being to install Chrome. Suspiciously, when I got fed up and fired off a pissed (like fire and brimstone pissed) message to Avast, the popups stopped. Come to think of it, maybe I'll replace it with MSE next time the license comes up...

Speaking of, no freakin wonder Chrome has the marketshare it does, since it's now part of the shovelware that comes along with every piece of free software on the entire internet (Chrome's not bad, but getting lumped in with the Ask toolbar is...I dunno, kinda degrading?)
SlowMotionSuicide 18th January 2013, 08:28 Quote
Quote:
Originally Posted by fluxtatic
Chrome's not bad, but getting lumped in with the Ask toolbar is...I dunno, kinda degrading?

It's like bundling a product with a free handjob from a hobo with if you ask me.
LordPyrinc 19th January 2013, 05:19 Quote
Never heard of AV-Test before this article. I've run MSE for over 4 years and have had only one brief issue which I was able to remedy with very little effort. Mainly due to some unscrupolous browsing on my part. Norton on the other hand is a resource hog and prone to corruption of its own files. Try removing Norton cleanly from a computer... not even possible. Norton should be defined as malware.
chrismarkham1982 19th January 2013, 08:40 Quote
Have had to do a few machines for people with viruses, always the same 3 or 4 people and always the same types of infections and yet since putting MSE on their machines and making sure auto update is on, ive not had to go out to any of those machines.
My own experience of MSE is good as-well, stopped using Avast to use MSE and havnt had any problems since.
Tattysnuc 19th January 2013, 09:26 Quote
Quote:
Originally Posted by Parge
Agree with Phalanx. Whats more, most other anti virus applications feel like malware themselves!

This^ I've not experienced any difficulties with MSE, but I'd like a bit more faith in it since I came off Symantec..

Go MSE!
GravitySmacked 19th January 2013, 10:04 Quote
I started using MSE on all my boxes a while ago (previously used NOD32) and I've not had any issues. I'm sure there are better paid products out there but MSE sits there quitely and does its job and I like that.
theshadow2001 19th January 2013, 17:00 Quote
Quote:
Originally Posted by Gareth Halfacree
Microsoft Security Essentials was subjected to exactly the same testing procedure on the same malware corpus as every other package on test. Most passed with flying colours; Microsoft's did not. These are the facts, and no amount of arguing from Microsoft will alter them.

Indeed, MSE seems to be quite pants. Anecdotal evidence supplied by readers is hardly a convincing argument to use MSE, especially when confronted with the results of empirical testing.
tigertop1 21st January 2013, 15:21 Quote
Quote:
Originally Posted by theshadow2001
Quote:
Originally Posted by Gareth Halfacree
Microsoft Security Essentials was subjected to exactly the same testing procedure on the same malware corpus as every other package on test. Most passed with flying colours; Microsoft's did not. These are the facts, and no amount of arguing from Microsoft will alter them.

Indeed, MSE seems to be quite pants. Anecdotal evidence supplied by readers is hardly a convincing argument to use MSE, especially when confronted with the results of empirical testing.

Well if it was anecdotal evidence you might be right -- 'Anecdotal' as defined implies that verification is dependent on the credibility of the party presenting the evidence. Seems to me that the vast majority of those posting on this thread have good evidential reason to trust MSE-as they are quoting from their own experience. I will stick with MSE thanks.
Gareth Halfacree 21st January 2013, 15:33 Quote
Quote:
Originally Posted by tigertop1
Well if it was anecdotal evidence you might be right -- 'Anecdotal' as defined implies that verification is dependent on the credibility of the party presenting the evidence. Seems to me that the vast majority of those posting on this thread have good evidential reason to trust MSE-as they are quoting from their own experience. I will stick with MSE thanks.
But 'experiential' evidence is no evidence at all. I eat far more crisps than can possibly be good for me; I have yet to catch the bad AIDS; ergo crisps offer adequate protection from AIDS. QED.

In other words: people use MSE, they have not fallen victim to malware, ergo MSE works. Same logic, and same flaws: would they have fallen victim to malware if they did not have MSE installed? In many cases - with the readership of this 'ere site, anyway - the answer's no: most people here aren't stupid enough to put themselves at risk by using out-of-date software, dodgy cracks from random Russian webpages, or clicking on the interminable "reset your PayPal access" links they receive via email. Thus it's impossible to say whether MSE is providing protection or their behaviour is providing protection.

That's why independent testing exists: the only way to be sure anti-malware software works is to expose it to malware, not to install it on client machines and say "our telemetry says nobody's infected." It's also why failing AV-TEST is a bigger deal than Microsoft is making out.

Not that I'm trying to convince anyone to switch anti-virus packages here. Frankly, I don't care: use whatever works (or appears to work) for you. I run Linux, so it's a moot point.
fdbh96 21st January 2013, 15:51 Quote
Actually I appear to have changed my mind. Just read the latest edition of PC pro and it also lists MSE way down the bottom. I can ignore the odd test or two, but when every test done proves its rubbish, I think its time to change. Apparently its not as lightweight as I thought either, so I'll be trying Avast and see how it goes (I bet I get a virus tomorrow ;))
theshadow2001 21st January 2013, 16:57 Quote
Quote:
Originally Posted by fdbh96
Actually I appear to have changed my mind. Just read the latest edition of PC pro and it also lists MSE way down the bottom. I can ignore the odd test or two, but when every test done proves its rubbish, I think its time to change. Apparently its not as lightweight as I thought either, so I'll be trying Avast and see how it goes (I bet I get a virus tomorrow ;))

hooray for evidenced based reasoning!
fdbh96 21st January 2013, 18:13 Quote
Quote:
Originally Posted by theshadow2001
hooray for evidenced based reasoning!

I know right, eventually you have to give in :D
Log in

You are not logged in, please login with your forum account below. If you don't already have an account please register to start contributing.



Discuss in the forums