14 DAY TRIAL // UK security vendor Prevx is proposing a method of measuring how fast antivirus programs respond to malware threats in real world scenarios via a small program installed voluntarily on people's computers.
Mr. Chris Bolin, Prevx's new president, who previously served as chief technology officer at McAfee, presented the company's initiative to the BBC.
Once installed on a computer the tool is supposed to monitor local file activity and record malware detection events from antivirus products.
In theory, this should allow it to determine how much time passed since a malicious file made its way into the system until it was detected and removed.
Mr. Bolin hopes that the program will be ready by November and will get installed by thousands of users and companies around the world.
The statistics gathered through it are supposed to help consumers make a more informed opinion about how well different antivirus products are able to protect them.
"Innovation needs to occur on the anti-malware side because it's growing exponentially on the malware side. We need to bring about change in an industry that is not changing," the Prevx president told BBC.
However, while this idea has its merits and could better reflect real life behavior than on-demand comparative tests performed in the lab, it still poses serious issues.
Graham Cluley, senior technology consultant at Sophos, enumerates some of them on his blog, starting with the privacy aspect.
"Are people who install the tool going to be happy with a third-party application keeping a record of when every program is installed?" he asks, especially since nowadays malware also comes via non-executable file formats like PDF, DOC or XLS.
Then, how will this tool be able to determine if the infection was properly cleaned? If it relies on antivirus alerts alone, there is a real possibility that sometimes the antivirus might report cleaning an infection, when it in reality it failed to remove all components.
Also, how will the program act when multiple anti-malware programs run on the same computer, or when the user employs a stand-alone tool to clean an infection. Which application will it credit for the removal or how will it determine that?
Another problem we see with this concerns the type of infections monitored. It's not clear if this tool monitors only active or passive infections, or both.
A passive infection refers to a malicious file, which only sits inactively on the hard drive. It has the potential to infect the system, but only if it is run.
Let's take a scenario where a user copies a folder from a network share, containing an .exe file infected by a new variant of the Sality virus.
Then let's suppose the antivirus program running on the system doesn't yet detect this variant and allows the file to be copied on the computer.
A detection routine is then added the next day, but the user never browses that folder or performs a full system scan for another two weeks. Will Prevx's program consider that the antivirus product left the user unprotected for two whole weeks?
"I think we all recognise that there is room for improvement in security products, and we need better ways to measure the effectiveness of the different solutions in the marketplace.
"Groups like the Anti Malware Testing Standards Organisation (AMTSO) are working hard to build standards for testers, which should make things better for purchasers of products.
"Prevx's tool may have noble aims - but to my mind, the concept of a free tool that measures the effectiveness of your anti-virus product on your PC is half-baked until the above concerns are adequately dealt with," Mr. Cluley concluded.