A flawed methodology, but interesting results nevertheless

Nov 2, 2015 13:05 GMT  ·  By

NirSoft, a small company that makes all kinds of software, decided it had enough of false positive alerts about its own files on VirusTotal, and ran a test to see what the most common offenders are.

Because the company develops software that can be used by both good and bad guys, meaning hackers, lots of the antivirus engines in VirusTotal flag it as a virus or malware, despite the software not containing any kind of malicious code.

Since Nirsoft produces a lot of "password-cracking" applications, you might understand why some AV engines mark it as malware.

This is an old debate that has also surrounded applications like keygens and game cracks, which, despite not containing malicious code, are generally marked as malware.

This behavior, as antivirus engines argued in the past, is for the user's protection, because many such files are usually detected as infected.

This explanation was not satisfactory for NirSoft, which set out to test each of its files, and see what antivirus engines showed the most false positives.

In-depth infection explanations were taken into account

Besides false positives, the company also analyzed the warnings users were getting, and more specifically, they were looking for antiviruses that showed explanations about the detected infection by telling users the file was not actually malware, but a program that might be used by others to harm their data, privacy, and computer.

Since NirSoft's results are based only on scans of its own applications, the results should be taken with a grain of salt. The full report, results and in-depth methodology can be found on their blog. Below are just the main test results.  

AV Name Total Alerts No virus No PR* Trojan Alerts Score
AegisLab 0 0 0 0 100
Alibaba 0 0 0 0 100
ALYac 0 0 0 0 100
ByteHero 0 0 0 0 100
ClamAV 0 0 0 0 100
Emsisoft 0 0 0 0 100
Panda 0 0 0 0 100
Qihoo-360 0 0 0 0 100
Tencent 0 0 0 0 100
TotalDefense 0 0 0 0 100
VBA32 0 0 0 0 100
Zoner 0 0 0 0 100
nProtect 1 0 0 0 98.5
Microsoft 3 3 0 0 97
F-Prot 2 1 1 0 96
... ... ... ... ... ...
Comodo 13 2 0 11 26.5
Antiy-AVL 27 19 7 13 -6..5
TheHacker 113 0 104 1 -230.5
Bkav 175 0 162 175 -1280.5
No PR = Number of alerts for programs that are not a password recovery tool. Meaning alerts for normal software applications.