Antivirus Performance Test (February 2010)

Table of Contents:

Introduction
Antivirus Effect on Operation System Boot Time
Antivirus Resource-Intensiveness Comparison
On-access Antivirus Scanner Performance Comparison
On-demand Antivirus Scanner Performance Comparison
Antivirus Performance Comparison for Office Applications

 

Introduction

Antivirus performance is the most important characteristic for most users as well as the quality of protection itself. This characteristic is the one that both home users and corporate customers pay their attention to when buying an antivirus. Nobody needs powerful but too resource-intensive protection with which you just cannot use your computer for doing what you would like to.

If the protection quality is very hard to evaluate all by yourself it is quite easy to notice immediately when the operating system and other programs slowdown or file copying and web-pages downloading "hang up". A reliable and practically unnoticeable antivirus is the biggest dream of every usual user.

The objective of this test is to show how personal antivirus software influences the typical operations performed by the user, slows down its work and utilizes the system resources. 

While performing the tests, we measured and compared parameters having a direct influence on the user's perception of antivirus performance, namely:

  1. Operation system boot time.
  2. Memory & CPU used by antivirus software.
  3. File copying performance (on-access antivirus scanner performance testing).
  4. Scan speed (on-demand antivirus scanner performance testing).
  5. Boot time for the five most popular office applications.

The test results give a clear idea of the performance of antivirus represented in the market. Having compared this information with Anti-Malware.ru test results, every user can make an informed choice in favor of this or that antivirus solution.

Methodology used for antivirus performance testing »
Analysis of the test results and awards »

Sixteen popular antivirus programs participating in this testing included:

  • Avast Antivirus Professional 4.8.1368
  • AVG Anti-Virus & Anti-Spyware 9.0.716
  • Avira Antivir Preminum 9.0.0.75
  • BitDefender Anti-Virus 2010 (13.0.18.345)
  • Dr.Web 5.01.1.11171
  • Eset Nod32 4.0.4680
  • F-Secure Anti-Virus 2010 (10.10 build 246) (СТРИМ.Антивирус)
  • Kaspersky Anti-Virus 2010 (9.0.0.736 (a,b))
  • McAfee VirusScan Plus 2010 (13.15.113)
  • Microsoft Security Essentials 1.0.1611.0
  • Norton Anti-Virus 2010 (17.1.0.19)
  • Outpost Antivirus Pro 2009 (6.7.1.2983.450.0714)
  • Panda Antivirus 2010 (9.01.00)
  • Sophos Anti-Virus 9.0.0
  • Trend Micro Antivirus plus Antispyware 2010 (17.50.1366)
  • VBA32 WinNT Workstation 3.12.12.0

The test was performed on a computer with the following configuration: Intel Core 2 Duo E6550 / 2.33 ГГц / MSI P35 Neo-F / 2048 MB / GeForce 8800 GTS 640MB / 80 GB (ST380013AS); Microsoft Windows XP Pro Rus SP3 since December 20, 2009 up to January 5, 2010. See the detailed testing platform and software description as well as all the step by step testing scenarios in the testing methodology.

Acronis True Image program by Aflex Software (Acronis representative in Russia and CIS) was used to save the system images before and after antivirus downloading.

Not to allow any errors, all the test measurements were performed consistently five times with coming back into the initial position after each measurement. The obtained results were averaged out less limit values (maximum and minimum). The detailed description of all the testing stages is given in the testing methodology.  

 

Antivirus Effect on Operation System Boot Time

When you use an antivirus on your computer it usually increases operation system boot time. It is an undesired effect for a user. The results of this parameter measurement are given in Table 1 as well as on Diagram 1 and 2. 

Table 1: Antivirus influence on the operation system boot time

Antivirus Boot time [sec] Delay as compared to the initial system [sec] Delay as compared to the initial system [%]
No antivirus 38,24 0,00 0,00
Avira 39,06 0,82 2,14
Trend Micro 39,15 0,91 2,38
F-Secure 39,27 1,03 2,69
Norton 41,66 3,42 8,94
McAfee 43,91 5,67 14,83
VirusBlokAda 43,97 5,73 14,98
Outpost 43,99 5,75 15,04
BitDefender 44,48 6,24 16,32
Kaspersky 44,57 6,33 16,55
Eset 48,40 10,16 26,57
Avast 51,80 13,56 35,46
Dr.Web 56,04 17,80 46,55
AVG 59,33 21,09 55,15
Microsoft 59,62 21,38 55,91
Panda 61,47 23,23 60,75
Sophos 63,15 24,91 65,14

 

Diagram 1: Operation system boot time

Antivirus test - Operation system boot time  

In Diagram 2, Operation system boot time is calculated in percentage as compared to the initial system (before antivirus download).

 

Diagram 2: Booting slowdown as compared with the initial system

Antivirus test - Booting slowdown as compared with the initial system 

As we can see from Diagrams 1 and 2, antivirus download can substantially influence the operation system loading performance. Avira, Trend Micro, F-Secure and Norton antivirus are the best ones in this characteristic. These products have about 10% influence on the operation system boot time. Antivirus scanners having the worst characteristic are Sophos and Panda that slow down operation system load by 60%. When speaking about absolute values, operation system load delay is from 1 to 10 seconds with the most antivirus applications.

Unlike our previous test performed in August 2008, we did not award the best antivirus for the minimum influence on the operation system boot tile because this characteristic is not critical for antivirus performance assessment and an average user does not restart an operation system too often.

 

Antivirus Resource-Intensiveness Comparison

Memory using (Physical Memory & System Cache) by an antivirus at rest can differ by dozens of megabytes and system cache using – by hundreds of megabytes. The results of measuring memory usage with installed antivirus at rest are given in Diagram 3.

Diagram 3: Idle memory usage

Antivirus test - Idle memory usage

 

It is obvious that BitDefender, Kaspersky, Panda and Outpost antivirus use the minimum memory at rest as 50Mb is quite enough for them at rest. Dr.Web, Norton and AVG use the most memory at rest.

It is worth mentioning that Microsoft, Panda, VirusBlokAda, Trend Micro, McAfee and Avast use system cache more actively than others. The detailed information of memory and CPU utilization are given in detailed test report.

If we analyze both parameters (Physical Memory & System Cache usage) at a time then Outpost and BitDefender antivirus can be recognized as the best ones.

 

On-access Antivirus Scanner Performance Comparison

For an antivirus performance assessment, an on-access scanner performance is of the most importance. It is well-known that when loading, creating, copying or changing files on hard-drive they are checked by antivirus scanner. It is quite obvious that antivirus "intrusion" into file operations can really slow down the system.

Table 3 and Diagrams 4-5 represent the test files copying time on computers with different antivirus and its delay as compared to the system without any antivirus.

Test collection and file copying time measuring procedure are described in testing methods. 

Table 3: Antivirus influence on the file copying performance

Antivirus Copying time
Time
[hour:min:sec]
Delay
[hour:min:sec]
Delay [%]
No antivirus 0:05:27 0 0
Avast 0:06:25 0:00:58 17,74
Avira 0:07:25 0:01:58 36,09
Norton 0:07:38 0:02:11 40,06
BitDefender 0:07:58 0:02:31 46,08
Sophos 0:08:10 0:02:43 49,85
AVG 0:08:21 0:02:54 53,21
Kaspersky 0:08:32 0:03:05 56,57
Panda 0:08:40 0:03:13 59,02
Trend Micro 0:09:30 0:04:03 74,31
F-Secure 0:09:30 0:04:03 74,31
Outpost 0:09:31 0:04:04 74,62
Eset 0:12:00 0:06:33 120,18
McAfee 0:16:10 0:10:43 196,64
Microsoft 0:16:22 0:10:55 200,31
Dr.Web 0:16:52 0:11:25 209,48
VirusBlokAda 0:30:33 0:25:06 460,55

 

Diagram 4: The clean file collection copying time

Antivirus test - The clean file collection copying time

 

In Diagram 5, test files collection copying time is translated into slowdown percent as compared with 5 minutes 27 seconds (the same collection copying time in the initial system before an antivirus installation).

 

Diagram 5: The clean files collection copying slowdown

Antivirus test - the clean files collection copying slowdown

 

In accordance to the awarding scheme for the testing of the fastest antivirus on-access scanners win special awards (see Table 4). 

 

Table 4: The fastest antivirus on-access scanners 

Antivirus Slowdown [%] Award
Avast 15

Platinum Performance Award On-Access Scanning
PlatinumPerformance Award
On-Access Scanning

Avira 30 Gold Performance Award On-Access Scanning
Gold Performance Award
On-Access Scanning
Norton 34
BitDefender 39
Sophos 42
AVG 45
Kaspersky 47
Panda 49
F-Secure 62 Silver Performance Award On-Access Scanning
Silver Performance Award
On-Access Scanning
Trend Micro 62
Outpost 63
Eset 101 Bronze Performance Award On-Access Scanning
Bronze Performance Award
On-Access Scanning
McAfee 165  
Microsoft 168
Dr.Web 176
VirusBlokAda 386

 

By the test results, Avast antivirus scanner have unsurpassed performance (test collection copying slowdown is 15% less as compared to the initial system without antivirus). This antivirus result is much better than that of the others and deserves Platinum Performance award: Award On-Access Scanning.

A big group of seven antiviruses with Avira, Norton, BitDefender, Sophos, AVG, Kaspersky and Panda among them has high and close results in the range of 30-50%. All these antivirus applications win Gold Performance Award: On-Access Scanning.

We would like to pay special attention to Avira, Norton, BitDebender and Kaspersky antiviruses in this "Gold Group" that can be much faster in real conditions at the expense of further scanning optimization. These technologies availability was checked in the next part of the test.

F-Secure, Trend Micro and Outpost antiviruses also showed good real-time testing performance and won Silver Performance Award: On-Access Scanning. These products slowed down testing file collection copying by 62-63%. Eset antivirus with the slowdown result of 101% won Bronze Performance Award: On-Access Scanning.

F-Secure and Outpost antivirus scanners from the "Silver Group" can work much faster at the expense of their further real time testing optimization technologies. VirusBlokAda can also work much faster in real-time conditions and it is only due to the awarding scheme imperfection that it has not win any award.

It is worth mentioning that performance characteristics of antivirus scanners by different manufacturers have been improved and averaged. Actually, we have eleven different antiviruses with the real-time testing performance that hardly differs from one another. These are the products having Silver Performance Award: On-Access Scanning or higher. 

 

On-demand Antivirus Scanner Performance Comparison

By analogy with antivirus monitors testing described above, we also measured the time necessary for antiviruses to perform on-demand scanning of the user's files collection. In this case the initial system was not taken into account and antiviruses competed only with one another. Tables 5 and Diagram 6 represent the time of the first scanning for antiviruses tested. 

 

Table 5: File collection scanning time

Antivirus Scanning time [hour:min:sec]
Avira 0:01:28
Kaspersky 0:02:42
Norton 0:03:10
Trend Micro 0:03:10
Avast 0:03:30
BitDefender 0:04:03
F-Secure 0:04:10
Outpost 0:04:17
Sophos 0:04:53
AVG 0:05:00
Panda 0:05:27
McAfee 0:06:45
Eset 0:09:49
Microsoft 0:10:24
Dr.Web 0:16:51
VirusBlokAda 0:28:33

 

Diagram 6: The clean files collection scanning time

Antivirus test - the clean files collection scanning time

 

As we can see from Diagram 6, the performance of the first scanning for the same file collection by this or that antivirus can differ by dozens of times. Scanning time for the best antiviruses was from 1.5 up to 5 minutes whereas the time for the worst ones exceeded 10 minutes.

To determine which antiviruses use scanning optimization algorithm (for example, at the expense of missing the files that have been checked before), a testing file collection rescanning was performed as a supplement for the test.

As a result, they found out that the scanning time considerably reduced with seven antivirus products including BitDefender, F-Secure, Kaspersky, Norton, Outpost and VirusBlokAda (see Diagram 7). It obviously demonstrates that in real conditions the file scanning time on the computers using these antiviruses can be much less at the expense of optimization used. 

 

Diagram 7: The clean files collection rescanning time

Antivirus test - the clean files collection rescanning time

 

As a comparison, our previous test of that kind in 2008 showed optimization only with three antiviruses: Kaspersky, Outpost and VirusBlokAda. Thus, optimization in seven antivirus scanners now proves the positive tendency for performance optimization and checking technology development in antivirus industry on the whole.

An average memory and CPU usage during on-demand scanning start was measured in addition. This information in given in detailed testing report. In accordance with the award scheme for this test, the fastest antivirus scanners won special awards (see Table 6).

It is worth mentioning, that unlike the previous test, we now took into consideration both the first scanning performance and optimization availability for further iterations. 

 

Table 6: The fastest on-demand antivirus scanner

Antivirus Scanning time Rescanning time
Award
Avira 0:01:28 0:00:35 Platinum Performance Award On-Demand Scanning
Platinum
Performance Award
On-Demand Scanning
Kaspersky 0:02:42 0:00:06 Gold Performance Award On-Demand Scanning
Gold Performance Award
On-Demand Scanning
Norton 0:03:10 0:00:18
BitDefender 0:04:03 0:00:01
F-Secure 0:04:10 0:00:01
Outpost 0:04:17 0:00:25
Trend Micro 0:03:10 0:02:28 Silver Performance Award On-Demand Scanning
Silver Performance Award
On-Demand Scanning
Avast 0:03:30 0:03:27
Sophos 0:04:53 0:04:47
AVG 0:05:00 0:04:50
Panda 0:05:27 0:05:25
McAfee 0:06:45 0:06:40 Bronze Performance Award On-Demand Scanning
Bronze Performance Award
On-Demand Scanning
VirusBlokAda 0:28:33 0:03:03
Eset 0:09:49 0:09:46
Microsoft 0:10:24 0:10:27  
Dr.Web 0:16:51 0:16:35

 

Avira antivirus showed the fastest scanning performance. So, it was the only product that won Platinum Performance Award: On-Demand Scanning as it combines high scanning performance and rescanning optimization.

Kaspersky, Norton, BitDefender, F-Secure and Outpost also proved to be very fast and won Gold Performance Award: On-Demand Scanning. By the first scanning performance these antiviruses are almost as good as the leader. At the same time, they use powerful technologies for rescanning optimization.

Trend Micro, Avast, Sophos, AVG and Panda also proved to be fast enough and won Silver Performance Award: On-Demand Scanning as well as McAfee, VirusBlokAda and Eset that won Bronze Performance Award: On-Demand Scanning.

In spite of the fact that VirusBlokAda antivirus has much worse first scanning time as compared to the leader, its essential rescanning performance optimization technology allows it to win the bronze award in this part of the test in accordance with the scheme adopted.

 

Antivirus Performance Comparison for Office Applications

One more very important characteristic of antivirus performance is its influence on the work of different applications used by the user. We chose five among them: Internet Explorer, Microsoft Office Word, Microsoft Outlook, Adobe Acrobat Reader and Adobe Photoshop. All these programs are widely used and some of them are quite resource-intensive that is why an antivirus work influence can be quite noticeable.

In accordance with the testing methods, the boot time for every program in a system with installed antivirus products was measured. The results are given in seconds in Table 7. 

 

Table 7: Office applications start slowdown with installed antiviruses (%)

Antivirus Word IE8 Outlook Acrobat
Reader
Adobe
Photoshop
Avast 43 103 112 47 42
AVG 11 200 30 73 29
Avira 16 69 13 100 13
Dr.Web 521 8 80 73 11
Eset 6 30 16 122 11
F-Secure 1061 96 362 221 1278
Kaspersky 1108 175 198 48 2500
McAfee 56 94 76 48 56
Microsoft 55 16 54 93 30
Outpost 72 180 73 196 33
Panda 700 34 409 23 11
Sophos 27 131 57 172 11
Norton 1437 258 129 23 2744
Trend Micro 1218 780 70 98 1522
VBA 18 351 30 122 11
BitDefender 15 98 9 73 11

 

As we can see from Table 7, the starting time of the most office applications after installing an antivirus increases by the dozens of times (1000% of more). Diagrams 8-12 show the results of office applications start slowdown as compared to the system without any antivirus. 

 

Diagrams 8-12: The office applications start time in the system with an installed antivirus

The office applications start time in the system with an installed antivirus Microsoft Word

 

 The office applications start time in the system with an installed antivirus - Microsoft Internet Explorer

 

 The office applications start time in the system with an installed antivirus - Microsoft Microsoft Outlook

 

The office applications start time in the system with an installed antivirus - Adobe Acrobat Reader

 

 The office applications start time in the system with an installed antivirus - Adobe Photoshop

 

To determine the fastest antivirus working with the office applications, we used a 10-point scale where every antivirus received its grade for its influence on the start time of every office application mentioned. At that, the sample starting time for an office application was taken for 10 points and the worst test result was taken for 0 points. Thus, the best possible result is 50 (zero delay for all the five programs start).

The point total and awards received by antiviruses for this part of the test are shown in Table 8. Complete calculations and award criteria can be found in detailed testing report and award scheme description. 

 

Table 8: The fastest antiviruses for working with office applications

Antivirus Point total Point total (% of max.) Award
BitDefender 45 90% Gold Performance Award Office Software
Gold Performance Award
Office Software
Avira 44 88%
McAfee 44 88%
Microsoft 44 88%
Eset 44 88%
Avast 43 86%
AVG 43 86%
Dr.Web 41 82% Silver Performance Award Office Software
Silver Performance Award
Office Software
VBA 39 78%
Sophos 38 75%
Outpost 36 73% Bronze Performance Award Office Software
Bronze Performance Award
Office Software
Panda 34 67%
Kaspersky 24 48%  
Norton 22 45%
F-Secure 21 42%
Trend Micro 20 40%

 

No antivirus won Platinum Performance Award: Office Software in this part of the test as no antivirus passed a 95% limit.

BitDefender, Avira, McAfee, Microsoft, Eset, Avast and AVG showed insignificant office applications start slowdown and won Gold Performance Award: Office Software.

Dr.Web, VBA, Sophos, Outpost and Panda also showed good results and won Silver Performance Award: Office Software and Bronze Performance Award: Office Software respectively. Their effect over office applications loading is not significant.

It is worth mentioning that methodology for this part of the test does not take into consideration that some antiviruses will need much more time for the first program start with behavioral analysis. At that, further on programs can load without any delay as there is no need for additional analysis. For example, Kaspersky, Norton (Norton Insight technology) and Trend Micro (Smart Protection Network) can performance up significantly.

We would like you to pay attention that this test report misses many interesting figures and details that are of certain interest for antivirus experts.

 

Ilya Shabanov, Managing Partner at Anti-Malware Test Lab:

"Everyone uses PC for work, receiving some information, communicating or just for fun. Its basic aim as well as that of any other device is to be of good use. That is why the basic PC resources must be used for intended purpose that is for its protection and operation. Many users always complain that their antivirus is too "greedy", so it was very important for us to perform this test and show the state of things for different antiviruses performance and resource-intensiveness."

"I am very pleased to see that, by the results of this test, many vendors in fact, not in word consider the task of their products performance optimization to be of vital importance. They pay great attention to the scanning optimization (there are now seven antiviruses of that kind instead of only three ones in 2008), the size of the memory used and the performance of different product modules. Unfortunately, some leaders of the previous years relaxed a lot and lost their positions in productivity fight but I hope they will draw the right conclusions by the test results."

"I think, if the vendors pay due attention to this characteristic in the nearest 1-2 years, the market will have the choice of about ten personal antiviruses having the same performance and all the users will benefit from it." 

AttachmentSize
Performance_test2_results_eng.xls420.5 KB