Methodology for Antivirus Performance Test (February 2010)


This testing methodology is the result of co-work and an intellectual property of an expert community Our forum offered an open discussion of this test methodology long time before the test and everyone who wanted could bring in his or her own suggestions concerning the future test.

The following antivirus programs took part in the test:

  • Avast Antivirus Professional 4.8.1368
  • AVG Anti-Virus & Anti-Spyware 9.0.716
  • Avira Antivir Preminum
  • BitDefender Anti-Virus 2010 (
  • Dr.Web
  • Eset Nod32 4.0.4680
  • F-Secure Anti-Virus 2010 (10.10 build 246)
  • Kaspersky Anti-Virus 2010 ( (a,b))
  • McAfee VirusScan Plus 2010 (13.15.113)
  • Microsoft Security Essentials 1.0.1611.0
  • Norton Anti-Virus 2010 (
  • Outpost Antivirus Pro 2009 (
  • Panda Antivirus 2010 (9.01.00)
  • Sophos Anti-Virus 9.0.0
  • Trend Micro Antivirus plus Antispyware 2010 (17.50.1366)
  • VBA32 WinNT Workstation

NB! Antivirus programs were updated after installation till they were actual for the moment of performing the test. The system was forcedly restarted and update function was disabled after that to avoid the influence on measurement results.

Also, all the scheduled scanning tasks were disabled to avoid the test results misrepresentation. If it was impossible, for example in case of autorun files checking, then a special technical pause was done to shutdown (controlled through Process Explorer) right before the test measurements.


Test environment configuration

The test environment was prepared before the beginning of the test. To do that, Microsoft Windows XP Pro Rus SP3 was installed on the clean computer with all updates available for that moment as well as additional software necessary for that test.

The basic characteristics of the computer where the test was performed (see Table 1) as well as the software installed (see Table 2) are given below.

Table 1: The test platform description

Processor Intel Core 2 Duo E6550 2.33 ГГц
Motherboard MSI P35 Neo-F
Videocard GeForce 8800 GTS 640MB


2048 MB
Hard drive 80 GB (ST380013AS)

100 Мбит/сек Ethernet

Table 2: The list of software installed

Operation system Microsoft Windows XP Pro Rus SP3 with all updates available for the test moment
Software installed

DirectX 9c
Internet Explorer 8.0.6001.18702
Microsoft Office Word 2003 (11.8307.8221) SP3
Microsoft Outlook 6 ( SP3
Adobe Acrobat Reader 9.2.0
Adobe Photoshop CS4 Extended

Performance Test 6.1
Windows Boot-Timer
Process Explorer

Operation system configuration You can see the full list of services on the test computer and their status here.


After operation system and all the necessary software have been installed, the following actions were undertaken:

  1. Disabling all operation system functions that can influence the test results (update, screen saver, Windows Security Notification, Windows Firewall, Windows Defender, Prefetching).
  2. Setting Windows Boot-Timer to activate with the system restart.
  3. Creating the system snapshot with Acronis True Image.
  4. Forming the collection of clean files of different type and size (Windows system files, installed program files, distribution files, documents, media-files and archives).

Thus, the source system spanshot was created and used as an initial one.

The license for Acronis True Image was provided specially for this test by Aflex Software, Acronis, Parallels, ASPLinux representative in Russia and CIS.

Windows Boot-Timer is a special tool for measuring the Widows start up time. The program starts measuring the time after BIOS initialization and till the system loading is completed (with all automatic start up processes).

AppTimer is a special PassMark Software tool for measuring applications start up time. This tool was used with the default settings.


Initial system performance measurement (without installing an antivirus)

To calculate an antivirus system slowdown it is necessary to measure all the initial system characteristics. The following things were done for that purpose:

  1. Restoring the system for the initial snapshot.
  2. Writing the system boot-time (using Windows Boot-Timer).
  3. Measuring the files copying time (using a special BAT-file).
  4. Measuring five office applications boot-time (Microsoft Internet Explorer, Microsoft Office Word, Microsoft Outlook, Adobe Acrobat Reader, Adobe Photoshop).
  5. Returning to Step 1 – repeating all the measurements (5 cycles).

NB! All the performance and resource-intensiveness characteristics were measured 5 times and the results were averaged less the limit values. At the same time, the system was restored from the initial snapshot with any iteration.


Measuring the boot-time for the system with antivirus software

The system boot-time is one of the most important system characteristics. To measure the system boot-time we used Windows Boot-Timer tool allowing to measure the time from the BIOS initialization finish to the complete system boot (with all automatic start-up processes).

Every antivirus was installed into a separate snapshot of the initial system that provided absolutely the same conditions for every tested product.

After each antivirus installation, update and several technical restarts, the system image was fixed (saved another snapshots) that allowed to save a lot of testing time.


The measurement algorithm for this part of the test is as follows:

  1. Restoring the system from the initial snapshot.
  2. Antivirus installation and the system restart.
  3. Antivirus update.
  4. Writing the default settings into a summary table (checking the file types, installed modules, etc.).
  5. System restart.
  6. Disabling the antivirus automatic update other scheduled default tasks.
  7. Windows Boot-Timer launch (system restart delay).
  8. Creating the image for the system with installed antivirus software using Acronis True Image (further on mentioned as "antivirus image").
  9. Restoring the system from the new image (Step 8).
  10. Writing the boot-time (using Windows Boot-Timer).
  11. Returning to Step 9 – repeating all the changes (5 cycles).

NB! The system boot-time was measured 5 times and the system was restored to its initial snapshot with an installed antivirus every time before the measurements (returning to Step 9). The results were averaged less the limit values.


Measuring of on-access and on-demand scanners performance and an antivirus influence on the office applications boot-time

At this stage of the test, we checked on-access monitor influence on the file collection copying (clean files folder) from one drive into another. Besides, we checked the time spent on on-demand scanner checking of the same file collection. See the files characteristics in Supplement 1.

For estimating the antivirus software influence on the office applications work, we measured the boot-time of the five top applications (AppTimer tool was used):

  1. Microsoft Internet Explorer 8.0.6001.18702;
  2. Microsoft Word 2003 (11.8307.8221);
  3. Microsoft Outlook 6 (;
  4. Adobe Acrobat Reader 9.2.0;
  5. Adobe Photoshop CS4 Extended.

To receive the full and objective picture, we measured CPU and memory utilization at system idling and during the on-demand test scanning (controlled by Process Explorer).


The following characteristics were measured in this part of the test:

  1. Free idle system resources (5 cycles for each antivirus).
  2. Test files copying time, time measurements (5 cycles for each antivirus). System restoring after every scanning.
  3. On-demand test files scanning (5 cycles for each antivirus). System restoring after every scanning.
  4. Free system resources during every testing copying and scanning (5 cycles for each antivirus).
  5. Test files rescanning time for determining the efficiency of checking optimization technologies used in antivirus products. Rescanning is performed only after the system restart when the first scanning has been completed (1 cycle for each antivirus).
  6. Office applications boot-time (5 cycles for each antivirus). System restoring after every set.


The algorithm for this part of the test is as follows:

  1. System restoring form the image with antivirus.
  2. Idle free system resources measurement.
  3. Measuring test files copying time (using a special BAT-file) and free resources.
  4. Returning to Step 1 – repeating all the measurements (5 cycles).
  5. System backup from the image with antivirus.
  6. Measuring test files scanning time and free system resources.
  7. * Single measurement of the files re-scanning time.
  8. Measurement of five office applications boot-time (Microsoft Internet Explorer, Microsoft Office Word, Microsoft Outlook, Adobe Acrobat Reader, Adobe Photoshop).
  9. Returning to Step 5 – repeating all the measurements (5 cycles).


NB! The measurements of test files copying and scanning time, office applications boot-time and idle resources were performed 5 times and every time the system was restored to its primary image with an antivirus installed (returning to Step 1 or Step 5). The results were averaged less the limit values.


Test results processing

All the test data needed special processing that can be divided into several stages:

  1. Filtration and averaging of the results of five measuring iterations (the function was chosen when two limit values were not taken into consideration and the remaining three were averaged).
  2. Assessment of the system slowdown by each parameter depending on the antivirus software and as compared to the initial system.
  3. Results normalization in percentage as compared to the initial system values.
  4. Compiling the antivirus performance rating for key characteristics.
  5. Awarding the best antiviruses.


Supplement 1: File collection

The collection included the files of the following type and size: Windows system files, installed program files, distributives, documents and, media-files. Total: 5031 files of 67 types (by extension); the total collection size is 5.7 Gb.



Состав файловой коллекции


Extention Size % Files
.dll 2364870,3 КБ 41,2% 2888
.exe 1409433,7 КБ 24,5% 939
.djvu 401891,4 КБ 7,0% 37
.sys 288746,5 КБ 5,0% 458
.pdf 259261,2 КБ 4,5% 13
.mp3 247349,2 КБ 4,3% 31
.jpg 199124,0 КБ 3,5% 59
.avi 156655,8 КБ 2,7% 1
.bak 112555,3 КБ 2,0% 47
.cpl 30567,5 КБ 0,5% 55
.qts 30319,0 КБ 0,5% 3
.bpl 28990,5 КБ 0,5% 80
.pif 26675,5 КБ 0,5% 3
.pptx 20986,0 КБ 0,4% 15
.ocx 20329,8 КБ 0,4% 28
.ax 18127,1 КБ 0,3% 20
.ppt 14794,0 КБ 0,3% 4
.api 14621,0 КБ 0,3% 5
.msi 13133,5 КБ 0,2% 2
.rar 11278,7 КБ 0,2% 2
.qtx 10192,0 КБ 0,2% 15
.doc 9901,7 КБ 0,2% 11
.htm 9490,5 КБ 0,2% 11
.kdl 8382,5 КБ 0,1% 21
.ppl 7985,8 КБ 0,1% 50
.tbp 4047,5 КБ 0,1% 2
.so 2740,8 КБ 0,0% 74
.docx 2702,8 КБ 0,0% 1
.setup 2472,0 КБ 0,0% 1
.apl 2194,5 КБ 0,0% 1
.pyd 2023,5 КБ 0,0% 7
.cav 1879,2 КБ 0,0% 12
.fmt 1047,5 КБ 0,0% 15
.rsc 1009,0 КБ 0,0% 4
.nls 762,6 КБ 0,0% 2
.scr 715,9 КБ 0,0% 3
.js 706,7 КБ 0,0% 37
.plugin 664,0 КБ 0,0% 5
.cab 628,8 КБ 0,0% 1
.fb2 604,6 КБ 0,0% 1
.ngr 604,0 КБ 0,0% 1
.acm 587,8 КБ 0,0% 9
.0 535,1 КБ 0,0% 2
.jar 508,0 КБ 0,0% 1
.rus 396,0 КБ 0,0% 4
.bin 396,0 КБ 0,0% 1
.crl 320,0 КБ 0,0% 1
.tsp 225,0 КБ 0,0% 1
.wcx 189,5 КБ 0,0% 2
.rll 152,0 КБ 0,0% 3
.wfx 136,0 КБ 0,0% 1
.nlr 104,0 КБ 0,0% 1
.mui 64,0 КБ 0,0% 3
.com 57,0 КБ 0,0% 3
.msc 40,2 КБ 0,0% 1
(no extention) 24,0 КБ 0,0% 3
.ico 22,0 КБ 0,0% 1
.wlx 20,5 КБ 0,0% 1
.xml 20,4 КБ 0,0% 1
.loc 13,3 КБ 0,0% 1
.vbs 3,4 КБ 0,0% 1
.bat 1,7 КБ 0,0% 6
.atr 0,4 КБ 0,0% 8
.lnk 0,3 КБ 0,0% 1
.sal 0,2 КБ 0,0% 1
.idx 0,2 КБ 0,0% 8
.m3u 0,1 КБ 0,0% 1
Total: 67 5744283 100 5031