Home » All tests » Tests Methodologies

Personal Backup Software Self-Protection Testing Methodology (January 2017)

Submitted by Ilya Shabanov on Wed, 01/18/2017 - 10:52

We tested 7 most popular personal backup products in their most recent versions issued by the date of the test (December 20, 2017) for Windows 7 x86. The list included:

  1. Acronis True Image 2017 New Generation 20.0.0.6106
  2. Carbonite Home 6.2.16804
  3. CrashPlan Free 4.8.0.331
  4. EaseUS ToDo Backup Free 10.0.0.0 build 20161212
  5. iDrive 6.5.1.22
  6. Macrium Reflect Home Edition 6.3 build 1665
  7. NovaBackup PC 18.5 build 926

Testing was conducted on a specially prepared test stand running under VirtualBox 5.1.12. A “clean” Windows 7 Home Premium x32 SP1 (6.1.7601) virtual machine was cloned for each backup product. VMware Workstation ran under Windows 10 (10.0.14393 x64) as well.

We used the default settings recommended by each product vendor when installing the backup software. All of the actions recommended by the installation programs (e.g., system restart, updating, etc.) were performed. All protection components that were not automatically enabled after installation were enabled manually.

Testing of backup product self-protection capabilities included the following parameters:

  1. Protection of the backup product's own files:
    1. Modules modification/removal;
    2. Removal upon reboot.
  2. Protection of the backup product’s registry keys:
    1. Modification/deletion of important registry keys (manually):
      • Startup keys;
      • Service keys;
      • Configuration keys.
  3. Protection of the backup product’s processes:
    1. Prevention of process termination:
      • From the TaskManager;
      • User-level API:
        1. Getting the handle of the process and using ZwTerminateProcess;
        2. Getting a list of all process threads, getting the handle of the process and use of ZwTerminateThread;
        3. Getting the handle of the process and use of ZwterminateJobObject;
        4. Getting the handle of the process and use of ZwDEbugActiveProcess;
        5. Using WinStationTerminateProcess.
      • System messages (SendMessage API):
        1. Getting process main window and sending WM_CLOSE using SendMessage, PostMessage, SendMessageCallback, SendNotifyMessage, PostThreadMessage, SendMessageTimeout;
        2. Getting process main window and sending WM_QUIT using SendMessage, PostMessage, SendMessageCallback, SendNotifyMessage, PostThreadMessage, SendMessageTimeout;
        3. Getting process main window and sending WM_SYSCOMMAND (SC_CLOSE) using SendMessage, PostMessage, SendMessageCallback, SendNotifyMessage, PostThreadMessage, SendMessageTimeout;
        4. Getting all process windows and sending all possible window events in the loop using SendMessage and PostMessage.
    2. Process/code modification:
      • Code injection (CreateRemoteThread):
        1. Getting the handle of the process and using ZwCreateThread with ExitProcess address.
      • Code injection (Set New Thread Context):
        1. Getting a list of all process threads, getting the handle of the process and use of ZwSetContextThread with ExitProcess address;
        2. Getting a list of all process threads, getting the handle of the process and use of ZwQueueApcThread/ZwQueueApcThreadEx with the ExitProcess address.
      • DLL injection;
        1. Getting a list of all process threads and injection of DLL closing the process using SetWindowsHookEx;
        2. Getting a list of all process threads and injection of DLL closing the process using SetWinEventHook.
      • Memory Attributes Modification:
        1. Getting the handle of the process and setting PAGE_NOACCESS attribute to every  available memory unit using ZeProtectVirtualMemory;
        2. Getting the handle of the process deallocation all available memory using ZwFreeVirtualMemory;
        3. Getting the handle of the process and unmap all mapped objects using ZwUnmapViewOfSection;
        4. Getting the handle of the process and allocation of all available memory using ZwAllocateVirtualMemory;
        5. Getting the handle of the process and allocation of all available memory using ZwMapViewOfSection.
      • Writing in process memory (WriteProcessMemory):
        1. Getting the handle of the process and override available memory using ZwWriteVirtualMemory. 
      • Modification of process objects:
        1. Getting the handle of the process and deallocation of all handles of the process using ZwDuplicateObject;
        2. Getting the handle of the process and allocation of all free handles of the process using ZwDuplicateObject.
  4. Cloud backups protection:
    1. External manipulation using command-line tools.
    2. Internal manipulation using embedded in the main process of backup tool code.
  5. Data recovery from the cloud to HDD:
    1. Remote backup server addresses manipulation using the hosts file modifications.
    2. Full disk encryption by ransomware, that modifying the MBR.

PLEASE NOTE: all tests were performed in User Mode. We did not test the kernel-level self-protection capabilities of the products as there is no use in such test in this particular case.

Testing of the self-protection capabilities was conducted manually or using specially developed utilities that imitate attacks. After each attack, the correct operation of the backup software (including individual modules, active processes, services and drivers) was verified.

If one of the processes was terminated during processes termination/modification test (ie the attack was successful), then all the other processes have been attacked again.

The testing process included the following steps:

  1. Installation of a backup software program on a clean virtual machine.
  2. System restart.
  3. Verification of the successful installation and correct operation of all modules.
  4. Saving an image of the virtual machine.
  5. Testing of one self-protection parameter.
  6. Verification of the correct operation of the program’s modules.
  7. Rollback to the saved image (step 4)

A separate clean virtual machine was used for each backup program (step 1). After testing the backup product’s self-protection capability based on one of the criteria, the virtual machine was rolled back to its original state after product installation (step 4).