Parental Controls Test (December 2012)

Nowadays parents and the government try hard to protect children from unwanted online content. That's why special filtration programs called “parental control software” are highly sought after. In this test we checked the effectiveness of popular programs filtering unwanted for children Internet sites. The results of this test should help parents choose the highest quality protection for their children surfing through the global network space.

 

Contents:
- Introduction
- List of Tested Products
- Test Platform Description
- Measuring Parental Control Product Effectiveness
- Final Test Results
- Change Analysis Comparison with the Previous Tests
- Test Comments
- Anti-Malware.ru Partner Comments

 

 

Introduction

All leading antivirus producers state that they have a parental control function in their multi-purpose products to protect home computers. Under the impact of the Russian Federation legislation lots of new companies opened up offering the market their own Internet filters with often questionable effectiveness.

In this third test we checked the real effectiveness of popular filtration programs sorting out unwanted for children Internet sites. The results of this test should help parents choose the highest quality protection for their children surfing through the global network space.

It's important to note that we didn't compare the product functionality in our test, that is, we didn't compare the presence or absence of any settings, functions, etc. We specifically checked the filter work effectiveness based on the supposition that a child has access to the Internet and «parental control» is set up in accordance with producer recommendations.

Filtration effectiveness was checked only in the category of «sites for adults» as the most massive and complicated one from the filtration point of view. However, the results of the test with certain assumptions can be used for other site categories as well.

 

List of Tested Products

We chose 15 products for this test, and these products include both specialized products and «parental control» service options, as well as products belonging to the Internet Security class, containing «Parental Control» modules.

1. Avira Internet Security 2013 (13.0.0.2832)
2. Bitdefender Internet Security 2013 (16.24.0.1684) /* disqualified */
3. ContentKeeper Express
4. Dr.Web Security Space 8.0 (8.0.0.11210)
5. Entensys KinderGate Parental Control 1.5.6371
6. F-Secure Internet Security 2013 (1.71)
7. Kaspersky Internet Security 2013 (13.0.1.4190)
8. McAfee Internet Security 11.6
9. NetPolice Child /* disqualified */
10. Norton Internet Security 2013 () + Norton Safety Minder /* disqualified */
11. Panda Internet Security 2013 (18.00.01)
12. SkyDNS
13. Trend Micro Titanium Maximum Security 2013 (6.0.1215)
14. Microsoft Windows Live Family Safety *
15. Internet Censor 2.2

 

We have to note right away that Bitdefender Internet Security, NetPolice and Norton Internet Security (Norton Safety Minder) fell out of the race and we never tested them.

The reasons for disqualification were the following: NetPolice and Norton Safety Minder services were found to be completely dysfunctional (due to errors, it's impossible to get on their website and download a client program), and Bitdefender dropped out of the test due to a critical error, not permitting to correctly determine the category of the site being blocked. That's why the data of these three products is considered disqualified.

Due to objective limitations we weren't able to test all possible Internet filters for children including specialized utilities. That's why the list has only the well-known products that are actively promoted in the Russian market.

 

Test Platform Description

The test was performed during October-November 2012 on a testing platform working on Microsoft Windows 7 x64 SP1 (Home Expanded) OS with all installed relevant updates. We used Microsoft Internet Explorer 9 as the standard browser for this OS.

Obviously, it's completely impossible to check the effectiveness of «parental control» products for all unwanted sites. That's why the most important task when preparing the test was choosing a representative set of links that would correctly reflect the effectiveness of the tested products' work overall.

According to the methodology we decided to limit ourselves in this test by checking «parental control» products on site blocking effectiveness for only porn and erotic items. For that purpose, based on a set of specifically chosen popular keywords from Yandex, Google and Bing search engines, we chose 800 links to «adult content» sites (400 in Russian and 400 in English). We are not publishing the exact list or search terms due to coarse language, and it's available only to representatives of the tested product producers upon request.   

 

Measuring Parental Control Product Effectiveness

To analyze the results we broke down the total mass of links into several categories. Consequently, the results of unwanted sites' filtration effectiveness based on the results found in different languages (Russian and English) in different search engines (Bing, Google and Yandex), are shown in tables 1-2 and pictures 1-4.

 

Table 1: The Results of Blocking Unwanted Sites based on Russian Search Requests

Russian	Bing		Google		Yandex		Average
Интернет Цензор	400	100%	397	99%	398	100%	99,6%
KinderGate	399	99,8%	395	99%	389	97%	98,6%
Dr.Web	390	98%	388	97%	394	99%	97,7%
Kaspersky	388	97%	378	95%	382	96%	95,7%
Avira	329	82%	361	90%	378	95%	89,0%
ContentKeeper	326	82%	352	88%	359	90%	86,4%
Panda	314	79%	352	88%	366	92%	86,0%
SkyDNS	319	80%	344	86%	358	90%	85,1%
F-Secure	316	79%	342	86%	355	89%	84,4%
Trend Micro	291	73%	354	89%	343	86%	82,3%
McAfee	294	74%	280	70%	311	78%	73,8%
Microsoft	266	67%	311	78%	290	73%	72,3%

 

Picture 1: The Results of Blocking Unwanted Sites based on Russian Search Requests for Different Search Engines

 

 

As you can see in picture 1, the effectiveness of unwanted sites' blocking in Russian language search requests got dependent on the search engine. The sites found through Bing search engine are not blocked as well as they were a year ago, when Google was less safe for children.

 

Picture 2: Average Results of Blocking Unwanted Sites in Russian Search Requests for all Search Engines

 

 

As you can see in picture 2, like a year ago, all, except for two of the tested products and services, showed high results, blocking more than 80% of sites found in Russian language search requests. Even the outsiders crossed the 70% line which has to be encouraging.

Russian producers are traditionally strong in blocking unwanted Russian sites for children. Internet Censor, KinderGate, Dr.Web Secure Space and Kaspersky Internet Security were able to show the blocking effectiveness at over 95%. The results of foreign competitors turned out to be considerably lower. As far as blocking unwanted sites, found by English language search requests, overall, everything looks a bit better.

 

Table 2: The results of Blocking Unwanted Sites Based on English Search Requests

English	Bing		Google		Yandex		Average
Интернет Цензор	398	99,5%	386	97%	396	99%	98,3%
Dr.Web	396	99%	390	98%	392	98%	98,2%
Kaspersky	395	99%	386	97%	393	98%	97,8%
KinderGate	392	98%	384	96%	393	98%	97,4%
ContentKeeper	384	96%	364	91%	376	94%	93,7%
Microsoft	375	94%	364	91%	370	93%	92,4%
SkyDNS	351	88%	362	91%	380	95%	91,1%
Avira	361	90%	356	89%	374	94%	90,9%
Trend Micro	360	90%	360	90%	358	90%	89,8%
Panda	351	88%	354	89%	373	93%	89,8%
F-Secure	351	88%	358	90%	366	92%	89,6%
McAfee	351	88%	343	86%	344	86%	86,5%

 

Picture 3: The Results of Blocking Unwanted Sites Based on English Search Requests for Different Search Engines

 

 

If we look at the breakdown by search engine in picture 3, then, overall, there is no significant difference in terms of the quality of blocking unwanted sites for children, found by English language search requests in different search engines.

 

Picture 4: Average Results of Blocking Unwanted Sites in English search requests for all search engines

 

 

As you can see from Table 2 and Picture 4, almost all tested «parental control» products were able to block 89% of unwanted sites, and only one of them showed results lower than this value. By the way, four products -   Internet Censor, Dr.Web, Kaspersky and KinderGate reached the result of 97%.

As per industry average, the results of blocking unwanted sites for children in English are a bit better than for the Russian language. The largest difference for the English language is noticeable on less effective products, they are substantially worse in dealing with unwanted site filtration in the Russian language.

 

Final Test Results

To conclude on the overall results for «parental control» product testing on blocking unwanted sites effectiveness it's necessary to additionally consider their false response rate. The thing is that the point of Internet-filtration technology is in blocking access to unwanted site categories and providing complete access to normal, legitimate sites.

That's why it's not enough to create an Internet filter that will block all 100% of unwanted sites for children. It's important at the same time to create access to sites with normal content. Otherwise, the disadvantages from using filtration will outweigh the advantages from it and it will just become impossible to use the Internet. This is exactly the principle that we considered in the system of awarding the best «parental control» products based on the results of this test.

Consequently, the overall results for the «parental control» product testing for unwanted site blocking effectiveness, irrespective of the language of search requests, as well as their false response rate, are shown in picture 4 and table 3. 

 

Picture 5: Final Test Results for Unwanted Site Blocking

 

 

Table 3: Final Test Results and Received Awards

Parental Control	Average	FP	Award
KinderGate Parental Control 1.5	98,0%	0,6%	Gold Parental Control Award
Kaspersky Internet Security 2013	96,8%	0,8%
ContentKeeper Express	90,0%	0,5%
Avira Internet Security 2013	90,0%	0,8%
SkyDNS	88,1%	0,9%	Silver Parental Control Award
Panda Internet Security 2013	87,9%	0,5%
Trend Micro Titanium Maximum Security 2013	86,1%	0,8%
F-Secure Internet Security 2013	87,0%	2,6%	Bronze Parental Control Award
Microsoft Windows Live Family Safety	82,3%	2,4%
McAfee Internet Security 11	80,1%	0,6%
Dr.Web Security Space 8.0	97,9%	5,1%	No award
Интернет Цензор 2.2	99,0%	73,0%
Bitdefender Internet Security 2013	-	-	Disqualified
Norton Internet Security 2013 + Norton Safety Minder	-	-
NetPolice	-	-

Best on filtration effectiveness for the testing collection was KinderGate Parental Control, that blocked in total 98% of unwanted sites for children. Kaspersky Internet Security came only a bit behind, having blocked 96.8%. Their false response rate didn't go over 0.8%. These two parental controls get the Gold Parental Control Award.

ContentKeeper Express and Avira Internet Security are also getting the Gold Parental Control Award, having blocked 90% of unwanted sites and at the same time their false response rate didn't exceed 0.8%.

The service SkyDNS, Panda Internet Security and Trend Micro Titanium Maximum showed very good effectiveness with the results from 86% to 89% and with low false response rate. They are receiving the Silver Parental Control Award.

F-Secure Internet Security, Microsoft Windows Live Family Safety and McAfee Internet Security, that blocked more than 80% of unwanted sites and that have not permitted more than 3% of false response are receiving the Bronze Parental Control Award.

The other «parental control» products, unfortunately, were left without awards. Two of them – Dr.Web Security Space and Internet Censor could have got an award but for the false response rate.

In Dr.Web Security Space case there was a high false response rate due to incorrect site categorization when a number of large sites (social networks, blogospheres, video hosting sites, etc) fell in the porn category and were completely blocked. Full blocking is a deliberate manufacturer's decision, however, despite the similarity with the approach first used in resolving the famous Gordian knot dilemma, this solution only puzzles the users.

Internet Censor, working on the white list principle, blocks the majority of legitimate Alexa.com world Top-1000 sites. Its false response rate was over 70%. In other words, your child will be disallowed to visit 7 out of 10 very reasonable sites for no legitimate reason. This approach cannot be called technologically mature at all. Obviously, the developers took the least costly solution. However, even when they were using the tough filtering approach based on white lists, they didn't manage to achieve the level of 100% filtration that they stated.

It turned to be a very unpleasant surprise this year that three tested products fell out of the race, and those were Bitdefender Internet Security, NetPolice and Norton Internet Security (Norton Safety Minder). These three products have been disqualified.

In the case of Bitdefender it was a program error that disturbed the testing and didn't permit to correctly determine the category for blocked sites. The services by NetPolice and Norton didn't work at all. Testers took a month of attempts trying to download installation packages for the test, and weren't able to do it, despite getting notices from the producers.

We seriously hope that the results of this test will positively impact «parental control» makers, and this important issue of protecting children online will not be neglected.

 

Change Analysis Comparison with the Previous Tests

In conclusion, we would like to analyze the results of «parental control» product effectiveness based on the total of two tests for 2011-2012. To get the total we add the results of this test to the previous test, done using the same basic methodology.

Therefore, we can see the changes in unwanted for children porn-related site blocking effectiveness (except for the products Bitdefender, Norton, NetPolice, and Internet Censor, as we don't have the data for 2 years for these products) — see pictures 6-7.

 

Picture 6: The dynamics of changes in «parental control» product effectiveness changes

 

 

Picture 7: The dynamics of parental control effectiveness changes

 

 

As you can see from pictures 6-7 over the past year, only some of the vendors were able to significantly improve the effectiveness of their unwanted for children content filtration technology. KinderGate, Dr.Web and Trend Micro showed positive changes. The rest stayed at the same level or showed worse results.

It's important to note good progress for many participants in terms of the drop in false response rate. Microsoft Windows Live Family Safety especially stands out here, whose number of false responses decreased twice.

If we look at the changes in the picture overall, there is no more equalizing of the results. The producers seriously involved in web filtration technologies and Internet site categorization keep a high quality standard and, if possible, improve the result. Others support the average effectiveness of their products, and don't try to follow the leaders that closely.

 

Test Comments

Ilya Shabanov, Anti-Malware.ru managing partner:

«Parental control test results this year evidence the starting product split based on the apparent filtration quality class and the false response rate. On the one hand, we notice a group of vendors who always improve the quality level and are striving to achieve the class of 99.9% protection keeping the false response rate close to zero. On the other hand, we see a certain level of relaxed attitude from other vendors who don't consider web filtration as the developmental priority, and who are happy enough with an average result and are not trying to become the leaders in this technological segment.

For the first time in the history of performing these tests we have come across the completely not working and unusable parental controls. This is a signal for me that as customers pay more attention to the child protection issue, the intention to provide fake parental controls for someone's personal money or governmental money will also grow. This kind of absence of service will not just permit unwanted content but simply won't work».

 

Mikhail Kasimov, Anti-Malware.ru expert:

«The products for testing can be divided into those that need some client installation software, and those that request changes in some personal provider access settings, or changing the DNS-server for an alternative one. I think that the second option is a more attractive option from the system load and use point of view. If we look at the overall results, the average false response rate for products is not above a few percent. At the same time, the main task of protecting children from harmful content is fulfilled by a half of products. If a child cannot open some good site, then it's not a big problem, the child will tell his parents about it and they will put the site on the exception list. False response in parental control is less critical than low level of detection of sites with dangerous content. Overall, the average detection of dangerous sites hasn't significantly changed, taking into account the fact that there are more and more sites like that appearing per unit of time, and the technologies of filter circumvention are becoming more and more technological. We can say for now that the use of «parental control» class products is justified.»

 

Valery Ledovskoy, Anti-Malware.ru expert:

«The issue of parental control is quite relevant now. Especially in connection with the latest legislation changes, connected with blocking of sites, containing the information capable of harming child psyche. Independent research showed that using the state register of prohibited sites is far from achieving the goal of protecting children from unwanted content at the moment. In particular, providers renew the list of blocked sites either late or incompletely. Therefore, some sites that have to be blocked are not blocked, and the sites that should have been unblocked a long time ago stay blocked. Providers block sites based on their IP addresses because they don't have the technical capability to block separate site pages. With that happening, personal products of «parental control» class, based on the results of the test supplied, look much more effective. Even the most ineffective parental control in this testing looks better than the effectiveness of using the state register of blocked sites. More than that, some providers give the parental control functionality based on the SaaS model for free or for a nominal payment. And families that have children can use it. I think that this test is not just information for the user, who wants to protect his children from the unwanted content on his computer. This is very important information for people who make decisions, that impact the used methods of protecting children from the unwanted content online on the country-wide scale. You can learn a lot from some of the products shown here».

 

Anti-Malware.ru Partner Comments

Dmitry Kurashev, Entensys Company Director:

«First of all I would like to thank Anti-Malware.ru resource just for performing this test. This is very helpful for us, producers of filtration systems. We constantly improve the accuracy and neatness of how our product works, and these tests help in that, by all means. I feel very proud that our KinderGate showed the best result among all the participating products, though, of course, it's sad that we missed the threshold for the platinum award by 0.07% for false response, and that award for «parental control» testing product has never been given to anyone before».

 

Oleg Andrianov, senior certification manager, Kaspersky Lab:

«The issue of unwanted content protection quality is extremely important and we are glad that the information-analytical center Anti-Malware.ru regularly performs such research. It's especially important for the users of this component to choose products with high quality protection. High results of finding and blocking unwanted content shown by our product in this test clearly state the result of our efforts to develop parental control technologies».

 

Vladimir Doodchenko, SoftBCom company General Director; SoftBCom distributes ContentKeeper products in Russia:

«The published test results show the culmination of unique in terms of its content work. This work has been performed for three years now completely free, and the experts from Anti-Malware.ru volunteer to do it. The provided data and the used methods and practices in this situation are especially valuable, and that is happening when many decisions, even on the state level, are made without the necessary scientific and technological work done beforehand, like we say «without giving it a second thought». The tests performed showed depth and multifacetedness of the filtration theme, though they have encompassed only one «pornographical» component of this problem.

ContentKeeper Express cloud product that was part of the test is one of the ContentKeeper product line components, and it's able to satisfy the needs of all user classes — from those working on separate computers at home or in small organizations (for example, schools or libraries), to medium and large structures. Internet provider traffic filtration is also within the sphere of ContentKeeper use.

The test results have clearly proven that Russian web-zone is always within ContentKeeper developers' and support service attention zone». 

 

Dmitry Vostretsov, SkyDNS Marketing Director:

«I am very sad to see these results, but our system is set up based on a two-level model using our own system of safe search that makes the filter up to 100% reliable. This is proven by multiple public prosecution office checks in schools using our filter».