Number of URL’s In An Internet Filter Database: Does It Matter?
Recently I noticed a post on David Burt’s blog, filteringfacts.org, that NetSweeper had told him they had exceeded 1 billion URL’s in thier database. They estimate that they will be to the second billion within 6 months.
I’m often asked in interviews “How big is your database”. What people are really trying to ask is “How effective is your filter.” URL count, simply can’t be used as the only method of determining if an Internet Filter is effective. Here is an example why:
http://www.internetsafety.com/
http://www.internetsafety.com/safe-eyes-parental-control-software.php
http://www.internetsafety.com/ethershield-internet-filter.php
http://www.internetsafety.com/internet-safety-tips-for-parents.php
There are four URL’s. But from an Internet Filtering point of view there probably isn’t a use case where I really want to treat one of those pages differently than the other. So one company could have the 4 URL’s listed above in their database, and another company could have just internetsafety.com in their database and inherit the categories down to the other URL’s. Both databases are just as effective, the extra URL’s are just database bloat.
So how can you know if your Internet Filter is effective enough? Again David Burt has compiled a really comprehensive list of Internet Filtering effective tests that have been performed.
However, after spending nearly 10 years in the Internet Filtering industry I have found that main stream filtering providers are only going to be separated by 1 or 2% difference in filtering effectiveness. That just is not going to make a really big difference in the blocking capabilities for your organization.
If you are looking at products from InternetSafety.com, Secure Computing, 8E6 Technologies, or Websense you are better off making your purchase decision on other functionality in the solution, support options, or pricing rather than how many URL’s are in the database.
Aaron Kenny
**Full Disclosure: This comment is from a NetSweeper employee.
Hello.
Great points about URL Filtering.
Netsweeper has an extreme advantage over the other Filters where not only do they have Premium Granular, per user Functionality, reporting support options, security and a low cost provider, Netsweeper’s Architecture and how they update New URL’s is more important than the data base size.
When a new URL is not known, Netsweeper will update the world in 1-2 seconds or less. Netsweeper only provides categorization on where customers have visited therefore they only categorize the parts of the Internet that are important to the users.
With Netsweeper’s scanning and AI Technologies Netsweeper also maintains this dynamic data base on a regular basis. Unlike the other list based filters, Netsweeper does not have to chase URL’s then maintain a huge data base. The Netsweeper technology and architecture is designed and proven for today and tomorrows growth of the Internet.