I need to start this post with an important disclosure. I was on the Board of ComScore for nine years from the summer of 1999 until earlier this month. And I am still a shareholder in ComScore through the Flatiron partnerships. In no way is this post intended as a plug for ComScore. And I cannot and will not disclose any confidential information about ComScore in this post.
Now that I have that on the table, I’d like to share my impressions after a first look at Google’s new Trends For Websites service. I’ve been involved in the web measurement market for almost ten years and we use third party measurement services every day in our job as investors in Internet companies. I’ve blogged about our approach (USV’s) to web measurement in the past.
In my opinion, there is no service that gets third party web traffic measurement exactly right. We like to triangulate at Union Square Ventures. We use ComScore actively and believe their data is the most accurate of all the services. But it is not perfect and it is particularly weak at measuring web sites with small (sub 500k uniques) audiences. We also use Compete, Alexa, and Quantcast on almost a daily basis. And we will most certainly start using Google’s Trends For Websites.
There are a lot of misconceptions about third party measurement services out there. They are on display in the comments in this Techcrunch post. People say things like "ComScore’s data is not worldwide" or "Compete doesn’t have good statistical algorithms." Neither statement is true. ComScore, Compete, Hitwise, and NetRatings have been around for a long time and have invested heavily in the statistical talent that is required to get a third party measurement service right. And certainly a bunch of the leading services, particularly ComScore and NetRatings have built large international panels and report international data.
Google’s service reminds me more of Alexa than anything else. And I don’t think it’s particularly accurate. Here’s an example. We run monthly queries of all the third party services against our entire portfolio and share that data internally in our firm. I am very familiar with the numbers on our portfolio companies, both what their own analytics and server logs are telling them and what the third party services tell them.
Here are three charts of Twitter, Etsy, and Indeed. I picked these three services because they have large enough audiences which makes them fairly easy to measure by third parties and have been around a while and are well known.
This is what ComScore says the worldwide unique visitor trends have been over the past year:
This is what Compete says:
And this is what Google Trends says:
Since we are investors in these three companies and know what their internal numbers are saying, I can safely say that ComScore and Compete are lower but directionally correct. Google is like Alexa in that they don’t report absolute numbers but even if they did, they are not directionally correct on this particular set of companies.
Once again, this story demonstrates that many in the tech blog world don’t really understand the third party measurement market. Google announces that they are offering a website traffic measurement service and everyone instantly assumes it is going to be great. I don’t think it is, at least not yet.
Mike Arrington points out in this post that,
all the data is being gathered by Google for the product is from Google
users (their toolbar, for example), the data for Google’s sites would
be skewed to 100% of all Internet users. It points out an inherent flaw
in the product, and I’m not sure Google can easily solve it.
This is an important fact. Everyone who provides third party measurement starts out with a data set that is skewed in some way. The trick is to understand how your data set is skewed and apply statistics to take that bias out. Firms like ComScore, Compete, Hitwise, and NetRatings who sell their data have invested heavily for many years in reconciling their data to server logs and internal analytics. And that makes their data better.
If Google wants to be a player in this market, they’ll have to invest time and energy doing the same. It will be interesting to see if they will do that. If not, they’ll be more like Alexa. A helpful tool but not one that you can completely rely on.