Overcounting Web Audiences


  Tossed Cookies 
  Originally uploaded by LeggNet.

When you hear a web site’s audience size, it’s most often stated in unique visitors. And those unique visitor counts are usually generated by analytics packages that use cookies to determine unique visitors (browsers). There are other methods (beacons and ip addresses) but those methods are not as popular as cookies because they suffer from a number of problems.

Well today comScore released the results of a study on the role of cookie deletion in web audience reporting that they’ve been working on for months. And they determined that using cookies to determine web audience will on average overcount your audience by 2.5x.  That’s right – 150% higher than it actually is.

And it’s worse if you have an audience of hard core cookie deleters. Seven percent of the web audience (as measured by comScore’s panel), deletes cookies more often than four times a month and on average they will be overcounted by 12.5x a month.

It’s been a rule of thumb that third party cookies (set by ad servers and ad networks) are the primary target of cookie deletion software so if you set first party cookies, you don’t have to worry so much. Wrong. It turns out cookie deleters make almost no distinction between third party cookies and first party cookies and delete them with about equal frequency.

I got a preview of comScore’s press release and wrote this post on Sunday because I am flying most of the day on Monday. I’ll link to it when I land but it should be linked to on this page. It’s a good release, filled with lots of numbers and quotes from academics, ad agency execs, and industry insiders who all agree that this is important research.

It proves something I’ve always said. You cannot rely on your own analytics data. You need third party data as well. That’s not to say that third party data (primarily panel data) is perfect either. You have to triangulate between all the numbers to get a decent view of what’s actually going on.

People often say that comScore’s numbers (and it’s competitors numbers too) are too low. It’s true that  panel data is generally a lot lower than your own server logs. But that doesn’t mean your server logs are right.