Flaky?

From Michael Arrington’s post on Digg acquisition rumors:

One point of controversy was around Digg’s claim of 20 million unique
monthly visitors and steep monthly growth, whereas the Comscore’s most
recent September report shows only 1.3 million monthly unique visitors
and flat growth since April (see chart below). Comscore is notoriously
flaky, and these numbers are for U.S. households only. Comscore is
almost certainly significantly under-reporting Digg traffic.

Michael is one of the best bloggers ever and I read Techcrunch every day. But I think he got this one wrong. Comscore is not "flaky". They are a third party measurement service. They don’t always get everything right. None of the third party measurement services do. But they are the best of the lot in my opinion. Now I am biased as I have been an investor in Comscore since 1999 and have been on the board since then.

I do not think Wall Street, Madison Avenue, and all of the major Internet companies would spend millions of dollars a year on data that is "flaky". Comscore’s data is always being reconciled, debated, improved, dicussed, and analyzed and it gets better and better every year.

Now let’s talk about Digg’s Comscore numbers.

1 – Comscore is a "consumer panel". It measures mainstream web users. It is not a "leading edge" panel and it will almost certainly undercount "geek" services like Delicious and Digg. But it won’t be off by 20x. It probably won’t even be off by 2x.

2 – Comscore counts real viewers in its panel, not cookies. Cookies get deleted by spyware removal software. If you remove your cookies once a week, you’ll look like four users every month to someone using cookies as a basis for UVs. The more sophisicated a user base is, the more likely they use cookie removal. And that results in significant UV overcounting.

3 – If you visit a service from multiple computers, you will be counted as mutiple users by most analytics programs. I suspect a decent subset of Digg’s user base does that.

4 – Comscore has a US panel and a International panel. The 1.3mm monthly uniques is US data. Comscore’s worldwide number for Digg’s UVs in September is 3.1mm.

So let’s look at Digg’s claim that they have 20 million UVs. Do you believe that? As Bryce said in his comments on Techcrunch:

Wow, if 20 million visitors is true – that’s a lot – the entire population of Australia.

YouTube had 20mm unique visitors in September. Do you think that Digg has as many uniques as YouTube? I don’t.

My guess is that Digg has something like 5mm monthly unique visitors worldwide. Not 20mm. The difference probably results from cookie counting, multiple browsers, and a few other factors.

And I’d like to encourage everyone out there to sit down and understand third party measurement services before calling them "flaky". My bet is they are more accurate than internal analytics numbers a lot of the time.