Audience Measurement Across Web and Mobile
With leading web properties like Facebook and Twitter now seeing up to half of their traffic on mobile, it's hard to get a sense of what is going on with their audiences by looking at Alexa, comScore, Quantcast, etc. The same is true of our portfolio companies and their competitors. Gone are the days when we could pull up several of these third party measurement services, put in a few URLs, do a little triangulation and see how our companies are doing against the competition.
Today its a hodgepodge of web audience measurement (done the way I described above) and app audience measurement. App audience measurement is tougher. Services like appannie will tell where an app ranks in the app store based on download activity. Services like appdata will tell you how many facebook logged in sessions a mobile app and web app get but it doesn't break it down between web and mobile. And some popular mobile apps don't support login with Facebook so they don't even show up in appdata (our portfolio company Kik is an example of one).
As far as I know, there isn't a consolidated audience measurement services across web and mobile that can tell you how an app is doing in the absolute and against its peers/competitors. There should be.
comScore is in a good position to provide such a service. I used to be an investor and board member of comScore so I know the company well. I still have a small position in comScore stock. The recently launched Mobile Metrix 2.o is the analog to comScore's market leading Media Metrix web measurement product. If they combined the data into a single audience measurement product, that would be what I am looking for.
There are some other potential providers of a service like this. Quantcast could team up with Flurry (a USV portfolio company) and provide self reported data on a combined basis for companies that run Quantcast and Flurry on their web and mobile apps. But that would not be a comprehensive data set.
It's not easy to deliver a third party measurement service across web and mobile. If it were, it would have been done by now. But we neeed one, badly. I hope someone brings it to market soon.
Comments (Archived):
Yes – it’s a big deal. Not least because we need data to distinguish between the nature of the mobile vs web usage. And imho we really need even more detailed information. I would like to know the nature of usage on large smartphones such as the Galaxy Note vs say the iphone. Imho we are very early in the process of understanding the impact of such different form factors on usage and I suspect they may be more different than a gross measure such as ‘mobile’ suggests.
For checking app sales their’s MopApp. A friend helped build that up.
Sounds like a great problem to solve. Lots of technology, data, analytics, integration and UI challenges to pull together. I can’t say I’ve been very impressed with comScore because they seem behind the technology curve. You’ve put a nice carrot out there.
I am no computer geek … honestly… i don’t understand how web and mobile are different… mobile also access the web right … with similar OS accessing different URL ? Any link that i can read to understand better?Now i understand why i should not be commenting here at AVC 🙂
Knowing where your traffic is coming from is an important data point, if you put all of that into one Web bucket it does not give you the same insight as what most of us want to see. We would like to know how many of the users are coming from Mobile vs Desktop/Laptop vs Tablet, granularity is the word. Within each cohort of channels/devices, what platforms, region, browser type, OS, OS version, gender, demography, preferences, likes/dislike (I wish) are all important data points. I know in this age of big data more is better with tools that can synthesize the data and give you insight is the pain. We all live and die by metrics and I hope someone brings it all together so we get an overview of how this thing called the internet is evolving. I guess Fred is just drawing attention to the fact that we need a service that does that. The internet is evolving faster than we can measure the evolution that is the opportunity
thanx.Then for mobile IMHO as @misteroo:disqus says the data is with the carriers.
Yes and no, there are ways for third parties to get the information. The carriers have it but they probably are not really using the information as they should be. Carriers are fat cats they don’t have any incentive to innovate they would like to stop innovation so they can seek more and more rent for their capital outlay… exactly why we need to make it easy for new entrants to compete on even terms with the carriers
the web is the web.there is no ‘mobile web’, in the same way there is no ‘desktop web’the web just gets accessed on different devices. under different contexts.but because of the different devices–which in tern have different limitations and functionality–certain design choices are made.it’s often a mobile site (an ‘m dot’ site E.g. m.twitter.com), responsive design, or just letting the mobile browser deal with a fixed width (designed for the desktop) siteall of the above are made with web technologies (html, http, javascript etc).and then there’s ‘native applications’, that run on mobile. they run on the handset with ‘native code’.it starts to get more complex as different technologies overlap. i don’t think there is one link that explains it all. it’s each technology, how they are used, how they interoperate, and how they are evolving.
thanx @chrishuntis:disqus and @BalaInIceland:disqus …for spending your time explaining things.Yes. that is what i had in mind “there is no ‘mobile web’, in the same way there is no ‘desktop web'” … and could not comprehend “web and mobile” as the title says.P.S. I never knew there is ‘m dot’ exist … and I assume they are just the ‘dot’ with UI re-sized for mobile screens.
there may not be conceptually, but from an audience measurement POV and an AD dollar POV there definitely is.I was at the Coke Mobility summit two weeks ago — the $$$ going to mobile vs. the people who are on it is totally out of whack.
I’m interested…Can you expand on this?
Rightly or wrongly it’s about the way traditional media agencies plan and purchase. They are still very heavily focused on GRPs and linear broadcast. The complexity of the digital market place in general, alongside the lack of sophistication of both clients and media planners topped off with the fact that the recession and unstable markets means everyone is doing more with less = f&ck it just buy TV and maybe some banner ads. It’s even worse in Canada — i’m seeing overall budgets where the % of dollars to digital (that’s digital asset creation and/or media) being less then 5% of the total. It’s crazzzzzyyyyyy.But we are about to cross the chasm. Just need more great little strategy agencies who are reinventing marketing planning (and of course, we just need some more dollars to build the data platform to feed the reinvention 😉
Less than 5%!Truly crazy indeed.
The Tipping Point beckons?
Our 3 biggest clients are changing their current marketing planning processes (one of them globally) bc have seen the light that there is a better way. Definitely work in progress but with a focus on the word progress 🙂
sophisticated media planning is closer to running a hedge fund and should be payed like it if they want to people to do it.(very personal opinion)
@leigh:disqus Yes.I understand the business logic … mobile is going to be more about proximity, neighborhood, …mostly based on position and intend (push and pull market operating together).Totally a different market than desktop or immobile web…from business perspective. I think… in that way there is going to be a mobile web and immobile web (not technically though).& btw, what did your cute little fellow said at the end in the video… i am not that good at English pronunciations.
where did you see the video? Not sure i can say yet (has it been sent to the powers that be?)
@leigh:disqus I am talking about fred’s wedding day video… is that not you and your kid … my bad and sorry.
lol oh yes it is =– wasn’t sure anyone sent it to him yet 🙂
We must get you up to speed. http://bit.ly/LAIwo3Card delivered too.And @rrohan189:disqus chronicled it: http://bit.ly/LAILiT
So sorry you’re getting to hear of it this way.Thanks Donna, for cc’ing me on the comment.I meant to do a follow up and I will.. I’m just amidst a crazy bunch of days.
Yes, that was Leigh and her little boy. And at the end, after wishing Fred and Joanne a Happy Anniversary, he said “and Peter Pan!” Cute!
LOL…damn cute …I have a 10-year old … but 2-5 year old are the cutest..now i understand the next slide by @3fdd3d39cb0b8459be27c89654b37acf:disqus
yes it was happy anniversary to fred and joanne and then he slipped in, and peter pan
I’d like to hear more about why that is.
The enormous discrepancy between Mobile usage and Mobile ad spend is driven by the fact that most advertising platforms don’t support mobile (are not optimized yet to display ads on mobile) and even the ads that show on a desktop website, do not show on that same website when it is accessed from a mobile device. When users access the same website from a Native App, ads don’t display at all. For example: Wall Street Journal and New York Times. A lot of ads if you access their sites on a desktop, none at all if you access them through their iOS apps.It’s just a matter of time before the advertising platforms get optimized to serve ads on mobile at the extend that they do it on desktop. And then the gap between mobile usage and ad spend will narrow.
In my experience there are very real differences between the way mobile web and desktop web function.The main differences fall into a few different categories:1) Browser compatibility across OS’s2) Resolution of devices3) Javascript parsers (more trouble in earlier iOS and Android releases – 3.x for the former, pre-2.2 for the latter. Also, ICS had loads of trouble with this in development).4) Elements that you embed in the site – ads, the measurement of ads, the delivery of the ads, the k weight of images, widgets (e.g. embeddable code that renders some thing)Fred’s last comment about it being difficult to “deliver a third party measurement service across web and mobile” is spot on. But I also think it helps to differentiate what you are trying to measure. Site statistics on mobile and desktop web is being accomplished today with incredible accuracy (Full disclosure: I’m the metric/analytics/product guy at work). In app measurements are a little trickier and you either need to sacrifice accurate data in favor of only online data or integrate the app with some backend hack to store the data.App-based measurement is also doable but generally not through the same services.I find that most of the trouble with data is not the services that help you track the data (kissmetrics, GA, Flurry, Quantcast, other ad delivery providers, app development houses with home servers, etc.) because the javascript events are not particularly complex to put in and fire. The tricky part is figuring out where your company adds value and then parsing the data for quality, accuracy, relevancy, and sellable value (think: engagement rate, percentages, and denominators for an advertisement).First post to AVC. Thanks for the good read.
Not sure i get your post. Seemed like Fred was just speeking to the Raw numbers on mobile users across apps.Â
“In my experience there are very real differences between the way mobile web and desktop web function.”it’s still the same web, though.what about tablets? are they the desktop web or the mobile web? what about if we get google glasses? what about if we get The Internet of Things? all new webs?or the same web accessed on different devices?”1) Browser compatibility across OS’s2) Resolution of devices3) Javascript parsers (more trouble in earlier iOS and Android releases – 3.x for the former, pre-2.2 for the latter. Also, ICS had loads of trouble with this in development).4) Elements that you embed in the site – ads, the measurement of ads, the delivery of the ads, the k weight of images, widgets (e.g. embeddable code that renders some thing)”these things inform design decisions.there’s also the method of input: touch. extra sensors. battery considerations. context of use.1) progressive enhancement. usage of things like boilerplate, as well: http://html5boilerplate.com/2) also the viewport http://dev.opera.com/articl…3) progressive enhancement4) responsive web design, or choosing to remove ads entirely, or something else based on constraints. page size is going to effect speed: number of http requests, usage of CDNs, zipping of files etc. steve souders is the most prominent in this area (that i know of) http://stevesouders.com/ ex-yahoo, publishes for o’reilly”I find that most of the trouble with data is not the services that help you track the data (kissmetrics, GA, Flurry, Quantcast, other ad delivery providers, app development houses with home servers, etc.) because the javascript events are not particularly complex to put in and fire. The tricky part is figuring out where your company adds value and then parsing the data for quality, accuracy, relevancy, and sellable value (think: engagement rate, percentages, and denominators for an advertisement).”yeah. with GA it’s just a copy and paste job. it uses a self executing anonymous function to dynamically create the script tag–this ensures it’s non-blocking.avinash kaushik has a good blog for understanding the data http://www.kaushik.net/avin…
And a damn good one to start off with
Unfortunately if you don’t do a a custom integration with KM or another platform, you miss the crucial events in a single page app. It would be nice if there was a platform that could determine what to track from the jQuery Mobile (or whatever) markup, inject ads into DOM, and do whatever is needed with the live document. Maybe you could adapt the JS request model to a more traditional URI layout for integration with existing delivery platforms? Maybe explicit Javascript isn’t the answer and it would be possible to have a context in the content itself that is oblivious to the delivery platform but the content tracker could tie it to views? (with a small js script picking up the views and delivering them to the backend) I can think of a few possible implementations of this.
Thanks, Chris.
Mobile apps pull internet data, but aren’t the internet itselfA good example of why measuring all of this is problematic is the following:I tend to use foursquare a lot on muy phone. Same with Wunderlist (http://www.wunderlist.com/). Depending on the day/week/month one could be used much more heavily than the other.Because they are used so differently, what counts as a very active user for both can and should look very different.
Of course you should be commenting on AVC!I think the key difference here is not mobile web vs. desktop web. It’s web vs. mobile apps.Our traditional measurements on the web: visits, uniques, signups, DAU, MAU…they are accomplished by dropping some code into your web app to report this data back to an analytics engine.It’s a totally different world in mobile apps because you need to measure slightly different things using different code (across multiple app platforms) when it comes to mobile apps.The final layer is analytics for internal use vs. publicly collected and reported analytics. Third parties like Comscore and Quantcast have figured out ways to report broad analytics data about web sites, which is more credible for investors and journalists.And I think what Fred is saying here is that it would be really cool if you could unify all of this and have that same kind of third party credibility for a unified view of analytics across all of the iterations of an app – web, mobile and even desktop.
thanx @aaronklein:disqus @benapple:disqus @ShanaC:disqus … got enough dose to digest.
Being willing enough to ask questions, especially when you don’t (yet) understand something, is *exactly* why you SHOULD be commenting here on AVC :-)That’s actually one of the core things that orig. pulled me in each day here…the ability to ask questions and learn around a large variety of things I have no idea about or only limited personal exposure to…the mix of the group, and the various ‘experts’ on just about any topic, always make for interesting and fun learning in my book…
+100 🙂
Also though I think a simple answer to differentiate between the two is to say that when I log on something like facebook on my computer, that can be easily tracked by a site such as Alexa. Now with mobile, I can use facebook through an app, so I could essentially be using facebook all day long, and it will not show up on Alexa.Also the way we use these services is different now. When facebook came out (as a college student at the time, I can attest to this), people would browse the site for hours, seeing what their friends were doing.Now, you get alerts to your phone when anything interesting happens, you can easily post something and get out, so less time is spent browsing. This is why its more difficult now- its no longer as simple as “this user was on this site for 10 minutes”- there are much more metrics to think about.
In addition to app download conversion tracking, you can add this to the list of premium data that Apple controls… and will likely use to help sell iAds.
The problem fred is that “audience” does not mean much in the App store business. the only immediate metrics are downloads but are generally useless because unlike on a website where visit more or less equal usage download is definitely not associated with usage. Appannie is not helpful in that manner. Flurry is more but they can t make their data public because of privacy reasonsThe only way maybe would be to create significant sample of apps and compare their usage on a month per month basis. but that would be only statistical.Only Apple and maybe carriers have that real data. Maybe looking at carriers is the way to go.
comscore uses a sample for mobile metrix i believe. that has worked well for web measurement of large audiences if the sample is big enough
indeed this works well on the web. But because of the fragmentation of app usage and the “hit” based system that would not work well enough for apps. a sample would break the logic and would not help meauring more than 2 to 3 % of the app usage. In addition that would be meaningful only with apps participating in the operation. On the web you can pretty much measure anything by registering the logs from the browser. but on app you can t unless you have an SDK embedded in the App. (on IOS)The only working approach would be a hitwise like approach where the company would be plugged to the operators logs back stream and analyze what is being done and doing this with the 3 main operators.
the Apple way will only work for IOS market… and as Android is growing in numbers… Yes, Carriers would be The Solution. Just if… 😉
Now where is the LIKE button for this comment?
totally useless – what does usage mean, for what sort of app? This is so app specific that I would find even comscore’s panel sort of useless.
btw fred, ‘bartender’ was better than ‘MOD’ title … Is it disqus limitation or you changed to MOD.
I didn’t do anything. My guess is disqus is changing stuff
You’re a punk, not a mod, aren’t you?
Ringo summed it up best – “I’m neither a mod, nor a rocker – I’m a mocker”.In this context I can’t see ‘pod’ or ‘munk’ working, however… 🙂
Ha ha! That quote immediately popped into my mind, too!
Correct
Maybe “sherpa”. You’ve been up Everest before and made it back.
Is it the user numbers of the web/app that is most significant indicator of popularity/relevance, or the data/content they are subsequently creating/curating/accessing?Usage where passive I’d suggest is not as significant as interactive/proactive usage.Facebook is I’d tender a case in point – I, like many now it seems, dip in now and then as some of my friends/family/business associates only use the Fb environment – but, I hardly ever interact with ‘it’ (Facebook), now. However, I am sure I am still ‘seen’ as an active user and so measured for audience reach, etc. I can’t even be arsed to reset my preferred email back to what it should be, frankly. Nice one, Facebook :-/Twitter/Instagram (eg), however, I interact with several times a day, typically – consuming, curating and creating therein. This type of user profile is infinitely more valuable, it seems to me – more than just a number.
Never under estimate the power of number 🙂 …number sells like currency … numbers are valued more than ounce of gold 🙂
Once upon a time, yes. Crude metrics, however, exposed since the PPC/ARPU Excel naive business plan myths have been somewhat exposed – look at recent poor conversion figures from Facebook re: ads – handily fully exposed after the IPO – it is far more important to look at engagement – hence why Facebook was in such a state of panic re: Instagram – and its own Places checkin service has failed miserably when compared to Foursquare.Numbers will always be important at a superficial headline level but drill down just a little bit and engagement is far more relevant to depicting the success of a web environment/app and the true revenue/growth potential.And that is far more subtle a task to measure and assess – so, Excel biased business plans like numbers – which frequently don’t tell the whole story…
Yes agree 100% @egoboss:disqus … I was comment sarcastically about numbers… that is why so many smileys in the comment.But the general belief is that even if you take 10% of those users that comes to 100 million and if we sell X to those 100 million then the value is few billion etc, and if you extend that to next decade with growth rate of 18% … it is a trillion dollar business… when i hear this it reminds me of the famous business model …”Chinese and Indians drink 4-tea /day and even if we capture the 1-tea/day market that is a few billion dollar business”
Lol ;-)Cool. All about economies of scale, ultimately, I guess – be confident/aspire to reach an audience of tens of millions and even a small ARPU will be highly profitable – otherwise, in the real world for most services/products, succeed in your chosen niche/add value, charge a premium, and be confident your product/service will be profitable based on merit and on recurring customer engagement.These are pretty polarised examples, of course – there’s a huge middle market/ground inbetween the two, obviously, with a variety of challenges and opportunities – just decide what you are aiming at helps.I have been in so many situations where the management goal was mass-market for a product/service that was at best average in a niche let alone when measured in global terms(!). Excel is wonderful but dangerous if used to reflect dreams that are unfounded – too many business models/plans are based solely on numbers in cells and multipliers in other cells, until the desired goal-seeking is reached.Simple. Erm….The mass market model doesn’t really care about engagement – just throw enough crap at the wall enough times and some of it will stick! 😉
numbers are gold
VC should be about Venture and not about trend spotting – which can be automated.I like to think of this containment as “chicken and egg” eggs contain chickens (and cockerels) chickens contain eggs (cockerels don’t)1) An app can deliver net content (including other net access software)2) Net accessing devices (browser / appstore) can deliver apps .So a browser is only a specific type of “app” and any app may contain a browser.Standardised metrics rely on predictable activity (eg downloads, visits).In case 1) above the app (and app provider) can choose whether to support the metrics Fred wants.In case 2) the browser must comply with certain protocols so if it “visits” a web “site” it can be forced to record the metrics Fred wants. IF the web “site” owner wants to track and publish such things. Your app, accessing my service can be completely opaque to Fred if we choose. Fred doesn’t like this – I think it is of essense. I also think it is not a massive issue but using Fred (jokingly) as a proxy for VC …Fred has no direct power in this situation. However, if I want Fred’s money it becomes another matter – I may need to prove that visitors exist – so Fred wants people to bare their souls to the market.By implication he wants to KNOW where best to invest rather than speculate (based on historical trends and not forward looking) This accountability costs us all as a small overhead and means late developers (newcomers who may prove better) are penalized . So there is a strong incentive to be secretive unless you want money from non-clients, and also if you are smarter than Fred 🙂 Secrecy therefore has market value if you are smart but there is an implicit “Tragedy of the commons”! If we all tell Fred everything he wants to know, it costs us all and we get no more – old technologies that we have identified as useful “en-masse” are sponsored, rather than those that an “entre-preneur” might offer. Equally however – when Fred is wisely promoting those that are better regardless of track record – he adds value, creates jobs and imporves the market status quo. So for an evolving (technical) environment Fred is essential – but in a stable one he is a burden and long term the burden tends to “tragedy”. They call this burden not Fred but banking and Insurance – everyone pays – no net benefit is delivered! (If insurance pays the insurer it costs the insured – we should only insure when we cannot afford to lose – insurance is a commons in many cultures where a community fire service serves all at least cost)So how do you like your Feed – pampered with your data or hungry to serve the market – you must decide! -I say let Fred eat cake – when he works he certainly deserves it – but as a lap dog he is on no value ! Note generally VC and Fred are in reality quite distinct ! – but here we took a walk on the dark side
We done! But, Fred and other VCs do struggle from not being perfect. In other words even though Fred wants data, he wants it to help himself and others.
the problem with measuring metrics on the web is the over reliance on quantity over quality. I understand that quantity (i.e number of downloads, Alexa traffic, number of likes, etc…) will always be important but what we are missing is a metric that can capture the quality of use or degree of engagement. I know that @egoboss:disqus touched a bit upon this in his answer and @wmougayar engagio is one of these companies that is trying to measure social conversation as an important metric. The next challenge after such measurements will be to demonstration ROI
You could say the same about Nielsen Broadcast data.
that is a good example. This is where companies such as GetGlue are trying to add social engagement for those consuming media
Fred,I feel most strongly about this – I have blogged a response which you might care to read – thanks :)http://blog.kwiqly.com/2012…
I think you are reading way too much into my post. We use third party audience data as one of many inputs on what is going on in the market and in our portfolio. As audiences move to mobile the techniques we have been using are getting less useful.
Fred – No offence intended . More simply where fewer proven data points exist risk is greater. This is exactly where the qualification of value (rather than quantification metrics) eg Team, Market, Vision, Defensibilty and Technical risk are the great deciders between a lead investor and a me too follower.In my view (and I hope obviously) you have a history of success on the right side of that divide, and it is from a Startup perspective evident that these are the VC’s that are worth their weight !
It’s even worse then that (or better then that?)We need a new measurement system that looks across all media types.The inability to compare — or the forcing of digital media, apps, mobile, web to have mass audience measures applied to them, is consistently stunting brand dollars from travelling outside of linear TV.The starting place is X GRPs — and everything else suffers from there.
So, what you’re trying to say is: It’s all about ad mix.
The branding and art direction on the App Annie site is blowing my mind. The site is beautiful, but it looks like a toy and not something I’d trust with my analytics data.Now I’m going to spend the whole day rethinking enterprise branding and design.
as we enter the world of multiple webs this type of analytics will only get more impossible with higher margins of error. privacy, competition, and technical standards all conspire to make it so.but in my opinion, the mobile world has gotten too far ahead of itself. just like bubble 1.0 got too far ahead of the customer in terms of web apps, bubble 2.0 has done so with mobile apps, in my opinion. all of these questions will be much easier to correctly assess in my opinion once the market is cleaned out and the customer catches up.
Off-topic alert: Gotham Gal crushed it yesterday. Just sayin’ ya gotta go read it. http://www.gothamgal.com/go…
I second that!
I want to be her when I grow up…..
It was a great post.
There are parts of it that are really good.Pointing out the failings of media people is like complaining about the weather.
You saw the gist of her post as being about the failings of media, or even about media, period?
No, but she led and closed with it.The issue of how we work, why each of us works and how to insure that you are not wildly unhappy with your outcomes, is a broader societal issue – its not just a women’s issue.The ‘your decision define you’ part of the post – including the part about warning young women that having it all is a myth (no man ever had it all) – is really powerful to put on the table and discuss.Framing it personally dilutes everything. Joanne should adopt Dick C’s approach to media relations – barely concealed contempt.
Oh, this is the blog that Joanne Wilson’s husband writes… I had heard about him. 🙂
touche 🙂 and upvoted.
Well done. She would upvote this comment.
Thanks for the reminder, Jim. I saw the post in my Engag.io email and meant to read it, then spaced it. I enjoyed the post and comments.
Yes she did
If this is a wish list post, you have to expand. @leigh:disqus is right it comes down to ad $ and those that are pushing marketing strategies have to have something to back the game plan. So you have to get numbers that break market brackets including the webs and those that attach.As more apps that have to do with direct purchase and/or put the customer in the store grow, you need to have the numbers on the promotion delivery vehicle 2 steps away that will get those customers to where that app is.Do this and you begin to define more precise the soft and driven.
who has put an app out here? who has seen itunes.iconnect data?it becomes obvious what apple thinks of data when you see these tools – hint – nothing.The metrics world is very complicated. We use Google (analytics and ad-words), kissmetrics, crashlytics, hootsuite and several other backend tools – i find it a challenge to weed through these daily and find meaning.Just yesterday we seemed to pivot away from looking at “account” as a measure of performance and began looking at location as a better measure…..I digress…..
Indeed. And that’s before we look at the alleged(?) tactic used by some(?) services of up-selling deals to users who are on a Mac. Pretty crude demographics trickery if this is true – I have not read enough about it to determine if story is valid – but can see these kind of tricks being adopted more to elicit money from targeted users/consumers.
So called “mixed model” are a stat technique that can handle most of these issues.
got a spreadsheet with an example “mixed model”?
It is not something that can be done via a spreadsheet. You can look to ucla for some good free tutorials . Maybe Fred can add a Dropbox account to the blog so we can share files?
That is today’s post. You read my mind.
Is Alexa or Comscore more important? I think if VC is making an investment on these rankings, they’re not doing their research to measure an actual audience or the real reason for an investment, the big idea(s), as traffic can turn in one day. Many are easy to manipulate, and some are in the business to only sell traffic to Alexa toolbar, when in fact there is no traffic at all to the service. I guess this may be good if you want to look good, and most of our world is about looking good, yet it’s not fair to fool the advertiser to say you have an audience based on this rating. With regards to mobile, it’s a much harder to track, as many people are on mobile services all the times, and just keep them running in the background.
This has nothing to do with investment decisions. This has to do with staying on top of what is going on in the markets we invest in.
Thanks, think this post still holds true.http://www.avc.com/a_vc/200…
Just got off a LightReading webinar sponsored by Cisco and the 1st 3 bullets were:1) Mobile operators and their suppliers face major challenges in improving the mobile broadband user experience, given the current lack of intelligence at device or application level.2) Most operators are limited to network-centric metrics for coverage and connectivity, and there is no consensus on best practices for user-centric mobile analytics.3) Even the leading mobile operators have not met customer expectations on 3G while planning their 4G networks.Good luck having the vertically integrated monopoly carriers solve this dilemma. Now we have control/middle and bandwidth/lower layer bottlenecks. More reason for either Apple, Google or MSFT/NOK (or all 3 simultaneously) to disrupt the telecom stack.
A wise man once told me, there is research to know, and research to show.”Research to know” is a combination of your internal log files, ad serving data, purchase histories, conversion rates, etc… This data is highly proprietary, and depends heavily on quality tools, as opposed to standardization. The information is generally about seeking truth, as opposed to economic advantage. But the core issues to worry about in “research to know” are consumer privacy, data protection, and coordinating international laws around these things.”Research to show”, on the other hand, is much more about establishing a common “currency” that is accepted by the publishers and by data users. This data will inherently attract a lot of criticism because the numbers will result in a value exchange of some kind – this data is used to set ad rates, valuations, or other market prices. Independence, diplomacy, integrity are essential attributes that are must-haves in addition to data quality….the data supplier cannot be perceived as “in the pocket” of either the buyers or sellers (eg: publishers, ad agencies, VCs or entrepreneurs).Given how much baggage comes along with this independent currency, it is unlikely that a enough companies will provide internal analytics for public syndication. As a result, this “research to show” will almost always be sample-based as opposed to census-level.Once you reconcile yourself to sample based data, the problem becomes easier in some ways – it is about attracting a critical mass of publishers, distributors and networks to acknowledge your data source as an independent and honest broker of usage analytics.
Another good analysis with good points. This is now my “This week in sleepless nights” topic.
Good point. And the problem has been solved to a degrre on the “research to know” side at various large companies. What I learned working on it is that having a logged in userbase is a huge advantage, since then you see how people use multiple devices. Counting cookies or UUIDs isn’t enough.The issue of benchmarking the competition remains.
agreed. benchmarking is a totally different beast – can be more of a “market maker” function than a “research” function.
That’s a neat way of thinking about it. But when you are sitting on the outside looking at a market from a macro level you need data and the only stuff available is the to show stuff
agreed – but a lot of companies seem to focus on solving the “data collection” part, when its the “trusted broker” part that is the most important (and often the hardest) part.The key, in my view, is to be an early mover in important segments – place bets on nascent segments that are too small to attract incumbent players…for example, be the only company that compared app performance in 2007.The companies (like flurry in apps, or telephia in the early days of mobile video) that do “good enough” work in barely existent markets often become the defacto standard of measurement. This becomes lucrative if/when that segment becomes big.
We backed the predecessor to flurry the month the ios app store opened As JLM would say, “I agree with you more than yourself”
Would a company necessarily want its metrics to be public?
‘hodgepodge’ needs to make a comeback.Surprised a comScore or deloitte has not hit major mobile OS platforms up for the 3P measurement contract…..
20+ years into the measurable, trackable, accountable internet revolution and still, audience measurement is… elusive.hmm. maybe “half your marketing budget is wasted but you don’t know which half” is an eternally thorny problemor maybe, truly solving that problem is, actually, to no one’s advantage;)
It seems like services who are willing to contribute usage data into an audience measurement service should be able to learn more back in return, either in the form of better data or money. Put another way: If you contribute your [web] + [mobile] + [social] apps’ engagement, you learn something about how your service indexes.And if VCs like Fred want to see multiple services’ engagement data, those who contribute their data can be paid — A CPQ (cost-per-query) model of sorts — but where real money accrues to those who contribute data. Or perhaps the disclosure occurs for a short time or on a project-specific basis — like ‘Kaggle for Audience Measurement’ or somesuch.I am sure this is an old idea that those in the measurement business can pick apart, but I also bet many readers of AVC thought AngelList would never happen — that early-stage investment data was ‘too secret’ to be openly-available. Another example is SellerCrowd — the sharing within that community is amazing and counter-intuitive.If anyone is working on this, I have an interest in contributing some of my ideas — find me at chris (at) phenner (dot) org if you’d like to meet/talk in NYC.
I’m not sure if this will post, but Disqus is not playing nice with Opera 12 today (was fine yesterday). All I see is the 3 spinning cogs.
Interestingly, I can see my own post, and it shows me as a mod. Flattering, but I suspect incorrect.Bug report submitted anyway 🙂
another great way to check the audience is via Google’s AdPlanner: https://www.google.com/adpl…
In a totally unrelated note, Fred, how in the world do you find time to post on vacation?? Props to you man!
I get up earlier than the rest of my family
As an old Israeli colleague of mine used to say – “There’s plenty of time to rest when you’re dead.”
compete.com says that they measure both web and mobile traffic. They post a complex methodology on their webpage. it is not clear whether they mean “web traffic” from mobile or they also capture traffic directly from mobile apps.
Customer satisfaction measurement driven by voice of customer is a valuable measure and predictor of future mobile app and website performance. foresee.com provides a benchmarked Customer Satisfaction measurement score across web and mobile, as well as offline (in-store and call center) capabilities, it’s not just a “number of total logins” type measurement.
Total Universe is a product in Beta with comScore. Measurement based on tags and active unique monthly audience is de-duped across platforms. Not based on downloads and quite accurate. A few big publishers are not tagging yet (FB/Google), but this is where the best cross-platform measurement can be found currently.
Wow, timing couldn’t be sweeter. I just wrapped up a blog post covering the need for new metrics that are not simply in the web context.http://subprint.com/blog/th…”In our new world of constant, personal streaming data points such as followers, retweets, pins, comments, shares, plus ones, watchers, likes, likes, and likes it only seems fitting that at some point, we are going to have to have some new way of measuring the intersection and union of all of these data points and more importantly, their relevance. To me, measuring something that is only in the web context, a website’s pageview, is not only archaic and irresponsible but screams for the undeniable need for new network metrics.”
I think this is much easier to achieve for a new entrant in the market who doesn’t have a legacy of focusing on web pages/clicks/sessions.My company (http://www.pipewise.com) measures at the level of each individual user. The user could have signed up via a mobile app, come back via a web app, switched to an Android app, etc., but we’ll always track all of the activities and purchases by that user across each client. We’re also working on being able to contact that user via multiple channels – email, push notifications, in-app pop-ups, etc.And as the number of devices proliferate (TV apps, in-car apps, Google Glasses?) it is going to get even more interesting!IMO we are only a few years into a process that is going to take a decade or two to sort out, and startups who focus on helping companies sort this all out will do well.