Posts from Comments

Today we are going to talk about a new feature right here on AVC. It's been running for a day or so, so some of you may have already noticed it. Right next to the comments link, there is a new link that says "disqus commenter breakdown". It looks like this:

If you click on that you will get an alternative view of the comment thread – broken down by the most active commenters. You can scroll down, find the commenters you enjoy most, click on them, and read what they have to say.

Like many of the hacks featured on this blog, this was built by Kevin Marshall. He built it on top of the Disqus API.

I like applications that offer an API to developers to build alternative views for end users. And I think Disqus does this better than anyone else in the comment space.

Given how long and busy some of the comment threads are here on AVC, I can imagine a number of alternative views that one could construct that would be useful. If anyone else wants to hack on the Disqus API and create something useful, I am happy to give them similar real estate.

Sometime over the summer, there was a discussion of analytics in the AVC comments and Shana said something like "I would love to do a serious data analysis on AVC's analytics." So I reached out to Shana and told her that I would give her access to my Google Analytics and Disqus Analytics accounts and she could go crazy on the numbers. But I told her that she had to produce a post out of all of that work. She agreed and this guest post is the result of those efforts.

——————————————————————————————-

One of the things I find really hard to wrap my head around about AVC.com is that for all intents and purposes it runs like any other media site.  To me, this is my bar where I hang out, but in reality, this site functions much like many other media sites such as the Atlantic, or Refinery29.  There is content, there are analytics, and there are ways of pushing out content, there are some ads and tools to push them out, there are some tools to make the community more social, but not much else.  If AVC.com had a business model (which it doesn’t, the advertising money goes to charity), it would be one similar to many content sites out there: Increase Users; Increase pageviews; Sell ads.  What makes this site unusual is that there is a large community of users, primarily driven by technology built by the team at Disqus.

It also leads to some interesting questions about this site in comparison to other media sites.  Most content sites are still trying to figure out the role of comments.  Do they ignore them?  Do they not have them?  Do they feature some content? Do they write about the comments?  Do they reward commenting behavior?  Does having a community make a difference to the business model of content sites?

On this site, it does.  Not only does it make a difference, comments here are highly correlated with unique pageviews of repeat users, uniques in general (not just for repeat users), time on site by repeat users, and time on site by everyone.

One half of unique pageviews over the past 9 months have been generated by repeat users.   

I wanted to see if unique pageviews of returning users was correlated with the number of comments. I used a correlation coefficient (Spearman’s Rank Correlation Coefficient) which is a measure of correlation variables that behave monotonically, or in other words, the variables move up and down together. The correlation coefficient for unique pageviews generated by repeat visitors is 0.7973 to comments. This is a high correlation coefficient and suggests the two are linked.

To give a comparison point to explain this correlation coefficient, SEOMOZ reports for Good SEO Experiments a correlation coefficient(a linear measure of correlation) of 0.3 is considered quite good, even though 0.3 usually implies fairly low correlation.  Having a community on your site is therefore way more likely to be a factor that would generate significant traffic than SEO efforts, if we compare statistical significance.

I also looked at the rate of change for the percent of returning users versus the percent of new users. They line up quite nicely. They have a correlation coefficient of .9387.  However, the rate of change for new users as well as repeat users is quite small.  Granted, this is a niche audience, so I’m not totally surprised.  Still, it is nice to know that total user activity is very much driven by regular user activity.

However, average time on site for all users to comments is less correlative (though still significantly so), with a correlation coefficient of 0.6733.   Similarly, there is correlation coefficient of 0.6848 for average time on site for returning users versus comments.  I suspect the reason is that some people like emailing back replies, some people like to go the site to write replies, and some people like using Engagio to write replies.  Unfortunately there is no way to directly measure which people on this site are using email, Engagio, or the site itself to reply to comments.

The  correlation of all unique visitors is also highly correlative to comments.  (correlation coefficient =0.8413). 

This data leads me to believe that people are in fact coming to the site not just for the posts, but for the community surrounding the posts.  People are more curious about the chatter and the interactions that come out of the posts than the post itself.  Building out community means over time you will build out a growing site.

If you are a web publisher/media company and you are looking at this post, having a strong commenting platform (like Disqus) is going to be essential to your long-term success as a media outlet.  Communities can be bigger drivers of traffic than Search Engine Optimization.  Having a strong moderation/community management team in place is more essential than having SEO staff in the long term, since there is a higher correlation to factors that matter to growth and ad sales (pageviews, uniques, time one site) to having community.  The reason is that people are not just on your media sites to read: They are there to interact with other readers about what they have read.  Teaching your writers and your community to stick to your site to discuss articles in depth ends up causing long-term growth.

(some notes:)

1)My friend Daniel Choi, a PHD Candidate in Molecular Biology/Computational Biology at Princeton, helped me understand rho based correlations. Thank you Daniel.  

2)For the sake of discussion, Disqus and Google Analytics are two different reporting tools.  GA also samples when you are looking at daily data for 9 months for a site of this size.  Please therefore take this post with a grain of statistical salt.

3) William Mougayar was kind of enough to give me some data about Fred to see if Fred’s presence in the comments matters.  It didn’t make it into the post for a variety of reasons.  Thank you William, anyway.

4) Thank you Fred for cleaning up some of the language about correlations during the editing process

5)IRL I’m a web analyst who is job hunting for my next gig while handling some side projects.  If you like this post, feel free to get in touch)

I'm sure the AVC community has noticed a bit more comment spam slipping in. That's for a number of reasons. As the number of comments rises across multiple threads every day, it takes me longer to get through them all. One of the many reasons I read every comment is that I am also the moderator and I delete and report all comment spam, as well as very infrequently I'll delete an entirely inappropriate comment. It is also true that as the Disqus network has increased in size and reach, it has become a bigger target for comment spammers. Even if you filter out 99.9% of all comment spam, if enough comes at you, that 0.1% starts to amount to a real number.

I've made a small change in the AVC Disqus moderation settings. If enough people flag a comment (by clicking on the little red flag that comes up when you hover below a comment), that comment will no longer appear in the thread. I will still see the comment when I moderate the thread and I can reinstate it or delete and report it as spam. This puts the power of moderation into the hands of the community which is something I should have done a long time ago. I'm not going to say how many flags will do this. I don't want anyone gaming the system.

I do not want the community flagging comments you don't agree with. This is a community open to all voices, even when they express ideas that aren't popular. In particular, comments that hate on me personally should not be moderated. I can take the heat. I want the heat. I'd like to remain the only person who can moderate "entirely inappropriate comments." So if that kind of comment gets flagged off the thread, I will put it back.

Finally, I also do have the ablity to make certain community members moderators with the same powers I have. There are a few, not very many, who I would bestow this right onto. Shana and William come to mind in particular. If you would like to have mod powers and you think you've earned them, let me know in this thread and I will respond to the request in this thread. I will also report on this blog who I've given these powers to. I reserve the right to revoke them at any time.

I've leveraged the power of this community for so many things over the years, it is ironic that it has taken me so long to leverage the power of the community to keep the bar clean.

Tereza asked me to comment on the NY Times piece that ran this weekend on news sites' decision to move away from anonymous commenters. I think anonymous commenting leads to a lot of bad behavior and it should be discouraged. But I think anonymous commenting should be allowed and I allow it here. There are enough examples out there of why someone would want to comment anonymously that I think it has a place in the online conversation.

In the world of user generated content, you are always going to get posts that you don't want. Fortunately, there are a number of techniques that can be used to downgrade or even largely hide that behavior from the vast majority of users without taking it down. I think anonymous comments should be subjected to some of those techniques.

For blogs and online publications that get a lot of comments, and this blog is on the cusp of that place, I think we need a way to highlight top comments for each post. Disqus does allow the comment reader to "sort by" most popular or "best rated" but that requires user action. I think Disqus should offer blogs with a lot of comments the ability to run a window above the comment thread with the half dozen or dozen best comments that would be automatically calculated with the possibility of override by the blog author. Some blogs are already doing this like Business Insider, Gawker, and Huffington Post.

We need to introduce game mechanics into commenting systems and I think Disqus can and will be at the forefront of this effort. Game mechanics will reward the kind of behavior the community wants and will punish the kind of behavior the community does not want. The anonymous commenter who has valuable information but can't publish in their own name will be rewarded. The anonymous commenter who leaves a hostile name calling piece of crap will be punished. And the comment thread and community will be better off for it.

The other benefit of this approach is the community can police the comment thread. I do a fair amount of that today helped out by our community bouncer Kid Mercury. Turning that job over to the community in its entirety is the obvious next step.

The comment threads on this blog are now well over 100 comments on most days and get up to 300 or more comments on the most popular posts. I continue to read every comment because it is important to me to see them all. I also want to make sure they aren't spam or hate filled crap. But you may notice that I've cut back on the number of replies. It used to be that out of a 100 comment thread, my replies might be 30 or more. Now its probably 10 or less. I make up for some of that by liking comments a lot more.

But this community is following a pattern that all online communities follow. In the beginning, I was a huge part of the threads. Now I've cut back and let the community do more of the talking. And I think that's a good thing. Hopefully Disqus will give us more features like the ones I talked about earlier in this post to take the community powered moderation and rating and presentation to the next level. I'm looking forward to it.

Regular commenters know that yesterday's post's comment thread was not on Disqus. I tried something new (posting from typepad's mobile web app via the android browser) and in the process accidentally turned typepad comments on which turned disqus comments off. I figured it out too late and there was already a discussion going on so I left the whole post on typepad comments.

As one commenter said:

Wow, I didn't realize how much I'd miss Disqus commenting until it was
gone. Perhaps that's an unplanned benefit from this little glitch?

I said the same thing to Daniel Ha, founder/CEO of Disqus, via email yesterday and he replied back:

What are 3-4 things that you miss?

I'd like anyone who is interested help me answer Daniel. I'll post my 3-4 things here and please leave your thoughts in the comments. I'll make sure Daniel reads them, although that won't be tough.

Things I missed:

1) Threaded discussions. If you look at yesterday's comment thread, you'll see that I was replying (via email actually) to the comments but they are not shown as replies. I found it impossible to follow and stopped replying as a result.

2) Email reply. Typepad has email reply, at least for me, the author, but I don't think it works for the person leaving the comment when someone replies to them. And the reply is not shown as a reply in the thread. And it doesn't pick up my avatar when it shows the reply. Without those features, email reply really isn't useful to me.

3) No avatars. I've got so used to seeing people's avatars next to their comments. It really allows the community to thrive. The comment thread feels so empty without them.

4) Easy login. If you are a frequent commenter and are "logged in" as I almost always am, Disqus recognizes you and invites you to leave a comment. If you aren't logged in, you can log in right in the comment thread. Those two features mean that a lot more people comment.

Here's some data to show the difference between Disqus and Tyepepad's comment system. I average about 100 comments per post. I got 25 yesterday (though it says 49, I think some of the comments may be missing). Either way I got way less than normal. And yesterday was one of the biggest days ever on AVC with 19k visits and 22k page views, four times my normal traffic. Four times the traffic, a quarter to half the comments? That's the Disqus difference in action.

Ok, now it's time for everyone else to chime in (via Disqus thankfully).