Community Moderation

The Verge has an incredible post up about “content moderation.”

I have always felt that the hardest part of running an Internet business was insuring the trust and safety of the users and I am thrilled to see some light being shone on this part of the business.

There is always so much talk about the product and engineering parts of the business and so little about the extremely difficult work that goes into policing the product. And yet when you look at churn, so much of it in a scale Internet business is a result of users running out of patience with spam, trolling, and worse. This comment by Dick Costolo from the piece is telling:

“We lose core user after core user by not addressing simple trolling issues that they face every day, We’re going to start kicking these people off right and left and making sure that when they issue their ridiculous attacks, nobody hears them.”Β 

Well as the post points out, that is not so simple. And, of course, there are free speech issues too. I constantly hear people criticizing Twitter for blocking users.

But trolling, as bad as it is, is not the worst part of this work.

A trust and safety team has to deal with the most awful kinds of people and actions imaginable. I often suggest that everyone should sit in a trust and safety organization for a week. Then a lot of the conversations we have about free speech, privacy, and the like would get a lot more nuanced. There are bad people out there doing bad stuff.

Sadly, as I have seen again and again, startups don’t understand how challenging these problems are going to be until some sort of situation forces them to react. Then they throw people at the problem but never their precious “engineering resources.” When trust and safety, fraud, compliance, and moderation teams start getting their own engineering resources, something that often takes years to happen, then you know the company is finally acknowledging the importance and seriousness of the work.

The people profiled in this Verge story are heroes in my book. They do hard work, are not paid as much as they should be, and they are working in incredibly difficult and dangerous (for their mental health) situations. It is high time we start acknowledging them and their work and investing in it.


Comments (Archived):

  1. LIAD

    Real names be proofBiometric data based distributed key registry. Signed public key required to open social accounts. Misuse services, key blacklisted. No more cat and mouse shenanigans.

    1. William Mougayar

      What? You must be kidding. Anonymity is a right, and that doesn’t equal being bad.Satoshi Nakamoto is one of fhe most famous anonymous person(s).

      1. LIAD

        Anonymity is never broken.

        1. William Mougayar

          But it’s tricky today for the average person to figure out anonymity in a safe way. Not that easy, especially if open social accounts are involved.

      2. Kirsten Lambertsen

        So, I’m in favor of allowing anonymity. But it isn’t really a right when it comes to using a service. The service can set any rules it wants (like Facebook has). If anonymity is important to you, then you don’t use that service.But, I’m with you. Anonymity doesn’t have to = bad. And I want it in a lot of places for my own protection. But sadly it enables a lot of had behavior.

        1. Eric Satz

          Correct, anonymity not a right. And I wonder about the times you feel you need it for your own protection — aren’t you choosing to fish in the wrong pond? While real names/identities won’t eliminate all bad actors, I’d guess it eliminates 90% of them which makes the job of the trust and safety team more focused and effective.

          1. William Mougayar

            “I wonder about the times you feel you need it for your own protection — aren’t you choosing to fish in the wrong pond?”- In other countries, it’s not so easy being open, as the fear of repercussion is real.

          2. Eric Satz

            I agree with you. I also think it need not be one size fits all. The solution employed in the US, Europe etc may be different from the solutions employed elsewhere. And yes it is complicated by the fact that social media attracts a global community, but I suspect we can alter user id requirements based on traffic origination.

  2. William Mougayar

    The end-result of what an online community looks like is a direct reflection on the company/host. And in some cases, it is the company itself. Therefore, having a hand in shaping and pruning how it develops is a key responsibility.It starts with the fact that it is all about “User-generated content”, and it means that users shape the community, but the moderators (or curators) have a right to shape the users.Cordial discourse/debate and voicing opinion is a wonderful thing. It leads to progress and evolution. But when it crosses the line into calling names, disrespect for others, hate, provocation, and personal attacks, it leads to anarchy, brings the whole experience down, and distorts the picture.I prefer the word “curation” to “moderation”, because you’re deciding what can stay and what doesn’t belong, just like curators decide what pieces they’d like to have in a museum, because it shapes the experiences of all visitors.

    1. Kamal

      Yet, boards are plastered with moronic posts like yours.

      1. fredwilson

        Please. You calling his comment moronic is proof that his comment is not

        1. Matt Kruza

          so it seems kamal is trolling or just rather moronic.. what i am curious about is we see relatively littlle of that here at avc. Do you, william and the other few mods have to delete man comments? On average I would say you get between 50-200 comments (outside of that range is an anomaly), could you give a ballpark idea of how many might be moderated / blocked per post? Just ballpark. 5, 10 , 20 or even is it 0? Thanks for any perspective!

          1. William Mougayar

            Tiny tiny number get deleted overall. We’re talking 5-10 total maybe in the past 4 years that I’ve been involved as a mod. Generally, they cross the line into the highly personal or obscene.As you just saw, Fred prefers to call that person out, and make them an example by magnifying the “bad” behavior for others to see, and that’s a pretty good deterrent. The joke gets put back on that person before they know what hit them.

          2. Matt Kruza

            Nice. Really appreciate that perspective and insight. Serves as a great example how to keep a nice tight knit community. Seems like once you get to mass media it doesn’t work because there isn’t the respect and awareness that exists in a relatively close knit site (although avc is still a pretty big community so its impressive that you mods and all the readers have kept it so great!)

          3. William Mougayar

            Although for mass media, sadly for many of them, the community comments are beyond repair. Once a community starts with the right footing, it’s difficult to derail it, because its organic nucleus drives it. And if a community is full of trolls, personal attacks and name calling,- a newcomer will be tempted to perpetuate that similar behavior. They would need a big reset. YouTube for example recognized this in the past two years, and if you noticed, the comments on YouTube are more decent than they were before.

          4. Vasudev Ram

            On some other sites,1) mods’ egosand2) groupthink and groupspeak (a la George Orwell, though those I’m talking about would (pretend to) faint from shock or be scandalized if they heard this,are a big issue, IMO.

          5. Vasudev Ram

            Wow. Reading this thread some time later, noticed there are 3 Orwell references – mine and two others (search and ye shall find).Wonder if it means anything, in the context of this post or the overall topic.

          6. ShanaC

            mods on a lot of mainstream media sites don’t have egos. Usually they are just out of college kids…

          7. jason wright

            i thought they were still in college working from their dorm rooms

          8. ShanaC

            Seconding what William says. Disqus also autotakesdown spam if it is flagged enough times. I’ve put comments back up that weren’t spam that aren’t well lovedWe rarely block due to IP issues and the fact that many people’s IP addresses do get re-assigned. Unusually, we will also reach out to people if necessary if they are regularlish and there are problems. People have reached out to us about stuff on their minds as well.We keep an informal eye on special population groups (newcomers, young people, women in code, VIPs, ect) to make sure they are welcome and that the conversation is flowing smoothly.we also have some semi-informal community behaviors and rules that come from nowhere, and doesn’t come from us at all. example, there was a period where nearly every post mentioned @andyswan:disqus and bourbon, because it was funny. That stopped when people started to realize we were accidentally SEOing him to look like an alcoholic (which he isn’t), so there was a conscious effort by everyone to not mention bourbon.The one thing that is a pain in a neck is that images have a delay as they go through Disqus’s system. while it is a nice luxury to have disqus pre-moderate images to some degree, Disqus’s policies in some ways are both more and less broad and definitely more formal than this site’s. Currently we don’t have an image policy, mostly because the only really active user is @MsPseudolus:disqus with funny gifs, but if someone decided to get edgy with it, we would need one.

          9. Vasudev Ram

            Wow, that’s great to hear.Didn’t know this was going on in the back end of AVC.Thanks to both you and William.

        2. AlexHammer

          I agree that reputation matters (a great deal). As is said, consider the source.

      2. ShanaC

        oy. I don’t think it’s moronic. I disagree with him. Meanwhile we both moderate.Which I do think is weird…

    2. awaldstein

      Great comment but in actuality it is way more messy than this.You curate content but you will never curate people unless you are designing the Matrix.If you have every been a community manager or sysop you know at your gut that even moderation is term that borders on being too non emotional for the task at hand.The people you remove don’t matter.The people that are impacted by the comments and the removal are dynamic, emotional and critical by nature.Emotions are invariably non trivial and basically unmanageable in group environments without impact.

      1. William Mougayar

        It’s not easy of course, but people follow the direction of the community.

      2. AlexHammer

        I believe that emotions are manageable by culture.Good culture self-corrects those with good intent, and excludes others.

    3. AlexHammer

      Well navigated and spoken.I agree with you that it is nuanced. On the one extreme you have trust and safety On the other you have censorship.Perhaps a lot of the decision-making rests on culture, and how the company thinks of the user.What does the fact that you can’t (generally) contact Google for customer service say about the company? Are they encouraging self-reliance? Are they authoritarian?Is customer service an expense or a value-add?Do people act better in communities in which it is possible to speak with a live person? Does this humanize the company?

      1. PhilipSugar

        That is the most frustrating thing about google. I would pay to make a call but I can’t. It will be their achilles heel.

        1. AlexHammer

          Why are there no major search companies which you can call and speak with a live person?

          1. Anne Libby

            About 20 years ago in a public-facing division of a NYC-based financial services firm, we calculated that each call into one of our service reps cost us around $15. If I remember correctly.This was pre-social media. Today a green, still being trained, service rep’s honest gaffe could wind up smeared all over the Internet. I don’t know how you figure the cost of that…

          2. AlexHammer

            The $15 was a known figure but you need to dig deeper.How many customers do you lose if you don’t take those calls? How much does it cost you each customer that you lose?How much more do customers spend in the future if customer service is done well?These are the types of questions which need to be answered in any const/benefit analysis.

          3. Anne Libby


          4. AlexHammer

            Everyone is a skeptic πŸ™‚

          5. AlexHammer

            A well intentioned gaffe is much more managable than pretending that the customer’s issue doesn’t exist, much less matter.

          6. PhilipSugar

            Agree but to let me recover a password for a child’s internet address I’d pay $50.

          7. Anne Libby

            Agree! I’m not suggesting that companies *shouldn’t* provide service.Pet peeve, the user forums filled with incomplete, inaccurate, and outdated “solutions” that substitute for solid documentation.It’s “service debt,” in a way. Which comes back to a few of Fred’s points, and some in the Verge article, too.

          8. Drew Meyers

            Because the vast, vast, vast majority of people who use them don’t pay a dime to the companies that operate/invest in/market them.

          9. AlexHammer

            F–k our customers, we take them for granted. Is that what you are implying?

          10. Drew Meyers

            Not at all. But there are very real financial costs associated with servicing non paying customers with live people.

          11. AlexHammer

            Where did all of the cash that Google is sitting on come from?Your argument doesn’t make sense to me.

          12. Drew Meyers

            If it was easy to contact Google, here’s my hunch of what would happen:millions of people would call/email and ask “why does search result x contain site Y as # 4 (which there are millions, all personalized to the user). this is wrong. page x is clearly more relevant than page y for reason 1, 2 and 3. why does that other page show up there”Could Google afford to deal with all those inquires for free?

          13. AlexHammer

            Bing could if they wanted to surpass Google.

          14. AlexHammer

            Do paying customers on Adsense have good support? (I am not sure).

          15. Drew Meyers

            I’m not sure either.

          16. AlexHammer

            Monopolies quickly grow lazy.

      2. Vasudev Ram

        >What does the fact that you can’t (generally) contact Google for customer service say about the company? Are they encouraging self-reliance? Are they authoritarian?Good question.I’m not sure about your points, but it seems like they are algorithmic, aka like to automate the shit out of everything. Might not always be a good approach.

        1. AlexHammer

          Google’s approach seems like a good one, until someone else with as good or close to as good a search engine decides to take a different approach with customer service.

    4. Twain Twain

      Evan Williams on social media hosts:*…These are problems not isolated to Twitter but across the Web.At some point, the junk food affects the financial health of the system.=> DATA CLEANSE across-the-board.

    5. ShanaC

      facebook is too big to say it is a reflection of just its users. Same with youtube. we’re talking billions of views in both categories

      1. William Mougayar

        But each person’s timeline is like a mini-community, no? Or some groups, for that matter.

        1. ShanaC

          it’s complicated, due to the network structure for any given person. some people are much more likely to not actually be part of “a given community” as much as “between given communities” or “leaders of a super-community” and modeling that is very difficult

  3. andrewparker

    Wired had a similar story a few years ago with similar first-party sources. This one from The Verge is so much better, more nuanced, well researched. This is a representative example of everything Vox does. They (recode, Vox, verge, polygon) pretty much never are the breaking source of news. But the quality of their stories is always top notch.

    1. fredwilson

      I think it’s interesting that this story was funded (at least in part) by a third party that invests in investigative journalism

      1. ShanaC

        bothers me so much that long form journalism is going nonprofit

  4. jason wright

    ‘witch hunt’ to become ‘troll hunt’ at the Guardian newspaper. if a reader writes a comment not in keeping with the G’s world view the reader gets the ‘troll’ treatment and the comment is deleted on that basis. it is just so incredibly self serving ‘neo liberal’ authoritarian bullshit that the G has just launched as a campaign to halt ‘trolls’. It’s George Orwell on steroids.

    1. LE

      I don’t read the Guardian comments they really do that?

      1. jason wright

        the G has taken ‘moderation’ to a new level of censorship. Katharine Viner should be sacked. the hypocrisy is stunning.Owen Jones is possibly the most opinionated journalist it has. He simply will not accept that readers may take a position that does not agree with his. in Owen’s world they must be sexist, racist, homophobic, et.c., et.c., and be expunged. only his political discourse counts. alternative views are ‘abuse’;…many articles are simply never opened up to comments. the political coverage is too ‘sensitive’ for the G to contemplate allowing the readers to express an opinion.the G is financially stressed btw. it will fail in time.

        1. JimHirshfield

          Old media newspaper mentality thinking that comments are supposed to be “letters to the editor”, not conversations. What comes with that is a sense that they know better than you, that’s why they’re the writer and you’re the reader.

          1. jason wright

            institutional myopia.

      2. Cam MacRae

        Yes, they’ve completely lost the plot. Ironic, given the G has all but replaced reportage with opinion pieces from notorious above the line trolls.

    2. Russell

      I couldn’t disagree more. The second article you linked to below is a big data analysis of the 70m comments they’ve had on their website. As I read the article, they are removing comments that attack their journalists. I would love to hear @wmoug:disqus ‘s view of this piece.

  5. andyswan

    1) Charge $5 to be verified as a real person.2) Display name = name on credit card3) Allow users option to “only show tweets from verified users.”

    1. Anne Libby

      That would be one approach.I’ll tell you, though, some of most surprising (edit) offenders I’ve seen in the last few years were trolls on the “official” Linked In group for my bschool. Real names, and arguably in a space that connects behavior to career. Doubly so, because of the bschool connection.#itscomplicated(edited, changed “worst” to “most surprising.”)

      1. LE

        A good way to change a word that you previously hated don’t like is to use the strike tag….

        1. Anne Libby

          Good idea! Next time, thanks!

    2. JimHirshfield

      That won’t work in Aleppo, Cairo, or Tehran.

      1. sigmaalgebra

        And, it won’t work in the septic tank in my backyard, either, and to me all four are all the same. For all four, there’s no hope of anything decent. To heck with them.

        1. JimHirshfield

          You’ve got a shitty attitude today

          1. sigmaalgebra

            Do I sense something pejorative? Gee, ironic given the thread today!For Aleppo, Cairo, or Tehran, etc., I’ve had it, fed up to my ear lobes.My considered judgment is that there is no hope for those places — none. There hasn’t been for 1000+ years, and that stands to remain the case for another 1000 years.Those places just don’t belong on the same planet with Western Civilization.Those places are totally dominated by Islam. That’s not just a religion but runs essentially everything in the country down to daily life in the home and everything between. It runs social relations, marriage, dress styles, food, architecture, education, law, foreign policy, government, etc. And, it very strongly keeps out any change. So, it’s a closed system, all locked up, self-perpetuating, unchanging. Except for oil and what they can buy with the oil revenue — vehicles, guns, construction companies to put up buildings, etc. — they haven’t changed in 10, 100, 1000+ years.They come in two version, Sunni and Shiite. Each regards everyone else, especially the West, as infidels to be converted or killed. Literally.We have no reasonable hope of changing them. We can easily defeat them militarily, and, if they attack or threaten us, then we should so defeat them. Otherwise, we should just wall them off and leave them to suffer alone. Maybe they will discover civilization, and maybe they won’t, but in either case we don’t have to pay any attention to them.That won’t work in Aleppo, Cairo, or Tehran.Of course it won’t work. Essentially nothing new from the past 1000 years will. It’s totally a spit to windward to hope for anything better for those places. It’s astounding that a culture can be so locked into degeneracy, but it is. We’re fools if we hope for anything better for them.

          2. Drew Meyers

            Not a very positive outlook on life you have.

          3. sigmaalgebra

            “Positive”, negative — I’m just trying to understand the reality. My best understanding of the Islamic countries is just what I wrote.My view is that we are just fools if we don’t look at reality. In that case we can make some serious mistakes, e.g., trying to bring secular, constitutional, parliamentary democracy to Iraq after we dumped Saddam, e.g., as in W’s remark: “The Iraqi people are perfectly capable of governing themselves.” No they aren’t. Not even close. Instead, they are capable of returning to their three way civil war — Shiites, Sunnis, and Kurds. That little, itsy, bitsy, mistake-y cost us thousands of US lives and ballpark $3 trillion. Big mistake. Huge. Dumb.We need to learn. We should be learning, $3 trillion is high tuition.

          4. Drew Meyers

            Have you been to any of the areas you speak about? Sat in their homes? Listened to their issues and perspective?With all due respect, if you haven’t done that, you have no idea what reality is.

          5. sigmaalgebra

            > you have no idea what reality is.That’s a bit too strong. Instead, since 9/11/2001, I have been awash in reports of what the “reality is” in many Islamic countries, and above I gave a summary of what I learned.Sure, visiting, talking in their homes, etc. would let me know more. But, so far I’ve seen what looks like a quite consistent picture, and I have lost patience with that picture and have contempt for it.Part of that picture is the Sunni-Shiite 1000+ year war: Basically special cases of that war are the long war of Iraq and Iran, the hostility internal to Iraq that Saddam put down with Stalinist tactics, the Shiite-ISIS war since the US largely left Iraq, the Syrian Assad and Syrian rebel war, and the Syrian and ISIS war.They are fighting religious wars: Europe did that for a long time about 500 years ago. No doubt the major rivers of Europe ran red with blood for many decades. Finally Europe decided mostly to separate church and state. Nearly all of Islam has still not figured out how to do that.For us here in the US, a biggie is the part of Islam that everyone else is an infidel and should convert or die — details from the Qu’ran in…If I’m wrong about Islam, then I would be really happy to learn how and what the good truth is that I’ve been missing.I’m not xenophobic: E.g., I don’t understand Japan, but I have a lot of respect for them and nothing against them. I don’t want to mix my culture with theirs, and my understanding is that they very much feel the same way — fine with both of us. I don’t understand Russia very well, but I have a lot of respect for them, especially in science and music. I do better understanding Italy, France, and Germany and have a lot of respect for each of them and likely would enjoy visiting. Italy? Fantastic, just unbelievable art and architecture, music, food, a quite nice history in science and technology, and more.

          6. Cam MacRae

            Careful, jokes like that will get you in the poo.

      2. ShanaC

        or India…or anywhere where banking is still emerging and cell companies work better than banks

    3. jason wright

      just rip a 6P fingerprint scan

    4. Jess Bachman

      But I’m already on Facebook.

  6. andyswan

    The problem is the only people that want to be on these Orwellian “trust and safety councils” are exactly the people that you don’t want in control of content distribution/censorship.They are the “safe space” losers from college

    1. Anne Libby

      That’s actually not true, Andy. “Trust and safety” teams exist at many public facing startups.Maybe you didn’t understand Fred’s use of the term, but that’s a group of startup employees you’re talking about…the people I know in these roles are thoughtful, smart, and caring.

      1. andyswan

        Have you taken a look at the team at Twitter?

    2. sigmaalgebra

      Uh, since that content is so corrupting, soon the moderators will be corrupted so that all the moderation is by corrupted people, thus, permitting corrupted content! Extra credit for the original source of this idea!

    3. ShanaC

      would you want me on this team?

    4. Susan Rubinsky

      I don’t know if agree. Some of the work I do is in the moderation space in the public sector (as a consultant). My main clients are public transit. I consider this work part of doing good for the public.As you can imagine, people can get very upset about public transit. The policies we’ve evolved over time basically allow all posts to say live unless there is a bullying/threatening and/or offensive language. In those cases, we send a private message to the person, indicating why we are removing their post. We give them a direct contact email and phone number if they would like to call us to talk about why we removed their comment.We address all complaints live in the format in which we are contacted (eg., if the comment is live on our Facebook page, we respond live on the Facebook page.)I think I am definitely someone you would want on your council/board. Insuring a safe environment where people can air their thoughts is key. We would not allow someone to stay on a bus or in the bus station if they were actively bullying or threatening someone so we should not allow that to occur in an internet space either.But active discussion, even if it exposes a problem with, say, bus service, is welcome. We consider it a way in which we can publicly discuss known service issues, and to find out about ones which we were unaware of.

  7. Robert Rogge, CEO Zingword

    Before starting Zingword, I owned a successful multilingual services business serving a variety of e-commerce sites with different business models. We did a lot of community management and “content validation.”It’s expensive work. The key point for a startup is to effectively build an outsourced team in the Philippines or elsewhere using tools like Upwork. Costs can escalate quickly…We’re thinking a lot about fraud protection and trust in Zingword, so this article is timely for us.

  8. pointsnfigures

    agree. and especially agree that it’s sometimes hard to figure out when to censor, and when not to censor. problem is when you are doing it in an online world, the comment can stay up and go viral while you are debating the problem.I toured Facebook’s office in Austin, TX. They have an entire legal team there dealing with all kinds of issues. It’s very complicated.Recently I saw Twitter was trying to stop terrorists from opening up accounts. As soon as they root them out and crush the account, they set up a new one. Whack a mole.

  9. JimHirshfield

    Brings back memories of all the issues mentioned in the article. Angelfire, Tripod, and GeoCities all had these issues, perhaps at lower scale, in the late 90’s.And fast forward to the last few years at Disqus. Many of these issues are existential to communities.

    1. AlexHammer

      Most people respond (to varying degrees) to the principle of reciprocity. The more respected they feel by the system, by the culture, by the company, the greater they reciprocate in kind.Those who are just antisocial should be banned.

      1. JimHirshfield

        Hard to define anti social in every context

        1. AlexHammer

          although we could rely on the “know it when we see it”perhaps anti-social is those who attack rather than criticize, those who seek to belittle and /or destroy.

  10. Kirsten Lambertsen

    I’m just getting into reading the Verge piece, but I don’t think free speech applies here (to Twitter, to YouTube, to AVC, etc.). Free speech isn’t about one’s right to show up on private property (which is essentially what these services represent) and say whatever one wants.Any given service can set its rules about discourse on the site and enforce them. They can change those rules any time they like. No one’s right to free speech is ever violated by this. If the government starts shutting down personal blogs because they criticize the president of the U.S., then we’re talking about violation of free speech.One of the earliest high profile bloggers, Duce, put it quite well, nearly ten years ago when she said (to the best of my recollection), “It’s my blog. If I want to print out your comment, take it outside and run it over with my car before deleting it, I can.”For a lot of people, free speech does not mean what they think it means. I can’t waltz into Starbucks and start spouting racial hate. They can kick me out for that. My right to free speech is in tact. I don’t have a right to post stuff to YouTube that breaks its rules.I have to go to Comicon now πŸ™‚

    1. JimHirshfield

      Spot on!

    2. PhilipSugar

      Yes, yes yes.I am always stunned when people say something is “my right”.

      1. LE

        Agree and that’s a group dynamic. People simply parroting what they see others doing. Entitlement. Definitely more now than it ever was as a result of blogging and social media and the internet.One of my biggest “people gripes” is that they don’t think businesses have a right to make money. For example when airlines (who apparently traditionally lose money) start to find ways to make money (charge for bags or things that they always gave away for free) everybody is in a tizzy over being charged for something as if it’s a right to get to check a bag on a flight for free (instead of simply being charged more for the ticket). Or that because the CEO of Verizon made a boatload of money so they can afford to pay the union guys more or they should feel the bern.Was at a car dealer yesterday (in DE) discussing with the dealer a hot car that they felt they could easily sell over list. Said they can’t do that people will freak out so they try to restrict who they sell it to to make sure it’s not flipped. You have to pass the filter. (Some dealers do it of course but not this dealer). Customer entitlement. They don’t care about the cars that sit unsold that they make no money on or lose money on. All they care is whatever their idea of being treated fairly is, in their uninformed never operated a business or car dealer opinion of the world and the way it should work.

        1. PhilipSugar

          I think I know the car dealer. They have sold cars to a co-founder. I think part of what they do is they know if they get somebody that is going to come back versus just flip it for a profit.Yes, I am always amazed when people complain about prices for things they have control of purchasing. There are sometimes where people “make up” pricing and don’t have a policy on it or a set price list, unfortunately healthcare leads the pack. I have a hard time not hating everybody in the system. Doctors that just want to churn through people, administrators who just want to build fiefdoms and castles, insurance companies that just don’t want to provide insurance, and scum sucking lawyers. They all suck.There are two restaurants in my Chesapeake Bay town that charge what people think are “rip-off” pricing.They want to tax them extra because they “charge too much”. What???? Don’t go.

          1. LE

            I had the funniest thing happen the other week. I had a small physical therapy bill (roughly $650) for the part insurance wouldn’t pay for. The therapy was shitty so I thought [1] “I will let this bill lapse and when they call me I will negotiate them down off the amount that I owe”. So I got several months of dunning notices, nothing scary, actually just a statement etc. Obviously I had the money to pay it. In fact I wanted to pay it in December at the PT office but they wouldn’t let me pay it (because it was done by central billing). I figured if I payed it in December I could figure out a tax angle (let’s just leave it at that).Anyway one night I am going to sleep and I realize “oh shit they have my ss# what if they turn this to a collection agency!!!”. Since I have zero medical bills (or any bills) and always pay bills literally before they are due this is not something I even thought of when I first had my idea to negotiate the bill down. (Or I would have built the strategy test over that fact and acted accordingly..)So the next morning I call central billing and they say “oh we just turned that over to collection”. I am like “oh fuck bet wrong on that one”. So I was able to pay it with a credit card. They told me that the credit collection gives you 90 days (before reporting) and they probably would have let me negotiate it down. I didn’t want to take a chance though. Definitely learned something.Back when I was dating a girl who was divorcing an anesthesiologist (with a private practice) they had literally millions of collectibles and something like (from memory) 40% not paying the bills. So if you drag them out you are in a position to pay pennies on the dollar (or certainly something less) if you are willing to negotiate it before it goes to collection (or has a vig) or it’s bought outright on pennies on the dollar by some company that buys receivables. I was amazed at how they service people and then bill later. (Just like the therapy place did with me they didn’t care about payment at all and billed me much later…)[1] Of course that wasn’t the only reason it was primarily for the fun of it.

          2. LE

            Well as you know I have little experience with corporations and CEO pay. But isn’t part of the purpose of high pay and benefits to create a pyramid that gets those beneath the top to both work harder and also to be able to strive under the false impression that they might one day be able to earn such high pay? Isn’t the same dynamic at work with people in sports? In other words aren’t there other soft benefits to paying high pay in terms of egging on people’s aspirations?Adapted in part by considering what Dubner said about drug dealers living with their mother.

          3. PhilipSugar

            That is the argument I’ve heard. Get all the SVP’s to work really hard.Sports is different. A great player can make a huge difference.But here is the thing: In the sport with the most people American Football you are 22 starters. In Basketball its five.The difference between winning and losing is VERY defined and the number of fans based on that record is too.In business, it’s much more nuanced. I’d rather have 10,000 great workers than 10,000 mediocre.Now Verizon workers that are making $130k might have to realize there a lot of people that would love that job.

          4. LE

            I have spoken to those Verizon guys. They are really single function machines. Try to have a conversation even about something in telecom that is not their expertise of POTS lines and it’s like talking to a 12 year old literally. They have no curiosity they have been trained to do one job and that’s the job they do (and then they go home drink beer and watch the game). Back in the 80’s I tried to get one guy to run cctv wires for me and he said “I’ve never done that sorry”. I said “it’s running coax cable you can do it easily I will give you everything, you just fish the wires and secure them”. And he did it and I saved a ton of money. He couldn’t visualize doing something that was, to me, just like fishing wires in a building like he had done for 20 years.<verbatim>That is the argument I’ve heard. Get all the SVP’s to work really hard.Where there is smoke there is fire.The other dynamic is that ever since they had to disclose CEO pay it’s well know (I think) that pay has gone up. This is an example of “can only be as honest as the competition” variation in action. That is, the group determines what can and does happen with any individual in the group because they all know what is going on. Will you be the schmuck that takes less money? Of course not.That said the life of a top guy must be rough. Not that it’s not rough for the guys below him (the SVP feeder) but let’s face it they do have plenty of responsibility and don’t have the freedom to just be themselves always having to watch (like any corporate person) every word that comes out of their mouth.

          5. Matt Kruza

            I think you are describing the “tournament economics” theory. Hope this helps put the formal name to it if you were curious!

          6. LE

            Thanks I will look at that article!

        2. PhilipSugar

          I also will add on the CEO pay. I have said for decades two things. Companies should get benefits for having employees in the U.S., and public CEO pay is out of control.If you and I owned a company and we had a CEO running it, we would not have them take a $10mm+++ pay package for running it if they didn’t start it. I understand you start it you take all of the risk you make money as an equity holder. Great, as much as you want.But if you are a cunning enough political person to become the CEO of a company you didn’t start, you are an employee. You need to be paid like an employee. $6mm base and another $12mm in stock??? Not so much. I think I could do that job for a bit less. Now what is the stock??? Not so sure, sometimes that gets evaluated strangely.

      2. jason wright

        where do you draw the line to separate a person’s rights allowed and rights not allowed?

        1. LE

          Well a good place to start is if you are getting something and not paying for it your rights go way way down.

        2. PhilipSugar

          Simple. If you are deriving any benefit especially not paying, it is not a “right”.I’m going to set up the flames here, I guess just fun because I never do it.It is not your “right” to buy whatever you want if you are on foodstamps.It is not your “right” to have as many kids as you want when you are on aid.We can go on.

          1. jason wright

            Basic Income Guarantee has no place in your world?

          2. PhilipSugar

            None, none whatsoever. That is a fantasy dreamed up by people that live in an Ivory Tower among other rich people. I can’t even express how stupid that idea is. Anybody that says we need that needs to live in a poor area for 2 years. They need to see how people need a hand up not a hand out, they need to see how people will take advantage of kindness. Nope, never.

          3. howardlindzon

            The first part was good

          4. PhilipSugar

            I am serious. It is easy for people that live in a homogeneous group of people with no outliers of people that exhibit really bad behavior to believe that.That is not America.Denmark, Iceland, Germany Japan, Australia, Canada yes.Other places like Indonesia, Quatar, Brazil, Russia, and the rest of the world….No.We are a beautiful mix between the two.But traveling as much as I do, it disgusts me when people think they can extrapolate something to our society…..This is going to sound like one of those “soundbites” but I bet anyone I have more working class friends than anybody else that spouts on this subject.Charlie might be close, but I don’t have any that I employ, just those that are friends.

    3. LE

      I can’t waltz into Starbucks and start spouting racial hate.You probably can’t waltz into Starbucks and start to really “spout” anything. Meaning if you walk in and stand there and start to disturb the other patrons by making a speech (of any type) you probably will (and should) get kicked out. [1]Now if you want to spout racial hate you do have a few options:a) Make a speech at a Universityb) Write it into a movie, song or comedy act or artwork. [2]c) Write an op-ed for the newspaper.d) Write a book and have the book create enough controversy that your thoughts are repeated.e) Be a member of the race you are dissing. You get a special exception if you are saying bad things about your own group. [3][1] Now if you have kids that are making a racket otoh they won’t say anything to you at all.[2] Creative pursuits are allowed much greater latitude with respect to free speech.[3] As if to say “being a member of the group is prima facie evidence of operating in the groups self interest. Which is about as reliable as thinking the carnival ride is safe because they guy pulling the lever let’s his own kids ride it.

      1. Rick Mason

        When is the last time you’ve visited a university? Liberals have free speech, on campus while conservatives do not. A racist would have no chance at all. In fact priests sometimes trigger panics simply walking across a Big 10 campus.

        1. LE

          Good point.

        2. ShanaC

          that is because most priests/rabbis/nuns/monks/theologians/whathaveyou don’t wear traditional garb day to day. I’ve met rabbis in running clothing when running. I’ve dharma holders priests in the zen tradition in sneakers and jeans. Why would they wear monk’s kimonos.It would take a couple of moments for me to realize it was a catholic monk’s robes, and I think i would only know that from disney’s sword and the stone. if i hadn’t seen that in real life I think I would freak out, because I would have no idea why a guy would be dressed like that

    4. jason wright

      ‘that breaks the rules’ – blockchain breaks the rules set by corporate publishers. that’s a good thing.i say people should be allowed to express whatever opinion they want.

      1. Kirsten Lambertsen

        They can. On their own blogs.

        1. jason wright

          the blogchain will replace centralised mass media platforms and centralised social platforms.

    5. Twain Twain

      Where’s Comicon?!!!

      1. Kirsten Lambertsen

        Ha! It’s at Meadowlands – you still have an hour and twenty minutes to make it πŸ˜‰ But never fear – the *big* one is at Javits in October. See you there?

        1. Twain Twain

          Ok, putting it into schedule!!! Another great reason to visit NYC at that time of year to add to the amazing apples thing!

          1. ShanaC


          2. Twain Twain

            Saturday mornings at Borough Hall farmers’ market there’s Wilklow Orchards’ apples.*…Better than the ones at Union Square.

          3. ShanaC

            prospect park is closer to me

    6. howardlindzon

      Exactly. Twitter chose scale and with that comes the cleanup later on public shareholders dime.

      1. Kirsten Lambertsen

        I’m still optimistic that Twitter will find a way to not just cope but to lead in this area. It’s a great opportunity.

        1. howardlindzon

          Still a giant opportunity which is why the market cap is still ginormous

      2. Twain Twain

        Evan Williams on Twitter’s junk food.*…Of course, it’s possible to scale quantitatively whilst also cultivating quality but there aren’t many startups that have solved that type of engineering.And, if investors keep sending value signals that scale is all that matters …Then junk will keep circulating in the system until the inevitable data cleanse happens to restore product & financial health.

        1. ShanaC

          you can’t force people to eat brocollini (I’ve actually tried this :/)

          1. Twain Twain

            I wouldn’t eat brocollini!!! (So wouldn’t serve it, :’).It’s a joint Product-Engineering-QA-DevOps problem. The quality of the food (junk or varied) is something they can develop process for.

          2. ShanaC

            what is with all the people I know being against cruciferous vegetables? They are tasty!!!And – sort of. Trickery only goes so far, even with “real food” or “real content” At some point people do push back, whether brocollini disguised in “junk food” or serious news disguised in listicles

    7. JasonBoisture

      Yes! Reminds me of a favorite quote:”People demand freedom of speech as a compensation for the freedom of thought which they seldom use.”β€”Soren Kierkegaard

      1. Kirsten Lambertsen

        Love it πŸ™‚

    8. ShanaC

      enjoy comiconThe idea that the web is a set of spaces that you can be kicked out of as best as possible is bizarre to many people, mostly because it is hard to imagine the “space” context

    9. Stephen Voris

      Conversely, though, “freedom of speech” is practically tautological if it’s merely “freedom of speech we like”. When it comes to the First Amendment, “we” would be “the government”, but it’s not so hard to extend the analogy to say that “we” in these contexts refers to “whoever handles moderation on this website/app/etc.”In that sense – you’re correct, there’s generally no explicit “freedom of speech” clause on these services, and indeed there are usually lengthy Terms and Conditions setting out precise legalese detailing what you are not free to say (and, given that it’s written in legalese, very few problem users are likely to read them).The danger is (and has been) that in throwing out the spam and the trolling, you also catch “honest opinions, poorly expressed”.

      1. Kirsten Lambertsen

        “The danger is (and has been) that in throwing out the spam and the trolling, you also catch “honest opinions, poorly expressed”.”That’s a business problem, I’d say.When I use the term “free speech” I am referring to how it’s described in the U.S. Bill of Rights. It’s pretty clear and one the amendments that has stood the test of time well. So, my point is that free speech is not at play here and doesn’t belong in a conversation about moderating commercial or personal website comments or UGC.And so, in that context I don’t see it making sense to extend the ideas of free speech as it applies to our government to whomever handles moderation of a commercial or personal web site. In fact that’s the point πŸ™‚

        1. Stephen Voris

          And my point is that while the actual amendment itself isn’t at play, at least some of the reasons behind that amendment are. We already have examples of limits on free speech in the offline domain – fighting words, shouting “fire” in a crowded theater, and the like – so the fact that there are limits in any given online domain does not preclude the existence of free speech as a desirable principle in its own right. It may well be a business problem in the short term, but if politics these days is anything to go by it doesn’t stay just a business problem.

          1. Kirsten Lambertsen

            Sure, I follow. I certainly don’t intend for anything I said to de-value free speech or freedom of anything. I just think it’s pretty clear cut that deleting a comment doesn’t constitute a violation of freedom of speech.I’m curious where you see this going in terms of your last sentence?

          2. Stephen Voris

            The gist of it is simply: while in theory it’s possible to fix problems one is not aware of – or that one refuses to acknowledge – in practice that approach is the source of the phrase “kicking the can down the road”, or its worse cousin, “kicking the snowball down the hill”.In American politics, for example, both of the mainstream parties have been increasingly polarized over the past couple of decades; I’m fairly sure that there are multiple causes to this, but one of the main ones in my mind is self-segregation: Democrats don’t feel welcome to post on Republican-leaning sites (and so they stop visiting them), and vice versa: the worldviews gradually diverge as all incompatible viewpoints are shunted to other blogs.For an example from a different perspective, China, as I understand it, has had rather substantial difficulties with the official statistics reporting what the local officials wanted to hear instead of the actual situation. It’s also had difficulties with its citizens posting unflattering commentary about its government (with varying degrees of sophistication), and with fixing the problems so posted instead of throwing the aforementioned citizens in jail.

    10. Prokofy

      It’s funny how this obvious fact which you’ve stated and I’ve stated just never gets to the top of these discussions although it’s the immediate issue. None of these companies are the government, nor do they enforce government law or provide civic goods in terms of “negative rights”. No one has asked them to take this role they usurp. But I can see the day where there will be lawsuits that get serious attention that prove that these companies have in fact defeated the civic good of free speech for say, a political candidate they didn’t like to the point where the First Amendment will have been neutralized by them precisely because of their dominance of the “public commons.” Imagine a world where the entire world of lefty media and social media didn’t ridicule or express horror about Donald Trump, but didn’t cover him and blocked the accounts of him and all his supporters. That world is not so inconceivable. Then what happens? Too many people would think “problem solved”.

      1. Kirsten Lambertsen

        You make a great point (even if I would think “problem solved,” personally, heh). But this isn’t something we’ve never encountered before, and it’s where there’s always been a lot of tension around balancing the freedoms of corporate entities and the right of the public to have choices. We have long had laws to prevent and dismantle monopolies.Whether or not our current models work now and/or need serious modification and/or amendment in order to apply to digital media and services is debatable.I think the big differentiating factor here is that with traditional media it isn’t possible for just anyone to launch their own newspaper or television network. It takes considerable resources to even start. The Internet is a powerful leveler, however, and nearly anyone can start a blog or website and has the opportunity (for now anyway) to gain a strong audience with very little resources (and many have). So, at what point does Facebook or Twitter become an actual monopoly? One might argue that one or both is already a monopoly, but I’d say we aren’t there yet b/c of the nature of the Internet at the moment.

  11. Bruce MacDonald

    Thanks for calling attention to the article Fred, not sure I would have read it otherwise. I agree that those people need acknowledgement.

  12. Mario Cantin

    Talk of long-form journalism: there are books shorter than this article! Nonetheless, it’s a must-read for any founder — or anyone — thinking of doing a startup that wants to build an interactive community. There are way more ramifications than initially “meet the eyes” that need to be thoughtfully considered.

  13. Twain Twain

    Moderation should be solvable with engineering, specifically AI — although it can’t be done by the same pattern recognition techniques that are used in image recognition and verbal speech recognition.At the moment, moderation relies on limited human resources of user self-reporting which is followed up by community moderators. That’s fine on sites like AVC where, to an extent, our community pre-filters itself for content quality and we’re talking a few thousand commentators who, mostly, use real identities.Less so on sites like Twitter, obviously.Unfortunately, Natural Language AI is simply not at that level of intelligence yet — as could be seen by how MS Tay chatbot on Twitter exploited the flaws in NL structures and amplified the trolling, racism and other forms of unacceptable speech behavior.Current Natural Language Processing (NLP) can do simple things like frequency count the incidence of black-listed words that may be offensive, e.g. the N-word. When that count exceeds a limit, the system flags the moderation team who then investigate.Current NLP has no frameworks for the subtleties and context of how language is perceived, though. For example, the N-word used amongst rappers and between people of the same African-American heritage isn’t perceived as offensive whilst in other situations it is offensive.Sure, NLP has corpus such as PANAS (positive and negative affect schedule) which does things like classify words so “sad” is negative and “happy” is positive.But, as the experts in AI acknowledge, Natural Language is the hardest problem to solve.For the last 50+ years, the structures in AI have been created by male researchers (so AI has had fathers training its brain).What it needs more than anything are “Mothers of AI” to train it in language — in much the same way our human brains are socially conditioned by our mothers to understand human emotions, values, concepts like trust and respect, art, beauty, humor and more as well as the technicalities of navigating the world. Mothers are even the human brain’s first points of reference wrt how to pattern recognize faces.Sadly, Silicon Valley (and elsewhere) has all sorts of issues around diversity and inclusion that make women not incentivized and encouraged to stay within technology generally.Then when we get into this particular specialism of AI there are even less women. That means an almost total absence of mentors for someone like me.In fact, no mentors because a lot of people I’ve met have told me to use the same tired, broken approaches and I’ve refused to continue perpetuating models that my intuition and scientific skills tell me are broken and that need to be solved.To say I’ve had to fight against loneliness (and wanting to quit every single day), chauvinistic and, often, patronizing dogma over the last few years would be an understatement.However, at least, FINALLY the Fathers of AI are acknowledging they haven’t been able to solve the Natural Language problem and don’t know where to begin to do so.Meanwhile, I already invented and coded a system.Natural Language is a really BIG problem because it affects so many areas of our online lives, including community moderation.The male CEOs and engineers are too busy with low-hanging toys like MS Tay chatbots to solve the harder technical problems in NLP that have much wider and deeper impact on technology.

    1. ShanaC

      last I heard AI can understand some sarcasm…

  14. Twain Twain

    @fredwilson:disqus – Re. “precious engineering resources,” the values incentives in startups mean that engineers are assigned to do everything to increase the QUANTITY of users not the QUALITY of users. This includes engineering A/B tests so the growth in MAU (monthly average users) makes investors happy.When MAUs are hockey-sticking, no one cares about the quality factors such as community curation.Worse, quantitative KPIs are presented as if they are proxies for quality of user experience when they’re not.So … there are all sorts of business model innovations ahead for us as well as technical innovations if we’re to solve big, common problems like community cultivation and startup growth wherein there are quantitative KPIs AND qualitative KPIs of different types to those that currently exist.

  15. AlexHammer

    I agree with you that it is nuanced. One the one extreme you have trust and safety On the other you have censorship.Perhaps a lot of the decision-making rests on culture, and how the company thinks of the user.What does the fact that you can’t (generally) contact Google for customer service say about the company. Are they encouraging self-reliance. Are they authoritarian?Is customer service an expense or a value add?Do people act better in communities in which it is possible to speak with a live person. Does this humanize the company?

  16. LaMarEstaba

    I worked in content moderation. It was called something else, but that’s what it effectively was. I remember on the first day seeing something that I thought didn’t even exist…and here people were posting about it in a public space. I read a lot of really bad stuff while I worked in content moderation. You don’t want to know what people do or think in the dark. They get reported, the mod team takes care of it, and the average Joe never has to think about it.Free speech is really important. I get that. I think that Youtube’s solution to put up a mature content button is a good one, and I hope that Facebook has some kind of mature content filter where you can opt in or out, with underage kids automatically locked out.Google has a landing page for “adult content” blogs, which are totally ridiculous. Google’s definition of “adult content” is a bunch of romance novel reviews. Absurd.I remember that Patrick Rothfuss (famous fantasy author) shared an article on feminism. In the preview pane on FB, there was half of a cartoon nipple (it was talking about clothing). FB shut down his account in response to him sharing that article with 10k likes.It’s not easy, that’s for sure.

  17. sigmaalgebra

    The article describes a lot of ugly, disgusting stuff.Yes, the article and the issue of content moderation is relevant to my startup in Internet content recommendation and discovery — I’ve known that.My first approach was to ensure my site has no content not safe for work (NSFW). But, of course, the article discusses much more than just NSFW or not.For my startup, my solution is, at least initially and likely for a long time, to reject all content unless there is a really good reason to include it. I have always felt that the hardest part of running an Internet business was insuring the trust and safety of the users Well, maybe not “the hardest part” but definitely an important part. Sure, user “trust and safety” is a crucial part of the service much as for the contents of bottle or can on a shelf in a grocery store. In a word, we’re talking about the brand of the service.E.g., for my startup, discovery and recommendation of music and still images? SURE! So, classical music from major orchestras, opera houses, concert performers. Images from, say, the world’s leading art galleries. Michelangelo? Renoir? Terrific! Robert Mapplethorpe? Likely just omit it! To decide on such content, too much botheration.Rap music? Not a chance, not for a long time. Some of that stuff is deliberately degenerate, degrading, and disgusting, and I don’t want to filter through it on the small chance of finding something worthwhile.Pop music, say, something like the Britney Spears “hit me baby one more time”? Nope. Ms. Spears was an astoundingly pretty teenage girl, but no way am I interested in filtering through and/or moderating pop music to decide when/when not to recommend such content.For videos of political demonstrations in countries run by ugly, brutal dictators? Not interested. Simple solution to such countries: If they attack or threaten us, then fix them so that they are no longer able to do any such thing. How to do that? A-10s, F-15s, F-16s, F-18s, F-117s, F-22s, B-1s and B-52s with JDAMS, etc. Otherwise just f’get about them. Civilian casualties? That’s what they are usually doing to their own people — including deliberate genocide including women and children — and very much want to do to us. So, if our JDAMs also result in some civilian casualties, note that the JDAMs also, net, reduce the total number of civilian casualties. Otherwise, just ignore such cesspool countries.For providing recommendation and/or discovery of such content? Not a chance. Not the market I’m going for.More generally, try to get nearly all the content from what are well known to be highly respected sources. Can do this? Sure: Even from only just such sources, there’s a LOT of content on the Internet.Q. But then your site won’t have comprehensive coverage of all the content on the Internet.A. And, you expected something else?The article seems to be an example of some of the techniques of journalism of getting readers all concerned, highly concerned, and, thus, of the standard journalistic goal of getting attention and eyeballs for ad revenue by grabbing people by the heart, the gut, and/or below the belt.In this case, the authors appear to be useful idiots, that is, personally highly concerned when, in my view, they need not be so concerned, and, thus, are being exploited to get eyeballs for their publisher.The authors are so highly concerned that — as no doubt correctly outlined in the article — the moderators could have their mental health and their health and lives more generally seriously damaged. Bummer. Slogging around in cesspools is not healthy; stay the heck out of cesspools.Why are the authors so concerned? I can’t respond to all the high concerns of the article, but it appears that heavily the authors are just stuck, obsessed, about freedom of speech for various noble political goals of alleviating human suffering and pushing back against bad politicians, etc.To me, such motivations are part of the common liberal distaff compulsions to save the world.There is an easier solution: Don’t strain so hard to try to save the world that have to filter through disgusting sewage.The article is an example of the journalistic technique of borrowing from formula fiction borrowed from the dramas of the ancient Greeks: Start with a person, show that the person has a problem, let the audience become concerned about what happens to this person, then keep the attention of the audience through to the end of the situation for that person. That is, there is a story with a protagonist.The article is also an example of the journalistic technique of taking a problem, making it seem as bad as possible, keeping the attention of the audience by keeping up the claim that the problem is really bad, discussing lots of poor solutions and deliberately avoiding discussing simple and obvious solutions — e.g., accepting content only from what are well known to be good sources.Then, to keep up the high concern, claim that there are big issues of human rights, freedom of speech, democratic principles, rule of law, equality of women, protection of children, etc. Of course, in a lot of cesspool countries run by ugly dictators, correcting such situations is essentially hopeless, maybe even for another 1000 years.E.g., for YouTube, accept submissions only from approved sources. To be an approved source, have to apply and supply, say, several pieces of obviously good content. Simple.

  18. V Tyree

    Hello, It seems that this post has resonated with a lot of people! I think that the humans (if, in fact they are humans), hide behind anonymity; or they think they are anonymous. They forget that their vulgar, profanity-laced comments NEVER go away. My conclusion is they are in a low mood. And, there are simple ways to address the issues. Be polite, free speech is important, humor, and consistency are helpful. When they see their nonsense is not encouraged, they go away… This has been my experience. Be well-keep posting. v P.S.: Thanks for the post and all the posters for their input.

  19. howardlindzon

    I don’t ever buy that there are free speech rules. There is policy the company’s CEO and maybe board, investors agree on. If StockTwits allowed penny stocks and spam from day one we could have likely raised way more money on bigger growth numbers but we would have had a cesspool and even so it is impossible in real time and in a pseudononynous internet to have spam and troll free environment. The hate mail we get is absurd despite relentless community efforts. Phil Pearlman and @chicagosean and @zerobeta and I set tone early often and with basic house rules that are the sites Verison of free speech.

  20. brittblaser

    We fixed comments in our new bolt-on group-forming, buzzword-compliant PaaS called “vEngine”, honoring Vasil Savovski, our brilliant developer in Belgrade. I’m Britt Blaser, CEO of the NewGov Foundation and Vasil’s Head Cheerleader.If I can back up that outrageous assertion, could someone at USV advise the NewGov Foundation on the best way to serve NGOs, politicians and companies? We think the service is better than Jive, and so do some Jive users.FWIW, vEngine’s bigger deal is IaaS – Influence as a Service, meaning political influence. it connects the NGO’s members to their politicians at 8 levels of government, in 51,231 jurisdictions.Imagine that Ms. Union S. Venture lives at 915 Broadway, NYC. With opt-in, vEngine tells her dozens of reps that Ms. Venture lives in their district. Her constituent hashtag is #PZNY122875-51000,2 and it’s shared with a few thousand of her neighbors so it’s not personally identifiable. Her constituency map is at….So, if I’m Rosie Mendez, the NYC Councilperson for District 2, I’m interested in any hashtags that start with “#PZ” and include “51000,2”.

  21. ShanaC

    This is interesting for me to watch, in part because I am part of a banned then reconstituted facebook group that facebook works with closely about content moderation.The reasonIt is a semi-private group for previvors, specifically geared at the under 40 crowd (aka people with strong risk for Breast and/or Ovarian cancer^1). The first version of the group, which I join around 22-24? had before/afters of full mastectomies and reconstructions of various types, including nipple saving, tattooing nipples, no nipples, what it looks like to have expanders, ect, ect, ect.^2 Even though these images weren’t sexual, they were really graphicThe group eventually got banned because Facebook really couldn’t keep making exceptions for all the nipples. Plus because of the sheer amount of discussion, images being posted, debates about what to do, the images kept overloading feeds at really inopportune times . (there was a point where I actually hid there stuff from my feed because otherwise all I would see is boobs in various stages of surgery). On top of that, because the images were graphic, they would almost always be eventually caught in a spam filter at some point because of the way AI around images work, which also created a lot of tension with Facebook.End result, the group got banned, then there was an outcry, the group was reconstituted, but now all the images are hosted elsewhere and I think for that group if you want to look at images, you have to go to a specific site to see what the process looks likes. At the time it causes a commotion, but at the end of the day it actually ended up working out really well and ended up being one of many factors involved with building out some types of moderation tools for groupsThe article is right – people who are involved get stuff built.^1If you ever want to have a fun conversation with me, it would be about big data, genomics/epigenomics, and AI^2 Yes, I’m high risk, which explains me seeing so many pictures. No, I am not going for surgery any time soon. In fact, because of issues involving data, I don’t even qualify (and if I did, I still wouldn’t do it right now).

  22. obarthelemy

    I think trolling is an important (bad) part of the comments issue, but getting good comments is more important. Actually, trolling is not so much bad per se (quality comments sections will downvote them into oblivion), as bad because it drives away “good” commenters.It’s also not purely about moderation. If you write clickbait, you’ll get trolls. If the pieces’ writers and other staff show up in comments,you’ll get less. I think troll handling is a part of a larger, community-building issue.Also, the definition of trolling is fluid. I’m currently life-banned on 2 sites, and have been spot-banned on a third one several times, not so much for trolling (as in insulting, menacing, kicking up a fuss for no reason…) but for disagreeing with the site’s mods/writers.

    1. Susan Rubinsky

      If you listen to Trump, how has he not been downvoted yet?

      1. obarthelemy

        Because we’re still waiting for good contributors to provide an alternative, and a good community to spot them ?

  23. cavepainting

    Great long-form journalism !The mental and psychological toll on the people doing these jobs has to be extremely high. What we see influences us in more ways than we think and give credit for. People like to think they are strong minded, but no one really is when it comes to witnessing such depravity. It is bound to leave deep impressions.What is a disgrace is that they are paid a pittance relative to developers, given what they have to go through every day and carry with them for the rest of their lives. At the least, there has to be a hazard premium when we ask people to step into jobs that put their mental health at risk. Yes, the policy and free speech questions are important but they will always be tough issues to grapple with in any civil society and have cultural dependencies.The bigger question is how best to minimize the impact on the people doing these jobs, and how to ensure that they are fairly compensated and taken care of. Clearly, technology can make a big difference in automating detection, self-learning policy based on rules and past data, and surfacing potential exceptions. In the long term, I hope this is a job that can be done mostly by AI with minimal human oversight.

  24. Susan Rubinsky

    Online dating is being destroyed by this. I have given up on it due to the vast amount of rude (often pornographic) messages I receive. The online dating services (I’ve tried practically all of them) put the onus on the user to report, block or hide the abusers which is ludicrous.

    1. Stephen Voris

      I (still) think the solution here would be “you get one free message to/from a new person a day; the rest you have to pay for.” The present user-must-report paradigm is useless for anyone who struggles to reach inbox zero, and for any actual person who wants to talk to one of those strugglers.

      1. Susan Rubinsky

        I would gladly pay a subscription if it reduced the inappropriate comments and the spammers. However, I have seen this type of activity on both paid and free sites. If Google has effectively created algorithms that have removed spam from my in-box, I don’t see why the online dating sites don’t do it.

        1. Stephen Voris

          At a guess, “because Google has already hired most of the people who think about these things enough”. As for paid sites also having this, I’m guessing (based on my own experience, but nonetheless guessing) they were subscriptions on a per-month rather than a per-message basis; the former doesn’t provide quite the same anti-spam incentive as the latter.

  25. Stephen Voris

    The dangers when it comes to moderation – in my mind, at any rate – are that it becomes a force for ideological segregation, and that it becomes a force for cultural imperialism. The latter may well be inevitable – after all, who doesn’t think their own values are correct?As for the former, “becomes” is probably the wrong term – users self-segregate already – but there’s no clear dividing line between “difference of opinion” and “troll”, nor between “honestly thinks a product is useful” and “spam”. There are factors that a good moderator will use to break the two apart – but not all moderators are above average (by definition), and as the article notes, the automation ain’t there yet (and even if it were, it’s unlikely it’d be publicly available… startup opportunity, perhaps? Selling moderation services to companies small enough not to have their own staff for it).

  26. Drew Meyers

    A topic near and dear to my heart… the least fun part of my job at Zillow (2005-2010) was certainly the nights/weekends spammers got ahold of the site and I had to spend hours manually deleting content.

  27. ajabog

    Fred, I’d be interested in hearing your thoughts on what we’re building here at Civil: https://www.civilcomments.comWe have a new approach to comment moderation; we’re calling it ‘user-moderated content’. Our customers are seeing dramatic results; with our crowdsourced peer-review system, they’ve had a 90% reduction in time spent moderating, increased engagement, and a wider range of participants. Testimonial video here: BogdanoffCEO, Civil

  28. Prokofy

    Tech libertarians become too clutched up about their imagined role as “providers of free speech”. The First Amendment is about *government* getting out of the way of free speech and the obligation of *government* not to suppress free speech. Providing the physical or virtual venues for the enjoyment of this right is *not* the government’s job; you have free speech because the government does not get in your way (it’s called a “negative right” for that reason) but you are not entitled to a printing press or an Internet connection — you have to pay for those things unless well-meaning but destructive leftists try to insist on government paying for broadband for “poor people” to “obtain education and job opportunities” — in between their bouts of game-playing and porn-viewing, of course.The reality is that no one needs tech companies to supply this civic good and they are absolutely terrible about supplying it for all the reasons Verge says, and because of their culture of suppressing speech they don’t like, and allowing speech they do like, i.e. Facebook’s selectivity on racism against blacks or intolerance of gays versus anti-semitism or Twitter’s boundless leniency on misogyny but trigger-sensitivity on discussion of race — because the left has learned that the quickest way to police the right or even someone two degrees from them on the left is to invoke “racism.”The answer to this as I’ve said for years is only one thing: the market. There must be a market place in free speech. If you want a “shock site” as the worker called it in the Verge story, you go there, usually anonymous and snicker with other channers or ebaumers. If you want more serious and responsible discourse, say, on politics and culture you go to Facebook where identity is king and your real name and real reputation are usually on the line.Twitter harassment by anonymous Kremlin trolls got you down? Go to a Facebook page or group you and your colleagues control and get rid of them. And so on.The problem here is that because these services are free, neither they themselves or the market of the public can really properly value them, but when the connection between payment of a subscription and a far less troll-free experience become united in the public’s mind, this will be a profitable area for everyone.So that has to happen — people have to take their business of speech to the place that supplies the kind of it they want and need best, whether 4chan or Twitter or Facebook or G+ even which isn’t worthless. No one of these entities alone has the awesome challenge of “supplying” free speech to us even in an era when they have too much control over the public commons. None of them can or should be in the business of deciding the limits of speech for all of us as if they are governments. The answer is not to liberate these spaces in the valuation sense and make them even more free or God forbid get government involved in them even more, but to subject them to the marketplace of ideas and dollars.

  29. LE

    I famously got into a big fight with an ex girlfriend over the James Frey fraud book “A million little pieces” that fooled Oprah. We were flying on a plane somewhere and she was reading the book. I picked it up and read a few pages and said “this is fake no way he could remember all of this detail if he was truly drugged the way he says he was”. She was so pissed off at me for that. Later of course I was proven right. I think part of it was not so much that I am a skeptic (in this case) but that I didn’t let emotion overcome logic (or something like that).In the end of course with respect to Frey I say “better to have loved and lost than never to have loved at all”. In the end we know who he is and we don’t know about the honest author who played by the rules, eh? (And there are other cases like that and I expect this will get worse not better in society.)I suspect at one time or another one of the “gang members” would have pointed the gun at the reporter and forced them to smoke crack with them.I didn’t read the article however to get by that the writer could have social engineered any number of reasons why they couldn’t smoke. Like a health reason or some other thing that seemed believable enough. To me it’s suspect but it’s possible to overcome that type of situation with the right preparation in advance since you are playing (just like in negotiation) a role like an actor.