Facial Recognition

I would like to start this post with a disclosure. USV portfolio company Clarifai has one of the best facial recognition models on the market and is very active in the facial recognition market. Now that I have disclosed that, we can move on.

Facial recognition has come of age. Machines can figure out who we are and more.

One of the most popular booths with students at The Annual CS Fair this year was the Microsoft booth where they were showing some of their facial recognition technology.

The delight and amazement on the students’ faces was infectious.

But of course, not everyone is excited about facial recognition technology being deployed in the market.

I particularly like that question in the embedded image in that tweet:

How does Jet Blue know what I look like?

The answer turns out that there are many ways to know what we look like and you can start with the federal government and go from there.

Like all technologies, facial recognition can be used for good and bad. And it will be.

I like what my partner Albert wrote on this topic recently:

And then some things are incredibly hard. Such as face and object recognition. There are tons of amazing positive applications for such technology. And yet they could also be used to bring about a dystopian future of autonomous killer weapons chasing citizens in the streets. Does that mean we should not develop these capabilities? Should we restrict who has access to them? Is it OK for corporations to have them but not the military? What about the police? What about citizens themselves? Those are hard questions and anyone who thinks they have obvious answers I submit hasn’t thought long enough about them.
So what is to be done? A good start is personal responsibility. 

We used to have to stop at toll booths and wait in long lines to get across bridges and tunnels. Now we drive past the tolls at 60mph and the machines detect our license plates and debit our accounts.

The same is going to happen with our faces and that will be great for many things. But, of course, it will also freak us out on a regular basis and add to the “technology is turning everything into a surveillance state” narrative that has more truth than we would like to admit.

So what is my point? Well for one, the technology is here and we had better get used to finding it deployed in the wild. And second, that it will bring a lot of good. So we should not over react. But we should be mindful of the downsides and those of us who are working on this technology, those of us who are financing the development of it, and those of us who are deploying it, need to take great care with it.

#machine learning

Comments (Archived):

  1. jason wright

    There has to be a way to outfox this technology.

  2. William Mougayar

    I would love it if they use more facial recognition at airports to speed up security and checkpoint delays. Click my face, please.

    1. awaldstein

      On this one thing, we agree.

  3. Tom Labus

    Our town in NJ recently swapped out the parking meters. The new ones requires that you put your license plate # in to activate them. There are new cameras also installed. Shop owners and everyone else seem pretty pissed.

  4. creative group

    CONTRIBUTORS:”But of course, not everyone is excited about facial recognition technology being deployed in the market.” – fred wilsonYou don’t say!Let’s not allow scientific evidence get in the way of money. After we make our money we will declare we will do better and be better stewards. Just say we are sorry and they will get over it.https://www.marketwatch.com…Captain Obvious!#UNEQUIVOCALLYUNAPOLOGETICALLYINDEPENDENT

  5. Ryan Frew

    I’m in the camp that’s generally comfortable with facial recognition, but with the obvious caveat that we need to be cautious with it. The JetBlue conversation gave me pause. Using facial recognition at security checkpoints? Sure, makes sense. But at the bridge? Does that benefit anyone besides JetBlue? If safety isn’t in the question, I don’t think DHS should be choosing to distribute my passport photo to private companies. The precedent just isn’t necessary.Disclaimer: I work for Microsoft. Doesn’t seem particularly relevant, here, but they were mentioned in the post so ¯_(ツ)_/¯Happy Easter!

  6. Dan G

    I’m ok with companies, organizations, agencies having my information, as long I have control with what I provide, I also benefit from providing my information and trust that information is secure, and there are safeguards to my identity if there’s a data beach.For the most part, this my situation with Google. I like Google, the many things they do, and reasonably trust the company with my data. Actually, it gets annoying when it shuts down a service I like- Reader for one, but now Feedly is as useful. I use many many GB of data per month, last cycle it was 120, most of that used on Google- Youtube, Music, Photos, Maps, etc. Traffic info on Maps saves commute time, Drive and Photos makes it much easier to keep records, many other benefits etc.I’m more reluctant to share data with new, unproven companies that I don’t really know about. If there was some sort of mechanism that builds confidence in these largely unheard of companies, then I’d be more apt to sign-up for their services.

  7. Gustavo Melo

    Here is a doozy. If you develop and sell facial recognition technology as a product, and an organization with an alleged track record of misusing technology to do what is generally considered to be bad things wants to buy your product, should you consider this track record before selling it to them?This question is important because as always, technological innovation far outpaces our government’s ability to regulate it, and anyway governments at large (including ours) have developed a reputation for being examples of such organizations so their speed and ability to effectively regulate is suspect at best.I am optimistic about democracy and believe that in the long term we will end up with a good set of laws and guidelines around what’s permissible with technology that invades our private space. But what about the tremendous damage that bad actors can do with this tech in the short- and medium-term?This isn’t an easy question to frame correctly and there aren’t easy answers either, but I think it’s a foundational issue of our times and will define a big part of the legacy of our generations.

    1. bogorad

      This is presuming you can do anything about the development of technology. Which you can not. Just look at Chinese genome experiments. Someone will do it anyway.

  8. bogorad

    As someone who successfully introduced an FR-system to the Customs Enforcement in the largest Russian airport (SVO), some four years ago, I’d like to clarify a couple of things.First, we’re in the very beginning. Truly useable algorithms started to appear after Google published its FaceNet paper mid-2015. It was mostly alchemy before that, including the systems previously employed by the US government (DMV, State Dept, etc.). The error rate was appalling, and although useable in some situations, useless in most real-life applications. All those billion-dollar valuations, oh well. As these algorithms mature, expect the computational requirements to go down, as well as error rate, which brings us to my next point.Second, there is absolutely no difference between an excellent algorithm and a live person who remembers a million faces. Do you object to people looking at your face in the street? What about genuinely panoptic and invasive cameras in London and other cities in the UK, monitored by such an individual? There is no practical way of avoiding it. So why bother even arguing? Anyone with a little bit of technical skill can buy a camera, a computer, and join a network of guerilla-style people-watchers.What are you going to do about it? You could supposedly elect a different breed of politicians who could outlaw these practices (imagine taking down all cameras in London, haha), but there is absolutely nothing you can do against a guerilla network run by individuals.In any case, the current use case is a straightforward one: the airline requests pictures of passengers from the DHS. I do not know the protocol they use, but it is theoretically possible that the airline doesn’t even get the photographs, just hashes that can be used in search by a computer, but not by a human (since it’s impossible to re-engineer the image from a hash). So matching some 200 people to 200 photographs is an effortless task. Even the older, ‘alchemic’ systems were quite good at it. But matching a face to a hundred million – that’s a totally different story, even today. But it might not be tomorrow ;)UPD: Looks like the airline just feeds faces to the DHS servers and they make the determination. Which changes nothing really.

    1. Richard

      What’s the breakthrough in the current 2015+ algo?

      1. bogorad

        Machine learning, of course.

        1. Richard

          Well there goes your credibility. ML algos have been around well before 2015. Fred stated that clarity has the best facial recognition algos. Is there a proprietary algo being used ?

    2. daryn

      Second, there is absolutely no difference between an excellent algorithm and a live person who remembers a million faces.There are actually two big differences, both of which favors the AI over the human (besides that no live person could remember a million faces).First, the algorithm is always on, doesn’t get fatigued, and can multi-task much better than one individual. Compare that to the security guard who is bored and half-watching a set of cameras for activity.Second, we can be more intelligent about biases, by being intentional about the content of the training data and its diversity. There’s a lot of work that goes into this process, and there’s still a ways to go, but ultimately we’ll be better off than with a human.

      1. bogorad

        In terms of total surveillance scare there is no difference. As to biases etc – as I said, we’re in the very beginning, everything will improve eventually.

  9. Richard

    What makes their system one of the best ? Are the algorithms proprietary? It seems lime there is very little proprietary in the classification data analysis space ?

    1. kidmercury

      No one knows how good it is, and how to measure accuracy is also very conditional. But I would expect all these commercial systems to be 99%+ in terms of a simple confusion matrix, hopefully six 9s.

      1. Richard

        When a VC (Fred) calls out their facial recognition product to be one of the best, is it just trumpian puffery?

        1. kidmercury

          That’s a fun troll. though I think probably many believe statements like that, or want to believe it, but really there is no way to prove you have the best ML algorithm when you don’t know what others have

          1. Richard

            I suppose questioning can come off as trolling when you toss in kicker. I studied ML 10 years ago – was wondering if the algos have advanced.

          2. tolstoy77

            10 years ago is when things really started to pick up and get interesting with deep learning. Launches of competitions like imageNet really changed the landscape for vision. Lots of neat stuff out there now.

  10. Lee Schneider

    What good will it bring apart from convenience? I realize that it is here and can’t be stopped. But I see more downside than upside.

    1. kidmercury

      Convenience leads to other innovations. Pay or validate with your face leads to shorter lines, fewer ushers, which holding other factors constant will lead to lower prices. How many venues and event planners struggle with fraud and cannot hold certain eventse because of it? also how about Airbnb, unlock the door with your face?

      1. David Gobel

        this is true. However, our faces are the most personal asset we have excepting private thoughts…and even these private thoughts are written on the face over time…and this time series of facial reactions will become recorded, analyzed and available for sale with omnipresent surveillance…leading to disclosure of presumed belief patterns of individuals without the knowledge nor permission of the one with the face.The 4th amendment of the Constitution guarantees the right of the people to be secure in their persons, and applies to states and municipalities via the principal of incorporation as impelled by the 14th and 15th amendments.In the US, corporations and businesses are creatures of government by granting of license or corporation charter.Unlimited facial recognition’s ubiquity and wild westness will inevitably rise to the attentions of the Supreme Court.We are seeing a preview of this in the “Social wars” where corporations sell people’s private information and thoughts and worse, to run roughshod over First Amendment rights to free speech by invoking the principle of private property. Marsh Vs. Alabama clearly states that “company towns” cannot preempt free speech. That’s how this issue will end.I predict the principle of incorporation will be extended to devolve to commercial and non-profit enterprise for facial and other forms of public recognition as well. It will be a truly fascinating (pun intended) decade!

    2. ann marie clarke

      Yes you are right they just put you up and follow you around for no reason

  11. Kirsten Lambertsen

    Reading that JetBlue conversation, all I could think was, “So they can invest in facial recognition technology but not more leg room for coach class, more overhead baggage space and eliminating baggage fees?”Time to invest in products that trick facial recognition technologies 😉

    1. Michael Elling

      Called a millinery.

      1. Kirsten Lambertsen

        Ha! Sheet facial mask is more fun though.

  12. awaldstein

    Benefits of this are boggling and I embrace the upside more than lament or fear the downside.Seems like there can be frameworks established to draw a hard line on what is ok and what is not.

  13. alwayslookaround

    “…those of us who are working on this technology, those of us who are financing the development of it, and those of us who are deploying it, need to take great care with it.”As an investor who is financing this kind of technology, what level of care are you putting into it? What stipulations are you putting in place with your investment? How are you ensuring that the companies you invest in are not going to use this technology against your wishes? What are your wishes for how facial recognition will and won’t be used?The current situation with Amazon selling their tech to governments is a bit alarming, especially their pushback against the desires of stockholders. How can this scenario be prevented in the future with other companies?Also, if you’re willing to share more of your personal logic around this topic…knowing that this kind of technology will not always be used for “good”, and will inevitably be used for “bad”, how do you justify investing in this tech? Is it just for monetary gain?I can’t help but think of the development of the atomic bomb, and the regrets that many of those involved had about their work on the project.

    1. fredwilson

      it starts with our board work, keeping management accountable to the board, and asking the hard questions

  14. Pointsandfigures

    Depends on your govt. What they are doing in China makes the book Brave New World seem like a how to manual

  15. kevando

    “Now we drive past the tolls at 60mph and the machines detect our license plates and debit our accounts.”If thats your positive narrative, Yikes!

    1. Michael Elling

      You prefer to wait in line?

  16. Brent Naseath

    “And yet they could also be used to bring about a dystopian future of autonomous killer weapons chasing citizens in the streets.” Instead of the extreme unlikely example, why not use a real-life current example of China using facial recognition to help assign citizenship ratings with punishments for bad citizens? We already allow ourselves to be controlled to “have the privilege” of driving and flying, i.e. moving around. In China, that is restricted if you are a “bad citizen.” We view every potential crime now on par with a terrorist attack and as a whole (and from the other comments) seem willing to give the government total control to protect us. This dependent attitude (fostered by the government) will change society. It is not privacy that is threatened, that has long since been lost. It is independence. We think of freedom in terms of life and death examples, not in terms of increasing societal controls where you are branded “bad” or “crazy” if you do not accept the beliefs and values of the majority.We are already seeing that information is presented according to our actions and past choices, enforcing an individual’s beliefs and biases. This will further that and allow companies and the government to further profile us, for good and for bad. We can’t stop this, but the good AND the bad of facial recognition will be here VERY soon, not just in some remote scifi example. Life will likely be safer, easier, and much more controlled.

    1. Michael Elling

      Except when there are different measures of right and wrong. When dissent is considered a wrong or crime then independence goes way down.

  17. OurielOhayon

    I don’t think you can put all face recognition in the same basket. It is important to separate two categories. Those that private by design and built on device and cannot be accessed or created by anyone but the user (eg faceID) and those that are services in particular unencrypted designed as in the case of jet blue to be used as a deanonymiser. Labeling those two the same would create unnecessary tension and confusion on the first one which is not susceptible to systemic hack and privacy holes.It’s like talking about blockchains without making the distinction between decentralized servires échanges and wallets and centralized servies like coinbase and others. They belong to the same category but do not operate the same risk or trade offs.

  18. Cristiane Sens O. Bastos

    Last year I visited the Canton Fair in China. It was my first time in the country. My arrival airport was Guangzhou. They checked my passport and also screened my face. At the Canton Fair they also use facial technology. I was happy not to be in line for more than a few minutes at each location. A friend of mine that lives in China mentioned that this is very standard there now and police locate criminals in matter of minutes. I am excited to see the bad guys located rapidly more than I am afraid of it. I agree that there should be a discussion about what is appropriate and what is not, but that can’t stop the advances and practical uses it will from adopting it.

  19. tolstoy77

    Feel like Clarifai is a company with so much potential but bogged down by an incompetent CEO. Never seen a company go through so many VPs so quickly. Hope the best for them and that they get their act together.Also while I can see that their face detection model has significantly improved recently, strongly feel that their real secret weapon is the ability to equip developers with ability to quickly build powerful image classification models.

  20. Mike

    Facial recognition is one of many new biometric identification tools and more are likely to come in the future as people come up with clever ways to apply advanced processing algorithms to extract unique individual signatures. Voice, gait and other similar gestures will be used for identification. Internal biometrics like heat rate.Fingerprints have been used since the early 20th century? And now common on smart phones with optical sensing technologies. What might be alarming in some cases about facial recognition is that perhaps this biometric can be obtained and utilized without the subject’s knowledge? You generally know when someone, or some device, is taking your fingerprints. Whether you consent to the process depends on the circumstances, but you know the biometric is being obtained.I agree, the development of new biometrics creates interesting possibilities and use cases. Different biometrics will be more suitable to certain applications depending on power, accuracy requirements (false detect/reject), speed, convenience, cost and other specifics of the use case. Anyone who travels can probably speak to the challenges of the modern airport experience. I think this is an example of Jet Blue using a new biometric technology to try and improve the customer travel experience. I expect there will be some trial and error involved and an ongoing balance between privacy and convenience.

  21. Richard

    Facial recognition is totally needed. Sri Lanka is just another example of just how many screwballs exist. When the denominator is 6 billion, it doesn’t take much to cause chaos.

  22. Michael Elling

    There are many examples in our society where there should be outside audit groups to hold “closed” institutions to account. This is not only important for government agencies, but particularly corporations. To wit, Disney’s pay structure. There are simply no more checks and balances in our, ironically, over-regulated society.