Feature Friday: Photo Search

I’ve been uploading my smartphone photos to Google Photo for my last two phones. Earlier this week, I was in a meeting with some architects and I said that I really liked the way they do the showers at the Soho House in Berlin. They asked if I had a photo of them. I opened up Google Photos, typed “Soho House Berlin”, and got this result.

soho house berlin

Sure enough, I had taken some photos of the shower. I showed them to the architects and we were able to talk about the features I liked in the shower. That was kind of magical because other than taking the photos, I had done nothing to tag or categorize them.

This works for all sorts of searches. I remember seeing a painting I really liked at The Hammer Museum in LA. A search on “Hammer Museum” produces the image I was looking for.

hammer museum

If you are looking for photos you took on a trip, you can do the same thing. Here are some photos I took of Grand Bazaar in Instanbul:

grand bazaar istanbul

Google Photos isn’t perfect. Some searches that I would expect to work don’t. But it is pretty good.

Our portfolio company Clarifai has a similar service in an iOS app called Forevery. If you don’t want to upload your photos to Google Photo and want to search them locally on your iPhone, Forevery is a great way to get a similar experience. Forevery will also search your photos in Dropbox.

Since I’m on an Android right now, I am using Google Photos but I use both apps on my iPhone.

Photo search is amazing. You no longer need to create albums and tag and categorize your photos to be able to find them. You just search for them. Kind of like how email changed when Gmail arrived.

#machine learning

Comments (Archived):

  1. William Mougayar

    I’m getting zero search results for anything I try. Do you have to enable something prior?

    1. Gustavo Melo

      Do you often have Google Location Service turned off when you’re taking photos on your phone? Without location information attached to the photo, a search for ‘Soho house Berlin’ e.g. would probably not have return results.I take photos of whiteboards all the time at work, and that’s another example of search usefulness – I can search for ‘whiteboard’ and it will pull up just those.The photo app itself should have the ‘back up & sync’ setting turned on, so the images are actually going to Google’s servers – the search magic seems to happen there.

      1. Susan Rubinsky

        I do this too 🙂

      2. William Mougayar

        thanks for those tips. i’ll play with it.

  2. PJ Kershaw

    Fred. I’m with you that it’s amazing, however the caveat is what are we giving away in privacy. You think about this (DuckDuckGo) so how come you don’t worry about this at this level for convenience?

  3. Varun

    Gorilla fallacy as CV caveat – http://www.wnyc.org/story/d….Clarifai is a great piece of applied computer vision and CV is going to be a game changer in many great and not so great ways.Here is Adam Berenzweig, Clarifai’s CTO speaking at Data & Society – he used calrifai and a life logger to show other interesting everday applications of Clarifaihttp://www.datasociety.net/…i’m just waiting until someone uses CV on the stanley cup gigapixel image …http://www.gigapixel.com/im

  4. scottythebody

    I want to see what’s so cool about the showers

  5. JimHirshfield

    Who takes pictures in the shower?

    1. Mac

      Thank you for asking that question.

      1. JimHirshfield

        ‘Cause you were too drained to ask?

        1. Mac


    2. panterosa,

      Design weenies like me. I may have more shower pix than Fred, not that we’re competing:)

    3. Jess Bachman

      There is precedent for this.

      1. Kirsten Lambertsen

        Is that Scoble?

        1. LE

          Yes it is. I was wondering if Scoble was still relevant. According to google trends (see below) he’s not. Looks like he got a bit of a bump right after the intro to google glass but now interest in him appears marginal….

          1. jason wright

            when i see the photo i understand the graph, although it looks a lot like Philip Seymour Hoffman on drugs.

          2. LE

            Yes, Scoble totally looks like the late gay Phillip Seymour Hoffman character. Boogie Nights Clip below of pool scene.https://www.youtube.com/wat

        2. Jess Bachman

          Yep, RoSco in the flesh.

          1. Kirsten Lambertsen

            If that was an animated gif, it would break the internet. Awesome!

      2. Kurt Kumar

        mind you the only precedent available 🙂

    4. pointsnfigures

      not going there.

  6. cavepainting

    Google Photos sucked in almost ten thousand photos and videos of our family across multiple devices and disk back-ups, auto-tagged them by face, location, background etc. and allows us to search simply and easily. Yes, it lacks some capabilities, but is still light years ahead of other options. No wonder it has gotten to 100 M users.Succeeding in photos is at the intersection of a lot of technologies: cloud, face and object recognition, machine-learning, handling multiple formats, dealing with very large scale etc. Apple icloud, Dropbox Carousel and other tech companies seem to be really behind. A good first step may be for one of them to buy Forevery – which does a pretty good job in recognizing photos.

  7. mikenolan99

    Had the same experience the other day – needed my son’s license plate number. Remember having taken a photo of it – typed “Audi” and up it came. Read the plate off of the photo.And… Mrs. Dr. Jules and I a re-doing the bathrooms… post bigger photos of cool showers!

  8. BillMcNeely

    Just went to samsung galaxy note 5 . Thanks for the note on Google photo

    1. Susan Rubinsky

      I LOVE my Galaxy Note. I’m on my second one. It will save everything for you!

      1. BillMcNeely

        Pretty easy to use. More memory , works with Google products camera is nice pretty easy switch from iPhone

        1. Susan Rubinsky

          I think it’s the best smartphone camera on the market and I also provide photography services to my clients and have a collection of high end cameras. It certainly says something that the Galaxy Note finally enabled me to ditch my high-end pocket camera.

    2. creative group

      Bill McNeely:How do you rate the Samsung Note 5?Not being able to remove or purchase an extra battery. (Copying the Apple model) Inability to expand memory..

  9. LIAD

    The examples you give are location based results.What I find impressive is their image AI.Search for shower and see if they come up. Search for art and see if they come up. That’s when it feels like magic.Needed a photo of my grandmother earlier this week. Searched for Grandma. Bunch of photos of elderly women came up. Didn’t return the photo I wanted. Searched for grey hair. Loads of false positives but the photo was there. Saved me trawling through 20k+/images.Auto backup. HD images. Unlimited. Cross device. Intelligent. Free. Not much more you can ask for really.

  10. JaredMermey

    Searching of files in all of its forms (apple Finder, google photo image search, etc) has nurtured incredible laziness into my directory organization skills.It is not a terrible thing for personal directories, but shared drives are becoming a mess.

    1. Susan Rubinsky

      I still organize most of my photos and other files, believe it or not. Sometimes I have clients call me and I can find their stuff quickly. Recently a client called whom I have not worked with in over five years. He lost his password to an account and had a different email address now. I was able to find it for him in seconds 🙂

  11. creative group

    FRED:We get hyped when you share your worldly travels. On occasion we can reminisce about destinations we have traveled also. But our travels pale in comparison. This is what we don’t get excited about. You promote a service (Google photos) and then slide an Envangelical Chief promotion for one of your portfolio companies that doesn’t service the largest group of people which is Android and getting good larger. We get it but don’t want to get it.

  12. jason wright

    where’s the magnification?

  13. awaldstein

    I’m a fan.Photo search was one of the last pieces of legacy confusion in an otherwise searchable and organized world.

  14. jason wright

    so how is the P6?does anyone have the new P9, and how is the Emotion UI? i hear it’s not good.

  15. OurielOhayon

    i just wished google photos took a very little extra step and automatically set aside screenshots (easy) and documents (a little more difficult) aside in separate folders

  16. LE

    I was in a meeting with some architects and I said that I really liked the way they do the showers at the Soho House in Berlin.If you are doing a shower with a tile floor it’s super critical to make sure that, after the pan is done and the tile is laid, it drains perfectly from all edges with no standing water. Otherwise you will be in squeegee hell. Don’t trust or take someone’s word for it. Personally test.

    1. creative group

      LE:Should the design of the tile shower have a slant to prevent the water from pooling?Does the type of drain matter? (Laticrete Hydroban and the Laticrete bonding flange drain) The slopes should come before the tiling and not during the tile install. No puddles is industry standard when employing a honest and good contractor. Unlike a Politician a good Contractor exists.

      1. LE

        Sure the slant is exactly what matters and is created in part by the bed created under it in cement. Yes the slope is done before the puddling I don’t think the type of drain makes any difference at all.

        1. creative group

          LE:Thanks for replying.

  17. pointsnfigures

    I have to use Forevery. Store all our photos on a separate hard drive, and I can never find photos when I want to show someone something. The bazaar in Istanbul was pretty fun. However, my most moving experiences there came at the Hagia Sophia and Chora Church

    1. awaldstein

      I seem to be the only person whose fondest memories of my many trips to Turkey are the gaggles of cats everywhere–from every alley, hotel lobby to sitting on every statue in every place i traveled to. A beautiful country with seriously screwed up human rights issues and a government that has done everything it could to suppress a possibly great artisanal wine industry.

      1. pointsnfigures

        Love the cats, but I am allergic to them. I didn’t see a lot of mice or rats like I do in Chicago. Istanbul is an incredibly interesting city. I found in most sections of the city people to be similar to me. Of course, there are sections where it’s not safe for Americans to go anymore and that is sad.

  18. LE

    I’ve been uploading my smartphone photos to Google Photo for my last two phonesWhat would be good is if you could upload very low res versions of the photos to Google Photo and then have them indexed to the hi res version that you store locally on your hard drive.That said the amount of times that I would need this is not outweighed by the fear of having photos kept at google (and time to upload as well). Plus it doesn’t solve the problem for videos.

  19. Laura Yecies

    Google photos is magical. I uploaded about 200gb of photos and was blown away by the automatic tagging and search. I then realized that I wanted the premium version with my original quality photos. Unfortunately to get that for the already uploaded photos (future photos are automatically original quality) I would need to delete and reupload manually. Google if you’re reading this discussion please add an automatic sync feature!

  20. Christian Keil

    My buddy from college made a commercial for Google Photos on this very feature! (More or less.)Check it out: Google Search by Emoji. https://m.youtube.com/watch

  21. jason wright

    is SoHo House a chain, a franchise,…?

  22. jason wright

    are Google Photo photos private albums searchable only by the taker of the photos?or, will they show up in a Google image search?

  23. sigmaalgebra

    So far, machines don’t learn. Google’s computers have not learned ANYTHING about showers or architecture. Instead, the text “Soho”, etc. had to come from somewhere, from a human, read from a sign, from some other image metadata, etc. We’re not in some land of magic, folks.

  24. John

    Do you always leave your GPS on so the photos have location tagged automatically as well?

  25. Joe Lazarus

    I’m interested in ways image recognition tech could be applied outside of search. For example, it would be really interesting to see what Pinterest could do with the Clarifai API, if they aren’t already doing something like that. I bet they could infer a lot about my interests, expose related images, and things like that by analyzing millions of photos. Are there any cool examples of companies using Clarifai like this?

  26. Fraser

    A few months ago I did a search for an esoteric term (banana) in Google Photos and it returned three photos out of my 10,000+ that had a banana in it. It was perhaps the most magical moment I’ve experienced from technology since the first time I tried the iPhone. It opened my eyes to the fact that machine learning is going to change the world, far faster than we think, and in ways far beyond what we think.

  27. Markus Buhatem Koch

    I assume those pictures of the Grand Bazaar where tagged with GPS info, but Google Photos is also great at tagging photos with their location based on the landscape or buildings alone. Face recognition is also amazing: it accurately tags pictures of my 5 month baby from since he was a newborn, and pictures of my 85-year-old grandmother from back when she was 25. Pretty impressive!

  28. JimHirshfield

    I think we just discovered the next big thing: ShowerChat? Showergram?

  29. creative group

    Paula Robert Cary:How do you rate the waterproof Galaxy S7?

  30. Mac

    Before we take this thread further down the drain……ShoweredIn? ShowerBook? ShowerHub?

  31. sigmaalgebra

    Ah, let’s get to what you really have in mind: ShowerTinder, ShowerMadison, ShowerMeetup, ShowerGGW, ShowerFriends.