Hacking Amazon Echo
My friend Stephen and I have been talking about hacking the Amazon Echo so it can be a front end to our Sonos systems. Stephen decided to make it more than talk and downloaded some code on Github and stitched together a system that works. I was at his house yesterday evening and he showed it to me. Here’s a blurry video taken from a phone that shows it in action.
Stephen’s system uses the cloud based Skills interface to parse the commands, then sends instructions to some of that Github code running on Amazon’s Lamba cloud based code execution service, which then sends instructions to a local server on Stephen’s laptop that runs the code that controls the Sonos locally on his home network. It’s a kludge for sure, but it works and I am going to get this hack working on my Sonos system over the next few weeks.
But the whole experience got me thinking about hacking the Echo to do other things. The possibilities are endless.
Here’s an idea for some enterprising engineer out there. We own some bungalows in Venice that are rented out on Airbnb and VRBO. We have a Sonos Play in each of them. And we have a Google Doc that has frequently asked questions on it hanging in the kitchen of each bungalow. What if there was an app for Amazon Echo that could take in that Google Doc, parse the frequently asked questions and answers, and train Alexa (the voice command interface in Amazon Echo) to answer the frequently asked questions. That would get me to swap out the Sonos Plays in each bungalow for Amazon Echos. And Airbnb could promote that app to all of its hosts.
Anyway, there are all sorts of things that would be fun to make by hacking Echo. And I suspect they will. These voice based devices could be the next big platform for developers to make things on.
Comments (Archived):
What if there was a 3rd party Sonos / Echo API integrated dashboard that actually let you also see the music your guests play and use those lists to upgrade playlists?The best playlists I got on Spotify were from the right pane seeing what friends discover
I like this – you’re saying to tie them together and download their songs you decide you like? Very cool, I want to get an Echo to play around with these now.
Exactly!
Airbnb should build that app. “Ask Airbnb what’s the wifi password?” “Ask Airbnb where are there extra towels” “Sorry I don’t know the answer to that question, let me ask the host and I’ll get back to you” (building the database of FAQ in the background)
Can’t see why Airbnb sees this as a value above the cost.Brand wise a differentiator–doubtful.Cool app–for certain.
Its a virtual front desk. Airbnb loves stuff that puts its hosts on par with hotels.
Airbnb issue is not competing with hotels. It is competing with their lobbying money for the states regs.An app is not where I would spend my money.
I wouldn’t put this effort in the “competing with hotels” category, and would rather put it in the “keeping hosts happy” category.
Agree – and “keeping guests happy”, if it takes off and is widely adopted among hosts.Even if it doesn’t move the needle relative to costs, it improves the product and widens the moat that competitors have to cross to compete with you.
Categorically I agree.I prefer a concierge service that you arrange for green blends delivered every morning or anything delivered any time prearranged and put in the fridge.That matters,
I tend to agree with you on this. It’s a bell and whistle that won’t move the needle in any significant way, relative to costs. [1] What you are saying is really classical marketing thinking. (That might not be the right way to put it.) Similar to what the car companies did for years. Any added feature had to be justified as bringing in more sales. Otherwise leave it out. The departure to this thinking, at least for car companies, was possibly around the time the Honda Accord was released (1976) and had nifty features like a place to put your toll change. But that wasn’t the reason people were buying the Accord. Nobody bought the Accord because of that but it was a good after the fact justification for buying it.Recently I stayed at the Fontainbleu in Miami Beach. They put Imacs in every room. Forgetting the fact that they were slow and an old model, I don’t think anyone is choosing that hotel (over another) for that particular feature. If they are doing that it’s probably because other hotels are, and they need to match features (that’s my guess at least). I’d rather that money be spent on early or late checkout which might be a reason that I’d choose a hotel. [2] The killer app for that hotel was a) the size b) the boardwalk outfront that runs to South Beach c) I stayed there as a kid[1] Where costs equals not only the build cost but the maintenance and operational cost. Also if the product doesn’t work correctly you have a negative created and frustration possibly. (Like the IMAC that was slow in my hotel room).[2] But that’s me. I can see how some people who don’t bring laptops might choose the hotel by an IMAC in the room of course when they find out how bad it is a negative will be created.
While people didn’t buy an Accord because it had a place to keep your change, it was the mindset of doing things that make the car more useful. Once you do that 15 times, the whole driver experience is upgraded and then you have a real advantage.Adding one small feature that makes the experience better doesn’t matter. But keep doing that over time and eventually it becomes an advantage.
All these subtle details in products and services, as you say, may not influence a buying decision but certainly contribute to customer satisfaction and loyalty.
dunno.i use these services and to me not how I choose.But I get your point.
Special feature for SF and NYC: “Tell Airbnb to alert me when the cops are coming”
Booyah
Sometimes I am absolutely sure that I visit the primordial soup of the continuously emerging and ever optimizing digital economy and world – every time I check in here. From the laymans gallery – my visits here minimize the need to go other places for creative vision gathering while maximizing views into probable futures. Thanks Alvin – uh – Fred and gang.
Then we are doing it right
That’s some tasty primordial soup!
Re the Google Doc tie up with Alexa… This is the future. Right now it is “Go search it up on YouTube.” Soon it will be “Ask Alexa.” When a new product comes into the house it will be “Alexa, here is a Samsung Model XYZ 123 refrigerator.” And she’ll take it from there calling for / scheduling the annual maintenance and being able to answer questions from the owner’s manual. … The owner’s what? 🙂
Integration with Google docs with standard platforms is pretty horrible in my experience.Want to be wrong but tie ins to inventory, accounting systems to my knowledge are non existent.
Hi AW, Re Google Docs… I was just “going with it”. Something else, like a internet platform that converts free text, Fred’s bungalow guidance, into something akin to XML, is probably more to the point you are making. Such a platform would be helpful if it had a autocorrect / “do you mean this or this or this…?” feature that helped smooth the input. Re tie ins to inventory, accounting systems… someone has to drive/invent/back that. It does happen… the Quick Response System (QRS) Universal Product Code (UPC) data base has been fairly successful for retail. Why not a similar system for these home convenience devices from Amazon/Apple/Google?
No knocking it at all!!Build more than one thing just by doing it but Gdocs is just and issue and because it is so poor to integrate it is allowing massive vertical closed systems to exist.That’s all!
Have you seen mycroft.ai? It’s an open source competitor for Echo that is infinitely extensible. No hack needed. The team just graduated from the Sprint Accelerator (Techstars) after completing a successful Kickstarter.
That’s actually a really trivial app to build for the echo, so I am sure someone will have it for you within a couple of days.The beauty/fun of building for the echo right now is that it’s all about what you can imagine and dream up to stitch together…and that’s what builds passion for programming. I encourage everyone to dig into it!
Reminds of what http://Josh.ai does
This is so awesome. Incredibly practical and super encouraging for kids to learn how it works.
Seriously – it’s a new dimension to add to the mix – love it.
I continue to be amazed at how fast Alexa is at providing a vast range of info as well as functions around the house. Next hack: getting Echo to open my garage door, code also from github.
Hack it for finance. Sell Alexa Sell
Wait, they’re selling?! Buy Alexa, Buy!
Alexa, get me the bid/ask for the $APPL August weekly 90/100 strangle, and can you give me the skew?
:-)Source: Economist.
How do you hang a Google Doc in the kitchen? Sometimes I feel like I will never catch up to technology these days.
We print it out
I’ve been waiting a long time for someone to figure this out. Great find Fred. Possibilities are endless.
The Google Doc idea is a nice start but maybe some instructional video clips on YouTube that Alexa could show thru the FireTV box…
Tangentially related: saw a great post on Reddit last night about hacking the Amazon Dashhttps://np.reddit.com/r/Ple…
“The possibilities are endless” is an understatement. Once you put programming in the hands of regular users, they end-up modifying the product’s intent for their own, in unique ways. And that can be even be done without hacking or programming. Check out the IFTTT Alexa channel with hundreds of existing recipes.https://ifttt.com/amazon_alexaActually, you and Stephen aren’t the only ones wanting to integrate Sonos & Echo:https://www.google.com/sear…
Browsing the IFTTT recipes for Alexa right now – so many cool ideas. Use Alexa to control the temperature, to open the blinds, to trigger “Find my iPhone” (that’s gold)…
We hacked Amazon Echo to help visually-impaired people & kids cross the road:* http://digitalcane.comOn the surface and for shallow IFTTT, Alexa works reasonably. However, it’s incapable of dynamic learning: e.g., if a person says, “I’m ready to cross the road now” and Alexa guides them but quarter-of-the-way the person gives new instructions, Alexa has to start over from default command origin.The other thing we discovered is that it isn’t GPS-enabled and has no camera so couldn’t assist road crossing in the way I had in mind — whereby it could calculate the distance (kerb to kerb), detect cars & their distance/velocity away AND count the number of steps the person crossing the road needs.What Amazon should do is build Alexa into traffic lights with cameras. Now, THAT would enable smart traffic monitoring and road crossing.It’s something I pointed out to Alexa team but there are hardware+software things they need to fix for that to happen.Yeah, I always push what’s hackable beyond the standard norms of playing music, switching on lights, opening doors.That way, it becomes clear where the $billion market gap is. Smart traffic control and toad crossing safety are HUGE value areas.LOL.
Looking forward to hacking your product 🙂
Longer than endless – hmmm eternal?Seriously – IoT solutions are often vertical specific, but when there is value in your lawnmower talking to your fridge, with good APIs it becomes feasible.Forget semantics and ontologies and focus on compatible plugs and sockets and soon platforms evolve where the demand exists.
You’re not kidding, it’s awesome to see the use cases people develop for Alexa. Alexa was a big part of the Disrupt NY Hackathon this year…. Both first prize and second runner-up used Alexa to tackle everyday tasks: https://techcrunch.com/2016…First was AlexaSite, to help you edit websites (backgrounds, text alignment) just with your voice, and second runner-up was Hungry Host, which allows you to search for recipes based on what’s in your fridge and receive step-by-step guidance on how to make it (I was lucky to be part of the Hungry Host team! The video you included reminds me of all the testing and tweaking and talking to Alexa for hours). It is really cool to see what this can do.The Airbnb idea is great; you could hook it up to Nest or Canary or whatever you have to welcome guests once they walk through the door or switch on a light (although, that could be a terrifying experience for those not expecting it).What about tying Alexa to Yelp or Foursquare as you’re getting ready in your Airbnb, to be able to ask for dinner recommendations and things to do while you’re visiting a new place, based on what previous guests have enjoyed? Possibilities here really are endless (some cool ideas in the comments as well, love the sharing playlists idea…).
Exa.ai has been working on this. Our airbnb pilot launches in Jersey City in two weeks. You can see video demos of the Alexa skills we’ve built for hotels at: https://exa.ai/hotels-and-r…
What if Alexa was just an App on your smartphone?Behind the scenes, it’s just web services tied together with a voice output.
I don’t have an Echo but from what I gather the idea here is that it’s always on and waiting for your commands. Currently Apple hardware can’t do that. You have to hit a button to get it to listen I believe. In order for a hardware company to get around this functionality they would have to probably deal with patent issues I am assuming Amazon has a zillion patents covering Echo.Then he finds this, from today (my always on comment still stands):http://www.slashgear.com/le…
but it’s not the hardware that’s doing most of the work; it’s the software.i’ve had early experience with Echo’s predecessor 3 years ago, the UBI.
Right but what I am saying is that I don’t believe on a smartphone, at least on an iphone, you can get that “always on” functionality unless the hardware manufacturer enables it.Interesting look at what they offer now (people who did UBI):http://www.ucic.io/Hah! I see now that you are involved with them!http://www.ucic.io/about
Except, well, you’re wrong about the always on functionality. Please stop saying this as if you know it for a fact when it’s just flat out not correct. Yes, it’s a setting in iOS, but if it’s on, it’s on.
On the Iphone 6s “hey siri” does not work unless your phone is unlocked. This removes the functionality that I am referring to that would be similar to the Amazon device which is always listening.What is an example of an app that runs on the iphone that is not logged in that allows you to execute commands or get anything from it?
Again, you’re just wrong. The entire POINT of Hey Siri is that it works whenever you say it. Hell, I *have* a 6S Plus and it works when locked, so I can personally verify that what you’re saying simply is not accurate.In the 6S/6S+ you use Hey Siri when not plugged in. Prior to that (I think), the device had to be plugged in as they couldn’t afford the battery drain of always listening (the S use the M9 motion coprocessor to listen which reduces this issue).Hey Siri has NEVER depended on the device being unlocked. For some devices it HAS depended on whether it’s plugged in. Again, please stop.”What is an example of an app that runs on the iphone that is not logged in that allows you to execute commands or get anything from it?”I’m not even sure what you’re asking here. Obviously, right now, we’re limited to Apple approved/implemented apps because the Siri SDK is in beta. We won’t really have a good idea of how well Siri integration can work for 6-12 months – after devs have had some time with a fully implemented, stable OS and SDK and time to write for it.PS: I think the Echo is a great device and I’m almost intrigued enough to buy one even though I don’t need it. Nothing I’m saying on this thread should be construed as a hit on Amazon in this respect.
well, they pivoted to ucic.but previously, their original product was exactly like the Echo. they were 3-4 years ahead of the market, and were in talks with Amazon a few years ago. long story … i had that product in my family room 2.5 years ago, and was giving it commands to play songs, do actions, etc…Actually, the interesting thing is that you could command it from a smartphone App too, and it would output something at the other end. I’m not sure if Echo has that.
On the iPhone 6 and beyond you can just say ‘Hey Siri” and it will activate. Same on the Watch. Not sure about the OS X version as it’s just out in Sierra b2 and I don’t have it installed on my Macbook.
So the question is can an app developer enable that functionality. My “guess” is that they can’t that Apple has that locked down but that’s a guess.
I don’t mean to be too rude here but… you have no idea. Neither do I. We’ll need to see, but your assumption is indication of bias.I’d make the opposite assumption – Hey Siri does nothing more than wake Siri up. At that point (now), Siri can be asked to do anything she knows how to do.The easiest, most rational assumption is that in Sierra, once Siri is paying attention she will be able to do anything she knows how to do, period, regardless of how she’s been woken, including developer defined things. Otherwise the user experience would be confusing – Hey Siri allows some things, but pressing the button allows everything. There’s no reason to do that. Again, bias.From what I hear the real impressive thing about Echo isn’t that it responds to your voice but that it has very good far-field microphones so you can be in another room and it will respond. That, I don’t see in Siri for OS X since her microphone is merely the builtin ones on the Mac. Of course, Apple could develop a standalone device with better mics and I’d assume they will at some point.
My old Moto X had ‘always listening’ capability, and without pressing buttons, I could speak commands to it.I actually found that I used that a lot.
I don’t know if this leads to the exact use cases Alexa allows, but Apple announced last month at the WWDC the inclusion in iOS 10 of an API for Siri. This will allow apps to receive and process voice commands coming from Siri. Android also has a “SpeechRecognizer” class which is still not quite usable imo, but should get better really soon as Google transfers their research work to Android.Natural voice recognition and processing in consumer products is a strong trend indeed.https://developer.apple.com…http://research.google.com/…
will be smart of apple to allow all the assistants to run in parallel , better than ending up with a bing
There is a new framework for developing on Amazon Echo called Flask-Ask written in python: https://alexatutorial.com/f….
From Kevin Kelly’s latest book: “anything that will not be intensely interactive will be broken”
all of this reminds me of the iPhone moment. different scale and opportunity set, but same impulse.
Semil Shah:Impulse and forget fad! The current numbers reported by Apple supports it. The issue is companies that become successful rarely make that shift quickly enough. Reliant on singular products. Oversaturation! Not being open to change!
Hey Fred and readers! You should come by the NYC Amazon Alexa Meetup on July 20th if you’re free – would love to meet other Alexa enthusiasts and learn about skills you’re building. More details here: http://www.meetup.com/NYC-A…
just wait until i use the echo to replace play calling in the NFL (this is the formation, the down, yardline, etc, whats the best play based on statistical expectancy)
No. 1 requirementtelephony
Alexa, start a live video stream from my iPhone camera to my Twitter account now.
FRED:respect your judgment in a majority of posts and views. But this post falls short of the weekday posts. The wealthy, geek, too much time on your hands, all of the above segment which may appear to be majority who responded appeared to get hyped over solving a problem that is like rigging servers together with wires everywhere from 1985.We will do it for all blog members interested who desire this enterprising engineering, cloud base interfacing and whatever whatcha call it (Soros hookup with Echo) for a small fee of $100,000. (We will label it the Hookup)Sounds outrageous? So is this entire endeavor. There are more serious world and national topics. (Talking to ourselves while typing the above-Do you realize this is Fred Wilson’s blog? If you want to blog a topic get your own blog)HOT TOPICS!The Brexit consequences based on ill-advised Nationalistic pride that is effecting the world economy, two countries that relied upon the EU to bail them out are now in play with Ita-leave and Portu-go exit the EU movements.Etsy serious processing of payments over the weekend that caused havoc on bank accounts.Twitter acquiring Magic Pony.Microsoft announced its intention today to combine the Dynamics CRM and ERP products into an integrated platform on top of which it is hoping third parties will build applications.Big deal that will have Salesforce.com looking over their shoulder.Tesla and the May 7, 2016 autopilot crash that was actually reported to the Government nine days after the accident. Elon Musk not listening or never heard of Herman Edwards advice to Rookies and Veterans in the NFL regarding social media and Twitter.Don’t press send! Do not press send.Microsoft announced its intention today to combine the Dynamics CRM and ERP products into an integrated platform on top of which it is hoping third parties will build applicationsWe will take our licks with the geeks and wealthy on the blog. Had to be addressed.www.youtube.com/watch?v=gIb…
There were sooo many Echo hacks at the disrupt hackathon this year, in fact one of them won the whole shebang. Feels like a deep developer community is growing here. Google and Apple have a not-insignificant amount catching up to do. https://techcrunch.com/2016…
Echo is to Siri what iPhone was to Blackberry.My wife bought me an Echo last Friday for my birthday even after I told her I don’t think I’d use it… I was too quick to speak! I love my Echo. I went to Best Buy and bought WeMo power plugs and LIFX lightbulbs, now I walk into my apartment and say “Alexa, Turn on the Lights” and my lights turn on. Beats walking around to three separate lamps to turn on and off the lights!Such a good product.
This would be super-easy to build, though you’d probably need some decent NLP to make it work well.
AirBnB and VRBO short term rentals by absentee landlords are coming to an end in Venice – https://docs.google.com/vie…Long overdue. Neighbors of absentee landlord properties using these service finally get some relief.
Hallo friend how do in order to get a dollar off online
https://www.technologyrevie…
The newer Sonos Play5 have microphones, so hopefully Sonos itself will offer some voice services. I’m always hope Apple buys them and it’s Siri.
Fred, who do you use for as your vacation rental property manager in Venice? I have a couple bungalows that I’m planning to switch from long term rentals to Airbnb’s. I’d really appreciate your recommendation.