Can Amazon Run The Table On Cloud Services?

Comments (Archived):

1. Think the commenter is on the money about geographic being the only factor and not much of a factor for most. Think AMZN has done a great job at being the first really good supplier of cloud computing services doesn’t mean someone can’t enter and provide a better, more cost effective service. I just can’t see it happening.

   1. Yeah. Amazon are the only guys doing it right at the moment. The API works, it’s quick and it’s reliable. The forum support is good, the startup events are good, it just feels good.

2. Amazon’s cloud services are not just for startups. I argue that large enterprises should start abandoning their server farms and let amazon or other providers run them in the cloud.I predict Amazon will roll out a Service Level Agreement (SLA) that rivals enterprise providers.See my white paper for BT at Davos for more info :http://www.globalservices.b…J

   1. Hi JackThanks for stopping by and sharing your thoughtsI’ll check out your white paper

   2. Jack: good stuff !

3. short term yes amazon has a brand equity & can deliver a good quality service, but longerterm there will be others who can do better at a lower price.Also the concept of a cloud maybe futile in 10-12 years when individuals will have a datacenter in their mobile devices i.e a Petabyte drive. That will be the bigger challenge for the Amazon’s & Googles of the world as they have invested large amounts of capital into building these cloud computing infrastructures and in the interim it works but long term it is just a Placeholder and they will be required to find ways to participate inside the individuals datacenter that they will be carrying around.

   1. Wow, now that’s far out thinkingI like it!

      1. Depends how you define ‘far out’ ……. being in a couple billion pockets and getting pushed by Moore’s law is a decent trajectory 🙂

      2. Mainframe to PC to Cloud to mobile device to… makes sense.

   2. I tend to agree with baba. The history of computing has been a back-and-forth between centralized and distributed architectures, as networks, storage and processing have evolved. Think mainframe (centralized) – minicomputer (distributed) – PC (distributed) – client-server (partially centralized) – cloud (centralized) – … A distributed competitor is coming at some point.

4. Amazon does not always win. See, for example http://www.deploymentzone.c…However, Amazon is a number of things, including a set of API’s and the execution of the API’s. With the announcement by Canonical of their support for Eucalyptus and the Amazon API’s in the forthcoming Karmic Koala version of Ubuntu Amazon will gain applicability. http://fridge.ubuntu.com/no…Now, if you use Amazon, you don’t need to feel trapped. If you want to run a version of the Amazon API’s inside your fire wall in the short term because you don’t feel comfortable with security in the Cloud, you can.So, Amazon is in a good position to become the defacto standard. Coupled with Amazon’s genetic focus on operational excellence, the future’s so bright you might have to wear shades.

   1. Albert touched on Mosso in the post that got all this startedIt will be interesting to see if rackspace can commoditize themselves fastenough to be a real competitor in this space

5. I’m a huge fan of Amazon Web Services. The ability to spin up multiple servers in about 5 minutes (EC2), seamlessly store content and push content to a dozen CDN servers (S3 and CloudFront), and even tap into an army of on-demand workers for tasks better accomplished by humans (Mechanical Turk) are amazing, and essential to any startup (any business?). And Amazon has done a fantastic job of defining their roadmap, executing against that roadmap, and knocking down any objections to using the service. Most recently, they launched the much-anticipated web front-end to managing EC2 servers. Load balancing servers isn’t built-in yet (although it can be done via various solutions) but that’s coming, per their roadmap.I find it simply amazing that a company could find out they are going to be on The Today Show tomorrow, and in an hour set up enough server capacity to handle the expected spike, then shut it down the next day — paying only for the actual resources consumed.In my mind, there are only a handful of reasons NOT to use Amazon Web Services: 1) If Amazon is competitive to your business, 2) Compliancy concerns (not sure how Amazon can handle PCI compliancy, for example), or 3) Usage limits or software licensing issues that make it economically more attractive to set up your own server farm. That’s about it. Otherwise, I agree, Fred — every startup should be looking to the cloud for the cost savings and flexibility offered.

   1. PCI compliance is an important issueThat’s probably a point of differentiation for a cloud startup (or a bigcompany looking to get into cloud svcs)

6. agree that I have not met a start up company (and some larger ones too) that isn’t defaulting to using AMZN cloud services to build their business. it is truly driving down the cost and complexity of how one starts a business. if GOOG doesn’t really make a play here AMZN will run away. on being long AMZN, my only issue there is that AMZN’s books and dvd retailing business (which is a lot of top line but less bottom line as I understand) is likely to go through a transformation that might be “violent” to the P&L -I dont know how to read how that plays out…

   1. I guess they have Kindle and mp3 store for the same deal

      1. Besides that for DVD, i think Amazon might be one of the biggest seller right now. Media companies cannot dump Amazon while they want to make switch to online in a hurry. Besides that media companies are never in a hurry in the first place. Also, media companies got burnt badly via apple mp3 fixed cost deal for years, which is one of the reason promoting all kinda competition – Lala, Rhapsody-iLike deal, Myspace, Playlist.com…… I dont expect media companies to promote similar setup in the online movie space, when newer releases start getting rolled on to Netflix and Apple. Pretty sure Amazon and Youtube will be part of the big push when it happens in the next 1-3 years

7. I’m pro cloud services for small biz, but I can’t believe it would produce the same performance medium/large corp would have with local systems. Plus as you rely more & more on cloud services; your bandwidth needs would increase. There gets a point where latency & SLA for bandwidth aren’t as cheap as everyone thinks. Global cloud services are needed if you’re planning to sell your product worldwide. Such as the current fastest latency from the east coast of US to China is 240ms. For your China or Asia clients, your product is going to seem damn slow.Optimally you’d have 3 sites if you’re global…if you’re selling purely to the US or Euro market, you could probably get away with 2. East/West Coast or US/Euro, so when that MAN or undersea cable is cut; you do have a better uptime(a cable in the Mediterranean has been cut twice, & severely impacted India). But I’d still go for as much redundancy as possible, as well as making sure you have backups of your servers/data locally at your sites. Too many webapps have failed recently due to not having mysql backups on seperate media/locations.

   1. I agree that EC2 is not all rosy. Simply because it’s easy and appears cheap, people opt for it very quickly. Bandwidth is one hidden cost, and it can become significant for certain types of applications. Also, there are some severe problems with inter-node latency that have less obvious costs in terms of performance and architecture. I’ve seen the intrinsic limitations of EC2’s performance shape application architecture- and that can end up costing you a lot more than you pay in hosting in headcount and development costs to maintain.I certainly don’t buy the assertion that every startup *must* be in the cloud. There are significant trade-offs in EC2, and one needs to be aware of them when building on the platform. What I think is a no-brainer is using EC2 for its elasticity. It’s hard to beat the ability to create 10 new nodes to handle a load spike in EC2, whether your primary hardware is in the cloud or not.

      1. agreed for elasticity it is the best option…at the moment. Though I wonder if IBM or such will get into this business? The hope is that Amazon is over expanding their services to handle the needs. To handle a load, such as if multiple vendors wanted to 10x their capacity at a single cloud location; would Amazon be able to handle?

         1. If i were a startup who wanted to get 10x more capacity, i would for sure get in touch with AMZ support before making the final switch.

            1. I’m just thinking about if there was some event, where a bunch of startups thought they’d get a lot more hits after such; I’d just be concerned if all the startups had their servers in a single cloud location.

            2. why would you be concerned about that? Can you be more specific. If you go with one cloud service what would the drawback be?

            3. Clouds – no matter what you name it, it’s just a bunch of VM servers clustered together w/ shared storage & fat pipes between datacenters.I’m not wholly sure how Amazon locates a VM server that you create. Is it located close to your location, or where low latency is?My concern is with overload at a particular location. Let’s say multiple startups have their VMs located in LA, as well as some large media organizations & Apple. So while at SXSWi or similar, there’s a real buzz for 60% of these startups, Apple releases another fanboy object, & OJ escapes jail. So now there’s massive load of internet traffic & I/O traffic at this DC in LA.Well all the startups & Apple have informed Amazon that there may be increased load….but no one expected the OJ thing.Has Amazon or other sufficiently provided enough bandwidth & I/O capabilities at that site to handle the load…or will VMs be moved to other DCs which could mean inaccessibility for some VMs during the transition and/or increased latency issues for the end user trying to find out about your product…if a customer gets any sort of error accessing your website; they’re less likely to return.

            4. Don’t you think that’s a little paranoid for a startup. AWS is pretty well distributed across the US and at an early startup it’s all about cost. Myspace and Facebook would crap out all of the time when they hit a real growth spurt and I doubt their outages and javascript failures stunted their growth.

            5. yes, it is paranoid. Depends what you’re building, MySpace & Facebook are making money by adverts only, at least till recently. If you’re going to be selling a service/product, and not just providing an open platform you want to insure limited if any outages.look at pownce, a great service (free & paid)- it went offline for several days, and lost (what seemed like) 70% of it’s user base.My primary concern would be to insure you have daily backups of your data at your office location or some other provider service(not the same company providing the cloud). Look at Ma.gnolia, or Twitter, when they’re mysql DBs corrupted & they lost all their data &/or site/service.

            6. Valid point. I think we need to look into this. Thanks again for the detail.

8. Fred, I’m surprised there was no mention of force.com, the salesforce.com Cloud computing offering. This field is still evolving. The Cloud offerings out there are different and may match some product offerings better than others. Full disclosure, I’m betting force.com does very, very well.

   1. But force.com is a vertical platform, right?I agree that they are doing some great stuff but you can’t build a startupon top of force.com can you?

9. Consider Amazon a “bookstore” peddling “illegal drugs” (i.e. cloud hosting) in the back. When you mention Amazon, the overwhelming response is a bookstore. Well, by 2012 I believe, Amazon is already estimating that their hosting revenue is going to make up the biggest portion of all of their offerings! That tells me the stock price has been a steal for a longgg time!!

   1. I wonder if anyone has done a five year projection on the AWS businessIf so, I’d love to see it

   2. yes but precludes Google, Microsoft, or a “new”, dedicated player from joining this space…these companies are not going to allow AMZN just to run away with all the cookies.

10. I agree with much of what Albert has said except for the part about being suspect about teams not building for the cloud. This is an awfully aggressive posture. Amazon still doesn’t have a world-class SLA. If you are building a company that you think can truly be world-class I would argue that using a service without a serious SLA is malpractice. Someone tell me why I’m wrong.

    1. well he qualified his aggressive stance by saying “time to be somewhat suspect”i didn’t write the words, but i work with albert every day and i know that he was saying that we should be asking every startup tech team “why not use the cloud?”

    2. What sort of compensation do you expect from a “world class SLA”?If it’s something like “a month of free service for each hour of downtime”, you’re paying a premium for nothing.An SLA that doesn’t cover loss of revenue is basically a joke.And, most SLAs are written such that they’re not in violation much of the time that your service isn’t available.

       1. Well, I know it sounds stupid but if they want to get larger public companies beyond startups, they are going to have to deal with corporate legal. And if you need to have a disaster recovery document, insurance, and all the legal cover-off that’s required in a large company, you will need an SLA…..it’s just a required thing if you are considering outsourcing a major function to a vendor like that.

11. I don’t know much about AWS and cloud services in general, but I know a bit about software engineering.If you get your abstraction right, it should be dead easy to move your server side to the cloud. On that basis, I tend to disagree with Albert’s advice. I takes enough effort to get a decent first iteration going these days (because the bar just gets higher) and so I would employ all my resources on core functionality rather than scalability.The key point is that the back-end be *designed* with the cloud in mind, rather than using it straight from day one.

    1. This is hard to disagree with my friend. Makes all the sense in the world. If you have the right team and the right design you can remain flexible.

    2. Certainly agree here. I’m building on a simple Slicehost account right now, because there’s less overhead and cost at this early stage.But I’m doing things in an agnostic way, so that I can move to whatever makes sense later.

12. While Amazon services are impressive, it’s ridiculous to say that everyone should use them. The problem is that their top of the line instances are still very expensive relative to just leasing a server through a good host like Softlayer, and they’re still nowhere near as powerful as the box you’d get there.The problem with that is scalability. Amazon makes scaling CPU-intensive apps (such as the posterboy, Animoto) relatively easy. That’s awesome. What they suck for is database driven apps.For instance, I’m running a few pretty popular Facebook apps on one Softlayer server pair. The power of the boxes available lets me run (we think) an app almost as popular as Mob Wars on just that one pair. On Amazon, the machines have so much less horsepower that we’d be forced to deal with sharding or something similarly painful to allow us to expand the information storage to multiple boxes, while paying 2-3x the hosting bill.Also, the ping time between Facebook and Amazon (every Facebook app has that extra round trip involved) is significantly higher too, so our apps, which are probably the best-performing of their size, would become significantly less responsive to users. This is a largely geographic factor for anyone building anything on an API.Old-fashioned hosting still offers the ability to run a significantly-trafficked database driven app for much less time and money than Amazon, and to locate it where you need to for maximum performance. That may one day change, but until then developers shouldn’t consider it the only tool at their disposal.

    1. Great comment and great real world examplesThanks!

    2. i gotta agree with you and Kiril.AWS is a great inovation, but it`s not for everyone (at least it shouldn’t), it`s not the best cost effective solution out there even in the “cloud” field and i can think of only a few web apps that could (should) entirely rely on their services. This may change in the future, but at this moment, if you know what you are doing, the best approach IMO is to combine dedicated servers + dedicated virtual + cloud services. And the hosting company that is better placed in this scenario rigth now is rackspace.

    3. I love Amazon (we host our infrastructure on AWS, and I was advising friends to buy there stock a while back) but the latencies can really bounce around to a troubling degree.

13. Don’t count out Google yet. Their App Engine isn’t the same animal as Amazon Web Services, but in the end it similarly enables developers to build a site/app that scales. And they’re making it easier to use…this new remote_api functionality is very interesting:http://code.google.com/appe…

    1. I’d have to agree that GAE is very compelling and Google is just getting started. And carlospero is right that it is a different animal. Basically GAE offers a real (albeit limited) web API. This is not true for Amazon where you have very granular smaller building blocks available which give you more control but at the cost of having to do much more engineering yourself. But for casual development GAE gets you an app up and running much quicker. Think of it as the Visual Basic of web app development. And I believe consumers (developers in this case) will go for convenience. Another way to look at it as that you enable a whole different class of developer (less sophisticated perhaps) but you are going to get a lot more developers. Visual Basic was hugely successful for this very reason. And in the world we are moving into development speed will be critical. Couple the speed with the huge number of developers you are enabling and you are potentially unleashing a huge number of apps. Then the market will decide which of those apps it wants. To be clear I am not saying that GAE does not have problems. It does but it is a different animal than AWS. And I think it has great potential. Lastly a shameless plug for a music *link* sharing app that I built on GAE in very little time. http://www.biome.fm.

       1. I’ve been thinking that Google has a “microsoft” problem in this space(cloud)They are “too big” and compete with too many businesses already to be seenas a safe hosting partnerBut that’s just me thinking out loudI wonder what others think about this issue

          1. I think MS was already “big” when Visual Basic took off. And again I think it’s all about convenience (even for programmers, especially younger ones that have not been through a few battles). Nothing like instant gratification. And programmers fundamentally want to have their work “be seen and be used”. The other thing I would add is that it’s not that difficult to migrate your GAE code to something like Django. So if you get uncomfortable with Google as your hosting partner you can take your code and leave. Lastly, one of the things that GAE makes trivial is scaling a DB. This is potentially the hardest part of building a web app. You get a free ride on “Big Table:” There is also the whole vertical app business. Typically these verticals are going to be too small for GOOG & MS to care about. But there are 1000s of app developers for this space.

             1. Can you migrate your DB from Big Table?The lockin issue is the one I am thinking about mostly

             2. Yes, although I don’t know that anyone has yet.http://code.google.com/appe…

             3. I recently ported a(n ever in-progress it seems) one-man project to GAE ( http://www.stockbubble.net ), with its Big Table and Python world, that had been running on a small dedicated server with classic LAMP. Having to learn Python was fine, and not so difficult, but wrapping my head around the limitations of Big Table with its inability to do traditional joins, required some significant refactoring and mental remodeling of approach. Going back the other way (away from GAE) would not be overly difficult I believe, though a developer having to structure code on GAE to store and search database stuff completely differently than a traditional RDBMS does, may be a large factor in avoiding GAE to some projects. In its defense, seeing the limitations of GAE forced a move of more code to the browser side, which was actually a good thing for performance, though there has now been a definite bandwidth increase. All engineering is compromise …

          2. Some companies may see hosting with Google a way to lower the barrier to an acquisition by Google. Then again, the same may go for Amazon or Rackspace.

          3. I use GAE for several small applications and your statement “compete with too many businesses already to be seen as a safe hosting partner” rings true. Amazon’s main business is something so different from most people’s software business that it isn’t seen as a competitor.Google on the other hand is competing in so many areas of software.Is the difference that Amazon is not known to make acquisitions?

          4. Google also has the Microsoft ‘problem’ that they get the biggest chunk of the revenue generated whenever someone hits an app in the competitor’s cloud.

       2. you could not be more wrong…and “Visual Basic” is a total joke among programming professionals. “real” developers are going to use the service that allows them the most power, flexibility and reliability, and where the cost is competitive.very few websites need “cloud” storage…probably less then 1/10 of 1% will ever approach the traffic necessary to trigger a real need. enterprise space is different, because they tend to overbuild anyway.

          1. I realize “real” programmers don’t think much of VB but I am sure MS was quite happy about the $ it brought to their developer division. However, what made them even happier is how it benefited the MS platform. You can look at GAE the same way. Can they bring other goodness to developers that benefits their platform? They have already done this to some degree in that you can authenticate users very easily and as Fred pointed out, they are getting people to put their data in bigtable.In terms of webapps not needing cloud storage, I am curious why you say this. I would argue that a significant amount of the value of your app will be directly related to how you can monetize the data that you store.Lastly Python (the only language available for GAE right now) is not like VB. It’s a very compelling language for development and lots of web apps already use Python (big pieces of google’s back end is running on Python). However, while a sophisticated language it’s also very easy to get started on. I believe MIT now uses it in their CS intro courses.

14. I don’t know if Amazon will ‘win’, but they are a few laps ahead of most of cloud computing pack.1. They make it dead simple to use (by offering up various API for each service and allowing your pick and choose which services you want to integrate or take advantage of).2. They have a brand name that instills confidence.3. They claim to use the services themselves for many of their popular services (which helps with point #2).Still it’s not clear that they’ll be the big winner in the end…while they have a great brand, it’s hard to say if they view themselves as a ecommerce business or a hosting business (and that’s an important distinction going forward for clients because you will want to set your service expectations for where they are going with things)…there’s also an issue of price (as a bunch of others have mentioned) and this alone will probably ensure that at least some competition sticks around…I think they also have a bit more communication to do with the potential user base…as it’s not yet clear to many people I talk with just when/where/why to use the various services (and there is some overlap/confusion between their S3 and SimpleDB services — disclosure: I’m currently writing a book for Apress on Amazon’s SimpleDB offering and will be attempting to clear much of this up in the book)Anyway, what I think Amazon (or any other cloud service) will really need to do is integrate some of their core features into their cloud offerings if they really want to ‘win’ the game…what I mean by that is a company like Amazon would need to tightly integrate something like their recommendation engine into various aspects of it’s cloud computing offerings (so for example, any cloud based offering would automatically be able to take advantage of some of Amazon’s secret sauce — without really having to know the inner workings of that sauce.).

    1. i think the decision to roll out an infrastructure business was a brilliant move on AMZN’s part, but IMHO it was a poor idea to do so under the AMZN name. i think it will create brand problems in the long run.

       1. Yes, that’s sort of my gut as well…the problem though seems to be without the Amazon name, would enough people have trusted the service and/or thought that it was a serious enough player (I think that’s one of the issues 10gen is having right now — it’s hard to tell people to build stuff on top of your technology when they’ve never really heard of your technology. So on top of having to ‘sell’ cloud services, 10gen also has to ‘sell’ their brand whereas Amazon only has to ‘sell’ the concept of cloud computing)…still, it seems they could have just called it something else and said ‘powered by Amazon’ or something silly like that…and then just put the word out everywhere that amazon.com uses ‘X powered by Amazon’ for it’s ecommerce offerings…

15. it’s a fascinating period of transition and very much captures the zeitgeist for start-ups/consolidation/pragmatism/eco-issues.another phase in the cycle: the mainframe approach being questioned by the mini computer, in turn being threatened by unix systems, in turn being threatened by networked pc’s in turn threatened by linux, VMWare, etc ….the key differentiator this time is the barrier to entry to having a ‘proper’ IT infrastructure behind your business/apps is removed and is a compelling one.the legacy/enterprise IT business model (for both vendors and users of) faces very radical changes – especially in licensing models and customisation of solutions.the success of salesforce.com has done more than anything to create the mind-share and give credibility for such an approach.

16. IMHO way too early to be forecasting AMZN as winner of the cloud game. i dont know if they have the right marketing approach for it, though i tend to think that is the case for all the cloud services. they all need to do a much better job of significantly simplifying their sales pitch (or find a reseller that can do it for them).and of course let’s see how AMZN deals with the economy, crisis hasn’t even begun yet (though we’re getting closer to the starting point).just looked at AMZN on google finance. remarkable how well it’s held up over the past 6 months, only down 3% (GOOG is down 21.55%). AMZN’s P/E of over 46 is a bit ridiculous in this environment though, IMHO. but in any event the fed has broken the stock market and turned it into a casino, so we can toss logic out the window, save for technical analysis.

17. I’ve been using both Amazon Web Services and Rackspace’s cloud offerings in Mosso. As of right now I think Mosso/Rackpsace have the lowest barrier of entry for businesses because of its ease of use to set up. Also add in fanatical customer service and they have provided a very differentiated product from Amazon.I am currently using Amazon S3 + CloudFront as backup storage and Content Delivery Network. I am simultaneously using the Mosso cloud for web hosting.

    1. That’s very useful input. The exact kind of thing I was hoping to get fromthis post.Thanks!

18. Sorry Amazon IS a bookstore – a retail outlet and if you look at economy these days that isn’t the best business to be in.As far as cloud services – this will go the way of other web service ISP ,content delivery – matter of fact look at Akamaithey were the top Content delivery company in 2003 – video was exploding on the web and Akamai had no rivals and then the rivals to the market flooded in .Look at akamaii stock price AKAM went from 250 bucks a share to .60 cents a share -after first internet stock crash – to 60 bucks a share now it is 18 bucks . The leader in a commodity business no more no less.

19. For a web or SaaS startup there is little reason not to use cloud infrastructure services, they scale much better for your business and are cheaper than hosting your own (it can’t be compared to shared hosting, that is like comparing apples and oranges). To say that Amazon will win is going a bit far. They are the current leader, however, and putting together a nice set of services. Few networks effects exist however on the user side. If I were evaluating IaaS providers it doesn’t matter much to me if Jim, Bobby and Steve also did. It will matter if I can have a persistent instance and a rich ecosystem of tools available. Amazon is the best positioned to remain the leader in cloud services, we have only just begun however and many innovations still remain. I always wonder who will be the “Google” to Amazon’s Yahoo? Maybe Msft?Adam

20. EC2 is well suited for elasticity. It’s not well suited for running a mildly popular site as a pseudo dedicated server.The exact applicability of EC2 aside, I am personally a distinct DIS-believer in first-mover advantage for these situations. I would say Amazon is the first-mover in this space, and if/when the cloud computing server model ever really becomes a truly viable stand-alone business someone will come along and eat their lunch. A lot of tech companies spend millions of dollars developing a product and a market, and then somebody comes along and stands on the shoulders of their learnings and create a better/faster/cheaper solution. Amazon seems (to me) to be prime to be an also-ran in this category because I really don’t think EC2 is core to their business model, it’s almost more of a grand experiment.

21. I use AWS extensively both at work (http://nytlabs.com and vimeo.com before that) and for side projects (http://enjoysthin.gs). As far as I’m concerned, it’s already game over.Amazon won a while ago. Google is a non-starter because you have to live within google’s development rules to get your app up and running, which means you can’t easily take an existing app and move it to their cloud. With amazon, porting existing apps is very easy. I can get my app running on whatever server is easiest, and then move it piece by piece to the cloud.I’m still kicking myself for not picking up AMZN when it was close to $30 a few months ago…

22. I find AWS *very* impressive, but there are 3 issues that have to be overcome for broader adoption:1 – Privacy issues. 4th amendment protection does not extend to virtually hosted environments. With no physical asset being leased or rented, you have no “reasonable expectation” of privacy under SCOTUS ruling. Practical implications include subpoenas being served to Amazon for your data, not you.2 – Security issues. This one is solvable but tricky – how does an enterprise maintain security of internal networks with servers being extended out into EC2?3 – Licensing issues. Enterprise software licenses are tied to physical processors. Vendors will have to be pushed, screaming, into something more aligned with virtual servers. This seems sort of trivial but in fact will send repercussions through the entire IT industry, and will be resisted by the big vendors as long as they possibly can.BTW, IDG claims the number of virtual servers in use has surpassed that of physical servers…

23. A few comments:1) As a reformed sysadmin who worked closely with one of the early cloud providers (Loudcloud), I know that it is incredibly painful to debug complex systems and application issues if you don’t know exactly what the underlying infrastructure and configuration looks like. It’s nice to say that AMZN will take care of everything but if there is a production problem, you will have limited information on AMZN’s supporting infrastructure and thus your ability to debug will be hampered. If you’re a small site and your cost of downtime is small, then this is most likely a risk you’re willing to take, but the enterprise will not stand for this level of obscurity. I wonder if AMZN is willing ot give out this kind of detailed info, and if so, to which customers?2) Related to #1, for AMZN to run the table on the cloud hosting market, they would need to sell their solution like other hosting providers, which would mean they’d need to build the requisite support, sales, and channel relationships. I’m not saying AMZN can’t figure this out, but to date I’m not sure if they’ve put these building blocks in place to support enterprise customers.3) Security in cloud environments is another one of those things that will stop an enterprise dead in their tracks if they don’t get a high comfort level that their business processes and data are safe. The virtualization software vendors (e.g. VMW) have heard this for years and have had to build both services and product to address those security concerns. I think AMZN and the rest of the cloud gang are going to need to do the same.

24. In reply to jmcaddell: A distributed competitor exists, p2p. Just give it a bit more time.That said, just a few things to add:- The Berkeley cloud paper is a good read. – Will, at least, the AWS API become a de facto standard?- Ultimately, I don’t think Amazon will be able to–or even want to–be in this business at the scale that’s required. It’s not their “core competency.” I think they’ll license/sell the technology to someone BIG, e.g., who knows how to run IT at the scale that’s required, who wants to–and can–get into this business, etc.

25. Funny this. Just had a conversation yesterday about whether cloud service potential is priced into AMZN.Anyway, from our small vantage point, we have seen a number of startups using AWS, but its not yet clear how adoption is going in larger firms – maybe just toe in the water stuff.Can they win? Well I think there are some interesting effects in terms of being first1. they have benefited already from lots of feedback from early users, so have been able to refine pricing, product, etc multiple times. And they have built a community or promoters, experts etc. So its a great environment to get started, as a developer, I believe. Just check out the level of activity in the forums – http://developer.amazonwebs…2. it doesnt appear to be like a traditional hosting decision – since it requires learning an API and integration, so its more like an OS, CMS or maybe DB selection, I think (happy to learn something here). The distinction I think gets to switching costs. 3. positive feedback loop – more people use and refer and it becomes harder to break in, particularly since they seem to be doing an excellent job. Again benefits of the early community.Looking forward to seeing more comments.On a somewhat related note (in terms of winners and losers as a result of cloud plays). A question, if I may. Who is going to suffer as a result?- hardware vendors like servers, networking equipment etc- hosting providers?- what about DB and security folks?Seems like a lot of pieces go away in a cloud world.

26. There is one problem – usability.I’m a HUGE Amazon Web Services fan and have been trying to find an excuse to use EC2, S3 and all the other goodies (I found one but I’ll stick to stealth mode for a bit).That said, I think usability is a major issue. The barrier to entry is very high. I spend a fair amount of time coding and developing (as a hobby) and it took me quite a bit of time to get started with AWS. Once you are working on it there are areas that lack refinement. Everything is done on a command line. Sure, there are 3rd party apps that help you visualize what you are doing (for Firefox and stand-alone) however there isn’t really a solid interface.In my mind, the only way Amazon will succeed is if they also invest some time into the interface. Not from a development side but from a management and reporting side. For a small startup this means being able to outsource development but still be able to manage day-to-day and run strong reporting.Then again, Amazon is leaving all of this up to 3rd parties. Not a bad idea but it diversifies the user base and thus makes it a bit harder for them to “rule” every aspect of cloud computing.

    1. Avertiz, check out the new web-based console Amazon launched about a month or so ago. Significantly easier to launch instances, assign elastic IPs, set security protocols, etc. And I have a panel running on my EC2 instance, so I rarely need to use the command line (although it’s still there when I need it).

       1. Thanks. I’ll give it a try. Guess that kills my objection. Sounds like AWS will definitely be the strongest player.

          1. Forgot to include the URL: https://console.aws.amazon….

27. Just got off the RightScale webinar for EU deployments on AWS. Sure is nice to have the same tools managing our deployments worldwide.By using RightScale as our “management console” we increase our portability between cloud platforms (AWS-US, AWS-EU, Mosso, and FlexiScale) and eventually, racked servers (RackSpace). This is a big consideration for bandwidth hungry businesses like ours.

28. whoa – another great set of comments for me to learn from – thanks all!

29. You also need to consider that the biggest advantage for a startup to use AWS is it allows them stretch their capital even longer than you may think. Using AWS is an operation expense, “OpEx”, whereas owning their own servers mean is a capital expense, “CapEx”. This may not seem like a big deal at first, but it has some tax implications that is not favorable to the startup, usually when they are still bootstrapping and small. This is important to a VC that is invested in a startup, I think…..On the question of lock-in, you need to consider the AWS services individually. EC2 gives you compute resources and it is your choice as too what software you install (open source or commercial). The source of any lock-in in EC2 can only be derived from the software you have running of host. If you know of any commodity server vendor that has lock-in on the basis on the hardware alone, I would like to know more about them. Lock-in is more likely with Google and Microsoft.With S3, moving your data to another backing store cannot be a problem. SQS can be replaced with a handful of mature open source queuing and messaging frameworks, you just have to change the interface. If you use SimpleDB, there are backup strategies for your data, and importantly, other cloud databases will most likely converge, over time, on some kind RESTful SQL that will allow different cloud databases to be queried and manipulated with the similar URL structures.On the question of whether commoditized hardware (PetaFlop handheld servers and the likes) will render AWS useless, I can only say that it is a possibility assuming AWS does not evolve to adapt over the time it will take this commoditization to occur. Even if it occurs sooner, the cost benefit from not keeping the staff required to administer and maintain your own server fleet is significant, at least to the VC.Lastly, AWS is still growing and things like interface and UI issues are typically not fully resolved as there are frequent changes in those parts. Over time as AWS penetrates deeper into the market, these issues will be reasonably resolved.

30. I think Google App Engine’s XMPP, and background job processing support can be a game changer… Scalability of these kinda jobs will be automatically guaranteed on app engine, while on Amazon you are left on your own or depend on RightScale or Elastra likes….

31. We’ve used S3, EC2, and some of the Rackspace offerings in our prototype services and realized that for our target customer (enterprise data backup) there isn’t enough transparency for them to be comfortable with those third parties. Specifically, they were concerned about certain types of compliance (financial and HIPAA), data security (can they trust the other party), and visibility of problems (what is the codebase and infrastructure like that is hosting the application and data). To address this, we’ve used an open-source platform for distributed data storage and backup and offer support and consulting services to help each company meet their specific needs.

32. I have been considering moving things to AWS for a while and I’m not sure why we haven’t done it yet……probably just the fear of downtime on an ecommerce site when making the transfer but I know it’s the right thing to do…..it just has it’s place on our priority list at a startup.

33. Great thread! There’s no question AWS has it all wrapped up for the moment. Smaller cloud platforms are starting to force pricing changes, such as the recent announcement regarding AWS “reserved” instances, but no single or handful of smaller cloud providers will encroach on AWS’s ubiquity anytime soon. The development that will upset AWS’s dominance will be a repeat of history; just as OS openness caused monopolies to wither and new lines of business to develop, just as the Web spelled commoditization for bandwidth vendors, cloud resource openness and coopetition will lead to a “golden age” of cheap, high-quality computing resources provided by a great many businesses. Imagine buying servers, storage, and data persistence from the lowest bidder at any given moment. Imagine an interface that can bring up a server instance anywhere based on open provisioning protocols and virtual hardware profile standards. Pricing, reliability, geographic distribution, and network throughput will determine which vendor gets your business at any particular time, and instances across vendors will be able to talk to each other. We are a long way from this being a technical reality, to be sure. But it’s the thing that will push up the quality bar for cloud services and reduce the cost. I’m afraid it will also spell the slow death of well-architected, resource-constrained applications 🙂

    1. So, yes for now, but not for long?

       1. Correct. AWS will have to play ball when protocols and standards level the playing field for virtual resource management. It’s hard to say how long it will take, but if I were a small cloud business now I would put some energy into standards definition and start rallying my competitors to collaborate on them.

34. Amazon is way ahead of the game and they are building Amazing services. I think it is going to be hard to overtake them at this point.

    1. I knew you’d be in that camp alexBut its pretty interesting all the different reactions in this threadLot’s of opinions to filter through before I buy more AMZN

35. Application latency is one issue. Another is custom components built into the database. We have a number of custom built search tools which can not be supported by Amazon. Would have greatly helped us get started and is a much more capital efficient way to get started.

36. I just chatted with a Mosso representative. And this is what he said about the price compared to dedicated hosting:ok, so you’re looking at about $87.60 – $175.20 a month which is going to be much cheaper then a dedicated sever which starts at $399 a monthMuch cheaper than dedicated hosting, for sure. Here is the link to the pricing modelhttp://www.mosso.com/clouds…

    1. Amazon’s cheapest EC2 for 1 year comes for $50+bandwidth/month

37. For those of not in the US – be aware that could be legal implicationsIf you are European and under safe harbour laws – where is your data?It could actually be under that Patriot Act –

    1. ugh

    2. AWS has a datacenter in Europe to serve European customers, you can restrict your data to stay within that region.

38. Love Amazon cloud services–but cant use it—until they figure out how to offer PCI or security as a part of their platform–I gotta build my own….that holds for all financial apps

39. Haven’t had the time to read all of the comments here so if I repeat anything said above please excuse me…Most of the time when people talk about Amazon AWS they are talking about today and not looking at the path that Amazon has followed and is following. Amazon has been extremely intelligent in making sure that what they are doing is open. I can replicate anything that I have created on Amazon AWS outside of Amazon. It reduces my fear of lock-in. Of course, they are locking me in by the quality of their service and the price point that they give me. And they are constantly improving that price point and service. Look at the recent announcement regarding pre-paid EC2 instances. You can pre-pay for EC2 instances you know that you will be keeping at Amazon and save 30+% on the cost. Yet they still stay open. I don’t have to make every EC2 instance a pre-paid instance, only the ones that I want to be. So it makes sense for me to lock in the instances I know I want to keep at Amazon and keep the others regular instances. Yet…they are still locking me in, and I am going willingly.If you follow what they are building, they are preparing for the Enterprise. This is where the really big money is. This year they gave presentations in NYC where they invited the Enterprise, last year it was the Startups. They have designed Amazon from the beginning with security in mind. One of the first things that impressed me was their layered security model. Have you heard of anyone taking control of someone else’s instances? (I am not talking about taking over a web site here. I am talking about booting up or bringing down the instances themselves using Amazon’s apis), I haven’t. Each piece they build out is ultimately with the enterprise in mind. Startups are just fledgling Enterprises and are perfect testing partners for Amazon. It is not just that Startups flocked to Amazon, Amazon actively recruited Startups.I realize that I have been talking about EC2 a lot because, for me, EC2 is the gateway to all of Amazon’s Cloud services. If they gave EC2 away their business would explode! Think about it, if people could run a server (or your own workstation even) for free, they would. Now they need storage…S3, then they need instance persistence…EBS, then they need a database…SimpleDB, then they need a message queue…SQS, then they get bigger and need a content delivery network…Cloudfront, etc.etc.etc. EC2 works so well, why would you look somewhere else?Amazon will not come to dominate Cloud Computing, like Google does search, they already do! They have taken their first mover advantage and pushed it so hard and fast that I don’t see anyone taking a significant amount of the market away from them. Even Google will not be able to do it. While Amazon locks you in, Google does it in a blatant way and that will scare the Enterprise and the Enterprise is where Cloud Gold will be won. If Amazon and Google were sitting across from me, and I was the CIO of the company, and Amazon told me I could place my existing operating systems and legacy applications on their cloud infrastructure and Google told me I would have to re-write everything in Python, I would have to give pause. Then Google tells me that, ‘Oh, by the way, the stuff you create only will run on our servers so you won’t be able to leave us’…hmmm. ‘Nice to talk to you both, goodbye. By the way, Amazon, what is your number again?’I have a lot more to say about this so, rather then say it here in the rambling words of a comment, I will put together a blog post and point you there….Doug

    1. This is basically the thesis that prompted this postYou are in agreementBut its important to note that many are notMy takeaway at this point (8 hours in) is that Amazon is running the tablefor now, but that could change and there are opportunities for others tobuild differentiated products

    2. If the world stopped today you would be correct. But what happens in the next 3 years, the Enterprise is much different than the startup world and yes Amazon has the lead in the space, but they need to catch up on the Enterprise experience. Other players understand enterprise more and will need to catch up in the large scale infrastructure. Who will win is still very much in the air.And there is a fair amount of lockin for an enterprise on Amazon, they would need to take all of their existing applications and modify them for SimpleDB, S3 and EC2 apis. On a small scale it is very portable, at an enterprise scale many more tools and “glue” frameworks will need to be added.Also don’t forget that no large company shifts at once. There will be a lot of experiments and many larger companies will start with internal clouds that may connect to public clouds in case of extra computing needs.

    3. Excellent comments

40. I think there is another angle with Amazon and that is what I call their “religion”.Look at the other players in cloud computing and question whether their cloud offerings are really channels to deliver their products. With Microsoft’s cloud, you can bet that you are getting locked in to their offerings. Similarly with Oracle. Google has its own agenda. You can imagine the same with Dell, IBM, HP, EMC, etc. Contrast that with Amazon who has nothing to peddle with AWS. This agnosticism means that as an architect, I can build without concern that my vision violates the sales strategy of the infrastructure folks. I know I’m not locked in. It ensures that I can keep picking the best ideas, the best technology and the best design. It’s the freedom to optimize!

    1. YupI said it a different way somewhere else in this comment thread but this isa political issue and google and microsoft aren’t in the right party on thisone

41. Amazon is doing some great stuff in the cloud space and I think they will continue to be a major player. But if I was a startup building the next big thing I would avoid “vendor lock” by building on top of SimpleDB or GAE as these are one way roads and it is very hard to turn back and go a different route if you run into a road block. Especially because all these services are in Beta which means they could make major changes to their architecture/design/pricing (such as Microsoft did last week with SDS) which could cause major disruption to your plans.Instead at this point I would choose to use a traditional stack in an Amazon virtual environment, or if I want to go bleeding edge then at least a movable stack including movable layer (maybe built around Mongo, Couch, Volde or whatever).I think building major apps on SimpleDB or GAE’s abstraction over BigTable without an easy migration path is risky at this point. Having said that I know of smart people working on this problem who are putting together kind of a cloud data fabric which will allow you to access any data store you want using a standard interface, behind it you can plug in whatever platform you want (SimpleDB, Mono, CouchDB or even traditional relational MySQL, SQL Server, Oracle etc) which will make this somewhat better.

42. Google will have something to say on this, they won’t ignore a big market next doorMaybe its just me, but does anyone else think the recession ended this week?

    1. I sure hope the recession ended this weekI kind of doubt the bad economy is over but I do think we may have foundbottom

43. I beg to differ on this one.Amazon.com has not demonstrated they have the scalability, support, and SLA to support large-scale enterprise customers. So far, most AWS customers are start-ups. A lot of Amazon.com’s internal software were built in late 1990’s or early 2000’s. Some of the AWS API calls the internal “legacy codes”. I’m still skeptical about AWS’s scalability and reliability.Also, in recent years, Amazon.com had several major outages and they never revealed what exactly happened. Transparency and meeting SLA requirements are going to be crucial for any large enterprise customers.Amazon is fundamentally a retail company. It’s going to be a tough transition for them to move into a cloud computing providers with the level of service and supported required by enterprise customers.

44. An oldie but a goodie – http://www.wallstrip.com/20…

    1. Great cameo DanThat was when Wallstrip was greatThanks for the memories

45. Fred, I would ask you to wait till tomorrow to know about a competitor against Amazon Web Services. I am pretty sure they will give a good fight to Amazon Web Services, especially EC2. I have a gut feeling that their move in this field will bring down the prices much more in the coming years.

    1. I am all earsI have not bought AMZN yet but still thinking hard about it and this commentthread has certainly made me a lot more intelligent about the issues

46. I was at the Startonomics conference in socal earlier this year, and I believe Jim Benedetto who used to be the CTO of Myspace until just recently talked about cloud computing.One thing that he talked about what data ownership. It was more of a “what if” type of idea but what happens if there is any legal issue and amazon steps in and they cut your off from their services. I don’t think this has come up yet, but most likely will in the future.

47. I agree 100% with what you said here.We use EC2 to run Kanchoo (http://www.kanchoo.com), our service that publishes content to native iPhone applications. If it weren’t for AWS I don’t think our start-up would have been possible.

48. The way it’s looking, yes. Especially in the last 12 months with introductions of EBS, the console, and promises of load balancing and automatic scaling coming soon. If someone tries to overtake them they have a lot of catching up to do!If someone were to overtake them they would have to a) provide the services they offer and more and b) have a name people could trust.I’m building my next service using EC2, EBS, S3 and SQS and even if a startup or medium sized company tried to offer the same services and more I would not go would them simply because I cant trust that they’ll be around and committed to cloud commuting 2 years from now.So assuming you dont want to use Windows for everything Google emerges as the only viable competitor (maybe Sun could have a go?) and they don’t seem to be in a terrible hurry to improve their App Engine product.

49. Don’t know about “run the table” but at least own a key piece. Especially if they can transfer their UX skills to the apps roll-out, management and billing process.Reliability and price are a given – but ease of deployment, ease of change and simple, transparent billing are great ways to help app owners. Amazon should excel there.

50. We are only 3 weeks into Amazon EC2 as our hosting service (have been using S3 for images for about a year), and they haven’t been fun weeks. I also think this is the way to go — having experienced two non-cloud offerings elsewhere — but at least for Java-based web companies, there is some significant adjustment to Amazon. They seemed to be an easier transition for Ruby and PHP shops.

51. Great post and comment thread – we use AWS for a large part of our infrastructure (http://www.3scalesolutions.net) and its made things possible that would have been a major headache with normal hosting. Right now I don’t see any other player touching Amazon in the space if they keep going (more data centres seem invevitable in the next 12-18 months though I think they’ll take their time).One thing I saw mentioned only in passing though is problems with geographical proximity – which is one area where other players might emerge. By far the biggest threat is the huge existing hosting market which is massively fragmented – but adds up to huge number of hosting centers worldwide. If one of the players can set up a franchise play by which may of these join together to form a distributed cloud that could be a major threat to amazon. Companies like 3tera (http://www.3tera.com/ – no relationship to 3scale!) are already doing this – penning deals with data centers all over globe and having them run their cloud software. Whether they’re big enough to make it stick is another issue – but it makes a lot of sense. There are lot of companies (including us) that would love to have identical, API managed hosting conditions in 10-20 strategic places around the globe.Whoever makes headway on tying together this “meta-cloud” could give amazon a very hard time.Hard to see how smaller current cloud players are going to compete – going to head-to-head with amazon on price is going to be tough, but lets hope someone emerges to keep amazon honest!

52. I’m late to the party here, but just read this article about Sun’s entry: http://venturebeat.com/2009… …. emphasizing data portability as a selling point….probably was overshadowed by the other Sun news today.

53. Stay long. We moved our sales collaboration solution called Broadchoice Workspace from a dedicated hosting environment to Amazon. Hosting cost went from $12k per month to $1k. Response times are much better, and in my opinion, it’s more dependable. When we need to increase capacity, it takes less than 20 minutes. Oh, did I mention we no longer need a dedicated IT person as the AWS work is being done by our engineering team. Game over – Amazon wins!

54. Short term, AWS has the best cloud option out there. We have put three of our clients on their server infrastructure and we use their disk based services for backups.However, there is still a long way to go in the cloud space and it is crazy to think that cloud services will mean the same thing to different people. Salesforce has some interesting cloud based services but they are completely different from what AWS does.Also, do NOT count MS out. We are a beta tester on their cloud based database services. From what we have seen, these have the potential to blow away anything else that’s out there – imagine SQL server in the cloud. Their MESH software is unique and we use it to synch files across sites and to easily manage computers remotely.Cloud based computing is in its infancy. AWS is a strong player, but there will be many out there that will compete head to head with them (probably not the greatest idea given how strong they are in their “space”) and others whose approach to cloud competing will be very different and may prove superior.