A Look Back At Summize
In July 2008, Twitter acquired a company called Summize. The stated reason was to bring search into Twitter. And that was the main reason that Twitter did the deal.
But Twitter also got a very strong engineering team in that acquisition. I remember asking Ev and Jack at the time "how are we going to integrate the engineering team?" and Jack replying "we aren't going to integrate them, they are going to integrate us".
At that time Twitter was experiencing very difficult scaling problems. The service was down a lot and was slow when it was up. The "fail whale" was the symbol of Twitter and it was a huge problem.
While Twitter hasn't slayed that beast entirely, it has come a very long way in less than two years. In the crudest measure of its traffic (monthly UVs to Twitter.com), Twitter has grown 50x in that time and has increased its reliability and scalability dramatically.
And the Summize engineering team had a lot to do with that. Greg Pass who was Summize's co-founder and VP Engineering became Twitter's engineering leader in the summer of 2008 and has built the team from roughly a dozen to somewhere around ten times that number. In my view, Greg is one of the unsung heros of the Twitter success story. He brought a calm, steady hand to a ship that was caught in a storm. He got it going in the right direction and headed for calmer waters.
I remember asking Greg during the Summize due diligence what his plan was for stabilizing Twitter. He answered that there was no magic bullet. He said they weren't going to do one big thing, they were going to do lots of small things. The first thing they did was instrument the hell out of the system, they started measuring everything and finding the bottlenecks, and then they started knocking them down one by one.
Twitter has an entirely new architecture now. But they did not rebuild Twitter, they just replaced one thing at at time and evolved it. They went from a monolithic beast where everything was connected to a distributed set of services that work together but are separate from each other. And that is Greg's legacy.
I'm telling this story today because Twitter has announced that Mike Abbott will be joining the Company as VP Engineering. And Greg Pass will become Twitter's first CTO. As Greg passes the engineering baton to Mike, I want to make sure that we recognize the extraordinary contribution that Greg and the entire Summize team has made to Twitter.
There is a saying that success has a thousand fathers. That is true. But only a few of those fathers are well known in most successes and many of the big contributors are unsung heros. That's how it is with Greg and the Summize team and so I am singing their praises today. Well done.
Twitter responded extremely well to what was truly an inflection point for their business. It is absolutely just as likely that twitter could have become a zero had the problems continued or gotten worse.The fact that an acquisition actually HELPED solve problems and helped turn a major, major turning point from “fail whale” to a platform worthy of Presidential candidates is amazing.You’ve praised Greg here, and I’m sure very rightfully so….but I think that this also speaks volumes about the existing upper management of twitter (and yes, twitter’s investors/board) that they were so willing and able to make full use of the intellectual capital and ability of their new team.MOST organizations would have simply plugged in “search” and had Greg and those guys running the search unit.Maybe it’s the desperation of the fail-whale or maybe it’s the winness of Biz and the team (likely both), but I admire all involved and I would love to hear some of the “here’s why and how we integrated the way we did” stories.
Agreed. the quote from Jack says it all, “”we aren’t going to integrate them, they are going to integrate us”.They knew that the Summize team was gold and would only enhance Twitter.
Not too much ought to be read into that statement by Jack, as in don’t take it completely literally. That statement also goes on to show, besides that the Summize team was stellar coming in, that Jack is not threatened by smart people. http://democracyforum.blogs… Considering I know little about the guy, makes me respect him more than I already do.
it doesnt take Ghandi or Einstein to accept help when your house is burning to the ground.-mike
Heheh, good point!
Having scaling challenges is different from a house burning to the ground. Twitter was growing like crazy at the time. That was not a house burning. Wrong metaphor.
I think some of those stories might have to wait for the post-IPO phase. Maybe Fred will write a book some day, My Twitter Story by Fred Wilson. Even in this blog post you can feel the tension of Fred artfully trying not to reveal more than might be absolutely necessary while also singing praises of an important below the radar Twitter figure.
Maybe we can just get Kitty Kelly to make shit up in the meantime
Andy. Happens all the time. People make stories up all the time. Like that Fred post on Twitter that got such wide “press” coverage and raised so much dust, in that post Fred casually mentions this company from the early 80s. The day after I read in a newspaper article that Fred worked for that company. If he did, he did not mention that in his blog post.
don’t look for a book from me. books have endings. a blog doesn’t.
I was suspecting you’d say that because I think you have said that before. Well, let the cats (stories) out at your blog when you feel the time’s right. I am all ears.
For those who know him, Greg is a humble star, and a wonderful person.
Most humble people are. It takes a lot of training to be humble.
There is a saying that success has a thousand fathers. what, no thanks to mothers? damn boss didn’t expect you to go misogynistic on us. oh well. hope no one calls you out on it.
I’m pretty sure that saying comes about because everyone always KNOWS who the mother is.That’s kind of the point of the saying.
LMAOKid. Success has a thousand fathers is an old saying that Fred can’t take credit for, and many old sayings are sexist, but this is not one of them, this actually might be a rare, anti-male old saying. Success has one mother and one father, just like children. But once success happens 999 male jerks show up wanting to take credit. That has been seen to be the tendency. It is usually men!The number one is not to be taken literally here. Usually there are a few people who are truly responsible for the success. But many more show up to take credit.
Woosh!(that’s the sound of the joke going past you)
They were to busy covering themselves up so they don’t cause earthquakes!
I saw that quote- so infuriating…
the implication is only one mother. that’s the whole point of the saying
I assumed Kid was yanking your chain and wanted to give him a +1 for comic value.I laughed out loud.
lol yeah def just trying to start controversy 🙂
Cool story, thanks for sharing, and congrats to Greg, Mike, and the evolution of the team!
This reminds me of the old Henry Ford story. He sent men out to junk yards to see what parts of the Model T were breaking and what parts never broke. Then he told his engineers to fix what was breaking and to save money on what as lasting too long.
This a great success story on several levels. I love to hear of acquiring companies keeping their promises to the employees acquired and using them to accomplish something great. It’s all to common to see the opposite happen. Thanks for sharing this.
At the other end of the spectrum we get marriages from hell: http://technbiz.blogspot.co…
I like that piece about “they are going to integrate us.” I think that speaks volumes to the culture of Twitter, as does the attitude of knocking out one problem at a time. From a user perspective, I’ve been seeing these changes, one at a time, and to me, it’s been making a world of difference in how effective the service is. I’m excited to see what else Twitter rolls out in upcoming months/years.
yes it does. in all the years of doing acquisitions, i’ve never heard it said. it struck me.
Always good to see engineering/development talent getting recognized for great work. Too often I see business users forgetting who keeps the systems going that help drive and support their successes.
Great story. It’s a nice companion article to what I read earlier today:Want to Know the Difference Between a CTO and a VP Engineering?
This is an example of good engineering. I’m glad it is getting recognized. This reminds me of a story I had heard from a Naval submariner about fixing the ‘big noises’ first. In this situation, the ‘big noises’ were the bottlenecks that Greg identified and fixed quickly. These small fixes can really add up and were key for a service like Twitter because it scaled up so quickly.
I’m curious what Greg’s background is in. The great thing about his approach is that it rings true for just about any business problem.I’ve taken the same approach to sales processes before. Measure everything temporarily, figure out where the problems are, and make a series of small changes until the whole machine is singing. Then back off and measure only the two or three things that you need to know.Great story.
Agree with the comment concerning Surmize and Greg Pass. The Surmize team was made up of excellent techies with a whole lot of business sense to boot and averaging it out, a sense of humbleness and plain get it done mentality but without the “sure we will get it done in a week” keystone cops routine.
I remember using Twitter in 2008 was major fail, but I still used it because it was fun to use Twitter. From that point in 2008 to today, I have not seen the Fail Whale at all.Great Job Twitter Team and especially the Engineering team from Summize.
Wow, Greg has been very busy keeping the entire platform together. I had no idea. Twitter is the biggest observer design pattern gone company I’ve seen. Really love Greg’s technique of small iterative patches to shift the architecture. There was a bigger under the hood switch to some Scala but I forgot the precise details of what bottleneck they were ameliorating.My personal usage has been one of limited success with search on the twitter site. If Greg or folks in the search team are listening, there are few things I was spoiled by from other search engines (not just social):-user delimited lists or groups (massive inflow lists, not just a handful of users on the API Get call)-social review quality: favorited tweets, likes, reshared statuses, frequency of occurrence of links-time windows (even older than 7 days ;)-tweets from users that are on more than X lists, or have more than Y followers, or who are on any of my input follow list-broader categories (only exact string matching feels like it works for me)-semantic relationships (Zemanta could help here)These are all features which would make Twitter search enormously more powerful and have an impact if made available through the APIwish list:Full api access to search, heck just make it a protocol instead of an API so that other social services can adopt it, extend it, improve it and pass the learning back to Twitter
“….Greg’s technique of small iterative patches to shift the architecture…”Reminds me of the human body. We die about a half dozen deaths, counting all the cell deaths, before we actually, formally die. Some biologist help me with that number six.
Fred, nice post. Engineering rarely gets acknowledged outside companies.
Hello Bijan, was at your tumblog earlier in the day. The post about Tumblr. Followed a link to it from AllThingsD.
it takes a really cool VC to write a public thank you to an engineer in one of their portfolio companies.much respect.
Yep. It’s a gutsy move.Sam Walton has said in his autobiography that a knowledgeable, heartfelt thank you or word of praise to the deserving team members is one of the best things you can do to raise morale in your organization. I hereby “accuse” Fred of management through blogging. :-)Also another anecdote I am tempted to pull in here, I think this comes from Steve Wozniak. He said he hated corporations. When you do a good job as an engineer, they go ahead and promote you, and you no longer get to do what you love doing, now you are instead a manager, no longer an engineer getting down and dirty. 🙂
It was gracious of Fred (and good business, too), but I’m not sure what was “gutsy” about publicly praising a key employee in one of his portfolio companies.
What makes it gutsy is you don’t see it done all that often, not at all.
That makes it rare, not gutsy.
No. Gutsy. In my book.
Your book obviously isn’t a dictionary.
I think this is about our fundamental disagreements on immigration.
It has nothing to do with immigration. I’m just a stickler for precise language. “Gutsy” connotes that Fred took a risk by posting this. I don’t see any risk, save for getting hit with a wave of admiration for being nice enough to give some credit to this fellow.
Praising Greg Pass so very publicly was gutsy at so many different levels.(1) You have to have the brains to identify Greg Pass has abilities. More brains to be specific about what those abilities have been. (2) You have to be somebody to personally know people like Greg Pass and Jack Dorsey.(3) You are sticking your neck out in praising a Greg Pass that seemingly noone has heard of. (4) It requires skill and character to praise Greg Pass without coming across as self-serving or showing off. (5) Why pick Greg Pass? All of Fred’s 28 portfolio companies must have CTOs. And I know of at least four people senior to Greg Pass in rank at Twitter itself. I have never read a full blog post by Fred that praised Ev. (6) You have to have judgment to decide you will praise Greg Pass and get away with it. (7) You have to judge Greg Pass will be okay with all the attention that will follow. He obviously does not seem to seek limelight. (8) You have to make a call that this does not come across as too much of an inside story, a case of Fred giving the farm away. (9) It is gutsy to manage to praise the Summize team without in any way slighting the original Twitter team.I think I could go on. But the summary is this, it takes a strong leader to praise a junior member in a specific, meaningful way. And Fred is pretty precise in his praise. That is why it worked at so many levels.
It was smart and nice of Fred to praise Greg publicly. I can think up some other positive adjectives to apply to Fred’s decision. But, again, not gutsy. Here are Webster’s definitions of gutsy. I don’t think any apply to this post, but we can agree to disagree if you like: 1 : marked by courage, pluck, or determination 2 a : expressing or characterized by basic physical senses or passions <gutsy macho=”” talk=””> <gutsy country=”” blues=””> b : rough or plain in style : not bland or sophisticated
We can agree to disagree. Sure. But his act fits definition number one. To me it is so obvious it was gutsy to so publicly and profusely praise Greg Pass.
i remember thinking in 08 that the purchase would go down as one of the biggest bargains ever. is it safe to say twitter’s business is built around real-time search yet?
It sure was a compelling acquisition, as is tweetie. Twitter might end up a mobile web only thing for the large part in a few years. Although it is uniquely suited for both the mobile web and the old web.
It sounds like a big success leading the engineering team as well as changing the architecture.We had a scaling problem at Flycast after the business took off. It was frustrating because at the time, the team didn’t have the experience and confidence to make an architectural change.If some hotshot had walked in the door and told us to move piece by piece to “a distributed set of services that work together but are separate from each other” I would not have believed it was practical. I needed a lot of exposure to well-designed services and critique of my work from legitimate hotshots, before I “got it”.
I know this is a bit off topic, however I was unsure of where to go to ask it:On my blackberry curve I can view AVC fine, however with the default browser I cannot view/add/edit comments. Can anyone suggest a browser to do so?
Thanks Shana! I’ll check it out!
Your welcome (though if the comments get super layered, say like 5-7comments in a row it will fail- however it will let you post for under that)
Try operamini.com from your BB browser…..install that. It’s a good browser especially for those sites that have a lot going on and require a more “full” view…….I’m not sure if disqus works well on it or not though but it’s worth a shot.
The name ringed a bell so I searched my emails and found one I sent to the Summize team back in June 2008 about the geolocation information provided by the API.Some Greg Pass 🙂 replied almost inmediately and was very helpful, which was a nice surprise.
Great post! Makes you think about the importance of the “Kaizen” approach for goal setting. http://en.wikipedia.org/wik…Thanks Fred, Chris in Cambridge, MA
When my group did some contract development work for Twitter, I never actually met Greg in person, but it was clear he was the soft spoken Man in Charge– you just knew it at the critical moments. And in those moments he emerged as a total class act.Looking fwd to seeing the next phases with him as CTO.
http://twitter.com/paramend… Wow. What a story. And so very well told, by an early believer in Twitter, an insider. And a great story teller. This is a nugget of a blog post. Twitter is USV’s crown jewel, in my opinion. Actually I think that is a pretty objective statement. http://technbiz.blogspot.co… This story also highlights the number one or two reason why Twitter needs to eat into its ecosystem: to acquire talents like Greg Pass. http://technbiz.blogspot.co… I also liked the phrase the “first CTO.” Twitter kept that throne empty until they thought someone had become worthy of the crown. I am impressed.
Who is this condescending Paramendra guy ?
Condescending? Of whom? As for me, can google me up.
I just did. http://en.wikipedia.org/wik… lol.http://www.paramendra.motim…I wrote a book in 1993 that got me an admission into the University of Chicago into the Economics program, top in the world. Why didn’t you go?
They gave me a spot but not money. So I went to the college which has the best financial aid program in the country.
Dude Paramendra,Thanks you for the shameless plugs to your rather useless and content-free blog, your irrelevant posts, your redundant insights and your fine pic.Oh how we love you.Frustrated readers of this blog.
First, I don’t know why this guest decided to post with John again, but I don’t care. I don’t care that he’s making redundant insights and making his shameless plugs to his blog. At all, actually. What I do care about is his portrayal of himself. If you check out his aforementioned blog, there’s some stuff that I find a bit shady/wtf. If you got accepted to UofC in its econ. program, why didn’t you go? You could have deferred and gone back in a few years, which is acceptable and many do it all the time, but instead he chose to go to Berea College, which isn’t bad, but not comparable to UofC. And the talk about his feeling a sense of gravity that apparently contributed to his downfall of opportunities and grades, etc… I just don’t understand all of this. It’s just weird.
Dude, he’s just another fake entrepreneur massaging his ego. Take it easy and leave him alone. Paramendra – please continue with your insights. Infact, get on techcrunch!
I comment a lot at AVC. When what I am about to say I have already said in a blog post before, I link to it. Do you feel like I am overdoing it? And thanks for liking my picture.
It’s very good you did this for those talented engineers, bravo! The other side, however, is also true: there is always moochers and free-loaders who would take undeserved credit only because they were there, loosely associated with a success story…I hope people would start calling such parasites out; they only dilute credit for the stars, while building phony reputational capital, which can be dangerous at a later point
I’ve always wondered how a talent acquisition is valued. It must be difficult to get to a number.
dollars per engineer. more if you get some tech with it. less if you don’t.
friendfeed acquisition comes to mind as a recent pure talent acquisition given friendfeed has been left to whither. $47.5 MM in total, 12 employees (probably all engineering talent). Certainly came with some technology . . . My guess is as high as it goes on the $/engineer scale.
Good to hear about the awesome work done by Greg and his team.Kudos to them.I don’t think most of people heard of Greg or Summize.
Thanks for that story.
This post is a nice companion to mark suster’s recent post on the difference between a CTO and a VP of Engineering. http://www.bothsidesoftheta…
i knowwe wrote them on the same dayinteresting coincidence
I’m confused. I don’t understand why everyone is saying that Twitter’s scaling problems are behind them. I get connection problems just as much as before, the difference is that my browser says something like “The server dropped the connection” instead of seeing a fail whale. I’m not sure how that is any better. Maybe it’s just me.
Really it’s an awesome blog.I am very happy with your post.I like the details provided here.Thanks a lot for sharing.Keep up the good works.
Haha, this is pretty hilariously inaccurate. Ask anyone who’s actually worked under Greg, especially in EngOps and I think you’ll hear more or less the exact opposite. He’s really great at managing up and really really terrible at managing those below him.