Counting The Hits
Venture Capital is a hits business. All of the returns come from the top cohort of investments. So figuring out how many “hits” there are over a given time period turns out to be a useful exercise.
Aileen Lee posted her attempt to do that on TechCrunch a while back. I countered with a post of my own on the topic.
Those two posts started William Mougayar on a long process to figure out what the right number is. This week he published his findings.
William has found 235 global “tech” companies that were started since 2000 that have gone on to be valued at north of $250mm.
I think William’s efforts are the most exhaustive I’ve seen to date but I don’t think the list is anywhere near complete, particularly for the $250mm to $1bn cohort.
I pointed him to three or four of our portfolio companies that are carrying valuations north of $250mm that he did not have on his list. And I am sure there are many more like that out there.
My guess is an exhaustive list of global tech companies founded since 2000 that have gone on to be valued at north of $250mm would be 400-500 companies and possibly more.
It is very unusual for a company to get to $250mm in valuation in a year or two so you would not have many companies started since 2010 on this list, yet.
So if we take my guess and divide by ten years, that means there are 40-50 global tech companies started a year that go on to be worth $250mm or more.
That feels about right to me. Let’s help William compile that list. Leave him comments on his post or this post suggesting companies he has left out.
Comments (Archived):
Thanks Fred. Yes, I’d love to continue making that list as complete as possible & keeping it as a live updated document for all.Any suggestion will be vetted & curated for inclusion, which is where lots of the work is involved.
yup. the vetting part is important. otherwise you get something that looks like the hackpad we did.the hackpad is useful as a starting point but not an ending point
Exactly. Private valuations are a tough number to find (or to guesstimate). I’m sure not all of my estimates are correct.
just keep working on the list. it will get better and better
yep. Iteration…just like for a product :)I already updated/added 5 entries this morning based on feedback. Public crowdsourcing + private curation is good.
“I pointed him to three or four of our portfolio companies that are carrying valuations north of $250mm that he did not have on his list.”was that information in the public domain before this post?
yes, if you knew where to look
That should be on your tombstone, along with “Normal behaviours generate normal returns.”
i plan to be cremated
I think you’re still allowed a grave if you wanted one. 🙂
so you’re not planning to post from the other side then?
i am assuming there is no other side. if there is, then that is upside.
do you have an avc succession plan in place? who inherits the throne?
‘upside if there’ an other side.Will be requoted!
Your middle name should be Obtuse :-0Can we engrave an Epitaph or 2 on the urn?The top 5 Greatest Hits of Pithy Things Fred’s Said:1) The ‘Magic’ vimeo video2) Normal behaviours…..3) If you know where to look…..4) suggestions?5) suggestions?
Urned media?
bad pun of the day!!
I’ll wear that honor proudly, all day.
How long do they have to be valued at that level?I worked briefly for a company that was a disaster….18 months later it was valued north of $500M….18months after that, more like $20M.
there will most certainly be churn on this listi would guess you could see 20 to 30 companies a year churn off of itmaybe more
I bet many of them will be bailed out by getting acquired long before their revenues have to support their $500M+ valuations.
I think the valuation will need to be current, ie as of now, or at time of exit of course.
William — great job! As well as crunchbase is there any info from ChubbyBrain that might help?
I haven’t used ChubbyBrain. Are you part of CB Insights? Is it free or I need a subscription?
It has a free 7-day trial.
50 / total # startups launched annually = ?#depressing.
going down fast because the denominator is going up quickly
should the 50 number go up also – at what point is the number a proportion?
No that #depressing if you take into account outcomes that are still *AMAZING* but <250M in valuation. Also – even if the number of failures is going up dramatically, that’s still a good thing for humanity. As a greater denominator leads clearly to a greater numerator ( even if the ratio gets crappier )
William is a natural born analyst and industry forecaster!A noble undertaking.
Here’s my question: how many of them made it there without VC?Bloomberg always stands out as an example, but I’m curious how many others there might be.It will not be a large list. An even rarer form of unicorn.
Been thinking about this.Started then canned a post on the plane the other day–‘need to spend money to make money’.Truth–easier and cheaper to start company. Truth, except in the rare occasions, capitalization to grow, be it people, cloud management or marketing, is possibly more.Even large non VC seed round usually presuppose VC follow on.
Businesses need capital. I’m happy at this point that we haven’t raised VC. I don’t think it is constraining our growth. It does force us to make smart, focused investments with a reasonable payoff. But that’s a constraint that seems to have served us well.
Consider yourself a fortunate corner case.
I’m sure that’s true.
I look at myself in the mirror and see me lucky each and every day.Then I work like a maniac to make that true!
The best way to do it. The harder I work, the luckier I tend to get. 😉
We agree.No wonder we are doing so well and like each other.BTW–great new spot for dinner next time in town. And I always can get a table.
Can’t wait!
Pressure is good for focusing – similar to keeping one “hungry for a salary.”
Actually need is a bad motivator and as a pressure point clouds rather than clarifies.
Agreed, so long as you understand this and don’t make the decisions under pressure, I think people will be fine or rather be able to make better decisions.
Pressure is steady state of business whether you are growing or shrinking.Pressure of not making payroll trumps sleep each and every tme.The idea of zen in the face of pressure, pressure as positive, busy as a lifestyle are all code talk not reality.It just is. The best work within this but understand it as reality.
It’s true.
I started to simply reply “it’s true.” But the truth is, “it’s complicated.”Capital is about breathing room. The room to experiment, the room to figure things out and get things right.You can give a jet so little runway that it can’t possibly take off.You can be so enamored with the runway that you forget to start gaining altitude and become self-sufficient.
ONLY TAKE VC$ WHEN NEED IT. ESPECIALLY IF THAT NEVER.
I thought about adding a column with total $ raised. Should I?
Yes! Efficiency matters.
If time is money and people are given money/time, then the time it takes to reach a certain valuation should be increased as well. It would be neat to sort those who didn’t take VC and who performed most efficiently, most quickly.
Or just total capital raised.VC is just one source, and it’s the amount you need that really matters.
yes! think of the data analysis that can be done with more features!
MVP first…piano piano.
yes absolutely.
Braintree bootstrapped from 2007-2011. Never took angel money. Then they took money from NEA in 2011, 2012, and sold in 2013.
true. there are a few stories like that.
I would add a check box whether it was an exit or not.I’m stealing Fred’s line when he says when you put up a valuation when you are buying preferred stock and the money goes to the company, the valuation number is irrelevant. It really is just how much cash is going to the company and what percent does that take.
yes, there’s an Exited column. So if there’s a year there, it means the company has exited either by getting acquired or going public.I agree with your 2nd point, but that’s the number if there is one.
Add a column to pinpoint when exactly hockey stick growth happened.Also a column about market perception at point of exit, e.g.:* Sold too soon* Under-priced* Over-priced* Fair value
oy…that could be subjective :)tough one.but maybe let the crowd vote on these. good idea, but i’ll need some special software for that! maybe a future version.
Oy, indeed, haha.Years ago I was on the product team that created this database for deals:* https://www.bvdinfo.com/en-…It was a joint venture with the ‘FT’ and got acquired by Bureau van Dijk.As a banker at UBS, I was deluged by data from Bberg, Thomson Reuters, PwC Moneytree, Dun & Bradstreet etc.So I frameworked how to value transactions beyond the objective, quantitative numbers.Please read the comments between SigmaAlgebra and myself. Subjectivity is indeed a tough problem to solve in mathematics, systems and software.However, it’s do-able and that’s where I focus: data Analytics with subjective factors.
William, I have quite a bit of data on this matter for that I did for an (unnamed) VC firm. Interested in sharing some of it if you’re interested!
That would be great actually. Could you email me at wmougayar AT GMAIL please? Thanks a lot. (check the updated list, 320 companies now on it)
I believe GoPro got to those kinds of valuations long before they took VC money.
interesting
I think you’re right.
I am particularly interested in this with you Aaron – no VC.
Well, we shall see. Pretty happy so far, but the story isn’t finished yet. 🙂
Building a hard goods product company with manufacturing, warehousing, returns and the like without capital reserves is a daunting spreadsheet to internalize as your chosen path.
Since 2000 forward is the data set I believe they are solving for, Bloomberg is much older than that so would not work. Plus, Bloomberg took $30M+ from Merrill Lynch in 1983 to build out the terminal business so it took outside capital.Campaign Monitor, which Insight Ventures just put $250 million into had raised no outside capital.
Not one to quibble, but the Merrill deal is fascinating because it was largely customer financing with an equity deal mixed in. Not a deal I would do, but an interesting one. http://www.fundinguniverse….Campaign Monitor is a kick-ass company. Just used their tech for the first time yesterday. Holy cow.
Aaron, agreed with you. The Merrill deal was a bit of both, but my point was that it is impossible to say that Bloomberg grew without outside equity capital from a professional investor. While on a very large scale, I think you’d find a lot of similar deals in many of the non-VC backed companies that grew to a sizable scale. Customer funding, sometimes with equity, is a key way to grow in many enterprise spaces. Some of the best returns I’ve seen in mid-sized exit events no one would have ever heard of have been from similar deals.
Twas a labor of love. painful but useful. thanks!
I am skeptical of the divide by ten assumption because value accretes logarithmically not linearly.
but it declines that way too!
Finance geeks, cute.cc @disqus_kYmMT4Wqrn:disqus
:/ how we should think about debt, is in that statement
You are generally looking for equity returns, so I’d disregard debt capital raised. Debt is non-dilutive to the investors and has a fixed return…or a roughly fixed return if you want to account for pre-payment penalties, floating interest rates, etc. Plus debt in self-funded companies often correlates to revenue growth, such as a working capital line, so I’d set it aside. Debt with warrants or convertible debt is always hart to figure out where to put it.
Not sure why that matters if the claim is that more companies get to $250 mm earlier?
It seems like the term hits if you are thinking about it from a VCs perspective should be adjusted based on series A valuation as I assume investors are thinking about hits as big returns.e.g. Quora’s last valuation at $900M still probably doesn’t make it that much of a “hit” for benchmark who supposedly went into the first round at $86M pre and have seen them raise $130M since that may have some kind of preference.I know this is more complicated, but maybe hits should be defined as something like – expected cash on cash return for Series A investor > $100M with >100% IRR from investment?
simple is betteryou are going down a rat hole with that level of complexity
ha – you are prob right – especially for this context.Non-biased question: do you think LPs would feel the same way?
Nuances are where the value is at..
Michael — possible typo for Quora: 900M not 900B.
You’re right. Edited it. that certainly would have made it a hit 🙂
has any company held a nearly trillion dollar valuation?
Apple came close.
but no private companies
maybe berkshire hathaway; although you could buy shares, they are semi-private.
This is along the line of what I was thinking yesterday after seeing William’s list. It may not be worth the effort, as a back of the envelope global IRR number seems pretty strait forward to calculate, using valuations and investments at years 2000 through 2013
if we did naive math w/o dilution and used mark to market paper returns, Quora would still be around 10x. Not a grand slam, but definitely in the triple / home run range.Side note: why is baseball the go to comparison for VC returns?
Remember the difference between terms/phrases used in baseball vs. football.Was that Carlin?
Fred – a post of yours last year where you talked about a fund’s “names” loss ratio is another good take on your comment here about VC being a hits business. Good VC investing lives by the old adage that “you’ve got to speculate to accumulate” – much as you said in last year’s post “If it had a lower “names” loss ratio, I might not feel as good about it.”Paul Graham expressed a similar viewpoint in his ‘Black Swan Farming’ essay in 2012 when he stated that “If we ever got to the point where 100% of the startups we funded were able to raise money after Demo Day, it would almost certainly mean we were being too conservative.”
As a founder I’m irrationally optimistic about exiting and these lists are inspiring.Funnily though, yesterday this Reuters post argued that founding a startup is akin to buying the most expensive lottery ticket in the world:* http://blogs.reuters.com/fe…Personally, I HATE startup success being calculated by probabilities.Why? Well because no probability calculation can measure these factors that affect startup success:* PASSION of the founding team to work their socks off, overcoming challenges;* PROTECTION and PURPOSE shaping offered by investors;* PERCEPTIONS of the market and users to the startup and its product offeringsand other PESTLE dimensions
This list needs expertise and a healthy dose of luck added to it.
Exactly, Arnold :*).The wrong experts and the founder loses focus, goes on wild goose chases and burns their runway.Ha and probability can’t measure luck either — only the likelihood of defined variables and outcomes.
The best advisors don’t tell you what to do, they are your partners in decision making with experience and foresight as their value-add.Luck–you can make it. But you can never guarantee that you have enough.Or so I’ve learned now 4+ years into advising startups and mid stage companies and a career or building them.
.The best advisors find the genius within the entrepreneur and help guide it to the surface.I have never worked with anyone who did not have all the answers in them. They just need to know where to look.It does help to write them down so you can stop looking for them. They can get lost again.JLM.
Same thoughts, different words, different approaches I bet.But I agree.
So basically, coaching is like acupuncture, which believes the body has within it all it needs, but simply needs that flowing correctly to achieve optimal health and functioning.
And yet that blogger is leaving Reuters to go to an “internet venture”, or so I hear.
Ha! Thanks, I didn’t know that.After I read that blog, I reflected on how my “safe” corporate career might have shaped up if I hadn’t decided to do a startup.Of my original team of 4 at the bank, 2 are Global Heads in private equity and 1 is the CEO of the LSE’s Turquoise platform (one of the strategic investments my team originally put together).Meanwhile, I’m in the scrappy world of startups earning factorially less (so far) but with 100% ownership over my inventions and my destiny.The pain points can be excruciating; you go from having an entire infrastructure of resources to access at the bigco to having to build everything up from Base 0 (business plan, product, code, relationships etc).But, actually, I’ve developed so much as a person that that value exceeds the corporate career that could have been.
That last sentence is the key.It’s hard to not do, but don’t compare yourself to others (friends, past co-workers, etc). Compare yourself to your past self.
Thanks, Jim, wise words.My past self had been promoted into CEO-Chairman’s Office in my mid-20s so I gained insights into what my operational life in a corporate would be like.My current self has created and coded a patent-pending system from scratch.RCS MediaGroup in Italy has implemented my strategic frameworks for measuring user intelligence and tying that to content advertising.I’m about to be in NY to discuss a strategic project with a big brand consultancy.So the excruciating pains and knowing I am not a Global Head / CEO in banking is…..A-OK :*).
Hit up when you’re in NYC. Would love to meet.
Would be great. I am twainventures [at] gmail.comRegardless of whether it’s big corp or scrappy startup, the main opportunity and reward is actually about learning and applying and sharing that learning.
.You are being paid in an entirely different currency. That currency trades up every day you survive.Plus you have recognized and are feeding your own insanity. You cannot fail to be an entrepreneur when you are an entrepreneur.It is like being a scorpion. When you are a scorpion, you must follow your nature.A scorpion cannot find happiness as a ballerina regardless of how lovely it looks in a tutu.JLM.
Well can I be a black swan instead of a scorpion :*).It’s true. There is a huge difference in the nature of an entrepreneur from a corporate manager.A board member of Reuters said this at London Business School: “A manager does everything they can to reduce risks. An entrepreneur does everything they can to RUN towards risks and convert that potential into ROI.”
JLM, I spit my coffee out imagining a scorpion in a tutu.Waiting for @FakeGrimlock:disqus to draw it now.
oh, I dunno about that – Odile was a bit of a scorpion, and has one of the most famous dances in all of ballet.http://youtu.be/CxgI-PvndX4…Yay turns!
Wow! You nailed it! All I know about ballet is just some of the music, not the dances or stories. But from the music, honest, I guessed ‘Swan Lake’. Checking, yup, that was it.Astounding dancing, by the woman, e.g., very much in time with the music. Expressive. Amazing. Astounding. UNbelievable!
> Personally, I HATE startup success being calculated by probabilities.Well, yes, but there is a solution still within the math of probability.> Why? Well because no probability calculation can measure these factors that affect startup success:Presto! You essentially found the solution!Or, in probability we deal with ‘events’. With the set theory approach to probability theory, an ‘event’ is the set of all ‘experimental trials’ that might have been observed that make something true. E.g., in coin flipping, the event H may be the set of all trials where the coin came up heads.A probability P puts a number between 0 and 1, with both 0 and 1 possible, on each event. For event H, we write its probability, that is, the number, as P(H). Then for a ‘fair’ coin, P(H) = 1/2.Okay, suppose S is the event that the company has a good exit. So, you have said that the probability P(S) is not pleasant to consider. Right.But suppose A, B, and C are some more events, e.g., from criteria such as you listed. Then there is the ‘conditional probability’, right, another number between 0 and 1, with both 0 and 1 possible, of success S ‘given’ events A, B, and C which we write as P(S|A, B, C).The intuitive view here is right on target: We take the set of all trials in all of A, B, and C, that is, in set theory, the ‘intersection’ of sets A, B, and C, and the conditional probability is the fraction of that intersection ‘covered’ by event S. So, if set S is 1/3rd of the intersection of the sets of events A, B, and C, then the conditional probability is 1/3rd. The definition is,P(S|A, B, C) = P(S & A & B & C) / P(A & B & C)where here in the notation I am using & to mean AND or set intersection.Here’s the bottom line that resolves your concerns and frustration: In practice, if we are going to consider probability, we rarely care about something like P(S) and, instead, consider a conditional probability P(S|A, B, C) for some events A, B, and C, such as you listed, that we know are true. So, probability theory continues to apply, but mostly we want a conditional probability.We can argue that as long as P(S) > 0, there exist A, B, and C or some such so that P(S|A, B, C) = 1, that is, success in the business is certain — good news. So, in practice, for evaluating S, part of the effort is to look for A, B, C that hold and make the conditional probability higher. To ‘manage’ our startup, we find events A, B, C that, if true, will make P(S|A, B, C) high, hopefully 1, and then work to make those three events true.For more on conditioning, easilyP(A|B)/P(A) = P(A & B)/(P(B)P(A)) = P(B|A)/P(B)or Bayes rule.Also of interest isP(A & B) = P(A | B) P(B)That is, if somehow we know the conditional probability of A given B and also the probability of B, then, if we are interested, and sometimes we are, we can find the probability of A and B.Events A, B are ‘independent’ provided P(A & B) = P(A)P(B). When we apply probability to a real problem, one of the more important parts of ‘formulating’ the real problem as a problem in probability is looking for cases of independence. Easily, if events A and B are independent, then P(A}B) = P(A).For more on ‘conditioning’, suppose event N is that our project has a ‘ninja’ developer. Suppose that success is ‘conditionally independent’ given event A. We show that, given A, the ninja is worthless! Event A? Sure, it might be recruiting good students, having a good training program, using some ‘chief programmers’ as in F. Brooks, and having good, occasional access to some consultants.Drawing from the definition of independence, events S and N are ‘conditionally independent’ given A provided that P(S & N|A) = P(S|A) P(N|A).So, we show that P(S|A,N) = P(S|A) which does mean that the ninja is worthless!So, we argue,P(S|A,N) = (P(S,N,A) / P(A&N)) (P(A)/P(A)) = P(S&N|A)P(A)/P(A&N) = P(S|A)P(N|A)/P(N|A) = P(S|A).Done.’Conditioning’ is a major theme in probability and stochastic processes. So, for real valued random variable Y, we can generalize probability P to expectation (‘average’) E and write E[Y] — under mild conditions, the expectation exists. Then for random variable X we can write the ‘conditional expectation’ of Y given X by real random variable Z = E{Y|X] = f(X) for some real valued function f of a real variable.Intuitively, conditioning ‘gets at’ ‘information’ and is a mathematical case of ‘what do you know and when do you know it’.And we can work with conditional independence of random variables.A key result is the Radon-Nikodym theorem with says, in part, E[E[Y|X]] = E[Y]. Quickly we can show that Z = E[Y|X} = f(X) is the best nonlinear least squares approximation of Y in terms of X. Or, in ‘business intelligence’ or ‘ad targeting’, if we have X and are looking for Y, then our best answer is f; in principle ‘big data’ can help us find f.One of the nicest proofs of the Radon-Nikodym result is due to von Neumann and is in Rudin, ‘Real and Complex Analysis’.Conditioning is central in ‘sufficient statistics’ as done by Halmos, an assistant to von Neumann at the Institute for Advanced Study, and can reduce a lot of ‘big data’ to really small data with no loss of utility!Conditioning is also central in much of stochastic processes including Markov processes and martingales.
The bias in perceptions is different from the bias (or standard deviation) in probability.An investor’s personal chemistry with a founder contains biases in perception and the market’s pricing of a startup also carries implicit value based on perceptions not just on the probability matrix of the capitalisation table.Personal chemistry and perceptions are not discrete events as per flip of a coin and P(H) = 1/2 or roll of a dice and P(2) = 1/6.Compound onto this the legislation which is open to interpretation. It wouldn’t be accurate to model it, for example, as “Let Jobs Act be an event where P(J) = 1/n….”Probability (Bayesian, Markovian, Boltzmann) is indeed powerful………It does enable the compression of “Big Data” into bite sized.However, the question is “Does Probability qualify and not simply quantify?”If it cannot qualify and quantify the personal chemistry involved in startups then…..Probability is not the panacea tool.
> Probability is not the panacea tool.Right.My P(H) = 1/2 was just to give a simple, precise introduction to some of the central mathematical notation and not to suggest that probability is only for such simple, discrete situations.Most problems in life are too ‘messy’ for making much progress with applied math. A good application of probability is typically from careful problem selection and formulation and, then, some significant work.Still, there are many surprisingly good applications.Sometimes the applications are justified from some surprising theoretical results: The law of large numbers and the central limit theorem are common example, but, say, we have some ‘arrivals’ at a Web site. Each ‘unique user’ returns with a ‘stochastic point process’ of their own. Mostly the users are ‘independent’. Then there is a ‘renewal’ theorem in W. Feller’s second volume that says that what the Web site sees is essentially just a Poisson process — there the arrivals are ‘stationary’ in time, and the increments are independent and identically distributed (i.i.d.) with exponential distribution. That’s a lot of information with just some intuitive, qualitative input. All that’s left to find is the arrival rate, and just counting arrivals over, say, 10 minutes and dividing by 600 will give a relatively good estimate of the arrival rate in arrivals per second.A similar argument can work for the package load at a FedEx office. Then can calculate the probability that a plane schedule serving that office will be overloaded.For more, here is some of how once I got my wife and I a vacation at Shenandoah: The Navy wanted to know how long the US missile firing submarines would last under a special and controversial scenario of global nuclear war limited to sea. So, both Red and Blue had a wide variety of ships, planes, etc., and there was to be a continuing war but limited to sea. Curiously, B. Koopman wrote a paper arguing that the ‘encounter rate’ of one Red and one Blue had to be a Poisson process. Then with some not too rash assumptions, can argue that the whole war would go like a continuous time, discrete state space Markov (past and future conditionally independent given the present) process ‘subordinated’ to a Poisson process. I wrote the software to do the evaluations, and the Navy got their results, in the two weeks they specified, and my wife got her vacation.Obviously the scenario was awash in forbidding complexity, yet at least for a reasonable first cut the assumptions were not absurd. So, just my efforts for two weeks as a graduate student on a part time job yielded something of value. Indeed I was told that my work was later sold to another part of the US government; I could tell you which one, but then I’d have to …!Sometimes forbidding complexity of a really messy problem can boil down to a surprisingly accurate probability model. It turns out, forbidding complexity can lead to fairly accurate assumptions of independence or conditional independence, and then a probabilistic model can be surprisingly simple and good.E. Cinlar at Princeton has remarked that people can be too eager to rush to working with data and that theoretical attacks are underrated. He is correct.> The bias in perceptions is different from the bias (or standard deviation) in probability.Typically an application of probability needs some input data, and often good data is difficult to find. Of course “bias in perceptions” is one of the challenges.> An investor’s personal chemistry with a founder contains biases in perception and the market’s pricing of a startup also carries implicit value based on perceptions not just on the probability matrix of the capitalisation table.Yes, but the situation may not be as hopeless as it might appear: With the theory of probability and, in particular, of random variables, your “implicit value” essentially has to be a ‘random variable’ (the careful definition of a random variable is next to brilliant and nearly universally general and has next to nothing to do with popular views of what is ‘random’), and then have a shot at getting some estimates via a fairly long list of techniques.> Personal chemistry and percepti ons are not discrete events as per flip of a coin and P(H) = 1/2 or roll of a dice and P(2) = 1/6.Right, but this presents little or no problem. Temperature tomorrow is not discrete, either, but as I understand current, standard weather prediction probability models are standard.> Compound onto this the legislation which is open to interpretation. It wouldn’t be accurate to model it, for example, as “Let Jobs Act be an event where P(J) = 1/n….”Right. There are nearly always plenty of ways to make mistakes; you have described some.> However, the question is “Does Probability qualify and not simply quantify?”Sure. What I said about about a Poisson process from the renewal theorem is usually regarded as ‘qualitative’.For more, even just empirically, if only via simple cross tabulation, assuming quite a lot of data, we can find the probability distribution of time to first offer of marriage for a person on a computer dating site all as a function of various attributes of the person. That is, we are looking for a conditional probability. So, let t be a positive number for time, random variable T be the time until the first offer of marriage, and events A, B, and C be attributes of the person. Then we can look forP(T <= t|A, B, C)which is the cumulative, conditional probability distribution of the time T in terms of the attributes A, B, and C.Probability is a powerful tool but good facility in applications tends to need quite a lot of study of the theory and then quite a lot of practice involving both real problems and the theory.For doing applications, I gave one of my favorite lessons, one of the more important parts of ‘formulating’ the real problem as a problem in probability is looking for cases of independence.
The arrival at the website is the same problem set as a search result is the same problem set as the Fedex one or the diet problem or the queuing theory for planes on runways for take off.It’s OR optimization theory.The discreteness of temperature is not the same as the discreteness of a perception.Temp (t=0) = 60Temp (t=1) = 70 = +10 on (t=0)Temp (t=2) = 50 = -10 on (t=0) and -20 on (t=1)Perception (t=0) = “this founding team looks interesting.”Perception (t=1) = “their product needs work. They don’t have investors yet. The market is starting to get there, though……”Perception (t=2) = “this start up is phenomenal, going to kill it.”Therein is the current limitation of maths itself, Central Limit Thoerem et al :*) .And the reason natural language parsers in AI still cannot permutate the probabilities of figuring out what we mean in our online communications.For metaphysical and inorganic phenomena (weather elements, dice, coins etc.) their quantifiability lends itself to existing mathematical calculations and tools.For human phenomena (language, perceptions, social dynamics etc) our friend Mathematica will need to provide other tools.
Of course it’s all a classification problem.Since the days of John Graunt circa 1660s we’ve put quant data into tables to tally, differentiate and pattern-recognise events so William’s efforts are in that tradition. And probability plays its part on those defined data inputs.However, there are missing classifiers as well as how the problem set is framed if we’re talking about the qualities that make for startup success and not just the quantities (levels of investments, number of people in founding team, number of users, how many investors, levels of exit etc.)Probability provides an approximation rather than accuracy per se.For accuracy there’s need to be different classifiers.I say that from a previous life looking at tables and tables of investment transactions from PwC Moneytree, VentureOne, CapitalIQ, etc. similar in scope to William, Aileen Lee and Fred’s great work.Still, after I had run through my nice probabilistic models I wanted to know, “WHY is this investment opportunity better on a qualitative basis — not just because of the numbers?”
I gave only a short answer to your point about”the discreteness of a perception”.So, you gave an example of three possible ‘percemptions’ from VCs about a startup. Okay, certainly such things are from common to nearly universal.Then one connection with probability is to say that there is some set of discrete responses and the response of the next VC is a random variable with a distribution over the set of discrete responses.Maybe we would like to predict the next response. If the next response is independent of all other data we have, that is, probabilistically independent of the values of all other random variables we have, then our best estimate of the next response is just directly from the distribution we have. But if some of our data includes values of some random variables not independent of the next response, then we have a shot at a better prediction. If we want to predict random variable Y (that has an expectation) and have random variable X, the we know the form of the best least squares prediction, E[Y|X] = f(X), and there it is sufficient to approximate f. Easy? Not always. Always hopeless? No.Since perceptions from people commonly do differ, maybe if we knew more about some of the people involved, then we could make a useful prediction.Maybe we could rank the responses on a scale of how close the entrepreneurs were to a term sheet or, maybe, just the probability of a term sheet. Then a crude, first=cut model might be to build a regression model with independent variables MBA or not, founded a VC backed company with an exit that made the VC money or not, got a bachelor’s degree from Harvard, Princeton, Yale, Williams, Stanford, Cal Tech, or Berkeley (seven 0-1 variables here), personally worth over $100 million or not (or just their personal worth), etc.For such a problem, some people want to use ‘logistic regression’ because the dependent variable, that is, what are trying to predict, is between 0 and 1 so that don’t want predications outside of that range so use a real valued function of a real variable that is continuous, differentiable, strictly monotone increasing, and asymptotically 0 on the left and asymptotically 1 on the right or some such thing.Commonly predicting the behavior of people is challenging, but at times probability can help. Ad targeting is one important area.Of course since I can’t resist teasing the girls on AVC, there is the line in the movie ‘The Big Sky Country’ “Never can tell what a woman will do next.” — or long ago it was common knowledge that predicting the behavior of people, especially women, could be challenging, so challenging, no joke, that they dominate the list of my worst failures!A closely related problem is, say, to take a radar image and say if a Russian tank was there. So, there are two ways to be wrong, false positives (saying that a tank is there when it is not) and false negatives (saying that a tank is not there when it is).A common way to proceed is an ‘hypothesis test’, that is, assume that a tank is not there (the ‘null’ hypothesis), use what we know about the probability distributions of relevant observable random variables when no tank is there, and see where the variables we observed are in those probability distributions. Can do much the same for, say, monitoring the ‘health and wellness’ of a system in a server farm or network (the null hypothesis is that the system is healthy).In an important sense, the best way to do such an ‘hypothesis test’ is the classic Neyman-Pearson result, that’s K. Pearson, in statistics about 100 years ago and J. Neyman, long at Berkeley about 60 years ago. The Neyman-Pearson result is commonly presented in intermediate level courses in statistics. Fred and JLM would quickly understand the basic idea: Take the false alarm rate willing to tolerate, regard it as money, and invest it in the plots of area with the best ratio of detection rate to false alarm rate until are out of money. Then, if get data in one of the plots with an investment, conclude a Russian tank or a sick system in the server farm or network.As I recall, Raytheon made use of Neyman-Pearson in some work in military radar target detection (the US DoD can quickly understand Neyman-Pearson for detecting Russian tanks; don’t expect the VCs in Silicon Valley to for monitoring server farms!).At one time U. Grenander, long at Brown’s Division of Applied Mathematics, and a world class researcher in probability, worked on some ideas of just astounding originality and with possibly applications to various cases of object detection.> The arrival at the website is the same problem set as a search result is the same problem set as the FedEx one or the diet problem or the queuing theory for planes on runways for take off.Well, maybe these problems can all be considered operations research (OR), but with the usual criteria they will not all be considered optimization problems!But not all the boundaries are clear: E.g., Neyman-Pearson can be viewed as an optimization problem. Indeed, a year or so ago I worked out a proof in terms of nonlinear duality theory and the Hahn decomposition from, right, the Radon-Nikodym result. Indeed, looked at in a little more detail, Neyman-Pearson leads to a knapsack problem which is an optimization problem in the notorious class NP-complete.But, the main work in optimization is elsewhere. E.g., for FedEx, the main work might be in fleet scheduling; there a small part of the work might be to use the Poisson process to estimate the probability of a schedule having a plane overfull at, say, South Bend, IN. This problem, of course, is both large and in NP-complete so challenging.The main idea in optimization is ‘necessary conditions’ for having the best possible solution, and there the main idea is: Suppose are in a winding cave with an undulating floor and are looking for the lowest point in the cave. So, if put down a marble and it starts to role, then if walls of the cave reasonably well behaved must not yet be at the lowest point. So, a necessary condition for being at the lowest point in the cave is that the marble not roll. Grow this intuitive idea into mathematics and get at least the Pontryagin maximum principle (deterministic optimal control, P. Falb, H. Kushner, M. Athans, e.g., how to climb, cruise, and descend an airplane; e.g., the least fuel trajectory to an outer planet) and the Kuhn-Tucker conditions (the workhorse of constrained nonlinear optimization) and for the second quite a bit in theoretical economics and finance and a famous paper in economics by Arrow, Hurwicz, and Uzawa.It’s a bit much to look for the “limitations of math itself”. The limitations you mention are really of available input data, not the math.> And the reason natural language parsers in AI still cannot permutate the probabilities of figuring out what we mean in our online communications.More directly the “reason” is that so far we don’t have even as much as a weak little hollow hint of a tiny clue about how intelligence, or even language understanding, works. Some of the problems attacked by humans with intelligence and/or natural language understanding can also be attacked with some math; maybe some such attacks can be quite good, maybe in some narrow problems better than what humans can do, but there is little hope that the math is how human intelligence works.> For human phenomena (language, perceptions, social dynamics etc) our friend Mathematica will need to provide other tools.No. Math remains one of the best tools, and with good work often the best tool by a wide margin, including for such problems.Maybe a better point would be that physics and the ‘hard’ sciences remain especially powerful for their “physical” and “inorganic phenomena” because so far ‘social science’ is nowhere nearly as powerful for its problems as physics is for its.Still often math can be successful for “human phenomena” even where social science is not; that is, math is sometimes able to get results without actually creating new results in social science. E.g., even if “Never can tell what a woman will do next”, the next dress she wears is still a random variable and can be approximated by other random variables, possibly fairly accurately, with values of other random variables we do have. So, we get no great progress in social science, but still we get a good prediction. Thank you A. Kolmogorov and J. von Neumann!
Ha, our exchanges are such fun!You didn’t factor in the presence of wind or other force on the marble that would give it momentum to roll. The marble, for all we know, could be static in a saddle point in the winding undulation. After all there are wallows in undulations!More seriously, Mathematica will need to provide new tools because now in the cutting edge of Quantum Theory there is a preponderance about “Perceptronium, the most general substance that can feel subjectively self-aware.”Ha but neither Maths nor Physics can solve the probability paradox nor the integration problem for this.There are some nice Hamiltonian operators involved too.My reading of the problem set is we’re arriving at a time point where a mathematical tool needs to emerge that coheres both metaphysical and human phenomena.It needs to be beyond Gaussian, Poisson and Nashian. It’s also beyond du Boisian, Webberian and Granovetterian.
> You didn’t factor in the presence of wind or other force on the marble that would give it momentum to roll. The marble, for all we know, could be static in a saddle point in the winding undulation. After all there are wallows in undulations!We would assume that the cave had no wind.The cave and marble were just an intuitive view of constrained nonlinear optimization. There is a chemical engineering professor at Princeton that much of Houston is interested in, and he may be interested in such optimization.In the actual mathematics of the Kuhn-Tucker conditions (KTC), we consider gradients of the objective function (the floor) and the constraints (the walls). The part of the KTC I considered were the necessary conditions. Then a saddle point in the objective function does not provide a contradiction.With the saddle point you seem to have been considering KTC sufficiency, and that is a bit different with usually a convexity assumption. Quite generally in optimization, sufficiency is a difficult subject because it is asking for ‘global optimality’ instead of just local optimality.A tricky part of KTC is the constraint qualifications (CQ); the constraints have to be relatively well behaved, that is, not pathological, or the results don’t hold. For the KTC theory to hold at a point need at least one CQ to hold at that point; there are many CQ. One of my papers solved an old problem in CQ. In practice CQs are nearly always satisfied.The KTC are yet another case of Lagrange multipliers and, thus, can provide some insight into that important, large, and old topic.It may be that Nash was at Princeton when H. Kuhn and A. Tucker were.T. Parthasarathy and T. E. S. Raghavan, ‘Some Topics in Two-Person Games’, has a proof by Lemke of Nash’s main result. The proof is in terms of linear programming (often a surprisingly powerful tool in proving theorems), and a good start on linear programming theory is immediate from the KTC.The Chair of my dissertation orals committee, as I recall, had been a Tucker student.The KTC are an old subject but a bit tricky in the details and, still, commonly misunderstood. Can find good information in texts by Zangwill, Mangasarian (long in the oil industry), and likely also Bertsekas. I have some good notes and maybe someday will type them into Knuth’s TeX.We started with just probability. Since that subject seems to be taken more seriously in France, Russia, and Japan than in the US, I thought I’d help people in the US understand that subject a little better.Then we got off into optimization which is quite different. If we get off into quantum mechanics, then we will never get done, and fixing the bug in my software will get delayed and having my project go live will get delayed.The bug has to do with class instance de/serialization, which can be super nice when it works! I know everything is okay early in the code, and I know where some deserialization fails, but there is quite a lot of code between those two. So I am putting in statements to write to log files, for the Web page and for the server, to trace the logic and confirm that everything is okay. The problem should be minor because nearly all the code where the bug is is well tested in my session state server and just copied from there. Bugs; they happen. So, we chase them down and fix them.
I think you could add HauteLook: http://techcrunch.com/2011/…Also, what about companies valued > 250MM that subsequently lost value before a full acquisition? Ideeli is one that comes to mind – they were likely valued around 250MM at one point but Groupon acquired them this year for ~40MM.
As I said, I prefer to keep it at the “ending” valuation, where the end is today, or an exit event. I will check/add HauteLook by eod. Got to go and get some work done 🙂 Thanks.
@wmoug:disqus Can you integrate with CrunchBase?
Hmm. How so? Crunchbase isn’t 100% accurate everywhere. That’s why supersets of it like Mattermark have a higher accuracy.The only place that attempted to include Valuations that I saw was the VentureBeat database, but it was full of holes and unusable.Tracking private valuations is not easy. It takes continuous research/updating, and some estimations, and you’ll be still wrong in 25% of the time.
Yeah, not saying to connect for valuation data, but for all the other data. So I can browse your list/hackpad and from their link off to CrunchBase to find more info on the specific company (founders, news, etc)
I just need one hit, man. Just one hit.
Too bad need has nothing to do with success 😉
I just want one hit man. Just one hit.
yep 🙂
.Hit or hit man?Different things.JLM.
missing a comma, mea culpa
.No apologies. Freud might have a problem with that.Needing a hit man is OK.Get what you really need, man.JLM.
coughed laughing – almost woke Michelle @1am reading (up early not late)
Or work effort sometimes. It’s proper execution that wins, not just playing the game, even if you’re playing to the best of your abilities. Sometimes, one’s best still isn’t good enough and they either need to practice more or pivot.
Or luck. Or changes in the market. Or team dynamics.Or….
Agreed Arnold. My point is that I get so tired of the whine “but I worked so _hard_ !!!”. If working hard were all something took, I’d be a pro basketball player (just not for the Knicks), have 20 Superbowl rings, cured cancer, etc. Some things take talent, not just work. Even Malcolm Gladwell agrees.On a side note, I’m a firm believer that people make their own luck. Luck favors the prepared.
Whining is boring and unattractive no question.
you sound like the Cubs.
They “play” baseball, right?
Was talking to an angel from Ohio. He said VC funding for almost every startup in Ohio doesn’t make sense. There have been 0 exits over $40M. There have been 0 IPOs. There has been little or no activity with local big corps buying startups. Interesting.In Chicago, we are starting to see that change. I have a gut feel on who could be there in the next two to three years, but they aren’t there yet.
Chicago’s tech scene is really starting to hum. I’m loving it.
Is it possible to tell over this time frame how valuations have evolved? Is the list weighted more to current companies? @wmoug:disqus
Not really. It’s supposed to go as far as 2000 for year founded.
Here are a few more: Rally software, Heroku, Solar City, Kiva systems, Admob, Foundation Medicine, Nanosolar, Stubhub, Hotel Tonight, Instacart (Rumored), Wildfire Interactive, Nextdoor, Spacex, Quirky, Pacific Biosciences, Practice Fusion, Brightcove. Fleetmatics
Great. I’ll check them & add them. Thx!
Wotif. 2000 / ~$550MKogan 2006 / ~$315M
Thanks Cam. I added Wotif, but when I checked Kogan, they looked mostly like a re-seller, so they don’t qualify as an info-tech related company.
Sure thing. Thanks for sharing your work. BTW, I got (most of) my data from AngelList.
Great work by William. I love those kind of studies and it also shows the value that companies such as Mattermark, CB Insights, Datafox are doing to this space.
How real to we consider private valuations?(No offence) – Their data exists in a very non-liquid markets.I’m also really curious what SecondMarket data would look like
For the $250M-$1B range, I’d consider the valuation to be very real. Businesses in that size range are typically raising money from new investors after being evaluated by multiple sophisticated investors, have meaningful revenue and very high growth rates, plus most have some level of in-bound acquisition interest and comparable company exits from M&A or IPOs. Once you accept that valuation is an art not a science, so perhaps a company carried at $400M is worth $300M or perhaps $500M, it all seems about right.Above $1B, similar factors all apply. Due to the amount of late stage money sloshing around resulting in very high valuations, I have a question on how those investors can generate their targeted returns and private company investments at super-high, $10B valuations seem like high risk endeavors to generate their targeted returns–perhaps higher risk than the investors are accounting for.It doesn’t mean the valuation is bad, it just means getting 3X off the high valuation is hard. Some of the investors in that range have a habit of over-paying and being late to the market so it will be interesting to see how it turns out.
Yeah, the high end is funny. It feels to me there are several solid $1B-$2B companies out there with $5B-$10B private market valuations.
I always am reminds of eugene fama in this regard – what makes these investors more qualified than very active market in which way way way more players are involved
At sub-$1B, maybe even up to $2B, I think the valuations are real and generally discounted appropriately to reflect the risk of the investment versus the likelihood of a big return at least based on what I know of many of those companies. Broadly speaking, it is a fairly well understood market from a risks and rewards standpoint when you consider any of the regular growth equity type investors–Summit, Insight, Meritech, etc.–with a good investment return historyAbove $1B, I noted my own reticence. You seem to have much more competition for those deals in what is a relatively new segment. Certainly when you get above $5B you have very few historical private VC financing data points. My take is you seem to have something that is going to either end very well or very badly. Easy argument to say you have inflated valuations, a bunch of new investors to that sort of valuation and space (TPG in airbnb), a bunch of investors who saw this story before in the last boom and in many cases busted (T Rowe and various mutual fund players), etc. but it is also possible you see huge returns.
It’s typically based on the last round that was raised.
Is there information elsewhere where we could find out what last rounds look like – there are plenty of companies that never hit mattermark or crunchbase
I’d also argue the SecondMarket data is more imperfect than using last round raised. Limited information available in the market, typically companies with very limited float, funky rights/shares, etc. It is like buying on the small cap public exchanges/pink sheets unless driven by an investor with significant knowledge of the company (e.g., Sacca with Twitter).
@JLM:disqus – how accurate is the pink sheet market (I know you’ve been involved in a pink sheet company)
Here’s one for your list William – NaturalMotion bought by Zynga for $527m 30/01/2014
Thanks. Missed that one. Will add it.
CTRL + F “thinkorswim” no results whoops
Yup, Think or Swim was acquired by TDAmeritrade for $750M.
was founded in 1999, so it doesn’t meet the cutoff, but it’s a good one.
Great list by William. I built this to track the most valuable startups (pre-exit) – http://www.thestartup100.com. We’re currently tracking 49 startups with $1B+ valuations, and I know we’re missing some. Anybody can sign up to add companies or make edits – it’s meant to be collaborative.
Thanks. I will check it out.
I’m most curious about what seed or A rounds looked like on average in terms of valuations and money raised. Any idea?
I think this post highlights how hard the media has it when trying to get a story right.This is a fairly technical story ( from business degree holder at a second or third tier school)The three people involved in the coverage are really competent people.Aileen Lee is a partner at Kleiner Perkins educated at HBS and MITWilliam Mougayar has been working with world class CIO for 30 years and has advanced degrees from Canada’s best universities.Fred Wilson has been doing VC for 20 years and educated at MIT and Wharton.The story is still being refined.If your just the average journalist what chance do you have of getting something like this right? What are the consequences of informing the few in the public incorrectly who care about weighty issues?
Good observation Bill. This is “analyst” type work. It takes time to do it, whereas a journalist needs to move on to the next story 45 seconds after they finish the previous one.- ps: actually my B.Sc. is from the University of Washington in Seattle 🙂
Fred, Rory O’Driscoll at ScaleVP put together 145 billion dollar+ vc exits; download here http://www.scalevp.com/from…
Good list. Thanks.
Very interesting topic. I wonder in pre IPO valuations do they consider revenue or earnings. A lot of recent high profile companies basically have zero earnings and not much revenue relative to the attached valuation.There is a lack of any consideration for earnings at all. Looks like profits are not important to many buyers and analysts as well.I do not know if this an indicator of a bubble but something is wrong. An enterprise with no profits and the stock price keeps rising.Please chime in with your wisdomEarnings important factor in valuations or a relic from the past. Earnings going the way of then donosaurs.
40-50 per year, so what’s the funnel look like to get to that number? How many is USV in — 2-3 per year? Thus if you were to somehow scale your firm and invest in 20x as many startups, you’d be more? Kind of the approach A16Z is taking?If there’s 40-50 per year now, what’s that number in 10 years? 20 years?
I’d love to see numbers showing the time required to reach $250mm from founding. My bet is that valuations of that size are being achieved significantly faster today than they were 10 years ago.
You’re prob right. That would require some work to find out.
what a serious bunch of bs. it’s an amusing by-product of the vc echo chamber (or too much time at b-school) that someone actually thinks they can count/quantify the number of hits, misses or otherwise in the start-up economy. despite lots of notice and notoriety around equity capital, an equally if not more important engine – the real dynamic – is actually those organizations that don’t raise capital and bootstrap, achieve real results and value.
TripAdvisorQuidsi (Diapers.com)BillMeLaterCouponsNgmocoTinyprintsHuffington PostDocusignGlamLiving SocialMantaNasty Gal?Path?RedditStella & Dot360buyIetvCloudarytrivagoGmarketBuzzfeedRightMediaPlaydomAdMobClub PenguinApartments.comMeeticLovefilmRevolution MoneyAdifyFotoliaBlueLithiumLast.fmPhotobucketTacodaGittiGidiyor
This is a amazing list of add-ons. I’m going through them now, one by one, and adding if they were founded 2000+ and over $250mil in valuation or acquisition price.Thank you very much. Curious how did you spurt out such a great list?
+blue apron
We took a look at it. Actually did have an offer but not one worth taking. Now that I don’t need it, we get an inbound inquiry a week. Gotta love it.I came out of that process with a list of three VCs I’d ever want to take money from. Fred is one and I’ll leave the other two as guesses. I can only wish we were building an engaged network but we just don’t fit USV’s thesis. :)We raised all of our outside capital from angels. Nobody famous but all great investors. I’m the luckiest fully-funded CEO in the world.
“Now that I don’t need it, we get an inbound inquiry a week”Ain’t it always that way?
I’m sure your inbound interest will continue to grow. If you scale a bit more, the growth equity folks will salivate. They love companies built to real scale with limited capital. It is a nice option if you ever need big capital to grow or need liquidity for some of your early investors but don’t want to sell the company. Many of the buyout firms are looking at similar deals.
> Now that I don’t need it, we get an inbound inquiry a week.Maybe those VCs are hoping, fishing, trolling for you to have a nasty divorce, a drinking problem, a heart attack, a search for a trophy wife, a mid-life crisis, a sudden desire to compete for the America’s Cup with all the attached, stuffed, wild bikinis, to sail solo across the Pacific, etc.But the VCs should be concerned that if you do take their cash, the reason was that you saw a really serious problem on the way that the VCs can’t see yet!It looks like such VCs are evaluating an investment based just on really simplistic criteria from accounting, etc.You, however, no doubt, had to have some good reasons to invest a big chunk of your life in the company, that is, a good reason to ‘invest’, on other criteria and long before the VCs were trying to contact you.Then you might wonder: The business needs to continue to grow in various respects from various new projects, and just how to do that will be, initially, as obscure as the business itself was early on. So, if the VCs couldn’t evaluate the business early on, how will they, as Board members, be able to evaluate the new projects and, then, approve the budgets, hiring, interim success criteria, ‘pivots’, milestones, due dates, etc.?While this may all seem like high level, really complicated top management and high finance, somehow I keep getting reminded of the story of the Little Red Hen.
.I hope you packed a BIG bag because today they are offering free visas and you will likely not be able to return.On Earth as it is in Texas!JLM.
+1
.Texas doesn’t want to secede. Who the Hell would we compare ourselves to?The Legislature meets every other year for 140 days. Then they’d really have to work.A million .5 Californians can’t be wrong plus today we got you.JLM.
Good point!
I think they’re responding simply to our traction. An inbound inquiry definitely doesn’t equal an investment. And many times they just want to get intel for another investment they’ve made in an adjacent space.But yes, you never know what they’re thinking. 😉