Death by statistics

The ever-insightful Pmarca writes about Age and the entrepreneur, mostly quoting from historical research by Dean Simonton at UC Davis.

This topic is very close to my heart, as I'm currently at the exact age where genius-programmer productivity is rumoured to fall off, and I'm recently left my first startup and joined/co-founded/spunoff/?? a second one. What does that mean for the success of my new company?

Without drawing any conclusions about that for now - maybe some other time - I'd just like to point out that some of Pmarca's interpretation of the data is slightly wrong. He seems to make the mistake of confusing statistical averages with individual cases. For example,

Simonton: Indeed, because an earlier productive optimum means that a writer can die younger without loss to his or her ultimate reputation, poets exhibit a life expectancy, across the globe and through history, about a half dozen years less than prose writers do. Pmarca: You know what that means -- if you're going to argue that younger entrepreneurs have a leg up, then you also have to argue that they will have shorter lifespans. Fun with math!

This may just be a joke, but it warrants clarification anyway: Simonton has just pointed out a mathematical fact, not a statistic. Imagine a particular creative genius dies at age 40. If he was a poet, he was already past his productive peak, so he'll be famous anyway; if he was a novelist, he hadn't hit his peak yet, so he probably won't be famous. If you take a bunch of people like this and let them all die at random ages, but we know that novelists tend to peak later in life than poets, then your sample set of famous novelists will necessarily have more long-lived people than your sample set of poets. It's just a statistical bias. No, peaking earlier doesn't make you die sooner. Of course not.

Here's a more important but less obvious example:

Simonton: Another way of saying the same thing is to note that the "quality ratio," or the proportion of major products to total output per age unit, tends to fluctuate randomly over the course of any career. The quality ratio neither increases nor decreases with age... Pmarca: Quality of output does not vary by age... which means, of course, that attempting to improve your batting average of hits versus misses is a waste of time as you progress through a creative career.

Youch! Once again, the statistics of the average group don't define the behaviour of an individual; that's backwards! Think of it this way: even if each person's "quality ratio" is completely random, what are the chances that nobody's quality ratio trends upwards throughout their life? Almost zero. What are the chances that nobody's quality ratio trends upwards dramatically as their life goes on? Very low. It's just another mathematical fact - if there's any particular pattern that's especially unlikely, then your data isn't really random. (And data that's not really random has a cause, which means you can improve it.)

What Simonton is saying is that there's no overall trend; statistically speaking, you're just as likely to get worse as to get better... and some of the world's best creative geniuses have gotten worse over time, not better.

But that assumes your improvement is completely random - that is, out of your control, and based only on things you can't understand. Maybe some people are "natural" geniuses, and everything they try to do to make themselves better is misguided and just pollutes their natural genius. But maybe other people are naturally stupid, and through lifelong education and practice they manage to weed out the useless parts and fine-tune their skills. Isn't that more likely than that their improvement was "random" and someone just lucked out and got better over time, while another genius was just "unlucky" and got worse?

Let's put it this way: the average genius doesn't improve with age. Do you want to be just an average genius?