Trust No one

adewhurst linked to Paul Graham's recent article about business and Open Source. Adrian's question essentially is: so is this a great article, or does it just sound really convincing?

The answer is, from just the context of the article, you don't know. The problem is that the author suffers badly from what's called the selection bias: wanting to believe something, and then examining only the evidence that supports your belief.

I learned about the term "selection bias" by reading Harvard Business Review, a really excellent magazine, even if you think (like I do) that most businesspeople are stupid. Ironically, I can use HBR as a counterexample to Graham's point about "blog knowledge" being more interesting than stuff published by traditional companies. Meaning no offense to Paul Graham, nearly every article in every issue of HBR is better researched and more instructive than any of Graham's essays that I've read. Why? I don't know, but if you follow my link, you're going to have to pay to read those articles, and Mr. Graham gives away his knowledge for free.

Has anyone ever told you about selection bias before? If not, then like me, you're going to make lots of stupid decisions. Browsing the Internet didn't get me that information. Paying HBR some money did.

So, what specifically in this article suffers from selection bias? Nearly every statistic and every fact in the entire article. Not to say that he's wrong... but the proofs are all invalid.

For example: Open Source produces better software, eg. Linux and Firefox. The evidence? People - maybe more than 50%, depending who you ask - are "switching to them" from their proprietary counterparts. That's pretty convincing, right? Not really. First of all, they haven't switched yet. The proprietary software still has the vast majority of the market. Secondly, Linux is obviously inferior to Windows for certain types of uses, ie. anything a small business might want. (There's no way a small business could figure out how to set up a Linux system to do, say, what Microsoft Exchange does out of the box.) And Firefox? Okay, maybe it's better, although I find it horrendously slow, and by the way, Microsoft hasn't released a new version of IE in something like three or four years. (Longer, if you consider the fact that nobody could really tell the difference between IE 4 and IE 6 just by looking.) Firefox has had a lot of time to work on becoming just barely comparable to IE. But, even though Microsoft just politely let them catch up for a few years (something I claim was their actual, smart business strategy!), Firefox is just now becoming barely better than IE. But yes, some people are switching to Firefox. Congratulations.

You can prove anything you want with statistics. Apache is the best web server, yes. Linux is the best OS platform, maybe. Firefox is the best web browser at the moment, quite possibly. But three examples doesn't prove anything; what's the best graphics package, word processor, music library, media player, game platform, game, or accounting package? The answer to almost all those questions isn't in Open Source. Now, by skipping some examples I didn't like, I just used selection bias on purpose to prove my point. Did you notice? And do you see why, after this whole rant, we still don't know if Paul Graham is right or not?

If so, at least you've learned something. Hope it works out for you.