Logic can only help you achieve your goals, not set them

Today, a quickie on the respective values of logic vs. emotion (intuition).

At one point, a person I know turned down an offer to work for my company in favour of an offer at Google. He had worked both with me and for Google at different points in the past, so he knew a lot about both choices. Our job offer was competitive with Google's in terms of salary and benefits, and as a bonus, he wouldn't have to relocate to a foreign country as with Google. Still, after a lot of deliberation, he went with Google, and I encouraged him to. Why?

Because he felt like it.

You can't argue with logic like that. You shouldn't try. The reason is simple: because doing what you feel like makes you happy. Doing what you don't feel like doesn't make you happy. And that's just the end of it. No amount of thinking it through will change those basic facts.

Unfortunately, a lot of people don't know what they feel like doing; they don't know what they want out of life, and they certainly don't know why. So they try to figure it out. That's where consumerism, among other things, comes from (weird article about escaping from consumerism). "I don't know which job will make me happier, so I'll take the one with the higher salary." "Maybe if I had a bigger house or a nicer car or whiter teeth, then I could be happy." If those things will actually make you happier (and if you're honest with yourself, you already know if they will), then that's it; you win. But if you don't know anything that will make you happier, and you're trying to choose on an "objective" basis, then you'll lose for sure.

This problem - overthinking your desires - is surprisingly widespread, not just among programmers, but for almost everyone. Programmers happen to be especially stubborn at rationalizing, so they sound more convincing than most, but it's all the same game.

So what is logic good for, then? It's good for achieving your desires. If you're honestly having trouble deciding which of two job offers makes you feel better, maybe you honestly don't like either of them. Maybe, regardless of how objectively great they both sound, neither is subjectively good at all.

When that happens - and this is a last resort for only that case - then you can use logic. First, imagine what you want your life to be like. It can be anything you want. Seriously. Imagine it in excruciating detail. Then figure out, in just as much excruciating detail, all the steps you'll have to take to get there. If you do this exercise properly, all the way back to the present, then you'll know what to do next. You'll have figured it out using logic, but that's okay, because your logic was guided by emotion, not the other way around.

--

For more on logic vs. intuition vs. several other human attributes you can combine for use in decision making, you might want to read the book On Equilibrium by John Ralston Saul. It has lots of big words, but it's enlightening.

Also beware that if you believe what I've said above, you probably now believe in fluffy emotional stuff called the Law of Attraction. Hope you don't mind. If you're like me, you'll find the traditional explanations of it (eg. follow the link) unbelievable and silly, but what I've said here is just a sanitized translation, honest.

Architecture Astronauthood

Joel Spolsky has a recent article called Architecture Astronauts take over. The only comments I can seem to find on his article are negative ones, which is too bad.

Now, I'm a bit of an architecture astronaut myself sometimes. Even so, I believe Joel's totally right.

...but he's missed (maybe deliberately for effect) a couple of important points.

One is that "architecture problems" really can be important to customers in the end. Not directly, but because they can make it easier to solve a certain class of problems that customers do have. Synchronization is, I believe, one of those problem classes. I've spent a lot of time thinking about and working on (real customer) problems that come down to the synchronization problem, and wishing there were a magic "synchronize this" button I could just click and have things always work out. There still isn't, and maybe there never will be, but I still hope there will. (rsync is the closest thing we have, and it's a huge architectural step forward - but there's more I need to synchronize than just files, and librsync seems to be largely ignored for some reason.)

The second point he misses is that giant non-user-focussed architectural projects are not always a waste of time. "The only difference between genius and insanity is success."⁽¹⁾ Look at MacOS X, for example; Apple threw out pretty much everything from the original MacOS, and spent years rewriting the entire thing from scratch. It was a total loser for years and years of development, and the pain continued even for years after its initial release, because it was slow and buggy and incompatible. And yet now it's powering some of the world's most popular (or at least coveted) laptops and cell phones.

It was a huge risk. For every MacOS X, there are dozens of examples of failed, mind-bogglingly expensive and time-consuming, attempts to build a new desktop operating system. Even Apple hasn't won more than a tiny fraction of the market. Thus, Joel is exactly right; anybody who would try this basically has no concept of risk vs. return.

And yet, thank goodness for those people.⁽²⁾

Footnotes

⁽¹⁾ Can anybody find a good source for that quote?

⁽²⁾ Not that Microsoft's latest attempt seems any more likely to succeed than the previous ones. I said rsync was a huge step forward, but it's really just a tiny little tool based on a brilliant concept. The "synchronization problem" needs more of those, not another giant multibillion dollar "synchronization platform." An operating system needs to support everything I need to do, while synchronization is just one of the difficult things I need to do sometimes.

Oil usage trends

Every now and then, you see a trendline that actually tells you something cool.

Check out the average number of vehicle-miles travelled in the U.S. since 1983... and how it's finally stopped increasing. Via Paul Kedrosky.

Great moments in probability

Years ago (around 1999) when dcoombs and I were debugging the first versions of our "weaver" Linux-based server appliances from our apartment in Waterloo, we used to test on the cheapest hardware we could obtain for cheap.

One of these boxes absolutely refused to boot weaver, but the symptoms were strange. We had three ways of booting: boot from a CD, install an image on the hard drive and boot that, or load Etherboot from a floppy and use that to network-boot the kernel over tftp.

The symptoms were as follows:

1. Booting from CD worked fine.
2. Installing from CD to the hard drive and booting that worked fine.
3. Booting a weaver image from the hard drive (with a kernel downloaded via ftp) always gave kernel decompression error.
4. The etherboot TFTP process would always abort with a timeout after a few packets. (Etherboot of the era would do that occasionally even on a good day, but here it happened every time.)

The obvious conclusion here was that our weaver kernel image was broken, because you could boot the Debian kernel from either CD or hard disk without a problem. Right?

Well... as it turned out, no. The actual problem was a horribly broken network card that would randomly corrupt bits. About 9 out of 10 packets would be corrupted. You'd think that would be obvious, right?

Well, no. In fact, TCP/IP is specially designed to deal with the occasional corrupted packet. TCP and UDP have a 16-bit checksum on every packet, and if it doesn't match, the receiver simply throws the packet away; the sender is supposed to resend (and it does!).

I had noticed the FTP transfers were surprisingly slow, but not that slow, and back in those days, you could never quite remember if your network card was 10 MBit or 100 MBit. This happened to be a 100 MBit card, but 9/10 packets were getting thrown away, so we got around 10 MBit performance from ftp.

But here's what killed us: a 16-bit checksum can only detect 65535 out of 65536 possible errors. A 9/10 error rate means you're sending 10x as much data as you think you are, so a 12MB kernel+rootdisk package is actually about 120MB of packets; that is, about 80000 packets at 1500 bytes each. Thus, virtually every transfer was destined to have a tiny number of incorrect bytes! Ha!

Of course TFTP is extra dumb and doesn't deal well at all with packet loss, so it would just time out. But I remain very impressed at how well TCP managed to paper over a 90% broken network. That's the power of the Internet for you, right there.

(Thanks to jwz for having a hopefully-unrelated problem that reminded me of this.)

Engineers without Borders

I went to an Engineers Without Borders conference in Montreal a while ago, and I was very impressed; these people seem to have their goals straight. And little did I know that it was founded in 2000 by University of Waterloo students who were in school at the same time I was. Considering it now has 25000+ members, that's incredibly fast growth.

Anyway, as a sample of the sort of things they do, here's a report of a field study of how a proposed drought-resistant crop worked out in a heavy rainfall season. Summary: not so well. But that's not the point; the point is that they actually went to check, and then reported the results honestly, just like you'd expect engineers to do. I really respect that.

A note on market research

I left one company after several years of faithful service. The CEO himself was elected to get me a going-away present on behalf of everyone. He got me... cuff links. (I include the wikipedia link to help out those of you who know as much about cuff links as I do.)

By comparison, where I work now, an office manager was elected to buy me a wedding present. She didn't know what I would want, so she went through my emergency contact information (the privileges of being an office manager!), called my parents, and asked them what I would want. My parents didn't know, so they sneakily called me under false pretenses, got the information out of me, and reported back. What I got, among other things, was a shower squeegee. (I include the patent office link so you can truly understand what an amazing gift that is.)

So what's the moral of this story?

Well, sometimes your office manager knows a lot more about pleasing customers than your CEO does.

jwz's collected bicycle wisdom

jwz has a remarkably insightful article about bike ownership. Notably, item #6 makes me feel better about getting my own bike stolen last summer:

Get a u-lock. Lock through the frame and the back wheel. Your bike will be stolen, so don't get too attached to it. This also means, don't waste your money on junk like baskets and lights. Just get a backpack. It doesn't matter how crappy your bike looks: any bike is worth stealing for $2 worth of crack. Your bike is temporary. Accept this and move on.

On projects that don't die

I find it interesting to track which of the things I create that continue to exist and flourish when I get bored with them, and which just drop dead when I stop working on them, and everything in between.

For example, I've posted several times before about random people still do with my obsolete-before-I-even-wrote-it Linux Arcnet driver.

Another one that I find interesting is the Debian Popularity Contest, which I created in... hmm, late 1998, almost ten years ago now. I intended it as a simple way to demonstrate that Debian really didn't need to distribute a 5-CD set; the packages most people wanted would happily fit onto one CD. Also, I was hoping I could forge enough results to get joe installed by default.

The first goal worked out great; Debian and Ubuntu now use popcon to fill their CDs, and your system will be just fine even if you only download a single CD image. The second goal unfortunately still hasn't come to pass... perhaps someday.

But more interesting is where these projects went beyond my original intention. The reason arcnet is still popular is that it's used in quite a few factories, because of its extremely good noise resistance and because it's good enough that nobody wants to replace it.

And popcon? Well, it now seems to have become the de-facto method of resolving things like vi vs. emacs wars or how soon git will be more popular than cvs, including a graph all the way back to 2004.

Amazing.

Update 2010/01/04: Thanks to Richard Soderberg for an updated link to the popularity-contest chart. He also pointed out that git-core now is more popular than cvs; it happened on 2009-03-21. At the current trend, it looks like git will overtake svn in another year or so.

On the causes of poverty

To seek "causes" of poverty in this way is to enter an intellectual dead end because poverty has no causes. Only prosperity has causes.
-- Jane Jacobs, The Economy of Cities

Ctrl-scrollwheel

Today I accidentally discovered the craziness of Ctrl-scrollwheel on a Mac. It also works with two-finger scrolling on a touchpad (although you need an extra finger for Ctrl), so MacBook users need not feel left out. Try it! It's ridiculous.

I'm totally going to use this next time I do a presentation. Which I think is in 15 minutes.