Assorted Writings

2007-07-03 »

On urban planning

Stationnement de Montreal has off-street parking sites available for a monthly fee. "52 lots!" they proclaim. "A network of 4000 parking spaces!" Those numbers surprised me a bit, given Montreal's 1.8 million residents.

It sounds like a Microsoft interview question. If there are 1.8 million people in Montreal and 4000 city-provided parking spaces, how long will the waiting list be? (Of course, you'd have to estimate the number of cars per person, number of private lots, etc.)

Answer: about four years.

Suggestion: park elsewhere.

Commentary: At least they're not building lots of ugly parking lots just because of massive demand. This is just one more part of the apparent Montreal policy of punishing people with cars.

July 3, 2007 14:40

2007-07-09 »

Leadership

"You know you're a good leader when people follow you, if only out of curiosity."
-- Colin Powell, via Pmarca

July 10, 2007 02:17

2007-07-17 »

True Love, Revisited

Last year at NITI I did a presentation I called True Love. It was about Lotus Domino. Follow the link if you don't believe me.

So how weird is it when, more than a year later, a review of Domino for Linux (which mentions Nitix) talks about true love in its first paragraph?

I knew it wasn't my imagination.

July 17, 2007 20:54

2007-07-20 »

I'm not lazy, I'm forward-thinking!

Apparently allowing blog comments is passé, according to people in the know.

I don't allow them on my site, but I thought it was just because I was too lazy to screen them, fight spammers, etc. As it turns out, allowing comments would restrict my freedom of speech! Holy cow! Who knew?

(As usual, people who want to email me or link to me about anything they find here are more than welcome to do so.)

July 20, 2007 18:59

2007-07-23 »

One of these sponsors is not like the others

Forget Web 2.0, it's Banking 2.0!

July 23, 2007 15:14

2007-07-25 »

The horrors of Setup.exe

Recently Jeff Atwood wrote about Mac software installation, complaining that it's not as easy as Windows setup.exe. There followed a barrage of comments, but I think most of them miss the point.

The point is that "installing" software is a very suspicious exercise anyway. What exactly is it doing to my computer that takes half an hour or more to install Microsoft SQL, for example? Why can't it just copy some files around and then run the server daemon?

Installers are arbitrarily slow

Since the ancient DOS days, this question has always been very mysterious to me. Once upon a time long long ago, I was installing a game (Wing Commander II, I think) from floppies on my 386, and wondering why it took absurdly long to do compared to, say, good old diskcopy.exe. There was no good reason; a few months later, I built an installer for an app I wrote, and it could copy itself from a floppy in a fraction of the time. And I didn't do anything special to achieve it; I just used plain fread() and fwrite().

To this day, programs like InstallShield continue to astonish me. Have you ever watched the installation log messages as they scroll by? They sometimes say things like "Creating registry key HKEY_LOCAL_blah\blah\blah\blah" - one per registry key. That's fine, but why do I have time to read them as they go by? On a dual-core 2 GHz processor, creating registry keys happens in a timeframe humans can see? How? I'm sure it's the same reason as Wing Commander II was slow, so many years ago.

InstallShield, by the way, is the worst offender I've ever seen. For some reason, it's also the most common installer platform. If you're writing an installer, take my advice: just use NSIS instead. It's awesome. We used it to build the ExchangeIt Plugin installer for Outlook. It writes registry keys during the installation process too. Know how long it takes? Me neither - it's immeasurably fast.

Actually, the entire ExchangeIt install process takes as long as it takes you to press Enter-Enter-Enter-Enter. I can't quite figure out what I'm doing right.

"Install" is the wrong concept

But there's another, more fundamental problem with all this: installers are doing too much stuff. "Install" is completely the wrong word for the operation we want to be doing. I don't "install" a toaster or a TV - I plug it in and push the button and it works. Anybody can do it. But you "install" a kitchen sink - and it's a scary operation, best done only by experts, because it involves plumbing and might spew water everywhere and ruin your whole house if you do just one thing wrong.

There's a reason people are scared to install Windows software - because it is like plumbing, and you might screw up and ruin your whole computer if you do it wrong. That's scary!

On Windows, installing software involves copying DLLs around (sometimes into the Windows directory where it overwrites ones from other apps), changing registry keys (sometimes ones that conflict with other apps), and creating Start Menu shortcuts. The installation process takes forever and has lots of unfriendly status messages just to underscore the fact that you should thank goodness for setup.exe, because you sure don't want to have to do all that stuff by hand.

On a Mac, most of those problems are solved. An application is actually a "package" folder containing all the app's critical files; most applications don't need to put any files outside that package. The "Start Menu" is the same thing as the Applications folder; unlike Windows, there's no need to create "shortcuts" to make the program visible. No user should ever have trouble figuring out the new application is in the Start Menu, because they're the person who put it there.

And you know what's cool? If they don't put it there - if they don't drag it anywhere at all - clicking on the application will work just fine. They never actually need to install it. And furthermore, they can drag it anywhere they want - to another disk, to their desktop - and it'll work fine there too. It's right where they left it.

Okay, how does uninstalling work? Well, you go to the applications folder and delete it, just like any other file. In Windows, users try that too: deleting the icon from the Start Menu to make the application go away and free up some disk space. In fact, they must have tried it a few too many times, because when you try it in Windows XP, a dialog pops up warning you that it won't work. Yeah, too bad about that.

The whole Mac thing with the .dmg files and virtual disks and needing to "eject the application" afterwards is mysterious to me, though. The design of application packages is extremely clever - but the front end to it is clunky. Why does the package need to be in a virtual disk at all? I'm not sure. At least it's just a UI issue and can be fixed in a future version.

(Incidentally, in .NET Microsoft is trying to do roughly the same thing as Mac packages: they call it Xcopy Deployment.)

Next time, some advice on how to write a better setup.exe. Because it doesn't really matter if setup.exe is a good idea or not; we're stuck with it.

July 26, 2007 02:04

2007-07-26 »

6 Steps to Writing a Better Windows Setup.exe

Yesterday's post on software installers mostly explained why the Windows software installation process is gross. But when I've written installers, even for Windows, they're not too gross. Here's some advice you can actually use.

1. Make the installer as small as possible

   Writing the installer really is a pretty unglamourous job to get stuck with, mostly because debugging it is such a pain. After all, it takes forever to run, and it leaves gunk all over your registry. And it breaks every time someone adds one more new file to the build, so it's much easier to write it at the very end of the cycle, right?

   There's an easy solution to this: minimize the amount of work that the installer actually does. Move all setup and configuration code to the application; the less that's in the installer, the less there is to debug, super-slowly, at the last minute right before release.

   Specifically, why on earth are you creating all your app's registry entries in the installer? Does that mean you've created them all by hand on your development machine? You mean all your developers did that, for the entire development cycle? Why? Why not just have your app create them automatically from default values if it can't find them at startup?

   Nitix puts all configuration outside the install process, and it's been easy and painless to maintain for years. I'd never do it any other way.

2. Use a better installer platform

   Use a good, fast installer like NSIS. Ditch InstallShield, it's terrible.

   NSIS is worth it just for the speed increase alone. I understand that the end user doesn't care how fast the installer runs; after all, they only run it once. But don't do it for the user, do it for yourself: that speed increase translates directly into a shorter compile-run-crash cycle for you, the installer developer. And that means you'll write better code in less time. You really will, trust me.

3. Use VMware Snapshots

   Install a fresh Windows system into VMware, take a snapshot, install your application, rewind, and repeat. Your test process isn't depending on the fact that your uninstaller really uninstalls everything, right? You're not reinstalling Windows from scratch every time, are you?

   Statistically speaking, you probably are. Stop it. VMware is free.

4. Use static linking

   Whenever possible, just link things statically into your application and make one big .exe file. Don't worry, Windows is smart enough to only page in parts as you need them. DLLs are only a memory saver if they're shared between apps - but they almost never are anymore, because everyone is desperately trying to avoid DLL hell. The default Delphi behaviour is to statically link everything; take their advice, it works.

5. Always just install everything

   Every installer always seems to have the stupid "Custom Install" option where you can choose which modules you do or don't want. Once upon a time, this made sense: why, you could cut an installation down from 40 megs to less than 10, if you were careful!

   But nowadays it's just a disaster. First of all apps that do this nowadays don't seem to have the same percentage flexibility in install size: either you install 1.2 gigs of stuff, or 1.15 gigs of stuff, or anywhere in between. Gee, thanks, that's a lot of flexibility, guys. It's still a 50 megabyte difference, but it just doesn't matter anymore.

   Secondly, every optional installation feature exponentially increases the number of combinations you need to test. And you're probably writing your installer right at the last minute. Are you really testing all the combinations? I bet you aren't.

   Try this: pick a random application. Uncheck all the boxes in the custom installation dialog. See how small it gets (not small). Now install it. See if it runs. Chances are, something weird will explode when you run it. Why? Well, because nobody actually does that, and certainly nobody tests it. If someone calls in to tech support about the problem, guess what they say? "Reinstall it with the default settings and it'll be fine."

   If you're not going to test the option, just leave it out.

6. "Kram" your data files into the .exe too

   Surprisingly, one of the slowest parts of installation is creating and copying a million tiny files: the Windows filesystem is horrendously slow at creating files, but fast at reading and writing existing files.

   Here's a trick some DOS programs were using in the early 1990's and it still works today (on DOS, Windows, Mac, and Unix!): just concatenate your data files to the end of your .exe as part of the build process, and tack an index (with the filenames, offsets, and lengths) on the end. You'll be amazed at the order-of-magnitude decrease in install time.

   And you can do some neat tricks with this too: for example, if you often load a whole bunch of related files at almost the same time (eg. an html file and its contained images), you can improve "locality of reference" on disk by simply concatenating the files together in the right order. If you create them as separate files, Windows has to guess (usually incorrectly) which ones belong together. But if they're all in one big file, Windows knows they're related and will try to keep them together on the disk.

   Useless trivia: this is also how "self-extracting" zip files work. There's no magic whatsover: just take your unzip.exe and tack the .zip file to the end. (Of course, your unzip.exe has to be smart enough to seek from the end of the file instead of the start, since the zip file no longer starts at offset 0.)

   I used to see this trick all the time in DOS-based "demo" programs - they'd be distributed as a single .exe that contained all the music and images tacked to the end. It's a shame this idea has been mostly abandoned, because it still works great and makes your application seem a lot less unwieldy.

The end result

If you follow all the above steps carefully, you should be able to get your installer down to its bare minimum: copy your .exe file to Program Files somewhere, create a Start Menu entry, and register the uninstaller. The Uninstaller does exactly the opposite, of course.

Exactly one test case, and it's trivial to verify if you got it right. And you know your app works identically with the installer and without, because the installer doesn't do anything.

Even if you assign a total newbie developer to that, he shouldn't be able to screw it up.

July 26, 2007 02:26

2007-07-27 »

Linus Torvalds discusses version control

It's not very often that you get to see Linus Torvalds discussing anything at all for a full hour. Follow the link for a discussion about version control, git, and why CVS sucks.

Surprisingly to me, he's a bit of an unexciting presenter. The content is interesting, though.

July 27, 2007 23:46