(No, I still haven't finished transcribing my handwritten notes from when I was laptopless in early September. Here's another entry. Note to self: I know it's where the term "copy and paste" came from, but yeesh, handwritten notes are a really inferior medium for actually doing it.)

Delirium 3: Specs and Constraints

So let's say you managed to assemble a team of obsessive, maniacal geniuses who are all "signed up" and ready to get the job done. Great! So, uh... what job, exactly?

There's more than one reason people don't run teams like this too often. The most basic reason is obvious enough: the rules are so strange, undocumented, counterintuitive, and completely opposite from normal (successful!) engineering practice that people just don't try to assemble the "right" conditions very often. But if, after takeoff, the results tended to work out spectacularly... well, people would get over all that other stuff, right? ... Right? ... Actually, yes, I think so.

But life isn't so simple. There's a much more serious problem with the "obsessive genius" technique. The problem is that the results are entirely out of control. You'll almost certainly get something cool - probably more cool stuff than you know what to do with. But you won't know in advance exactly what it'll be. And that, my friends, makes it hard to run a business.

Some companies do it anyway. They start a "research lab" or "skunkworks," put the wheels of obsession in motion, and pray like crazy that something good comes out.

Well, research labs aren't for me. I'm not going to beg "the establishment" for their precious resources, with nothing more to offer than a random splash from my really, really big shiny innovation firehose. When all I have to choose from is total randomness in large quantities or total determinism in disappointing quantities, I'm in a classic compromise situation. The non-compromise condition is massive, sustainable, directed ingenious output. And my theory is that it's a lot easier to direct output that's already massive, sustainable, and ingenious than it is to take small deterministic output and make it bigger and more ingenious. It's easier to learn to aim a big firehose than to put out a big fire with a really accurate squirt gun. Mind you, no matter how well you aim, some tangential stuff is still going to get a little wet. But it'll dry off eventually.

Which all finally brings us around to my point. It is possible to direct the output of a team of obsessives - sort of. You just have to be careful about what exactly you direct.

Determinists love specs - clear descriptions of the ins and outs, which must be captured exactly. Then you leave your geniuses to do any design they want, as long as it implements the spec precisely.

I tried that method. I really did. I sliced and diced specs vs. designs in every way I could think of. UI design isn't specification, it's design!, I declared once.

But with each and every attempt at this technique, I failed in one of two ways. Either the spec was too dumb and restrictive - or the result was too crazy and random. Either the product was overspecified or it was underspecified. And now I realize that it always will be. Trying to "right-size" your spec is just a tradeoff - a compromise. The right answer doesn't even lie on that axis.

But even though I've never really managed to separate the spec from the design (for management purposes at least - it works for post-documentation, or what we sometimes call a "retrospec"), I have successfully produced non-random, ingenious results - and I've worked with other teams, like NVS and GourD, that produced them. What's the secret?

The secret is "end-to-end." The problem with our spec-design separation is actually very simple: it draws a line between the real, deterministic world - the spec - and the world of genius - the design. But that's nonsense. How can your product be ingenious if its workings were defined by someone deterministic? How can it be useful if the workings are defined by someone crazy? The spec, too, must be written by someone in genius mode. Those were the real successes I've seen - where the spec, the design, and the product were all done in genius mode, "proper processes" be darned. And I've been fooled like everyone else: GourD had a Requirements doc, then a Spec, then a Design, then an implementation. Deterministic, right? The model of good engineering!

Yeah, sure. The author of all those parts was the same person, and while they were written down in sequence - which is a very fair, respectable, methodical approach applicable to geniuses as well as anyone else - I know for sure that the documents were "non-causal." The contents of the design changed the requirements long before anyone wrote down the design. I was there. You can't fool me. Just try to tell me the requirements would have been exactly the same if Mediawiki hadn't existed as a model. It was all a good exercise in in thinking clearly. But it wasn't a "deterministic engineering process." It was a sham. It was an ingenious bunch of work that couldn't help but come out because all the right pieces, including the right people with the right motivations, were in place.

So the question is, then, how did it get to be the right product? Where did the genius stop and the real world begin?

The answer to that question is the key to everything.