Like an orange
Everything here is my opinion. I do not speak for your employer.
August 2013
October 2013

2013-09-04 »

After observing cloud infrastructure for a while, I offer you this observation:

We have extended the CAP theorem to the CAPS theorem. Consistency, Availability, Partition Tolerance... Speed.  Pick any two.

And now, back to using sqlite.  Which picked C and S, of course.

2013-09-06 »

I really liked this article, because it convinced me of the opposite of what they were trying to convince me of.

"This is about a simple rule: Brand design follows brand management, not the other way around."

Here's what I think Mayer wants to do with the brand: make it clear to every person inside the company that nothing is sacred.  You can fire your high-priced consultants and replace their work in a weekend.  You can do it yourself.  The CEO is not afraid to step in and make that happen.  The CEO is not afraid to blog about it afterwards and take full responsibility for a decision.  And the CEO knows how to make a splash without breaking actual important parts (like usability).

I also suspect that if you worked at Yahoo, there was some kind of hidden message in there about being data driven, although I don't know exactly what that message was if they really did just pre-select this logo before doing 30 days of other ones.

I don't know if this Yahoo turnaround will work out.  It's a near-impossible job. But I really like that the turnaround is a complete rejection of the person who wrote this article and everything he represents.


2013-09-11 »

The ID (internal delay) setting has three control bits:

1. Override hardware default setting: enabled by default. (What!?)

2. Disable 2ns delay feature: disabled by default.  (Ok, though the negative phrasing makes this confusing.)

3. Bypass 2ns delay feature: enabled by default.  (Errr....)

So by default it is not-disabled, so the delay function is on.  But bypassed.  And the hardware defaults are irrelevant; to use them, your software has to explicitly disable the software override.

Thanks, guys.

2013-09-18 »


In 1929, a ragtag band of engineers at the BBC started sending TV over the air! We'll celebrate this momentous occasion by doing it again.... in 2013... (wait, seriously?)

Seriously. TV boxes without wires. Way cool.

2013-09-19 »

Thanks for trying the new version of the driver we sent you that we said would solve your problem.  Yes, we're aware of the crash you reported.  It happens to us all the time too.

2013-09-24 »

A story about mocks:

A couple of weeks ago I got into a minor argument with someone about whether/when one should use mocks vs. trying to actually call the library in question.  My argument was that whenever you use mocks instead of real code, you're reducing code coverage, so if you're going to use them, you'd better have a good reason (such as a library that has an actual external dependency on the network).

So okay, there was not much resolution to that argument, and I was too depressed to bother looking to see if anyone took my advice.  But a few days later I ran into a problem with a program that I had helped write, related to its handling of HTTP redirects.  I looked at the HTTP client test code and... well, it didn't test HTTP redirects at all, which isn't surprising, because really, who tests HTTP redirects.

But it also didn't use real HTTP.  The http "client" was talking to the http "server" using a mock library that left out HTTP altogether.  Which worked okay, but wasn't going to let me add the HTTP redirect tests.

So I converted the thing to use real http client and server classes instead of mocks.  No big deal.  In fact, the resulting program was slightly shorter.  Excellent.  It even passed, leading me to doubt all this advice a bit, since avoiding mocks had failed to benefit me in any way.  Then I added the HTTP redirect test I came here to add, which after all that, was the easy part really.  It passed too!

Great, right?  Well, no, because I hadn't fixed the bug yet.  Sigh.

What I learned was that the main() program was configuring the http client library to use libcurl.  But the class under test was not doing that.  So my new unit tests were testing a totally different http client library.  Oops.  Fixed that, and now the test failed as expected.  Except it failed in the wrong way; the test just froze upon receiving a redirect.  It wasn't supposed to do that, it was supposed to redirect the wrong way (using GET instead of POST, in case you're wondering; we needed it to redirect using POST).

Oh, why is that? To make a long story short, a really weird race condition in libcurl that triggers pretty easily when using epoll, but is harder to trigger otherwise (but which would have made it dangerously error prone if you use threads).  Word to the wise, upgrade to libcurl 7.31 or higher if you use threads or epoll.  (Explanation: it closed the socket, then waited a while, then unregistered for events on the socket.  If someone opened a new socket and got assigned the same fd number before the event unregistration, bad news.)

So what started out as a one-line fix turned into a whole day of screwing around, and my no-longer-mocked unit tests found a bug in a library that wasn't even mine, which I wasn't looking for, and that I didn't really want to know about.  Also, it was probably responsible for the way our client randomly freezes up in the field sometimes, on a small number of devices, with very low probability.  And now it's fixed.

So here's my updated view on mocks: overusing mocks is the lazyperson's backup strategy to not writing tests at all.  If you don't write tests, you don't have to find bugs you never wanted to find.  Eventually, someone shames you into writing tests.  But you want to find bugs in your code, not bugs in someone else's code, so you write mocks.  That saves you time debugging other people's code, which you never wanted to do and isn't how you get promoted and make an impact, right?

But if you want to fix all the weirdo rare bugs in your product, you try to avoid mocks.  Because you do find edge cases, even in other people's code.  And other people's code is part of your program, like it or not.  And clearly their own tests didn't find all the bugs.


I checked in my fix (which was upgrading to libcurl 7.32, of course).  It broke SSL and I didn't notice.  I hate computers.

2013-09-26 »

Contractual non-screwage agreement

Today I was reminded of some legalese I put into an employment contract at one of my startups, because the person who had been employed under said contract has a friend who wanted to copy some of the legalese for use at his own company.  Since the contract was, in theory, confidential, he wanted to check with me first before sharing the text.

The parties agree to not screw each other over, to not do stupid shit, to not give away each others’ secrets (if you don’t know what’s a secret, you should ask first), and to not use the term “nyah nyah” to refer to any aspect of their business or personal relationships, even in their heads, upon penalty of death.  Contractor agrees not to start any new business or activities intended to compete with the Company, nor to solicit the Company’s customers to switch to any competing product except with the express written permission of the Company.

Naturally I told him that legalese is generally considered not to be copyrightable for the same reason C header files are considered not to be copyrightable.

Warning: this is not legal advice.  If this had been legal advice, I would be sending you a bill for it.

2013-09-27 »

Today in Learning (tm) with Real Actual Facts (tm):

2.4 GHz wifi has channels 1-11, but really only 3 non-overlapping 20 MHz channels, generally agreed to be 1, 6, and 11.  (Though some troublemakers apparently prefer to use 5 instead of 6.)  The existence of the other in-between channels serves mainly to confuse, since if your router uses an overlapping-but-not-aligned channel, you probably won't see its carrier, but it will still happily interfere with you like crazy.

5 GHz wifi is known to have "more channels", but how many more? Well, there are at least 21 channels licensed for use in the U.S.  But here's the catch: they're all 20 MHz wide instead of 5 Mhz!  So that's not just twice as many, that's 7x as many.  And as a bonus, the wifi standards people got it right and disallowed in-between channels, so you don't have weird overlapping interference like in 2.4 GHz.

As a double bonus, 5 GHz gets blocked better by walls.  That may not sound like a bonus, but when it comes to your neighbours not interfering with your signal, it definitely is.

No wonder 5 GHz is quieter.


2013-09-30 »

Today I'm debugging NAND flash drivers.

"Einstein repeatedly argued that there must be simplified explanations of nature, because God is not capricious or arbitrary. No such faith comforts the software engineer."
       – Fred Brooks

August 2013
October 2013

I'm CEO at Tailscale, where we make network problems disappear.

Why would you follow me on twitter? Use RSS.

apenwarr on gmail.com