When there's only one

...there's only one choice
Everything here is my opinion. I do not speak for your employer.
July 2012
September 2012

2012-08-04 »

You cannot record the entire olympics in HD in 2TB :(

2012-08-09 »

New word: bureaucrazy.  The best part is you can say it right in front of bureaucrats and they won't even know.

2012-08-10 »

py-monotime, CLOCK_MONOTONIC, and time.monotonic()

An increasingly common problem nowadays is caused by the fact that the time() or gettimeofday() system calls do not always return monotonically increasing values. That is, sometimes they go backwards, and sometimes they take giant leaps.

In Unix, thankfully, timezones don't affect this: you don't have to worry about, say, daylight savings time messing up your time calculations, as long as you use time_t or struct timeval instead of struct tm for any long-lived storage. (Example of not doing it right: if your log file includes the current local time of each message, then every autumn when daylight savings causes a one-hour backwards jump, you will have log messages that appear out of order in the file, and if you sort them by date, it'll be a lie.)

However, there are other things that do make a mess, because Unix time was not entirely well thought out when it was invented. The most commonly known problem is NTP, which jumps your clock around sometimes. People mostly try to ignore this by just assuming that after boot, NTP won't jump the time, it'll only slew the time (which it tries very hard to do), and this mostly works, so people get away with this assumption 99.9% of the time, and their program goes slightly bananas the other 0.1% of the time, after which they typically reboot and are happy.

There's another annoyingly niggly one though, which is virtually impossible to work around: leap seconds. As implemented in Unix, a leap second literally causes the same time_t to occur twice. That turns out to be most definitely the wrong answer; the right answer would have been to include leap seconds in the localtime() calculation, not in the kernel's implementation of time. But sadly, mistakes were made, and we have what we have. There are many stories in computer lore of programs that work great except at the exact moment of a leap second, at which time all sorts of things (notably multimedia timing algorithms) go completely wrong. Once again, though, rebooting fixes it.

Anyway, it turns out there is already a solution for all these problems, and it's been out there for a long time, but not well standardized. The solution is called a "monotonic clock." In a monotonic clock, absolute times aren't very meaningful (they are usually "seconds since the system booted" or something like that) but relative times, which are almost always what you care about, always do what you want. So if you say, "give me an event 60 seconds from now" then you schedule it for time monotonic() + 60, and life is simple and good, and you don't need any crazy hackery to make sure you deal correctly with backwards-flowing time.

If you want access to this lovely thing on a system compliant with POSIX.1-2001, such as Linux, what you want is the clock_gettime(CLOCK_MONOTONIC) system call. Unfortunately, non-Linux systems seem to largely not support it. On MacOS, you can use this advice from Apple instead.

What's worse, if you're writing in python, there is no access to monotonic time even on systems that do support it. Well, there is, starting in python 3.3 apparently (which adds a time.monotonic() function), but nobody uses python 3, so that doesn't really help most of us. For the rest of the world, I just made a new python module called monotime. If you import it, then time.monotonic() suddenly appears, and you can write your program to use it, just like if it were supported internally in your copy of python. It's licensed under the Apache 2.0 license. You can get it through pypi and I also added Debian packaging scripts.

I should also mention python-monotonic-time, which I didn't use for two reasons. First, it's under the GPLv3 (not the LGPL), which would infect any program that uses it. Secondly, it's written using python's ctypes, which is great for hackery, but is very brittle (it depends tightly on system-dependent library names and struct formats, without actually including the system header files during build) and much slower than a C module, which is what my monotime module uses. Programs that need monotonic time typically need a lot of monotonic time, and you want it to be fast.

For more time-related trivia than you can possibly imagine, check out python's PEP 418, which introduced the time.monotonic() function and a few other things.

By the way, getting approval to release this code through Google's super-streamlined open source releasing process took only 47 (monotonic) minutes and almost no CPU time.

2012-08-12 »

Protip: if it takes longer to load the index than to cat the entire dataset, then it's not an index. It's definitely something, but it's not an index.

2012-08-13 »

This is much better than a simplistic conservative vs. liberal dichotomy.  Just like Canadian politics is better than Democrats vs. Republicans. When you put the focus on the actual correct statement of the problem, political issues tend to melt away.

http://blog.ometer.com/2012/08/13/what-matters-in-software-development/

2012-08-16 »

All's quiet on the beta tester front.  Too quiet.

Maybe they're all... happily watching TV?

No.  Unlikely.

2012-08-18 »

Rule of thumb:

If the software watchdog timer wakes you up from a poweroff state, something is backwards.

2012-08-19 »

The Low-Latency Analog Measurement Adapter (LLAMA):

I want a device that will be able to measure, with sub-millisecond accuracy, the amount of latency in a complete audio network.  Assuming an audiovisual signal has proper lipsync, audio latency is a good proxy for video latency, which is what I actually care about, for things like video games and videoconferencing.

Here's how I think you could design a system for measuring the audio latency of a TV, for example, given a plain RCA-style or headphone-style input:

  • Create a device with two synchronized ADCs. For each sample, you always sample both ADCs at the same time.

  • Hook ADC1 to the audio signal with a simple three-way splitter (we don't care about analog signal degradation here).  The other splitter port goes onward to the TV.

  • Hook ADC2 to a microphone near the TV's (or home theatre system's, or whatever's) speaker.

  • Generate the audio signal from a PC.  Use a really simple signal that's amenable to a really clear FFT, such as alternating between two frequencies or sweeping up and down, or pulsing on and off.

  • You can't trust that a PC's audio output has low latency.  (Audio drivers are often crap.)  This is why we needed two ADCs.  Take both ADC inputs back in and FFT them.  The lag on ADC1, compared to your expected output, is how slow your PC sound card is.  The lag on ADC2 compared to ADC1 is how slow your sound/video delivery system is.

Does anyone know if a system like this already exists?  (I realize you could do it with an oscilloscope, but I'd rather have something that requires zero expertise to operate so it could be used to compare lots of different equipment objectively.)

2012-08-22 »

Apparently me starting to enjoy a service is a bit like the kiss of death:

http://www.zdnet.com/onlive-sells-assets-to-onlive-leaves-htc-and-other-investors-out-to-dry-7000002873/

On the other hand, you have to admire a video game company that has figured out how to respawn in real life.

2012-08-25 »

Oh look.  There's a perfectly sane standard for flash memory partition tables (where the partition layout is itself saved in flash, so the storage is self-describing) called RedBoot, and it's already supported by Linux and the brcmnand driver.    That might have been useful to know.

July 2012
September 2012

I'm CEO at Tailscale, where we make network problems disappear.

Why would you follow me on twitter? Use RSS.

apenwarr on gmail.com