Stuffing the stuff

without getting stuffy
Everything here is my opinion. I do not speak for your employer.
May 2016
July 2016

2016-06-04 »

I'm gonna need a frickin' army of yakthulus.

2016-06-11 »

"You presume correctly, Doctor."

"That's what I do. I'm a theoretical physicist."

2016-06-21 »

Truth in Amtrak announcements: "This will be the most important question you ask us today: why isn't the wifi working? Because we're in Canada. Wifi doesn't work in Canada. Once we cross over into the United States, we will pick up the wifi signal. And it will be terrible."

2016-06-23 »

wifitop improvements from my train ride on Tuesday. New and improved: it actually accepts user input, it creates friendly names for things based on OUI and SSID beacons, it bolds the rows that have shown activity (other than beacons) in the last second, and you can expand/collapse the list of attached stations using keyboard navigation.

The most important thing I learned? Someone around here has a Blackberry on the guest network! Maybe Obama is in town. He's getting pretty lousy MCS levels though, so I'm guessing his phone got stuck and failed to roam to a nearer AP.

2016-06-24 »

My secret: I don't even know what difference it will make if UK separates from EU.

Context: I visited the UK in summer 2014. At that time, I observed that a) I had to cross customs to get there from another EU country (but not from Ireland) because it's not part of Schengen. b) They don't use the Euro. c) They don't use the metric system. d) They don't speak multiple languages. e) They're not even physically attached to the European continent, map wise.

What exactly does EU membership even mean to the UK? They were obviously, at best, only half-hearted members anyway. Free trade? Whatever, you can have free trade agreements without being in the same federation. Military agreements? Different EU countries seem to have their own militaries anyhow, I think. Human rights? Lots of countries (see: Canada) do fine on human rights without having to federate first. The right to be forgotten? Okay, now I'm just being snarky.

But seriously, I really don't understand. This seems very minor. If, say, Germany left, that would be another story, since they seem to be all-in on the EU. I see no signs that the UK ever was. People seem pretty upset, so I'm obviously missing something. Can someone enlighten me?

2016-06-26 »

This is a little ranty (although I admit a fondness for rants) but does have some interesting things to say about the decrepitude/fossilization of the basics of Unix. At this point, we're mostly afraid to touch the basics, because we're unsure (or too sure) of what will break. Thus, all our new stuff is built in additional layers of abstraction on top. Even containers themselves are a way of loading the same fossilized layers as more layers on top of themselves.

One of the few exceptions is systemd, which just goes to show you that if you'd have to be crazy to try to fix the base system, then only crazy people try to fix the base system.

http://dtrace.org/blogs/wesolows/2014/12/29/fin/

2016-06-28 »

This graph may not look like much, but it's a pretty huge achievement, resulting from the work of several different people over 2+ years. In the test, there are two wifi stations: A, which is slow (low signal), and B, which is fast (good signal). Both are downloading data as fast as they can, so the AP's transmit queue is full. (We aren't lucky enough to be downloading from a server that uses, say, TCP BBR, so it just floods the transmit queue.)

In the old Linux kernel, all the traffic for both stations would be dumped into a single transmit queue, so traffic for both stations could be caught behind a big blob of traffic from the very slow node A. Thus, node B's latency would be fine right up until node A got busy, at which point, both nodes' latency would skyrocket. This is havoc for any kind of real-time streaming, such as Hangouts, GVC, or video game streaming.

With the recent set of patches now going into Linux, we divide up the wifi tx queue per station, then retrieve data round-robin from each of the queues. This has two effects: node A's impact on B's latency is much less (shown here), and nodes A and B get equal airtime rather than equal bytes (called airtime fairness, not shown here). Notably, this means there is now almost no benefit to "boosting" streams based on user input or heuristics, because all the streams get good results.

$VENDOR once came to us and did a demo of why you need three radios, so you can put low-end clients into the "penalty box" channel so they can't ruin performance for high-end clients. Of course, they wanted to sell more chips. These patches solve the software problem in software instead, where it belongs, at no extra cost.

This graph is a major improvement over what you'd see with the old way of doing things, but there are still two problems: a) node B is still perturbed more than it should be. We believe this is just a bug in ath9k (too-large hardware queue in the no-aggregation case) that we can probably fix soon, so the induced latency should be max ~12ms rather than the current max ~35ms, and usually much less. And b) node A's own self-induced latency needs to be controlled. Michal Kazior actually has a patch for this, to apply fq_codel to each station's subqueue, thus keeping its own latency under control. However, I don't trust that patch yet, as fq_codel is rather fiddly and was not designed for wifi's variability, so we haven't applied it to our kernel as is.

Known contributors: Felix Fietkau, Michal Kazior, Toke Høiland-Jørgensen, Tim Shepard, Dave Taht, Andrew Mcgregor, Eric Dumazet, Johannes Berg.

I would like to claim credit for any of this, but the truth is I mostly just evangelized at a lot of people until they eventually did what I wanted, whether or not it was causally related to said evangelization. I guess we'll never know. Oh well, mission accomplished! [1]

[1] in the style of George W. Bush. This is still wifi after all.

May 2016
July 2016

I'm CEO at Tailscale, where we make network problems disappear.

Why would you follow me on twitter? Use RSS.

apenwarr on gmail.com