Guspaz asks why routers only do one-way QoS (that is, in the upload direction).
The simple answer is that while yes, you can fiddle with TCP sliding windows to adjust the receive rate, it's very complicated to do so accurately. The way to make TCP slow down is literally to drop packets, not just to delay them; delay is a latency thing, not a bandwidth thing, and a server noticing a high latency should put more packets in flight at once, not less. But you'd have to drop outgoing ACKs, not incoming packets, to reduce the bandwidth, and that's just complicated.
Incidentally, Linux's QoS stuff can do it, but it has a fatal complexity: you have to tell it what your link bandwidth is, and most users just don't know. mag will remember how annoying it is to calculate that correctly. (BTW: no, we still don't.)
Conversely, outgoing QoS (as it's normally defined) is easy. You just take all the packets currently sitting in the queue, and sort them in priority order. Got a new packet and the queue is full? Throw out the lowest-priority one. No real "bandwidth management" is needed, the algorithms are trivial, and it doesn't matter what your actual bandwidth count is.
ssh+2FA to all your machines, anywhere, without opening firewall ports.