Uploading yourself for fun and profit (plus: sshuttle 0.20 "almost" works on MacOS)

After trying out the initial version of sshuttle that I produced this weekend, a few people asked me whether it would be possible to make it work without installing a sshuttle server on the server machine. Can it work with just an sshd, they wondered?

Good question.

Some people pointed out ssh's -D option (dynamic port forwarding using a SOCKS proxy). If we just used that (ie. the sshuttle client transproxies stuff into ssh's SOCKS server), then there wouldn't need to be a server side for sshuttle, and that would solve the problem. But sadly, sshd's latency management is pretty thoroughly awful - among other things, it sets its SO_SNDBUF way too high - so if you have a few connections going at once, performance takes a dump. sshuttle has some clever stuff to make sure that doesn't happen even if you've got giant ISOs downloading over your VPN link. I'd like to keep that.

So then I said to myself, hey, self, what if we just uploaded our source code to the remote server and executed it automatically? It works for viruses (technically worms), after all.

You know it's never a good thing when I start talking to myself. And yet the result is surprisingly simple and elegant. Here's a simplified version of what the "stage 1 reassembler" looks like:

   ssh hostname python -c '
    import sys;
    exec compile(sys.stdin.read(%d), "assembler.py", "exec")'

Where "%d" is substituted with the length of assembler.py. Assembler.py, by the way, is the "stage 2 reassembler," which looks like this:

    import sys, zlib

    z = zlib.decompressobj()
    mainmod = sys.modules[__name__]
    while 1:
        name = sys.stdin.readline().strip()
        if name:
            nbytes = int(sys.stdin.readline())
            if verbosity >= 2:
                sys.stderr.write('remote assembling %r (%d bytes)\n'
                                 % (name, nbytes))
            content = z.decompress(sys.stdin.read(nbytes))
            exec compile(content, name, "exec")
            # FIXME: this crushes everything into a single module namespace,
            # then makes each of the module names point at this one. Gross.
            assert(name.endswith('.py'))
            modname = name[:-3]
            mainmod.__dict__[modname] = mainmod
        else:
            break
    main()

Yeah, that's right, I gzipped it.

You know the best part? When the server throws an exception, it still gives the right filenames and line numbers in the backtrace, because we assemble each file separately.

If anybody knows the right python incantation to make it import each of the modules as a separate actual module object (rather than just dumping it all into the global namespace, as the comment indicates) please send a patch.

Grab the latest version of sshuttle VPN on GitHub.

"Almost" works on MacOS

Responding to popular request, I thought I would try to get the sshuttle client working on MacOS. (The sshuttle server already works - or at least it should work - on just about any platform with an sshd.)

MacOS, being based on BSD, uses the same ipfw stuff as FreeBSD seems to use. So it "should" be just a matter of having it auto-detect whether the current system uses iptables or ipfw, then run the right commands, right?

Well, almost. I did all that stuff, and I've almost got the rules working, but I just can't make it work right. I'm using MacOS X Snow Leopard on my laptop. I checked it in and pushed it anyway in case anybody wants to take a look; the final fix is probably a one liner.

For more information on my conundrum, see my (as yet unanswered) question on ServerFault. If you can contribute an answer, you'll forever be my hero. Even if you don't know anything about ipfw, if you could run through the steps on your version of MacOS or BSD and tell me what happens, it could help narrow things down.

Enjoy!

Update 2010/05/05: Thanks to Ed Maste and drheld for helping confirm. It seems that FreeBSD and MacOS 10.5 Leopard work fine - which means sshuttle 0.20 should work for you, yay! - while Snow Leopard consistently does not.

Really, that restores a bit of balance to the universe; since everyone I know who's upgraded to Snow Leopard went through huge pain and suffering, I was a little shocked to find that my own upgrade had been harmless, and one friend's upgrade really did make their computer go much faster (boggle). Now that Snow Leopard has finally screwed me after all, I once again feel like things are as they should be.