Variation on a theme: maxtime

Previously, I explained my simple runlock program. This time, a slightly different example: maxtime.

The purpose of maxtime is to automatically kill a process if it runs for more than a certain number of seconds. This is useful in gitbuilder to handle cases when your build process gets permanently stuck and never exits (such as if you have unit tests that go into an infinite loop or start waiting for user input). Last time, when discussing runlock, I mentioned that it was incredibly evil to simply break a lock because it's "old"; maxtime solves this correctly, by properly killing the program that owns the lock.

Here's the interesting part of maxtime:

    my $pid = fork();
    if ($pid) {
        # parent
        local $SIG{INT} = sub { kill 2, -$pid; };
        local $SIG{TERM} = sub { kill 15, -$pid; };
        local $SIG{ALRM} = sub {
        kill 15, -$pid; sleep 1;
        kill 9, -$pid; exit 98; };
        alarm $maxtime;
        my $newpid = waitpid($pid, 0);
        my $ret = $?;
        if ($newpid != $pid) {
            die("waitpid returned '$newpid', expected '$pid'\n");
        }   
        exit $ret >> 8;
    } else {
        # child
        setpgrp(0,0);
        exec(@ARGV);
    }

To the untrained observer, this looks a lot like the fork/exec chunk of runlock. But it's not. There are some really critical differences here.

The most basic difference is we're now calling alarm() to request an alarm signal after the timeout. When we receive the signal, we make sure the process is good and dead.

Another difference is that we kill "-$pid" in maxtime, versus "$pid" in runlock. That takes advantage of a little-understood Unix feature called process groups. Have you ever noticed how when you run a command in the background, then terminate it, your shell can reliably kill it and all its subprocesses? Like this:

    $ (sleep 100 & sleep 200 & sleep 300) &
    [1] 16130

    $ kill %1

In the shell, there's an important difference between killing %1 versus killing 16060. If you kill %1 in the above example, you actually send signal 15 (SIGTERM) to "process -16060", also known as "process group 16060." You'll explicitly terminate all the processes, not just the parent one. Watch:

    $ (sleep 100 & sleep 200 & sleep 300) &
    [1] 16130

    $ ps xjf
     PPID   PID  PGID   SID STAT COMMAND
    16123 16124 16124 31457 S     \_ sh
    16124 16130 16130 31457 S         \_ sh
    16130 16131 16130 31457 S         |   \_ sleep 100
    16130 16132 16130 31457 S         |   \_ sleep 200
    16130 16133 16130 31457 S         |   \_ sleep 300
    16124 16134 16134 31457 R+        \_ ps xjf

    $ kill 16130
    $ ps xjf
     PPID   PID  PGID   SID STAT COMMAND
    16123 16124 16124 31457 S     \_ sh
    16124 16135 16135 31457 R+        \_ ps xjf
        1 16133 16130 31457 S    sleep 300
        1 16132 16130 31457 S    sleep 200
        1 16131 16130 31457 S    sleep 100

    $ kill -15 -16130
    $ ps xjf
     PPID   PID  PGID   SID STAT COMMAND
    16123 16124 16124 31457 S     \_ sh
    16124 16136 16136 31457 R+        \_ ps xjf

Notice that after killing the primary subprocess (known as the "process group leader"), the child process's "sleep" calls are still running happily in the background. That's no good! But you can see that they all have the PGID (process group id) of the primary subprocess, 16130. If we kill the process group using -16130, the entire set of processes will die.

This is what we want with maxtime, of course. If the problem really is some unit tests that are frozen in the background, you can't guarantee that the process group leader will clean them up correctly; after all, he's obviously buggy, and that's why you're killing him.

One last note: how do you create a process group in the first place? Normally, when you fork() a process, the new process is in the same process group as its parent. (It wouldn't be much of a group if every new process got a new one!)

That's the purpose of the setpgrp(0,0) call⁽¹⁾ right before running exec(). This creates a new process group for the current process, and gives it the same number as the process itself. (That way, the parent will know what the pgid is.) Your shell does this automatically whenever you run a command interactively.

Okay, so what's that SID field, then?

The SID is the "session" ID, and it's related to your terminal. Session IDs are even more complicated, because of the restrictive rules on when you are/are not allowed to create a new session, who controls the session, and so on. Basically, the session is what's responsible for such handy features as this:

    $ cat &
    [1] 23298

    $ true
    [1]+  Stopped(SIGTTIN)        cat

Only the current foreground process in a particular session is allowed to receive input from your terminal, or things would get pretty confusing. Also, when you press CTRL-C, it sends the SIGINT signal only to the current foreground process. To read more about sessions, try "man 2 setsid".

Footnote

⁽¹⁾ Note that setpgrp(0,0) in perl is equivalent to setpgrp(void) in C, which is the same as setpgid(0,0) in C. I'm not sure why perl chose to name their system calls like that. Thanks to Aaron Davies for pointing out that my article mixed the two meanings.

::nv,li