UNIX ptrace() block child's system calls

I am deve开发者_如何学Cloping a grader for programming contests. Basically, the grader must run the solution program in an 'isolated' process. So, I would like the solution not to call any harming system calls (such as system(), fork(), etc.). Can I use ptrace() to achieve that?

I think there are 2 possible solutions:

1. Using the LD_PRELOAD mechanism to create 'shim' to replace the system calls you want to stop.
2. Use setrlimit() to limit what the calling process can do. Unfortunately these limits seem to be a per-user, not per-process, basis, which makes calculating the correct value to set very difficult.

EDIT: I have the first option working, and have included the necessary code below. Build binaries using make all and then test with make runtests:

$ make all
gcc -fPIC -shared -Wl,-soname,libmy.so.1 -o libmy.so.1.0 lib.c
ln -sf libmy.so.1.0 libmy.so.1
ln -sf libmy.so.1 libmy.so
gcc -o test test.c

$ make runtests
Without LD_PRELOAD:
./test
in child: retval=9273
in parent: retval=0
With LD_PRELOAD:
LD_PRELOAD=./libmy.so ./test
libmy.so fork!
fork error: error=Operation not permitted (1)

Makefile:

all: libs test

runtests:
    @echo Without LD_PRELOAD:
    ./test
    @echo With LD_PRELOAD:
    LD_PRELOAD=./libmy.so ./test


libs: lib.c
    gcc -fPIC -shared -Wl,-soname,libmy.so.1 -o libmy.so.1.0 lib.c
    ln -sf libmy.so.1.0 libmy.so.1
    ln -sf libmy.so.1 libmy.so

test: test.c
    gcc -o test test.c

clean:
    rm -f test libmy.so.1.0 libmy.so.1 libmy.so lib.o

lib.c:

#include <stdio.h>
#include <unistd.h>
#include <errno.h>

pid_t fork()
{
    printf("libmy.so fork!\n");
    errno = EPERM;
    return (pid_t)-1;
}

test.c:

#include <stdio.h>
#include <string.h>
#include <errno.h>

int main(int argc, char **argv)
{
    int retval = fork();
    if (retval == 0)
        printf("in parent: retval=%d\n", retval);
    else if (retval > 0)
        printf("in child: retval=%d\n", retval);
    else
        printf("fork error: error=%s (%d)\n", strerror(errno), errno);
    return 0;
}

Yes you can use ptrace() to block certain syscalls using the PTRACE_SYSCALL option. Here is a project using this feature:

https://github.com/t00sh/p-sandbox/blob/master/p-sandbox.c

If you only target Linux, I would suggest seccomp instead, which is a faster technique, to white list/blacklist certain syscalls or restrict their arguments.

An alternative would be Google's Native Client. This projects provides a cross-platform implementation of an application sandbox.

You could also run the applications as an unprivileged user within a container such as docker or LXC to limit the damage.

Using LD_PRELOAD alone is insecure as executable may come with their own syscall implementation and bypass the underlying libc.