fcntl.flock - how to implement a timeout?

I am using python 2.7

I want to create a wrapper function around fcntl.flock() that will timeout after a set interval:

wrapper_function(timeout):

I've tried calling on another thread and using thread.join(timeout) but it seems that fcntl.flock() continues blocking:

def GetLock(self, timeout):
    """Returns true if lock is aquired, false if lock is already in use"""
    self.__lock_file = open('proc_lock', 'w')

    def GetLockOrTimeOut():
        print 'ProcessLock: Acquiring Lock'            
        fcntl.flock(self.__lock_file.fileno(), fcntl.LOCK_EX)
        print 'ProcessLock: Lock Acquired'

    thread = threading.Thread(target=GetLock开发者_开发知识库OrTimeOut)
    thread.start()
    thread.join(timeout)

    if thread.isAlive():
        print 'GetLock timed out'
        return False
    else:
        return True

I've looked into solutions for terminating threads, the most popular solution seems to be sub-classing threading.thread and adding a feature to raise an exception in the thread. However, I came across a link that says this method will not work with native calls, which I am pretty sure fcntl.flock() is calling a native function. Suggestions?

Context: I am using a file-lock to create a single instance application but I don't want a second instance of the application to sit around and hang until the first instance terminates.

Timeouts for system calls are done with signals. Most blocking system calls return with EINTR when a signal happens, so you can use alarm to implement timeouts.

Here's a context manager that works with most system calls, causing IOError to be raised from a blocking system call if it takes too long.

import signal, errno
from contextlib import contextmanager
import fcntl

@contextmanager
def timeout(seconds):
    def timeout_handler(signum, frame):
        pass

    original_handler = signal.signal(signal.SIGALRM, timeout_handler)

    try:
        signal.alarm(seconds)
        yield
    finally:
        signal.alarm(0)
        signal.signal(signal.SIGALRM, original_handler)

with timeout(1):
    f = open("test.lck", "w")
    try:
        fcntl.flock(f.fileno(), fcntl.LOCK_EX)
    except IOError, e:
        if e.errno != errno.EINTR:
            raise e
        print "Lock timed out"

I'm sure there are several ways, but how about using a non-blocking lock? After some n attempts, give up and exit?

To use non-blocking lock, include the fcntl.LOCK_NB flag, as in:

fcntl.flock(self.__lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)

For Python 3.5+, Glenn Maynard's solution no longer works because of PEP-475. This is a modified version:

import signal, errno
from contextlib import contextmanager
import fcntl

@contextmanager
def timeout(seconds):
    def timeout_handler(signum, frame):
        # Now that flock retries automatically when interrupted, we need
        # an exception to stop it
        # This exception will propagate on the main thread, make sure you're calling flock there
        raise InterruptedError

    original_handler = signal.signal(signal.SIGALRM, timeout_handler)

    try:
        signal.alarm(seconds)
        yield
    finally:
        signal.alarm(0)
        signal.signal(signal.SIGALRM, original_handler)

with timeout(1):
    f = open("test.lck", "w")
    try:
        fcntl.flock(f.fileno(), fcntl.LOCK_EX)
    except InterruptedError:
        # Catch the exception raised by the handler
        # If we weren't raising an exception, flock would automatically retry on signals
        print("Lock timed out")

I'm a fan of shelling out to flock here, since attempting to do a blocking lock with a timeout requires changes to global state, which makes it harder to reason about your program, especially if threading is involved.

You could fork off a subprocess and implement the alarm as above, or you could just exec http://man7.org/linux/man-pages/man1/flock.1.html

import subprocess
def flock_with_timeout(fd, timeout, shared=True):
    rc = subprocess.call(['flock', '--shared' if shared else '--exclusive', '--timeout', str(timeout), str(fd)])
    if rc != 0:
        raise Exception('Failed to take lock')

If you have a new enough version of flock you can use -E to specify a different exit code for the command otherwise succeeding, but failed to take the lock after a timeout, so you can know whether the command failed for some other reason instead.

As a complement to @Richard Maw answer above https://stackoverflow.com/a/32580233/17454091 (Don't have enough reputation to post a comment).

In Python 3.2 and newer, for fds to be available in sub-processes one must also provide pass_fds argument.

Complete solution ends up as:

import subprocess
def flock_with_timeout(fd, timeout, shared=True):
    rc = subprocess.call(['flock',
                          '--shared' if shared else '--exclusive',
                          '--timeout', str(timeout),
                          str(fd)],
                         pass_fds=[fd])
    if rc != 0:
        raise Exception('Failed to take lock')