Why does my socket.makefile object block even after a close?

If I use socket.makefile and then close the file object as well as the underlying socket, then subsequent calls to read will throw an exception, just as I'd want it to. For example, the following code works as I'd expect:

import socket
from time import sleep
from threading import Thread

ADDR = ("localhost", 4321)

def listener(sock):
    client,addr = sock.accept()
    sleep(1)
    client.close()
    sock.close()

server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_sock.bind(ADDR)
server_sock.listen(1)
Thread(target=listener, args=[server_sock]).start()

sock = socket.create_connection(ADDR)
f = sock.makefile("r+b", bufsize=0)

f.close()
sock.close()
f.read(8)    # throws an exception, as I'd expect

However, if I ma开发者_Go百科ke a call to read while the file/socket are still open then that call will block, and then if I close the socket, the read method still doesn't return. In fact, it hangs indefinitely until the socket is closed on the other end. The following code demonstrates this distressing behavior:

import socket
from time import sleep
from threading import Thread

ADDR = ("localhost", 4321)

def reader(f):
    print("about to read")
    print("we read %r" % f.read(8))
    print("finished reading")

def listener(sock):
    client, addr = sock.accept()
    sleep(3)
    client.close()
    sock.close()

server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_sock.bind(ADDR)
server_sock.listen(1)
Thread(target=listener, args=[server_sock]).start()

sock = socket.create_connection(ADDR)
f = sock.makefile("r+b", bufsize=0)
Thread(target=reader, args=[f]).start()

sleep(1)
print("closing pseudo-file and socket")
f.close()
sock.close()
sleep(1)
print("we still haven't finished reading!")

This is a serious problem for me, since I'd like to make a blocking call to f.read in a thread, but then still be able to close the socket and have the thread return from that call (possibly by throwing an exception) and exit. However, all that happens is that the call blocks forever so long as the other side never closes the socket.

So is there any way for Thread1 to call read on the file-like object created by socket.makefile and then have Thread2 shut down the socket in a way that causes Thread1 to stop blocking on its read call?

EDIT: I tried re-writing my program to entirely use gevent and its socket and Greenlet approach to multithreading, but my program still does exactly the same thing:

from gevent import sleep, socket, spawn

ADDR = ("localhost", 4321)

def reader(f):
    print("about to read")
    print("we read %r" % f.read(8))
    print("finished reading")

def listener(sock):
    client, addr = sock.accept()
    sleep(3)
    client.close()
    sock.close()

server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_sock.bind(ADDR)
server_sock.listen(1)
spawn(listener, server_sock)

sock = socket.create_connection(ADDR)
f = sock.makefile("r+b", bufsize=0)
spawn(reader, f)

sleep(1)
print("closing pseudo-file and socket")
f.close()
sock.close()

sleep(1)
print("we still haven't finished reading!")
sleep(2)

I was surprised to learn that even using gevent sockets, calls to read will block even after the underlying socket is closed. If there's no way to prevent this, I'll probably just have to accept Thomas' depressing "this is not possible" answer :(

In situations like this, I've had success using eventlet and nowadays, gevent is doing the same types of things:

'monkey patching' the socket library to use non blocking I/O. Here's an example of what you could try:

>>> from gevent import monkey; monkey.patch_socket()
>>> import socket

and see how that'd affect your results

Yes, threads have distressing behaviour, especially when touching things like sockets from multiple threads at the same time. This is not something you can make work right. There is no way to force a thread that's blocked in a read to break out of the read or terminate -- closing the socket from under it is more likely to cause your entire program to segfault (or worse) than it is to do what you want.

The right way to deal with situations like this is to use non-blocking reads, instead. A decent event framework, like Twisted, can help you with this.