Python Socket Flush

I am trying to make sure that every 开发者_JAVA技巧time I call the socket.send function my buffer is sent (flushed) to my server (which is in C using unix socket).

From my understanding (and from what I see on this board) just disabling the naggle algo. should do it but my server still receive my data in chunk of 4096 bytes (default set)...

Im using the following code in Python v2.5.4:

 self.sck = socket( AF_INET, SOCK_STREAM )

 self.sck.setsockopt( IPPROTO_TCP, TCP_NODELAY, 1 ) # That doesn't seems to work...

 self.sck.connect( ( "127.0.0.1", "12345" ) )

 while( 1 ):
      self.sck.send( "test\n" )
      self.sck.send( "" ) # Still trying to flush...

Enabling/Disabling TCP_NODELAY seems that have no effect whatsoever... Is this a bug or I am missing something?

TIA

TCP does not provide any kind of guaranteed "packet" sending to the other end. You are sending data as fast as you can, and TCP is helpfully batching up the data into as much as it can send at once. Your server is receiving data 4096 bytes at a time, probably because that's what it asked for (in a recv() call).

TCP is a stream protocol and therefore you will have to implement some kind of framing yourself. There are no built-in message boundaries.

The only way I got a flush to work (back to a C client) was to use:

my_writer_obj = mysock.makefile(mode='w', ...)
my_writer_obj.write('my stuff')
my_writer_obj.flush()

The big advantage is that my_writer_obj defaults to text mode, so no more byte translation.

creating a reader object did not go as smoothly, but I did not need a flush on that side.

There is no way to ensure the size of the data chunks that are sent. If you want to make sure that all the data that you want to send is send, you can close the connection:

self.sck.close()

Note also, that n = socket.send() returns the number of actual sent bytes. If you definitely want to send all data, you should use

self.sck.sendall()

or loop over the data sending:

while data:
    n = self.sck.send(data)    
    data = data[n:]

(But that is roughly the same as sendall() ). If you want to receive the data in bigger chunks, you can increase the size of the buffer in recv(), but this only makes the possible chunk size bigger. There is no guaranty that the data arrives in these sizes.

This is a common question about TCP protocol. TCP itself has no way to send data in specific chunks. It's designed only for sending stream of data. If you need such functionality, you should implement it yourself. For example, send your chunks in separate lines or first send chunk size and then the chunk itself.

In most cases, you don't need to care about the Naggle algorithm. This algorithm is better described by the name TCP_NODELAY. If you disable it, you may achieve smaller delays for small chunks but lower speed for large chunks at the same time.

In Linux, this is possible with a call to ioctl:

SIOCOUTQ
Returns the amount of unsent data in the socket send queue. The > socket must not be in LISTEN state, otherwise an error (EINVAL) is returned. SIOCOUTQ is defined in <linux/sockios.h>. Alternatively, you can use > the synonymous TIOCOUTQ, defined in <sys/ioctl.h>.

Quick look to linux/sockios.h:

/* Linux-specific socket ioctls */
#define SIOCINQ         FIONREAD
#define SIOCOUTQ        TIOCOUTQ        /* output queue size (not sent + not acked) */

So, applying this is fairly easy:

import socket

def flush_socket(sock: socket.socket) -> bool:
    from ctypes import c_ulong
    from time import sleep
    from termios import TIOCOUTQ
    from fcntl import ioctl

    while sock.fileno() != -1: # if fd is -1, then it has been probably close()'d
        remaining = c_ulong.from_buffer_copy(
            ioctl(sock.fileno(), TIOCOUTQ, bytearray(8), False)).value

        if remaining == 0:
            # all data has been sent and ACKed
            return True

        # wait a bit before retrying,
        # sleep(0) was meant like yield current thread,
        # but will probably be close to busy-waiting,
        # feel free to change it to fit your needs
        sleep(0)

    # not all data has been sent
    return False