twisted server, nc client

Ill demonstrate the problem I am facing with a small example.

class TestProtocol(basic.LineReceiver):
    def lineReceived(self, line):
        print line

Everything works fine as long as I use the telnet client to connect to the server. However, the lin开发者_高级运维e is not received connect and send the data using netcat. I have a feeling that this has something to do with the default delimiter being "\r\n" in twisted.

How could I make a server such that both the clients(telnet and nc) would behave in a similar manner when connecting to the client?

LineReceiver only supports one delimiter. You can specify it, but there can only be one at a time. In general, if you want to support multiple delimiters, you'll need to implement a new protocol that supports that. You could take a look at the implementation of LineReceiver for some ideas about how a line-based protocol is implemented.

netcat sends whatever you type, so the delimiter is often \n (but it may vary from platform to platform and terminal emulator to terminal emulator). For the special case of \n, which is a substring of the default LineReceiver delimiter \r\n, there's another trick you can use. Set the TestProtocol.delimiter to "\n" and then strip the "\r" off the end of the line passed to lineReceived if there is one.

class TestProtocol(basic.LineReceiver):
    delimiter = "\n"

    def lineReceived(self, line):
        print line.rstrip("\r")

Another workaround is to use nc with the -C switch.

From the manual:

-C Send CRLF as line-ending

or as @CraigMcQueen suggested:

socket with -c switch (Ubuntu package).

Twisted's LineReceiver and LineOnlyReceiver only support one line ending delimiter.

Here is code for UniversalLineReceiver and UniversalLineOnlyReceiver, which override the dataReceived() method with support for universal line endings (any combination of CR+LF, CR or LF). The line breaks are detected with the regular expression object delimiter_re.

Note, they override functions with more code in them than I'd like, so there's a chance that they may break if the underlying Twisted implementation changes. I've tested they work with Twisted 13.2.0. The essential change is the use of delimiter_re.split() from the re module.

# Standard Python packages
import re

# Twisted framework
from twisted.protocols.basic import LineReceiver, LineOnlyReceiver

class UniversalLineReceiver(LineReceiver):
    delimiter_re = re.compile(br"\r\n|\r|\n")
    def dataReceived(self, data):
        """
        Protocol.dataReceived.
        Translates bytes into lines, and calls lineReceived (or
        rawDataReceived, depending on mode.)
        """
        if self._busyReceiving:
            self._buffer += data
            return

        try:
            self._busyReceiving = True
            self._buffer += data
            while self._buffer and not self.paused:
                if self.line_mode:
                    try:
                        line, remainder = self.delimiter_re.split(self._buffer, 1)
                    except ValueError:
                        if len(self._buffer) > self.MAX_LENGTH:
                            line, self._buffer = self._buffer, b''
                            return self.lineLengthExceeded(line)
                        return
                    else:
                        lineLength = len(line)
                        if lineLength > self.MAX_LENGTH:
                            exceeded = self._buffer
                            self._buffer = b''
                            return self.lineLengthExceeded(exceeded)
                        self._buffer = remainder
                        why = self.lineReceived(line)
                        if (why or self.transport and
                            self.transport.disconnecting):
                            return why
                else:
                    data = self._buffer
                    self._buffer = b''
                    why = self.rawDataReceived(data)
                    if why:
                        return why
        finally:
            self._busyReceiving = False

class UniversalLineOnlyReceiver(LineOnlyReceiver):
    delimiter_re = re.compile(br"\r\n|\r|\n")
    def dataReceived(self, data):
        """
        Translates bytes into lines, and calls lineReceived.
        """
        lines = self.delimiter_re.split(self._buffer+data)
        self._buffer = lines.pop(-1)
        for line in lines:
            if self.transport.disconnecting:
                # this is necessary because the transport may be told to lose
                # the connection by a line within a larger packet, and it is
                # important to disregard all the lines in that packet following
                # the one that told it to close.
                return
            if len(line) > self.MAX_LENGTH:
                return self.lineLengthExceeded(line)
            else:
                self.lineReceived(line)
        if len(self._buffer) > self.MAX_LENGTH:
            return self.lineLengthExceeded(self._buffer)