Python: subprocess.stdin.write is not working properly

I'm new to python but I can code and debug a bit. The following problem has been on my head for the past couple of days and looking out for an answer. Any help is much appreciated. Task: I wanted to do an interactive telnet (I know there is an telnet library, but we are not using it for various reasons). For this purpose, I use a subprocess.popen

p = subprocess.Popen(telnet_command,
                         stdin = subprocess.PIPE,
                         stdout = outputfileobj,
                         stderr = errorfileobj)

I do a poll() to see if the server has got connected to the session. Once I verified, I proceed to write to the stdin for interactive communication.

inputTxt = 'GET / HTTP/1.1\nHost: ' + hostheader + '\n\n'
p.stdin.write(inputTxt)
p.stdin.flush()

Here is where the problem has occurred. I get the http response (or atleast output) for 5/6 tim开发者_如何学运维es but 1/6 time, I do not get a output and the subprocess gets terminated - which is not possible.

I ran a system trace for the failure case and please find below the same.

16129 write(7, "GET / HTTP/1.1\nHost: www.google."..., 37) = 37
16129 gettimeofday({1310538497, 134474}, NULL) = 0
16129 futex(0x81993a8, FUTEX_WAKE, 1)   = 0
16129 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=56, ...}) = 0
16129 fstat64(4, {st_mode=S_IFREG|0644, st_size=520689, ...}) = 0
16129 _llseek(4, 520689, [520689], SEEK_SET) = 0
16129 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=56, ...}) = 0
16129 write(4, "2011-07-13 06:28:17,134 : pconns"..., 170) = 170
16129 futex(0x81d3b30, FUTEX_WAKE, 1)   = 0
16129 waitpid(16198, 0xffa945e8, WNOHANG) = 0
16129 gettimeofday({1310538497, 135160}, NULL) = 0
16129 fstat64(6, {st_mode=S_IFREG|0644, st_size=81, ...}) = 0
16129 fstat64(6, {st_mode=S_IFREG|0644, st_size=81, ...}) = 0
16129 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf74b9000
16129 _llseek(6, 0, [0], SEEK_SET)      = 0
16129 fstat64(6, {st_mode=S_IFREG|0644, st_size=81, ...}) = 0
16129 _llseek(6, 0, [0], SEEK_CUR)      = 0
16129 read(6, "Trying 74.125.236.48...\nConnecte"..., 4096) = 81
16129 read(6, "", 4096)                 = 0
16129 close(6)                          = 0
16129 munmap(0xf74b9000, 4096)          = 0
16129 gettimeofday({1310538497, 135778}, NULL) = 0
16129 futex(0x81993a8, FUTEX_WAKE, 1)   = 0
16198 <... select resumed> )            = 1 (in [0])
16198 read(0, "GET / HTTP/1.1\nHost: www.google."..., 8191) = 37
16129 stat64("/etc/localtime",  <unfinished ...>
16198 ioctl(1, TCFLSH, 0)               = -1 ENOTTY (Inappropriate ioctl for device)
16129 <... stat64 resumed> {st_mode=S_IFREG|0644, st_size=56, ...}) = 0
16198 select(4, [0 3], [3], [3], {0, 0}) = 1 (out [3], left {0, 0})
16129 fstat64(4, {st_mode=S_IFREG|0644, st_size=520859, ...}) = 0
16198 send(3, "GET / HTTP/1.1\r\nHos\377\363\377\375\6", 24, 0 <unfinished ...>
16129 _llseek(4, 520859,  <unfinished ...>
16198 <... send resumed> )              = 24
16198 select(4, [0 3], [3], [3], {0, 0} <unfinished ...>
16129 <... _llseek resumed> [520859], SEEK_SET) = 0
16198 <... select resumed> )            = 1 (out [3], left {0, 0})
16198 send(3, ": www.google.com\r\n\r\n", 20, 0 <unfinished ...>
16129 stat64("/etc/localtime",  <unfinished ...>
16198 <... send resumed> )              = 20

If you closely look at the line 16198 send(3, "GET / HTTP/1.1\r\nHos\377\363\377\375\6", 24, 0 in the trace, the string "Host" is replaced by some "Hos\377\363\377\375\6". I'm not sure why this occurs once in a while and also this closes the telnet connection I established. Please let me know if you need more data.

What if you try to use the communicate() method to send data to the subprocess instead of using stdin, as the latter is discouraged by Python's documentation:

Warning: Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

Try this:

p.communicate(input='GET / HTTP/1.1\nHost: ' + hostheader + '\n\n')

Though communicate is better, if you really have to use stdin.write you can make the calls non-blocking using:

fcntl.fcntl(p.stdout.fileno(), fcntl.F_SETFL, os.O_NONBLOCK)

for better understanding read this:

io non blocking

If there is no need to interact, I don't use communicate, but put the text into a pipe ahead of time. I've not found a more pythonic "text_to_stream" yet.

import os
import sys
import subprocess


def text_to_stream(text):
    p = os.pipe()
    os.write(p[1], text.encode('utf-8'))
    os.close(p[1])
    return os.fdopen(p[0], "r")


def run(cmd, stdin_text=None):
    stdin = text_to_stream(stdin_text) if stdin_text else sys.stdin
    result = subprocess.run(
        cmd, stdout=sys.stdout, stderr=sys.stderr, stdin=stdin)
    result.check_returncode()