How can I read output from another program? [duplicate]

This question already has answers here: read subprocess stdout line by line (10 answers) Closed last month.

How can I receive input from the terminal in Python?

I am using Python to interface with another program which generates output from user input. I am using subprocess.Popen() to input to the program, but I can't set stdout to subprocess.PIPE because the program does not seem to flush ever, so everything gets stuck in the buffer. The program's standard output seems to be to print to terminal, and I see开发者_开发技巧 output when I do not redirect stdout. However, I need Python to read and interpret the output which is now in the terminal.

Sorry if this is a stupid question, but I can't seem to get this to work.

---

Buffering in child processes is a common problem. Here are four possible approaches.

First, and easiest, you could read one byte at a time from your pipe. This is what I would call a "dirty hack" and it carries a performance penalty, but it's easy and it guarantees that your read() calls will only block until the first byte comes in, rather than wait for a buffer to fill up that's never going to fill up. However, this does not force the other process to flush its write buffer, so if that is the issue this approach will not help you anyway.

Second, and I think next-easiest, consider using the Twisted framework which has a facility for using a virtual terminal, or pty ("pseudo-teletype" I think) to talk to your child process. However, this can affect the design of your application (possibly for the better, but this may not be in the cards for you regardless). http://twistedmatrix.com/documents/current/core/howto/process.html

If neither of the above options works for you, you're reduced to solving gritty I/O concurrency issues yourself.

Third, try setting your pipes (all of them, before fork()) to non-blocking mode using fcntl() with O_NONBLOCK. Then you can use select() to test for read/write readiness before trying the read/write; but you still have to catch IOError and test for EAGAIN because it can happen even in this case. This may, depending on the behavior of the child process, allow you to wait until the data really shows up before trying to read it in.

The last resort is to implement the PTY logic yourself. If you've seen references to stuff like termio options, ioctl() calls, etc. then that's what you're up against. I have not done this before, because it's complicated and I have never really needed to. If this is your destiny, good luck.

---

Have you tried setting the bufsize in your Popen object to 0? I'm not sure if you can force the buffer to be unbuffered from the receiving size, but I'd try it.

http://docs.python.org/library/subprocess.html#using-the-subprocess-module