Needing Help Capturing STDOUT in real-time from pygtk GUI in python project

I am working on a project in which I want to use the command line utility cdparanoia from a python pygtk GUI. I'm using Glade for UI development. I have tried importing subprocess and using subprocess.Popen. It works, but it freezes my GUI (won't even allow repainting of the windows) while the process is executing. Not a very nice interaction for the user. How can I prevent this behaviour? I would like put a cancel button on the window but this would work as it "free开发者_StackOverflow社区zes" the program. Ultimately, I would like to capture stderr (as below, audio info is piped to sox via stdout) and present it in as a gtk.Expander with a similar look to Synaptic when it is installing a program with the ability of the user to see things happening in real time. Also, I would like to use the text from the progress indicator (as seen below) to build a real progress indicator widget. How can I get a shell to pass info back to python in real-time rather than once the process is finished (when it gives it all as one big info dump)?

Real-time info needing captured:

Working on me - me - DISK 01.flac
cdparanoia III release 10.2 (September 11, 2008)

Ripping from sector       0 (track  1 [0:00.00])
      to sector  325195 (track 15 [1:56.70])

outputting to stdout

 (== PROGRESS == [>                             | 004727 00 ] == :-) O ==)

Here is the code I've used so far:

        quick = " -Z" if self.quick == True else ""
        command = "cdparanoia -w%s 1- -| sox -t wav - \"%s - %s - DISK %s%s.flac\"" %\
                    (
                        quick,
                        self.book_name.replace(" ", "_"),
                        self.author_name.replace(" ", "_"),
                        "0" if disc < 10 else "", 
                        disc
                    )
        print command
        shell = subprocess.Popen(command, shell=True, executable="/bin/bash",
                                    stdin=subprocess.PIPE,
                                    stdout=subprocess.PIPE,
                                    stderr=subprocess.PIPE
                                )
        data, err = shell.communicate(command)

With Thanks, Narnie

I wrote a Python shell implementation once, and it did run wget and the actual Python console with fully functional output.

You need to use subprocess.Popen and write directly to sys.stdout:

process = subprocess.Popen(shlex.split(command), stdout = subprocess.PIPE, stderr = subprocess.STDOUT)
complete = False

while True:
  output = process.stdout.read(1)

  if output == '' and process.poll() != None:
    break

  if output != '':
    sys.stdout.write(output)
    sys.stdout.flush()

If you write a GUI program which reads from a file handle you have two use a dispatcher to integrate the file descriptor events into the GUI event loop. A general description of event loops can be found at Wikipedia. The specific description for Gtk+ can be found in the reference.

Solution for your problem: use the function g_io_add_watch to integrate your action into the main event loop. Here is an example in C. Python should be analogous.

Yes there are 2 issues here. The first is that you'll need to specify a read timeout, so that you don't block until the sub process is finished. The second is that there is probably buffering happening that is not desirable.

To address the first issue, and read from the sub process asynchronously you could try my subProcess module, with a timeout: http://www.pixelbeat.org/libs/subProcess.py Note this is simple but also old and linux only. It was used as a basis for the new python subprocess module, so you'd be better to go with that if you need portability.

To understand/control any additional buffering which may happen, see: http://www.pixelbeat.org/programming/stdio_buffering/