Python Shell Wrapper

I'm trying to write a simple Python version of the time command line program, except that instead of displaying the real/usr/sys times to the shell, it'll record it to a database.

What I currently have is:

wrapper.py

#!/usr/bin/python
import sys
from subprocess import Popen, PIPE
cmd = 'time ' + (' '.join(['"%s"' % v if ' ' in v else v for v in sys.argv[1:]]))
p = Popen(cmd, shell=True, st开发者_StackOverflow中文版din=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
print p.stdout.read()
print p.stderr.read()

For simplicity, I've excluded the database insert code.

However, to illustrate a problem, I'm using the test script:

#!/usr/bin/python
import time
for i in xrange(3):
    print i
    time.sleep(1)

If I run wrapper.py python delay.py, I'd like to see the seconds print out in real time, followed by something like:

real    0m3.057s
user    0m0.030s
sys 0m0.000s

Instead, I get no output for 3 seconds, and then this is printed:

0
1
2

0.02user 0.00system 0:03.03elapsed 0%CPU (0avgtext+0avgdata 21632maxresident)k
0inputs+0outputs (0major+1514minor)pagefaults 0swaps

How do you read and print output from a subprocess as it happens in real time?

Also, why does the output from time differ from when I run it directly in a shell and become jumbled when run from a subprocess inside a Python script?

First, why are you handling the I/O with python at all? Just let the sub process's stdout and stderr go to the same place as python's. Second, instead of actually using the time command, you can retrieve the resources directly from python. Try something like this:

#! /usr/bin/python

import os
import resource
import sys
import time

cmd = ' '.join(sys.argv[1:])
stime = time.time()
os.system(cmd)          # fire off the command
etime = time.time()

# now get the resource utilization
r = resource.getrusage(resource.RUSAGE_CHILDREN)
user_time = r.ru_utime
sys_time = r.ru_stime
# resource doesn't know "clock" time, so we'll just figure that ourselves
real_time = etime - stime

print "real\t" + str(real_time)
print "user\t" + str(user_time)
print "sys\t" + str(sys_time)

This prints the times in seconds. If you really want them to look just like the time command, you can format them accordingly.

To answer the second part of your question, there are actually different "time" commands. When you are running it as a child of python, you're getting the output of /usr/bin/time. When you run it by hand, you're getting the shell's built-in version of time. Try typing "type -a time" at the shell prompt. Also, try running your test program like this: "/usr/bin/time ./test.py" and you should see the second form.

You need to write the output in realtime:

process = subprocess.Popen(shlex.split(command), stdout = subprocess.PIPE, stderr = subprocess.STDOUT)

while True:
  output = process.stdout.read(1)

  if output == '' and process.poll() != None:
    break

  if output != '':
    sys.stdout.write(output)
    sys.stdout.flush()