Python - pipelining subprocess in Windows

I'm using Windows 7, and I've tried this under Python 2.6.6 and Python 3.2.

So I'm trying to call this command line from Python:

netstat -ano | find ":80"

under Windows cmd, this line works perfectly fine.

So,

• 1st attempt:

  output = subprocess.Popen(
             [r'netstat -ano | find ":80"'],
             stdout=subprocess.PIPE,
             shell开发者_如何学C=True
  ).communicate()
  

  An error is raised that 'find' actually didn't receive correct parameter (e.g. 'find ":80" \'):

  Access denied - \
  

• 2nd attempt:

  #calling netstat
  cmd_netstat = subprocess.Popen(
                  ['netstat','-ano'],
                  stdout = subprocess.PIPE
  )
  
  #pipelining netstat result into find
  cmd_find = subprocess.Popen(
               ['find','":80"'],
               stdin = cmd_netstat.stdout,
               stdout = subprocess.PIPE
  )
  

  Again, the same error is raised.

  Access denied - \
  

What did I do wrong? :(

EDIT:

• 3rd attempt (As @Pavel Repin suggested):

  cmd_netstat = subprocess.Popen(
                  ['cmd.exe', '-c', 'netstat -ano | find ":80"'],
                  stdout=subprocess.PIPE
  ).communicate()
  

  Unfortunately, subprocess with ['cmd.exe','-c'] results in something resembling deadlock or a blank cmd window. I assume '-c' is ignored by cmd, resulting in communicate() waiting indefinitely for cmd termination. Since this is Windows, my bet bet is cmd only accepts parameter starting with slash (/). So I substituted '-c' with '/c':

  cmd_netstat = subprocess.Popen(
                  ['cmd.exe', '/c', 'netstat -ano | find ":80"'],
                  stdout=subprocess.PIPE
  ).communicate()
  

  And...back to the same error:

  Access denied - \
  

EDIT: I gave up, I'll just process the string returned by 'netstat -ano' in Python. Might this be a bug?

What I suggest is that you do the maximum inside Python code. So, you can execute the following command:

# executing the command
import subprocess
output = subprocess.Popen(['netstat', '-ano'], stdout=subprocess.PIPE).communicate()

and then by parsing the output:

# filtering the output
valid_lines = [ line for line in output[0].split('\r\n') if ':80' in line ]

You will get a list of lines. On my computer, the output looks like this for port number 1900 (no html connexion active):

['  UDP    127.0.0.1:1900         *:*                                    1388', '  UDP    192.xxx.xxx.233:1900    *:*                                    1388']

In my opinion, this is easier to work with.

Note that :

• option shell=True is not mandatory, but a command-line window is opened-closed quickly. See what suits you the most, but take care of command injection;
• list of Popen arguments shall be a list of string. Quoting of the list parts is not necessary, subprocess will take care of it for you.

Hope this helps.

EDIT: oops, I missed the last line of the edit. Seems you've already got the idea on your own.

So I revisited this question, and found two solutions (I switched to Python 2.7 sometime ago, so I'm not sure about Python 2.6, but it should be the same.):

1. Replace find with findstr, and remove doublequotes

   output = subprocess.Popen(['netstat','-ano','|','findstr',':80'],
                             stdout=subprocess.PIPE,
                             shell=True)
                      .communicate()
   

   But this doesn't explain why "find" cannot be used, so:

2. Use string parameter instead of list

   output = subprocess.Popen('netstat -ano | find ":80"',
                             stdout=subprocess.PIPE,
                             shell=True)
                      .communicate()
   

   or

   pipeout = subprocess.Popen(['netstat', '-ano'], 
                              stdout = subprocess.PIPE)
   output = subprocess.Popen('find ":80"', 
                             stdin = pipeout.stdout, 
                             stdout = subprocess.PIPE)
                      .communicate()
   

The problem arise from the fact that: ['find','":80"'] is actually translated into ['find,'\":80\"']. Thus the following command is executed in Windows command shell:

>find \":80\"
Access denied - \

Proof:

• Running:

  output = subprocess.Popen(['echo','find','":80"'],
                            stdout=subprocess.PIPE,
                            shell=True)
                     .communicate()
  print output[0]
  

  returns:

  find \":80\"
  

• Running:

  output = subprocess.Popen('echo find ":80"',
                            stdout=subprocess.PIPE,
                            shell=True)
                     .communicate()
  print output[0]
  

  returns:

  find ":80"
  

New answer, after reading this old question again: this may be due to the two following facts:

• The pipe operator executes the following commands in a sub-shell; see for instance this interesting consequence).

• Python itself uses the pipe as a way to get the results back:

  Note that (...) to get anything other than None in the result tuple, you need to give stdout=PIPE and/or stderr=PIPE too.

Not sure if this 'conflict' is kind of a bug, or a design choice though.