recursive grep using python

I am new to python开发者_JS百科 and trying to learn. I am trying to implement a simple recursive grep using python for processing and here is what I came to so far.

p = subprocess.Popen('find . -name [ch]', shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
  for line in p.stdout.readlines():
    q = subprocess.Popen('grep searchstring %s', line, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    print q.stdout.readlines()

Can some one pls tell me how to fix this to do what it is supposed to?

You should use the os.walk function for going through your files. Use string methods or regex for filtering out the results. Check http://docs.python.org/library/os.html for informations about how to use os.walk.

import os
import re

def findfiles(path, regex):
    regObj = re.compile(regex)
    res = []
    for root, dirs, fnames in os.walk(path):
        for fname in fnames:
            if regObj.match(fname):
                res.append(os.path.join(root, fname))
    return res

print findfiles('.', r'my?(reg|ex)')

Now for the grep part, you can loop over the file with the open function

def grep(filepath, regex):
    regObj = re.compile(regex)
    res = []
    with open(filepath) as f:
        for line in f:
            if regObj.match(line):
                res.append(line)
    return res

If you want to get the line numbers, you may want to look into the enumerate function.

edited to add the grep function

You can use python-textops3 :

Example, to grep all 'import' in all .py files from current directory :

from textops import *

print('\n'.join(('.' | find('*.py') | cat() | grep('import')))) 

It is pure python, no need to fork a process.

p = subprocess.Popen('find . -name [ch]', shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
  for line in p.stdout.readlines():
    q = subprocess.Popen('grep searchstring %s', line, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    print q.stdout.readlines()

1. The indentation on line 2 will Exception, the for needs to be aligned with the p above
2. 'grep searchstring %s', line will not do the string replacement, you need to replace the , with %

With those changes and real search values, it works on my OS X box. Final script:

import subprocess
p = subprocess.Popen('find . -name *.py', shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in p.stdout.readlines():
    print line
    q = subprocess.Popen('grep import %s' % line, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    print q.stdout.readlines()

Maybe an example can help you, the command find . -print | grep "python" is equivalent to this:

import subprocess

pc1 = subprocess.Popen('find . -print', stdout=subprocess.PIPE, shell=True)
pc2 = subprocess.Popen('grep "python"', stdin=pc1.stdout, shell=True,
                       stdout=subprocess.PIPE)

print pc2.communicate()[0]