Parsing Python Code From Within Python? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. Closed 9 years ago.

We have an older C++ tool that generates some python code automatically. I tried to slog through the C++ source tool, today and pretty much wanted to shoot my self. The thing is what i want to do, is clean up the source created by the tool and link the classes to our开发者_如何学编程 internal documentation system via adding sphinx tags.

Now what i am wondering is there some sort of wonderful tool for parsing python code within python?

There is alot of stuff like this:

foo._methods_.append()

Snip 500 lines

foo._methods_.append()

ANy suggestions?

Basically i have a functional but insanely messy code structure, i basically want to extract certain chunks, move them to their own files. And cleanup all the miscellanous things that get created.

I looked quickyl at both parser and AST but i cannot find any real examples of it being used.

---

You may tokenize python code to parse individual tokens using tokenize module. e.g. Script to remove Python comments/docstrings

or you can use the parser module

or use ast module

---

Depending on your needs, you may also want to check out the 2to3 library. It was written to automatically facilitate the conversion of Python 2.x apps to Python 3.0, so its main use case is taking one Python source file, performing some transformations on it, and then spitting out the result source file.

One benefit that lib2to3 has over the ast module is that ast does not preserve whitespace and comments, whereas lib2to3 does. If you're already dealing with autogenerated code this might not be a problem for you.

---

Try one of the Python doc-generating utilities to see if this might help you with your overall problem. I've used epydoc with great success.

Or if you can get your boss to part with $200, buy a copy of Enterprise Architect from SparxSystems. It will reverse-engineer your Python code, generate a class diagram, and allow you to click on the class diagram and see the underlying code. Works for many languages besides Python, and an excellent design and documentation utility. (There is a $99 version, but this does not include the code import capability.)