Mini-languages in Python

I'm after creating a simple mini-language parser in Python, programming close to the problem domain and all that.

Anyway, I was wondering how the people on here would go开发者_高级运维 around doing that - what are the preferred ways of doing this kind of thing in Python?

I'm not going to give specific details of what I'm after because at the moment I'm just investigating how easy this whole field is in Python.

Pyparsing is handy for writing "little languages". I gave a presentation at PyCon'06 on writing a simple adventure game engine, in which the language being parsed and interpreted was the game command set ("inventory", "take sword", "drop book", etc.). (Source code here.)

You can also find links to other pyparsing articles at the pyparsing wiki.

I have limited but positive experience with PLY (Python Lex-Yacc). It combines Lex and Yacc functionality in a single Python class. You may want to check it out.

Fellow Stackoverflow'er Ned Batchelder has a nice overview of available tools on his website. There's also an overview on the Python website itself.

I would recommend funcparserlib. It was written especially for parsing little languages and DSLs and it is faster and smaller than pyparsing (see stats on its homepage). Minimalists and functional programmers should like funcparserlib.

Edit: By the way, I'm the author of this library, so my opinion may be biased.

Python is such a wonderfully simple and extensible language that I'd suggest merely creating a comprehensive python module, and coding against that.

I see that while I typed up the above, PLY has already been mentioned.

If you ask me this now, I would try the textx library for python. You can very easily create a dsl in that with python! Advantages are that it creates an AST for you, and lexing and parsing is combined.

http://igordejanovic.net/textX/

In order to be productive, I'd always use a parser generator like CocoPy (Tutorial) to have your grammar transformed into a (correct) parser (unless you want to implement the parser manually for the sake of learning).

The rest is writing the actual interpreter/compiler (Create stack-based byte code or memory AST to be interpreted and then evaluate it).