templated configuration in python or perl?

I'm trying to implement a templated configuration file. I'd prefer python, but I'd take an answer in perl too. I've used perl for my example.

I've searched a bit and found - python single configuration file - ConfigObj - python configuration file generator - ePerl but I could not from those solve my problem.

I'm trying generate a configuration file mostly in the INI format (with not even sections):

# Comments
VAR1 = value1
EDITOR = vi

and I need that generated from a template where I'm embedding a scripting language inside the text:

# Config:
MYPWD = <:   `pwd`  :>

The text in between the '<:' and ':>' would be in the scripting language (python or perl). As with a template, its stdout is captured and inserted in the resulting text. The templating used in the example is basically eperl, but I'd prefer python if available.

and finally, the defined variables should be reusable:

# Config:
CODE_HOME = /some/path
CODE_BIN = <:=$CODE_HOME:>/bin

Here's the test source file that I read in:

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# platform.cfg
# This one variable
VAR =value
# this is a templated variable. The langage is perl, but could be python.
HELLO= <: print 'World' :>
# This is a multi-line code which should resolve to a single line value.
LONGER = <:
 if (1) {
    print "abc ";
 }
 $pwd = `/bin/pwd`;
 chomp($pwd);
 print $pwd;
:>
# Another one开发者_运维知识库 to test the carriage returns.
MULTIPLE = /<: print "proj" :>/<: print "tahiti":>/<: 
print "pd/1/";
$system = `grep -w VAR platform.cfg | egrep -v 'print|platform.cfg' | cut -d = -f 2-`;
chomp($system);
print $system;
:>
# variables dependent from the previous variable definition
VAR1 = <: print $VAR :>1
# variables dependent from the previous variable definition
VAR2 = <: print $VAR1 :>2
# variables dependent from the previous variable definition
VAR3 = <: print $VAR2 :>3
# variables dependent from the previous variable definition
VAR4 = <: print $VAR3 :>4
# BTW, multi-line comments are significant
# and should be preserved as the documentation for the
# variable just below:
VAR5 = <: print $VAR4 :>5
VAR6 = <: print $VAR5 :>6
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And I'm looking to get this result out of the script. I could not figure how to have the variables defined in the config file be part of the interpreter?

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# platform.cfg
# This one variable
VAR =value
# this is a templated variable. The langage is perl, but could be python.
HELLO= World
# This is a multi-line code which should resolve to a single line value.
LONGER = abc /src/byop/CODE
# Another one to test the carriage returns.
MULTIPLE = /proj/tahiti/pd/1/value
# variables dependent from the previous variable definition
VAR1 = value1
# variables dependent from the previous variable definition
VAR2 = value12
# variables dependent from the previous variable definition
VAR3 = value123
# variables dependent from the previous variable definition
VAR4 = value1234
# BTW, multi-line comments are significant
# and should be preserved as the documentation for the
# variable just below:
VAR5 = value12345
VAR6 = value123456
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Thanks for your suggestions.

I developed Template::MasonLite for exactly this purpose. I was generating a whole lot of config files (Apache configs in my case) and I needed the script to run on multiple hosts without extra dependencies. So I developed a cut-down version of HTML::Mason that could be embedded directly in my script (an extra ~70 lines). The Mason templating syntax uses Perl for any logic (conditionals, loops etc) rather than making up a whole new language.

The Template::MasonLite implementation is not on CPAN (only at the link above) because I didn't want to clutter CPAN with yet another templating engine and also it wasn't clear what name the module should have.

At the time I developed it I was using cfengine which lacked any sane form of templating. I've since migrated to puppet which includes a templating language. I still use Template::MasonLite for generating presentation slides.

If you don't mind using a different syntax there are several template libraries you could use, mako is similar in spirit, Jinaj2 is also pretty nice. Go with a tried and tested library! If you really want to implement your own template library this might give you a start:

import re, StringIO, sys

def exec_block(block, variables):
    """Captures output of exec'd code block"""
    code = compile(block.strip(), '<string>', 'exec')
    _stdout, result = sys.stdout, StringIO.StringIO()
    sys.stdout = sys.__stdout__ = result
    exec(code, variables)
    sys.stdout = sys.__stdout__ = _stdout
    return result.getvalue()

def format_template(template):
    """Replaces code blocks with {0} for string formating later"""
    def sub_blocks(matchobj):
        """re.sub function, adds match to blocks and replaces with {0}"""
        blocks.append(matchobj.group(0)[2:-2].strip())
        return '{0}'

    blocks = []
    template = re.sub(r'<:.+?:>', sub_blocks, template, flags=re.DOTALL).splitlines()
    blocks.reverse()
    return blocks, template

def render_template(blocks, template):
    """renders template, execs each code block and stores variables as we go"""
    composed, variables = [], {}
    for line in template:
        if '{0}' in line:
            replacement = exec_block(blocks.pop(), variables).strip()
            line = line.format(replacement)
        if not line.startswith('#') and '=' in line:
            k, v = [x.strip() for x in line.split('=')]
            variables[k] = v
        composed.append(line)
    return '\n'.join(composed)

if __name__ == '__main__':
    import sys
    with open(sys.argv[1]) as f:
        blocks, template = format_template(f.read())
        print rend_template(blocks, template)

Which basically works like the above, except uses Python for the code blocks. Only supports one block per assignment, which actually seems like the best approach to me. You could feed it a configuration file like:

VAR = value
LONGER = <:
    print 'something'
:>
VAR1 = <: print VAR :>1
# comment
VAR2 = <: print VAR1 :>2
VAR3 = <: print VAR2 :>3
VAR4 = <: print VAR3 :>4

And it would exec each block render the variables out for you:

VAR = value
LONGER = something
VAR1 = value1
# comment
VAR2 = value12
VAR3 = value123
VAR4 = value1234