Creating Anonymous Modules in Python

Edit: I've added some more info on why I'm looking to do this.

The example code below is purely for example purposes. In the end, I need to have my code walk over a directory structure, and when it finds a python file, import the file to grab data and possibly function definitions from it. That will then be used when determining what to do with the remaining files in the directory (which are HTML templates, etc.)

Because the id开发者_开发百科ea is to make the directory structure be more suited for holding HTML templates and images, and the python files to really are more of a data container (although every once and awhile there will be logic in terms of function), I don't want to have __init__.py files all over the place to make sure that the python modules are part of a package, and I feel that messing with the sys.path is the wrong solution (there might be "meta.py" files in multiple directories).

So, I feel loading a python file by it's source to be the best solution. The problem with this is what I've shown below.

Imagine that I have code that does the following:

main.py:

import imp

a = imp.load_source('blah', 'a.py')
print dir(a)
b = imp.load_source('blah', 'b.py')
print dir(b)

a.py:

a = 'a'

b.py:

b = 'b'

The results of running main.py are:

['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'a']
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'a', 'b']

What I was hoping to happen was that the second call to load_source creates a new module and returns it, but it looks like it actually will overwrite the existing one.

For now, my solution is to change the 'blah' string in the load_source function to be some name that none of these modules will share (such as their filename), but I was wondering if there was some better way.

Mark, as I understand you wonder, if imp.load_source() can import a module with a name "blah" overwriting the previous import. Well, you can't do that via imp.load_source(), and here is the reason:

Take a look at a part of Python's import.c source code:

PyObject *
PyImport_ExecCodeModuleEx(char *name, PyObject *co, char *pathname)
{
   PyObject *modules = PyImport_GetModuleDict();
   PyObject *m, *d, *v;

   m = PyImport_AddModule(name);
   if (m == NULL)
       return NULL;
   /* If the module is being reloaded, we get the old module back
      and re-use its dict to exec the new code. */ 
   d = PyModule_GetDict(m);
   if (PyDict_GetItemString(d, "__builtins__") == NULL) {
       if (PyDict_SetItemString(d, "__builtins__",
                                PyEval_GetBuiltins()) != 0)
           goto error;
   }
   ...
   ...

Have you noticed the comment about reloading the module? So, a clear and non-hackish way to do what you're asking for, is to use another name instead of "blah" while importing b.py.

Don't use the same name for distinct modules. You can use e.g. os.path.splitext to get the filename and use that, or just make up a name.

More specifically, the first argument to load_source is the name of the module, like os or sys. You are telling imp to import what it assumes is the same module, but from different files. It's slightly odd that it doesn't re-import it from scratch, but I guess it's a reloading optimisation? Notice that sys.modules will get populated with the new module as "blah", not as "a".

Are you sure you want Python files? Without knowing more about your templating engine I can't really say anything, but it might be simpler if you just had e.g. csv or ini-style files in the directory hierarchy for settings.