Loading each .py file in a path - imp.load_module complains about relative import

I am trying to parse a given path for python source files, import each file and DoStuff™ to each imported module.

def ParsePath(path):
    for root, dirs, files in os.walk(path):
        for source in (s for s in files if s.endswith(".py")):
            name = os.path.splitext(os.path.basename(source))[0]
            m = imp.load_module(name, *imp.find_module(na开发者_如何转开发me, [root]))
            DoStuff(m)

The above code works, but packages aren't recognized ValueError: Attempted relative import in non-package

My question is basically, how do I tell imp.load_module that a given module is part of a package?

You cannot directly tell Importer Protocol method load_module that the module given is part of the package. Taken from PEP 302 New Import Hooks

The built-in __import__ function (known as PyImport_ImportModuleEx in import.c) will then check to see whether the module doing the import is a package or a submodule of a package. If it is indeed a (submodule of a) package, it first tries to do the import relative to the package (the parent package for a submodule). For example if a package named "spam" does "import eggs", it will first look for a module named "spam.eggs". If that fails, the import continues as an absolute import: it will look for a module named "eggs". Dotted name imports work pretty much the same: if package "spam" does "import eggs.bacon" (and "spam.eggs" exists and is itself a package), "spam.eggs.bacon" is tried. If that fails "eggs.bacon" is tried. (There are more subtleties that are not described here, but these are not relevant for implementers of the Importer Protocol.)

Deeper down in the mechanism, a dotted name import is split up by its components. For "import spam.ham", first an "import spam" is done, and only when that succeeds is "ham" imported as a submodule of "spam".

The Importer Protocol operates at this level of individual imports. By the time an importer gets a request for "spam.ham", module "spam" has already been imported.

You must then simulate what the built-in import does and load parent packages before loading sub modules.

The function imp.find_module always takes a plain module name without dots, but the documentation of imp.load_module says

The name argument indicates the full module name (including the package name, if this is a submodule of a package).

So you could try this:

def ParsePath(path):
    for root, dirs, files in os.walk(path):
        for source in (s for s in files if s.endswith(".py")):
            name = os.path.splitext(os.path.basename(source))[0]
            full_name = os.path.splitext(source)[0].replace(os.path.sep, '.')
            m = imp.load_module(full_name, *imp.find_module(name, [root]))
            DoStuff(m)

I had the same problem. Good news is that there is a way of doing it, but you have to use a combination of imp and importlib. Here's an illustrative example:

import imp
import importlib
package_path = r"C:\path_to_package"

package_name = "module"
module_absolute_name = "module.sub_module"
module_relative_name = ".sub_module"

# Load the package first
package_info = imp.find_module(package_name, [package_path]) 
package_module = imp.load_module(package_name, *package_info)

# Try an absolute import
importlib.import_module(module_absolute_name, package_name)

# Try a relative import
importlib.import_module(module_relative_name, package_name)

This will allow sub_module to import using relative module paths because we've already loaded the parent package and the submodule has been loaded correctly by importlib to know what it's being imported relative to.

I believe this solution is only necessary for those of us stuck in Python 2.*, but would need someone to confirm that.