Behavior of PYTHONPATH vs PERL5LIB

So behaviour of python path is different from PERL5LIB which makes it a little difficult to partition libraries into different repos. Let me describe how it works in PERL:

If the directory structure is as follows:

project/lib/bar/foo.pm

common/lib/bar/baz.pm

now PERL5LIB is set to 'project/lib:common/lib'

in my perl script I can do this:

use bar::foo; # this comes from project/lib/bar/foo.pm
use bar::baz; # this comes from common/lib/bar/baz.pm

in Python though with the same directory structure (with .py files instead) and adding the same dirs to PYTHONPATH (and of course adding dummy __init__.py in the project/lib/bar and common/lib/bar dirs):

import bar.foo # this successfully imports from project/lib/bar/foo.py
import bar.baz # this fails!

Is there a way around this in Python, because this makes p开发者_运维百科artitioning code a nightmare.

EDIT: to be little bit more clear where the __init__.py files are.

Your first problem is that you're trying to force a Perl paradigm into Python. Your second problem is that you don't seem to understand how the Python namespace system works. You can't have the same package imported twice into the global namespace. It just doesn't work like that.

Python is going to use the first match it finds. Since it's finding the package bar when you import bar.foo from the first location, it's not even trying the latter one when you try to import bar.baz residing at the latter location.

When you import a module it is added to the global namespace, and also tracked in the sys.modules dictionary. Starting with a fresh Python interpreter, if you import sys you see that it is in the global namespace:

>>> import sys
>>> globals().keys()
['__builtins__', '__name__', 'sys', '__doc__', '__package__']

If you get it by key, you get the module object:

>>> globals()['sys']
<module 'sys' (built-in)>

Now if you import foo you see it too also ends up in the global namespace:

>>> import foo
>>> globals().keys()
['__builtins__', '__package__', 'sys', '__name__', 'foo', '__doc__']

And also in sys.modules:

>>> sys.modules['foo']
<module 'foo' from 'foo.pyc'>
>>> globals()['foo']
<module 'foo' from 'foo.pyc'>

Even if you delete foo from the global namespace, it still resides in sys.modules:

>>> del foo
>>> 'foo' in globals()
False
>>> 'foo' in sys.modules
True

Why is this? Think of sys.modules as a registry of modules that have been imported. This is done as an optimization, so that if you import multiple parts of the same package, parts that have already been loaded don't keep getting reloaded. The only way to truly "unload" a module is to delete it from sys.modules and from the global namespace.

I hope that by illustrating this, you also see that each package or module object can only reside ONCE in the namespace of any given program. This is why what you're trying to do is failing. Python already succesfully imported bar the first time, so it's not going to try to re-import it a second time.

If you really want two distinct paths with the same package name to reside at different locations in the filesystem, you should research Python's namespace packages. This will allow you to install packages in alternate locations that bind themselves to the namespace of another package.

See also:

• Python module and packaging tutorial
• A guide to Python namespaces