Accessing resource files in Python unit tests & main code

I have a Python project with the following directory structure:

project/
project/src/
project/src/somecode.py
project/src/mypackage/mymodule.py
project/src/resources/
project/src/resources/datafile1.txt

In mymodule.py, I have a class (lets call it "MyClass") which needs to load datafile1.txt. This sort of works when I do:

open ("../resources/datafile1.txt")

Assuming the code that creates the MyClass instance created is run from somecode.py.

The gotcha however is that I have unit tests for mymodule.py which are defined in that 开发者_如何学编程file, and if I leave the relative pathname as described above, the unittest code blows up as now the code is being run from project/src/mypackage instead of project/src and the relative filepath doesn't resolve correctly.

Any suggestions for a best practice type approach to resolve this problem? If I move my testcases into project/src that clutters the main source folder with testcases.

I usually use this to get a relative path from my module. Never tried in a unittest tho.

import os

print(os.path.join(os.path.dirname(__file__), 
                   '..',
                   'resources' 
                   'datafile1.txt'))

Note: The .. tricks works pretty well, but if you change your directory structure you would need to update that part.

On top of the above answers, I'd like to add some Python 3 tricks to make your tests cleaner.

With the help of the pathlib library, you can explicit your ressources import in your tests. It even handles the separators difference between Unix (/) and Windows ().

Let's say we have a folder structure like this :

`-- tests
    |-- test_1.py <-- You are here !
    |-- test_2.py
    `-- images
        |-- fernando1.jpg <-- You want to import this image !
        `-- fernando2.jpg

You are in the test_1.py file, and you want to import fernando1.jpg. With the help to the pathlib library, you can read your test resource with an object oriented logic as follows :

from pathlib import Path

current_path = Path(os.path.dirname(os.path.realpath(__file__)))
image_path = current_path / "images" / "fernando1.jpg"

with image_path.open(mode='rb') as image :
    # do what you want with your image object

But there's actually convenience methods to make your code more explicit than mode='rb', as :

image_path.read_bytes() # Which reads bytes of an object

text_file_path.read_text() # Which returns you text file content as a string

And there you go !

in each directory that contains Python scripts, put a Python module that knows the path to the root of the hierarchy. It can define a single global variable with the relative path. Import this module in each script. Python searches the current directory first so it will always use the version of the module in the current directory, which will have the relative path to the root of the current directory. Then use this to find your other files. For example:

# rootpath.py
rootpath = "../../../"

# in your scripts
from rootpath import rootpath
datapath = os.path.join(rootpath, "src/resources/datafile1.txt")

If you don't want to put additional modules in each directory, you could use this approach:

Put a sentinel file in the top level of the directory structure, e.g. thisisthetop.txt. Have your Python script move up the directory hierarchy until it finds this file. Write all your pathnames relative to that directory.

Possibly some file you already have in the project directory can be used for this purpose (e.g. keep moving up until you find a src directory), or you can name the project directory in such a way to make it apparent.

You can access files in a package using importlib.resources (mind Python version compatibility of the individual functions, there are backports available as importlib_resources), as described here. Thus, if you put your resources folder into your mypackage, like

project/src/mypackage/__init__.py
project/src/mypackage/mymodule.py
project/src/mypackage/resources/
project/src/mypackage/resources/datafile1.txt

you can access your resource file in code without having to rely on inferring file locations of your scripts:

import importlib.resources

file_path = importlib.resources.files('mypackage').joinpath('resources/datafile1.txt')
with open(file_path) as f:
    do_something_with(f)

Note, if you distribute your package, don't forget to include the resources/ folder when creating the package.

The filepath will be relative to the script that you initially invoked. I would suggest that you pass the relative path in as an argument to MyClass. This way, you can have different paths depending on which script is invoking MyClass.