Are Python 2.5 .pyc files compatible with Python 2.6 .pyc files?

A while ago I had to upgrade some servers from Python 2.4 to Python 2.5. I found that .pyc files created under Python 2.4 would crash whe开发者_如何学运维n Python 2.5 tried to run them.

Will this happen again when I upgrade from 2.5 to 2.6?

EDIT: Here is a bit more detail

I have a fileserver that contains the python code. This is accessed by both Ubuntu and Windows servers to run the python code. When they run the code they produce .pyc files on the fileserver.

I found that when I upgraded one of the server machines from Python 2.4 to 2.5 I had problems with .pyc files. I'm now not sure whether it was a machine running 2.5 that tried to run 2.4 bytecode or whether it was a 2.4 machine trying to run 2.5 bytecode, but if I deleted the bytecode all went well until the next bytecode clash.

I upgraded all of the machines to 2.5 and the problem went away.

In general, .pyc files are specific to one Python version (although portable across different machine architectures, as long as they're running the same version); the files carry the information about the relevant Python version in their headers -- so, if you leave the corresponding .py files next to the .pyc ones, the .pyc will be rebuilt every time a different Python version is used to import those modules. "Trying to run" wrong-version .pyc files is something I never heard about. What architectures were involved? Were the .py files around as they should be?

Edit: as the OP clarified that the crashes came when he was running both Python 2.4 and Python 2.5 programs on the same .py files (from two different servers, sharing a network drive), the explanation of the crashes becomes easy. The .py files were all the time being recompiled -- by the 2.4 Python when the 2.5 had been the one running them most recently, and vice versa -- and therefore the .pyc files were frantically busy getting rewritten all the time. Proper file locking on network drives (especially but not exclusively across different operating systems) is notoriously hard to achieve. So the following must have happened (the roles could be switched): the 2.4 server had just determined that a .pyc file was fine for it and started reading it; before it could finish reading, the 2.5 server (having previously determined that the module needed to be recompiled) wrote over it; so the 2.4 server ended up with a memory buffer that had (say) the first 4K bytes from the 2.4 version and the next 4K bytes from the 2.5 version. When it then used that mangled buffer, unsurprisingly... crash!!!

This can be a real problem if you ever find yourself continuously trying to run a single set of .py files from two or more different versions of Python (even on the same server, without the added complications of network drives). The "proper" solution would be something like virtualenv. The (simple, but dirty-ish) hack we adopted at work (many years ago, but it's still in production...!) is to patch each version of Python to produce and use a different extension for its compiled bytecode files: .pyc (or .pyo) for Python 1.5.2 (which was the most stable "system" version back when we started doing this kludge to newer versions), .pyc-2.0 for 2.0, .pyc-2.2 for 2.2, and so forth (or equivalent .pyo-X.Y of course). I hear this is soon going away at long last (thanks Thomas!-), but it did tide us semi-decently over this ticklish problem for many, many years.

A much simpler solution is to keep a single version of Python around, if that's feasible for your system; if your system has any complications that make it unfeasible to have a single Python version (as ours did, and does), then these days I'd heartily recommend virtualenv, which I've already mentioned.

With the adoption of PEP 3147 in Python 3.2, pyc files for different Python versions are distinguished automatically by filename. This should solve most problems with different Python versions overwriting each other's pyc files.

If you have the source code then it will recompile it for you. So in general you are okay.

But, this could be bad for you if users with difference versions of Python run from a central installation directory.

It could also be bad if you just have the pyc files. I just ran a quick test for you. I created two .pyc files. One in 2.5 and one in 2.6. The 2.5 won't run in 2.6 and the 2.6 won't run in 2.5. Both throw "ImportError: Bad magic number in .." error, which makes sense because the magic number has changed from 2.5 to 2.6.

If you want to determine this ahead of time you can get the magic number of your Python as follows:

$ python -V
Python 2.6.2
# python
>>> import imp
>>> imp.get_magic().encode('hex')
'd1f20d0a'

To get the magic number for a pyc file you can do the following:

>>> f = open('test25.pyc')
>>> magic = f.read(4)
>>> magic.encode('hex')
'b3f20d0a'
>>> f = open('test26.pyc')
>>> magic = f.read(4)
>>> magic.encode('hex')
'd1f20d0a'

The Python version that creates the file is stored in the .pyc file itself. Usually this means that the .pyc is replaced by one with the correct Python version

some reasons this might not happen
- permissions
- .py file is not available

In the case of permission problem, Python will just use the .py and ignore the .pyc (at a cost to performance)

I think it is ok between minor versions though, eg a Python2.6.2 .pyc should work with Python2.6.4

Here is an excerpt from /usr/share/file/magic

# python:  file(1) magic for python
0   string      """ a python script text executable
0   belong      0x994e0d0a  python 1.5/1.6 byte-compiled
0   belong      0x87c60d0a  python 2.0 byte-compiled
0   belong      0x2aeb0d0a  python 2.1 byte-compiled
0   belong      0x2ded0d0a  python 2.2 byte-compiled
0   belong      0x3bf20d0a  python 2.3 byte-compiled
0   belong      0x6df20d0a  python 2.4 byte-compiled
0   belong      0xb3f20d0a  python 2.5 byte-compiled
0   belong      0xd1f20d0a  python 2.6 byte-compiled

So you can see that the correct Python version is indicated by the first 4 bytes of the .pyc file

See http://www.python.org/dev/peps/pep-3149/ for a proposed fix for this (hopefully in Python 3.2)

You will certainly need to recompile the bytecode files for them to be of any use. The bytecode magic number has changed over every major version of Python(*).

However, having non-version-matching bytecode files shouldn't ever crash Python. It will generally just ignore any bytecode that doesn't have the correct version number, so there shouldn't be an error, it'll just be slower the first time as it recompiles (or slower every time if the user running the scripts doesn't have write permission to update the bytecode).

(*: and often during the development phases, plus in earlier versions it sometimes changed over minor versions too. See import.c for a full list of magic numbers and their corresponding Python versions.)