python PIL YCbCr support

Trying to create an PIL image in YCbCr mode crashes even a fresh python/PIL installation from synaptic in my ubuntu 11.04

wim@wim-ubuntu:~$ python
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Image
>>> Image.VERSION
'1.1.7'
>>> Image.new('YCbCr', (640,480))
*** buffer overflow detected ***: python terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7f4a35e4a1d7]
/lib/x86_64-linux-gnu/libc.so.6(+0xfd0f0)[0x7f4a35e490f0]
/usr/local/lib/python2.7/dist-packages/PIL/_imaging.so(ImagingNewPrologueSubtype+0x97)[0x7f4a34bfb257]
/usr/local/lib/python2.7/dist-packages/PIL/_imaging.so(ImagingNewBlock+0xb)[0x7f4a34bfb78b]
/usr/local/lib/python2.7/dist-packages/PIL/_imaging.so(ImagingNew+0x4c)[0x7f4a34bfb86c]
/usr/local/lib/python2.7/dist-packages/PIL/_imaging.so(+0xfd22)[0x7f4a34bddd22]
python(PyEval_EvalFrameEx+0x361)[0x4965f1]
python(PyEval_EvalCodeEx+0x145)[0x49d325]
python(PyEval_EvalFrameEx+0x802)[0x496a92]
python(PyEval_EvalCodeEx+0x145)[0x49d325]
python(PyEval_EvalCode+0x32)[0x4ecb02]
python[0x4fdc74]
python(PyRun_InteractiveOneFlags+0x1e2)[0x42cd9e]
python(PyRun_InteractiveLoopFlags+0xc0)[0x42cebd]
python(PyRun_AnyFileExFlags+0x39)[0x42d2e1]
python(Py_Main+0xac9)[0x418c9e]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff)[0x7f4a35d6aeff]
python[0x4c62b1]
======= Memory map: ========
00400000-0062f000 r-xp 00000000 08:21 1049585                            /usr/bin/python2.7
0082e000-0082f000 r--p 0022e000 08:21 1049585                            /usr/bin/python2.7
0082f000-00897000 rw-p 0022f000 08:21 1049585                            /usr/bin/python2.7
00897000-008a9000 rw-p 00000000 00:00 0 
01f0a000-021a1000 rw-p 00000000 00:00 0                                  [heap]
7f4a34794000-7f4a347a9000 r-xp 00000000 08:21 659235                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f4a347a9000-7f4a349a8000 ---p 00015000 08:21 659235                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f4a349a8000-7f4a349a9000 r--p 00014000 08:21 659235                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f4a349a9000-7f4a349aa000 rw-p 00015000 08:21 659235                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f4a349aa000-7f4a349cd000 r-xp 00000000 08:21 1057568                    /usr/lib/x86_64-linux-gnu/libjpeg.so.62.0.0
7f4a349cd000-7f4a34bcc000 ---p 00023000 08:21 1057568                    /usr/lib/x86_64-linux-gnu/libjpeg.so.62.0.0
7f4a34bcc000-7f4a34bcd000 r--p 00022000 08:21 1057568                    /usr/lib/x86_64-linux-gnu/libjpeg.so.62.0.0
7f4a34bcd000-7f4a34bce000 rw-p 00023000 08:21 1057568                    /usr/lib/x86_64-linux-gnu/libjpeg.so.62.0.0
7f4a34bce000-7f4a34c0b000 r-xp 00000000 08:21 1312337                    /usr/local/lib/python2.7/dist-packages/PIL/_imaging.so
7f4a34c0b000-7f4a34e0b000 ---p 0003d000 08:21 1312337                    /usr/local/lib/python2.7/dist-packages/PIL/_imaging.so
7f4a34e0b000-7f4a34e0e000 r--p 0003d000 08:21 1312337                    /usr/local/lib/python2.7/dist-packages/PIL/_imaging.so
7f4a34e0e000-7f4a34e11000 rw-p 00040000 08:21 1312337                    /usr/local/lib/python2.7/dist-packages/PIL/_imaging.so
7f4a34e11000-7f4a34e30000 r-xp 00000000 08:21 1316665                    /usr/lib/python2.7/lib-dynload/_ctypes.so
7f4a34e30000-7f4a3502f000 ---p 0001f000 08:21 1316665                    /usr/lib/python2.7/lib-dynload/_ctypes.so
7f4a3502f000-7f4a35030000 r--p 0001e000 08:21 1316665                    /usr/lib/python2.7/lib-dynload/_ctypes.so
7f4a35030000-7f4a35034000 rw-p 0001f000 08:21 1316665                    /usr/lib/python2.7/lib-dynload/_ctypes.so
7f4a35034000-7f4a35035000 rw-p 00000000 00:00 0 
7f4a35035000-7f4a35075000 r-xp 00000000 08:21 655419                     /lib/li开发者_C百科bncurses.so.5.7
7f4a35075000-7f4a35274000 ---p 00040000 08:21 655419                     /lib/libncurses.so.5.7
7f4a35274000-7f4a35278000 r--p 0003f000 08:21 655419                     /lib/libncurses.so.5.7
7f4a35278000-7f4a35279000 rw-p 00043000 08:21 655419                     /lib/libncurses.so.5.7
7f4a35279000-7f4a352b2000 r-xp 00000000 08:21 655450                     /lib/libreadline.so.6.2
7f4a352b2000-7f4a354b2000 ---p 00039000 08:21 655450                     /lib/libreadline.so.6.2
7f4a354b2000-7f4a354b4000 r--p 00039000 08:21 655450                     /lib/libreadline.so.6.2
7f4a354b4000-7f4a354ba000 rw-p 0003b000 08:21 655450                     /lib/libreadline.so.6.2
7f4a354ba000-7f4a354bb000 rw-p 00000000 00:00 0 
7f4a354bb000-7f4a354c0000 r-xp 00000000 08:21 1316693                    /usr/lib/python2.7/lib-dynload/readline.so
7f4a354c0000-7f4a356bf000 ---p 00005000 08:21 1316693                    /usr/lib/python2.7/lib-dynload/readline.so
7f4a356bf000-7f4a356c0000 r--p 00004000 08:21 1316693                    /usr/lib/python2.7/lib-dynload/readline.so
7f4a356c0000-7f4a356c2000 rw-p 00005000 08:21 1316693                    /usr/lib/python2.7/lib-dynload/readline.so
7f4a356c2000-7f4a35d4c000 r--p 00000000 08:21 1055991                    /usr/lib/locale/locale-archive
7f4a35d4c000-7f4a35ed6000 r-xp 00000000 08:21 659207                     /lib/x86_64-linux-gnu/libc-2.13.so
7f4a35ed6000-7f4a360d5000 ---p 0018a000 08:21 659207                     /lib/x86_64-linux-gnu/libc-2.13.so
7f4a360d5000-7f4a360d9000 r--p 00189000 08:21 659207                     /lib/x86_64-linux-gnu/libc-2.13.so
7f4a360d9000-7f4a360da000 rw-p 0018d000 08:21 659207                     /lib/x86_64-linux-gnu/libc-2.13.so
7f4a360da000-7f4a360e0000 rw-p 00000000 00:00 0 
7f4a360e0000-7f4a36164000 r-xp 00000000 08:21 659244                     /lib/x86_64-linux-gnu/libm-2.13.so
7f4a36164000-7f4a36363000 ---p 00084000 08:21 659244                     /lib/x86_64-linux-gnu/libm-2.13.so
7f4a36363000-7f4a36364000 r--p 00083000 08:21 659244                     /lib/x86_64-linux-gnu/libm-2.13.so
7f4a36364000-7f4a36365000 rw-p 00084000 08:21 659244                     /lib/x86_64-linux-gnu/libm-2.13.so
7f4a36365000-7f4a3637c000 r-xp 00000000 08:21 659290                     /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f4a3637c000-7f4a3657b000 ---p 00017000 08:21 659290                     /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f4a3657b000-7f4a3657c000 r--p 00016000 08:21 659290                     /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f4a3657c000-7f4a3657d000 rw-p 00017000 08:21 659290                     /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f4a3657d000-7f4a366e3000 r-xp 00000000 08:21 655398                     /lib/libcrypto.so.0.9.8
7f4a366e3000-7f4a368e3000 ---p 00166000 08:21 655398                     /lib/libcrypto.so.0.9.8
7f4a368e3000-7f4a368f0000 r--p 00166000 08:21 655398                     /lib/libcrypto.so.0.9.8
7f4a368f0000-7f4a36909000 rw-p 00173000 08:21 655398                     /lib/libcrypto.so.0.9.8
7f4a36909000-7f4a3690c000 rw-p 00000000 00:00 0 
7f4a3690c000-7f4a36958000 r-xp 00000000 08:21 655454                     /lib/libssl.so.0.9.8Aborted
wim@wim-ubuntu:~$ 

I've tried

find /usr -name Image.py | xargs grep -n YCbCr 

and found what I believe to be the offending line

216:    "YCbCr": ('|u1', 4),

YCbCr images should be 3 channel, not 4. I changed this to 3 and deleted the .pyc, but the code still crashed in the same way when it goes into the c code (core.fill with mode='YCbCr'). OK so I thought maybe the error worked its way into the build somehow, marked PIL for complete removal in package manager, downloaded the sources, dependencies, and modified that number 4 to 3, and after a bit of mucking around with paths was able to build PIL again, with all the selftests passing, then installing. Unfortunately, still the same crash. Now I'm at a loss at to what to do - I need YCbCr colour space support in PIL but can't fix the bug.

edit: perhaps this can provide further insight, I've found that while Image.new('YCbCr', (640,480)) did not crash in 10.10, it can be provoked to segfault by using numpy.asarray(Image.new('YCbCr', (640,480))). It all smells like a memory problem in some C-extension of numpy and/or PIL, or the way they are talking to each other.

Tracing down the source and it seems that it's the strcpy() at libImaging/Storage.c:186 that caused the buffer overflow. Further looking at the code and I found that im->mode is not big enough to hold "YCbCr".

To solve this problem, just change libImaging/Imaging.h:78 from char mode[4+1]; to char mode[4+2];char mode[6+1]; and recompile PIL.

For numpy.asarray, according to another post, I think its also a problem of PIL and your fix on Image.py should work.

edit: A patch for Fedora was commited before and it suggest changing to char mode[6+1]; so the buffer is big enough to hold "RGB;32".

OOT: If Linus would see this he would definitely not accept my fix due to its magic..