Simple wrapping of C code with cython

I have a number of C functions, and I would like to call them from python. cython seems to be the way to go, but I can't really find an example of how exactly this is done. My C function looks like this:

void calculate_daily ( char *db_name, int grid_id, int year,
                       double *dtmp, double *dtmn, double *dtmx, 
                       double *dprec, double *ddtr, double *dayl, 
                       double *dpet, double *dpar ) ;

All I want to do is to specify the first three parameters (a string and two integers), and recov开发者_StackOverflow中文版er 8 numpy arrays (or python lists. All the double arrays have N elements). My code assumes that the pointers are pointing to an already allocated chunk of memory. Also, the produced C code ought to link to some external libraries.

Here's a tiny but complete example of passing numpy arrays to an external C function, logically

fc( int N, double* a, double* b, double* z )  # z = a + b

using Cython. (This is surely well-known to those who know it well. Comments are welcome. Last change: 23 Feb 2011, for Cython 0.14.)

First read or skim Cython build and Cython with NumPy .

2 steps:

• python f-setup.py build_ext --inplace
  turns f.pyx and fc.cpp -> f.so, a dynamic library
• python test-f.py
import f loads f.so; f.fpy( ... ) calls the C fc( ... ).

python f-setup.py uses distutils to run cython, compile and link:
cython f.pyx -> f.cpp
compile f.cpp and fc.cpp
link f.o fc.o -> f.so, a dynamic lib that python import f will load.

For students, I'd suggest: make a diagram of these steps, look through the files below, then download and run them.

(distutils is a huge, convoluted package used to make Python packages for distribution, and install them. Here we're using just a small part of it to compile and link to f.so. This step has nothing to do with Cython, but it can be confusing; simple mistakes in a .pyx can cause pages of obscure error messages from g++ compile and link. See also distutils doc and/or SO questions on distutils .)

Like make, setup.py will rerun cython f.pyx and g++ -c ... f.cpp if f.pyx is newer than f.cpp.
To cleanup, rm -r build/ .

An alternative to setup.py would be to run the steps separately, in a script or Makefile:
cython --cplus f.pyx -> f.cpp # see cython -h
g++ -c ... f.cpp -> f.o
g++ -c ... fc.cpp -> fc.o
cc-lib f.o fc.o -> dynamic library f.so.
Modify the cc-lib-mac wrapper below for your platform and installation: it's not pretty, but small.

For real examples of Cython wrapping C, look at .pyx files in just about any SciKit .

See also: Cython for NumPy users and SO questions/tagged/cython .

To unpack the following files, cut-paste the lot to one big file, say cython-numpy-c-demo, then in Unix (in a clean new directory) run sh cython-numpy-c-demo.

#--------------------------------------------------------------------------------
cat >f.pyx <<\!
# f.pyx: numpy arrays -> extern from "fc.h"
# 3 steps:
# cython f.pyx  -> f.c
# link: python f-setup.py build_ext --inplace  -> f.so, a dynamic library
# py test-f.py: import f gets f.so, f.fpy below calls fc()

import numpy as np
cimport numpy as np

cdef extern from "fc.h": 
    int fc( int N, double* a, double* b, double* z )  # z = a + b

def fpy( N,
    np.ndarray[np.double_t,ndim=1] A,
    np.ndarray[np.double_t,ndim=1] B,
    np.ndarray[np.double_t,ndim=1] Z ):
    """ wrap np arrays to fc( a.data ... ) """
    assert N <= len(A) == len(B) == len(Z)
    fcret = fc( N, <double*> A.data, <double*> B.data, <double*> Z.data )
        # fcret = fc( N, A.data, B.data, Z.data )  grr char*
    return fcret

!

#--------------------------------------------------------------------------------
cat >fc.h <<\!
// fc.h: numpy arrays from cython , double*

int fc( int N, const double a[], const double b[], double z[] );
!

#--------------------------------------------------------------------------------
cat >fc.cpp <<\!
// fc.cpp: z = a + b, numpy arrays from cython

#include "fc.h"
#include <stdio.h>

int fc( int N, const double a[], const double b[], double z[] )
{
    printf( "fc: N=%d a[0]=%f b[0]=%f \n", N, a[0], b[0] );
    for( int j = 0;  j < N;  j ++ ){
        z[j] = a[j] + b[j];
    }
    return N;
}
!

#--------------------------------------------------------------------------------
cat >f-setup.py <<\!
# python f-setup.py build_ext --inplace
#   cython f.pyx -> f.cpp
#   g++ -c f.cpp -> f.o
#   g++ -c fc.cpp -> fc.o
#   link f.o fc.o -> f.so

# distutils uses the Makefile distutils.sysconfig.get_makefile_filename()
# for compiling and linking: a sea of options.

# http://docs.python.org/distutils/introduction.html
# http://docs.python.org/distutils/apiref.html  20 pages ...
# https://stackoverflow.com/questions/tagged/distutils+python

import numpy
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
# from Cython.Build import cythonize

ext_modules = [Extension(
    name="f",
    sources=["f.pyx", "fc.cpp"],
        # extra_objects=["fc.o"],  # if you compile fc.cpp separately
    include_dirs = [numpy.get_include()],  # .../site-packages/numpy/core/include
    language="c++",
        # libraries=
        # extra_compile_args = "...".split(),
        # extra_link_args = "...".split()
    )]

setup(
    name = 'f',
    cmdclass = {'build_ext': build_ext},
    ext_modules = ext_modules,
        # ext_modules = cythonize(ext_modules)  ? not in 0.14.1
    # version=
    # description=
    # author=
    # author_email=
    )

# test: import f
!

#--------------------------------------------------------------------------------
cat >test-f.py <<\!
#!/usr/bin/env python
# test-f.py

import numpy as np
import f  # loads f.so from cc-lib: f.pyx -> f.c + fc.o -> f.so

N = 3
a = np.arange( N, dtype=np.float64 )
b = np.arange( N, dtype=np.float64 )
z = np.ones( N, dtype=np.float64 ) * np.NaN

fret = f.fpy( N, a, b, z )
print "fpy -> fc z:", z

!

#--------------------------------------------------------------------------------
cat >cc-lib-mac <<\!
#!/bin/sh
me=${0##*/}
case $1 in
"" )
    set --  f.cpp fc.cpp ;;  # default: g++ these
-h* | --h* )
    echo "
$me [g++ flags] xx.c yy.cpp zz.o ...
    compiles .c .cpp .o files to a dynamic lib xx.so
"
    exit 1
esac

# Logically this is simple, compile and link,
# but platform-dependent, layers upon layers, gloom, doom

base=${1%.c*}
base=${base%.o}
set -x

g++ -dynamic -arch ppc \
    -bundle -undefined dynamic_lookup \
    -fno-strict-aliasing -fPIC -fno-common -DNDEBUG `# -g` -fwrapv \
    -isysroot /Developer/SDKs/MacOSX10.4u.sdk \
    -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 \
    -I${Pysite?}/numpy/core/include \
    -O2 -Wall \
    "$@" \
    -o $base.so

# undefs: nm -gpv $base.so | egrep '^ *U _+[^P]'
!

# 23 Feb 2011 13:38

The following Cython code from http://article.gmane.org/gmane.comp.python.cython.user/5625 doesn't require explicit casts and also handles non-continous arrays:

def fpy(A):
    cdef np.ndarray[np.double_t, ndim=2, mode="c"] A_c
    A_c = np.ascontiguousarray(A, dtype=np.double)
    fc(&A_c[0,0])

Basically you can write your Cython function such that it allocates the arrays (make sure you cimport numpy as np):

cdef np.ndarray[np.double_t, ndim=1] rr = np.zeros((N,), dtype=np.double)

then pass in the .data pointer of each to your C function. That should work. If you don't need to start with zeros you could use np.empty for a small speed boost.

See the Cython for NumPy Users tutorial in the docs (fixed it to the correct link).

You should check out Ctypes it's probably the most easiest thing to use if all you want is one function.