Brief Introduction to Cython

March 2016 NoVa Python Meetup
Brief Introduction to Cython
Aleksandar Jelenak

What is Cython?
A programming language with Python-like syntax and C/C++-like static typing.
Optimizing static compiler that translates Python/Cython to C/C++ code.
2

● Speed up critical parts of Python code using static typing (measure performance
first!)
● Accelerate data exchange between array-like structures (multidimensional
NumPy arrays, str, bytes. bytearray, array.array, ctypes)
● Integrate Python with existing C/C++ applications and libraries
3
How can Cython help?

Further Information
4
Book: Cython: A Guide for Python Programmers, Kurt Smith
Home page: http://cython.org
Documentation: http://docs.cython.org
FAQ: https://github.com/cython/cython/wiki/FAQ
User mailing list: https://groups.google.com/forum/#!forum/cython-users

Installation
● C/C++ compiler (Mac OS X: Xcode; Windows: Visual Studio)
● Many Python distributions
○ Enthought’s Canopy
○ Continuum’s Anaconda Python
○ Python(x,y)
● Package management systems
○ Linux: apt-get, yum
○ OS X: MacPorts, Homebrew
● For the latest version
○ pip install cython
○ python setup.py install
5

● Implementation files (.pyx)
○ Cython source code
● Definition files (.pxd)
○ C-level constructs definitions (similar to C/C++ header files)
● Include files (.pxi)
○ Specific information for abstracting away any differences in order to produce consistent code
interface
6
Code Organization

Example: Fibonacci Numbers
Python: fib.py
def fib(n):
a = 0
b = 1
for i in range(n):
a, b = a + b, a
return a
Cython: cyfib.pyx
def fib(int n):
cdef int i
cdef double a = 0.0
cdef double b = 1.0
for i in range(n):
a, b = a + b, a
return a
7

● Python’s build tools
● Automatically at import
● From Jupyter (IPython) session
● Integrated into make or Cmake build systems
What happens:
● Cython compiler transforms Python/Cython source code into platform-
independent, optimized C or C++ code.
● C/C++ compiler generates platform-specific shared library from the C/C++ code.
● Produced shared library file is called extension module and can be imported as
any other Python module.
8
How to Compile Cython Code

● Command-line tool for translating Python/Cython code to C/C++ code
● Useful options:
○ -a, --annotate Produce a colorized HTML version of the source
○ -o, --output-file <filename> Specify name of generated C file
○ -t, --timestamps Only compile newer source files
○ -f, --force Compile all source files (overrides implied -t)
○ --line-directives Produce #line directives pointing to the .pyx source
○ --cplus Output a C++ rather than C file
○ -2 Compile based on Python-2 syntax and code semantics
○ -3 Compile based on Python-3 syntax and code semantics
9
The cython Command

Annotation file: cython -a fib.py
● Beside C/C++ code output, produce also an
HTML file with code annotations of the
Python/Cython source code.
● Yellow lines indicate the number of Python
C API calls.
● Less yellow generally indicates better
performance.
● Clicking on each line with “+” displays the
generated C/C++ code.
● The for loop lines have the largest amount
of Python C API calls because the types of
the n, a, and b variables are not specified.
10

Annotation file: cython -a fib.pyx
● Beside C/C++ code output, produce also an
HTML file with code annotations of the
Python/Cython source code.
● Yellow lines indicate the number of Python
C API calls.
● Less yellow generally indicates better
performance.
● Clicking on each line with “+” displays the
generated C/C++ code.
● Because of static typing of the n, a, and b
variables the for loop is now just the simple
C for loop.
11

$ ipython
In [1]: %load_ext Cython
In [2]: %%cython
...: def fib(n):
...: a = 0
...: b = 1
...: for i in range(n):
...: a, b = a + b, a
...: return a
...:
In [4]: %ls ~/.ipython/cython/
Users/
_cython_magic_65fad53c09eafb137d14870b4cf57507.c
_cython_magic_65fad53c09eafb137d14870b4cf57507.cpython-35m-darwin.so*
_cython_magic_65fad53c09eafb137d14870b4cf57507.pyx
In [5]: %whos
Variable Type Data/Info
--------------------------------------------------
fib builtin_function_or_method <built-in function fib>
12
How to Run Cython Code (IPython magic %%cython)

$ ipython
In [1]: import pyximport
In [2]: pyximport.install()
Out[2]: (None, <pyximport.pyximport.PyxImporter at 0x104dcba90>)
In [3]: import cyfib
In [4]: %whos
Variable Type Data/Info
-------------------------------
cyfib module <module 'cyfib' (/Users/a<...>b.cpython-35m-darwin.so)>
pyximport module <module 'pyximport' from <...>s/pyximport/__init__.py>
In [5]: %ls ~/.pyxbld/lib.macosx-10.10-x86_64-3.5/
cyfib.cpython-35m-darwin.so*
13
How to Run Cython Code (On import: pyximport)

setup.py file:
from distutils.core import setup
from Cython.Build import cythonize
setup(ext_modules=cythonize('cyfib.pyx'))
Build command:
$ python setup.py build_ext --inplace
14
How to Run Cython Code (Python build tools)

15
Performance Comparison
Version Runtime Speedup
Python 6.39 µs 1x
Compiled Python 4.1 µs 1.56x
Cython 167 ns 38.26x
%timeit fib(100)

● Most performance improvements for CPU-bound tasks
● Could somewhat help for memory-bound tasks
● Not appropriate for I/O- or network-bound tasks
● Value overflow possible when replacing Python data types with C data types
● Always first profile to locate and quantify execution bottlenecks!
16
When to use Cython?

Selected Cython Language Features
17

cdef int a = 0
cdef long b
cdef double c
cdef float **p = NULL
cdef list l
cdef tuple t
cdef object o
Variables without declared type in Cython code treated as standard Python variables.
18
Static Typing: cdef keyword

19
Declaration of structs, unions, enums
ctypedef struct xyz:
int x
float y
long z
ctypedef union u:
short a
double b, c
ctypedef enum H5Q_match_op_t:
H5Q_MATCH_EQUAL,
H5Q_MATCH_NOT_EQUAL,
H5Q_MATCH_LESS_THAN,
H5Q_MATCH_GREATER_THAN

Four ways:
cdef xyz v = xyz(1, 2.0, 3)
cdef xyz v = xyz(x=1, z=3, y=2.0)
cdef xyz v = {'z': 3, 'y': 2.0, 'x': 1}
cdef xyz v
v.x = 1
v.y = 2.0
v.z = 3
20
Assignment to structs

Pointer’s value at location 0 dereferences the pointer.
cdef int *p, a
a = 5
p = &a
print p[0] # prints: 5
21
C Pointers in Cython

● C’s char* and C++’s string types are automatically treated as the bytes type.
● Python string variables just work but the cython compiler will issue warning if
explicit decoding/encoding is needed.
● The bytes type is supported as is.
● Cython’s str type is treated as bytes in Python 2 and str in Python 3.
● Cython’s unicode type is unicode type in Python 2 and str type in Python 3.
● Implicit conversion between bytes and unicode/str types not allowed by default.
● Global compiler directives c_string_type and c_string_encoding control
implicit string conversion.
22
Strings in Cython

● Implementation files (.pyx) can be written in either syntax.
● The generated C/C++ source code is both Python 2 and 3 compatible.
● The default is Python 2 syntax.
● Can be specified with the -2 or -3 cython command line options.
● Imports from __future__ are supported.
23
Python 2 or 3

● Cython supports standard Python functions defined with the def keyword.
● cdef keyword defines a C-style function. Return value type can be specified.
cdef long long fib(int n):
pass
● cdef functions need wrapper def functions to be callable from Python.
● cpdef functions behave like cdef when called from Cython and like def when
called from Python. Only data types supported in both Python and C/C++ are
allowed, e.g. not void.
● inline keyword can be used in cdef and cpdef functions.
24
def, cdef, cpdef

● Exceptions generated in cdef and cpdef functions that don’t return a Python
object are not propagated automatically.
● except keyword declares an exception value.
cdef int foo(double x) except -1:
pass
● If an exception occurs in foo(), the returned value will be set to -1 which will
cause the exception to be propagated.
● Using except? (note the question mark) means that -1 is a legal return value from
the foo() function so the Cython-generated C/C++ code will include a call to
PyErr_Occurred() to check first if there was an exception.
● With except * calls to PyErr_Occurred() are always made.
25
Exception Handling

● Cython allows definition of classes that act as Python built-in types.
● Such extension type classes have C-level access to all methods and data.
cdef class Matrix:
cdef:
unsigned int nrows, ncols
double *_matrix
def __cinit__(self, nr, nc):
self.nrows = nr
self.ncols = nc
self._matrix = <double*>malloc(nr * nc * sizeof(double))
if self._matrix == NULL:
raise MemoryError()
def __dealloc__(self):
if self._matrix != NULL:
free(self._matrix)
26
Extension Types
Example code from: Cython, Kurt Smith, O’Reilly, 2015.

Brief Introduction to Cython

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Brief Introduction to Cython

Similar to Brief Introduction to Cython (20)

Recently uploaded

Recently uploaded (20)

Brief Introduction to Cython