Cython
Close to the metal Python
Kyiv.py #10
Taras Lyapun
Python is a wonderful language that
allows us to solve problems quickly and
elegantly, but...
...but nothing ideal in this world. As a
interpreted, dynamical language without
static typization, Python is slow.
How many times have you heard or said,
"We can always rewrite bottlenecks on
C"?
How can we do this?
- By hand using Python C API
- SWIG
- Boost.Python
- ctypes
- SIP, Py++, f2py, PyD, Interrogate, Robin, ...
- Cython
Cython
- Programming language based on Python
- The source code gets translated into
optimized C/C++ code and compiled as Python
extension modules
- Originally based on the well-known Pyrex
- This code is executed within the CPython
runtime environment, but at the speed of
compiled C and with the ability to call directly
into C libraries.
Install
pip install cython
Building with a distutils
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [Extension("hello", ["hello.pyx"])]
setup(
name = 'Hello world app',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules
)
python setup.py build_ext --inplace
Building with pyximport
import pyximport
pyximport.install()
Building manually
cython path/to/your.pyx
gcc ...
Workflow
- Python code
- Profiling
- Unittests
- Cython code
Original
mv util.py util.pyx
x2 performance
cython -a util.pyx
Just add static typization
ncalls tottime percall cumtime percall filename:lineno(function)
8 0.010 0.001 0.010 0.001 {qrcode.util.lost_point}
x16 performance.
Easy achieve 100-1000x!
cython -a util.pyx
Overview
- Almost like Python (with few exceptions)
- top-level classes and functions
- loops, with, try-except/finally...
- lambda
- generators
- import
- Py3 support
Type declaration
- cdef - for static typization
cdef double dx, s
- cdef - as a C function
cdef double f(double x):
return sin(x**2)
- cdef class - extensions
cdef class MyType:
cdef int field
- cpdef - C function + Python wrapper
Classes
cdef class MyClass(SomeClass):
- "builtin" extension type
- single inheritance, only from other extension
types
- fixed, typed fields
- C-only access by default, or readonly/public
- Python + C methods
Pure python mode
@cython.ccall
@cython.returns(cython.double)
@cython.locals(x=cython.double, y=cython.
double)
def float_mult(x, y):
return x * y
Pure python mode
# test.py
def float_mult(x, y):
return x * y
# test.pxd
cpdef double float_mult(double x, double y)
(but no access to C functions)
Exceptions
cdef double f(double x) except -1:
cdef double f(double x) except? -1:
cdef int spam() except *:
cdef int spam() except +:
Exceptions
File "run.py", line 9, in <module>
"""
File "qrcode/main.py", line 7, in make
return qr.make_image()
File "qrcode/main.py", line 169, in make_image
self.make()
File "qrcode/main.py", line 53, in make
self.best_fit(start=self.version)
File "qrcode/main.py", line 108, in best_fit
self.error_correction, self.data_list)
File "util.pyx", line 401, in qrcode.util.create_data (qrcode/util.c:7844)
File "util.pyx", line 418, in qrcode.util.create_data (qrcode/util.c:7513)
qrcode.util.DataOverflowError: Code length overflow. Data size (916) > size
available (128)
Profiling
# cython: profile=True
@cython.profile(False)
cython -X profile=True ...
Also
- debugging with GDB
- Easy wrap C/C++ libraries
- Numpy, cmath, etc
- with nogil - release GIL
- cython.parallel - for native parallelism (OpenMP)
- and a lot of other cool hackers things
Thanks!
http://cython.org/

Cython - close to metal Python

  • 1.
    Cython Close to themetal Python Kyiv.py #10 Taras Lyapun
  • 2.
    Python is awonderful language that allows us to solve problems quickly and elegantly, but...
  • 3.
    ...but nothing idealin this world. As a interpreted, dynamical language without static typization, Python is slow.
  • 4.
    How many timeshave you heard or said, "We can always rewrite bottlenecks on C"?
  • 5.
    How can wedo this? - By hand using Python C API - SWIG - Boost.Python - ctypes - SIP, Py++, f2py, PyD, Interrogate, Robin, ... - Cython
  • 6.
    Cython - Programming languagebased on Python - The source code gets translated into optimized C/C++ code and compiled as Python extension modules - Originally based on the well-known Pyrex - This code is executed within the CPython runtime environment, but at the speed of compiled C and with the ability to call directly into C libraries.
  • 7.
  • 8.
    Building with adistutils from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext ext_modules = [Extension("hello", ["hello.pyx"])] setup( name = 'Hello world app', cmdclass = {'build_ext': build_ext}, ext_modules = ext_modules ) python setup.py build_ext --inplace
  • 9.
    Building with pyximport importpyximport pyximport.install()
  • 10.
  • 11.
    Workflow - Python code -Profiling - Unittests - Cython code
  • 12.
  • 13.
  • 14.
  • 15.
    Just add statictypization ncalls tottime percall cumtime percall filename:lineno(function) 8 0.010 0.001 0.010 0.001 {qrcode.util.lost_point} x16 performance. Easy achieve 100-1000x!
  • 16.
  • 17.
    Overview - Almost likePython (with few exceptions) - top-level classes and functions - loops, with, try-except/finally... - lambda - generators - import - Py3 support
  • 18.
    Type declaration - cdef- for static typization cdef double dx, s - cdef - as a C function cdef double f(double x): return sin(x**2) - cdef class - extensions cdef class MyType: cdef int field - cpdef - C function + Python wrapper
  • 19.
    Classes cdef class MyClass(SomeClass): -"builtin" extension type - single inheritance, only from other extension types - fixed, typed fields - C-only access by default, or readonly/public - Python + C methods
  • 20.
  • 21.
    Pure python mode #test.py def float_mult(x, y): return x * y # test.pxd cpdef double float_mult(double x, double y) (but no access to C functions)
  • 22.
    Exceptions cdef double f(doublex) except -1: cdef double f(double x) except? -1: cdef int spam() except *: cdef int spam() except +:
  • 23.
    Exceptions File "run.py", line9, in <module> """ File "qrcode/main.py", line 7, in make return qr.make_image() File "qrcode/main.py", line 169, in make_image self.make() File "qrcode/main.py", line 53, in make self.best_fit(start=self.version) File "qrcode/main.py", line 108, in best_fit self.error_correction, self.data_list) File "util.pyx", line 401, in qrcode.util.create_data (qrcode/util.c:7844) File "util.pyx", line 418, in qrcode.util.create_data (qrcode/util.c:7513) qrcode.util.DataOverflowError: Code length overflow. Data size (916) > size available (128)
  • 24.
  • 25.
    Also - debugging withGDB - Easy wrap C/C++ libraries - Numpy, cmath, etc - with nogil - release GIL - cython.parallel - for native parallelism (OpenMP) - and a lot of other cool hackers things
  • 26.