Cython
Close to the metal Python
Kyiv.py #10
Taras Lyapun
Python is a wonderful language that
allows us to solve problems quickly and
elegantly, but...
...but nothing ideal in this world. As a
interpreted, dynamical language without
static typization, Python is slow.
How many times have you heard or said,
"We can always rewrite bottlenecks on
C"?
How can we do this?
- By hand using Python C API
- SWIG
- Boost.Python
- ctypes
- SIP, Py++, f2py, PyD, Interrogate, Robin...
Cython
- Programming language based on Python
- The source code gets translated into
optimized C/C++ code and compiled as ...
Install
pip install cython
Building with a distutils
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils...
Building with pyximport
import pyximport
pyximport.install()
Building manually
cython path/to/your.pyx
gcc ...
Workflow
- Python code
- Profiling
- Unittests
- Cython code
Original
mv util.py util.pyx
x2 performance
cython -a util.pyx
Just add static typization
ncalls tottime percall cumtime percall filename:lineno(function)
8 0.010 0.001 0.010 0.001 {qrc...
cython -a util.pyx
Overview
- Almost like Python (with few exceptions)
- top-level classes and functions
- loops, with, try-except/finally......
Type declaration
- cdef - for static typization
cdef double dx, s
- cdef - as a C function
cdef double f(double x):
return...
Classes
cdef class MyClass(SomeClass):
- "builtin" extension type
- single inheritance, only from other extension
types
- ...
Pure python mode
@cython.ccall
@cython.returns(cython.double)
@cython.locals(x=cython.double, y=cython.
double)
def float_...
Pure python mode
# test.py
def float_mult(x, y):
return x * y
# test.pxd
cpdef double float_mult(double x, double y)
(but ...
Exceptions
cdef double f(double x) except -1:
cdef double f(double x) except? -1:
cdef int spam() except *:
cdef int spam(...
Exceptions
File "run.py", line 9, in <module>
"""
File "qrcode/main.py", line 7, in make
return qr.make_image()
File "qrco...
Profiling
# cython: profile=True
@cython.profile(False)
cython -X profile=True ...
Also
- debugging with GDB
- Easy wrap C/C++ libraries
- Numpy, cmath, etc
- with nogil - release GIL
- cython.parallel - f...
Thanks!
http://cython.org/
Upcoming SlideShare
Loading in …5
×

Cython - close to metal Python

2,290 views

Published on

Little talk at Kiev-Py #10

Taras Lyapun

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,290
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
21
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Cython - close to metal Python

  1. 1. Cython Close to the metal Python Kyiv.py #10 Taras Lyapun
  2. 2. Python is a wonderful language that allows us to solve problems quickly and elegantly, but...
  3. 3. ...but nothing ideal in this world. As a interpreted, dynamical language without static typization, Python is slow.
  4. 4. How many times have you heard or said, "We can always rewrite bottlenecks on C"?
  5. 5. How can we do this? - By hand using Python C API - SWIG - Boost.Python - ctypes - SIP, Py++, f2py, PyD, Interrogate, Robin, ... - Cython
  6. 6. Cython - Programming language based on Python - The source code gets translated into optimized C/C++ code and compiled as Python extension modules - Originally based on the well-known Pyrex - This code is executed within the CPython runtime environment, but at the speed of compiled C and with the ability to call directly into C libraries.
  7. 7. Install pip install cython
  8. 8. Building with a distutils from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext ext_modules = [Extension("hello", ["hello.pyx"])] setup( name = 'Hello world app', cmdclass = {'build_ext': build_ext}, ext_modules = ext_modules ) python setup.py build_ext --inplace
  9. 9. Building with pyximport import pyximport pyximport.install()
  10. 10. Building manually cython path/to/your.pyx gcc ...
  11. 11. Workflow - Python code - Profiling - Unittests - Cython code
  12. 12. Original
  13. 13. mv util.py util.pyx x2 performance
  14. 14. cython -a util.pyx
  15. 15. Just add static typization ncalls tottime percall cumtime percall filename:lineno(function) 8 0.010 0.001 0.010 0.001 {qrcode.util.lost_point} x16 performance. Easy achieve 100-1000x!
  16. 16. cython -a util.pyx
  17. 17. Overview - Almost like Python (with few exceptions) - top-level classes and functions - loops, with, try-except/finally... - lambda - generators - import - Py3 support
  18. 18. Type declaration - cdef - for static typization cdef double dx, s - cdef - as a C function cdef double f(double x): return sin(x**2) - cdef class - extensions cdef class MyType: cdef int field - cpdef - C function + Python wrapper
  19. 19. Classes cdef class MyClass(SomeClass): - "builtin" extension type - single inheritance, only from other extension types - fixed, typed fields - C-only access by default, or readonly/public - Python + C methods
  20. 20. Pure python mode @cython.ccall @cython.returns(cython.double) @cython.locals(x=cython.double, y=cython. double) def float_mult(x, y): return x * y
  21. 21. Pure python mode # test.py def float_mult(x, y): return x * y # test.pxd cpdef double float_mult(double x, double y) (but no access to C functions)
  22. 22. Exceptions cdef double f(double x) except -1: cdef double f(double x) except? -1: cdef int spam() except *: cdef int spam() except +:
  23. 23. Exceptions File "run.py", line 9, in <module> """ File "qrcode/main.py", line 7, in make return qr.make_image() File "qrcode/main.py", line 169, in make_image self.make() File "qrcode/main.py", line 53, in make self.best_fit(start=self.version) File "qrcode/main.py", line 108, in best_fit self.error_correction, self.data_list) File "util.pyx", line 401, in qrcode.util.create_data (qrcode/util.c:7844) File "util.pyx", line 418, in qrcode.util.create_data (qrcode/util.c:7513) qrcode.util.DataOverflowError: Code length overflow. Data size (916) > size available (128)
  24. 24. Profiling # cython: profile=True @cython.profile(False) cython -X profile=True ...
  25. 25. Also - debugging with GDB - Easy wrap C/C++ libraries - Numpy, cmath, etc - with nogil - release GIL - cython.parallel - for native parallelism (OpenMP) - and a lot of other cool hackers things
  26. 26. Thanks! http://cython.org/

×