Cython - close to metal Python

  • 1,303 views
Uploaded on

Little talk at Kiev-Py #10 …

Little talk at Kiev-Py #10

Taras Lyapun

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,303
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
10
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Cython Close to the metal Python Kyiv.py #10 Taras Lyapun
  • 2. Python is a wonderful language that allows us to solve problems quickly and elegantly, but...
  • 3. ...but nothing ideal in this world. As a interpreted, dynamical language without static typization, Python is slow.
  • 4. How many times have you heard or said, "We can always rewrite bottlenecks on C"?
  • 5. How can we do this? - By hand using Python C API - SWIG - Boost.Python - ctypes - SIP, Py++, f2py, PyD, Interrogate, Robin, ... - Cython
  • 6. Cython - Programming language based on Python - The source code gets translated into optimized C/C++ code and compiled as Python extension modules - Originally based on the well-known Pyrex - This code is executed within the CPython runtime environment, but at the speed of compiled C and with the ability to call directly into C libraries.
  • 7. Install pip install cython
  • 8. Building with a distutils from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext ext_modules = [Extension("hello", ["hello.pyx"])] setup( name = 'Hello world app', cmdclass = {'build_ext': build_ext}, ext_modules = ext_modules ) python setup.py build_ext --inplace
  • 9. Building with pyximport import pyximport pyximport.install()
  • 10. Building manually cython path/to/your.pyx gcc ...
  • 11. Workflow - Python code - Profiling - Unittests - Cython code
  • 12. Original
  • 13. mv util.py util.pyx x2 performance
  • 14. cython -a util.pyx
  • 15. Just add static typization ncalls tottime percall cumtime percall filename:lineno(function) 8 0.010 0.001 0.010 0.001 {qrcode.util.lost_point} x16 performance. Easy achieve 100-1000x!
  • 16. cython -a util.pyx
  • 17. Overview - Almost like Python (with few exceptions) - top-level classes and functions - loops, with, try-except/finally... - lambda - generators - import - Py3 support
  • 18. Type declaration - cdef - for static typization cdef double dx, s - cdef - as a C function cdef double f(double x): return sin(x**2) - cdef class - extensions cdef class MyType: cdef int field - cpdef - C function + Python wrapper
  • 19. Classes cdef class MyClass(SomeClass): - "builtin" extension type - single inheritance, only from other extension types - fixed, typed fields - C-only access by default, or readonly/public - Python + C methods
  • 20. Pure python mode @cython.ccall @cython.returns(cython.double) @cython.locals(x=cython.double, y=cython. double) def float_mult(x, y): return x * y
  • 21. Pure python mode # test.py def float_mult(x, y): return x * y # test.pxd cpdef double float_mult(double x, double y) (but no access to C functions)
  • 22. Exceptions cdef double f(double x) except -1: cdef double f(double x) except? -1: cdef int spam() except *: cdef int spam() except +:
  • 23. Exceptions File "run.py", line 9, in <module> """ File "qrcode/main.py", line 7, in make return qr.make_image() File "qrcode/main.py", line 169, in make_image self.make() File "qrcode/main.py", line 53, in make self.best_fit(start=self.version) File "qrcode/main.py", line 108, in best_fit self.error_correction, self.data_list) File "util.pyx", line 401, in qrcode.util.create_data (qrcode/util.c:7844) File "util.pyx", line 418, in qrcode.util.create_data (qrcode/util.c:7513) qrcode.util.DataOverflowError: Code length overflow. Data size (916) > size available (128)
  • 24. Profiling # cython: profile=True @cython.profile(False) cython -X profile=True ...
  • 25. Also - debugging with GDB - Easy wrap C/C++ libraries - Numpy, cmath, etc - with nogil - release GIL - cython.parallel - for native parallelism (OpenMP) - and a lot of other cool hackers things
  • 26. Thanks! http://cython.org/