Your SlideShare is downloading. ×
Cython - Making Python as Fast as C
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Cython - Making Python as Fast as C

1,278
views

Published on

It introduces how to make Python as fast as C by Cython at Taipei.py [1], COSCUP 2014 [2], and Tainan.py [3]. …

It introduces how to make Python as fast as C by Cython at Taipei.py [1], COSCUP 2014 [2], and Tainan.py [3].

[1]: http://www.meetup.com/Taipei-py/events/169077442/
[2]: http://coscup.org/2014
[3]: http://www.meetup.com/Tainan-py-Python-Tainan-User-Group/events/219957183/

Published in: Technology

0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,278
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
25
Comments
0
Likes
8
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. CYTHON — 
 MAKING PYTHON AS FAST AS C Mosky
  • 2. MOSKY 2
  • 3. MOSKY Python Charmer at Pinkoi 2
  • 4. MOSKY Python Charmer at Pinkoi An author of the Python Packages MoSQL, Clime, … and more 2
  • 5. MOSKY Python Charmer at Pinkoi An author of the Python Packages MoSQL, Clime, … and more A speaker of the conferences, PyCon TW/JP mostly. 2
  • 6. MOSKY Python Charmer at Pinkoi An author of the Python Packages MoSQL, Clime, … and more A speaker of the conferences, PyCon TW/JP mostly. A Python instructor 2
  • 7. MOSKY Python Charmer at Pinkoi An author of the Python Packages MoSQL, Clime, … and more A speaker of the conferences, PyCon TW/JP mostly. A Python instructor mosky.tw 2
  • 8. OUTLINE 3
  • 9. OUTLINE Introduction 3
  • 10. OUTLINE Introduction Setup 3
  • 11. OUTLINE Introduction Setup Foundation 3
  • 12. OUTLINE Introduction Setup Foundation Practicing 3
  • 13. OUTLINE Introduction Setup Foundation Practicing Tips 3
  • 14. OUTLINE Introduction Setup Foundation Practicing Tips Uncovered Topics 3
  • 15. OUTLINE Introduction Setup Foundation Practicing Tips Uncovered Topics Is Cython the best solution? 3
  • 16. INTRODUCTION
  • 17. CYTHON 5
  • 18. CYTHON Cython is a source-to-source compiler (aka. transcompiler). 5
  • 19. CYTHON Cython is a source-to-source compiler (aka. transcompiler). Cython is a superset of Python. 5
  • 20. CYTHON Cython is a source-to-source compiler (aka. transcompiler). Cython is a superset of Python. Provides optional static type declarations. 5
  • 21. CYTHON Cython is a source-to-source compiler (aka. transcompiler). Cython is a superset of Python. Provides optional static type declarations. Makes writing C extensions for Python easier. 5
  • 22. CYTHON Cython is a source-to-source compiler (aka. transcompiler). Cython is a superset of Python. Provides optional static type declarations. Makes writing C extensions for Python easier. Makes Python program faster by pre-compiling and static type.
 (sometimes faster by orders of magnitude) 5
  • 23. 6
  • 24. 6 .pyx
  • 25. 6 .pyx .c Cython
  • 26. 6 .pyx .c Cython C compiler .so
  • 27. 6 .pyx .c Cython C compiler .so .py
  • 28. 6 .pyx .c Cython C compiler .so .py Python
  • 29. 6 .pyx .c Cython C compiler import .so .py Python
  • 30. SETUP
  • 31. INSTALL C COMPILER 8
  • 32. INSTALL C COMPILER Mac: xcode-select --install 8
  • 33. INSTALL C COMPILER Mac: xcode-select --install Ubuntu / Debian: sudo apt-get install build-essential 8
  • 34. INSTALL C COMPILER Mac: xcode-select --install Ubuntu / Debian: sudo apt-get install build-essential Other: http://docs.cython.org/src/quickstart/install.html 8
  • 35. INSTALL CYTHON 9
  • 36. INSTALL CYTHON Recommend to use PIP: sudo pip install cython 9
  • 37. INSTALL CYTHON Recommend to use PIP: sudo pip install cython Other: http://docs.cython.org/src/quickstart/install.html 9
  • 38. THE SETUP.PY from distutils.core import setup from Cython.Build import cythonize ! setup(     name = 'cython-lab',     ext_modules = cythonize('*.pyx'), ) 10
  • 39. THE HELLO_CYTHON.PYX print 'Hello Cython!' 11
  • 40. BUILD 12
  • 41. BUILD Into package folder for development: python setup.py build_ext --inplace 12
  • 42. BUILD Into package folder for development: python setup.py build_ext --inplace Into system for production: python setup.py install 12
  • 43. BUILD Into package folder for development: python setup.py build_ext --inplace Into system for production: python setup.py install If clang: error: unknown argument: '-mno-fused-madd', export CFLAGS=-Qunused-arguments 12
  • 44. FOUNDATION — the difference of Cython and Python
  • 45. DEFINE STATICTYPES cdef int i, j, k cdef float f, g[42], *h 14
  • 46. cdef struct Grail: int age float volume ! cdef union Food: char* spam float* eggs ! cdef enum CheeseType: cheddar, edam, camembert ! cdef enum CheeseState: hard = 1 soft = 2 runny = 3 15
  • 47. ctypedef unsigned long ULong ! ctypedef int* IntPtr 16
  • 48. cdef struct Point: int x int y ! # either `struct` or `ctypedef` is not need cdef Point p 17
  • 49. cdef: struct Point: int x int y ! Point p 18
  • 50. DEFINE FUNCTION def say_hello(name='World'): return 'Hello, %s!' % name ! cdef say_hello(name='World'): return 'Hello, %s!' % name 19
  • 51. cdef say_hello(object name='World'): return 'Hello, %s!' % name ! cdef say_hello(char* name='World'): return 'Hello, %s!' % name 20
  • 52. cdef int add(int a, int b): return a+b ! cpdef say_hello(char* name='World'): return 'Hello, %s!' % name 21
  • 53. .PXD EXPOSES CDEF FUNC. # mylib.pxd cdef say_hello(char* name=?) ! # another.pyx from mylib cimport say_hello 22
  • 54. USING C LIB. from libc.math cimport sin # or cdef extern from "math.h": double sin(double x) 23
  • 55. FUNCTIONVISIBILITY 24 SAME FILE OTHER .PYX .PY FUNC. IN .H/.C Visible
 directly Visible
 via cdef extern Invisible CDEF Visible
 via .pxd & cimport CPDEF Visible
 via import DEF
  • 56. BINDING DURING 25 SAME FILE OTHER .PYX .PY FUNC. IN .H/.C compile-time compile-time x CDEF CPDEF run-time DEF
  • 57. TYPE CONVERSIONS 26 C FROM PY TO PY [UNSIGNED] CHAR/SHORT INT int/long int LONG UNSIGNED INT/LONG long [UNSIGNED] LONG LONG
  • 58. 27 C FROM PY TO PY FLOAT/DOUBLE int/long/float float LONG DOUBLE CHAR* str/bytes STRUCT dict
  • 59. PRACTICING
  • 60. SUGGESTION 29
  • 61. SUGGESTION LIB_NAME.pyx has an execute_self_tests function 29
  • 62. SUGGESTION LIB_NAME.pyx has an execute_self_tests function test_LIB_NAME.py call the execute_self_tests function 29
  • 63. OVERFLOW 30
  • 64. OVERFLOW Static types may also overflow in Cython silently. 30
  • 65. OVERFLOW Static types may also overflow in Cython silently. Try to make an overflow! 30
  • 66. OVERFLOW Static types may also overflow in Cython silently. Try to make an overflow! Hint: http://j.mp/test_overflow_in_c_c 30
  • 67. OVERFLOW Static types may also overflow in Cython silently. Try to make an overflow! Hint: http://j.mp/test_overflow_in_c_c Ans: http://j.mp/overflow_in_pyx_pyx 30
  • 68. FUNCTIONS 31
  • 69. FUNCTIONS Write three func. defined in def, cdef, and cpdef. 31
  • 70. FUNCTIONS Write three func. defined in def, cdef, and cpdef. Try to call them in the same file, another .pyx file, and a .py file. 31
  • 71. FUNCTIONS Write three func. defined in def, cdef, and cpdef. Try to call them in the same file, another .pyx file, and a .py file. Hints: Refer to the table,“Function Visibility”. http://j.mp/lib_in_pyx_pyx 31
  • 72. FUNCTIONS Write three func. defined in def, cdef, and cpdef. Try to call them in the same file, another .pyx file, and a .py file. Hints: Refer to the table,“Function Visibility”. http://j.mp/lib_in_pyx_pyx Ans: http://j.mp/use_lib_in_pyx_pyx http://j.mp/test_lib_in_pyx_py 31
  • 73. USING C FUNCTION 32
  • 74. USING C FUNCTION Try to use the functions in C. 32
  • 75. USING C FUNCTION Try to use the functions in C. Playing with fork, the system call, may be fun. 32
  • 76. USING C FUNCTION Try to use the functions in C. Playing with fork, the system call, may be fun. Hint: http://j.mp/test_fork_c 32
  • 77. USING C FUNCTION Try to use the functions in C. Playing with fork, the system call, may be fun. Hint: http://j.mp/test_fork_c Ans: http://j.mp/fork_in_pyx_pyx 32
  • 78. TIPS
  • 79. CYTHON -A 34
  • 80. CYTHON -A cython -a NAME.pyx 34
  • 81. CYTHON -A cython -a NAME.pyx open NAME.html 34
  • 82. CYTHON -A cython -a NAME.pyx open NAME.html Lines are colored according to the level of “typedness” – white lines translates to pure C without any Python API calls. 34
  • 83. PYXIMPORT import pyximport; pyximport.install() import my_pyx_lib # compile .pyx into .so ! # or pyximport.install(pyimport=True) import my_py_lib # compile .py into .so 35
  • 84. UNCOVERED 
 TOPICS
  • 85. UNCOVEREDTOPICS 37
  • 86. UNCOVEREDTOPICS Differences between C and Cython expressions http://docs.cython.org/src/userguide/ language_basics.html#differences-between-c-and-cython- expressions 37
  • 87. UNCOVEREDTOPICS Differences between C and Cython expressions http://docs.cython.org/src/userguide/ language_basics.html#differences-between-c-and-cython- expressions Propagating Exceptions in cdef http://docs.cython.org/src/userguide/ language_basics.html#error-return-values 37
  • 88. 38
  • 89. Extension Type — cdef class http://docs.cython.org/src/userguide/extension_types.html 38
  • 90. Extension Type — cdef class http://docs.cython.org/src/userguide/extension_types.html Generic programming using Cython's Template http://docs.cython.org/src/userguide/fusedtypes.html 38
  • 91. Extension Type — cdef class http://docs.cython.org/src/userguide/extension_types.html Generic programming using Cython's Template http://docs.cython.org/src/userguide/fusedtypes.html Conditional Compilation http://docs.cython.org/src/userguide/ language_basics.html#conditional-compilation 38
  • 92. 39
  • 93. Profiling http://docs.cython.org/src/tutorial/profiling_tutorial.html 39
  • 94. Profiling http://docs.cython.org/src/tutorial/profiling_tutorial.html Parallelism (No GIL + OpenMP) http://docs.cython.org/src/userguide/parallelism.html 39
  • 95. Profiling http://docs.cython.org/src/tutorial/profiling_tutorial.html Parallelism (No GIL + OpenMP) http://docs.cython.org/src/userguide/parallelism.html Using C++ in Cython http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html 39
  • 96. IS CYTHON 
 THE BEST SOLUTION?
  • 97. PYTHON 
 COMMUNITY 
 IS 
 DORAEMON!
  • 98. OTHER SOLUTIONS 42
  • 99. Boost.Python — exposes C++ to Python OTHER SOLUTIONS 42
  • 100. Boost.Python — exposes C++ to Python Numba — compiles annotated code into LLVM by JIT compiler OTHER SOLUTIONS 42
  • 101. Boost.Python — exposes C++ to Python Numba — compiles annotated code into LLVM by JIT compiler PyPy — speeds up existent code by JIT compiler OTHER SOLUTIONS 42
  • 102. Boost.Python — exposes C++ to Python Numba — compiles annotated code into LLVM by JIT compiler PyPy — speeds up existent code by JIT compiler NumPy or Blaze — provides efficient array OTHER SOLUTIONS 42
  • 103. Boost.Python — exposes C++ to Python Numba — compiles annotated code into LLVM by JIT compiler PyPy — speeds up existent code by JIT compiler NumPy or Blaze — provides efficient array SciPy — provides fast scientific computing OTHER SOLUTIONS 42
  • 104. COOL DOWN 43
  • 105. COOL DOWN Algorithm still does matter in any case. 43
  • 106. COOL DOWN Algorithm still does matter in any case. Profile your program. 43
  • 107. COOL DOWN Algorithm still does matter in any case. Profile your program. Consider the portability — you are writing C program! 43
  • 108. COOL DOWN Algorithm still does matter in any case. Profile your program. Consider the portability — you are writing C program! Consider the improvement is enough or not. 43
  • 109. COOL DOWN Algorithm still does matter in any case. Profile your program. Consider the portability — you are writing C program! Consider the improvement is enough or not. Then pick the most suitable tools. 43
  • 110. DEMO?
  • 111. ENDING
  • 112. ENDING 46
  • 113. ENDING cdef static types functions extern for C func. 46
  • 114. ENDING cdef static types functions extern for C func. .pxd exposes cdef func. 46
  • 115. ENDING cdef static types functions extern for C func. .pxd exposes cdef func. Tips 46
  • 116. ENDING cdef static types functions extern for C func. .pxd exposes cdef func. Tips mosky.tw 46
  • 117. ENDING cdef static types functions extern for C func. .pxd exposes cdef func. Tips mosky.tw Any question? 46