Python Bindings Overview

11,370 views

Published on

Python always got a good relation with the C language, through its syntax affinity or with its own API integrated with C.

Presentation's goal is to describe and compare several ways of doing bindings in C/C++ for Python which allow to augment Python features through speed improvements or giving access to a large ecosystem of C/C++ (or other) libs.

Following is presented : Python C API, ctypes, SWIG, Cython speaking about qualities and weak points.

Published in: Technology
1 Comment
7 Likes
Statistics
Notes
No Downloads
Views
Total views
11,370
On SlideShare
0
From Embeds
0
Number of Embeds
50
Actions
Shares
0
Downloads
127
Comments
1
Likes
7
Embeds 0
No embeds

No notes for slide

Python Bindings Overview

  1. 1. Python Bindings Overview<br />RudáMoura : ruda.moura@gmail.com<br />Sébastien Tandel : sebastien.tandel@gmail.com<br />
  2. 2. Python C API<br />PythonBrasil[5]<br />Caxias do Sul, RS - Brazil<br />
  3. 3. 3<br />Python C API : philosophy<br />Everything is a PyObject*<br />Public symbols with Py prefix<br />Private symbols with _Py prefix<br />Objects are stored in memory heap<br />It’s you job to count the references<br />Py_INCREF & Py_DECREF<br />Yeah, that’s sucks!<br />There are functions to convert C data types to Python C data types and back<br />PyArg_ParseTuple()<br />Py_BuildValue()<br />
  4. 4. Python C API<br />A wrapper around two readline functions:<br />readline - return a malloced string<br />add_history - add a (const) string to the history<br />char *readline(const char *);<br />intadd_history(const char *);<br />
  5. 5. Python C API<br />readline()<br />#include &lt;Python.h&gt;<br />#include &lt;readline/readline.h&gt;<br />PyObject *<br />fn_readline(PyObject *self, PyObject *args)<br />{<br /> char *prompt;<br /> char *line;<br />PyObject *o;<br /> if (!PyArg_ParseTuple(args, &quot;s&quot;, &prompt))<br /> return NULL;<br />line = readline(prompt);<br />o = Py_BuildValue(&quot;s&quot;, line);<br />free(line);<br /> return o;<br />}<br />
  6. 6. Python C API<br />add_history()<br />PyObject *<br />fn_add_history(PyObject *self, PyObject *args)<br />{<br /> char *line;<br />int status;<br />PyObject *o;<br /> if (!PyArg_ParseTuple(args, &quot;s&quot;, &line))<br /> return NULL;<br />status = add_history(line);<br />o = Py_BuildValue(&quot;i&quot;, status);<br /> return o;<br />}<br />And if I want to return None?<br />Py_INCREF(Py_None); <br />return Py_None;<br />
  7. 7. Python C API<br />Registering functions and modules<br />static PyMethodDef<br />methods[] = {<br /> {&quot;readline&quot;, fn_readline, METH_VARARGS, NULL},<br /> {&quot;add_history&quot;, fn_add_history, METH_VARARGS, NULL},<br /> {NULL, NULL, 0, NULL}<br />};<br />PyMODINIT_FUNC<br />initreadline(void)<br />{<br /> (void) Py_InitModule(&quot;readline&quot;, methods);<br />}<br />
  8. 8. Python C API<br />Compiling...<br />$ gcc -dynamiclib -I/usr/include/python2.5 -lpython -lreadline mod_readline.c -o readline.so<br />Or you can use distutils / setup.py<br />
  9. 9. Python C API<br />The goods<br />Strong C/C++ and Python integration<br />The best performance possible<br />Full Python data types support<br />The bads<br />It’s hard work, only good for masochists<br />You’re supposed to be good in C<br />Reference counts nightmare<br />
  10. 10. Python C API : summary<br />
  11. 11. SWIG<br />Simplified Wrapper and Interface Generator<br />
  12. 12. SWIG : philosophy<br />Everything is defined in the interface file<br />Module name<br />Literal #include or source code<br />Define What symbols to export<br />The swig compiler translates the interface file into C source code<br />
  13. 13. SWIG<br />The interface file<br />// readline.i<br />%module readline<br />%{<br />#include &lt;readline/readline.h&gt;<br />%}<br />char *readline(const char *);<br />intadd_history(const char *);<br />
  14. 14. SWIG<br />Compiling...<br />$ swig -python readline.i<br />$ gcc -creadline_wrap.c -I/usr/include/python2.5<br />$ gcc -dynamiclibreadline_wrap.o -lpython -lreadline -o _readline.so<br />Or you can use distutils / setup.py<br />
  15. 15. SWIG<br />The goods<br />Full bindings process automatization<br />Strong C++ support (class, template, exception)<br />Not only for Python! Support for Perl, Ruby and others<br />It’s mature (since 1995)<br />The bads<br />The C file created is not for human consum<br />Callbacks and double references are not easy<br />
  16. 16. SWIG : summary<br />
  17. 17. CTYPES<br />
  18. 18. Ctypes : philosophy<br />Ctypes is a dlopen-like for Python<br />The basic steps :<br />Obtain a handle of a library<br />Use this handle to access library’s functions<br />from ctypes import cdll<br />libc_h = cdll.LoadLibrary(“libc.so.6”)<br />libc_h.mkdir(“python-mkdir-test”)<br />
  19. 19. Ctypes<br />OK, it’s a dlopen-like, and?<br />libc_h.mkdir(“python-mkdir-test”)<br /><ul><li>It’s a Python string!</li></ul>Transparent conversion for these types :<br />None, integers, long, byte strings and unicode strings<br />
  20. 20. Ctypes<br />What if I want specific C types?<br />c_int, c_long, c_char, c_short, c_ulong, …<br />i = c_int(10)<br />print i.value()<br />10<br />
  21. 21. Ctypes<br />Functions return values?<br />By default, assumed to be int<br />from ctypes import cdll, c_char_p<br />lr = cdll.LoadLibrary(&quot;libreadline.dylib&quot;)<br />while True:<br /> line = lr.readline(“C:&gt;&quot;)<br />c_line = c_char_p(line)<br /> if not c_line.value: break<br />
  22. 22. Ctypes<br />Return value as input for another C function?<br />from ctypes import cdll, c_char_p<br />lr = cdll.LoadLibrary(&quot;libreadline.dylib&quot;)<br />while True:<br /> line = lr.readline(“C:&gt;&quot;)<br />c_line = c_char_p(line)<br /> if not c_line.value: break<br />lr.add_history(c_line.value) # or<br />lr.add_history(c_line) # or<br />lr.add_history(line)<br />
  23. 23. Ctypes<br />Is there a way to simplify a bit?<br />from ctypes import cdll, c_char_p<br />lr = cdll.LoadLibrary(&quot;libreadline.dylib&quot;)<br />lr.readline.restype = c_char_p<br />while True:<br /> line = lr.readline(&quot;c:&gt;&quot;)<br /> if not line: break<br />lr.add_history(line)<br />You can define types for arguments too with .argtypes!<br />
  24. 24. Ctypes<br />Possible to access structures returned by C functions<br />Possible to define arrays … but with a fixed length!<br />Possible to create callback written in Python called by C lib.<br />On windows, check the number of parameters passed to a function.<br />Part of Python 2.5<br />
  25. 25. Ctypes : summary<br />
  26. 26. Cython<br />
  27. 27. Cython : philosophy<br />Cython is an extension of Python language<br />Three simple steps :<br />Write “python”<br />Compile the module it with C compiler<br />Import the package in your python code<br />
  28. 28. Cython<br />Compiling a module =&gt; setuptools<br />print “hello world!”<br />setup.py<br />from distutils.core import setup<br />from distutils.extension import Extension<br />from Cython.Distutils import build_ext<br />setup(<br />cmdclass = {&apos;build_ext&apos;: build_ext},<br />ext_modules = [Extension(”hello&quot;, [”hello.pyx&quot;])]<br />)<br />
  29. 29. Cython<br />cdef extern from &quot;readline/readline.h&quot;:<br /> char* readline(char*)<br />intadd_history(char*)<br />def readline_loop():<br /> while True:<br /> line = readline(&quot;c:&gt;”)<br /> if not line: break<br />add_history(line)<br />
  30. 30. Cython<br />setup.py<br />from distutils.core import setup<br />from distutils.extension import Extension<br />from Cython.Distutils import build_ext<br />setup(<br />cmdclass = {&apos;build_ext&apos;: build_ext},<br />ext_modules = [Extension(‘readlineloop’, <br /> [‘readline-loop.pyx’],<br />libraries = [‘readline’])]<br />)<br />
  31. 31. Cython<br />Last details :<br />Can access C structures<br />Possible to define callbacks<br />Check on number of args<br />Possible to wrap C++ classes!<br />Handle exceptions!<br />Can be used to optimize your code<br /><ul><li>need to learn a bit more</li></li></ul><li>Cython<br />Some limitations :<br />No support to yield<br />No nested def<br />No globals(), locals()<br />Class / func can’t be placed inside control structures<br />
  32. 32. Cython : summary<br />
  33. 33. Conclusions<br />
  34. 34. Thank You!<br />ruda.moura@gmail.com<br />sebastien.tandel@gmail.com<br />

×