Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Python Bindings Overview

11,907 views

Published on

Python always got a good relation with the C language, through its syntax affinity or with its own API integrated with C.

Presentation's goal is to describe and compare several ways of doing bindings in C/C++ for Python which allow to augment Python features through speed improvements or giving access to a large ecosystem of C/C++ (or other) libs.

Following is presented : Python C API, ctypes, SWIG, Cython speaking about qualities and weak points.

Published in: Technology
  • Login to see the comments

Python Bindings Overview

  1. 1. Python Bindings Overview<br />RudáMoura : ruda.moura@gmail.com<br />Sébastien Tandel : sebastien.tandel@gmail.com<br />
  2. 2. Python C API<br />PythonBrasil[5]<br />Caxias do Sul, RS - Brazil<br />
  3. 3. 3<br />Python C API : philosophy<br />Everything is a PyObject*<br />Public symbols with Py prefix<br />Private symbols with _Py prefix<br />Objects are stored in memory heap<br />It’s you job to count the references<br />Py_INCREF & Py_DECREF<br />Yeah, that’s sucks!<br />There are functions to convert C data types to Python C data types and back<br />PyArg_ParseTuple()<br />Py_BuildValue()<br />
  4. 4. Python C API<br />A wrapper around two readline functions:<br />readline - return a malloced string<br />add_history - add a (const) string to the history<br />char *readline(const char *);<br />intadd_history(const char *);<br />
  5. 5. Python C API<br />readline()<br />#include &lt;Python.h&gt;<br />#include &lt;readline/readline.h&gt;<br />PyObject *<br />fn_readline(PyObject *self, PyObject *args)<br />{<br /> char *prompt;<br /> char *line;<br />PyObject *o;<br /> if (!PyArg_ParseTuple(args, &quot;s&quot;, &prompt))<br /> return NULL;<br />line = readline(prompt);<br />o = Py_BuildValue(&quot;s&quot;, line);<br />free(line);<br /> return o;<br />}<br />
  6. 6. Python C API<br />add_history()<br />PyObject *<br />fn_add_history(PyObject *self, PyObject *args)<br />{<br /> char *line;<br />int status;<br />PyObject *o;<br /> if (!PyArg_ParseTuple(args, &quot;s&quot;, &line))<br /> return NULL;<br />status = add_history(line);<br />o = Py_BuildValue(&quot;i&quot;, status);<br /> return o;<br />}<br />And if I want to return None?<br />Py_INCREF(Py_None); <br />return Py_None;<br />
  7. 7. Python C API<br />Registering functions and modules<br />static PyMethodDef<br />methods[] = {<br /> {&quot;readline&quot;, fn_readline, METH_VARARGS, NULL},<br /> {&quot;add_history&quot;, fn_add_history, METH_VARARGS, NULL},<br /> {NULL, NULL, 0, NULL}<br />};<br />PyMODINIT_FUNC<br />initreadline(void)<br />{<br /> (void) Py_InitModule(&quot;readline&quot;, methods);<br />}<br />
  8. 8. Python C API<br />Compiling...<br />$ gcc -dynamiclib -I/usr/include/python2.5 -lpython -lreadline mod_readline.c -o readline.so<br />Or you can use distutils / setup.py<br />
  9. 9. Python C API<br />The goods<br />Strong C/C++ and Python integration<br />The best performance possible<br />Full Python data types support<br />The bads<br />It’s hard work, only good for masochists<br />You’re supposed to be good in C<br />Reference counts nightmare<br />
  10. 10. Python C API : summary<br />
  11. 11. SWIG<br />Simplified Wrapper and Interface Generator<br />
  12. 12. SWIG : philosophy<br />Everything is defined in the interface file<br />Module name<br />Literal #include or source code<br />Define What symbols to export<br />The swig compiler translates the interface file into C source code<br />
  13. 13. SWIG<br />The interface file<br />// readline.i<br />%module readline<br />%{<br />#include &lt;readline/readline.h&gt;<br />%}<br />char *readline(const char *);<br />intadd_history(const char *);<br />
  14. 14. SWIG<br />Compiling...<br />$ swig -python readline.i<br />$ gcc -creadline_wrap.c -I/usr/include/python2.5<br />$ gcc -dynamiclibreadline_wrap.o -lpython -lreadline -o _readline.so<br />Or you can use distutils / setup.py<br />
  15. 15. SWIG<br />The goods<br />Full bindings process automatization<br />Strong C++ support (class, template, exception)<br />Not only for Python! Support for Perl, Ruby and others<br />It’s mature (since 1995)<br />The bads<br />The C file created is not for human consum<br />Callbacks and double references are not easy<br />
  16. 16. SWIG : summary<br />
  17. 17. CTYPES<br />
  18. 18. Ctypes : philosophy<br />Ctypes is a dlopen-like for Python<br />The basic steps :<br />Obtain a handle of a library<br />Use this handle to access library’s functions<br />from ctypes import cdll<br />libc_h = cdll.LoadLibrary(“libc.so.6”)<br />libc_h.mkdir(“python-mkdir-test”)<br />
  19. 19. Ctypes<br />OK, it’s a dlopen-like, and?<br />libc_h.mkdir(“python-mkdir-test”)<br /><ul><li>It’s a Python string!</li></ul>Transparent conversion for these types :<br />None, integers, long, byte strings and unicode strings<br />
  20. 20. Ctypes<br />What if I want specific C types?<br />c_int, c_long, c_char, c_short, c_ulong, …<br />i = c_int(10)<br />print i.value()<br />10<br />
  21. 21. Ctypes<br />Functions return values?<br />By default, assumed to be int<br />from ctypes import cdll, c_char_p<br />lr = cdll.LoadLibrary(&quot;libreadline.dylib&quot;)<br />while True:<br /> line = lr.readline(“C:&gt;&quot;)<br />c_line = c_char_p(line)<br /> if not c_line.value: break<br />
  22. 22. Ctypes<br />Return value as input for another C function?<br />from ctypes import cdll, c_char_p<br />lr = cdll.LoadLibrary(&quot;libreadline.dylib&quot;)<br />while True:<br /> line = lr.readline(“C:&gt;&quot;)<br />c_line = c_char_p(line)<br /> if not c_line.value: break<br />lr.add_history(c_line.value) # or<br />lr.add_history(c_line) # or<br />lr.add_history(line)<br />
  23. 23. Ctypes<br />Is there a way to simplify a bit?<br />from ctypes import cdll, c_char_p<br />lr = cdll.LoadLibrary(&quot;libreadline.dylib&quot;)<br />lr.readline.restype = c_char_p<br />while True:<br /> line = lr.readline(&quot;c:&gt;&quot;)<br /> if not line: break<br />lr.add_history(line)<br />You can define types for arguments too with .argtypes!<br />
  24. 24. Ctypes<br />Possible to access structures returned by C functions<br />Possible to define arrays … but with a fixed length!<br />Possible to create callback written in Python called by C lib.<br />On windows, check the number of parameters passed to a function.<br />Part of Python 2.5<br />
  25. 25. Ctypes : summary<br />
  26. 26. Cython<br />
  27. 27. Cython : philosophy<br />Cython is an extension of Python language<br />Three simple steps :<br />Write “python”<br />Compile the module it with C compiler<br />Import the package in your python code<br />
  28. 28. Cython<br />Compiling a module =&gt; setuptools<br />print “hello world!”<br />setup.py<br />from distutils.core import setup<br />from distutils.extension import Extension<br />from Cython.Distutils import build_ext<br />setup(<br />cmdclass = {&apos;build_ext&apos;: build_ext},<br />ext_modules = [Extension(”hello&quot;, [”hello.pyx&quot;])]<br />)<br />
  29. 29. Cython<br />cdef extern from &quot;readline/readline.h&quot;:<br /> char* readline(char*)<br />intadd_history(char*)<br />def readline_loop():<br /> while True:<br /> line = readline(&quot;c:&gt;”)<br /> if not line: break<br />add_history(line)<br />
  30. 30. Cython<br />setup.py<br />from distutils.core import setup<br />from distutils.extension import Extension<br />from Cython.Distutils import build_ext<br />setup(<br />cmdclass = {&apos;build_ext&apos;: build_ext},<br />ext_modules = [Extension(‘readlineloop’, <br /> [‘readline-loop.pyx’],<br />libraries = [‘readline’])]<br />)<br />
  31. 31. Cython<br />Last details :<br />Can access C structures<br />Possible to define callbacks<br />Check on number of args<br />Possible to wrap C++ classes!<br />Handle exceptions!<br />Can be used to optimize your code<br /><ul><li>need to learn a bit more</li></li></ul><li>Cython<br />Some limitations :<br />No support to yield<br />No nested def<br />No globals(), locals()<br />Class / func can’t be placed inside control structures<br />
  32. 32. Cython : summary<br />
  33. 33. Conclusions<br />
  34. 34. Thank You!<br />ruda.moura@gmail.com<br />sebastien.tandel@gmail.com<br />

×