This document discusses different approaches for creating Python extensions and bindings to C/C++ libraries. It summarizes the author's experience using ctypes to create a minimal binding called PyMiniRacer to the V8 JavaScript engine. The author argues that combining ctypes, which allows shipping a single Python-independent binary, with pre-built wheel distributions can provide an optimal solution for packaging and distributing Python extensions.

1. Writing a C Python
extension in 2018
Paris.py 2018

2. Jean-Baptiste Aviat
CTO & Co-founder of sqreen.io
Former Hacker at Apple (Red Team)
PyMiniRacer author
jb@sqreen.io
@jbaviat
Who Am I?

3. Someday… we needed to
use V8 from Python.
What we ship:
• is public
• is widely used
• need to have frictionless install
$ pip install sqreen
—> it just works™!

5. PyMiniRacer: cool JS binding
github.com/sqreen/PyMiniRacer

6. The problem
• V8 is C++
• How do you run C++ in Python?
…We need some kind of binding
between these 2 worlds.

7. Many popular
packages rely
on binary
cryptography
numpy
pymongo
psycopg
simplejson
lxml
sqlalchemy…

8. People do it!
Let’s do it too.

9. What are our goals?
• minimize maintenance
• make setup easy
• make testing easy
• have great performance
• have low memory fingerprint
• dev time is a constraint
We want to: And (obviously)…

10. built-in pythonic
Python version
independant
open to other
languages
high
throughput
capable
CPython
✔ ✔ ✔
ctypes ✔ ✔ ✔
cffi ✔ ✔ ✔ ✔
Cython ✔ ✔
SWIG ✔ ✔

11. ctypes
Built into Python
Binary is Python independant:
• can be used on any Python
version
• can be used in other languages!
No tight integration to Python
• not high throughput
capable
• less Pythonic
Complex syntax (C types
wrapped in Python…)
Not for C++
👍 👎

12. $ python
>>> path = "./hello.so"
>>> import ctypes
>>> lib = ctypes.cdll.LoadLibrary(path)
>>> lib.hello_world()
Hello world!
C file
Python
interface
binary
object

13. Overview
V8 (C++ interface)C interface to V8Python interface
3rd party
binaries
import ctypes
class PyMiniRacer(object):
…
#include <v8.h>
int miniracer_init();
…
V8 library (libv8.a)
V8 headers (v8.h)
linkingctypes
C/C++
code
Python
library

14. How to put this together?
$ cat setup.py
from distutils.core import setup, Extension
extension = Extension('hello', ['hello.c'])
setup(name=‘hello',
version='1.0',
ext_modules=[extension])
$ python setup.py build
running build
running build_ext
building 'hello' extension
clang […] -c hello.c -o hello.o
creating build/lib.macosx-10.6-intel-2.7
clang -bundle […] hello.o -o hello.so

17. Ship it

18. Ever had trouble installing packages?

20. (at Sqreen, we are
dog friendly)
Come get your dog sticker!

21. Python packaging
history
sdist (source distribution)
eggs
wheels
—> manylinux wheels
(built distribution)
2004
2012
2016
Python 2.4
Python 3.3
Python 3.6 ❤

22. manylinux wheels
Python standard:
PEP513 / PEP571
Compatible on most
(real world) Linux
Only in pip >= 8.1
Need build for all Python versions
Binaries need to be built on
🚨🚨CentOS 5🚨🚨
👍 👎

23. CentOS 5??
PEP571 → bumped requirement
to CentOS 6!

24. Wheels vs compilation
iso builds (crash can
be reproduced)
many packages
need maintenance
one build per user
only one package
harder to install…

25. Best of 2 worlds
We can have:
ctypes: use a Python independant binary
wheels: ship compiled binaries
By combining these…
🥂🍾 ctypes + wheels = ship only one binary

26. Thanks!
Questions?
We’re
hiring!
https://sqreen.io/jobs