Python with a of C++
Bob McNaughton
PyData NYC 2015
• Simplified Wrapper and Interface Generator
• Allow the use of C/C++ from other languages
• http://swig.org/
• Originally developed by Dave Beazly
• First released in February 1996
• Latest release 3.0.7, August 3rd, 2015
• Actively maintained, several releases a year
What is SWIG?
What other languages?
$ swig -help
Target Language Options
-allegrocl - Generate ALLEGROCL wrappers
-chicken - Generate CHICKEN wrappers
-clisp - Generate CLISP wrappers
-cffi - Generate CFFI wrappers
-csharp - Generate C# wrappers
-guile - Generate Guile wrappers
-java - Generate Java wrappers
-lua - Generate Lua wrappers
-modula3 - Generate Modula 3 wrappers
-mzscheme - Generate Mzscheme wrappers
-ocaml - Generate Ocaml wrappers
-perl - Generate Perl wrappers
-php - Generate PHP wrappers
-pike - Generate Pike wrappers
-python - Generate Python wrappers
-ruby - Generate Ruby wrappers
-sexp - Generate Lisp S-Expressions wrappers
-tcl - Generate Tcl wrappers
-uffi - Generate Common Lisp / UFFI wrappers
-xml - Generate XML wrappers
Are there any alternatives?
• Cython
• cTypes
• Cpython C API
Why would you ever want to call C/C++ from Python?
• Speed
• Existing C/C++ libraries
• With or without source
• Develop faster initially with Python
• Speed up parts that the profiler shows as critical
C++ Fibonacci
if (n < 0)
nextTerm = -1;
else if (n == 0)
nextTerm = 0;
else if (n == 1)
nextTerm = 1;
else {
firstTerm = 0;
secondTerm = 1;
for (int i = 0; i < n-1; ++i) {
nextTerm = firstTerm + secondTerm;
firstTerm = secondTerm;
secondTerm = nextTerm;
}
}
Python Fibonacci
def fib(n):
if (n < 0):
return -1
elif (n == 0):
return 0
elif (n == 1):
return 1
else:
a,b = 0,1
for i in range(n-1):
a,b = b,a+b
return a
How do they compare?
• Had to run both multiple times to get something measurable
• C++ significantly faster
• Python doesn’t overflow
C++ Library routine
long long fib(int n)
{
long long firstTerm = 0, secondTerm = 1, nextTerm;
if (n < 0)
return -1;
if (n == 0)
return 0;
if (n == 1)
return 1;
for (int i = 1; i < n-1; ++i) {
nextTerm = firstTerm + secondTerm;
if (nextTerm < secondTerm) // Overflow!
return -1;
firstTerm = secondTerm;
secondTerm = nextTerm;
}
return nextTerm;
}
C++ Header File
long long fib(int n);
Make a shared object or dll
g++ -g –c –o fibl.o fibl.cpp
g++ -g –shared –fPIC –Wl,-soname,libfib.so –o libfib.so fibl.o
Use this .so from C++
#include <iostream>
#include <stdlib.h>
#include "fib.h"
using namespace std;
int main(int argc, char* argv[]) {
if (argc < 2)
{
cout << "Usage: " << argv[0] << " <fibonacci number to print>" << endl;
return -1;
}
long n = strtol(argv[1], NULL, 10);
for (int i=0; i<100000; i++) {
fib(n);
}
cout << "Fibonacci Number " << n << " is " << fib(n) << endl;
return 0;
}
Compile with:
g++ -g –L. –lfib –o fibm fibm.cpp
How does it compare to no .so?
Slightly slower
Still overflows
Now call the same .so from Python
SWIG interface file, fibl.i
%module fibl
%{
#include "fib.h"
%}
%include "fib.h"
Run SWIG on the interface file
swig –python fibl.i
Produces fibl_wrap.c and fibl.py
Turn the SWIG outputs into a .so
g++ -g –fpic –c fibl_wrap.c –I /usr/include/python3.5m
g++ -g –shared –L. –lfib fibl_wrap.o –o _fibl.so
$ ldd _fibl.so
linux-vdso.so.1 (0x00007ffdd6d9e000)
libfib.so => ./libfib.so (0x00007fcbdc07e000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fcbdbcc9000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007fcbdb9cb000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fcbdb7b4000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007fcbdb410000)
/usr/lib64/ld-linux-x86-64.so.2 (0x000055f6c1f6c000)
Now call the .so from Python
import sys
import fibl
x=int(sys.argv[1])
for i in range (1, 100000):
fibl.fib(x)
print("Fibonacci number", x, "is", fibl.fib(x))
Now how do the results compare?
C++ still faster, but it is much closer now
Python now overflows
What about a more complicated example?
#include <rw/sortvec.h>
#include <rw/collstr.h>
int main ()
{
RWSortedVector sv;
sv.insert(new RWCollectableString("dog"));
sv.insert(new RWCollectableString("cat"));
sv.insert(new RWCollectableString("fish"));
RWSortedVectorIterator next(sv);
RWCollectableString* item;
while( (item = (RWCollectableString*)next() ) != 0)
std::cout << *item << std::endl;
sv.clearAndDestroy();
return 0;
}
Try the obvious SWIG interface file
%module fibl
%{
#include <rw/sortvec.h>
#include <rw/collstr.h>
%}
%include <rw/sortvec.h>
%include <rw/collstr.h>
Conclusion
• Works fine for C
• C++ can be a problem.
• The only option if all you have is a header file and a dynamic library

Python with a SWIG of c++

  • 1.
    Python with aof C++ Bob McNaughton PyData NYC 2015
  • 2.
    • Simplified Wrapperand Interface Generator • Allow the use of C/C++ from other languages • http://swig.org/ • Originally developed by Dave Beazly • First released in February 1996 • Latest release 3.0.7, August 3rd, 2015 • Actively maintained, several releases a year What is SWIG?
  • 3.
    What other languages? $swig -help Target Language Options -allegrocl - Generate ALLEGROCL wrappers -chicken - Generate CHICKEN wrappers -clisp - Generate CLISP wrappers -cffi - Generate CFFI wrappers -csharp - Generate C# wrappers -guile - Generate Guile wrappers -java - Generate Java wrappers -lua - Generate Lua wrappers -modula3 - Generate Modula 3 wrappers -mzscheme - Generate Mzscheme wrappers -ocaml - Generate Ocaml wrappers -perl - Generate Perl wrappers -php - Generate PHP wrappers -pike - Generate Pike wrappers -python - Generate Python wrappers -ruby - Generate Ruby wrappers -sexp - Generate Lisp S-Expressions wrappers -tcl - Generate Tcl wrappers -uffi - Generate Common Lisp / UFFI wrappers -xml - Generate XML wrappers
  • 4.
    Are there anyalternatives? • Cython • cTypes • Cpython C API
  • 5.
    Why would youever want to call C/C++ from Python? • Speed • Existing C/C++ libraries • With or without source • Develop faster initially with Python • Speed up parts that the profiler shows as critical
  • 6.
    C++ Fibonacci if (n< 0) nextTerm = -1; else if (n == 0) nextTerm = 0; else if (n == 1) nextTerm = 1; else { firstTerm = 0; secondTerm = 1; for (int i = 0; i < n-1; ++i) { nextTerm = firstTerm + secondTerm; firstTerm = secondTerm; secondTerm = nextTerm; } }
  • 7.
    Python Fibonacci def fib(n): if(n < 0): return -1 elif (n == 0): return 0 elif (n == 1): return 1 else: a,b = 0,1 for i in range(n-1): a,b = b,a+b return a
  • 8.
    How do theycompare? • Had to run both multiple times to get something measurable • C++ significantly faster • Python doesn’t overflow
  • 9.
    C++ Library routine longlong fib(int n) { long long firstTerm = 0, secondTerm = 1, nextTerm; if (n < 0) return -1; if (n == 0) return 0; if (n == 1) return 1; for (int i = 1; i < n-1; ++i) { nextTerm = firstTerm + secondTerm; if (nextTerm < secondTerm) // Overflow! return -1; firstTerm = secondTerm; secondTerm = nextTerm; } return nextTerm; }
  • 10.
    C++ Header File longlong fib(int n);
  • 11.
    Make a sharedobject or dll g++ -g –c –o fibl.o fibl.cpp g++ -g –shared –fPIC –Wl,-soname,libfib.so –o libfib.so fibl.o
  • 12.
    Use this .sofrom C++ #include <iostream> #include <stdlib.h> #include "fib.h" using namespace std; int main(int argc, char* argv[]) { if (argc < 2) { cout << "Usage: " << argv[0] << " <fibonacci number to print>" << endl; return -1; } long n = strtol(argv[1], NULL, 10); for (int i=0; i<100000; i++) { fib(n); } cout << "Fibonacci Number " << n << " is " << fib(n) << endl; return 0; }
  • 13.
    Compile with: g++ -g–L. –lfib –o fibm fibm.cpp
  • 14.
    How does itcompare to no .so? Slightly slower Still overflows
  • 15.
    Now call thesame .so from Python SWIG interface file, fibl.i %module fibl %{ #include "fib.h" %} %include "fib.h"
  • 16.
    Run SWIG onthe interface file swig –python fibl.i Produces fibl_wrap.c and fibl.py
  • 17.
    Turn the SWIGoutputs into a .so g++ -g –fpic –c fibl_wrap.c –I /usr/include/python3.5m g++ -g –shared –L. –lfib fibl_wrap.o –o _fibl.so $ ldd _fibl.so linux-vdso.so.1 (0x00007ffdd6d9e000) libfib.so => ./libfib.so (0x00007fcbdc07e000) libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fcbdbcc9000) libm.so.6 => /usr/lib/libm.so.6 (0x00007fcbdb9cb000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fcbdb7b4000) libc.so.6 => /usr/lib/libc.so.6 (0x00007fcbdb410000) /usr/lib64/ld-linux-x86-64.so.2 (0x000055f6c1f6c000)
  • 18.
    Now call the.so from Python import sys import fibl x=int(sys.argv[1]) for i in range (1, 100000): fibl.fib(x) print("Fibonacci number", x, "is", fibl.fib(x))
  • 19.
    Now how dothe results compare? C++ still faster, but it is much closer now Python now overflows
  • 20.
    What about amore complicated example? #include <rw/sortvec.h> #include <rw/collstr.h> int main () { RWSortedVector sv; sv.insert(new RWCollectableString("dog")); sv.insert(new RWCollectableString("cat")); sv.insert(new RWCollectableString("fish")); RWSortedVectorIterator next(sv); RWCollectableString* item; while( (item = (RWCollectableString*)next() ) != 0) std::cout << *item << std::endl; sv.clearAndDestroy(); return 0; }
  • 21.
    Try the obviousSWIG interface file %module fibl %{ #include <rw/sortvec.h> #include <rw/collstr.h> %} %include <rw/sortvec.h> %include <rw/collstr.h>
  • 22.
    Conclusion • Works finefor C • C++ can be a problem. • The only option if all you have is a header file and a dynamic library