Thinking Hybrid - Python/C++ Integration

Slide 1: Python Extending/Integrating A Real World Example Tips Summary Python where we can, C++ where we must Source: http://xkcd.com/413/ Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 1/33

Slide 2: Python Extending/Integrating A Real World Example Tips Summary Thinking Hybrid – Python/C++ Integration (Python where we can, C++ where we must∗ ) Guy K. Kloss Computer Science Massey University, Albany Computer Science/Information Technology Seminar Auckland, 23 April 2008 ∗ Quote: Alex Martelli, Senior Google Developer Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 2/33

Slide 3: Python Extending/Integrating A Real World Example Tips Summary Source: http://xkcd.com/413/ Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 3/33

Slide 4: Python Extending/Integrating A Real World Example Tips Summary Outline 1 Python 2 Extending/Integrating 3 A Real World Example 4 Tips Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 4/33

Slide 5: Python Extending/Integrating A Real World Example Tips Summary Outline 1 Python 2 Extending/Integrating 3 A Real World Example 4 Tips Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 5/33

Slide 6: Python Extending/Integrating A Real World Example Tips Summary Why Python? C/C++ Python • dynamic • static • 50 % less code, 300 % pro- • high performance ductivity • extendable with C/C++ • extendable with Python • platform independent • close to hardware • garbage collector • small memory footprint • huge standard library • many other bindings While others are still debugging, you can publish your results! Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 6/33

Slide 7: Python Extending/Integrating A Real World Example Tips Summary Why Python? Source: http://xkcd.com/371/ Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 7/33

Slide 8: Python Extending/Integrating A Real World Example Tips Summary What is Python used for? System Utilities System admin tools, portable shell scripts Internet Scripting CGI scripts, parse HTML, process XML, email tools User Interfaces (UIs) & rapid prototyping Component Glue Scripting for apps, COM scripting Distributed Programming Web Services, COM, CORBA, XML–RPC Database Programming Scientific Computing PyODE, NumPy, SciPy, PyMol, . . . Image Processing Python Image Library, OpenCV OpenGL Programming, Writing Games PyOpenGL, Panda3D, PyOgre, Py3d, VisualPython Artifical Intelligence Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 8/33

Slide 9: Python Extending/Integrating A Real World Example Tips Summary Outline 1 Python 2 Extending/Integrating 3 A Real World Example 4 Tips Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 9/33

Slide 10: Python Extending/Integrating A Real World Example Tips Summary Extending/ Integrating Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 10/33

Slide 11: Python Extending/Integrating A Real World Example Tips Summary What if I could . . . Use this code more effectively . . . ? [NaSt2D demonstration (native executable)] Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 11/33

Slide 12: Python Extending/Integrating A Real World Example Tips Summary Extending/Integration The Contestants: The “classic way” . . . Extending and Embedding the Python Interpreter The “new way” . . . Python ctypes SWIG Boost.Python Others: SIP Pythonizer SILOON (Pyrex) Extensions also for Java, C#, Fortran, . . . available Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 12/33

Slide 13: Python Extending/Integrating A Real World Example Tips Summary Extending/Integration The Contestants: The “classic way” . . . Extending and Embedding the Python Interpreter The “new way” . . . Python ctypes SWIG Boost.Python Others: SIP Pythonizer SILOON (Pyrex) Extensions also for Java, C#, Fortran, . . . available Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 12/33

Slide 14: Python Extending/Integrating A Real World Example Tips Summary Extending/Integration The Contestants: The “classic way” . . . Extending and Embedding the Python Interpreter The “new way” . . . Python ctypes SWIG Boost.Python Others: SIP Pythonizer SILOON (Pyrex) Extensions also for Java, C#, Fortran, . . . available Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 12/33

Slide 15: Python Extending/Integrating A Real World Example Tips Summary Extending/Integration The Contestants: The “classic way” . . . Extending and Embedding the Python Interpreter The “new way” . . . Python ctypes SWIG Boost.Python Others: SIP Pythonizer SILOON (Pyrex) Extensions also for Java, C#, Fortran, . . . available Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 12/33

Slide 16: Python Extending/Integrating A Real World Example Tips Summary Extending/Integration The Contestants: The “classic way” . . . Extending and Embedding the Python Interpreter The “new way” . . . Python ctypes SWIG Boost.Python Others: SIP Pythonizer SILOON (Pyrex) Extensions also for Java, C#, Fortran, . . . available Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 12/33

Slide 17: Python Extending/Integrating A Real World Example Tips Summary Boost.Python Thinking Hybrid with Boost.Python Bottom up and . . . Top down possible Develop quickly Resolve bottle necks Donald Knuth’s: “Premature optimisation is the root of all evil.” or: Don’t Optimise Now! Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 13/33

Slide 18: Python Extending/Integrating A Real World Example Tips Summary Boost.Python Thinking Hybrid with Boost.Python Bottom up and . . . Top down possible Develop quickly Resolve bottle necks Donald Knuth’s: “Premature optimisation is the root of all evil.” or: Don’t Optimise Now! Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 13/33

Slide 19: Python Extending/Integrating A Real World Example Tips Summary Boost.Python Thinking Hybrid with Boost.Python Bottom up and . . . Top down possible Develop quickly Resolve bottle necks Donald Knuth’s: “Premature optimisation is the root of all evil.” or: Don’t Optimise Now! Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 13/33

Slide 20: Python Extending/Integrating A Real World Example Tips Summary Elegantly with Boost.Python Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 14/33

Slide 21: Python Extending/Integrating A Real World Example Tips Summary Hello World char const* greet(unsigned x) { static char const* const msgs[] = {"hello", "Boost.Python", "world!"}; if (x > 2) { throw std::range error("greet: Index out of range."); } return msgs[x]; } #include <boost/python.hpp> using namespace boost::python; BOOST PYTHON MODULE(hello) { def("greet", greet, "return one of 3 parts of a greeting"); } And here it is in action: >>> import hello >>> for x in range(3): ... print hello.greet(x) ... hello Boost.Python world! Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 15/33

Slide 22: Python Extending/Integrating A Real World Example Tips Summary Boost.Python One of a few libraries that make it easy to integrate C++ and Python code How does it pull off this trick? Template meta–programming (i. e. Don’t ask!) Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 16/33

Slide 23: Python Extending/Integrating A Real World Example Tips Summary Boost.Python One of a few libraries that make it easy to integrate C++ and Python code How does it pull off this trick? Template meta–programming (i. e. Don’t ask!) Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 16/33

Slide 24: Python Extending/Integrating A Real World Example Tips Summary See it Happen I’m making it work for you now . . . [“MyClass” demonstration (MyClass.cpp, MyClass.h, mymodule.cpp)] Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 17/33

Slide 25: Python Extending/Integrating A Real World Example Tips Summary Outline 1 Python 2 Extending/Integrating 3 A Real World Example 4 Tips Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 18/33

Slide 26: Python Extending/Integrating A Real World Example Tips Summary Real–World Examples Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 19/33

Slide 27: Python Extending/Integrating A Real World Example Tips Summary A Real World Example Re-visiting NaSt2D Wrapping NaSt2D Control the code Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 20/33

Slide 28: Python Extending/Integrating A Real World Example Tips Summary NaSt2D in Python This is what I’m going to show you: Code to be wrapped Generated wrapper code Generator script SCons (build system) How it works with Python [Wrapped NaSt2D demonstration (Wrapper.h, nast2dmodule.cpp, generate bindings.py, SConstruct, demo0.py)] Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 21/33

Slide 29: Python Extending/Integrating A Real World Example Tips Summary Extend Wrapper Class Inheriting from C++ classes Interfacing numerical values Change functionality Follow the Computation Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 22/33

Slide 30: Python Extending/Integrating A Real World Example Tips Summary Overriding in Python This is what I’m going to show you: Overriding a native method in Python Native method needs to be “virtual” Live data plotting with GNUplot [NaSt2D with plotting demonstration (demo1.py, demo2.py)] Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 23/33

Slide 31: Python Extending/Integrating A Real World Example Tips Summary Do more Computations Parameter Study Change input file Compute several cases Plot results automatically Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 24/33

Slide 32: Python Extending/Integrating A Real World Example Tips Summary Automating in Python This is what I’m going to show you: Using a template input file Batch–calculating several runs Plotting results with GNUplot [NaSt2D with parameter variation demonstration (demo3.py, demo4.py)] Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 25/33

Slide 33: Python Extending/Integrating A Real World Example Tips Summary Outline 1 Python 2 Extending/Integrating 3 A Real World Example 4 Tips Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 26/33

Slide 34: Python Extending/Integrating A Real World Example Tips Summary Some Tips . . . Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 27/33

Slide 35: Python Extending/Integrating A Real World Example Tips Summary Tips To override C++ methods: make them virtual C/C++ pit fall Call by reference/value Solution: calling policies Map to “other/sane” languages Java: Jython C#: IronPython Fortran: PyFort, Py2F (Native to other: SWIG) Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 28/33

Slide 36: Python Extending/Integrating A Real World Example Tips Summary Source: http://xkcd.com/353/ Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 29/33

Slide 37: Python Extending/Integrating A Real World Example Tips Summary Summary Why is Python good for you? How can performance bottle necks be resolved? Advantages of “Thinking Hybrid” Python–native wrapping using Boost.Python Automated wrapper generation SCons build system Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 30/33

Slide 38: Python Extending/Integrating A Real World Example Tips Summary Python and the “Need for Speed” Cuong Do – Software Architect YouTube.com “Python is fast enough for our site and allows us to produce maintainable features in record times, with a minimum of developers.” Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 31/33

Slide 39: Python Extending/Integrating A Real World Example Tips Summary Some Links Boost.Python http://www.boost.org/libs/python/doc/ Py++ http://www.language-binding.net/pyplusplus/ pyplusplus.html SCons http://www.scons.org/ NaSt2D ftp://ftp.lrz-muenchen.de/pub/science/ fluiddynamics/cfd/NaSt2D http://people.scs.fsu.edu/~burkardt/cpp_src/ nast2d/nast2d.html http://home.arcor.de/drklaus.bauerfeind/nast/ eNaSt2D.html Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 32/33

Slide 40: Python Extending/Integrating A Real World Example Tips Summary Questions? G.Kloss@massey.ac.nz Slides and code available here: http://www.kloss-familie.de/moin/TalksPresentations Guy K. Kloss | Thinking Hybrid – Python/C++ Integration 33/33