Bind Python and C
C Extensions for Python 3
StarNight @ COSCUP 2015
Who am I?
潘建宏 / Jian-Hong Pan (StarNight)
About Me : http://about.me/StarNight
出沒在~
GitHub : starnight
PTT : zack2004
plurk : StarNight
Facebook : Jian-Hong Pan
目前繼續在種花店當個打雜園丁 ~
Outline
● Standing on the shoulders of giants
● What I want
● Binding flow
● Examples:
○ Without arguments
○ With passed arguments
○ With variables return
● Difference of binding between Python 2 & 3
Stand on the Shoulders of Giants
● Cython - Making Python as Fast as C by
Mosky
● The Future of GUI Programming with Python
by TP
● "My grand scheme was to build a C API for
the web that's what I thought PHP was." by
Rasmus Lerdorf (The father of PHP)
● Many languages make wrappers bind C!
What I Want
● Python is convenient
and good to make
prototype.
● I am familier with C
more. (是個慣C
● Therefore, I write C
with my “right hand”,
and write Python
with my “left hand”.
From 灌籃高手漫畫
What I want (Cont.
● Compiled C program is more efficient than
Python.
● My OS default enviroment is “Python 3”.
● To be convenient to make prototype and to
have efficiency at the same time, I should
make Python 3 call some C libraries written
by myself, not Cython.
● Python 3 with C extensions.
test.py
lib***.h
lib***.c
bind.c
setup.py
C extension
import
module
Binding Flow
Binding Flow (cont.
1. test.py is the usual Python application.
2. The module is composited with lib***.h and
lib***.c.
3. To be imported by Python, the bind.c is the
wrapper of the module.
4. setup.py tells compiler how to build and
output the wrapped C extension module.
5. test.py imports C extension module.
Examples
https://github.com/starnight/python-c-extension
Traditional Chinese Description on StarNight’s Blog
#include <python.h>
To access the internal Python API.
PEP 3121
00-HelloWorld: Without Arguments
● libmypy.h & libmypy.c:
○ Function declaration & Implementation
○ PyObject * hello(PyObject *self)
● bind.c:
○ Define Python Methods: helloworld_funcs
○ Define Python Module: helloworld_mod
○ Initial Module Function: PyInit_helloworld calls
PyModule_Create(&helloworld_mod)
points to the returned object
points to this module object
Define Python Methods
struct PyMethodDef {
const char *ml_name; // name of method
PyCFunction ml_meth; // pointer to the C implementation
int ml_flags; // flag bits
const char *ml_doc; // pointer to the description string
};
typedef struct PyMethodDef PyMethodDef;
PS. flag bits could be:
METH_VARARGS, METH_KEYWORDS,
METH_NOARGS, METH_O
Define Python Module
typedef struct PyModuleDef{
PyModuleDef_Base m_base; // PyModuleDef_HEAD_INIT
const char* m_name; // pointer to the module name
const char* m_doc; // pointer to the module description
Py_ssize_t m_size; // -1 in this example
PyMethodDef *m_methods; // previous PyMethodDef
inquiry m_reload; // should be NULL
traverseproc m_traverse; // could be NULL
inquiry m_clear; // could be NULL
freefunc m_free; // could be NULL
}PyModuleDef;
00-HelloWorld: Without Arguments
● setup.py:
from distutils.core import setup, Extension
setup(
name = "helloworld",
version = "1.0",
ext_modules = [Extension("helloworld",
["bind.c", "libmypy.c"])]
);
00-HelloWorld: Without Arguments
● Makefile:
python setup.py build_ext --inplace
● test.py:
import helloworld
print(helloworld.hello());
help(helloworld);
module name module description
method name
method description
01-HeyMan: With Passed Arguments
● libmypy.h & libmypy.c:
○ Add heyman function
○ PyObject * heyman(PyObject *self, PyObject *args);
○ heyman calls
PyArg_ParseTuple(args, "is", &num, &name)
● bind.c
○ Add heyman to defined Python Methods:
helloworld_funcs
points to passed arguments object
int
character
string
01-HeyMan: With Passed Arguments
● test.py
import helloworld
print(helloworld.hello());
print(helloworld.heyman(5, "StarNight"));
help(helloworld);
02-Add: With Variable Return
● libmypy.h & libmypy.c:
○ Add add function
○ PyObject * add(PyObject *self, PyObject *args);
○ add calls
return Py_BuildValue("is", num1 + num2, eq)
● bind.c
○ Add add to defined Python Methods:
helloworld_funcs
points to the returned object
character
string
int
02-Add: With Variable Return
● test.py
import helloworld
print(helloworld.hello());
print(helloworld.heyman(5, "StarNight"));
print(helloworld.add(5, 6));
help(helloworld);
Recap
1. Include python.h
2. Declare and implement functions you want.
3. Define Python methods’ mapping table with
PyMethodDef structures which wraps the
functions.
4. Define Python module with PyModuleDef
structure which is the namespace of the
methods’ mapping table.
5. Implement the initial function which initials
the module.
03-CrossVersion
● It is the difference of binding between
Python 2 & 3.
● bind.c
○ Python 3 has module definition structure
(PyModuleDef), but Python 2 does not.
○ Python 3’s Initial funciton:
■ PyInit_<module name> calls PyModule_Create
○ Python 2’s Initial funciton:
■ init<module name> calls Py_InitModule3
Module Definition in Python 3
Python 3’s initial function
Python 2’s initial function
Reference
● Python Extension Programming with C
● Extending Python with C or C++
● Migrating C extensions
● Porting Extension Modules to Python 3
● Parsing arguments and building values
Thank you ~

Bind Python and C @ COSCUP 2015

  • 1.
    Bind Python andC C Extensions for Python 3 StarNight @ COSCUP 2015
  • 2.
    Who am I? 潘建宏/ Jian-Hong Pan (StarNight) About Me : http://about.me/StarNight 出沒在~ GitHub : starnight PTT : zack2004 plurk : StarNight Facebook : Jian-Hong Pan 目前繼續在種花店當個打雜園丁 ~
  • 3.
    Outline ● Standing onthe shoulders of giants ● What I want ● Binding flow ● Examples: ○ Without arguments ○ With passed arguments ○ With variables return ● Difference of binding between Python 2 & 3
  • 4.
    Stand on theShoulders of Giants ● Cython - Making Python as Fast as C by Mosky ● The Future of GUI Programming with Python by TP ● "My grand scheme was to build a C API for the web that's what I thought PHP was." by Rasmus Lerdorf (The father of PHP) ● Many languages make wrappers bind C!
  • 5.
    What I Want ●Python is convenient and good to make prototype. ● I am familier with C more. (是個慣C ● Therefore, I write C with my “right hand”, and write Python with my “left hand”. From 灌籃高手漫畫
  • 6.
    What I want(Cont. ● Compiled C program is more efficient than Python. ● My OS default enviroment is “Python 3”. ● To be convenient to make prototype and to have efficiency at the same time, I should make Python 3 call some C libraries written by myself, not Cython. ● Python 3 with C extensions.
  • 7.
  • 8.
    Binding Flow (cont. 1.test.py is the usual Python application. 2. The module is composited with lib***.h and lib***.c. 3. To be imported by Python, the bind.c is the wrapper of the module. 4. setup.py tells compiler how to build and output the wrapped C extension module. 5. test.py imports C extension module.
  • 9.
  • 10.
    #include <python.h> To accessthe internal Python API. PEP 3121
  • 11.
    00-HelloWorld: Without Arguments ●libmypy.h & libmypy.c: ○ Function declaration & Implementation ○ PyObject * hello(PyObject *self) ● bind.c: ○ Define Python Methods: helloworld_funcs ○ Define Python Module: helloworld_mod ○ Initial Module Function: PyInit_helloworld calls PyModule_Create(&helloworld_mod) points to the returned object points to this module object
  • 12.
    Define Python Methods structPyMethodDef { const char *ml_name; // name of method PyCFunction ml_meth; // pointer to the C implementation int ml_flags; // flag bits const char *ml_doc; // pointer to the description string }; typedef struct PyMethodDef PyMethodDef; PS. flag bits could be: METH_VARARGS, METH_KEYWORDS, METH_NOARGS, METH_O
  • 14.
    Define Python Module typedefstruct PyModuleDef{ PyModuleDef_Base m_base; // PyModuleDef_HEAD_INIT const char* m_name; // pointer to the module name const char* m_doc; // pointer to the module description Py_ssize_t m_size; // -1 in this example PyMethodDef *m_methods; // previous PyMethodDef inquiry m_reload; // should be NULL traverseproc m_traverse; // could be NULL inquiry m_clear; // could be NULL freefunc m_free; // could be NULL }PyModuleDef;
  • 16.
    00-HelloWorld: Without Arguments ●setup.py: from distutils.core import setup, Extension setup( name = "helloworld", version = "1.0", ext_modules = [Extension("helloworld", ["bind.c", "libmypy.c"])] );
  • 17.
    00-HelloWorld: Without Arguments ●Makefile: python setup.py build_ext --inplace ● test.py: import helloworld print(helloworld.hello()); help(helloworld);
  • 18.
    module name moduledescription method name method description
  • 19.
    01-HeyMan: With PassedArguments ● libmypy.h & libmypy.c: ○ Add heyman function ○ PyObject * heyman(PyObject *self, PyObject *args); ○ heyman calls PyArg_ParseTuple(args, "is", &num, &name) ● bind.c ○ Add heyman to defined Python Methods: helloworld_funcs points to passed arguments object int character string
  • 21.
    01-HeyMan: With PassedArguments ● test.py import helloworld print(helloworld.hello()); print(helloworld.heyman(5, "StarNight")); help(helloworld);
  • 23.
    02-Add: With VariableReturn ● libmypy.h & libmypy.c: ○ Add add function ○ PyObject * add(PyObject *self, PyObject *args); ○ add calls return Py_BuildValue("is", num1 + num2, eq) ● bind.c ○ Add add to defined Python Methods: helloworld_funcs points to the returned object character string int
  • 25.
    02-Add: With VariableReturn ● test.py import helloworld print(helloworld.hello()); print(helloworld.heyman(5, "StarNight")); print(helloworld.add(5, 6)); help(helloworld);
  • 27.
    Recap 1. Include python.h 2.Declare and implement functions you want. 3. Define Python methods’ mapping table with PyMethodDef structures which wraps the functions. 4. Define Python module with PyModuleDef structure which is the namespace of the methods’ mapping table. 5. Implement the initial function which initials the module.
  • 28.
    03-CrossVersion ● It isthe difference of binding between Python 2 & 3. ● bind.c ○ Python 3 has module definition structure (PyModuleDef), but Python 2 does not. ○ Python 3’s Initial funciton: ■ PyInit_<module name> calls PyModule_Create ○ Python 2’s Initial funciton: ■ init<module name> calls Py_InitModule3
  • 29.
    Module Definition inPython 3 Python 3’s initial function Python 2’s initial function
  • 31.
    Reference ● Python ExtensionProgramming with C ● Extending Python with C or C++ ● Migrating C extensions ● Porting Extension Modules to Python 3 ● Parsing arguments and building values
  • 32.