Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
DAE
DAE
Loading in …3
×
1 of 93

Python高级编程(二)

81

Share

Download to read offline

Happy Day 2011-10-28 @ Douban

Related Books

Free with a 30 day trial from Scribd

See all

Python高级编程(二)

  1. 1. Advanced Python Programming (Part II) Happy Day #5 2011.10
  2. 2. Topics • Performance Tuning • Garbage Collection • Extending Python
  3. 3. Performance Tuning
  4. 4. Python's Speed Among Most Popular Languages C C++ Java Lisp C# Pascal Python Ruby PHP Perl data source (on Oct.17, 2011): • http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html • http://shootout.alioth.debian.org/u64q/which-programming-languages-are-fastest.php
  5. 5. 7 Steps to Gain Speed 1) Find performance bottlenecks 2) Use better algorithms 3) Use faster tools 4) Write optimized code 5) Hire optimizers 6) Write your own extension modules 7) Parallelize the computation
  6. 6. Step 0 Is It Fast Enough Already?
  7. 7. Step 1 Find Performance Bottlenecks
  8. 8. Find Performance Bottlenecks • Profile, no guess - profile • a pure Python module - cProfile • written in C, new in Python 2.5 • same interface with profile, but lower overhead - hotshot • written in C, new in Python 2.2 • not maintained and might be removed
  9. 9. cProfile Usage • cProfile.run('foo()') • cProfile.run('foo()', 'profile.result') • python -m cProfile -o profile.result myscript.py • p = pstats.Stats('profile.result') • p.sort_stats('cumulative').print_stats() • sort by 'cumulative' to find what algorithms are taking time • sort by 'time' to find what functions are taking time • RunSnakeRun for GUI guys • RTFM, please • for IPython, type %prun?
  10. 10. Line Profile • line_profile and kernprof @profile def slow_function(): ... $ kernprof.py -l -v script_to_profile.py ... Line # Hits Time Per Hit % Time Line Contents ============================================================== 1 @profile 2 def slow_function(): 3 1 3 3.0 0.2 s = 0 4 1001 934 0.9 48.6 for i in xrange(1000): 5 1000 984 1.0 51.2 s += i 6 1 1 1.0 0.1 return s
  11. 11. Step 2 Use Better Algorithms
  12. 12. How To Know Which is Better? • timeit! • python -m timeit -s "setup" "statement" • e.g. which is faster, "d.has_key(k)" or "k in d"? $ python -m timeit -s "d=dict(zip(range(1000), range(1000)))" "d.has_key(500)" 1000000 loops, best of 3: 0.223 usec per loop $ python -m timeit -s "d=dict(zip(range(1000), range(1000)))" "500 in d" 10000000 loops, best of 3: 0.115 usec per loop
  13. 13. Use Bettern Algorithms • How to calculate sum([1, 2, ..., 100])?
  14. 14. Use Bettern Algorithms • How to calculate sum([1, 2, ..., 100])? s = 0 for i in range(101): 8.3usec s += i
  15. 15. Use Bettern Algorithms • How to calculate sum([1, 2, ..., 100])? s = 0 for i in range(101): 8.3usec s += i s = sum(range(101)) 2.8usec
  16. 16. Use Bettern Algorithms • How to calculate sum([1, 2, ..., 100])? s = 0 for i in range(101): 8.3usec s += i s = sum(range(101)) 2.8usec s = sum(xrange(101)) 2.03usec
  17. 17. Use Bettern Algorithms • How to calculate sum([1, 2, ..., 100])? s = 0 for i in range(101): 8.3usec s += i s = sum(range(101)) 2.8usec s = sum(xrange(101)) 2.03usec s = (1 + 100) * 100 / 2 0.109usec
  18. 18. Advanced Data Types • membership testing: • set & dict: O(1) vs. tuple & list: O(n) • return iterator instead of a large list • array, collections.deque, heapq, bisect
  19. 19. Examples lst = [] for i in xrange(10000): lst.insert(0, i) lst = collections.deque() for i in xrange(10000): 25317% faster lst.appendleft(i) sorted(lst, reverse=True)[:10] heapq.nlargest(10, lst) 613% faster
  20. 20. Do Less Computation • Pre-computation • Lazy computation • Cache • Approximation Algorithms
  21. 21. Example def fib(n): if n <= 1: return 1 fib(25): 59.8ms return fib(n-2) + fib(n-1)
  22. 22. Example def cache(func): c = {} def _(n): r = c.get(n) fib(25): 59.8ms if r is None: r = c[n] = func(n) return r return _ with @cache: @cache fib(25): 0.524us def fib(n): if n <= 1: return 1 112000 times faster! return fib(n-2) + fib(n-1)
  23. 23. Step 3 Use Faster Tools
  24. 24. Use Faster Tools • use iterator form • range() -> xrange() • map() -> itertools.imap() • list comprehension -> generator expression • dict.items() -> dict.iteritems() • for i in range(len(seq)): -> • for item in seq: • for i, item in enumerate(seq):
  25. 25. Use Faster Tools • SAX is faster and memory efficient than DOM • use C version of modules • profile -> cProfile • StringIO -> cStringIO • pickle -> cPickle • elementTree -> cElementTree / lxml • select has lower overhead than poll (and epoll at low number of connections) • numpy is essential for high volume numeric work
  26. 26. numpy Example from itertools import izip a=range(1000) b=range(1000) c = [ai+bi for ai, bi in izip(a, b)] import numpy a=numpy.arange(1000) b=numpy.arange(1000) c = a + b
  27. 27. Step 4 Write Optimized Code
  28. 28. Write Optimized Code • Less temporary objects • e.g. accumulator vs. sum • however, string concatenation has been optimized after Python 2.5
  29. 29. Write Optimized Code • use key= instead of cmp= when sorting lst = open('/Users/hongqn/projects/shire/luzong/ group.py').read().split() lst.sort(cmp=lambda x, y: cmp(x.lower(), y.lower())) lst.sort(key=str.lower) 377% faster
  30. 30. Write Optimized Code • local variables are faster than global variables def f(): for i in xrange(10000): r = abs(i) def f(): _abs = abs for i in xrange(10000): r = _abs(i) 28% faster • you can eliminate dots, too
  31. 31. Write Optimized Code • inline function inside time-critical loops def f(x): return x + 1 for i in xrange(10000): r = f(i) for i in xrange(10000): r = i + 1 187% faster
  32. 32. Write Optimized Code • do not import modules in loops for i in xrange(10000): import string r = string.lower('Python') import string for i in xrange(10000): r = string.lower('Python') 178% faster
  33. 33. Write Optimized Code • list comprehensions are faster than for- loops lst = [] for i in xrange(10000): lst.append(i) lst = [i for i in xrange(10000)] 213% faster
  34. 34. Write Optimized Code • use "while 1" for time-critical loops (readability lost!) a = 0 while True: a += 1 if a > 10000: break a = 0 while 1: a += 1 if a > 10000: 78% faster break
  35. 35. Write Optimized Code • "not not x" is faster than "bool(x)" (not recommended!) bool([]) not not [] 196% faster
  36. 36. Step 5 Hire Optmizers
  37. 37. Hire Optimizers • sys.setcheckinterval() • Python checks for thread switch and signal handling periodly (default 100 python virtual instructions) • set it to a larger value for better performance in cost of responsiveness
  38. 38. Hire Optimizers • gc.disable() • disable automatic garbage collection • gc.set_threshold() • collect less frequently
  39. 39. Hire Optimizers • Psyco • JIT for 32bit only. <=Python 2.6 • PyPy • Alternative implementation of Python • Shed Skin • Python-to-C++ compiler • numexpr • numpy expression evaluator
  40. 40. Step 6 Write Your Own Extension Modules
  41. 41. Write Your Own Extension Modules • Python/C API • Official API • ctypes • Call dynamic link library in Python • SWIG • Automatically generate interface code • Pyrex / Cython • write extension using Python-like language • Boost.Python • C++ API • Weave • Inline C code
  42. 42. Write Your Own Extension Modules • C-level Optimization
  43. 43. Step 7 Parallelize the Computation
  44. 44. Over CPUs • multi-threading • threading (be careful with GIL!) • multi-processing • fork • subprocess • multiprocessing • async • asyncore • twisted • greenlet/gevent • PyOpenCL / PyCUDA
  45. 45. multiprocessing Example sum(xrange(1, 10000001)) 172ms from multiprocessing import Pool pool = Pool() sum(pool.map(sum, (xrange(i, i+1000000) for i in xrange(1, 10000000, 1000000)))) 104ms on dual-core 49ms on 8-core
  46. 46. Over Cluster • XML-RPC / Json-RPC / Thrift / Protocol Buffer • Pyro • Parallel Python • dumbo on Hadoop • dpark
  47. 47. Gold Rule Premature optimization is the root of all evil. -- Donald Knuth
  48. 48. Garbage Collection
  49. 49. What Is Garbage • An object which will not be used in any future • i.e. no other object refers to it
  50. 50. Why Collecting Garbage? • re-sell it (recycle) • make programs simpler
  51. 51. Garbage Collection in Everywhere • by lilinghui, Sep.20, 2011 • http://svn.douban.com/projects/shire/wiki/ HallOfFire/Platform
  52. 52. Basic Garbage Collection Algorithms • Reference couting • Mark-and-sweep • Mark-and-compact • Copy
  53. 53. Garbage Collection in CPython • Reference couting • Mark-and-sweep
  54. 54. Reference Counting def f(): a = A() a.b = B() a.b.c = C() a.c = b.c
  55. 55. Reference Counting def f(): frame a = A() a.b = B() a.b.c = C() a b a.c = b.c 1 1 c 2
  56. 56. Reference Counting def f(): frame a = A() a.b = B() a.b.c = C() a b a.c = b.c 1 1 del a.c c 2
  57. 57. Reference Counting def f(): frame a = A() a.b = B() a.b.c = C() a b a.c = b.c 1 1 del a.c c 1
  58. 58. Reference Counting def f(): frame a = A() a.b = B() a.b.c = C() a b a.c = b.c 1 1 del a.c del a.b c 1
  59. 59. Reference Counting def f(): frame a = A() a.b = B() a.b.c = C() a b a.c = b.c 1 0 del a.c del a.b c 1
  60. 60. Reference Counting def f(): frame a = A() a.b = B() a.b.c = C() a a.c = b.c 1 del a.c del a.b c 0
  61. 61. Reference Counting def f(): frame a = A() a.b = B() a.b.c = C() a a.c = b.c 1 del a.c del a.b
  62. 62. Reference Counting def f(): a = A() a.b = B() a.b.c = C() a a.c = b.c 0 del a.c del a.b
  63. 63. Reference Counting def f(): a = A() a.b = B() a.b.c = C() a.c = b.c del a.c del a.b
  64. 64. Pros & Cons of Reference Counting • Pros • collect early • predictable run-time behavior • Cons • slow • circle reference
  65. 65. Circle Reference def f(): frame a = A() a.b = B() a.b.c = C() a b a.b.c.b = a.b 1 2 c 1
  66. 66. Circle Reference def f(): frame a = A() a.b = B() a.b.c = C() a b a.b.c.b = a.b 1 2 del a.b c 1
  67. 67. Circle Reference def f(): frame a = A() a.b = B() a.b.c = C() a b a.b.c.b = a.b 1 1 del a.b c 1
  68. 68. Circle Reference def f(): a = A() a.b = B() a.b.c = C() b a.b.c.b = a.b 1 del a.b c 1
  69. 69. Weak Reference a reference not strong enough to keep an object alive >>> import weakref import weakref >>> class Object: ... pass _id2obj_dict = weakref.WeakValueDictionary() ... >>> o = Object() def remember(obj): >>> r = weakref.ref(o) oid = id(obj) >>> o2 = r() _id2obj_dict[oid] = obj >>> o is o2 return oid True >>> del o, o2 def id2obj(oid): >>> print r() return _id2obj_dict[oid] None
  70. 70. Mark-and-Sweep frame 1. Mark root objects and all their referents reachable a b 2. Sweep unreachable 1 1 objects c 1
  71. 71. Generations • generations and thresholds • generation 0 (youngest): 700 • generation 1 (middle):10 • generation 2 (oldest): 10 • and long_lived_pending / long_lived_total > 25% (Python 2.7+)
  72. 72. Python's Optimization • Track only container objects • Use referent count to find root objects
  73. 73. __del__ • circle referenced objects with __del__() methods will be put in gc.garbage • do not use __del__
  74. 74. GC Module • gc.enable() / gc.disable() • gc.collect() • gc.get_threshold() / gc.set_threshold() • gc.set_debug() • gc.get_referers() / gc.get_referents() • gc.get_objects()
  75. 75. Extending Python
  76. 76. Python/C API
  77. 77. #include <Python.h> static PyObject *SpamError; static PyObject * spam_system(PyObject *self, PyObject *args) { const char *command; int sts; if (!PyArg_ParseTuple(args, "s", &command)) return NULL; sts = system(command); if (sts < 0) { PyErr_SetString(SpamError, "System command failed"); return NULL; } return PyLong_FromLong(sts); } static PyMethodDef SpamMethods[] = { {"system", spam_system, METH_VARARGS, "Execute a shell command."}, {NULL, NULL, 0, NULL} /* Sentinel */ }; PyMODINIT_FUNC initspam(void) { PyObject *m; m = Py_InitModule("spam", SpamMethods); if (m == NULL) return; SpamError = PyErr_NewException("spam.error", NULL, NULL); Py_INCREF(SpamError); PyModule_AddObject(m, "error", SpamError); }
  78. 78. Building with setuptools from setuptools import setup, Extension module1 = Extension('spam', sources = ['spam.c']) setup (name = 'PackageName', version = '1.0', description = 'This is a demo package', ext_modules = [module1]) python setup.py install python setup.py build_ext --inplace
  79. 79. Parse and Build Values • int PyArg_ParseTuple(PyObject *args, const char *format, ...) • int PyArg_ParseTupleAndKeywords(PyObject *args, PyObject *kw, const char *format, char *keywords[], ...) • int PyArg_BuildValue(const char *format, ...)
  80. 80. Manage Reference Counting • Py_INCREF(PyObject *o) / Py_XINCREF(PyObject *o) • Py_DECREF(PyObject *o) / Py_XINCREF(PyObject *o) / Py_CLEAR(PyObject *o) • New/Borrowed/Stealing references
  81. 81. Exception Handling • errno-like mechanism • PyErr_SetString() / PyErr_SetObject() to set error • PyErr_Occurred() to check error • Most functions return an error indicator, e.g. NULL, -1
  82. 82. Global Interpreter Lock • Release GIL before running blocking C code Py_BEGIN_ALLOW_THREADS ...Do some blocking I/O operation... Py_END_ALLOW_THREADS • Reacquire GIL before calling into Python functions PyGILState_STATE gstate; gstate = PyGILState_Ensure(); /* Perform Python actions here. */ result = CallSomeFunction(); /* evaluate result */ /* Release the thread. No Python API allowed beyond this point. */ PyGILState_Release(gstate);
  83. 83. Cython / Pyrex
  84. 84. Python Style Code from libc.stdlib cimport system as c_system class error(Exception): pass def system(char* cmd): cdef int retval = c_system(cmd) if retval < 0: raise error("System command failed") return retval
  85. 85. setup.py from setuptools import setup, Extension from Cython.Distutils import build_ext extmod = Extension("spam", sources=['spam.pyx']) setup( name="Spam", cmdclass = {'build_ext': build_ext}, ext_modules = [extmod])
  86. 86. #include <Python.h> #include "structmember.h" static PyGetSetDef Noddy_getseters[] = { Extension Types typedef struct { {"first", PyObject_HEAD (getter)Noddy_getfirst, (setter)Noddy_setfirst, PyObject *first; "first name", PyObject *last; NULL}, int number; {"last", } Noddy; (getter)Noddy_getlast, (setter)Noddy_setlast, "last name", static void NULL}, Noddy_dealloc(Noddy* self) {NULL} /* Sentinel */ { }; Py_XDECREF(self->first); Py_XDECREF(self->last); static PyObject * self->ob_type->tp_free((PyObject*)self); Noddy_name(Noddy* self) } { static PyObject *format = NULL; static PyObject * PyObject *args, *result; Noddy_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { if (format == NULL) { Noddy *self; format = PyString_FromString("%s %s"); if (format == NULL) self = (Noddy *)type->tp_alloc(type, 0); return NULL; if (self != NULL) { } self->first = PyString_FromString(""); if (self->first == NULL) args = Py_BuildValue("OO", self->first, self->last); { if (args == NULL) Py_DECREF(self); return NULL; return NULL; } result = PyString_Format(format, args); Py_DECREF(args); self->last = PyString_FromString(""); if (self->last == NULL) return result; { } Py_DECREF(self); return NULL; static PyMethodDef Noddy_methods[] = { } {"name", (PyCFunction)Noddy_name, METH_NOARGS, "Return the name, combining the first and last name" self->number = 0; }, } {NULL} /* Sentinel */ }; return (PyObject *)self; } static PyTypeObject NoddyType = { PyObject_HEAD_INIT(NULL) static int 0, /*ob_size*/ Noddy_init(Noddy *self, PyObject *args, PyObject *kwds) "noddy.Noddy", /*tp_name*/ { sizeof(Noddy), /*tp_basicsize*/ PyObject *first=NULL, *last=NULL, *tmp; 0, /*tp_itemsize*/ (destructor)Noddy_dealloc, /*tp_dealloc*/ static char *kwlist[] = {"first", "last", "number", NULL}; 0, /*tp_print*/ 0, /*tp_getattr*/ if (! PyArg_ParseTupleAndKeywords(args, kwds, "|SSi", kwlist, 0, /*tp_setattr*/ &first, &last, 0, /*tp_compare*/ &self->number)) 0, /*tp_repr*/ return -1; 0, /*tp_as_number*/ 0, /*tp_as_sequence*/ if (first) { 0, /*tp_as_mapping*/ tmp = self->first; 0, /*tp_hash */ Py_INCREF(first); 0, /*tp_call*/ self->first = first; 0, /*tp_str*/ Py_DECREF(tmp); 0, /*tp_getattro*/ } 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ if (last) { Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /*tp_flags*/ tmp = self->last; "Noddy objects", /* tp_doc */ Py_INCREF(last); 0,! ! /* tp_traverse */ self->last = last; 0,! ! /* tp_clear */ Py_DECREF(tmp); 0,! ! /* tp_richcompare */ } 0,! ! /* tp_weaklistoffset */ 0,! ! /* tp_iter */ return 0; 0,! ! /* tp_iternext */ } Noddy_methods, /* tp_methods */ Noddy_members, /* tp_members */ static PyMemberDef Noddy_members[] = { Noddy_getseters, /* tp_getset */ {"number", T_INT, offsetof(Noddy, number), 0, 0, /* tp_base */ "noddy number"}, 0, /* tp_dict */ {NULL} /* Sentinel */ 0, /* tp_descr_get */ }; 0, /* tp_descr_set */ 0, /* tp_dictoffset */ static PyObject * (initproc)Noddy_init, /* tp_init */ Noddy_getfirst(Noddy *self, void *closure) 0, /* tp_alloc */ { Noddy_new, /* tp_new */ Py_INCREF(self->first); }; return self->first; } static PyMethodDef module_methods[] = { {NULL} /* Sentinel */ static int }; Noddy_setfirst(Noddy *self, PyObject *value, void *closure) { #ifndef PyMODINIT_FUNC! /* declarations for DLL import/export */ if (value == NULL) { #define PyMODINIT_FUNC void PyErr_SetString(PyExc_TypeError, "Cannot delete the first attribute"); #endif return -1; PyMODINIT_FUNC } initnoddy3(void) { if (! PyString_Check(value)) { PyObject* m; PyErr_SetString(PyExc_TypeError, "The first attribute value must be a string"); if (PyType_Ready(&NoddyType) < 0) return -1; return; } m = Py_InitModule3("noddy3", module_methods, Py_DECREF(self->first); "Example module that creates an extension type."); Py_INCREF(value); self->first = value; if (m == NULL) return; return 0; } Py_INCREF(&NoddyType); PyModule_AddObject(m, "Noddy", (PyObject *)&NoddyType); static PyObject * } Noddy_getlast(Noddy *self, void *closure) { Py_INCREF(self->last); return self->last; } static int Noddy_setlast(Noddy *self, PyObject *value, void *closure) { if (value == NULL) { PyErr_SetString(PyExc_TypeError, "Cannot delete the last attribute"); return -1; } if (! PyString_Check(value)) { PyErr_SetString(PyExc_TypeError, "The last attribute value must be a string"); return -1; } Py_DECREF(self->last); Py_INCREF(value); self->last = value; return 0; }
  87. 87. #include <Python.h> #include "structmember.h" static PyGetSetDef Noddy_getseters[] = { Extension Types typedef struct { {"first", PyObject_HEAD (getter)Noddy_getfirst, (setter)Noddy_setfirst, PyObject *first; "first name", PyObject *last; NULL}, int number; {"last", } Noddy; (getter)Noddy_getlast, (setter)Noddy_setlast, "last name", static void NULL}, Noddy_dealloc(Noddy* self) {NULL} /* Sentinel */ { }; Py_XDECREF(self->first); Py_XDECREF(self->last); static PyObject * self->ob_type->tp_free((PyObject*)self); Noddy_name(Noddy* self) } { static PyObject *format = NULL; static PyObject * PyObject *args, *result; Noddy_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { if (format == NULL) { Noddy *self; format = PyString_FromString("%s %s"); if (format == NULL) self = (Noddy *)type->tp_alloc(type, 0); return NULL; if (self != NULL) { } self->first = PyString_FromString(""); if (self->first == NULL) args = Py_BuildValue("OO", self->first, self->last); { if (args == NULL) Py_DECREF(self); return NULL; return NULL; } result = PyString_Format(format, args); Py_DECREF(args); self->last = PyString_FromString(""); if (self->last == NULL) return result; { Py_DECREF(self); } cdef class Noddy: return NULL; static PyMethodDef Noddy_methods[] = { } {"name", (PyCFunction)Noddy_name, METH_NOARGS, "Return the name, combining the first and last name" }, """Noddy objects""" self->number = 0; } {NULL} /* Sentinel */ }; return (PyObject *)self; } static int static PyTypeObject NoddyType = { PyObject_HEAD_INIT(NULL) 0, /*ob_size*/ cdef object first, last "noddy.Noddy", /*tp_name*/ Noddy_init(Noddy *self, PyObject *args, PyObject *kwds) { PyObject *first=NULL, *last=NULL, *tmp; sizeof(Noddy), 0, /*tp_basicsize*/ /*tp_itemsize*/ cdef public int number (destructor)Noddy_dealloc, /*tp_dealloc*/ static char *kwlist[] = {"first", "last", "number", NULL}; 0, /*tp_print*/ 0, /*tp_getattr*/ if (! PyArg_ParseTupleAndKeywords(args, kwds, "|SSi", kwlist, 0, /*tp_setattr*/ &first, &last, &self->number)) 0, 0, 0, /*tp_compare*/ /*tp_repr*/ /*tp_as_number*/ def __cinit__(self, str first="", str last="", int number=0): return -1; if (first) { 0, 0, 0, /*tp_as_sequence*/ /*tp_as_mapping*/ /*tp_hash */ self.first = first tmp = self->first; Py_INCREF(first); self->first = first; Py_DECREF(tmp); 0, 0, 0, /*tp_call*/ /*tp_str*/ /*tp_getattro*/ self.last = last 0, /*tp_setattro*/ } if (last) { 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /*tp_flags*/ self.number = number tmp = self->last; "Noddy objects", /* tp_doc */ Py_INCREF(last); 0,! ! /* tp_traverse */ self->last = last; 0,! ! /* tp_clear */ Py_DECREF(tmp); 0,! ! /* tp_richcompare */ } 0,! 0,! 0,! ! ! ! /* tp_weaklistoffset */ /* tp_iter */ /* tp_iternext */ property first: return 0; } Noddy_methods, Noddy_members, Noddy_getseters, /* tp_methods */ /* tp_members */ /* tp_getset */ """first name""" static PyMemberDef Noddy_members[] = { {"number", T_INT, offsetof(Noddy, number), 0, "noddy number"}, {NULL} /* Sentinel */ 0, 0, 0, /* tp_base */ /* tp_dict */ /* tp_descr_get */ def __get__(self): 0, /* tp_descr_set */ }; static PyObject * 0, (initproc)Noddy_init, /* tp_dictoffset */ /* tp_init */ return self.first Noddy_getfirst(Noddy *self, void *closure) 0, /* tp_alloc */ { Py_INCREF(self->first); }; Noddy_new, /* tp_new */ def __set__(self, str value): return self->first; } static PyMethodDef module_methods[] = { }; {NULL} /* Sentinel */ self.first = value static int Noddy_setfirst(Noddy *self, PyObject *value, void *closure) { #ifndef PyMODINIT_FUNC! /* declarations for DLL import/export */ if (value == NULL) { #define PyMODINIT_FUNC void } PyErr_SetString(PyExc_TypeError, "Cannot delete the first attribute"); return -1; #endif PyMODINIT_FUNC initnoddy3(void) property last: { if (! PyString_Check(value)) { PyErr_SetString(PyExc_TypeError, PyObject* m; """last name""" "The first attribute value must be a string"); if (PyType_Ready(&NoddyType) < 0) } return -1; return; def __get__(self): m = Py_InitModule3("noddy3", module_methods, Py_DECREF(self->first); Py_INCREF(value); if (m == NULL) "Example module that creates an extension type."); return self.last self->first = value; return 0; return; Py_INCREF(&NoddyType); def __set__(self, str value): } static PyObject * Noddy_getlast(Noddy *self, void *closure) } PyModule_AddObject(m, "Noddy", (PyObject *)&NoddyType); self.last = value { Py_INCREF(self->last); return self->last; } static int def name(self): Noddy_setlast(Noddy *self, PyObject *value, void *closure) { if (value == NULL) { """Return the name, combining the first and last name""" PyErr_SetString(PyExc_TypeError, "Cannot delete the last attribute"); } return -1; return "%s %s" % (self.first, self.last) if (! PyString_Check(value)) { PyErr_SetString(PyExc_TypeError, "The last attribute value must be a string"); return -1; } Py_DECREF(self->last); Py_INCREF(value); self->last = value; return 0; }
  88. 88. GIL with nogil: ... cdef void my_callback(void *data) with gil: ... cdef void my_gil_free_func(int spam) nogil: ...
  89. 89. ctypes
  90. 90. Call C Function in Python from ctypes import cdll libc = cdll.LoadLibrary('libc.dylib') class error(Exception): pass def system(cmd): retval = libc.system(cmd) if retval < 0: raise error("System command failed.") return retval
  91. 91. GIL • ctypes.cdll: Release GIL • ctypes.pydll: With GIL
  92. 92. Q &A
  93. 93. Thanks!

×