Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
PyPy Compiling Dynamic Language Implementations
The Python Situation
The Python Situation <ul><ul><li>lots of implementations </li></ul></ul><ul><ul><ul><li>CPython - Posix </li></ul></ul></u...
What PyPy is <ul><ul><li> Python Implementation </li></ul></ul><ul><ul><li>Translation Framework </li></ul></ul>
What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul>
What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul>$> pytho...
What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul><ul><ul>...
What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul><ul><ul>...
What PyPy is <ul><li>2.  Translation Framework  </li></ul>$> python py.py $> ./pypy-c  $> ./pypy-jvm $> ./pypy-cli
some PyPy facts <ul><ul><li>founded in 2003 by Holger Krekel and Armin Rigo </li></ul></ul><ul><ul><li>2004 - 2007 EU Proj...
PyPy Components <ul><ul><li>Architecture </li></ul></ul><ul><ul><li>RPython </li></ul></ul><ul><ul><li>Interpreter  </li><...
PyPy components Architektur
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul>
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components Interpreter
PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul>
PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><u...
PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><u...
PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><u...
Python Interpreter
PyPy components Translation Framework
PyPy components Translation Framework
PyPy components Translation Framework
PyPy components Translation Framework
 
PyPy > CPython <ul><ul><li>High Level Language Implementation </li></ul></ul><ul><ul><li>JIT Generation </li></ul></ul><ul...
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul>
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to ...
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to ...
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to ...
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to ...
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to ...
PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluat...
PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluat...
PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluat...
PyPy > CPython JIT Generation
PyPy > CPython Object Spaces <ul><ul><li>Flow ObjSpace </li></ul></ul><ul><ul><li>Thunk ObjSpace </li></ul></ul><ul><ul><l...
Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><...
Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><...
Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><...
Taint Object Space <ul><ul><li>provides protection  for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not le...
Taint Object Space <ul><ul><li>provides protection  for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not le...
Taint Object Space <ul><ul><li>provides protection  for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not le...
PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li>...
PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li>...
PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li>...
Demo Lazy functions and objects Tainted objects Translator Shell
Upcoming SlideShare
Loading in …5
×

a quick Introduction to PyPy

5,208 views

Published on

Published in: Technology
  • Be the first to comment

a quick Introduction to PyPy

  1. 1. PyPy Compiling Dynamic Language Implementations
  2. 2. The Python Situation
  3. 3. The Python Situation <ul><ul><li>lots of implementations </li></ul></ul><ul><ul><ul><li>CPython - Posix </li></ul></ul></ul><ul><ul><ul><li>Jython - JVM  </li></ul></ul></ul><ul><ul><ul><li>IronPython - .NET </li></ul></ul></ul><ul><ul><li>specialized projects </li></ul></ul><ul><ul><ul><li>Stackless Python </li></ul></ul></ul><ul><ul><ul><li>unladen-swallow </li></ul></ul></ul><ul><ul><ul><li>Psyco </li></ul></ul></ul>
  4. 4. What PyPy is <ul><ul><li> Python Implementation </li></ul></ul><ul><ul><li>Translation Framework </li></ul></ul>
  5. 5. What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul>
  6. 6. What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul>$> python py.py
  7. 7. What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul><ul><ul><li>runs on top of CPython </li></ul></ul>$> python py.py
  8. 8. What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul><ul><ul><li>runs on top of CPython </li></ul></ul><ul><ul><li>about 2000 times slower than CPython  </li></ul></ul>$> python py.py
  9. 9. What PyPy is <ul><li>2.  Translation Framework  </li></ul>$> python py.py $> ./pypy-c  $> ./pypy-jvm $> ./pypy-cli
  10. 10. some PyPy facts <ul><ul><li>founded in 2003 by Holger Krekel and Armin Rigo </li></ul></ul><ul><ul><li>2004 - 2007 EU Project - Sixth Framework </li></ul></ul><ul><ul><li>since 2007 Open Source Project </li></ul></ul><ul><ul><li>occasional Funding by Google </li></ul></ul><ul><ul><li>Sprint-Driven Development </li></ul></ul>
  11. 11. PyPy Components <ul><ul><li>Architecture </li></ul></ul><ul><ul><li>RPython </li></ul></ul><ul><ul><li>Interpreter  </li></ul></ul><ul><ul><li>Translation Framework </li></ul></ul>
  12. 12. PyPy components Architektur
  13. 13. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul>
  14. 14. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul>
  15. 15. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul><ul><ul><ul><li>required to perform Type Inference  </li></ul></ul></ul>
  16. 16. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul><ul><ul><ul><li>required to perform Type Inference  </li></ul></ul></ul><ul><ul><ul><li>input to the Translation Framework </li></ul></ul></ul>
  17. 17. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul><ul><ul><ul><li>required to perform Type Inference  </li></ul></ul></ul><ul><ul><ul><li>input to the Translation Framework </li></ul></ul></ul><ul><ul><ul><li>no real specification - just some hints </li></ul></ul></ul>
  18. 18. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul><ul><ul><ul><li>required to perform Type Inference  </li></ul></ul></ul><ul><ul><ul><li>input to the Translation Framework </li></ul></ul></ul><ul><ul><ul><li>no real specification - just some hints </li></ul></ul></ul><ul><ul><ul><li>used for writing interpreters </li></ul></ul></ul>
  19. 19. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul><ul><ul><ul><li>required to perform Type Inference  </li></ul></ul></ul><ul><ul><ul><li>input to the Translation Framework </li></ul></ul></ul><ul><ul><ul><li>no real specification - just some hints </li></ul></ul></ul><ul><ul><ul><li>used for writing interpreters </li></ul></ul></ul><ul><ul><ul><li>can be used for writing extensions as well </li></ul></ul></ul>
  20. 20. PyPy components Interpreter
  21. 21. PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul>
  22. 22. PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><ul><li>bytecode compiler -> generates bytecode </li></ul></ul></ul>
  23. 23. PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><ul><li>bytecode compiler -> generates bytecode </li></ul></ul></ul><ul><ul><ul><li>bytecode evaluator -> interprets bytecode  </li></ul></ul></ul>
  24. 24. PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><ul><li>bytecode compiler -> generates bytecode </li></ul></ul></ul><ul><ul><ul><li>bytecode evaluator -> interprets bytecode  </li></ul></ul></ul><ul><ul><ul><li>object space -> handles operations on objects </li></ul></ul></ul>
  25. 25. Python Interpreter
  26. 26. PyPy components Translation Framework
  27. 27. PyPy components Translation Framework
  28. 28. PyPy components Translation Framework
  29. 29. PyPy components Translation Framework
  30. 31. PyPy > CPython <ul><ul><li>High Level Language Implementation </li></ul></ul><ul><ul><li>JIT Generation </li></ul></ul><ul><ul><li>Object Spaces </li></ul></ul><ul><ul><li>Stackless </li></ul></ul>
  31. 32. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul>
  32. 33. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to implement new features </li></ul></ul>
  33. 34. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to implement new features </li></ul></ul><ul><ul><ul><li>lazily computed objects and functions </li></ul></ul></ul>
  34. 35. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to implement new features </li></ul></ul><ul><ul><ul><li>lazily computed objects and functions </li></ul></ul></ul><ul><ul><ul><li>plug-able  garbage-collection  </li></ul></ul></ul>
  35. 36. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to implement new features </li></ul></ul><ul><ul><ul><li>lazily computed objects and functions </li></ul></ul></ul><ul><ul><ul><li>plug-able  garbage-collection  </li></ul></ul></ul><ul><ul><ul><li>runtime replacement of live-objects  </li></ul></ul></ul>
  36. 37. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to implement new features </li></ul></ul><ul><ul><ul><li>lazily computed objects and functions </li></ul></ul></ul><ul><ul><ul><li>plug-able  garbage-collection  </li></ul></ul></ul><ul><ul><ul><li>runtime replacement of live-objects </li></ul></ul></ul><ul><ul><ul><li>stackless concurrency  </li></ul></ul></ul>
  37. 38. PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler&quot;   </li></ul></ul>
  38. 39. PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler&quot; </li></ul></ul><ul><ul><li>says:  it's possible to take an interpreter and transform it into a compiler </li></ul></ul>
  39. 40. PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler&quot; </li></ul></ul><ul><ul><li>says:  it's possible to take an interpreter and transform it into a compiler </li></ul></ul><ul><ul><li>means: having such a thing is pretty cool because writing interpreters is a lot easier than writing compilers.  </li></ul></ul>
  40. 41. PyPy > CPython JIT Generation
  41. 42. PyPy > CPython Object Spaces <ul><ul><li>Flow ObjSpace </li></ul></ul><ul><ul><li>Thunk ObjSpace </li></ul></ul><ul><ul><li>Taint ObjSpace </li></ul></ul><ul><ul><li>Dump ObjSpace </li></ul></ul><ul><ul><li>Transparent Proxies </li></ul></ul>
  42. 43. Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><ul><ul><li>lazily computed functions </li></ul></ul></ul><ul><ul><ul><li>globally replaceable objects </li></ul></ul></ul>
  43. 44. Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><ul><ul><li>lazily computed functions </li></ul></ul></ul><ul><ul><ul><li>globally replaceable objects </li></ul></ul></ul><ul><ul><li>is implemented in about 200 lines of code </li></ul></ul>
  44. 45. Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><ul><ul><li>lazily computed functions </li></ul></ul></ul><ul><ul><ul><li>globally replaceable objects </li></ul></ul></ul><ul><ul><li>is implemented in about 200 lines of code </li></ul></ul><ul><ul><li>Example: </li></ul></ul>>>>> a = &quot;hello&quot; >>>> b = &quot;world&quot; >>>> a + b 'helloworld' >>>> become(a,b) >>>> a + b 'worldworld'
  45. 46. Taint Object Space <ul><ul><li>provides protection for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not leak </li></ul></ul></ul>
  46. 47. Taint Object Space <ul><ul><li>provides protection for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not leak </li></ul></ul></ul><ul><ul><li>provides protection from : </li></ul></ul><ul><ul><ul><li>untrusted data that needs to be validated </li></ul></ul></ul>
  47. 48. Taint Object Space <ul><ul><li>provides protection for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not leak </li></ul></ul></ul><ul><ul><li>provides protection from : </li></ul></ul><ul><ul><ul><li>untrusted data that needs to be validated </li></ul></ul></ul><ul><ul><li>Example: </li></ul></ul>>>>> password = &quot;secret&quot; >>>> password 'secret' >>>> password = taint(&quot;secret&quot;) >>>> password Traceback (application-level):    File &quot;<inline>&quot;, line 1 in <interactive>      password TaintError
  48. 49. PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li></ul></ul></ul><ul><ul><ul><li>regular PyPy limited to depth=5000 </li></ul></ul></ul>
  49. 50. PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li></ul></ul></ul><ul><ul><ul><li>regular PyPy limited to depth=5000 </li></ul></ul></ul><ul><ul><li>Microthreads </li></ul></ul><ul><ul><ul><li>Coroutines </li></ul></ul></ul><ul><ul><ul><li>Tasklets and Channels </li></ul></ul></ul><ul><ul><ul><li>Greenlets </li></ul></ul></ul>
  50. 51. PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li></ul></ul></ul><ul><ul><ul><li>regular PyPy limited to depth=5000 </li></ul></ul></ul><ul><ul><li>Microthreads </li></ul></ul><ul><ul><ul><li>Coroutines </li></ul></ul></ul><ul><ul><ul><li>Tasklets and Channels </li></ul></ul></ul><ul><ul><ul><li>Greenlets </li></ul></ul></ul><ul><ul><li>Coroutine Pickling </li></ul></ul><ul><ul><li>Coroutine Cloning  </li></ul></ul>
  51. 52. Demo Lazy functions and objects Tainted objects Translator Shell

×