PyPy Compiling Dynamic Language Implementations
The Python Situation
The Python Situation <ul><ul><li>lots of implementations </li></ul></ul><ul><ul><ul><li>CPython - Posix </li></ul></ul></u...
What PyPy is <ul><ul><li> Python Implementation </li></ul></ul><ul><ul><li>Translation Framework </li></ul></ul>
What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul>
What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul>$> pytho...
What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul><ul><ul>...
What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul><ul><ul>...
What PyPy is <ul><li>2.  Translation Framework  </li></ul>$> python py.py $> ./pypy-c  $> ./pypy-jvm $> ./pypy-cli
some PyPy facts <ul><ul><li>founded in 2003 by Holger Krekel and Armin Rigo </li></ul></ul><ul><ul><li>2004 - 2007 EU Proj...
PyPy Components <ul><ul><li>Architecture </li></ul></ul><ul><ul><li>RPython </li></ul></ul><ul><ul><li>Interpreter  </li><...
PyPy components Architektur
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul>
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components RPython <ul><ul><li>RPython  = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most...
PyPy components Interpreter
PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul>
PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><u...
PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><u...
PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><u...
Python Interpreter
PyPy components Translation Framework
PyPy components Translation Framework
PyPy components Translation Framework
PyPy components Translation Framework
 
PyPy > CPython <ul><ul><li>High Level Language Implementation </li></ul></ul><ul><ul><li>JIT Generation </li></ul></ul><ul...
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul>
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to ...
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to ...
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to ...
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to ...
PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to ...
PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluat...
PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluat...
PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluat...
PyPy > CPython JIT Generation
PyPy > CPython Object Spaces <ul><ul><li>Flow ObjSpace </li></ul></ul><ul><ul><li>Thunk ObjSpace </li></ul></ul><ul><ul><l...
Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><...
Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><...
Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><...
Taint Object Space <ul><ul><li>provides protection  for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not le...
Taint Object Space <ul><ul><li>provides protection  for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not le...
Taint Object Space <ul><ul><li>provides protection  for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not le...
PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li>...
PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li>...
PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li>...
Demo Lazy functions and objects Tainted objects Translator Shell
Upcoming SlideShare
Loading in...5
×

a quick Introduction to PyPy

4,011

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
4,011
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
49
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "a quick Introduction to PyPy"

  1. 1. PyPy Compiling Dynamic Language Implementations
  2. 2. The Python Situation
  3. 3. The Python Situation <ul><ul><li>lots of implementations </li></ul></ul><ul><ul><ul><li>CPython - Posix </li></ul></ul></ul><ul><ul><ul><li>Jython - JVM  </li></ul></ul></ul><ul><ul><ul><li>IronPython - .NET </li></ul></ul></ul><ul><ul><li>specialized projects </li></ul></ul><ul><ul><ul><li>Stackless Python </li></ul></ul></ul><ul><ul><ul><li>unladen-swallow </li></ul></ul></ul><ul><ul><ul><li>Psyco </li></ul></ul></ul>
  4. 4. What PyPy is <ul><ul><li> Python Implementation </li></ul></ul><ul><ul><li>Translation Framework </li></ul></ul>
  5. 5. What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul>
  6. 6. What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul>$> python py.py
  7. 7. What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul><ul><ul><li>runs on top of CPython </li></ul></ul>$> python py.py
  8. 8. What PyPy is <ul><ul><li>Python implemented in Python  </li></ul></ul><ul><ul><li>looks like this: </li></ul></ul><ul><ul><li>runs on top of CPython </li></ul></ul><ul><ul><li>about 2000 times slower than CPython  </li></ul></ul>$> python py.py
  9. 9. What PyPy is <ul><li>2.  Translation Framework  </li></ul>$> python py.py $> ./pypy-c  $> ./pypy-jvm $> ./pypy-cli
  10. 10. some PyPy facts <ul><ul><li>founded in 2003 by Holger Krekel and Armin Rigo </li></ul></ul><ul><ul><li>2004 - 2007 EU Project - Sixth Framework </li></ul></ul><ul><ul><li>since 2007 Open Source Project </li></ul></ul><ul><ul><li>occasional Funding by Google </li></ul></ul><ul><ul><li>Sprint-Driven Development </li></ul></ul>
  11. 11. PyPy Components <ul><ul><li>Architecture </li></ul></ul><ul><ul><li>RPython </li></ul></ul><ul><ul><li>Interpreter  </li></ul></ul><ul><ul><li>Translation Framework </li></ul></ul>
  12. 12. PyPy components Architektur
  13. 13. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul>
  14. 14. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul>
  15. 15. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul><ul><ul><ul><li>required to perform Type Inference  </li></ul></ul></ul>
  16. 16. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul><ul><ul><ul><li>required to perform Type Inference  </li></ul></ul></ul><ul><ul><ul><li>input to the Translation Framework </li></ul></ul></ul>
  17. 17. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul><ul><ul><ul><li>required to perform Type Inference  </li></ul></ul></ul><ul><ul><ul><li>input to the Translation Framework </li></ul></ul></ul><ul><ul><ul><li>no real specification - just some hints </li></ul></ul></ul>
  18. 18. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul><ul><ul><ul><li>required to perform Type Inference  </li></ul></ul></ul><ul><ul><ul><li>input to the Translation Framework </li></ul></ul></ul><ul><ul><ul><li>no real specification - just some hints </li></ul></ul></ul><ul><ul><ul><li>used for writing interpreters </li></ul></ul></ul>
  19. 19. PyPy components RPython <ul><ul><li>RPython = Restricted/Reduced Python  </li></ul></ul><ul><ul><ul><li>Restricted = most possible static subset of Python   </li></ul></ul></ul><ul><ul><ul><li>required to perform Type Inference  </li></ul></ul></ul><ul><ul><ul><li>input to the Translation Framework </li></ul></ul></ul><ul><ul><ul><li>no real specification - just some hints </li></ul></ul></ul><ul><ul><ul><li>used for writing interpreters </li></ul></ul></ul><ul><ul><ul><li>can be used for writing extensions as well </li></ul></ul></ul>
  20. 20. PyPy components Interpreter
  21. 21. PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul>
  22. 22. PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><ul><li>bytecode compiler -> generates bytecode </li></ul></ul></ul>
  23. 23. PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><ul><li>bytecode compiler -> generates bytecode </li></ul></ul></ul><ul><ul><ul><li>bytecode evaluator -> interprets bytecode  </li></ul></ul></ul>
  24. 24. PyPy components Interpreter <ul><ul><li>written in RPython </li></ul></ul><ul><ul><li>Internals: </li></ul></ul><ul><ul><ul><li>bytecode compiler -> generates bytecode </li></ul></ul></ul><ul><ul><ul><li>bytecode evaluator -> interprets bytecode  </li></ul></ul></ul><ul><ul><ul><li>object space -> handles operations on objects </li></ul></ul></ul>
  25. 25. Python Interpreter
  26. 26. PyPy components Translation Framework
  27. 27. PyPy components Translation Framework
  28. 28. PyPy components Translation Framework
  29. 29. PyPy components Translation Framework
  30. 31. PyPy > CPython <ul><ul><li>High Level Language Implementation </li></ul></ul><ul><ul><li>JIT Generation </li></ul></ul><ul><ul><li>Object Spaces </li></ul></ul><ul><ul><li>Stackless </li></ul></ul>
  31. 32. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul>
  32. 33. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to implement new features </li></ul></ul>
  33. 34. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to implement new features </li></ul></ul><ul><ul><ul><li>lazily computed objects and functions </li></ul></ul></ul>
  34. 35. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to implement new features </li></ul></ul><ul><ul><ul><li>lazily computed objects and functions </li></ul></ul></ul><ul><ul><ul><li>plug-able  garbage-collection  </li></ul></ul></ul>
  35. 36. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to implement new features </li></ul></ul><ul><ul><ul><li>lazily computed objects and functions </li></ul></ul></ul><ul><ul><ul><li>plug-able  garbage-collection  </li></ul></ul></ul><ul><ul><ul><li>runtime replacement of live-objects  </li></ul></ul></ul>
  36. 37. PyPy > CPython High Level Language Implementation <ul><ul><li>easy to experiment with  </li></ul></ul><ul><ul><li>easy to implement new features </li></ul></ul><ul><ul><ul><li>lazily computed objects and functions </li></ul></ul></ul><ul><ul><ul><li>plug-able  garbage-collection  </li></ul></ul></ul><ul><ul><ul><li>runtime replacement of live-objects </li></ul></ul></ul><ul><ul><ul><li>stackless concurrency  </li></ul></ul></ul>
  37. 38. PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler&quot;   </li></ul></ul>
  38. 39. PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler&quot; </li></ul></ul><ul><ul><li>says:  it's possible to take an interpreter and transform it into a compiler </li></ul></ul>
  39. 40. PyPy > CPython JIT Generation <ul><ul><li>based on Partial Evaluation via Yoshihiko Futamura (1971). &quot;Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler&quot; </li></ul></ul><ul><ul><li>says:  it's possible to take an interpreter and transform it into a compiler </li></ul></ul><ul><ul><li>means: having such a thing is pretty cool because writing interpreters is a lot easier than writing compilers.  </li></ul></ul>
  40. 41. PyPy > CPython JIT Generation
  41. 42. PyPy > CPython Object Spaces <ul><ul><li>Flow ObjSpace </li></ul></ul><ul><ul><li>Thunk ObjSpace </li></ul></ul><ul><ul><li>Taint ObjSpace </li></ul></ul><ul><ul><li>Dump ObjSpace </li></ul></ul><ul><ul><li>Transparent Proxies </li></ul></ul>
  42. 43. Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><ul><ul><li>lazily computed functions </li></ul></ul></ul><ul><ul><ul><li>globally replaceable objects </li></ul></ul></ul>
  43. 44. Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><ul><ul><li>lazily computed functions </li></ul></ul></ul><ul><ul><ul><li>globally replaceable objects </li></ul></ul></ul><ul><ul><li>is implemented in about 200 lines of code </li></ul></ul>
  44. 45. Thunk Object Space <ul><ul><li>provides: </li></ul></ul><ul><ul><ul><li>lazily computed objects  </li></ul></ul></ul><ul><ul><ul><li>lazily computed functions </li></ul></ul></ul><ul><ul><ul><li>globally replaceable objects </li></ul></ul></ul><ul><ul><li>is implemented in about 200 lines of code </li></ul></ul><ul><ul><li>Example: </li></ul></ul>>>>> a = &quot;hello&quot; >>>> b = &quot;world&quot; >>>> a + b 'helloworld' >>>> become(a,b) >>>> a + b 'worldworld'
  45. 46. Taint Object Space <ul><ul><li>provides protection for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not leak </li></ul></ul></ul>
  46. 47. Taint Object Space <ul><ul><li>provides protection for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not leak </li></ul></ul></ul><ul><ul><li>provides protection from : </li></ul></ul><ul><ul><ul><li>untrusted data that needs to be validated </li></ul></ul></ul>
  47. 48. Taint Object Space <ul><ul><li>provides protection for : </li></ul></ul><ul><ul><ul><li>sensitive data that should not leak </li></ul></ul></ul><ul><ul><li>provides protection from : </li></ul></ul><ul><ul><ul><li>untrusted data that needs to be validated </li></ul></ul></ul><ul><ul><li>Example: </li></ul></ul>>>>> password = &quot;secret&quot; >>>> password 'secret' >>>> password = taint(&quot;secret&quot;) >>>> password Traceback (application-level):    File &quot;<inline>&quot;, line 1 in <interactive>      password TaintError
  48. 49. PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li></ul></ul></ul><ul><ul><ul><li>regular PyPy limited to depth=5000 </li></ul></ul></ul>
  49. 50. PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li></ul></ul></ul><ul><ul><ul><li>regular PyPy limited to depth=5000 </li></ul></ul></ul><ul><ul><li>Microthreads </li></ul></ul><ul><ul><ul><li>Coroutines </li></ul></ul></ul><ul><ul><ul><li>Tasklets and Channels </li></ul></ul></ul><ul><ul><ul><li>Greenlets </li></ul></ul></ul>
  50. 51. PyPy > CPython Stackless <ul><ul><li>infinite Recursion </li></ul></ul><ul><ul><ul><li>CPython limited to depth=1000 </li></ul></ul></ul><ul><ul><ul><li>regular PyPy limited to depth=5000 </li></ul></ul></ul><ul><ul><li>Microthreads </li></ul></ul><ul><ul><ul><li>Coroutines </li></ul></ul></ul><ul><ul><ul><li>Tasklets and Channels </li></ul></ul></ul><ul><ul><ul><li>Greenlets </li></ul></ul></ul><ul><ul><li>Coroutine Pickling </li></ul></ul><ul><ul><li>Coroutine Cloning  </li></ul></ul>
  51. 52. Demo Lazy functions and objects Tainted objects Translator Shell
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×