PyPy London Demo Evening 2013
Upcoming SlideShare
Loading in...5
×
 

PyPy London Demo Evening 2013

on

  • 74,426 views

The slides we used at the PyPy London Demo Evening:

The slides we used at the PyPy London Demo Evening:

http://morepypy.blogspot.co.uk/2013/08/preliminary-london-demo-evening-agenda.html

Statistics

Views

Total Views
74,426
Views on SlideShare
2,463
Embed Views
71,963

Actions

Likes
2
Downloads
16
Comments
0

76 Embeds 71,963

http://morepypy.blogspot.com 21994
http://morepypy.blogspot.de 8920
http://morepypy.blogspot.co.uk 5767
http://morepypy.blogspot.fr 3448
http://morepypy.blogspot.ca 2543
http://morepypy.blogspot.it 2535
http://morepypy.blogspot.ch 2125
http://morepypy.blogspot.ru 2123
http://morepypy.blogspot.com.es 1922
http://morepypy.blogspot.nl 1767
http://morepypy.blogspot.fi 1542
http://morepypy.blogspot.se 1527
http://morepypy.blogspot.com.au 1523
http://morepypy.blogspot.cz 1091
http://morepypy.blogspot.jp 1068
http://morepypy.blogspot.com.br 1024
http://morepypy.blogspot.tw 1015
http://morepypy.blogspot.dk 979
http://morepypy.blogspot.kr 909
http://morepypy.blogspot.be 904
http://morepypy.blogspot.co.il 862
http://morepypy.blogspot.hu 779
http://morepypy.blogspot.no 719
http://morepypy.blogspot.co.at 704
http://morepypy.blogspot.in 578
http://morepypy.blogspot.sg 540
http://morepypy.blogspot.hk 473
http://morepypy.blogspot.gr 338
http://cloud.feedly.com 321
http://morepypy.blogspot.co.nz 286
http://morepypy.blogspot.com.ar 260
http://feeds.feedburner.com 202
http://morepypy.blogspot.pt 171
http://morepypy.blogspot.sk 165
http://morepypy.blogspot.ro 149
http://morepypy.blogspot.ie 132
http://morepypy.blogspot.mx 101
http://www.newsblur.com 65
http://newsblur.com 64
http://www.morepypy.blogspot.fr 53
http://www.morepypy.blogspot.com 35
http://digg.com 34
http://morepypy.blogspot.com.tr 28
http://morepypy.blogspot.ae 27
http://translate.googleusercontent.com 17
http://www.morepypy.blogspot.ru 14
http://www.goread.io 14
http://summary 13
http://yoleoreader.com 10
http://www.morepypy.blogspot.in 10
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

PyPy London Demo Evening 2013 PyPy London Demo Evening 2013 Presentation Transcript

  • Looking for Python developers! Speak to us at PyPy demo night or visit www.pitchup.com/jobs Python / Django Postgres Celery Redis nginx memcache Jquery Solr S3 Leading booking site for campsites and caravan parks, founded in 2009 by lastminute.com alumni ● 65k visits / day, £6m bookings / year ● 650 bookable sites ● Huge market ○ 26k campsites and 300m bednights in Europe ○ 600m bednights in US ○ 47% more bednights than hotels (GB) ○ More trips to campsites than holidays to France + Spain combined (GB) ● Team of 15, based in west London
  • Welcome to the PyPy Demo Evening Laurence Tratt 2013-08-27 1 / 5 http://soft-dev.org/
  • Why are we here? 2 / 5 http://soft-dev.org/
  • What do we do? (l-r) Vasudevan, Bolz, Tratt, Barrett, Diekmann 3 / 5 http://soft-dev.org/
  • What do we do? • Aim: identify important challenges in software development. 3 / 5 http://soft-dev.org/
  • What do we do? • Aim: identify important challenges in software development. • Strengths: language design and implementation. 3 / 5 http://soft-dev.org/
  • What do we do? • Aim: identify important challenges in software development. • Strengths: language design and implementation. • Immediate benefits: faster VMs. 3 / 5 http://soft-dev.org/
  • What do we do? • Aim: identify important challenges in software development. • Strengths: language design and implementation. • Immediate benefits: faster VMs. • Long-term benefits: language composition. 3 / 5 http://soft-dev.org/
  • This evening 1 Carl Friedrich Bolz PyPy overview. 2 Lukas Diekmann Storage strategies. 3 Maciej Fijalkowski NumPy. 4 Armin Rigo Software Transactional Memory (STM). 5 Edd Barrett Language composition using meta-tracing. 4 / 5 http://soft-dev.org/
  • Can you help? • Contributors. 5 / 5 http://soft-dev.org/
  • Can you help? • Contributors. • Resources. • Software Freedom Conservancy 5 / 5 http://soft-dev.org/
  • A Very Brief Introduction to PyPy Carl Friedrich Bolz PyPy Demo Evening, King’s College London, August 27, 2013 Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • CPython is slow CPython 1-3 orders of magnitude slower than C BinaryTrees Dhrystone FannkuchRedux Fasta Knucleotide Mandelbrot Nbody RegexDNA RevComp Richards SpectralNorm 0.1 1 10 100 1000 SlowerthanC,lowerisbetter C Java Cpython Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Reasons for Bad Performance interpretation overhead late binding dispatching boxing Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Enter PyPy a modern efficient implementation of Python Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Enter PyPy a modern efficient implementation of Python open source, MIT license written in Python itself, then bootstrapped to C uses a tracing JIT compiler to produce machine code at runtime Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Performance of PyPy significantly faster than CPython, typically in the same order of magnitude than C BinaryTrees Dhrystone FannkuchRedux Fasta Knucleotide Mandelbrot Nbody RegexDNA RevComp Richards SpectralNorm 0.1 1 10 100 1000 SlowerthanC,lowerisbetter C Java PyPy CPython Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Performance of PyPy significantly faster than CPython, typically in the same order of magnitude than C BinaryTrees Dhrystone FannkuchRedux Fasta Knucleotide Mandelbrot Nbody RegexDNA RevComp Richards SpectralNorm 0.1 1 10 100 1000 SlowerthanC,lowerisbetter C Java PyPy CPython on average about 6.3 faster than CPython Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Demo Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Architecture Python interpreter written in RPython JIT-compiler automatically added via meta-tracing Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Status Python 2.7.3 support, 2.7.4 coming soon beta-level support for Python 3.2, more coming eventually pure Python code fully supported, please report as bug if not Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Status Python 2.7.3 support, 2.7.4 coming soon beta-level support for Python 3.2, more coming eventually pure Python code fully supported, please report as bug if not C extension modules partially supported, if they are well-behaved they are slow use cffi (a ctypes replacement) instead Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Questions? PyPy is a fast JITted Python implementation (if something is not fast, please report it as a bug) open source under MIT license http://pypy.org Carl Friedrich Bolz A Very Brief Introduction to PyPy
  • Storage Strategies for Fast Containers Lukas Diekmann August, 27 2013 1 / 6 http://soft-dev.org/
  • Collection strategies introduced in PyPy 1.9 optimisation of collections for certain data types improving speed reducing memory 2 / 6 http://soft-dev.org/
  • Idea typical programs have homogeneously types collections create optimised versions of collections for certain types so far: lists: ints, floats, strings/unicode sets: ints, floats, strings/unicode dicts: ints, strings/unicode 3 / 6 http://soft-dev.org/
  • Optimisations collections automatically change to most efficient strategy store elements more memory efficiently fast elements access 4 / 6 http://soft-dev.org/
  • Further optimisations collection creation and initalisation split(d), set([1,2,3]) type based operations: contains, difference, issubset special strategies RangeListStrategy: calculates elements on the fly Tracing JIT interaction: faster (low-level) comparisons, remove type checks 5 / 6 http://soft-dev.org/
  • Results paper at OOPSLA on average ∼18% speedup ∼6% less memory usage more info at http://soft-dev.org/pubs/ 6 / 6 http://soft-dev.org/
  • Numpy on PyPy Maciej Fijałkowski King’s College London August 27 2013 fijal Numpy on PyPy
  • Goals fully compliant numpy replacement for PyPy fast looped operations fast vectorized operations fijal Numpy on PyPy
  • Why? fast looping single language fijal Numpy on PyPy
  • Model some programs have numerical kernels that can be written in C some don’t http://arxiv.org/abs/1301.1334 image manipulation demo abstraction unfriendly fijal Numpy on PyPy
  • Status fast looped operations ok vectorized operations fijal Numpy on PyPy
  • Future goals finish numpy make it fast make it compatible with more software (matplotlib, scipy) fijal Numpy on PyPy
  • Funding about $20k left we likely need more behind schedule, but not behind budget fijal Numpy on PyPy
  • Q&A Thank you! fijal Numpy on PyPy
  • Software Transactional Memory on PyPy
  • Pseudo-Goal “Kill the GIL” GIL = Global Interpreter Lock
  • Real Goals Multi-core programming But reasonable multi-core programming Using the recent model of Transactional Memory
  • PyPy-STM An executable pypy-stm which uses internally Software Transactional Memory Optimistically run multiple threads in parallel The only new feature is atomic: with atomic: piece of code...
  • Example of higher-level API def work(...): ... several more calls to: transaction.add(work, ...) ... Starts N threads, scheduling work() calls to them Each work() is done in an atomic block Multi-core, but as if all the work() are done sequentially
  • Status Kind of working without the JIT Roughly three times slower (you need four cores to see benefits) Working on the JIT support
  • Q&A Thank you! Budget of $10k left, likely more needed too
  • Unipycation Combining Prolog and Python Edd Barrett August 27, 2013 1 / 9 http://soft-dev.org/
  • Our Goal The softdev team is exploring language compositions. Ideally our compositions should be: Easy to implement. Transparent (as possible) to the user-programmer. High performance. Can meta-tracing help? 2 / 9 http://soft-dev.org/
  • Unipycation: A Language Composition Experiment PyPy + Pyrolog = Unipycation Unipycation Both interpreters implemented in RPython. About 600 LoC of integration code. A few months to develop. Languages communicate via an API. No syntax integration yet. 3 / 9 http://soft-dev.org/
  • Why? Explore composition of opposing paradigms. Evaluate RPython as a language composition framework. Performance/ease of development. Composition with many realistic applications. Example Scenario Data acquisition by JSON/XML/Sqlite. Easy in Python, not easy in Prolog. Some kind of knowledge inference based upon data. Perhaps not so easy in Python, trivial in Prolog. Visualisation of Results. Easy in Python, lack of library support in Prolog. 4 / 9 http://soft-dev.org/
  • Example Suppose we have a directed graph (London Underground?): a c b d e d g And we need to ask questions like: Where can I get to from ’b’ via at most 4 nodes and how? 5 / 9 http://soft-dev.org/
  • This is Easy with Prolog path.pl: edge(a, c). edge(c, b). edge(c, d). edge(d, e). edge(b, e). edge(c, f). edge(f, g). edge(e, g). edge(g, b). path(From , To , MaxLen , Nodes) :- path(From , To , MaxLen , Nodes , 1). path(Node , Node , _, [Node], _). path(From , To , MaxLen , [From | Ahead ], Len) :- Len < MaxLen , edge(From , Next), Len1 is Len + 1, path(Next , To , MaxLen , Ahead , Len1). query: path(b, To, 4, Path). 6 / 9 http://soft-dev.org/
  • Example: Python → Prolog from uni import Engine engine = Engine.from_file (" path.pl") paths = engine.db.path.iter for (to , nodes) in paths ("b", None , 4, None): print ("To %s via %s" % (to , nodes)) To b via [’b’] To e via [’b’, ’e’] To g via [’b’, ’e’, ’g’] To b via [’b’, ’e’, ’g’, ’b’] Calling from Prolog to Python also possible. E.g. python:somefunc(blah) 7 / 9 http://soft-dev.org/
  • Performance 8 / 9 http://soft-dev.org/
  • In Summary Compositions are relatively easy to implement with RPython. We were able to implement a fairly transparent API-like interface. Performance promising. Further Reading https://bitbucket.org/vext01/pypy http://soft-dev.org/ 9 / 9 http://soft-dev.org/
  • Looking for Python developers! Speak to us at PyPy demo night or visit www.pitchup.com/jobs Python / Django Postgres Celery Redis nginx memcache Jquery Solr S3 Leading booking site for campsites and caravan parks, founded in 2009 by lastminute.com alumni ● 65k visits / day, £6m bookings / year ● 650 bookable sites ● Huge market ○ 26k campsites and 300m bednights in Europe ○ 600m bednights in US ○ 47% more bednights than hotels (GB) ○ More trips to campsites than holidays to France + Spain combined (GB) ● Team of 15, based in west London