Little presentation about PyPy STM

1. STM
Francisco Fernandez Castano
@fcofdezc
Francisco.fernandez.castano@gmail.com
Upclose.me

4. The Free Lunch Is Over: A
Fundamental Turn Toward
Concurrency in Software
Herb Sutter
http://www.gotw.ca/publications/concurrency-ddj.htm

6. Inside the Python GIL
David Beazley
https://www.youtube.com/watch?v=ph374fJqFPE

8. Removing the GIL
● Fine-grained locking
● Shared-nothing
● STM

9. BEGIN;
UPDATE accounts SET balance = balance - 100.00
WHERE name = 'Alice';
UPDATE accounts SET balance = balance + 100.00
WHERE name = 'Bob';
COMMIT;

10. def update_account(src_account_id, dst_account_id, amount):
with atomic:
src_account = accounts[src_account_id]
dst_account = acconts[dst_account_id]
dst_account.deposit(account.withdraw(amount))

11. ● Hardware and Software support
● Some modern processors (Haswell) support it
– https://software.intel.com/en-us/blogs/2012/02/07/transactional-synchronization-in-haswell

12. ● Transaction is started
● Memory read are logged into read set
● Memory writes are logged into write set
● Commit is attempted

14. PyPy approach

15. Illustration taken from Armin Rigo

16. Problems
● Too experimental
● Only runs in 64 bit Linux and patched version of
Clang (just for translation)
● Limited to 1.5 GB of Ram (Configurable)
● JIT warm up still pretty bad
● Different GC from regular PyPy (Less efficient)
● Long running processes crash

17. Demo

18. References
● Rajwar, Herlihy and Lai:
Virtualizing Transactional Memory
● Herlihy and Moss: Transactional Memory:
Architectural Support for Lock-Free Data
Structures
● Dice, Shalev, Shavit: Transactional Locking II

19. Donations
● http://pypy.org/tmdonate2.html