The document summarizes transactional memory (TM) and software transactional memory (STM) approaches for concurrency in Python. It discusses removing the global interpreter lock (GIL) in Python through fine-grained locking, shared-nothing, and STM approaches. It provides examples of STM usage in Python and describes the hardware and software support for STM. It also outlines some current limitations of the PyPy STM implementation for Python including memory and performance limitations.

1. STM
Francisco Fernandez Castano
@fcofdezc
Francisco.fernandez.castano@gmail.com
Upclose.me

4. The Free Lunch Is Over: A
Fundamental Turn Toward
Concurrency in Software
Herb Sutter
http://www.gotw.ca/publications/concurrency-ddj.htm

6. Inside the Python GIL
David Beazley
https://www.youtube.com/watch?v=ph374fJqFPE

8. Removing the GIL
● Fine-grained locking
● Shared-nothing
● STM

9. BEGIN;
UPDATE accounts SET balance = balance - 100.00
WHERE name = 'Alice';
UPDATE accounts SET balance = balance + 100.00
WHERE name = 'Bob';
COMMIT;

10. def update_account(src_account_id, dst_account_id, amount):
with atomic:
src_account = accounts[src_account_id]
dst_account = acconts[dst_account_id]
dst_account.deposit(account.withdraw(amount))

11. ● Hardware and Software support
● Some modern processors (Haswell) support it
– https://software.intel.com/en-us/blogs/2012/02/07/transactional-synchronization-in-haswell

12. ● Transaction is started
● Memory read are logged into read set
● Memory writes are logged into write set
● Commit is attempted

14. PyPy approach

15. Illustration taken from Armin Rigo

16. Problems
● Too experimental
● Only runs in 64 bit Linux and patched version of
Clang (just for translation)
● Limited to 1.5 GB of Ram (Configurable)
● JIT warm up still pretty bad
● Different GC from regular PyPy (Less efficient)
● Long running processes crash

17. Demo

18. References
● Rajwar, Herlihy and Lai:
Virtualizing Transactional Memory
● Herlihy and Moss: Transactional Memory:
Architectural Support for Lock-Free Data
Structures
● Dice, Shalev, Shavit: Transactional Locking II

19. Donations
● http://pypy.org/tmdonate2.html