My talk on modern Python performance engineering at PiterPy 2017

1. Python performance
engineering in 2017
Alex Chistyakov, an engineer @ some company

2. Agenda
- Is Python slow?
- How to profile a Python program?
- We are not going to talk about blockchain, sorry
2

3. Is Python slow?
- Definitely, because of GIL and lack of proper JIT
3

4. Is Python slow?
- Definitely, because of GIL and a lack of a proper JIT
compiler
4

5. Is Python slow?
- Seriously, how do you define “to be slow”?
5

6. Okay, let’s do our homework finally
- Vagrant + https://github.com/alexclear/piterpy2017
6

7. Okay, let’s do our homework finally
- Vagrant + https://github.com/alexclear/piterpy2017
- 2-cores VirtualBox VM
7

8. Okay, let’s do our homework finally
- Vagrant + https://github.com/alexclear/piterpy2017
- 2-cores VirtualBox VM
- Something called Sanic:
https://github.com/channelcat/sanic because it’s 2017
and 2.x era should have ended years ago
8

9. Okay, let’s do our homework finally
- Vagrant + https://github.com/alexclear/piterpy2017
- 2-cores VirtualBox VM
- Something called Sanic:
https://github.com/channelcat/sanic because it’s 2017
and 2.x era should have ended years ago
- A load testing tool called ab because we love classics
9

10. What is a typical Python program?
- https://goo.gl/Abz8LX (Please note that this link will
produce different results in 5 years)
- Something not too server-side (or a framework)
- Something scientifical
- Some utility
10

11. What is a typical Python program?
- https://goo.gl/Abz8LX (Please note that this link will
produce different results in 5 years)
- Something not too server-side
- Something scientifical
- Some utility
- An async web application!
11

12. A crash course in modern profiling
- Collect stack samples at all costs
12

13. A crash course in modern profiling
- Collect stack samples at all costs
- Convert collected samples to Brendan Gregg’s
flamegraph.pl format
13

14. A crash course in modern profiling
- Collect stack samples at all costs
- Convert collected samples to Brendan Gregg’s
flamegraph.pl format
- Create a flamegraph
14

15. Collecting samples
15

16. Collecting samples
- Uber’s pyflame: https://github.com/uber/pyflame
16

17. A bit of internal dark magic
- pyflame uses ptrace(2) system call
17

18. A bit of internal dark magic
- pyflame uses ptrace(2) system call
- It attaches to a running process and traverses its memory
18

19. Limitations
- Linux only
- CPython only
- Should have enough privileges
- Can be slow (again, what is “to be slow”? I honestly have
no idea)
19

20. Enough on that, let’s get real
- Starting the server:
python3 ./sanic-app.py 1>/dev/null 2>/dev/null
- (It does basically nothing, just “hello world” stuff)
20

21. Enough on that, let’s get real
- Stressing the server:
ab -n 300000 -c 300 http://127.0.0.1:8000/
- Please note that using ab for real world simulations is
totally insane, but this is exactly what we want now
21

22. How slow is pyflame?
- We run tests with and without pyflame
22

23. Interactive demo time!
23

24. Conclusions
- Pyflame is surprisingly fast!
- Python is not slow, it’s just drawn that way
24

25. Questions, please?
- alexclear@gmail.com
- Skype, Telegram: demeliorator
- Shameless advertising: http://t.me/lhommequipleure
25