As hackers we love to understand how stuff works and how to optimize it. A very good tool to do both is software tracing. During the talk we'll see how tracing tools work and we'll zoom on one particular project called pyflame.

1. Tracing python applications
@stoitsev

3. “Bottlenecks occur in surprising places, so don't try
to second guess and put in a speed hack until you
have proven that's where the bottleneck is.”
Rob Pike

4. Measure Change

5. Software tracing

6. Deterministic Profilers

8. Example
https://pymotw.com/2/sys/tracing.html

9. cProfile
https://docs.python.org/2/library/profile.html

11. https://pymotw.com/2/profile/

12. https://pymotw.com/2/profile/

13. High overhead

14. No full call stack information

15. Services not written for profiling

16. Sampling Profilers

17. Python

18. Pyflame
https://github.com/uber/pyflame

19. Collect the full Python stack

20. Have low overhead

21. Just works for existing code

22. Can do flame graphs

23. Flame graph

24. http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html

25. http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html

26. main()
print()
compute_seq() send()
compute()
get() find_solution()
slow_multiplication()

27. main()
print()
compute_seq() send()
compute()
get() find_solution()
slow_multiplication()

28. main()
print()
compute_seq() send()
compute()
get() find_solution()
slow_multiplication()

29. https://pymotw.com/2/profile/

30. Demo

32. $ pyflame <PID>

33. script.py:<module>:27;script.py:main:23 122
script.py:<module>:27;script.py:main:22 34
script.py:<module>:27;script.py:main:23;script.py:run:13 11
script.py:<module>:27;script.py:main:23;script.py:run:15 114
script.py:<module>:27;script.py:main:23;script.py:run:17 141
script.py:<module>:27;script.py:main:23;script.py:run:16 95
script.py:<module>:27;script.py:main:23;script.py:run:14 4
script.py:<module>:27;script.py:main:23;script.py:run:16;script.py:square:9;script.py:multiply:5 44
script.py:<module>:27;script.py:main:23;script.py:run:15;script.py:square:9;script.py:multiply:5 41
script.py:<module>:27;script.py:main:23;script.py:run:17;script.py:multiply:5 39
script.py:<module>:27;script.py:main:23;script.py:run:16;script.py:square:9 100
script.py:<module>:27;script.py:main:23;script.py:run:15;script.py:square:9 119

34. pyflame <PID> | ./flamegraph.pl profile.svg
From https://github.com/brendangregg/FlameGraph

37. https://github.com/uber/pyflame

38. Thanks!

39. eng.uber.com