The document discusses Python profiling tools for optimizing code performance. It introduces common profiling tools like cProfile and memory_profiler and demonstrates using them to analyze a minification script. Additional tools like meliae, heapy, and valgrind are listed. Profiling Django applications is also covered, with mentions of debug toolbar, django-profiler, and New Relic.

1. globo
.com Profiling em Python
Friday, October 4, 13

2. porque profiling é importante?
Friday, October 4, 13

3. Friday, October 4, 13

4. Friday, October 4, 13

5. então vamos otimizar tudo!
Friday, October 4, 13

6. Friday, October 4, 13

7. “We should forget about small efficiencies, say
about 97% of the time: premature optimization
is the root of all evil” - Donald Knuth
Friday, October 4, 13

8. from timeit import timeit
if __name__ == "__main__":
setup = "from htmlmin.minify import html_minify;"
setup += "from data import raw_html"
t = timeit(
stmt="html_minify(raw_html)",
setup=setup,
number=100)
print(t)
benchmark.py
Friday, October 4, 13

9. $ python benchmark.py
25.8121981621
Friday, October 4, 13

10. conheça seu código
Friday, October 4, 13

11. ‣ cProfile
‣ Profile
‣ hotshot (deprecated)
‣ trace
‣ line profiler
‣ memory profiler
Friday, October 4, 13

12. from data import raw_html
from htmlmin.minify import html_minify
if __name__ == "__main__":
html_minify(raw_html)
profile.py
Friday, October 4, 13

13. $ python -m cProfile profile.py
Friday, October 4, 13

14. Friday, October 4, 13

15. Friday, October 4, 13

16. Friday, October 4, 13

17. Friday, October 4, 13

18. Friday, October 4, 13

19. Friday, October 4, 13

20. Friday, October 4, 13

21. Friday, October 4, 13

22. Friday, October 4, 13

23. Friday, October 4, 13

24. $ python -m cProfile -o out profile.py
Friday, October 4, 13

25. $ python -m cProfile -o out profile.py
Friday, October 4, 13

26. >>> import pstats
>>> p = pstats.Stats("out")
>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time
List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.092 0.092 1.358 1.358 profile.py:2(<module>)
1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:
7(<module>)
1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:
17(<module>)
1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:
1(<module>)
1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:
2(<module>)
1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:
12(<module>)
1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:
26(html_minify)
2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:
80(__init__)
2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:
193(_feed)
2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:
33(feed)
Friday, October 4, 13

27. >>> import pstats
>>> p = pstats.Stats("out")
>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time
List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.092 0.092 1.358 1.358 profile.py:2(<module>)
1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:
7(<module>)
1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:
17(<module>)
1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:
1(<module>)
1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:
2(<module>)
1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:
12(<module>)
1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:
26(html_minify)
2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:
80(__init__)
2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:
193(_feed)
2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:
33(feed)
Friday, October 4, 13

28. >>> import pstats
>>> p = pstats.Stats("out")
>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time
List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.092 0.092 1.358 1.358 profile.py:2(<module>)
1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:
7(<module>)
1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:
17(<module>)
1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:
1(<module>)
1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:
2(<module>)
1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:
12(<module>)
1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:
26(html_minify)
2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:
80(__init__)
2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:
193(_feed)
2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:
33(feed)
Friday, October 4, 13

29. >>> import pstats
>>> p = pstats.Stats("out")
>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time
List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.092 0.092 1.358 1.358 profile.py:2(<module>)
1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:
7(<module>)
1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:
17(<module>)
1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:
1(<module>)
1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:
2(<module>)
1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:
12(<module>)
1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:
26(html_minify)
2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:
80(__init__)
2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:
193(_feed)
2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:
33(feed)
Friday, October 4, 13

30. >>> import pstats
>>> p = pstats.Stats("out")
>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time
List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.092 0.092 1.358 1.358 profile.py:2(<module>)
1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:
7(<module>)
1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:
17(<module>)
1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:
1(<module>)
1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:
2(<module>)
1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:
12(<module>)
1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:
26(html_minify)
2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:
80(__init__)
2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:
193(_feed)
2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:
33(feed)
Friday, October 4, 13

31. >>> import pstats
>>> p = pstats.Stats("out")
>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time
List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.092 0.092 1.358 1.358 profile.py:2(<module>)
1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:
7(<module>)
1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:
17(<module>)
1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:
1(<module>)
1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:
2(<module>)
1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:
12(<module>)
1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:
26(html_minify)
2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:
80(__init__)
2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:
193(_feed)
2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:
33(feed)
Friday, October 4, 13

32. >>> import pstats
>>> p = pstats.Stats("out")
>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time
List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.092 0.092 1.358 1.358 profile.py:2(<module>)
1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:
7(<module>)
1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:
17(<module>)
1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:
1(<module>)
1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:
2(<module>)
1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:
12(<module>)
1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:
26(html_minify)
2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:
80(__init__)
2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:
193(_feed)
2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:
33(feed)
Friday, October 4, 13

33. $ kernprof.py -l -v minify.py
Friday, October 4, 13

34. Friday, October 4, 13

35. Friday, October 4, 13

36. Friday, October 4, 13

37. Friday, October 4, 13

38. Friday, October 4, 13

39. Friday, October 4, 13

40. Friday, October 4, 13

41. $ python -m memory_profiler minify.py
Friday, October 4, 13

42. Friday, October 4, 13

43. Friday, October 4, 13

44. Friday, October 4, 13

45. Friday, October 4, 13

46. Friday, October 4, 13

47. gui?
Friday, October 4, 13

48. Friday, October 4, 13

49. Friday, October 4, 13

50. outras ferramentas
‣ meliae
‣ heapy (guppy)
‣ benchy
‣ valgrind
‣ python object graphs (objgraph)
‣ plop
‣ pycounters
Friday, October 4, 13

51. bônus
Friday, October 4, 13

52. django profiling
Friday, October 4, 13

53. algumas ferramentas
‣ django-debug-toolbar
‣ django-profiler
‣ new relic
Friday, October 4, 13

54. Friday, October 4, 13

55. $ newrelic-admin run-program gunicorn -w 3
wsgi:application
Friday, October 4, 13

56. Friday, October 4, 13

57. globo
.com Estamos contratando!
Friday, October 4, 13

58. globo
.com
obrigada!
@flaviamissi
Friday, October 4, 13