Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Profiling em Python

635 views

Published on

Slides da palestra sobre profiling em python dada na PythonBrasil[9] em 2013

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Profiling em Python

  1. 1. globo .com Profiling em Python Friday, October 4, 13
  2. 2. porque profiling é importante? Friday, October 4, 13
  3. 3. Friday, October 4, 13
  4. 4. Friday, October 4, 13
  5. 5. então vamos otimizar tudo! Friday, October 4, 13
  6. 6. Friday, October 4, 13
  7. 7. “We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil” - Donald Knuth Friday, October 4, 13
  8. 8. from timeit import timeit if __name__ == "__main__": setup = "from htmlmin.minify import html_minify;" setup += "from data import raw_html" t = timeit( stmt="html_minify(raw_html)", setup=setup, number=100) print(t) benchmark.py Friday, October 4, 13
  9. 9. $ python benchmark.py 25.8121981621 Friday, October 4, 13
  10. 10. conheça seu código Friday, October 4, 13
  11. 11. ‣ cProfile ‣ Profile ‣ hotshot (deprecated) ‣ trace ‣ line profiler ‣ memory profiler Friday, October 4, 13
  12. 12. from data import raw_html from htmlmin.minify import html_minify if __name__ == "__main__": html_minify(raw_html) profile.py Friday, October 4, 13
  13. 13. $ python -m cProfile profile.py Friday, October 4, 13
  14. 14. Friday, October 4, 13
  15. 15. Friday, October 4, 13
  16. 16. Friday, October 4, 13
  17. 17. Friday, October 4, 13
  18. 18. Friday, October 4, 13
  19. 19. Friday, October 4, 13
  20. 20. Friday, October 4, 13
  21. 21. Friday, October 4, 13
  22. 22. Friday, October 4, 13
  23. 23. Friday, October 4, 13
  24. 24. $ python -m cProfile -o out profile.py Friday, October 4, 13
  25. 25. $ python -m cProfile -o out profile.py Friday, October 4, 13
  26. 26. >>> import pstats >>> p = pstats.Stats("out") >>> p.sort_stats("cumulative").print_stats(10) 315165 function calls (311828 primitive calls) in 1.334 seconds Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py: 7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py: 17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py: 1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py: 2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py: 12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py: 26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py: 80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py: 193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py: 33(feed) Friday, October 4, 13
  27. 27. >>> import pstats >>> p = pstats.Stats("out") >>> p.sort_stats("cumulative").print_stats(10) 315165 function calls (311828 primitive calls) in 1.334 seconds Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py: 7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py: 17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py: 1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py: 2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py: 12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py: 26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py: 80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py: 193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py: 33(feed) Friday, October 4, 13
  28. 28. >>> import pstats >>> p = pstats.Stats("out") >>> p.sort_stats("cumulative").print_stats(10) 315165 function calls (311828 primitive calls) in 1.334 seconds Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py: 7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py: 17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py: 1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py: 2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py: 12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py: 26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py: 80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py: 193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py: 33(feed) Friday, October 4, 13
  29. 29. >>> import pstats >>> p = pstats.Stats("out") >>> p.sort_stats("cumulative").print_stats(10) 315165 function calls (311828 primitive calls) in 1.334 seconds Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py: 7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py: 17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py: 1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py: 2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py: 12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py: 26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py: 80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py: 193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py: 33(feed) Friday, October 4, 13
  30. 30. >>> import pstats >>> p = pstats.Stats("out") >>> p.sort_stats("cumulative").print_stats(10) 315165 function calls (311828 primitive calls) in 1.334 seconds Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py: 7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py: 17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py: 1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py: 2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py: 12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py: 26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py: 80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py: 193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py: 33(feed) Friday, October 4, 13
  31. 31. >>> import pstats >>> p = pstats.Stats("out") >>> p.sort_stats("cumulative").print_stats(10) 315165 function calls (311828 primitive calls) in 1.334 seconds Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py: 7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py: 17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py: 1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py: 2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py: 12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py: 26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py: 80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py: 193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py: 33(feed) Friday, October 4, 13
  32. 32. >>> import pstats >>> p = pstats.Stats("out") >>> p.sort_stats("cumulative").print_stats(10) 315165 function calls (311828 primitive calls) in 1.334 seconds Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py: 7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py: 17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py: 1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py: 2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py: 12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py: 26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py: 80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py: 193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py: 33(feed) Friday, October 4, 13
  33. 33. $ kernprof.py -l -v minify.py Friday, October 4, 13
  34. 34. Friday, October 4, 13
  35. 35. Friday, October 4, 13
  36. 36. Friday, October 4, 13
  37. 37. Friday, October 4, 13
  38. 38. Friday, October 4, 13
  39. 39. Friday, October 4, 13
  40. 40. Friday, October 4, 13
  41. 41. $ python -m memory_profiler minify.py Friday, October 4, 13
  42. 42. Friday, October 4, 13
  43. 43. Friday, October 4, 13
  44. 44. Friday, October 4, 13
  45. 45. Friday, October 4, 13
  46. 46. Friday, October 4, 13
  47. 47. gui? Friday, October 4, 13
  48. 48. Friday, October 4, 13
  49. 49. Friday, October 4, 13
  50. 50. outras ferramentas ‣ meliae ‣ heapy (guppy) ‣ benchy ‣ valgrind ‣ python object graphs (objgraph) ‣ plop ‣ pycounters Friday, October 4, 13
  51. 51. bônus Friday, October 4, 13
  52. 52. django profiling Friday, October 4, 13
  53. 53. algumas ferramentas ‣ django-debug-toolbar ‣ django-profiler ‣ new relic Friday, October 4, 13
  54. 54. Friday, October 4, 13
  55. 55. $ newrelic-admin run-program gunicorn -w 3 wsgi:application Friday, October 4, 13
  56. 56. Friday, October 4, 13
  57. 57. globo .com Estamos contratando! Friday, October 4, 13
  58. 58. globo .com obrigada! @flaviamissi Friday, October 4, 13

×