Managing and Monitoring Application Performance

6,097 views

Published on

Writing your application is one thing. Making the application to perform well is another. We usually forget there is somebody else on the other side of the screen, that becomes very frustrated and upset when he needs to wait until this one page finally loads. It requires a lot of experience to predict specific behaviour and to know what kind of things to avoid. And even with that there is so many different factors that can affect the end user experience. During this talk I will talk about tools and techniques you can use to measure and monitor your application performance.

Published in: Technology

Managing and Monitoring Application Performance

  1. 1. Sebastian Marek, Internal Systems Technical Architect MANAGING AND MONITORING APPLICATION PERFORMANCE http://www.flickr.com/photos/wraith34/9377593973/
  2. 2. a Pole living in Sheffield over 12 years in software development big fan of process automation TDD and CI occasionally contributes to open- source projects wants to be a knight @proofek h"ps://joind.in/9301
  3. 3. HTTPS://WWW.CODECLUB.ORG.UK/ A nationwide network of volunteer-led after school coding clubs for children aged 9-11 Volunteer Code Club Projects Venue Children + + +
  4. 4. PERFORMANCE MONITORING Overview network level system level application level End-user level http://www.flickr.com/photos/global-jet/483825828/
  5. 5. PERFORMANCE MONITORING Network level failed TCP connections and packet losses bandwidth usage firewalls and loadbalancers connectivity to external serviceshttp://www.flickr.com/photos/sbrown6079/2038502257/
  6. 6. PERFORMANCE MONITORING System level CPU load memory consumption disk space http://www.flickr.com/photos/viagallery/3178699697/
  7. 7. PERFORMANCE MONITORING Application level new deployments response time key transactions memory usage http://www.flickr.com/photos/qualityfrog/3546657245/
  8. 8. PERFORMANCE MONITORING End-user level response time user experience http://www.flickr.com/photos/alesk/356136498/
  9. 9. remote debugging code coverage tracing profiling Xdebug
  10. 10. Installation XDEBUG # pecl install xdebug zend_extension=”/usr/lib/php/extensions/xdebug.so” configure in xdebug.ini web server restart (optional)
  11. 11. Configuration XDEBUG ;Profiling settings (always on) xdebug.profiler_enable = 1 xdebug.profiler_output_dir = /var/log/xdebug/ ;Profiling settings (triggered by GET/POST/COOKIE) xdebug.profiler_enable = 0 xdebug.profiler_output_dir = /var/log/xdebug/ xdebug.profiler_enable_trigger = 1
  12. 12. Browser integration XDEBUG
  13. 13. Profiling data XDEBUG fl=php:internal fn=php::spl_autoload_register 881 3 fl=/Users/Sebastian/git/app/src/include/HTMLPurifier/HTMLPurifier.standalone.php fn=HTMLPurifier_Bootstrap::registerAutoload 878 21 cfl=php:internal cfn=php::spl_autoload_functions calls=1 0 0 880 1 cfl=php:internal cfn=php::spl_autoload_register calls=1 0 0 881 3
  14. 14. Profiling tools XDEBUG KCacheGrind for *nix systems qcachegrind for MacOsX (and apparently Windows) MacCallGrind for MacOsX WebGrind via browser
  15. 15. qCacheGrind XDEBUG
  16. 16. WebGrind XDEBUG
  17. 17. profiling web interface xhProf
  18. 18. Installation XHPROF # pecl install xhprof extension=”/usr/lib/php/extensions/xhprof.so” configure in xhprof.ini web server restart (optional)
  19. 19. Configuration XHPROF ; XHProf extension = xhprof.so xhprof.output_dir = /var/log/xhprof/
  20. 20. Profiling tools XHPROF default web interface xhGui (old, also known as XH UI) xhGui (new, with MongoDB backend)
  21. 21. Default web interface - Installation XHPROF git clone git@github.com:facebook/xhprof.git
  22. 22. Default web interface - Installation XHPROF ;Webserver (VirtualHost) <VirtualHost *:80> DocumentRoot “/Users/Sebastian/git/xhprof/xhprof_html” ServerName xhprof.localdomain <Directory /Users/Sebastian/git/xhprof/xhprof_html> Order allow,deny Allow from all </Directory> ErrorLog “/private/var/log/apache2/xhprof-error_log” CustomLog “/private/var/log/apache2/xhprof-access_log” common </VirtualHost>
  23. 23. Default web interface - Installation XHPROF ;auto_prepend file - enable xhprof $xhProfHome = __DIR__ . ‘/..’; $profiler_namespace = ‘myapp’; // namespace for your application if (extension_loaded(‘xhprof’)) { include_once “$xhProfHome/xhprof_lib/utils/xhprof_lib.php”; include_once “$xhProfHome/xhprof_lib/utils/xhprof_runs.php”; xhprof_enable(XHPROF_FLAGS_CPU | XHPROF_FLAGS_MEMORY); }
  24. 24. Default web interface - Installation XHPROF ;auto_prepend file - collect data function xhprofShutdownFunction() { global $profiler_namespace; $xhProfUrl = “http://xhprof.localdomain/”; if (extension_loaded(‘xhprof’)) { $xhprof_data = xhprof_disable(); $xhprof_runs = new XHProfRuns_Default(); $run_id = $xhprof_runs->save_run($xhprof_data, $profiler_namespace); $profiler_url = sprintf(“$xhProfUrl/index.php?run=%s&source=%s”, $run_id, $profiler_namespace); echo “<a href=’$profiler_url’ target=‘_blank’>Profiler output</a>”; } } register_shutdown_function(‘xhprofShutdownFunction’);
  25. 25. Default web interface - Installation XHPROF ;enable profiling in your application (VirtualHost) <VirtualHost *:80> DocumentRoot “/Users/Sebastian/git/my_app/src” ServerName my_app.localdomain <Directory /Users/Sebastian/git/my_app/src > Order allow,deny Allow from all </Directory> php_admin_value auto_prepend_file /Users/Sebastian/git/xhprof/xhprof_html/header.php </VirtualHost>
  26. 26. Default web interface - Home page XHPROF
  27. 27. Default web interface - Drilldown XHPROF
  28. 28. Default web interface - Call graph XHPROF
  29. 29. Default web interface - Full Call graph XHPROF
  30. 30. XH UI - Installation XHPROF git clone git@github.com:preinheimer/xhprof.git
  31. 31. XH UI - Installation XHPROF ;Webserver (VirtualHost) <VirtualHost *:80> DocumentRoot “/Users/Sebastian/git/xhui/xhprof_html” ServerName xhui.localdomain <Directory /Users/Sebastian/git/xhui/xhprof_html> Order allow,deny Allow from all </Directory> ErrorLog “/private/var/log/apache2/xhui-error_log” CustomLog “/private/var/log/apache2/xhui-access_log” common </VirtualHost>
  32. 32. XH UI - Configuration XHPROF # vim xhui/xhprof_lib/config.php and reuse already provided xhui/external/header.php
  33. 33. XH UI - Installation XHPROF ;enable profiling in your application (VirtualHost) <VirtualHost *:80> DocumentRoot “/Users/Sebastian/git/my_app/src” ServerName my_app.localdomain <Directory /Users/Sebastian/git/my_app/src > Order allow,deny Allow from all </Directory> php_admin_value auto_prepend_file /Users/Sebastian/git/xhui/external/header.php </VirtualHost>
  34. 34. XH UI - Home page XHPROF
  35. 35. XH UI - Transaction detail XHPROF
  36. 36. XH UI - Transaction detail XHPROF
  37. 37. XHG - Installation XHPROF git clone git@github.com:preinheimer/xhgui.git
  38. 38. XHG - Installation XHPROF ;Webserver (VirtualHost) <VirtualHost *:80> DocumentRoot “/Users/Sebastian/git/xhgui/webroot” ServerName xhgui.localdomain <Directory /Users/Sebastian/git/xhgui/webroot> Order allow,deny Allow from all AllowOverride All </Directory> ErrorLog “/private/var/log/apache2/xhgui-error_log” CustomLog “/private/var/log/apache2/xhgui-access_log” common </VirtualHost>
  39. 39. XHG - Requirements XHPROF MongoDB server PHP MongoDB extension PHP mcrypt extension
  40. 40. XHG - Installation XHPROF # cd xhgui # php install.php
  41. 41. XHG - Installation XHPROF ;enable profiling in your application (VirtualHost) <VirtualHost *:80> DocumentRoot “/Users/Sebastian/git/my_app/src” ServerName my_app.localdomain <Directory /Users/Sebastian/git/my_app/src > Order allow,deny Allow from all </Directory> php_admin_value auto_prepend_file /Users/Sebastian/git/xhgui/external/header.php </VirtualHost>
  42. 42. XHG - Home page XHPROF
  43. 43. XHG - Transaction trace XHPROF
  44. 44. XHG - Transaction trace XHPROF
  45. 45. XHG - Transaction trace XHPROF
  46. 46. XHG - Transaction history XHPROF
  47. 47. XHG - Transaction call graph XHPROF
  48. 48. Glossary PROFILING call count - number of times a function/request was called wt (Wall Time/Wall Clock Time) - time it took to execute a function/request cpu - CPU time spent executing a function/request mu - Amount of memory used during function/request execution pmu - The peak amount of memory used during function/request execution exclusive - total time spent excluding time to execute other functions inclusive - total time spent including time to execute other functions
  49. 49. Process PROFILING Profile and record the results Identify functions/requests using the most CPU time/memory (exclusive) Find out what’s causing the issue Refactor and optimize the code - look for external resources - database, web services, filesystem Profile and compare the results start all over again
  50. 50. server monitoring application monitoring real user monitoring NewRelic
  51. 51. Supported web applications NEW RELIC
  52. 52. Supported PHP framework NEW RELIC
  53. 53. Installation NEW RELIC # RHEL $ rpm -Uvh http://yum.newrelic.com/pub/newrelic/el5/x86_64/newrelic- repo-5-3.noarch.rpm $ yum install newrelic-php5 $ /usr/bin/newrelic-install # Ubuntu / Debian $ wget -O - https://download.newrelic.com/548C16BF.gpg | sudo apt-key add - deb http://apt.newrelic.com/debian/ newrelic non-free $ apt-get install newrelic-php5 $ /usr/bin/newrelic-install
  54. 54. Server Monitoring NEW RELIC
  55. 55. Server Monitoring - Main Dashboard NEW RELIC Portal01 App01 App02 Portal02 SugarCRM SugarCRM Support Centre
  56. 56. Server Monitoring - Overview NEW RELIC App02 App02 SugarCRM (App02)
  57. 57. Application Monitoring NEW RELIC
  58. 58. Server Trace - Summary NEW RELIC App01(5.3.19+a2h(2.2.3):SugarCRM)
  59. 59. Server Trace - Details NEW RELIC
  60. 60. Server Trace - SQL queries NEW RELIC
  61. 61. Application Monitoring - Error Tracking NEW RELIC
  62. 62. Real User Monitoring - App Dashboard NEW RELIC
  63. 63. Understanding RUM NEW RELIC Web application - The time spent in the application code Network - The time it takes for a request to make a round-trip over the internet. DOM processing - Time spent in the browser parsing and interpreting the HTML. Page rendering - Time spent in the browser displaying the HTML, running in-line JavaScript and loading images.
  64. 64. Understanding RUM NEW RELIC Source: http://blog.newrelic.com
  65. 65. Real User Monitoring - Browser Page Load NEW RELIC
  66. 66. Real User Monitoring - Browser Page Load NEW RELIC
  67. 67. Apdex NEW RELIC
  68. 68. Apdex NEW RELIC 0 T F Satisfied zone: < T Tolerating zone: T - F (4T) Frustrated zone: > F
  69. 69. Apdex score NEW RELIC Satisfied customers + (Tolerating customers/2) Total number of customers
  70. 70. Apdex by regions NEW RELIC
  71. 71. Apdex - baseline NEW RELIC check the average response time set your Apdex to average response time find areas that require performance fixes when performance improves lower your Apdex
  72. 72. New Relic - Map NEW RELIC memcached.local crm.local
  73. 73. Keeping on top of the changes NEW RELIC
  74. 74. Server Monitoring - plugins NEW RELIC
  75. 75. Server Monitoring - APC NEW RELIC
  76. 76. Server Monitoring - MySQL NEW RELIC
  77. 77. Server Monitoring - Apache NEW RELIC
  78. 78. PERFORMANCE MONITORING Summary Xdebug - application profiling during development stage xhProf - application profiling during development stage and on live production platform New Relic - end-to-end application monitoring on live production platfrom
  79. 79. PERFORMANCE MONITORING Alternatives DataDog - http://www.datadoghq.com/product/ GraphDat - http://www.graphdat.com Scout - https://scoutapp.com/ Nagios - http://www.nagios.org/ Graphite - http://graphite.wikidot.com
  80. 80. Resources Xdebug - http://www.xdebug.org/docs/profiler kCacheGrind - http://kcachegrind.sourceforge.net WebGrind - https://github.com/jokkedk/webgrind MacCallGrind for MacOsX - http://www.maccallgrind.com xhProf - https://github.com/facebook/xhprof xhUI - https://github.com/preinheimer/xhprof xhGui - https://github.com/preinheimer/xhgui New Relic - http://newrelic.com New Relic Documentation - https://newrelic.com/docs Apdex - http://apdex.org http://techportal.inviqa.com/2009/12/01/profiling-with-xhprof/ http://techportal.inviqa.com/2013/10/01/profiling-php-applications-with-xhgui/
  81. 81. Q & A h"ps://joind.in/9301

×