PHP applications/environments monitoring: APM & Pinba Davide Mendolia - Patrick Allaert PHPBenelux Conference 2010
Davide Mendolia <ul><li>http://davidemendolia.blogspot.com/
dmendolia@php.net or @davideme (Twitter)
Lead developer of  APM
php-benchmark  contributor
http://www.php-ci.net/
Works at  Tuenti </li></ul>
<ul><li>Social network , based in Madrid
More than 20 billion page views per month
First Spanish website ahead Google, Microsoft and Facebook (Comscore)
International work environment (15 nationalities)
Developers wanted! </li></ul>
Patrick Allaert <ul><li>http://patrickallaert.blogspot.com/
patrickallaert@php.net or @patrick_allaert (Twitter)
Lead developer of  APM
Works at  eZ Systems  (eZ Publish, eZ Components)
PHPBenelux  staff member ( http://www.phpbenelux.eu/ ) </li></ul>
Patrick Allaert <ul><li>Start programming with PHP 3
Ten years of Free Software evangelism
Contributions: </li><ul><li>PHP (ext/ldap, QA: PHPT)
KDESvn (Subversion client for KDE)
XOo°f (eXtensible Open Object Oriented Framework)
Tabellio (collaborative suite for parliamentary assembly) </li></ul></ul>
Monitoring: Definition ? <ul><li>Being aware of the state of a system by watching and measuring it.
Indicators: </li><ul><li>the physical condition of a machine
load of a machine
application availability
the messages in system logs
network performance
known attacks </li></ul></ul>
Monitoring: Why? <ul><li>Need in terms of increasing availability
Economic reasons
User experience </li></ul>
Monitoring: Why? « When I get on the homepage I have a blank page […] » « Seriously? One moment, checking […] » « We are a...
Monitoring: Presentation <ul>Monitoring the P of LAMP. In particular: <ul><li>Error handling in PHP
Execution performance. </li></ul></ul>
What are the solutions ? <ul><li>Current: </li><ul><li>PHP
Zend Platform </li></ul><li>Newcomers: </li><ul><li>APM (Alternative PHP Monitor)
Pinba </li></ul></ul>
PHP: integrated functions <ul><li>error_reporting
display_errors
error_log </li><ul><li>file
syslog </li><ul><li>Windows => event log </li></ul></ul><li>set_error_handler() </li></ul>
Upcoming SlideShare
Loading in...5
×

PHP applications/environments monitoring: APM & Pinba

9,568

Published on

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
9,568
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
26
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Surveiller d&apos;eventuellle erreur comportementale de l&apos;application. Surveiller au niveau des performances, afin de prevoir d&apos;eventuel probleme
  • Page de demo avec E_NOTICE et autre, Display avec tail
  • Tail -f /var/log/php/error.log Page vide de apm.php Puis generation d&apos;une page d&apos;erreur Affichage de apm.php Recommecer avec mutilple erreur
  • Rappel, qu&apos;est ce que APC, Xdebug, … Compatbilité totale avec le monde du LL
  • Compatbilité totale avec le monde du LL
  • L&apos;extension PHP Pinba envoie un paquet UDP lors de la terminaison d&apos;une requête qui contient des informations comme: hostname - gethostname() result request_count - number of requests served by this process server_name script_name document_size - size of the response body memory_peak - memory allocation peak request_time - time spent of processing the request ru_utime - resource usage (user) ru_stime - resource usage (system) timers - array of timers (optional) De la nature UDP, aucune connexion n&apos;est réalisée, ce qui permet d&apos;avoir aucun impacte de performance, mais des pertes de paquets pourraient se produire. De l&apos;autre coté, le plugin MySQL Pinba écoute sur le port configuré pour récupérer les paquets et les décoder afin de les stocker dans ses pools d&apos;enregistrements ainsi que de mettre à jour les index, les rapports de bases et rapports de tags si ils existent.
  • Raw data tables contain raw request data (surprise!). Please bear in mind that access to the raw data is relatively slow (the number of records might reach millions and there are NO indexes except for primary keys, so almost any operation requires full table scan). Reports were created in order to speed up the most frequent operations by aggregating data on-the-fly, when new request data arrives.
  • PHP applications/environments monitoring: APM & Pinba

    1. 1. PHP applications/environments monitoring: APM & Pinba Davide Mendolia - Patrick Allaert PHPBenelux Conference 2010
    2. 2. Davide Mendolia <ul><li>http://davidemendolia.blogspot.com/
    3. 3. dmendolia@php.net or @davideme (Twitter)
    4. 4. Lead developer of APM
    5. 5. php-benchmark contributor
    6. 6. http://www.php-ci.net/
    7. 7. Works at Tuenti </li></ul>
    8. 8. <ul><li>Social network , based in Madrid
    9. 9. More than 20 billion page views per month
    10. 10. First Spanish website ahead Google, Microsoft and Facebook (Comscore)
    11. 11. International work environment (15 nationalities)
    12. 12. Developers wanted! </li></ul>
    13. 13. Patrick Allaert <ul><li>http://patrickallaert.blogspot.com/
    14. 14. patrickallaert@php.net or @patrick_allaert (Twitter)
    15. 15. Lead developer of APM
    16. 16. Works at eZ Systems (eZ Publish, eZ Components)
    17. 17. PHPBenelux staff member ( http://www.phpbenelux.eu/ ) </li></ul>
    18. 18. Patrick Allaert <ul><li>Start programming with PHP 3
    19. 19. Ten years of Free Software evangelism
    20. 20. Contributions: </li><ul><li>PHP (ext/ldap, QA: PHPT)
    21. 21. KDESvn (Subversion client for KDE)
    22. 22. XOo°f (eXtensible Open Object Oriented Framework)
    23. 23. Tabellio (collaborative suite for parliamentary assembly) </li></ul></ul>
    24. 24. Monitoring: Definition ? <ul><li>Being aware of the state of a system by watching and measuring it.
    25. 25. Indicators: </li><ul><li>the physical condition of a machine
    26. 26. load of a machine
    27. 27. application availability
    28. 28. the messages in system logs
    29. 29. network performance
    30. 30. known attacks </li></ul></ul>
    31. 31. Monitoring: Why? <ul><li>Need in terms of increasing availability
    32. 32. Economic reasons
    33. 33. User experience </li></ul>
    34. 34. Monitoring: Why? « When I get on the homepage I have a blank page […] » « Seriously? One moment, checking […] » « We are aware of the problem, our engineers have found the solution. Everything will be back to normal in 5 minutes » « Hell! You're right!? »
    35. 35. Monitoring: Presentation <ul>Monitoring the P of LAMP. In particular: <ul><li>Error handling in PHP
    36. 36. Execution performance. </li></ul></ul>
    37. 37. What are the solutions ? <ul><li>Current: </li><ul><li>PHP
    38. 38. Zend Platform </li></ul><li>Newcomers: </li><ul><li>APM (Alternative PHP Monitor)
    39. 39. Pinba </li></ul></ul>
    40. 40. PHP: integrated functions <ul><li>error_reporting
    41. 41. display_errors
    42. 42. error_log </li><ul><li>file
    43. 43. syslog </li><ul><li>Windows => event log </li></ul></ul><li>set_error_handler() </li></ul>
    44. 44. Zend Platform <ul>All-in-One commercial solution of Zend. Includes: <ul><li>monitoring
    45. 45. performance
    46. 46. high availability
    47. 47. … </li></ul></ul>
    48. 48. APM <ul><li>A lternative P HP M onitor
    49. 49. Dedicated on error handling
    50. 50. Born in october 2008 </li></ul>
    51. 51. APM demo !
    52. 52. Why ? <ul><li>LAMP + APC + Xdebug
    53. 53. Missing software building block in the Free Software landscape.
    54. 54. Performance problems indirectly related to the Zend Platform. Need of a solution with a minimum impact. </li></ul>+ APM (LAMPAXA)
    55. 55. Architecture
    56. 56. Database event id INTEGER ts TEXT type INTEGER file TEXT line INTEGER message TEXT backtrace TEXT slow_request id INTEGER ts TEXT duration FLOAT file TEXT
    57. 57. PHP functions <ul><li>void apm_get_events ([int limit[, int offset[, int order[, bool asc[, bool json]]]]]) </li><ul><li>Prints events from the database in an HTML or JSON format. </li></ul><li>void apm_get_slow_requests ([int limit[, int offset[, int order[, bool asc[, bool json]]]]]) </li><ul><li>Prints slow requests from the database in an HTML or JSON format. </li></ul><li>int apm_get_events_count (void)
    58. 58. int apm_get_slow_requests_count (void) </li></ul>
    59. 59. Future <ul><li>PHP monitoring framework
    60. 60. Customizable strategies backends: </li><ul><li>SNMP (in progress)
    61. 61. e-mail
    62. 62. custom command </li></ul><li>Alternative storage solutions: </li><ul><li>MySQL/MariaDB
    63. 63. Hadoop
    64. 64. Volatile backends </li></ul></ul>
    65. 65. Contributing
    66. 66. APM contact info <ul><li>Websites: http://pecl.php.net/package/APM/ http://code.google.com/p/peclapm/
    67. 67. Twitter: @php_apm
    68. 68. Mailing lists:
    69. 69. [email_address] [email_address] </li></ul>
    70. 70. Pinba <ul><li>P HP i s n ot a b ottleneck a nymore
    71. 71. Dedicated to realtime performance monitoring
    72. 72. Born in 2009 </li></ul>
    73. 73. Why ? <ul><li>LAMP + APC + Xdebug + APM
    74. 74. LAMPAXAP
    75. 75. Missing software building block in the PHP landscape.
    76. 76. Gather performance metrics in production environments. </li></ul>+ Pinba
    77. 77. Architecture
    78. 78. Pinba demo!
    79. 79. Data sent <ul><li>string hostname – gethostname() result
    80. 80. int request_count – number of requests served by this process
    81. 81. string server_name – $_SERVER[&quot;SERVER_NAME&quot;]
    82. 82. string script_name – $_SERVER[&quot;SCRIPT_NAME&quot;]
    83. 83. int document_size – size of the response body
    84. 84. int memory_peak – memory allocation peak
    85. 85. float request_time – time spent of processing the request
    86. 86. float ru_utime – resource usage (user)
    87. 87. float ru_stime – resource usage (system)
    88. 88. array timers – array of timers (optional) </li></ul>
    89. 89. Benchmarking ? <?php $timer  =  microtime (  true  ); // some code being benchmarked echo  microtime (  true  ) -  $timer ,  &quot; &quot; ;
    90. 90. PHP functions <ul><li>resource pinba_timer_start (array tags[, array data]) </li><ul><li>Creates and starts new timer. </li></ul><li>bool pinba_timer_stop (resource timer) </li><ul><li>Stops the timer. </li></ul><li>bool pinba_timers_stop (void) </li><ul><li>Stops all running timers. </li></ul><li>bool pinba_script_name_set (string script_name) </li><ul><li>Set custom script name instead of $_SERVER['SCRIPT_NAME'] used by default. </li></ul></ul>
    91. 91. PHP functions <ul><li>array pinba_timer_get_info (resource timer) </li><ul><li>Returns timer data.
    92. 92. Exemple:
    93. 93. array(4) { [&quot;value&quot;]=>float(0.0213) [&quot;tags&quot;]=>array(1) { [&quot;foo&quot;]=>string(3) &quot;bar&quot; } [&quot;started&quot;]=>bool(true) [&quot;data&quot;]=>NULL } </li></ul></ul>
    94. 94. PHP functions <ul><li>array pinba_get_info (void) </li><ul><li>Returns all request data (including timers user data).
    95. 95. Example: array(9) { [&quot;mem_peak_usage&quot;]=> int(786432) [&quot;req_time&quot;]=> float(0.001529) [&quot;ru_utime&quot;]=> float(0) [&quot;ru_stime&quot;]=> float(0) [&quot;req_count&quot;]=> int(1) [&quot;doc_size&quot;]=> int(0) [&quot;server_name&quot;]=> string(7) &quot;unknown&quot; [&quot;script_name&quot;]=> string(1) &quot;-&quot; [&quot;timers&quot;]=> array(1) { [0]=> array(4) { [...] } } } </li></ul></ul>
    96. 96. Tables <ul>The Pinba plugin provides two tables types: <ul><li>Raw data tables
    97. 97. Reports </li></ul></ul>
    98. 98. Raw data tables <ul><li>request: </li><ul><li>Stores all the request data.
    99. 99. The table is a cyclic buffer of a certain size. </li></ul><li>timer: </li><ul><li>Stores timer data. </li></ul><li>tag: </li><ul><li>Stores unique list of tags and their IDs. </li></ul><li>timertag: </li><ul><li>Stores tag values. </li></ul></ul>
    100. 100. Reports <ul><li>info;
    101. 101. report_by_script_name;
    102. 102. report_by_server_name;
    103. 103. report_by_hostname;
    104. 104. report_by_server_and_script;
    105. 105. report_by_hostname_and_script;
    106. 106. report_by_hostname_and_server;
    107. 107. report_by_hostname_server_and_script. </li></ul>
    108. 108. Reports fields <ul><li>req_count;
    109. 109. req_per_sec;
    110. 110. req_time_total;
    111. 111. req_time_percent;
    112. 112. req_time_per_sec;
    113. 113. ru_utime_total;
    114. 114. ru_utime_percent;
    115. 115. ru_utime_per_sec; </li></ul><ul><li>ru_stime_total;
    116. 116. ru_stime_percent;
    117. 117. ru_stime_per_sec;
    118. 118. traffic_total;
    119. 119. traffic_percent;
    120. 120. traffic_per_sec. </li></ul>
    121. 121. Tag(s) reports <ul>Two report types: <ul><li>« tag info »
    122. 122. « tag report » </li></ul>Both of them being available for: <ul><li>1 tag
    123. 123. 2 tags </li></ul></ul>
    124. 124. Tag info for 1 tag Timer data grouped by the value of the specified tag. CREATE TABLE `tag_info_foo` ( `tag_value` varchar ( 32 ) DEFAULT NULL , `req_count` int ( 11 ) DEFAULT NULL , `hit_count` int ( 11 ) DEFAULT NULL , `timer_value` float DEFAULT NULL ) ENGINE =PINBA DEFAULT CHARSET =latin1 COMMENT = 'tag_info:foo'
    125. 125. Tag info for 2 tags Timer data grouped by the value of two specified tags. CREATE TABLE `tag_info_foo_bar` ( `foo_value` varchar ( 32 ) DEFAULT NULL , `bar_value` varchar ( 32 ) DEFAULT NULL , `req_count` int ( 11 ) DEFAULT NULL , `hit_count` int ( 11 ) DEFAULT NULL , `timer_value` float DEFAULT NULL ) ENGINE =PINBA DEFAULT CHARSET =latin1 COMMENT = 'tag2_info:foo,bar'
    126. 126. Tag report for 1 tag Timer data grouped by script name and value of the specified tag. CREATE TABLE `tag_report_foo` ( `script_name` varchar ( 128 ) DEFAULT NULL , `tag_value` varchar ( 32 ) DEFAULT NULL , `req_count` int ( 11 ) DEFAULT NULL , `hit_count` int ( 11 ) DEFAULT NULL , `timer_value` float DEFAULT NULL ) ENGINE =PINBA DEFAULT CHARSET =latin1 COMMENT = 'tag_report:foo'
    127. 127. Tag report for 2 tags Timer data grouped by script name and value of two specified tags. CREATE TABLE `tag_report_foo_bar` ( `script_name` varchar ( 128 ) DEFAULT NULL , `tag1_value` varchar ( 32 ) DEFAULT NULL , `tag2_value` varchar ( 32 ) DEFAULT NULL , `req_count` int ( 11 ) DEFAULT NULL , `hit_count` int ( 11 ) DEFAULT NULL , `timer_value` float DEFAULT NULL ) ENGINE =PINBA DEFAULT CHARSET =latin1 COMMENT = 'tag2_report:foo,bar'
    128. 128. Performance <ul><li>Overhead +/- 2 µs/request on a personal computer.
    129. 129. Invariable from the availability of MySQL (Reminder: UDP protocol). </li></ul>
    130. 130. Pinba contact info <ul><li>Website: http://pinba.org/
    131. 131. Mailing list: [email_address] </li></ul>
    132. 132. Credits <ul><li>Andrei Nigmatulin - initial design and first implementation.
    133. 133. Alexey Rybak - design, ideas and lots of (useful) critics.
    134. 134. Special thanks to Sergey Frolovichev.
    135. 135. Current implementation is done by Antony Dovgal. </li></ul>
    136. 136. Questions ? Suggestions ?
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×