• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
PHP applications/environments monitoring: APM & Pinba
 

PHP applications/environments monitoring: APM & Pinba

on

  • 10,131 views

 

Statistics

Views

Total Views
10,131
Views on SlideShare
10,083
Embed Views
48

Actions

Likes
3
Downloads
23
Comments
0

3 Embeds 48

http://martindekeijzer.nl 23
http://www.martindekeijzer.nl 15
http://www.slideshare.net 10

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Surveiller d'eventuellle erreur comportementale de l'application. Surveiller au niveau des performances, afin de prevoir d'eventuel probleme
  • Page de demo avec E_NOTICE et autre, Display avec tail
  • Tail -f /var/log/php/error.log Page vide de apm.php Puis generation d'une page d'erreur Affichage de apm.php Recommecer avec mutilple erreur
  • Rappel, qu'est ce que APC, Xdebug, … Compatbilité totale avec le monde du LL
  • Compatbilité totale avec le monde du LL
  • L'extension PHP Pinba envoie un paquet UDP lors de la terminaison d'une requête qui contient des informations comme: hostname - gethostname() result request_count - number of requests served by this process server_name script_name document_size - size of the response body memory_peak - memory allocation peak request_time - time spent of processing the request ru_utime - resource usage (user) ru_stime - resource usage (system) timers - array of timers (optional) De la nature UDP, aucune connexion n'est réalisée, ce qui permet d'avoir aucun impacte de performance, mais des pertes de paquets pourraient se produire. De l'autre coté, le plugin MySQL Pinba écoute sur le port configuré pour récupérer les paquets et les décoder afin de les stocker dans ses pools d'enregistrements ainsi que de mettre à jour les index, les rapports de bases et rapports de tags si ils existent.
  • Raw data tables contain raw request data (surprise!). Please bear in mind that access to the raw data is relatively slow (the number of records might reach millions and there are NO indexes except for primary keys, so almost any operation requires full table scan). Reports were created in order to speed up the most frequent operations by aggregating data on-the-fly, when new request data arrives.

PHP applications/environments monitoring: APM & Pinba PHP applications/environments monitoring: APM & Pinba Presentation Transcript

  • PHP applications/environments monitoring: APM & Pinba Davide Mendolia - Patrick Allaert PHPBenelux Conference 2010
  • Davide Mendolia
    • http://davidemendolia.blogspot.com/
    • dmendolia@php.net or @davideme (Twitter)
    • Lead developer of APM
    • php-benchmark contributor
    • http://www.php-ci.net/
    • Works at Tuenti
    • Social network , based in Madrid
    • More than 20 billion page views per month
    • First Spanish website ahead Google, Microsoft and Facebook (Comscore)
    • International work environment (15 nationalities)
    • Developers wanted!
  • Patrick Allaert
    • http://patrickallaert.blogspot.com/
    • patrickallaert@php.net or @patrick_allaert (Twitter)
    • Lead developer of APM
    • Works at eZ Systems (eZ Publish, eZ Components)
    • PHPBenelux staff member ( http://www.phpbenelux.eu/ )
  • Patrick Allaert
    • Start programming with PHP 3
    • Ten years of Free Software evangelism
    • Contributions:
      • PHP (ext/ldap, QA: PHPT)
      • KDESvn (Subversion client for KDE)
      • XOo°f (eXtensible Open Object Oriented Framework)
      • Tabellio (collaborative suite for parliamentary assembly)
  • Monitoring: Definition ?
    • Being aware of the state of a system by watching and measuring it.
    • Indicators:
      • the physical condition of a machine
      • load of a machine
      • application availability
      • the messages in system logs
      • network performance
      • known attacks
  • Monitoring: Why?
    • Need in terms of increasing availability
    • Economic reasons
    • User experience
  • Monitoring: Why? « When I get on the homepage I have a blank page […] » « Seriously? One moment, checking […] » « We are aware of the problem, our engineers have found the solution. Everything will be back to normal in 5 minutes » « Hell! You're right!? »
  • Monitoring: Presentation
      Monitoring the P of LAMP. In particular:
      • Error handling in PHP
      • Execution performance.
  • What are the solutions ?
    • Current:
      • PHP
      • Zend Platform
    • Newcomers:
      • APM (Alternative PHP Monitor)
      • Pinba
  • PHP: integrated functions
    • error_reporting
    • display_errors
    • error_log
      • file
      • syslog
        • Windows => event log
    • set_error_handler()
  • Zend Platform
      All-in-One commercial solution of Zend. Includes:
      • monitoring
      • performance
      • high availability
  • APM
    • A lternative P HP M onitor
    • Dedicated on error handling
    • Born in october 2008
  • APM demo !
  • Why ?
    • LAMP + APC + Xdebug
    • Missing software building block in the Free Software landscape.
    • Performance problems indirectly related to the Zend Platform. Need of a solution with a minimum impact.
    + APM (LAMPAXA)
  • Architecture
  • Database event id INTEGER ts TEXT type INTEGER file TEXT line INTEGER message TEXT backtrace TEXT slow_request id INTEGER ts TEXT duration FLOAT file TEXT
  • PHP functions
    • void apm_get_events ([int limit[, int offset[, int order[, bool asc[, bool json]]]]])
      • Prints events from the database in an HTML or JSON format.
    • void apm_get_slow_requests ([int limit[, int offset[, int order[, bool asc[, bool json]]]]])
      • Prints slow requests from the database in an HTML or JSON format.
    • int apm_get_events_count (void)
    • int apm_get_slow_requests_count (void)
  • Future
    • PHP monitoring framework
    • Customizable strategies backends:
      • SNMP (in progress)
      • e-mail
      • custom command
    • Alternative storage solutions:
      • MySQL/MariaDB
      • Hadoop
      • Volatile backends
  • Contributing
  • APM contact info
    • Websites: http://pecl.php.net/package/APM/ http://code.google.com/p/peclapm/
    • Twitter: @php_apm
    • Mailing lists:
    • [email_address] [email_address]
  • Pinba
    • P HP i s n ot a b ottleneck a nymore
    • Dedicated to realtime performance monitoring
    • Born in 2009
  • Why ?
    • LAMP + APC + Xdebug + APM
    • LAMPAXAP
    • Missing software building block in the PHP landscape.
    • Gather performance metrics in production environments.
    + Pinba
  • Architecture
  • Pinba demo!
  • Data sent
    • string hostname – gethostname() result
    • int request_count – number of requests served by this process
    • string server_name – $_SERVER["SERVER_NAME"]
    • string script_name – $_SERVER["SCRIPT_NAME"]
    • int document_size – size of the response body
    • int memory_peak – memory allocation peak
    • float request_time – time spent of processing the request
    • float ru_utime – resource usage (user)
    • float ru_stime – resource usage (system)
    • array timers – array of timers (optional)
  • Benchmarking ? <?php $timer  =  microtime (  true  ); // some code being benchmarked echo  microtime (  true  ) -  $timer ,  &quot; &quot; ;
  • PHP functions
    • resource pinba_timer_start (array tags[, array data])
      • Creates and starts new timer.
    • bool pinba_timer_stop (resource timer)
      • Stops the timer.
    • bool pinba_timers_stop (void)
      • Stops all running timers.
    • bool pinba_script_name_set (string script_name)
      • Set custom script name instead of $_SERVER['SCRIPT_NAME'] used by default.
  • PHP functions
    • array pinba_timer_get_info (resource timer)
      • Returns timer data.
      • Exemple:
      • array(4) { [&quot;value&quot;]=>float(0.0213) [&quot;tags&quot;]=>array(1) { [&quot;foo&quot;]=>string(3) &quot;bar&quot; } [&quot;started&quot;]=>bool(true) [&quot;data&quot;]=>NULL }
  • PHP functions
    • array pinba_get_info (void)
      • Returns all request data (including timers user data).
      • Example: array(9) { [&quot;mem_peak_usage&quot;]=> int(786432) [&quot;req_time&quot;]=> float(0.001529) [&quot;ru_utime&quot;]=> float(0) [&quot;ru_stime&quot;]=> float(0) [&quot;req_count&quot;]=> int(1) [&quot;doc_size&quot;]=> int(0) [&quot;server_name&quot;]=> string(7) &quot;unknown&quot; [&quot;script_name&quot;]=> string(1) &quot;-&quot; [&quot;timers&quot;]=> array(1) { [0]=> array(4) { [...] } } }
  • Tables
      The Pinba plugin provides two tables types:
      • Raw data tables
      • Reports
  • Raw data tables
    • request:
      • Stores all the request data.
      • The table is a cyclic buffer of a certain size.
    • timer:
      • Stores timer data.
    • tag:
      • Stores unique list of tags and their IDs.
    • timertag:
      • Stores tag values.
  • Reports
    • info;
    • report_by_script_name;
    • report_by_server_name;
    • report_by_hostname;
    • report_by_server_and_script;
    • report_by_hostname_and_script;
    • report_by_hostname_and_server;
    • report_by_hostname_server_and_script.
  • Reports fields
    • req_count;
    • req_per_sec;
    • req_time_total;
    • req_time_percent;
    • req_time_per_sec;
    • ru_utime_total;
    • ru_utime_percent;
    • ru_utime_per_sec;
    • ru_stime_total;
    • ru_stime_percent;
    • ru_stime_per_sec;
    • traffic_total;
    • traffic_percent;
    • traffic_per_sec.
  • Tag(s) reports
      Two report types:
      • « tag info »
      • « tag report »
      Both of them being available for:
      • 1 tag
      • 2 tags
  • Tag info for 1 tag Timer data grouped by the value of the specified tag. CREATE TABLE `tag_info_foo` ( `tag_value` varchar ( 32 ) DEFAULT NULL , `req_count` int ( 11 ) DEFAULT NULL , `hit_count` int ( 11 ) DEFAULT NULL , `timer_value` float DEFAULT NULL ) ENGINE =PINBA DEFAULT CHARSET =latin1 COMMENT = 'tag_info:foo'
  • Tag info for 2 tags Timer data grouped by the value of two specified tags. CREATE TABLE `tag_info_foo_bar` ( `foo_value` varchar ( 32 ) DEFAULT NULL , `bar_value` varchar ( 32 ) DEFAULT NULL , `req_count` int ( 11 ) DEFAULT NULL , `hit_count` int ( 11 ) DEFAULT NULL , `timer_value` float DEFAULT NULL ) ENGINE =PINBA DEFAULT CHARSET =latin1 COMMENT = 'tag2_info:foo,bar'
  • Tag report for 1 tag Timer data grouped by script name and value of the specified tag. CREATE TABLE `tag_report_foo` ( `script_name` varchar ( 128 ) DEFAULT NULL , `tag_value` varchar ( 32 ) DEFAULT NULL , `req_count` int ( 11 ) DEFAULT NULL , `hit_count` int ( 11 ) DEFAULT NULL , `timer_value` float DEFAULT NULL ) ENGINE =PINBA DEFAULT CHARSET =latin1 COMMENT = 'tag_report:foo'
  • Tag report for 2 tags Timer data grouped by script name and value of two specified tags. CREATE TABLE `tag_report_foo_bar` ( `script_name` varchar ( 128 ) DEFAULT NULL , `tag1_value` varchar ( 32 ) DEFAULT NULL , `tag2_value` varchar ( 32 ) DEFAULT NULL , `req_count` int ( 11 ) DEFAULT NULL , `hit_count` int ( 11 ) DEFAULT NULL , `timer_value` float DEFAULT NULL ) ENGINE =PINBA DEFAULT CHARSET =latin1 COMMENT = 'tag2_report:foo,bar'
  • Performance
    • Overhead +/- 2 µs/request on a personal computer.
    • Invariable from the availability of MySQL (Reminder: UDP protocol).
  • Pinba contact info
    • Website: http://pinba.org/
    • Mailing list: [email_address]
  • Credits
    • Andrei Nigmatulin - initial design and first implementation.
    • Alexey Rybak - design, ideas and lots of (useful) critics.
    • Special thanks to Sergey Frolovichev.
    • Current implementation is done by Antony Dovgal.
  • Questions ? Suggestions ?