Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Measure your app internals with InfluxDB and Symfony2

3,781 views

Published on

Introduction to InfluxDB and PHP Symfony2 and how to monitor your application workflow

Published in: Engineering

Measure your app internals with InfluxDB and Symfony2

  1. 1. Measure your app internals with InfluxDB (time-series database) Symfony Day 2015 - Reggio Emilia
  2. 2. Trust me
  3. 3. Why we measure things?
  4. 4. Measure - Learn - Build - Measure
  5. 5. Measure for a change
  6. 6. Measure for discovery
  7. 7. Measure for compare
  8. 8. What we need? * A data provider (our application) * A database that will collects data * A dashboard for representing information
  9. 9. What is a time-series database? A database optimized for handling time series data: arrays of numbers indexed by time (a datetime or a datetime range) sign elem is 4 -> @10:13 elem is 4 A time series is a sequence of data points, measured typically at successive points in time spaced at uniform time intervals
  10. 10. Time series Databases InfluxDB github.com/influxdb
  11. 11. InfluxDB Easy to use Written in Golang (no external dependencies) ready for bilion of rows (scalable) UDP/IP and HTTP network adapters Several client libraries (PHP, Golang, etc...) Integrated query web panel Ready for external dashboards (grafana, chronograf, etc...)
  12. 12. Write information in your InfluxDB curl ­XPOST 'http://localhost:8086/write?db=mydb'      ­d 'cpu load=2 1434055562000000000'
  13. 13. Read data back from InfluxDB curl ­G http://localhost:8086/query?pretty=true ­­data­urlencode "db=mydb"     ­­data­urlencode "q=SELECT * FROM cpu WHERE host='server01' AND time < now() ­ 1d"
  14. 14. For example: InfluxDB tags You can mark new points with a tag information sign @10.32 key=value tag1=value tag2=value sign @10.32 temperature=21.3 location=home env=test sign @10.32 cpu=21.3,mem=123.3 location=eu­west­1 env=dev instance=i­a91afcd1
  15. 15. Use tags in your queries Example SELECT * FROM home_sensors WHERE   location="home" AND   temperature > 19.2 AND   time >= now() ­ 10m Example for websites: SELECT * FROM index_page WHERE   env="prod" AND   load_time > 19.2 AND   time >= now() ­ 1d Just use them like string column values
  16. 16. Play from the InfluxDB web console http://your-domain.tld:8083/
  17. 17. GRAFANA DASHBOARD An open source, feature rich, metrics dashboard and graph editor for several time series databases (InfluxDB, Graphite, etc.)
  18. 18. Annotate your graphs Thanks to annotations you can use a serie as an annotation list Mark a new software release in your graph (Continuous Delivery) Mark any critic event in all your graphs (Information Point)
  19. 19. CHRONOGRAF DASHBOARD A new data visualization tool for InfluxDB.
  20. 20. How to send my application data? * Send every single point directly in real-time * Collect data during the application life-cycle and send those points once per request
  21. 21. InfluxDB supports two network adapters HTTP or UDP/IP Which one we should have to use? UDP/IP must be enabled by configuration
  22. 22. HTTP * TCP/IP based * Data point delivery guaranteed * Error handling * ~ 374 Ops/second (for compare with UDP/IP)
  23. 23. UDP/IP * No delivery guarantees * No error handling * ~ 22917 Ops/second (for compare with TCP/IP)
  24. 24. HTTP * Sensitive data * Good for once at a time strategy (or if we do not care about monitor impact on our application) UDP/IP * Good for immediate delivery
  25. 25. InfluxDB-PHP-SDK https://github.com/corley/influxdb-php-sdk Supports both: HTTP and UDP/IP Star 46 Fork 13 composer require corley/influxdb-sdk
  26. 26. How it works $client­>mark("serie_name", [     "customer_name" => "walter",     "item_price" => 100.12,     //... ]); Concise Format (good for immediate delivery)
  27. 27. How it works (2) $client­>mark([     "points" => [         [             "measurement" => "instance",             "fields" => [                 "cpu" => 18.12,                 "free" => 712423,             ],         ],         [             "measurement" => "page_load",             "fields" => [                 "listeners_speed" => 1.12,                 "counted_elements" => 341143,             ],         ],     ] ]); Extended Format (good for a bulk strategy)
  28. 28. InfluxDB Client UDP/IP + HTTP $httpOptions = new Options(); $reader = new HttpReader($httpOptions); $udpOptions = new Options(); $writer = new UdpWriter($udpOptions); $client = new Client($reader, $writer); HTTP only $http = new GuzzleHttpClient(); $options = new Options(); $reader = new HttpReader($http, $options); $writer = new HttpWriter($http, $options); $client = new Client($reader, $writer); Supports Guzzle versions: ~4, ~5, ~6
  29. 29. Symfony: Dependency Injection Container parameters:   influxdb_host: "localhost"   influxdb_port: 4444 services:   influxdb:     class: InfluxDBClient     arguments: ["@influxdb_reader", "@influxdb_writer"]   influxdb_writer:     class: InfluxDBAdapterWriterUdp     arguments: ["@influxdb_udp_options"]   influxdb_reader:     class: InfluxDBAdapterHttpReader     arguments: ["@influxdb_http_options"]
  30. 30. Symfony: Dependency Injection Container #Only options   influxdb_udp_options:     class: InfluxDBOptions     calls:       ­ ["setHost", ["%influxdb_host%"]]               # InfluxDB host       ­ ["setPort", ["%influxdb_udp_port%"]]           # UDP/IP Port       ­ ["setTags", [{"env": "%kernel.environment%"}]] # tag every point with env   influxdb_http_options:     class: InfluxDBOptions     calls:       ­ ["setHost", ["%influxdb_host%"]]               # InfluxDB host       ­ ["setPort", ["%influxdb_http_port%"]]          # HTTP Port parameters:   influxdb_host: "influxdb.domain.tld"   influxdb_udp_port: 4444   influxdb_http_port: 8086
  31. 31. Don't try this at home class DefaultController extends Controller {     public function indexAction(Request $request)     {         $client = $this­>container­>get("influxdb");         $client­>mark(...);     } } Goals: single-responsibility, reduce coupling, simplifing testing
  32. 32. Controllers as a service? For a monitor? Make sense?
  33. 33. With events class DefaultController extends Controller {     public function indexAction(Request $request)     {         //...         $dispatcher­>dispatch("my_event_name", $event);     } } Event listener class MyMonitorAwareForMyEventListener {     public function __construct(Client $client) { ... }     public function onMyEvent(Event $event)     {         $this­>client­>mark(...);     } } Deregister this listener in order to stop data collection
  34. 34. Compose your listener monitor_listener:   class: AppBundleListenerMonitorListener   arguments: ["@influxdb", "@debug.stopwatch"]   tags:     ­ { name: kernel.event_listener, event: kernel.request, method: startStopwatch }     ­ { name: kernel.event_listener, event: kernel.response, method: stopStopwatch } The stopwatch will help us to collect page timing information
  35. 35. A listener example with stopwatch class MonitorListener {     public function startStopwatch(GetResponseEvent $event)     {         $routeName = $event­>getRequest()­>attributes­>get("_route");         $this­>stopwatch­>start($routeName);     }     public function stopStopwatch(FilterResponseEvent $event)     {         $routeName = $event­>getRequest()­>attributes­>get("_route");         $events = $this­>stopwatch­>stop($routeName);         foreach ($events­>getPeriods() as $measure) {             $this­>client­>mark($routeName, [                 "memory" => $measure­>getMemory(), //...             ]);         }     } }
  36. 36. Check your data via InfluxDB
  37. 37. Prepare your dashboard with grafana
  38. 38. Checkout out the app example https://github.com/wdalmut/symfonyday2015-example
  39. 39. Thanks github.com/wdalmut

×