Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Docker {at,with} SignalFx

3,013 views

Published on

Slides of Maxime Petazzoni's talk at the Palo Alto Docker Meetup on September 1st, 2015. Discusses how we use Docker to power our software development lifecycle and run our production environments, as well as how to monitor Dockerized deployments and applications, in particular with SignalFx.

Published in: Software
  • If you want to download or read this book, copy link or url below in the New tab ......................................................................................................................... DOWNLOAD FULL PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m6jJ5M } .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • If you want to download or read this book, copy link or url below in the New tab ......................................................................................................................... DOWNLOAD FULL PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m6jJ5M } .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Docker {at,with} SignalFx

  1. 1. MAXIME PETAZZONI — PALO ALTO DOCKER MEETUP — SEPT. 1ST, 2015 DOCKER {AT,WITH} SIGNALFX
  2. 2. MAXIME PETAZZONI — PALO ALTO DOCKER MEETUP — SEPT. 1ST, 2015 Me? github.com/mpetazzoni @mpetazzoni
  3. 3. MAXIME PETAZZONI — PALO ALTO DOCKER MEETUP — SEPT. 1ST, 2015 What is SignalFx?
  4. 4. The pitch • Real-time monitoring system for modern applications • User-defined, streaming and interactive analytics • Alerting and anomaly detection • Easy to deploy, to use and to integrate • Scales; from small deployments to large, complex infrastructures
  5. 5. MAXIME PETAZZONI — PALO ALTO DOCKER MEETUP — SEPT. 1ST, 2015 Docker at SignalFx
  6. 6. Docker at SignalFx • Started using Docker in November 2013 • Running Docker in prod ever since we’ve had a “prod” • Back when Docker’s README said “DO NOT RUN IN PRODUCTION”
  7. 7. Objectives • Separate infrastructure concerns • Stronger independence from hosting provider • Development lifecycle • Application packaging and delivery • Higher-level orchestration
  8. 8. Infrastructure vs application • Separation of concerns • Infrastructure provides appropriately spec-ed hosts and network • Abstracts from cloud provider • Currently on AWS • Amazon Linux AMIs + Salt to setup a few things, up to Docker
  9. 9. Infrastructure vs application (cont.) • “Application” stack takes it up from there • Everything is containerized, including 3rd party components • No dependence on Amazon specifics • We can spin up the stack anywhere (laptop, GCE, DO, …)
  10. 10. SDLC • Run locally, in tests and in prod environments • Same bits, same tools, same arguments, etc. • Crucial for software validation • Maven, Jenkins, Quay.io, MaestroNG • maestro start
  11. 11. SDLC (cont.) docker-maven-plugin
  12. 12. SDLC (cont.) • Continuous build of the release branch • Builds and pushes :latest tagged Docker images • Promotion track through :lab, :prod and :mon • One-click promote + upgrade, orchestrated by MaestroNG • Automated integration tests environment
  13. 13. Application packaging and delivery • All components of the stack are packaged as Docker images • 3rd-party component packaging was figured out once (Dockerfile) • SignalFx components all have their Dockerfile too • Built by docker-maven-plugin • All components have a run.py init script
  14. 14. Discovery • ZooKeeper / Curator service discovery • We built announcing wrappers for Cassandra and ElasticSearch • Only need to know “realm” name and zkConnectString • Everything else is discovered from ZooKeeper • Need to make sure to announce host IP address/port • Maestro gives us that via environment
  15. 15. Orchestration • We built MaestroNG • Orchestrator of multi-host Docker environments • github.com/signalfuse/maestro-ng • Jinja2/Yaml environment file • Defines what runs where, and how • Knows and respects service dependencies
  16. 16. MAXIME PETAZZONI — PALO ALTO DOCKER MEETUP — SEPT. 1ST, 2015 Monitoring containers
  17. 17. The basics • Docker’s /stats API provides the bare minimum • CPU • Memory • Network I/O • Disk I/O • Pitfalls; GET /stats is blocking, up to a second • Use CollectD and docker-collectd-plugin • github.com/signalfx/docker-collectd-plugin
  18. 18. The basics (cont.) • Data returned by /stats API needs “math” to yield usable numbers • See Docker client code • Better: use SignalFx! • Docker Dashboard • Collect host metrics too if you can (with CollectD) • System/container-level metrics are very limited • We need to go deeper
  19. 19. Application monitoring • Usually needs more specific visibility • Instrument first, ask questions later • Metrics are cheap, both in CPU and $ • For 3rd-party apps, use CollectD and its plugins • We’re doing a lot of work to improve/provide CollectD plugins for common FOSS • And provide curated dashboards for each of them
  20. 20. Application monitoring (cont.) • For 1st-party software, depends on language platform • For VM-based services (Java, Scala, Go, …), get key metrics • Heap • # threads • Time spent in GC and # collections • Use metric libraries and instrument everything • Bring it all together in dashboards and anomaly detectors
  21. 21. Docker with SignalFx • Objective: time-to-data as short as possible • Easiest route: use our packaged CollectD • Good default set of host metrics • Install/enable plugins as needed; docker-collectd-plugin • SignalFx automatically imports curated dashboards • That’s it!
  22. 22. More monitoring with SignalFx • Use our client libraries or API to send your monitoring data • Or tee off your existing Graphite, StatsD, … pipeline towards SignalFx • Leverage the power of the real-time, streaming analytics • Correlate system, container and application metrics • Build better dashboards and alerts from higher-level signals
  23. 23. Thanks! github.com/signalfuse/maestro-ng github.com/signalfx/docker-collectd-plugin signalfx.com/free-trial.html @signalfx — signalfx.com/careers.html @mpetazzoni — github.com/mpetazzoni

×