Monitoring with
Stackdriver
Or: How I Learned How to Stop Worrying and Dig Hosted
Monitoring.

denise stockman - denise@st...
Mad Props
Thanks to Coupa for hosting
and to Stackdriver for sponsoring
and to Advanced AWS for organizing.
Hi, I’m denise
●
●
●
●
●

System Architect @ Maverix Biomics
18 years button pushing
Musical Chair Champion
Data center to...
What is Stackdriver?
“Intelligent Monitoring for your AWS
Infrastructure, Systems and App”
Agenda
●
●
●
●
●
●

The Environment
Selection Process
Implementation
Day to Day
TODO
Questions
A Diverse Stack
Further Context
●
●
●
●
●

Green field!
We treat instances like cattle*
Modern and crusty backend services
“Quaint” team
M...
What to do for monitoring?
Evaluate & Select
● In-house or hosted?
● Modern vs proven?
● Specialized vs general?
● Paid vs...
Let’s Be Honest
●
●
●
●

Not enough time
Small environment
More important work to do
Everyone else is doing it
Winner, Winner, Chicken Dinner
●
●
●
●
●
●
●

System & service metrics
Alerting
Custom metrics & annotations
collectd agen...
Implementation
Consistent Resource Naming
Resource name format
env-function-id.location.domain.com
Tags for resources
env, owner, purpose...
●
●
●
●

Installs base agent on all hosts
Configures API key
Includes plugins for supported services
Auto-upgrades agent
n...
●
●
●
●

Create groups based on filtering criteria
Dashboards auto-created for each group
Groups used in custom dashboardi...
Alerting
Day to Day Use
Custom Dashboards
Anomaly Detection
Event Correlation
Event Logging
curl -X POST
-H "x-stackdriver-apikey: DOOOOWEEEEOOOO"
-H "Content-Type: application/json"
-d '{"message":"Mind taking!", ...
Ruby example
customer_id = 31337
api_key = "DOOOOWEEEEOOOO"
StackDriver.init customer_id, api_key
redis = Redis.new
mindsT...
TODO.txt
●
●
●
●
●

Expand app metrics with statsd
Automate group creation & management
Annotate orchestrated events
Cloud...
Thanks!
Questions?
denise stockman
denise@stockmans.org
github.com/maverixbio/puppet-stackdriver-agent
forge.puppetlabs.co...
Upcoming SlideShare
Loading in...5
×

Monitoring with Stackdriver

1,339

Published on

A presentation on how we came to find Stackdriver for system and service monitoring, how we implemented it and how we use it on a day to day basis.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,339
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
30
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Monitoring with Stackdriver"

  1. 1. Monitoring with Stackdriver Or: How I Learned How to Stop Worrying and Dig Hosted Monitoring. denise stockman - denise@stockmans.org
  2. 2. Mad Props Thanks to Coupa for hosting and to Stackdriver for sponsoring and to Advanced AWS for organizing.
  3. 3. Hi, I’m denise ● ● ● ● ● System Architect @ Maverix Biomics 18 years button pushing Musical Chair Champion Data center to cloud Early Stackdriver user & puppet module maintainer
  4. 4. What is Stackdriver? “Intelligent Monitoring for your AWS Infrastructure, Systems and App”
  5. 5. Agenda ● ● ● ● ● ● The Environment Selection Process Implementation Day to Day TODO Questions
  6. 6. A Diverse Stack
  7. 7. Further Context ● ● ● ● ● Green field! We treat instances like cattle* Modern and crusty backend services “Quaint” team Monitor & report system, service, event and application statistics in one place. * As much as possible
  8. 8. What to do for monitoring? Evaluate & Select ● In-house or hosted? ● Modern vs proven? ● Specialized vs general? ● Paid vs free?
  9. 9. Let’s Be Honest ● ● ● ● Not enough time Small environment More important work to do Everyone else is doing it
  10. 10. Winner, Winner, Chicken Dinner ● ● ● ● ● ● ● System & service metrics Alerting Custom metrics & annotations collectd agent Anomaly detection AWS integration & cloud friendly Cost per resource
  11. 11. Implementation
  12. 12. Consistent Resource Naming Resource name format env-function-id.location.domain.com Tags for resources env, owner, purpose, version
  13. 13. ● ● ● ● Installs base agent on all hosts Configures API key Includes plugins for supported services Auto-upgrades agent node /^dev-redis.*.stockmans.org$/ { class { "stackdriver::redis": stackdriver_redis_port => 6379, stackdriver_redis_host => localhost, stackdriver_redis_timeout => 2000, } } Agent Deployment
  14. 14. ● ● ● ● Create groups based on filtering criteria Dashboards auto-created for each group Groups used in custom dashboarding and alerting policies Set it once and fuggetaboutit* Groups
  15. 15. Alerting
  16. 16. Day to Day Use
  17. 17. Custom Dashboards
  18. 18. Anomaly Detection
  19. 19. Event Correlation
  20. 20. Event Logging
  21. 21. curl -X POST -H "x-stackdriver-apikey: DOOOOWEEEEOOOO" -H "Content-Type: application/json" -d '{"message":"Mind taking!", "annotated_by":"Mentok"}' https://event-gateway.stackdriver.com/v1/annotationevent Annotations
  22. 22. Ruby example customer_id = 31337 api_key = "DOOOOWEEEEOOOO" StackDriver.init customer_id, api_key redis = Redis.new mindsTaken = redis.zcard("minds:taken") StackDriver.send_metric "mentok.minds.taken", mindsTaken, Time.now.to_i Custom Metrics
  23. 23. TODO.txt ● ● ● ● ● Expand app metrics with statsd Automate group creation & management Annotate orchestrated events Cloudwatch custom metrics Stackdriver @ our colo
  24. 24. Thanks! Questions? denise stockman denise@stockmans.org github.com/maverixbio/puppet-stackdriver-agent forge.puppetlabs.com/dstockman/stackdriver
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×