SlideShare a Scribd company logo
1 of 41
Download to read offline
Which watcher watches
cloudwatch?
David Lutz
@dlutzy
about me
sysadmin
about me
sysadmin
devops
about me
sysadmin
devops
operations
about me
sysadmin
devops
web operations
about me
sysadmin
devops
web operations
about me
#monitoringlove
#infracoders
I “drive” website
infrastructure
What do I mean by “drive”?
Operate, anything not build related.
● Gather telemetry or metrics on running system
● Use metrics to make decisions about capacity and
architecture
● Know what the bottlenecks are and when we’ll hit
them
● Advise the business about cost vs performance
tradeoffs
● Fix things when the break
Work with my friends the
developers…
They build the things I drive.
What the developers think they built
What they actually built
How operations see themselves
How dev sees ops
The Perfect Dashboard
How fast are we
going?
(km/h)
How hard is the
engine working
(RPM)
Is anything
broken?
(alerts)
How fast can we
go before redline
How far can we go
before running
out of fuel
(Thresholds)
How fast are we going?

latency (ms)
How hard are the servers working?
throughput (rpm)
Is
anything
broken?
What about thresholds?
You must load test to find the
breaking points and bottlenecks
yourself.
Load testing is hard.
AWS CloudWatch pros and cons.
Comparison of three SAAS products
that consume CloudWatch metrics.
NewRelic
Stackdriver
Librato
CloudWatch limitations
CloudWatch limitations...
●
●
●
●

two week data retention
can’t create custom dashboards
UI is better than it used to be…
can’t time shift metrics to compare last
week to this week
However in CloudWatch’s defense
●
●
●
●

good for ad hoc exploration of data
most up to date
can set thresholds and alert on them
can push custom metrics (does anyone use
this?)
Setting up access
Create IAM group and
user
Give Access Key and
Secret Key to Librato
Setting up access
Create IAM group and user
Launch t1.micro AMI from
marketplace (or run it on your
own server)
Configure and run Sync process
(is a ruby gem)
Setting up access
Create role for
“cross account
access” for “3rd
party”, “read only”
Give stackdriver the
Role ARN.
Data retention
NewRelic

Stackdriver

Librato

3 months

13 months
(Elite tier)

12 months
(rolled up to
1 hour
resolution)
Features
NewRelic

Stackdriver

Librato

Custom dashboards

✓

✓

✓

Real time update

✓

✓

✓

✓

Metric lag time display
Time shift overlay

✓

Server agent

✓

Mobile client

✓

Custom metrics

✓

✓

✓

Events (like deployments)

✓

✓

✓

✓
Key benefits IMHO
NewRelic

Stackdriver

integration
with existing
system

easily define and integration
clusters and
with existing
aggregate
system

open up access
to those who
wouldn’t
otherwise have
access

imports
CloudTrail data,
who did what
when?
anomaly detection

Librato

flexibility of
data
collection and
retention
Costs (per month)
NewRelic

Stackdriver

Librato

Free if you
already use
NewRelic

$12 per resource
(Elite tier)

AWS Billing $0.60
DynamoDB $1.10
EBS $0.50
EC2 $0.50
ELB $0.65
ElastiCache $1.95
EMR $1.30
OpsWorks $0.75
RDS $0.70
Redshift $0.75
Route53 $0.05
SNS $0.20
SQS $0.40
...

But you need a
server to run
the collector

(per instance)
Say g’day to me
@dlutzy

More Related Content

What's hot

Public and private cloud metadata and why it is useful
Public and private cloud metadata and why it is usefulPublic and private cloud metadata and why it is useful
Public and private cloud metadata and why it is useful
DevSecCon
 
Dynatrace: DevOps, shift-left & self-healing a performance clinic with andi
Dynatrace: DevOps, shift-left & self-healing a performance clinic with andiDynatrace: DevOps, shift-left & self-healing a performance clinic with andi
Dynatrace: DevOps, shift-left & self-healing a performance clinic with andi
Dynatrace
 

What's hot (19)

Migrating .NET and .NET Core to Pivotal Cloud Foundry (1/2)
Migrating .NET and .NET Core to Pivotal Cloud Foundry (1/2)Migrating .NET and .NET Core to Pivotal Cloud Foundry (1/2)
Migrating .NET and .NET Core to Pivotal Cloud Foundry (1/2)
 
Mastering Azure Monitor
Mastering Azure MonitorMastering Azure Monitor
Mastering Azure Monitor
 
Journey to cloud openstack nexus ipma 2013
Journey to cloud openstack nexus ipma 2013Journey to cloud openstack nexus ipma 2013
Journey to cloud openstack nexus ipma 2013
 
MongoDB World 2018: Using Puppet, Ansible and Ops Manager to Create Your Own ...
MongoDB World 2018: Using Puppet, Ansible and Ops Manager to Create Your Own ...MongoDB World 2018: Using Puppet, Ansible and Ops Manager to Create Your Own ...
MongoDB World 2018: Using Puppet, Ansible and Ops Manager to Create Your Own ...
 
The Essentials of AWS Cost Control
The Essentials of AWS Cost ControlThe Essentials of AWS Cost Control
The Essentials of AWS Cost Control
 
Intelligent Cloud Conference 2018 - Building secure cloud applications with A...
Intelligent Cloud Conference 2018 - Building secure cloud applications with A...Intelligent Cloud Conference 2018 - Building secure cloud applications with A...
Intelligent Cloud Conference 2018 - Building secure cloud applications with A...
 
Customer Story: Property Partner
Customer Story: Property PartnerCustomer Story: Property Partner
Customer Story: Property Partner
 
Serverless Logging Architecture
Serverless Logging ArchitectureServerless Logging Architecture
Serverless Logging Architecture
 
Public and private cloud metadata and why it is useful
Public and private cloud metadata and why it is usefulPublic and private cloud metadata and why it is useful
Public and private cloud metadata and why it is useful
 
Azure Saturday 2017 - Hold my beer...
Azure Saturday 2017 - Hold my beer...Azure Saturday 2017 - Hold my beer...
Azure Saturday 2017 - Hold my beer...
 
Dynatrace: DevOps, shift-left & self-healing a performance clinic with andi
Dynatrace: DevOps, shift-left & self-healing a performance clinic with andiDynatrace: DevOps, shift-left & self-healing a performance clinic with andi
Dynatrace: DevOps, shift-left & self-healing a performance clinic with andi
 
Intelligent Cloud Conference 2018 - Next Generation of Data Integration with ...
Intelligent Cloud Conference 2018 - Next Generation of Data Integration with ...Intelligent Cloud Conference 2018 - Next Generation of Data Integration with ...
Intelligent Cloud Conference 2018 - Next Generation of Data Integration with ...
 
Level up your SQL and Azure, by using Rubrik
Level up your SQL and Azure, by using RubrikLevel up your SQL and Azure, by using Rubrik
Level up your SQL and Azure, by using Rubrik
 
Supersizing Magento
Supersizing MagentoSupersizing Magento
Supersizing Magento
 
Friends Don't Let Friends Build Landing Zones
Friends Don't Let Friends Build Landing ZonesFriends Don't Let Friends Build Landing Zones
Friends Don't Let Friends Build Landing Zones
 
IoT 'Megaservices' - High Throughput Microservices with Akka
IoT 'Megaservices' - High Throughput Microservices with AkkaIoT 'Megaservices' - High Throughput Microservices with Akka
IoT 'Megaservices' - High Throughput Microservices with Akka
 
Real time dqmm on flink
Real time dqmm on flinkReal time dqmm on flink
Real time dqmm on flink
 
Iot meets Serverless
Iot meets ServerlessIot meets Serverless
Iot meets Serverless
 
AWS Sydney Meetup April 2016 - Paul Wakeford
AWS Sydney Meetup April 2016 - Paul WakefordAWS Sydney Meetup April 2016 - Paul Wakeford
AWS Sydney Meetup April 2016 - Paul Wakeford
 

Viewers also liked

DevOps/Flow workshop for agile india 2015
DevOps/Flow workshop for agile india 2015DevOps/Flow workshop for agile india 2015
DevOps/Flow workshop for agile india 2015
Yuval Yeret
 
Devops the Microsoft Way
Devops the Microsoft WayDevops the Microsoft Way
Devops the Microsoft Way
Patrick Chanezon
 
DevOps monitoring: Feedback loops in enterprise environments
DevOps monitoring: Feedback loops in enterprise environmentsDevOps monitoring: Feedback loops in enterprise environments
DevOps monitoring: Feedback loops in enterprise environments
Jonah Kowall
 

Viewers also liked (18)

DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)
DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)
DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)
 
Survey on article extraction and comment monitoring techniques
Survey on article extraction and comment monitoring techniquesSurvey on article extraction and comment monitoring techniques
Survey on article extraction and comment monitoring techniques
 
Practical Monitoring Techniques
Practical Monitoring TechniquesPractical Monitoring Techniques
Practical Monitoring Techniques
 
Customer Ops: DevOps <3 customer support
Customer Ops: DevOps <3 customer supportCustomer Ops: DevOps <3 customer support
Customer Ops: DevOps <3 customer support
 
DevconTLV 2014 (Jan) - DIY DevOps
DevconTLV 2014 (Jan) - DIY DevOpsDevconTLV 2014 (Jan) - DIY DevOps
DevconTLV 2014 (Jan) - DIY DevOps
 
Measured availability - Sanjay Singh - DevOps Bangalore meetup March 28th 2015
Measured availability - Sanjay Singh - DevOps Bangalore meetup March 28th 2015Measured availability - Sanjay Singh - DevOps Bangalore meetup March 28th 2015
Measured availability - Sanjay Singh - DevOps Bangalore meetup March 28th 2015
 
5 Ways ITSM can Support DevOps, an ITSM Academy Webinar
5 Ways ITSM can Support DevOps, an ITSM Academy Webinar5 Ways ITSM can Support DevOps, an ITSM Academy Webinar
5 Ways ITSM can Support DevOps, an ITSM Academy Webinar
 
DevOps Roadtrip Minneapolis
DevOps Roadtrip Minneapolis DevOps Roadtrip Minneapolis
DevOps Roadtrip Minneapolis
 
DevOps/Flow workshop for agile india 2015
DevOps/Flow workshop for agile india 2015DevOps/Flow workshop for agile india 2015
DevOps/Flow workshop for agile india 2015
 
Devoxx 2014 monitoring
Devoxx 2014 monitoringDevoxx 2014 monitoring
Devoxx 2014 monitoring
 
Run IT Support the DevOps Way
Run IT Support the DevOps WayRun IT Support the DevOps Way
Run IT Support the DevOps Way
 
Jelastic - DevOps PaaS Business with Docker Support for Service Providers
Jelastic - DevOps PaaS Business with Docker Support for Service ProvidersJelastic - DevOps PaaS Business with Docker Support for Service Providers
Jelastic - DevOps PaaS Business with Docker Support for Service Providers
 
AWS re:Invent 2016: Amazon CloudWatch Logs and AWS Lambda: A Match Made in He...
AWS re:Invent 2016: Amazon CloudWatch Logs and AWS Lambda: A Match Made in He...AWS re:Invent 2016: Amazon CloudWatch Logs and AWS Lambda: A Match Made in He...
AWS re:Invent 2016: Amazon CloudWatch Logs and AWS Lambda: A Match Made in He...
 
Paris Devops - Monitoring And Feature Toggle Pattern With JMX
Paris Devops - Monitoring And Feature Toggle Pattern With JMXParis Devops - Monitoring And Feature Toggle Pattern With JMX
Paris Devops - Monitoring And Feature Toggle Pattern With JMX
 
Devops the Microsoft Way
Devops the Microsoft WayDevops the Microsoft Way
Devops the Microsoft Way
 
DevOps in the Cloud with Microsoft Azure
DevOps in the Cloud with Microsoft AzureDevOps in the Cloud with Microsoft Azure
DevOps in the Cloud with Microsoft Azure
 
DevOps monitoring: Feedback loops in enterprise environments
DevOps monitoring: Feedback loops in enterprise environmentsDevOps monitoring: Feedback loops in enterprise environments
DevOps monitoring: Feedback loops in enterprise environments
 
VIZBI 2015 Tutorial: Cytoscape, IPython, Docker, and Reproducible Network Dat...
VIZBI 2015 Tutorial: Cytoscape, IPython, Docker, and Reproducible Network Dat...VIZBI 2015 Tutorial: Cytoscape, IPython, Docker, and Reproducible Network Dat...
VIZBI 2015 Tutorial: Cytoscape, IPython, Docker, and Reproducible Network Dat...
 

Similar to Which watcher watches CloudWatch

Similar to Which watcher watches CloudWatch (20)

Microservices and Prometheus (Microservices NYC 2016)
Microservices and Prometheus (Microservices NYC 2016)Microservices and Prometheus (Microservices NYC 2016)
Microservices and Prometheus (Microservices NYC 2016)
 
Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)
Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)
Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)
 
Prometheus (Microsoft, 2016)
Prometheus (Microsoft, 2016)Prometheus (Microsoft, 2016)
Prometheus (Microsoft, 2016)
 
An Introduction to Prometheus (GrafanaCon 2016)
An Introduction to Prometheus (GrafanaCon 2016)An Introduction to Prometheus (GrafanaCon 2016)
An Introduction to Prometheus (GrafanaCon 2016)
 
Prometheus: A Next Generation Monitoring System (FOSDEM 2016)
Prometheus: A Next Generation Monitoring System (FOSDEM 2016)Prometheus: A Next Generation Monitoring System (FOSDEM 2016)
Prometheus: A Next Generation Monitoring System (FOSDEM 2016)
 
Prometheus - Open Source Forum Japan
Prometheus  - Open Source Forum JapanPrometheus  - Open Source Forum Japan
Prometheus - Open Source Forum Japan
 
Thinking DevOps in the era of the Cloud - Demi Ben-Ari
Thinking DevOps in the era of the Cloud - Demi Ben-AriThinking DevOps in the era of the Cloud - Demi Ben-Ari
Thinking DevOps in the era of the Cloud - Demi Ben-Ari
 
Prometheus for Monitoring Metrics (Fermilab 2018)
Prometheus for Monitoring Metrics (Fermilab 2018)Prometheus for Monitoring Metrics (Fermilab 2018)
Prometheus for Monitoring Metrics (Fermilab 2018)
 
Ci tips and_tricks_linards_liepins
Ci tips and_tricks_linards_liepinsCi tips and_tricks_linards_liepins
Ci tips and_tricks_linards_liepins
 
Prometheus for the traditional datacenter
Prometheus for the traditional datacenterPrometheus for the traditional datacenter
Prometheus for the traditional datacenter
 
From Monoliths to Microservices at Realestate.com.au
From Monoliths to Microservices at Realestate.com.auFrom Monoliths to Microservices at Realestate.com.au
From Monoliths to Microservices at Realestate.com.au
 
Monitoring Kubernetes with Prometheus (Kubernetes Ireland, 2016)
Monitoring Kubernetes with Prometheus (Kubernetes Ireland, 2016)Monitoring Kubernetes with Prometheus (Kubernetes Ireland, 2016)
Monitoring Kubernetes with Prometheus (Kubernetes Ireland, 2016)
 
Skynet project: Monitor, analyze, scale, and maintain a system in the Cloud
Skynet project: Monitor, analyze, scale, and maintain a system in the CloudSkynet project: Monitor, analyze, scale, and maintain a system in the Cloud
Skynet project: Monitor, analyze, scale, and maintain a system in the Cloud
 
Prometheus (Prometheus London, 2016)
Prometheus (Prometheus London, 2016)Prometheus (Prometheus London, 2016)
Prometheus (Prometheus London, 2016)
 
AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)
AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)
AWS re:Invent 2016: Automating Workflows for Analytics Pipelines (DEV401)
 
Operational Visibiliy and Analytics - BU Seminar
Operational Visibiliy and Analytics - BU SeminarOperational Visibiliy and Analytics - BU Seminar
Operational Visibiliy and Analytics - BU Seminar
 
Go Observability (in practice)
Go Observability (in practice)Go Observability (in practice)
Go Observability (in practice)
 
Expecto Performa! The Magic and Reality of Performance Tuning
Expecto Performa! The Magic and Reality of Performance TuningExpecto Performa! The Magic and Reality of Performance Tuning
Expecto Performa! The Magic and Reality of Performance Tuning
 
Prometheus for Monitoring Metrics (Percona Live Europe 2017)
Prometheus for Monitoring Metrics (Percona Live Europe 2017)Prometheus for Monitoring Metrics (Percona Live Europe 2017)
Prometheus for Monitoring Metrics (Percona Live Europe 2017)
 
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
 

More from David Lutz (7)

2014 devops conferences
2014 devops conferences2014 devops conferences
2014 devops conferences
 
antifragility - DevOps Melbourne November 2013
antifragility - DevOps Melbourne November 2013 antifragility - DevOps Melbourne November 2013
antifragility - DevOps Melbourne November 2013
 
Devops, Dungeons & Dragons
Devops, Dungeons & Dragons Devops, Dungeons & Dragons
Devops, Dungeons & Dragons
 
Infrastructure coders logstash
Infrastructure coders logstashInfrastructure coders logstash
Infrastructure coders logstash
 
Monitoring and metrics in the cloud
Monitoring and metrics in the cloudMonitoring and metrics in the cloud
Monitoring and metrics in the cloud
 
Graphite
GraphiteGraphite
Graphite
 
Kanban thank you maam
Kanban thank you maamKanban thank you maam
Kanban thank you maam
 

Recently uploaded

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 

Recently uploaded (20)

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 

Which watcher watches CloudWatch