SlideShare a Scribd company logo
1 of 26
CloudWatch
  the In’s and Out’s




AWS-DC 2012-01-25
What is CloudWatch?
At its most basic -
AWS instrumentation
Every AWS Service has “Metrics”:

 • ELB Front- and Back-end Response Codes
 • ElastiCache hits and misses
 • EBS IOPS
 • SQS Queue Length
Can even combine them
Every Metric can be converted
       into an Alarm
Alarms can take actions:

• Send message through SNS
• Trigger autoscaling
Even Better
 You can create your own metrics

So you can trigger your own events
Terminology
Metric - a time-ordered set of data points

Dimension - A name/value pair that helps you to
uniquely identify a metric.  e.g.:  EC2 InstanceID

NameSpace - container

Statistic Set - Aggregated set of data points (as
often as once per minute)
Example from the AWS tutorial:

 Pick an arbitrary set of data points

•   Hour one: 87, 51, 125, 235

•   Hour two: 121, 113, 189, 65, 89

•   Hour three: 100, 47, 133, 98, 100, 328
Hour 1 - Individual points
       Hour   Raw Data
        1        87

        1        51

        1       125

        1       235
Hours 2&3 - Stat Sets
       Four predefined keys:  Sum, Minimum, Maximum, and SampleCount




                                                       Sample
Hour             Raw Data                  Sum Min Max
                                                       Count
 2          121,113,189,65,89               577       65     189      5

 3       100,47,133,98,100,328              806       47     328      6
Push with CLI
# For Hour 1
# The unit of measurement is optional
mon-put-data -m RequestLatency -n "Nathan"   -t   2012-01-24T11:00:00   -v   87 -u Milliseconds
mon-put-data -m RequestLatency -n "Nathan"   -t   2012-01-24T11:00:00   -v   51 -u Milliseconds
mon-put-data -m RequestLatency -n "Nathan"   -t   2012-01-24T11:00:00   -v   125 -u Milliseconds
mon-put-data -m RequestLatency -n "Nathan"   -t   2012-01-24T11:00:00   -v   235 -u Milliseconds

# For Hour 2
mon-put-data -m RequestLatency -n "Nathan" -t 2012-01-24T12:00:00 -s "Sum=577,Minimum=47,Maximum=189,SampleCount=5" -u Milliseconds

# For Hour 3
# If no timestamp is provided, it defaults to the current time
mon-put-data -m RequestLatency -n "Nathan" -s "Sum=806,Minimum=47,Maximum=328,SampleCount=6" -u Milliseconds
When you use the mon-put-data command, you must use a
date range within the past two weeks. There is currently no
function to delete data points. Amazon CloudWatch
automatically deletes data points with a timestamp more than
two weeks old.

Can include --dimensions "x=y,u=v" in both puts and gets
Retrieve Stats with CLI
MacBook-Pro:~ user$ mon-get-stats -n Nathan -m RequestLatency -s "Average" --start-time 2012-01-24T11:00:00 --period 3600 --headers

Time          Average Unit
2012-01-24 11:00:00 106.0 Milliseconds
2012-01-24 12:00:00 122.5 Milliseconds
View Online
Quirks of the View
One drawback to CloudWatch is that can be
     difficult to understand the graphs
It’ll report what you ask for - Literally

 E.g. If you leave "Sum" selected and select "Healthy Host Count",
 it adds up all the data points supplied during the period selected.
 So instead of "10" you get "2500".
In this case you'd want min, max or avg.
Have to experiment with different
        view parameters to get an
             accurate picture
 E.g.:  ELB Response Codes - the data points don't represent numbers of coded responses during a
period.  Each one represents one instance of a code received.  So to see the number of 2xx response
                      codes for a period, you need to select the "Sum" statistic
If there aren't enough data-points, it
   won't draw the connecting lines.
Amazon CloudWatch does not aggregate
        data across Regions

                  List of available endpoints and regions:  

  http://docs.amazonwebservices.com/general/latest/gr/rande.html?r=5025
Bottom line:
Create information out of your system statistics
       and then act on it - automatically
Docs and Tools
Documentation:
      http://aws.amazon.com/documentation/cloudwatch/




CloudWatch CLI tools:
Setup Page:
        http://docs.amazonwebservices.com/AmazonCloudWatch/latest/GettingStartedGuide/SetupCLI.html

Set JAVA_HOME on OSX Lion:
        http://steveswinsburg.wordpress.com/2011/07/22/java_home-on-os-x-lion/

Reference for AWS Service Metrics
        http://docs.amazonwebservices.com/AmazonCloudWatch/latest/DeveloperGuide/CW_Support_For_AWS.html



Great How-To with Python and Boto:
         http://loggly.com/blog/2011/05/send-custom-metrics-to-cloudwatchs-api/
Contact Info
 Nathan McCourtney
     @beaknit
   gmail: beaknit

More Related Content

What's hot

Monitoring Modern Applications: Best Practices - SRV338 - Chicago AWS Summit
Monitoring Modern Applications: Best Practices - SRV338 - Chicago AWS SummitMonitoring Modern Applications: Best Practices - SRV338 - Chicago AWS Summit
Monitoring Modern Applications: Best Practices - SRV338 - Chicago AWS SummitAmazon Web Services
 
Training AWS: Module 9 - CloudWatch
Training AWS: Module 9 - CloudWatchTraining AWS: Module 9 - CloudWatch
Training AWS: Module 9 - CloudWatchBùi Quang Lâm
 
ENT203 Monitoring and Autoscaling, a Match Made in Heaven
ENT203 Monitoring and Autoscaling, a Match Made in HeavenENT203 Monitoring and Autoscaling, a Match Made in Heaven
ENT203 Monitoring and Autoscaling, a Match Made in HeavenAmazon Web Services
 
SRV421 Deep Dive with AWS Mobile Services
SRV421 Deep Dive with AWS Mobile ServicesSRV421 Deep Dive with AWS Mobile Services
SRV421 Deep Dive with AWS Mobile ServicesAmazon Web Services
 
AWS re:Invent 2016: How DataXu scaled its Attribution System to handle billio...
AWS re:Invent 2016: How DataXu scaled its Attribution System to handle billio...AWS re:Invent 2016: How DataXu scaled its Attribution System to handle billio...
AWS re:Invent 2016: How DataXu scaled its Attribution System to handle billio...Amazon Web Services
 
Securing Serverless Architectures
Securing Serverless ArchitecturesSecuring Serverless Architectures
Securing Serverless ArchitecturesAmazon Web Services
 
AWS re:Invent 2016 Day 2 Keynote re:Cap
AWS re:Invent 2016 Day 2 Keynote re:CapAWS re:Invent 2016 Day 2 Keynote re:Cap
AWS re:Invent 2016 Day 2 Keynote re:CapIan Massingham
 
AWS September Webinar Series - Infrastructure Deployment and Monitoring with ...
AWS September Webinar Series - Infrastructure Deployment and Monitoring with ...AWS September Webinar Series - Infrastructure Deployment and Monitoring with ...
AWS September Webinar Series - Infrastructure Deployment and Monitoring with ...Amazon Web Services
 
(SEC309) Amazon VPC Configuration: When Least Privilege Meets the Penetration...
(SEC309) Amazon VPC Configuration: When Least Privilege Meets the Penetration...(SEC309) Amazon VPC Configuration: When Least Privilege Meets the Penetration...
(SEC309) Amazon VPC Configuration: When Least Privilege Meets the Penetration...Amazon Web Services
 
Cloudwatch: Monitoring your Services with Metrics and Alarms
Cloudwatch: Monitoring your Services with Metrics and AlarmsCloudwatch: Monitoring your Services with Metrics and Alarms
Cloudwatch: Monitoring your Services with Metrics and AlarmsFelipe
 
AWS re:Invent 2016: Workshop: Building Your First Big Data Application with A...
AWS re:Invent 2016: Workshop: Building Your First Big Data Application with A...AWS re:Invent 2016: Workshop: Building Your First Big Data Application with A...
AWS re:Invent 2016: Workshop: Building Your First Big Data Application with A...Amazon Web Services
 
Amazon CloudWatch Logs and AWS Lambda: A Match Made in Heaven | AWS Public Se...
Amazon CloudWatch Logs and AWS Lambda: A Match Made in Heaven | AWS Public Se...Amazon CloudWatch Logs and AWS Lambda: A Match Made in Heaven | AWS Public Se...
Amazon CloudWatch Logs and AWS Lambda: A Match Made in Heaven | AWS Public Se...Amazon Web Services
 
Introduction to Amazon Lightsail
Introduction to Amazon LightsailIntroduction to Amazon Lightsail
Introduction to Amazon LightsailAmazon Web Services
 
Cloudwatch: Monitoring your AWS services with Metrics and Alarms
Cloudwatch: Monitoring your AWS services with Metrics and AlarmsCloudwatch: Monitoring your AWS services with Metrics and Alarms
Cloudwatch: Monitoring your AWS services with Metrics and AlarmsFelipe
 
Manage Security & Compliance of Your AWS Account using CloudTrail
Manage Security & Compliance of Your AWS Account using CloudTrailManage Security & Compliance of Your AWS Account using CloudTrail
Manage Security & Compliance of Your AWS Account using CloudTrailCloudlytics
 
Gaining Operational Insights out of Your Logs
Gaining Operational Insights out of Your LogsGaining Operational Insights out of Your Logs
Gaining Operational Insights out of Your LogsAmazon Web Services
 
Stream Processing in SmartNews #jawsdays
Stream Processing in SmartNews #jawsdaysStream Processing in SmartNews #jawsdays
Stream Processing in SmartNews #jawsdaysSmartNews, Inc.
 
(DVO303) Scaling Infrastructure Operations with AWS
(DVO303) Scaling Infrastructure Operations with AWS(DVO303) Scaling Infrastructure Operations with AWS
(DVO303) Scaling Infrastructure Operations with AWSAmazon Web Services
 

What's hot (20)

Monitoring Modern Applications: Best Practices - SRV338 - Chicago AWS Summit
Monitoring Modern Applications: Best Practices - SRV338 - Chicago AWS SummitMonitoring Modern Applications: Best Practices - SRV338 - Chicago AWS Summit
Monitoring Modern Applications: Best Practices - SRV338 - Chicago AWS Summit
 
Training AWS: Module 9 - CloudWatch
Training AWS: Module 9 - CloudWatchTraining AWS: Module 9 - CloudWatch
Training AWS: Module 9 - CloudWatch
 
ENT203 Monitoring and Autoscaling, a Match Made in Heaven
ENT203 Monitoring and Autoscaling, a Match Made in HeavenENT203 Monitoring and Autoscaling, a Match Made in Heaven
ENT203 Monitoring and Autoscaling, a Match Made in Heaven
 
SRV421 Deep Dive with AWS Mobile Services
SRV421 Deep Dive with AWS Mobile ServicesSRV421 Deep Dive with AWS Mobile Services
SRV421 Deep Dive with AWS Mobile Services
 
Amazon S3 Deep Dive
Amazon S3 Deep DiveAmazon S3 Deep Dive
Amazon S3 Deep Dive
 
AWS re:Invent 2016: How DataXu scaled its Attribution System to handle billio...
AWS re:Invent 2016: How DataXu scaled its Attribution System to handle billio...AWS re:Invent 2016: How DataXu scaled its Attribution System to handle billio...
AWS re:Invent 2016: How DataXu scaled its Attribution System to handle billio...
 
Securing Serverless Architectures
Securing Serverless ArchitecturesSecuring Serverless Architectures
Securing Serverless Architectures
 
AWS re:Invent 2016 Day 2 Keynote re:Cap
AWS re:Invent 2016 Day 2 Keynote re:CapAWS re:Invent 2016 Day 2 Keynote re:Cap
AWS re:Invent 2016 Day 2 Keynote re:Cap
 
AWS September Webinar Series - Infrastructure Deployment and Monitoring with ...
AWS September Webinar Series - Infrastructure Deployment and Monitoring with ...AWS September Webinar Series - Infrastructure Deployment and Monitoring with ...
AWS September Webinar Series - Infrastructure Deployment and Monitoring with ...
 
(SEC309) Amazon VPC Configuration: When Least Privilege Meets the Penetration...
(SEC309) Amazon VPC Configuration: When Least Privilege Meets the Penetration...(SEC309) Amazon VPC Configuration: When Least Privilege Meets the Penetration...
(SEC309) Amazon VPC Configuration: When Least Privilege Meets the Penetration...
 
Cloudwatch: Monitoring your Services with Metrics and Alarms
Cloudwatch: Monitoring your Services with Metrics and AlarmsCloudwatch: Monitoring your Services with Metrics and Alarms
Cloudwatch: Monitoring your Services with Metrics and Alarms
 
AWS re:Invent 2016: Workshop: Building Your First Big Data Application with A...
AWS re:Invent 2016: Workshop: Building Your First Big Data Application with A...AWS re:Invent 2016: Workshop: Building Your First Big Data Application with A...
AWS re:Invent 2016: Workshop: Building Your First Big Data Application with A...
 
Amazon CloudWatch Logs and AWS Lambda: A Match Made in Heaven | AWS Public Se...
Amazon CloudWatch Logs and AWS Lambda: A Match Made in Heaven | AWS Public Se...Amazon CloudWatch Logs and AWS Lambda: A Match Made in Heaven | AWS Public Se...
Amazon CloudWatch Logs and AWS Lambda: A Match Made in Heaven | AWS Public Se...
 
Introduction to Amazon Lightsail
Introduction to Amazon LightsailIntroduction to Amazon Lightsail
Introduction to Amazon Lightsail
 
Cloudwatch: Monitoring your AWS services with Metrics and Alarms
Cloudwatch: Monitoring your AWS services with Metrics and AlarmsCloudwatch: Monitoring your AWS services with Metrics and Alarms
Cloudwatch: Monitoring your AWS services with Metrics and Alarms
 
Manage Security & Compliance of Your AWS Account using CloudTrail
Manage Security & Compliance of Your AWS Account using CloudTrailManage Security & Compliance of Your AWS Account using CloudTrail
Manage Security & Compliance of Your AWS Account using CloudTrail
 
Gaining Operational Insights out of Your Logs
Gaining Operational Insights out of Your LogsGaining Operational Insights out of Your Logs
Gaining Operational Insights out of Your Logs
 
SRV408 Deep Dive on AWS IoT
SRV408 Deep Dive on AWS IoTSRV408 Deep Dive on AWS IoT
SRV408 Deep Dive on AWS IoT
 
Stream Processing in SmartNews #jawsdays
Stream Processing in SmartNews #jawsdaysStream Processing in SmartNews #jawsdays
Stream Processing in SmartNews #jawsdays
 
(DVO303) Scaling Infrastructure Operations with AWS
(DVO303) Scaling Infrastructure Operations with AWS(DVO303) Scaling Infrastructure Operations with AWS
(DVO303) Scaling Infrastructure Operations with AWS
 

Viewers also liked

Monitoring, troubleshooting,
Monitoring, troubleshooting,Monitoring, troubleshooting,
Monitoring, troubleshooting,aspnet123
 
S4 trouble shooting, AgencY S4
S4 trouble shooting, AgencY S4S4 trouble shooting, AgencY S4
S4 trouble shooting, AgencY S4S4 (sale4.me)
 
Site24x7 Cloud Monitoring
Site24x7 Cloud MonitoringSite24x7 Cloud Monitoring
Site24x7 Cloud MonitoringSite24x7
 
Java performance and trouble shooting
Java performance and trouble shootingJava performance and trouble shooting
Java performance and trouble shootingAnna Choi
 
Redis trouble shooting_eng
Redis trouble shooting_engRedis trouble shooting_eng
Redis trouble shooting_engDaeMyung Kang
 
PyconJP: Building a data preparation pipeline with Pandas and AWS Lambda
PyconJP: Building a data preparation pipeline with Pandas and AWS LambdaPyconJP: Building a data preparation pipeline with Pandas and AWS Lambda
PyconJP: Building a data preparation pipeline with Pandas and AWS LambdaFabian Dubois
 
(DVO315) Log, Monitor and Analyze your IT with Amazon CloudWatch
(DVO315) Log, Monitor and Analyze your IT with Amazon CloudWatch(DVO315) Log, Monitor and Analyze your IT with Amazon CloudWatch
(DVO315) Log, Monitor and Analyze your IT with Amazon CloudWatchAmazon Web Services
 
Using dynaTrace to optimise application performance
Using dynaTrace to optimise application performanceUsing dynaTrace to optimise application performance
Using dynaTrace to optimise application performanceRichard Bishop
 

Viewers also liked (9)

Monitoring, troubleshooting,
Monitoring, troubleshooting,Monitoring, troubleshooting,
Monitoring, troubleshooting,
 
S4 trouble shooting, AgencY S4
S4 trouble shooting, AgencY S4S4 trouble shooting, AgencY S4
S4 trouble shooting, AgencY S4
 
Site24x7 Cloud Monitoring
Site24x7 Cloud MonitoringSite24x7 Cloud Monitoring
Site24x7 Cloud Monitoring
 
Java performance and trouble shooting
Java performance and trouble shootingJava performance and trouble shooting
Java performance and trouble shooting
 
Redis trouble shooting_eng
Redis trouble shooting_engRedis trouble shooting_eng
Redis trouble shooting_eng
 
PyconJP: Building a data preparation pipeline with Pandas and AWS Lambda
PyconJP: Building a data preparation pipeline with Pandas and AWS LambdaPyconJP: Building a data preparation pipeline with Pandas and AWS Lambda
PyconJP: Building a data preparation pipeline with Pandas and AWS Lambda
 
(DVO315) Log, Monitor and Analyze your IT with Amazon CloudWatch
(DVO315) Log, Monitor and Analyze your IT with Amazon CloudWatch(DVO315) Log, Monitor and Analyze your IT with Amazon CloudWatch
(DVO315) Log, Monitor and Analyze your IT with Amazon CloudWatch
 
Using dynaTrace to optimise application performance
Using dynaTrace to optimise application performanceUsing dynaTrace to optimise application performance
Using dynaTrace to optimise application performance
 
Metrics 101
Metrics 101Metrics 101
Metrics 101
 

Similar to Cloudwatch - The In's and Out's

Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017Amazon Web Services
 
Amazon CloudWatch - Observability and Monitoring
Amazon CloudWatch - Observability and MonitoringAmazon CloudWatch - Observability and Monitoring
Amazon CloudWatch - Observability and MonitoringRick Hwang
 
StackWatch: A prototype CloudWatch service for CloudStack
StackWatch: A prototype CloudWatch service for CloudStackStackWatch: A prototype CloudWatch service for CloudStack
StackWatch: A prototype CloudWatch service for CloudStackChiradeep Vittal
 
Build a custom metrics on aws cloud
Build a custom metrics on aws cloudBuild a custom metrics on aws cloud
Build a custom metrics on aws cloudAhmad karawash
 
Using AWS CloudWatch Custom Metrics and EC2 Auto Scaling -VSocial Infrastructure
Using AWS CloudWatch Custom Metrics and EC2 Auto Scaling -VSocial InfrastructureUsing AWS CloudWatch Custom Metrics and EC2 Auto Scaling -VSocial Infrastructure
Using AWS CloudWatch Custom Metrics and EC2 Auto Scaling -VSocial InfrastructureChristopher Drumgoole
 
Cloudera Movies Data Science Project On Big Data
Cloudera Movies Data Science Project On Big DataCloudera Movies Data Science Project On Big Data
Cloudera Movies Data Science Project On Big DataAbhishek M Shivalingaiah
 
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic SystemTimely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic SystemAccumulo Summit
 
AWS re:Invent 2016: IoT Blueprints: Optimizing Supply for Smart Agriculture f...
AWS re:Invent 2016: IoT Blueprints: Optimizing Supply for Smart Agriculture f...AWS re:Invent 2016: IoT Blueprints: Optimizing Supply for Smart Agriculture f...
AWS re:Invent 2016: IoT Blueprints: Optimizing Supply for Smart Agriculture f...Amazon Web Services
 
Metrics simplified
Metrics simplifiedMetrics simplified
Metrics simplifiedlinmark333
 
AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...
AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...
AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...Amazon Web Services
 
AutoScaling and Drupal
AutoScaling and DrupalAutoScaling and Drupal
AutoScaling and DrupalPromet Source
 
AWS re:Invent 2016: Deep Dive on Amazon EC2 Instances, Featuring Performance ...
AWS re:Invent 2016: Deep Dive on Amazon EC2 Instances, Featuring Performance ...AWS re:Invent 2016: Deep Dive on Amazon EC2 Instances, Featuring Performance ...
AWS re:Invent 2016: Deep Dive on Amazon EC2 Instances, Featuring Performance ...Amazon Web Services
 
Harness the Power of the Cloud for Grid Computing and Batch Processing Applic...
Harness the Power of the Cloud for Grid Computing and Batch Processing Applic...Harness the Power of the Cloud for Grid Computing and Batch Processing Applic...
Harness the Power of the Cloud for Grid Computing and Batch Processing Applic...RightScale
 
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Amazon Web Services
 
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...Amazon Web Services
 
Getting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise MonitorGetting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise MonitorMark Leith
 
Flink Forward San Francisco 2018: David Reniz & Dahyr Vergara - "Real-time m...
Flink Forward San Francisco 2018:  David Reniz & Dahyr Vergara - "Real-time m...Flink Forward San Francisco 2018:  David Reniz & Dahyr Vergara - "Real-time m...
Flink Forward San Francisco 2018: David Reniz & Dahyr Vergara - "Real-time m...Flink Forward
 
Auto-Scaling Apache Spark cluster using Deep Reinforcement Learning.pdf
Auto-Scaling Apache Spark cluster using Deep Reinforcement Learning.pdfAuto-Scaling Apache Spark cluster using Deep Reinforcement Learning.pdf
Auto-Scaling Apache Spark cluster using Deep Reinforcement Learning.pdfKundjanasith Thonglek
 
Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...
Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...
Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...MSAdvAnalytics
 

Similar to Cloudwatch - The In's and Out's (20)

Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
Tooling Up for Efficiency: DIY Solutions @ Netflix - ABD319 - re:Invent 2017
 
Amazon CloudWatch - Observability and Monitoring
Amazon CloudWatch - Observability and MonitoringAmazon CloudWatch - Observability and Monitoring
Amazon CloudWatch - Observability and Monitoring
 
StackWatch: A prototype CloudWatch service for CloudStack
StackWatch: A prototype CloudWatch service for CloudStackStackWatch: A prototype CloudWatch service for CloudStack
StackWatch: A prototype CloudWatch service for CloudStack
 
Build a custom metrics on aws cloud
Build a custom metrics on aws cloudBuild a custom metrics on aws cloud
Build a custom metrics on aws cloud
 
Using AWS CloudWatch Custom Metrics and EC2 Auto Scaling -VSocial Infrastructure
Using AWS CloudWatch Custom Metrics and EC2 Auto Scaling -VSocial InfrastructureUsing AWS CloudWatch Custom Metrics and EC2 Auto Scaling -VSocial Infrastructure
Using AWS CloudWatch Custom Metrics and EC2 Auto Scaling -VSocial Infrastructure
 
Cloudera Movies Data Science Project On Big Data
Cloudera Movies Data Science Project On Big DataCloudera Movies Data Science Project On Big Data
Cloudera Movies Data Science Project On Big Data
 
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic SystemTimely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
 
AWS re:Invent 2016: IoT Blueprints: Optimizing Supply for Smart Agriculture f...
AWS re:Invent 2016: IoT Blueprints: Optimizing Supply for Smart Agriculture f...AWS re:Invent 2016: IoT Blueprints: Optimizing Supply for Smart Agriculture f...
AWS re:Invent 2016: IoT Blueprints: Optimizing Supply for Smart Agriculture f...
 
Metrics simplified
Metrics simplifiedMetrics simplified
Metrics simplified
 
AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...
AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...
AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...
 
AutoScaling and Drupal
AutoScaling and DrupalAutoScaling and Drupal
AutoScaling and Drupal
 
AWS re:Invent 2016: Deep Dive on Amazon EC2 Instances, Featuring Performance ...
AWS re:Invent 2016: Deep Dive on Amazon EC2 Instances, Featuring Performance ...AWS re:Invent 2016: Deep Dive on Amazon EC2 Instances, Featuring Performance ...
AWS re:Invent 2016: Deep Dive on Amazon EC2 Instances, Featuring Performance ...
 
Load Data Fast!
Load Data Fast!Load Data Fast!
Load Data Fast!
 
Harness the Power of the Cloud for Grid Computing and Batch Processing Applic...
Harness the Power of the Cloud for Grid Computing and Batch Processing Applic...Harness the Power of the Cloud for Grid Computing and Batch Processing Applic...
Harness the Power of the Cloud for Grid Computing and Batch Processing Applic...
 
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
 
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
More Nines for Your Dimes: Improving Availability and Lowering Costs using Au...
 
Getting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise MonitorGetting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise Monitor
 
Flink Forward San Francisco 2018: David Reniz & Dahyr Vergara - "Real-time m...
Flink Forward San Francisco 2018:  David Reniz & Dahyr Vergara - "Real-time m...Flink Forward San Francisco 2018:  David Reniz & Dahyr Vergara - "Real-time m...
Flink Forward San Francisco 2018: David Reniz & Dahyr Vergara - "Real-time m...
 
Auto-Scaling Apache Spark cluster using Deep Reinforcement Learning.pdf
Auto-Scaling Apache Spark cluster using Deep Reinforcement Learning.pdfAuto-Scaling Apache Spark cluster using Deep Reinforcement Learning.pdf
Auto-Scaling Apache Spark cluster using Deep Reinforcement Learning.pdf
 
Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...
Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...
Cortana Analytics Workshop: Real-Time Data Processing -- How Do I Choose the ...
 

Recently uploaded

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Jeffrey Haguewood
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native ApplicationsWSO2
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesrafiqahmad00786416
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Bhuvaneswari Subramani
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Angeliki Cooney
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Zilliz
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfOrbitshub
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...Zilliz
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Victor Rentea
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 

Recently uploaded (20)

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 

Cloudwatch - The In's and Out's

  • 1. CloudWatch the In’s and Out’s AWS-DC 2012-01-25
  • 3. At its most basic - AWS instrumentation
  • 4. Every AWS Service has “Metrics”: • ELB Front- and Back-end Response Codes • ElastiCache hits and misses • EBS IOPS • SQS Queue Length
  • 5.
  • 7. Every Metric can be converted into an Alarm
  • 8. Alarms can take actions: • Send message through SNS • Trigger autoscaling
  • 9. Even Better You can create your own metrics So you can trigger your own events
  • 10. Terminology Metric - a time-ordered set of data points Dimension - A name/value pair that helps you to uniquely identify a metric.  e.g.:  EC2 InstanceID NameSpace - container Statistic Set - Aggregated set of data points (as often as once per minute)
  • 11. Example from the AWS tutorial: Pick an arbitrary set of data points • Hour one: 87, 51, 125, 235 • Hour two: 121, 113, 189, 65, 89 • Hour three: 100, 47, 133, 98, 100, 328
  • 12. Hour 1 - Individual points Hour Raw Data 1 87 1 51 1 125 1 235
  • 13. Hours 2&3 - Stat Sets Four predefined keys:  Sum, Minimum, Maximum, and SampleCount Sample Hour Raw Data Sum Min Max Count 2 121,113,189,65,89 577 65 189 5 3 100,47,133,98,100,328 806 47 328 6
  • 14. Push with CLI # For Hour 1 # The unit of measurement is optional mon-put-data -m RequestLatency -n "Nathan" -t 2012-01-24T11:00:00 -v 87 -u Milliseconds mon-put-data -m RequestLatency -n "Nathan" -t 2012-01-24T11:00:00 -v 51 -u Milliseconds mon-put-data -m RequestLatency -n "Nathan" -t 2012-01-24T11:00:00 -v 125 -u Milliseconds mon-put-data -m RequestLatency -n "Nathan" -t 2012-01-24T11:00:00 -v 235 -u Milliseconds # For Hour 2 mon-put-data -m RequestLatency -n "Nathan" -t 2012-01-24T12:00:00 -s "Sum=577,Minimum=47,Maximum=189,SampleCount=5" -u Milliseconds # For Hour 3 # If no timestamp is provided, it defaults to the current time mon-put-data -m RequestLatency -n "Nathan" -s "Sum=806,Minimum=47,Maximum=328,SampleCount=6" -u Milliseconds
  • 15. When you use the mon-put-data command, you must use a date range within the past two weeks. There is currently no function to delete data points. Amazon CloudWatch automatically deletes data points with a timestamp more than two weeks old. Can include --dimensions "x=y,u=v" in both puts and gets
  • 16. Retrieve Stats with CLI MacBook-Pro:~ user$ mon-get-stats -n Nathan -m RequestLatency -s "Average" --start-time 2012-01-24T11:00:00 --period 3600 --headers Time Average Unit 2012-01-24 11:00:00 106.0 Milliseconds 2012-01-24 12:00:00 122.5 Milliseconds
  • 18. Quirks of the View One drawback to CloudWatch is that can be difficult to understand the graphs
  • 19. It’ll report what you ask for - Literally E.g. If you leave "Sum" selected and select "Healthy Host Count", it adds up all the data points supplied during the period selected. So instead of "10" you get "2500".
  • 20. In this case you'd want min, max or avg.
  • 21. Have to experiment with different view parameters to get an accurate picture E.g.:  ELB Response Codes - the data points don't represent numbers of coded responses during a period.  Each one represents one instance of a code received.  So to see the number of 2xx response codes for a period, you need to select the "Sum" statistic
  • 22. If there aren't enough data-points, it won't draw the connecting lines.
  • 23. Amazon CloudWatch does not aggregate data across Regions List of available endpoints and regions:   http://docs.amazonwebservices.com/general/latest/gr/rande.html?r=5025
  • 24. Bottom line: Create information out of your system statistics and then act on it - automatically
  • 25. Docs and Tools Documentation: http://aws.amazon.com/documentation/cloudwatch/ CloudWatch CLI tools: Setup Page: http://docs.amazonwebservices.com/AmazonCloudWatch/latest/GettingStartedGuide/SetupCLI.html Set JAVA_HOME on OSX Lion: http://steveswinsburg.wordpress.com/2011/07/22/java_home-on-os-x-lion/ Reference for AWS Service Metrics http://docs.amazonwebservices.com/AmazonCloudWatch/latest/DeveloperGuide/CW_Support_For_AWS.html Great How-To with Python and Boto: http://loggly.com/blog/2011/05/send-custom-metrics-to-cloudwatchs-api/
  • 26. Contact Info Nathan McCourtney @beaknit gmail: beaknit

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n