Monitoring on Amazon AWS Cloud


Published on

This document explains about the Monitoring on Amazon AWS cloud.
8KMiles provides Cloud Computing Consulting services in following AWS technologies like Amazon EC2 , Elastic Cloud compute , Simple Storage Service , S3 , Simple Queue service , SQS, Simple DB, AutoScaling , Elastic Load Balancing , ELB, Amazon CloudWatch, Amazon CloudFront, Amazon MapReduce and SNS.
Visit us at

Published in: Technology
1 Comment
  • There is a brand new Oracle Middleware and Cloud Computing book available now covering AWS Cloud Watcht etc.:
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Monitoring on Amazon AWS Cloud

  1. 1. Monitoring on Amazon AWS Cloud Cloud computing concepts like on-demand scalability, high-availability and usage based pricing models have pushed real-time monitoring from the side alleys onto center stage. Monitoring tools have become an absolute necessity and constitutes an important part of any distributed/cloud application. It is a challenge for an AWS Cloud administrator to manage and efficiently monitor these new breed of elasticity in their production environment. An ideal monitoring tool on AWS cloud must be  Highly powerful, with minimum (or no) configuration to get started.  Have minimum (or no) impact on the monitored machines' device utilization.  Consume minimal network bandwidth during data collation (as network bandwidth in the cloud comes with a cost overhead).  Support monitoring performance parameters across different types of resources including the app server, database, network, etc and AWS cloud resources like Elastic cloud compute EC2 ,Simple storage Service S3 , Simple Queue Service SQS , Amazon RDS , Elastic Block storage EBS etc  Provide access to all the metrics with user based dashboard views, reports, APIs As part of our work on Cloud Computing Consulting on 8KMiles, we constantly research and try out various monitoring tools continuously. The top 2 monitoring tools that we use for our internal systems are Hyperic HQ and Amazon CloudWatch. Let us see what they are and how they compare in broad level: Cloud Consulting Services Copyright 8KMiles 2009 All rights reserved
  2. 2. Hyperic HQ Hyperic HQ is an open source systems monitoring, server monitoring, and IT management software. Hyperic is easy to deploy and configure. It automatically discovers, monitors, and manages software and network resources. It provides a single pane view of performance and availability of apps running on any OS. It also ensures apps are meeting their SLAs through alerts and automated corrective actions. Hyperic is built for web operations making web-based applications more manageable and to ease the burden on the web operations team. Hyperic HQ is a Java-based platform for monitoring and managing software resources. The key elements of the architecture are the HQ Server, which provides centralized management and persistence, and the HQ Agent, which provides monitoring and control on a per-monitored-platform basis. You run an HQ Agent on each machine you want to manage with Hyperic HQ. Upon first startup, the agent auto-discovers the software running on the EC2 instance, and periodically re- scans for configuration changes. HQ Agents gather availability, utilization, performance, and throughput metrics; perform log and event tracking; and allow you to perform software control actions, for instance, starting and stopping web and application servers. Agents send the inventory data and metrics they collect to a central HQ Server. The HQ Server receives inventory and metric data from HQ Agents and stores it in the HQ database. The server provides facilities for managing your software inventory—it implements the HQ inventory and access model, grouping your software assets in useful ways to ease monitoring and management. Cloud Consulting Services Copyright 8KMiles 2009 All rights reserved
  3. 3. Amazon CloudWatch Amazon CloudWatch is an easy-to-use web service that provides comprehensive monitoring for Amazon Elastic Compute Cloud (Amazon EC2), Elastic Load Balancing (ELB), and Amazon Relational Database Service (Amazon RDS) and Amazon Elastic Block Storage (EBS). It’s very easy to get started with Amazon CloudWatch, there is no upfront setup in order to start getting the metrics. Amazon CloudWatch provides the data you need to make informed decisions. You specify the metric you want and you receive statistics for that metric. This allows you to build complex queries in order to access different views of the data, for periods of one-minute up to two weeks in length. Amazon EC2 Instance Amazon Elastic Load Amazon RDS metrics Amazon EBS Volume Metrics Balancing Metrics metrics CPUUtilization Latency CPUUtilization VolumeReadBytes NetworkIn RequestCount FreeStorageSpace VolumeWriteBytes NetworkOut HealthyHostCount DatabaseConnections VolumeReadOps DiskWriteOps UnHealthyHostCount ReadIOPS VolumeWriteOps DiskReadBytes WriteIOPS VolumeTotalReadTime DiskReadOps ReadLatency VolumeTotalWriteTime DiskWriteBytes WriteLatency VolumeIdleTime ReadThroughput VolumeQueueLength WriteThroughput Information about the metrics is available in Amazon CloudWatch developer guide With Amazon CloudWatch, you only pay for what you use. Amazon CloudWatch is charged on hourly basis by the number of Amazon EC2 instances we monitor. There is no charge for monitoring EBS Volumes, Elastic Load Balancers or RDS Database Instances. Example, a Cloud administrator may want to monitor 10 Amazon EC2 instances 24×7 for a 30-day period. The Amazon CloudWatch cost would be $108 (or $0.015 per Amazon EC2 instance hour x 10 Amazon EC2 instances x 24 hours per day x 30 days). Partial hours will be billed as full hours. (Refer Amazon CloudWatch page for latest rates and updates) Cloud Consulting Services Copyright 8KMiles 2009 All rights reserved
  4. 4. Hyperic HQ Vs Amazon CloudWatch  Hyperic needs installation of HQ Agent on all the EC2 instances to be monitored and HQ server on an EC2 instance to monitor and manage all the other machines. Amazon EC2 needs no software to be installed. Monitoring needs to be enabled to collect all the metrics of the instances, Elastic load balancer, EBS and Amazon RDS.  With Hyperic it’s easy to setup alerts and take automated corrective action. Amazon CloudWatch does not support alerts and automated corrective action feature as of now. We have to develop ourselves the alert and notifications using the AWS CloudWatch API.  Hyperic monitors can monitor databases, web and application servers, middleware, network devices, etc. Amazon EC2 monitors only the specific set of metrics mentioned in the above table. Hyperic is not very efficient in monitoring Amazon RDS and Amazon ELB.  Hyperic HQ open source edition is free and Amazon CloudWatch is charged on usage basis. But we have to pay for the EC2 Instance on which Hyperic HQ server is running even though it is open source.  Configuring Hyperic HQ on AWS cloud is complex compared configuring Amazon CloudWatch on AWS.  Introducing Hyperic HQ in the middle of our production environment life cycle will need re bundling of your new images in Amazon for reusability purpose; Whereas AWS CloudWatch can be introduced and removed at any time with ease.  Amazon CloudWatch must be enabled if we need Auto Scaling features of AWS.  Monitored Data is persisted for 2 weeks in CloudWatch even if the monitored instances are terminated. In Hyperic HQ it can be persisted as much time as we want. At 8KMiles, we use both Hyperic HQ Open Source and Amazon CloudWatch to monitor our production setup. We have provided assistance and consultation service to many of our customers to configure, manage and monitor their production systems using Hyperic HQ and Amazon CloudWatch on AWS. We feel that the blend of both these tools is required to monitor the production systems at this stage because Amazon CloudWatch is in its evolution stage and it is not a comprehensive monitoring Service which will help the Cloud System administrator to monitor a complex production system involving Queues, Amazon MapReduce , Notifications , Simple DB and S3. But we expect to see more features added into Amazon CloudWatch in coming months and we are sharing our inputs and experiences with Amazon AWS team constantly on this front. Cloud Consulting Services Copyright 8KMiles 2009 All rights reserved
  5. 5. Author Santhosh D is a Cloud Engineer at 8KMiles specialized in Cloud Infrastructure and Developing Large Scale Web Apps using Cloud Computing. He is a vital player of the Virtual Computing Environment team where he integrates the AWS Cloud services into 8KMiles platform. Santhosh helps companies to get started on Amazon AWS and then fall in love with it. About 8KMiles 8KMiles is an internet company that is focused on building solutions around cloud computing. 8KMiles Cloud Solutions group offers cloud consulting, engineering and Cloud migration services to help companies leverage the power of cloud computing. Our team of AWS Cloud experts – located in North America and India provides Cloud Consulting, Migration, Remote infrastructure management, Backup, Auto Scaling, GEO Load balancing and Disaster Recovery services over AWS. For more information, visit Contact India Harish Ganesan Co Founder and CTO USA Paddy Co Founder and Chief Evangelist 1-408-647-1217 Cloud Consulting Services Copyright 8KMiles 2009 All rights reserved