Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building a Global Multi-Tenant Monitoring Platform


Published on

Datapipe, an AWS Premier Consulting Partner, has built and customized a global monitoring platform specifically for AWS. This presentation discusses the challenges encountered when architecting this solution and provides a live demonstration of the platform and its specific monitoring capabilities.

Published in: Technology
  • SaaS Life Cycle Management encompasses of all the typical phases/activities that are performed on a day to day SaaS business model.
    Are you sure you want to  Yes  No
    Your message goes here

Building a Global Multi-Tenant Monitoring Platform

  1. 1. © 2014, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of, Inc. Building a Global Multi-Tenant Monitoring Platform Jeff Moody, Datapipe
  2. 2. #> whoami • Jeff Moody – Cloud Engineer with Datapipe – Principle engineer for architecting and building the Datapipe Cloud Monitoring Platform product – Extensive work in Cloud Computing and DevOps tool chains
  3. 3. Disclaimer • My opinions are my own, not necessarily representative of my employer • Your Mileage May Vary
  4. 4. Who is Datapipe? • Managed Service Provider – Headquartered in Jersey City, NJ – Premier AWS Consulting Partner – Single provider solution for mission critical IT – Offices and data centers in New Jersey, Northern Virginia, Silicon Valley, Iceland, London, Hong Kong, Shanghai, and Singapore – Fully and partially managed solutions running on Linux, UNIX, and Windows
  5. 5. What are Datapipe’s Managed Service Offerings? • Operating System Management & Support • Application Stack Management & Support • Network Management & Support • DBA Management & Support • Security & Compliance Management & Support – PCI, HIPAA
  6. 6. What does Datapipe bring to AWS? • AWS Premier Consulting Partner – Exchange, MSP, Oracle, and SharePoint Competencies • Managed AWS – Single Vendor Billing & Governance – Cloud Provisioning of all AWS services including: EC2, S3, ELB, and CloudFront – Infrastructure and Application Monitoring – DBA Tuning and Optimization – Managed Scaling – Managed Security Groups – 24x7 Issue Response and Resolution
  7. 7. What does Datapipe bring to AWS? • AWS Optimization Governance, security, and financial optimization tools – Cloud Reports • Implement cloud budgeting and Reserved Instances across multiple AWS accounts • Push-button 24x7 response on critical security and utilization insights – Datapipe Cloud Management Portal with Cloud Apps • Single pane view for security, technology & billing – Committed Instances • The benefits of Reserved Instances without the upfront costs
  8. 8. What does Datapipe bring to AWS? • AWS Direct Connect - Simplify Hybrid IT – Datapipe AWS Direct Connect Enabled Services • Turnkey DX solutions - No CAPEX • DBaaS (Oracle RAC, MS SQL Enterprise) • Enterprise SANs, Crypto equipment, and more – AWS Direct-Connect from: • NJ to AWS US-East • Virginia to AWS US-East • Silicon Valley to AWS US-West (North California) • London to AWS EU (Ireland) • Singapore to AWS Asia Pacific (Singapore)
  9. 9. What is Cloud Reports? Optimize and Forecast Implement cloud budgeting with confidence and optimize Reserved Instances across multiple AWS accounts and internal departments. Insight Response and Resolution Push-button 24x7 response on critical security and utilization insights from Datapipe’s award winning global managed services support. Custom Reporting for Individual Business Units Dashboards and reports for the CEO, CTO, CFO, and development teams to get the most out of AWS. PROBLEM: How do I efficiently use the vast amounts of operational data on my AWS environments to provide insightful, actionable information? ANSWER: Datapipe Cloud Reports for AWS
  10. 10. The Problem • Late 2011 • Growing number of customers leveraging AWS • Current monitoring platform built to work inside datacenters • Elasticity
  11. 11. The Solutions? • Fix legacy monitoring platform – Non-starter as Datacenter monitoring moving to new SNMP- based platform – SNMP not the best choice for the Cloud • Roll a new platform – Substantial development effort – Support and maintenance nightmare • Build a new platform from a different, non-SNMP vendor
  12. 12. The Solution • Monitoring Vendor Provided Solution Challenges – Multi-tenancy – Agents? • Legacy Monitoring was Agent Based, caused a lot of finger-pointing • Setup? (Automation toolset? Still often requires an agent) – Centralized Management • Easy deployment of “local” data collection to keep latency low
  13. 13. Datapipe Cloud Monitoring • Powered by Zenoss Service Dynamics – Substantial number of customizations – Agentless (leverages SSH and WMI) – Scalable – Supportable
  14. 14. Deployment Architecture Master Collector Customer Server Customer Server Customer Server Collector Customer Server Customer Server Customer Server Customer Server Customer Server Customer Server Collector VPC
  15. 15. Deployment Challenges • AWS ZenPack requires one key per account for auto discovery • Distributing keys to all collectors is bad • Programmability
  16. 16. Development Challenges • Enrollment of Instance through a single-click – Update Security Group Rules – Ensure presence of necessary keys on appropriate collector – Ensure presence in Datapipe inventory – Provide synchronization of information between monitoring, inventory, and ticketing system – Clean removal of instance on shutdown
  17. 17. Hybrid Data Solutions • Datapipe already has collectors in every data center – VLANs can be extended into the local collector for physical host monitoring – Single tool for full physical and virtual Cloud Monitoring
  18. 18. User Interface • Zenoss UI is confusing • Zenoss permissions are difficult to manage • Abstracted into Datapipe Cloud Portal – One-click setup
  19. 19. What does Cloud Monitoring mean to me? • Solution Escalation Action Plan (SEAP) – Alerts and procedures based on observed events – i.e.: Scale an environment during a traffic spike, or who to contact when an error is seen • SLAs – Clear reporting to ensure SLAs are met • Accountability & Peace of Mind – Understand the health of your environment
  20. 20. What does Cloud Monitoring Provide? • Process monitoring – Auto-discovery for most major applications • Web Stacks: Apache, nginx, HAProxy, Rails, Zope, IIS, Tomcat • Databases: MySQL, MSSQL, PGSQL, Mongo, Riak • URL Monitoring – HTTP content checks – HTTP response codes – Response times • System monitoring – CPU, Memory, Disk
  21. 21. Instance Registration/Monitoring Demo • This is a live demo • Please be patient
  22. 22. Questions? Jeff Moody Cloud Engineer with Datapipe E-Mail: Twitter: @fifthecho
  23. 23. Please give us your feedback on this presentation As a thank you, we will select prize winners daily for completed surveys!