Your SlideShare is downloading. ×
0
© 2013 Cloud Technology Partners, Inc. / Confidential
1
Cloud Technology Partners / April 2014 / www.cloudtp.com
Monitorin...
© 2013 Cloud Technology Partners, Inc. / Confidential
2
About the Presenter
@madgreek65
mikekavis
madgreek65
VP/Principal ...
© 2013 Cloud Technology Partners, Inc. / Confidential
3
Topics of Discussion
1. Service Centric Ops
2. Logging Strategies
...
© 2013 Cloud Technology Partners, Inc. / Confidential
4
Service Centric Ops
© 2013 Cloud Technology Partners, Inc. / Confidential
5
What needs to Change?
Shift thinking away from product-centric to ...
© 2013 Cloud Technology Partners, Inc. / Confidential
6
What needs to Change?
Traditional Challenge – Dev needs speed, Ops...
© 2013 Cloud Technology Partners, Inc. / Confidential
7
What needs to Change?
Shift thinking away from product-centric to ...
© 2013 Cloud Technology Partners, Inc. / Confidential
8
What needs to Change?
Whoever prioritizes the backlog must
be acco...
© 2013 Cloud Technology Partners, Inc. / Confidential
9
Logging Strategies
© 2013 Cloud Technology Partners, Inc. / Confidential
10
Top Log Use Cases
– Troubleshooting – debugging information and
e...
© 2013 Cloud Technology Partners, Inc. / Confidential
11
Centralized Logs
– Pipe logs to Sysout and direct to log
services...
© 2013 Cloud Technology Partners, Inc. / Confidential
12
Without standards, Logs are “Garbage in, Garbage out”
– Things to...
© 2013 Cloud Technology Partners, Inc. / Confidential
13
Best Practices
– Log Everything, Monitor Everything
• Infrastruct...
© 2013 Cloud Technology Partners, Inc. / Confidential
14
Common Logging Solutions
Open Source Commercial
© 2013 Cloud Technology Partners, Inc. / Confidential
15
Monitoring Strategies
© 2013 Cloud Technology Partners, Inc. / Confidential
16
Nagios is not a Monitoring Strategy
Blind spots can kill you
© 2013 Cloud Technology Partners, Inc. / Confidential
17
What needs to be Monitored?
Data Category Description
Performance...
© 2013 Cloud Technology Partners, Inc. / Confidential
18
End to end Monitoring is Required
There is no ONE tool that does ...
© 2013 Cloud Technology Partners, Inc. / Confidential
19
Who needs Monitoring/Logging Data?
Actor Purpose
Product Manager ...
© 2013 Cloud Technology Partners, Inc. / Confidential
20
Synthesized Production Data and Monitoring
Production data that i...
© 2013 Cloud Technology Partners, Inc. / Confidential
21
Think ahead: Create strategies for logging & monitoring
– Log and...
© 2013 Cloud Technology Partners, Inc. / Confidential
22
Thank you for your time and interest.
Upcoming SlideShare
Loading in...5
×

Monitoring in the DevOps Era

1,173

Published on

Today's cloud implementations require a different approach to monitoring. This presentation discusses the mindset required and discusses logging and monitoring strategies and tools.

Transcript of "Monitoring in the DevOps Era"

  1. 1. © 2013 Cloud Technology Partners, Inc. / Confidential 1 Cloud Technology Partners / April 2014 / www.cloudtp.com Monitoring in the DevOps Era
  2. 2. © 2013 Cloud Technology Partners, Inc. / Confidential 2 About the Presenter @madgreek65 mikekavis madgreek65 VP/Principal Architect @ Cloud Technology Partners Mike Kavis The Virtualization Practice madgreek65 DevOps.com
  3. 3. © 2013 Cloud Technology Partners, Inc. / Confidential 3 Topics of Discussion 1. Service Centric Ops 2. Logging Strategies 3. Monitoring Strategies
  4. 4. © 2013 Cloud Technology Partners, Inc. / Confidential 4 Service Centric Ops
  5. 5. © 2013 Cloud Technology Partners, Inc. / Confidential 5 What needs to Change? Shift thinking away from product-centric to service-centric Operating a Service 24x7x365Shipping Product
  6. 6. © 2013 Cloud Technology Partners, Inc. / Confidential 6 What needs to Change? Traditional Challenge – Dev needs speed, Ops needs control Speed APIs Security Compliance Availability Auditing The Great Balancing Act
  7. 7. © 2013 Cloud Technology Partners, Inc. / Confidential 7 What needs to Change? Shift thinking away from product-centric to service-centric Old Way New Way Software is built and shipped Services are running and managed Development of features are done Services are never done until they are turned off Product owner focus only on features Product owner owns operational results along with product feature set Each silo owns their own area All groups focus on end user satisfaction Dev must go through Ops to get work done Ops enables Dev to get work done Ops monitors Apps Ops provides Dev with tools to operate Apps Reactive monitoring/Ops Proactive monitoring/Ops Dev, Ops, Security and Product owners must work together throughout the SDLC and have a shared responsibility for the overall quality and reliability of the services
  8. 8. © 2013 Cloud Technology Partners, Inc. / Confidential 8 What needs to Change? Whoever prioritizes the backlog must be accountable for reliability and quality, not just speed to market Don’t be a crash test dummy Speed to market should not negatively impact customer satisfaction!
  9. 9. © 2013 Cloud Technology Partners, Inc. / Confidential 9 Logging Strategies
  10. 10. © 2013 Cloud Technology Partners, Inc. / Confidential 10 Top Log Use Cases – Troubleshooting – debugging information and error messages are collected for analyzing what is occurring in the production environment – Security – tracking all user access, both successful and unsuccessful access attempts. Intrusion detection leverages this information – Auditing – providing a trail of data for auditors is extremely important for audits. It is one thing to have a process flow on paper, it is another to show real data in the logs – Monitoring – identifying trends, anomalies, thresholds and other variables proactively allow companies to resolve issues before they become noticeable and/or critical to the end users Logging Strategies
  11. 11. © 2013 Cloud Technology Partners, Inc. / Confidential 11 Centralized Logs – Pipe logs to Sysout and direct to log services – Consider SaaS solutions so logging service does not go down with apps (e.g. Splunk) Best Practices – Block all developer access to servers – Direct developers to logging app instead – Standard log message codes and severity codes Logging Strategies
  12. 12. © 2013 Cloud Technology Partners, Inc. / Confidential 12 Without standards, Logs are “Garbage in, Garbage out” – Things to consider • Logs need to be easy to search • Logs must be easy to use or people won’t use them • External consumers of APIs expect standards – Standard codes • HTTP Status codes (200, 404, 503, etc.) • RFC 5424 Severity Levels – Standard Message Formats • Settle on a standard format • Build an API Logging Strategies Source Wikipedia http://en.wikipedia.org/wiki/Syslog#Severity_levels
  13. 13. © 2013 Cloud Technology Partners, Inc. / Confidential 13 Best Practices – Log Everything, Monitor Everything • Infrastructure logs • App Stack Logs (OS, app server, database, programming language) • API logs • Application logs • Security logs • Events, notifications, alerts • Changes, config mgmt., deployment • Access • Patching history, machine images What to collect
  14. 14. © 2013 Cloud Technology Partners, Inc. / Confidential 14 Common Logging Solutions Open Source Commercial
  15. 15. © 2013 Cloud Technology Partners, Inc. / Confidential 15 Monitoring Strategies
  16. 16. © 2013 Cloud Technology Partners, Inc. / Confidential 16 Nagios is not a Monitoring Strategy Blind spots can kill you
  17. 17. © 2013 Cloud Technology Partners, Inc. / Confidential 17 What needs to be Monitored? Data Category Description Performance Page loads, query times, response times, upload/download speeds, etc. Capacity Disk space, memory, CPU, bandwidth, etc. Uptime Availability (e.g.. Four 9’s) Throughput Every layer (web, cache, database, network, app stack, etc.) SLAs Availability, reliability, security, etc. KPIs Examples: Revenue per minute, Avg concurrent users, etc. User Metrics Registrations, page views, bounce rates, click rates, etc. Governance/Compliance Access, permissions, intrusion detection, intrusion prevention, cost containment, etc. Log file analysis Predictive analytics, pattern recognition, etc.
  18. 18. © 2013 Cloud Technology Partners, Inc. / Confidential 18 End to end Monitoring is Required There is no ONE tool that does it all Application Presentation Session Transport Network Data Link Physical Infrastructure Monitoring User Metrics, KPIs Web, Browser Metrics Sessions, Transactions App Svr, Database, Cache Packets, Access, Data Transfer Bandwidth, Trace routes, Requests CPU, Memory, Disk
  19. 19. © 2013 Cloud Technology Partners, Inc. / Confidential 19 Who needs Monitoring/Logging Data? Actor Purpose Product Manager Owns Features, reliability, and quality of product Developers Trace transactions, understand performance/bottlenecks, troubleshoot issues Testers Performance and regression testing, requirements traceability for the “ilities” Operations Support infrastructure NOC and Help Desk First level support and customer support Business Stakeholders Manage key business metrics, understand user behavior, forecasting, profitability Deployment team Validate deployment, ensure no negative impact of deployments Security team Enforcement of policies, intrusion detection & prevention Compliance team SLA Management, auditing, customer requests for information Customers/Users Account information, real time billing, application specific metrics
  20. 20. © 2013 Cloud Technology Partners, Inc. / Confidential 20 Synthesized Production Data and Monitoring Production data that is artificially created to simulate real users within a system in order to test and monitor system features, performance, reliability, and/or scalability What is Synthetic Data? Example Use Cases: 1. Test customer in a live production environment 2. Test user ID in a live production account 3. Netflix’s Simian Army (Purposely creating failures to test resiliency)
  21. 21. © 2013 Cloud Technology Partners, Inc. / Confidential 21 Think ahead: Create strategies for logging & monitoring – Log and monitor everything – Create standards to prevent “Garbage in Garbage out” in your logs – Put both reactive and proactive monitors in place – Know what your baseline metrics are and raise alerts when they change – Be prepared before auditors walk in the door – Make sure everyone is accountable for reliability and quality Summary
  22. 22. © 2013 Cloud Technology Partners, Inc. / Confidential 22 Thank you for your time and interest.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×