Metrics Monitoring Is So Critical - What's Your Best Approach?

Metrics Monitoring is So Critical:
What’s Your Best Approach
Live Webcast, Sponsored by Wavefront
Rob Markovich – Chief Marketing Officer
Conor Beverland – Director, Applications Engineering
February 14, 2017

What’s Ahead for the Next 50 Minutes
I. The what and why of metrics monitoring
II. The comparative approaches to implementing a metrics platform
III. The Metrics Maturity Model – where you are, why it matters
IV. Scenarios / case studies – Intuit, Shooju, Medallia, Workday
V. Learning from cloud ops pioneers; checklist for decision process
VI. Q&A session
© 2017, Wavefront. All rights reserved. Wavefront Confidential. 2

About Wavefront
Your First Pane of Glass to Manage Cloud Applications at Scale.
Industry’s Most Advanced
Metrics-as-a-Service:
A Different Approach
for TechOps, SREs,
DevOps, Developers
Monitor Every Corner of Your
Cloud Application Environment
Inspired by web-scale
monitoring platforms at:
Get fast and deep answers
to how it’s all working in
production.
Leverage your existing
monitoring investment.CUSTOM & STANDARD METRICS
APPLICATIONS CLOUD TOOLS LOGS INFRASTRUCTURE
UNIFIED REAL-TIME METRIC STORE
ANALYTICS, ALERTS, DASHBOARDS
ANSWERSQUESTIONS

Data-Driven Companies Outperform
1. Data-driven culture is about more than Data Specialists
2. Top-performing companies have adopted a data-driven culture
3. Data-driven culture requires an executive champion
4. Data-driven companies democratize data
FROM IDG
Home > Software > Business Intelligence (BI)
FEATURE
Data-Driven Companies Outperform Competitors Financially
Companies that rate themselves substantially ahead of their piers in their use of data are three
times more likely to rate themselves as substantially ahead in financial performance, according to
findings from the Economist Intelligence Unit.
http://www.cio.com/article/2387460/business-intelligence/data-driven-companies-outperform-competitors-financially.html

Metrics Monitoring (& Platform) Explained
…all with Performance & Reliability at Scale
Analytics-Driven
ALERTING
Rich, Custom
VISUALIZATIONS
UNIFIED REAL-TIME METRIC STORE
ANALYTICS
The wide and continuous use of operational metrics across the organization to
reactively and proactively manage the health of a cloud application environment.
 See anomalies earlier
 Investigate iteratively, faster
 Democratize monitoring
Query-Powered
Automate
Proactive Monitoring

Relevant Diversion – Logs vs. Metrics
Metric and log entries both provide value, but they tell different stories. There’s a right tool for each to
evaluate the respective data.
 A log entry should give you information detail about
a specific event, whereas, a metric should just let you know
that an event occurred.
 The specificity that a log message allows is what makes it
valuable, but you only need that specificity when a
problem arises. In normal operation, log files contain
anomalies, not stacks of information telling you everything
is ok (ideally, separate the information types).
 Metrics can be incredibly smaller (than log messages)
because they convey considerably less information. They’re
also extremely easier to evaluate. This has real impact
around how metrics can be stored, processed and retained.
 Metrics are better suited to giving you a good idea on how
the system is performing, i.e. for overall monitoring of the
system (uptime, KPIs, etc.).
See also: “Logs vs. Metrics” allthingsdork.com

Metrics Monitoring – Why It’s So Critical
Business Value: Move Faster with Stable Infrastructure
The first pane of glass to
assess overall health,
reduce tool sprawl
Know the leading
indicators before the
”oops” moments
Enables the DevOps mantra:
collect everything,
monitor what matters
The best way to visualize
how dynamic / ephemeral
cloud environments operate
See things earlier that
other tools can’t see
(with analytics)
Democratize
monitoring
economically

Who Uses Metrics Monitoring
Developers
Tech Ops, SREs
DEV OPS
Platform / DevOps / Centralized
Tooling / Observability Teams
Self-Service Metrics

Potential Approaches to a Metrics Platform
Do Nothing
Do with an
APM
Tool
Do with an
Infrastructure Check
Tool
Do with a
Log Monitoring
Tool
Do in-house,
e.g. with an
open-source TSDB
Do with a
commercial
metrics service

Potential Approaches to a Metrics Platform
Do Nothing
Do with an
APM
Tool
Do with an
Tool
Do with a
Log Monitoring
Tool
Do in-house,
e.g. with an
open-source TSDB
Do with a
commercial
metrics service

The Metrics Maturity Model
RUNNING
FLYING
WALKING
CRAWLING
How advanced is your use of operational metrics to create business value? 20%
30%
40%
10%
Reactive: investigate, some detect.
Poor, ad-hoc.
Reactive: investigate, detect.
Reactive & proactive: investigate,
detect, validate, plan.
None.
None.
Minimal.
Not in use.
Static, domain-specific, growing levels
of alert noise.
Siloed.
None.
Some siloed.
Consolidated, sustaining alert noise,
escalation process for critical.
Metrics defined centrally, viewable by
others.
Realization of need and have started to
understand relationships and model.
Collaborative but irregular. Processes
started to get metrics and analyzed.
Consolidated, analytics-driven alerts
increase quality / reduces noise,
escalation process for critical & inform.
Self-service metrics across entire
organization.
Well understood relationships and
model – identified and monitor
computed metrics / KPIs.
Collaborative, regular, automated.
Effective knowledge sharing and
individual/team empowerment.
Metrics-driven decision making
System model understanding
Metrics democratization at scale
Alerting design & mgmt. at scale
Metrics use over monitoring cycle

The Metrics Maturity Model
0 1 2 3
RUNNING
FLYING
WALKING
CRAWLING
Companies across many verticals are already flying with metrics.

How to Self-Assess Your Metrics Maturity
1. How many engineers (software developers, operations engineers, platform engineers, etc.)?
2. Do you write application code or do you rely on 3rd party applications?
3. Does your application code directly make $ for your business?
4. DevOps/continuous delivery adoption – you live and breathe it as native tongue, or just starting?
5. Progression of developers taking on greater responsibility for code in production?
6. Cloud-native or running on your own data centers?
7. Do you send any operational data into the cloud?
8. Are there existing monitoring tools in use already, not much in place?
9. Messaging queue already in place?
10. How often is your “operational timescale”?
SoftwareProclivityDataProclivity

Mapping Approaches to the Maturity Model
0 1 2 3
RUNNING
FLYING
WALKING
CRAWLING
APM
Log Monitor
Open-Source TSDB
Entry Metrics Service
Advanced Metrics Service
Immediate value when you leap to fly!

Metrics Platform Case Study #1
Started with an in-house time-series database,
upgraded to Wavefront’s advanced metrics-as-a-service.

Started with an entry-level metrics-as-a-service,

Started with infrastructure check and log mon tools,

Was doing nothing with metrics monitoring,

Metrics Platform Decision Process
1. Document overall goals and needs.
2. Assess your current / desired position on the Metrics Maturity curve.
3. Determine strategy for advancement– e.g. progress serially or leap-frog.
4. Plan for instrumentation, tool integration / migration (if applicable).
5. Prepare your metrics platform evaluation checklist.
6. Test your ideas – e.g. hands on trial, or PoC.
7. Finish business value assessment (BVA) for best platform selection.

Metrics Platform Evaluation Checklist
 Functionality requirements
 Framework requirements
 Integration requirements
 Deployment requirements
 Performance requirements
 Vendor support requirements
 Value (vs. TCO) requirements

5 Lessons from Data-Driven CloudOps Pioneers
1. They foster a data-driven, analytics-
enabled culture across the organization.
2. They build models to determine what
operational metrics best track customer
experience and business effectiveness.
3. They gather metrics from all corners of
their environment.
4. They aggregate metrics to one place with
common analytics tools for all.
5. They use metrics-driven monitoring at
every stage of the operational continuum.

In Closing – Key Takeaways
I. Unified metrics monitoring with real-time analytics is critical
for DevOps teams to move faster with stable infrastructure.
II. Many organizations choose not to progress serially through
the Metrics Maturity Model – instead, they leap-frog and
leverage advanced tooling to move faster.
III. Only an advanced metrics service like Wavefront gives you:
(a) the analytics depth for alerting and anomaly detection,
(b) the flexibility breadth for self-service metrics, and
(c) the performance, reliability and usability at scale for an
enterprise-wide rollout.

Time for Q&A – Message Your Questions Now

Thank You For Attending!
For more information, visit wavefront.com
Try it Out Now – online demo and trial
Blogs and other consultative content
Industry and local events
Community pages

Metrics Monitoring Is So Critical - What's Your Best Approach?

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Metrics Monitoring Is So Critical - What's Your Best Approach?

Similar to Metrics Monitoring Is So Critical - What's Your Best Approach? (20)

Recently uploaded

Recently uploaded (20)

Metrics Monitoring Is So Critical - What's Your Best Approach?