From the TeamQuest live webinar "Dashboards Don't Work - Unless You Have a Metrics Management Strategy".
Given the wide variety of measurement challenges found in IT, not many have taken a step back to assess the big picture and formulate an end-to-end strategy for metrics. Instead, the easy route is to find a dashboard tool, sometimes the incumbent enterprise ‘standard’, to fit each localized need. When the tool is force-fit to a specialized set of needs, the results are predictable- the dashboards at some point cease to be useful.
In this webinar, you will learn how to create an effective Business Value Dashboard. Topics covered include:
1. How to communicate IT’s business value to executives utilizing metrics.
2. What metrics matter to business leaders.
3. How to develop valuable strategic insights from operations data.
4. The most effective methods of dashboard visualization to simplify interpretation.
Live Q & A will follow the presentation.
Your presenter, Rich Razon, has over thirty years of experience in IT Service Management and has overseen the deployment of transformative IT metrics solutions in some of the largest and most complex IT organizations in the world. Prior to TeamQuest, Rich co-founded PureShare, a company focused on real-time and business-value metrics for IT.
To watch the on demand version of this live webinar, visit:
http://www.teamquest.com/en/resources/resource_center/videos/webinar-dashboards-dont-work/
2. Learn how to put metrics to work
in making dashboards as useful
and effective as possible.
Our Objective
3. How to Communicate IT’s Value to Execs through Metrics
Optimizing Communications through Visualization
Transforming Operations Metrics into Business Value
Today’s Agenda
6. As you go higher in the organization, the scope of metrics expands and the
need for summarization increases
Metrics empower all levels of management
Teams focus on operations
7. As you go higher in the organization, the scope of metrics expands and the
need for summarization increases
Metrics empower all levels of management
Teams focus on operations
Senior managers provide leadership,
direction, and oversight
8. Metrics empower all levels of management
Teams focus on operations
Senior managers provide leadership,
direction, and oversight
Executives concentrate on decisions that
shape the company
Each level has a unique profile
Tailored and interconnected metrics
As you go higher in the organization, the scope of metrics expands and the
need for summarization increases
10. = Key metrics that develop penetrating understanding
Must be valuable to the user, worthy of attention
“a bite-sized chunk of knowledge” for rapid review
Types include any combination of:
Inform – Educate, provide context and/or talking points
Awareness – Make aware of current status, recent events
Progress – Show trending vs. baselines
Results – Metrics showing results vs. targets & objectives
Anticipate – Leading indicators, contributing factors
What are strategic insights?
12. New insights from combining metrics
Added areas make framework richer
Other Internal
Supporting
Areas
Industry
Benchmarks
Strategic
Insights
Layer other areas
Marketing, Sales, Finance, HR, etc.
Intertwine results with the business
Industry metrics as baseline
comparison
Market share, cust sat, profitability,
efficiency, etc.
Strategic Insights
Internal
Core Areas
Manuf.Int’l
28. 5 Executive Metrics and Reporting Tenets: Clarity
Executive view:
Answers the question ‘Is there anything I
should focus on right now?’
HOW ARE THINGS TODAY?
33. Overall IT Performance
Too Low
Medium
Real Good
2016 Target
FY16 Productivity
Targets exceeded as a
result of certification
program, we all get a
bonus
5 Executive Metrics and Reporting Tenets: Context
36. 5 Executive Metrics and Reporting Tenets
Transparency
Open access to performance measures that apply to all organizational levels, top-to-bottom.
Automation
Eliminate manual handling and consolidation where possible to reduce errors and report latency.
Automation builds trust and confidence.
Clarity
Unambiguous as to what metrics are, what they mean, who they are for and why they are important.
Context
Answers the question ‘So what?’
Continual Improvement
Metrics and performance management is not a ‘project’ with a finite end. It is a process that will
continue as the business evolves.
37. 5 Executive Metrics and Reporting Tenets
Transparency
Open access to performance measures that apply to all organizational levels, top-to-bottom.
Automation
Eliminate manual handling and consolidation where possible to reduce errors and report latency.
Automation builds trust and confidence.
Clarity
Unambiguous as to what metrics are, what they mean, who they are for and why they are important.
Context
Answers the question ‘So what?’
Continual Improvement
Metrics and performance management is not a ‘project’ with a finite end. It is a process that will
continue as the business evolves.
38. How to Communicate IT’s Value to Execs through Metrics
Optimizing Communications through Visualization
Transforming Operations Metrics into Business Value
Today’s Agenda
39. Graphic Visualization IQ Test
Pick the graphic that you think is easier to understand: A or B
B
A
41. A B
Studies have found that it is more difficult for people to judge differences in area, such
as slices of pie, than it is to judge differences in length such as the length of bars.
45. A B
The grid, fill colors, unnecessary precision, and redundant use of the dollar
signs in the table A all distract from the data and make it unnecessarily
difficult to read and compare values.
47. A
B
Bars work very well for comparing
differences in magnitude, but when
you need to see the shape of change
over time, nothing is more effective
that a line.
51. A B
To use Map B, you’re required to memorize the meanings of seven colors (which
exceeds the limits of short term memory) or constantly refer to the legend. With Map
A, you can remember the meanings of red and blue, and the rest is intuitive.
53. A
B
This isn’t a video game! 3-D effects make
graphs harder to read and can hide some of
the values altogether, through occlusion.
The small ‘multiples’ display allows for easy
comparison of all of the bars.
57. A
B
Look at the graphs
before answering!
Bright colors are
important for making
things stand out but
when they’re overused
nothing stands out
and it becomes more
difficult to focus on
the data.
59. A
B
Color should be
used sparingly.
Overuse of color is
distracting.
Additionally, when
everything is
colored, the most
important things
don’t jump out as
they should.
60. Now here is your chance to be a dashboard critic!
61.
62. • How to Communicate IT’s Value to Execs through Metrics
• Optimizing Communications through Visualization
• Transforming Operations Metrics into Business Value
Today’s Agenda
63. Business Value
Metrics
Leadership
Eliminate below-the-line information in
above-the-line communication,
but maintain continuity in views so that
below-the-line details are always accessible
when needed.
‘The Line’
Above-the-Line Metrics
Below-the-Line Metrics
Operations Metrics
Management
Operations
64. Operations Metrics
Tool-level
Process metrics
Technical/Operations-level focus
Business Value
Metrics
Align priorities and investment with business strategy
Provide transparency of operation and cost
Showcase business value
BV versus Operations Metrics
71. Core Business
Metrics
Business
Dimensions
GRC
Governance,
Risk Management
and Compliance
ITSM
IT Service Mgmt
Dash Template
ITOM
IT Operations
Management
ITAM
IT Asset
Management
SIAM
Service Integration
and Management
SIEM
Security Information
and Event Mgmt
CCM
Contact Center
Metrics Template
BVD
Business Value
Dashboards for IT
Operations Metrics
Business Value
Metrics
Operations
Management
Leadership
73. Business Value Metrics and Dashboards:
1. Eliminate your leadership’s need for to
constantly justify budget
2. Quantify how people, process and
technology help business
3. Build business agility
4. Become operationally mandatory
74. Starting down the BVD path? Here are the steps:
1. Get support
Obtain key leadership support for an incremental but
sweeping metrics initiative.
Leverage controversial metrics if needed to stimulate
conversation with leadership.
P1 Cost per Minute = $2,666,666
75. The Path to Business Value Dashboards
1. Get support
2. Discover pressing, unanswered ‘above-the-line’ questions
Collaborate with leadership to find pressing
questions BVDs can answer, including finding
the proper way to present answers.
How much has IT investment added
to the business bottom line?
76. The Path to Business Value Dashboards
1. Get support
2. Discover pressing, unanswered ‘above-the-line’ questions
3. Catalog the process required to derive business-value metrics
+ = BVD
77. The Path to Business Value Dashboards
1. Get support
2. Discover pressing, unanswered ‘above-the-line’ questions
3. Catalog the process required to derive business-value metrics
4. Test and validate. Your audience knows its metrics and sound
numbers boost credibility
78. The Path to Business Value Dashboards
1. Get support
2. Discover pressing, unanswered ‘above-the-line’ questions
3. Catalog the process required to derive business-value metrics
4. Test and validate. Your audience knows its metrics and sound
numbers boost credibility.
5. Incrementally expand metrics footprint, automate
where possible
79. People Processes Technology Resources
Organization, Roles,
Culture, Skills, Training
Metrics
Focus, Standards,
Integration, Metrics
Standards, Efficiency,
Agility, Service Quality,
Tools, Metrics
Planning, Financial
Management, Metrics,
Governance, Sourcing,
Project Management
Advancing Organizational Maturity through Metrics
80. 1. Metrics can significantly impact business outcomes
2. Dashboards don’t work (without a metrics strategy)
3. Access to answers should be effortless!
In Summary:
81. Connect IT to the business
value it delivers
Automated capacity
managementintelligence
Analyze real-time and
historical IT performance
Vityl Suite
Editor's Notes
Typical Scenario- need some visibilty into some aspect of operations- answer used to be build a report, now its build a dashboard. The proper answer is to ask
what is the objective,
which metrics are appropriate,
where do they come from,
how do I get them,
how often do I get the metrics,
what is the context needed to make them meaningful,
what is a good range,
what is a bad range,
what is the target,
what is the stretch target,
how do we make access to the metrics effortless?
how do we leverage automation that the metrics are automatically monitored?
Dashboards Don’t Work: Unless You Have a Metrics Management Strategy
Given the wide variety of measurement challenges found in IT, not many have taken a step back to assess the big picture and formulate an end-to-end strategy for metrics. Instead, the easy route is to find a dashboard tool, sometimes the incumbent enterprise ‘standard’, to fit each localized need. When the tool is force-fit to a specialized set of needs, the results are predictable- the dashboards at some point cease to be useful.In this webinar, you will learn how to create an effective Business Value Dashboard. Topics covered include:• How to communicate IT’s business value to executives utilizing metrics.• What metrics matter to business leaders.• How to develop valuable strategic insights from operations data.• The most effective methods of dashboard visualization to simplify interpretation.
Dashboards Don’t Work: Unless You Have a Metrics Management Strategy
Given the wide variety of measurement challenges found in IT, not many have taken a step back to assess the big picture and formulate an end-to-end strategy for metrics. Instead, the easy route is to find a dashboard tool, sometimes the incumbent enterprise ‘standard’, to fit each localized need. When the tool is force-fit to a specialized set of needs, the results are predictable- the dashboards at some point cease to be useful.In this webinar, you will learn how to create an effective Business Value Dashboard. Topics covered include:• How to communicate IT’s business value to executives utilizing metrics.• What metrics matter to business leaders.• How to develop valuable strategic insights from operations data.• The most effective methods of dashboard visualization to simplify interpretation.
-How to communicate IT’s business value to executives utilizing metrics.-What metrics matter to business leaders.-How to develop valuable strategic insights from operations data.-The most effective methods of dashboard visualization to simplify interpretation.
Dashboards Don’t Work: Unless You Have a Metrics Management Strategy
Given the wide variety of measurement challenges found in IT, not many have taken a step back to assess the big picture and formulate an end-to-end strategy for metrics. Instead, the easy route is to find a dashboard tool, sometimes the incumbent enterprise ‘standard’, to fit each localized need. When the tool is force-fit to a specialized set of needs, the results are predictable- the dashboards at some point cease to be useful.In this webinar, you will learn how to create an effective Business Value Dashboard. Topics covered include:• How to communicate IT’s business value to executives utilizing metrics.• What metrics matter to business leaders.• How to develop valuable strategic insights from operations data.• The most effective methods of dashboard visualization to simplify interpretation.
Profitability – Revenue per Employee
Efficiency – PUE Metrics
Dashboards Don’t Work: Unless You Have a Metrics Management Strategy
Given the wide variety of measurement challenges found in IT, not many have taken a step back to assess the big picture and formulate an end-to-end strategy for metrics. Instead, the easy route is to find a dashboard tool, sometimes the incumbent enterprise ‘standard’, to fit each localized need. When the tool is force-fit to a specialized set of needs, the results are predictable- the dashboards at some point cease to be useful.In this webinar, you will learn how to create an effective Business Value Dashboard. Topics covered include:• How to communicate IT’s business value to executives utilizing metrics.• What metrics matter to business leaders.• How to develop valuable strategic insights from operations data.• The most effective methods of dashboard visualization to simplify interpretation.
Here is an example of the result of the idea presented in the previous slide – (explain)
Let’s take a dive into the details behind this to show how we get here
This graph (best shown building up, one metric at a time) shows a drop in risk scores (metric 1) even if release volume (metric 2) is trending up. More significantly, the number of emergency changes (metric 3) is also dropping. When put in the context of organizational maturity self-assessment (metric 4), it indicates that the initiatives to improve processes (including deployment of Adviser which precipitated the drop in risk scores to begin with) are surely working and can be quantified by these metrics.
This graph (best shown building up, one metric at a time) shows a drop in risk scores (metric 1) even if release volume (metric 2) is trending up. More significantly, the number of emergency changes (metric 3) is also dropping. When put in the context of organizational maturity self-assessment (metric 4), it indicates that the initiatives to improve processes (including deployment of Adviser which precipitated the drop in risk scores to begin with) are surely working and can be quantified by these metrics.
This graph (best shown building up, one metric at a time) shows a drop in risk scores (metric 1) even if release volume (metric 2) is trending up. More significantly, the number of emergency changes (metric 3) is also dropping. When put in the context of organizational maturity self-assessment (metric 4), it indicates that the initiatives to improve processes (including deployment of Adviser which precipitated the drop in risk scores to begin with) are surely working and can be quantified by these metrics.
This graph (best shown building up, one metric at a time) shows a drop in risk scores (metric 1) even if release volume (metric 2) is trending up. More significantly, the number of emergency changes (metric 3) is also dropping. When put in the context of organizational maturity self-assessment (metric 4), it indicates that the initiatives to improve processes (including deployment of Adviser which precipitated the drop in risk scores to begin with) are surely working and can be quantified by these metrics.
This is an example of a dashboard that provides a consolidated view of IT Metrics. The bottom half shows operations level (below the line) metrics vs the business (above-the line metrics) appearing on the top half. The graphs and table on the bottom half shows how well various processes within IT are working.
This dashboard aggregates metrics from multiple systems and data sources to provide a consolidated view of what is going on. This provides the unique perspective of being able to see how improvements in IT processes impact the business’ bottom line.
Metrics in the lower left quadrant highlight business-affecting priority 1 incidents, rolled up to each line of business. This level of summarization makes the metrics of interest to business stakeholders. The Metrics Catalog is leverage to input organizational info to the system.
The lower center graph indicates the proportion of capacity–related P1s in comparison to all other P1 incidents. There is a general decrease even as release volume (red line) is increasing. Seeing the release volume, major incidents and capacity-related incidents tells a story that IT is improving its ability to cope with increasing workloads while improving quality.
The graph on the lower right (best shown building up, one metric at a time) Shows a drop in risk scores (presumable since Adviser was installed) even if release volume is trending up. More significantly, the number of emergency changes is also dropping. When put in the context of organizational maturity self-assessment, it indicates that the initiatives to improve processes (including deployment of Adviser) are surely working and can be quantified by these metrics.
If you put out a dashboard, it better be consistent with this the idea that every executive yearns for organizational transparency. This is driven by the idea of of shared success and as well, situational awareness for all which results in organizational agility.
This is driven by the idea of of shared success and as well, situational awareness for all which results in organizational agility.
Automate where possible
Level of metrics resolution or summarization matches the organizational level of the audience. Again, it needs to be perfectly clear what the metric is, who it is for, what is a bad indication, what is a good indication.
How to communicate IT’s business value to executives utilizing metrics.• What metrics matter to business leaders.• How to develop valuable strategic insights from operations data.• The most effective methods of dashboard visualization to simplify interpretation.
How to communicate IT’s business value to executives utilizing metrics.• What metrics matter to business leaders.• How to develop valuable strategic insights from operations data.• The most effective methods of dashboard visualization to simplify interpretation.
Call attention to P1s on previous slide rolled up by business
Consolidated view
Summaries by business hierarchy
Typically this will involve IT leadership as well as a sponsor in a line-of business.
Publishing controversial metrics is also an effective technique to start a conversation with business stakeholders.
Typically this will involve IT leadership as well as a sponsor in a line-of business.
Publishing controversial metrics is also an effective technique to start a conversation with business stakeholders.
If I accelerate project X to Q1, what is the impact on capacity.. And cost?
If I use the 300M for project X and use on project y, what is impact?
This exercise involves:
Identifying the data and metrics involved in creating a composite ‘answer’
Identifying the process by which the data is collected, filtered and manipulated.
Identifying the calculations used to derive the ‘answer’
Create a catalog that documents every business value metric and every attribute. This includes dimensions associated with metrics, calculations, filtering and analytics and how the metrics are derived, who they are for, why they are important, what is bad vs good.
In many cases it can be as simple as dividing a technical metric (critical system outage minutes) by a business metric (orders per hour) to create a business value metric
If you live and die by metrics, some level of assurance needs to be associated with the process of generating business value metrics. Introduce the idea of ‘data assurance metrics’ (sort of a mini MDM)
Data assurance metrics are simply metrics that indicate a level of data integrity (it will never be perfect) ex – how many records exist where an entry does not appear on a field when status of a record is at a certain level. Or how many times a customer record appears in different ways.
The idea is data will never be perfect but it will be good to gain an idea of the level of ‘imperpectness’ and and ieda of whether is getting better or getting worse.
Errors injected into data by way of a bad process usually is reflected in metrics- by then it is too late.
Replicate the small successes, incrementally expand
Almost everyone starts with a search for a ‘dashboard’. What they are really looking for is an end-to-end metrics management strategy. With it, the means to measure can evolve with the business. Without it, dashboards are short-lived and unmaintainable. Consider this end-to-end approach when looking for tooling. Consider what you put on an end user’s desktop or mobile device (do they really want to cube and pivot data? 99% of people in any average busy organization just want answers!)
New product suite will deliver a single tool to help organizations adopt to the speed of change and increase maturity
Define the problems (SLA, meeting new demand, understanding value, controlling costs)
Best practices to resolve them (broad and deep data collection, algorithms, adaptive)
Focus on the benefits of tying operational metrics to business metrics
Important considerations: workflow, collaboration, customized views for stake holders