• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Why meaning matters - Outcomes, Benefits, CSFs, KPIs, Metrics and Measures
 

Why meaning matters - Outcomes, Benefits, CSFs, KPIs, Metrics and Measures

on

  • 887 views

Slide deck for BrightTalk webinar September 11, 2013

Slide deck for BrightTalk webinar September 11, 2013

Statistics

Views

Total Views
887
Views on SlideShare
885
Embed Views
2

Actions

Likes
1
Downloads
0
Comments
0

1 Embed 2

http://t.co 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Why meaning matters - Outcomes, Benefits, CSFs, KPIs, Metrics and Measures Why meaning matters - Outcomes, Benefits, CSFs, KPIs, Metrics and Measures Presentation Transcript

    • BSS Nexus Global © 2013April 2013 Why Meaning Matters: CSFs, KPIs, Metrics, Outcomes and Benefits Larry Cooper Senior Partner, BSSNexus
    • BSS Nexus Global © 2013April 2013 Why meaning matters... “There’s a difference between... knowing you’re sh*t and... knowing your sh*t”
    • BSS Nexus Global © 2013April 2013 Meaning • Noun 1. word meaning - the accepted meaning of a word  the meaning of a word or expression; the way in which a word or expression or situation can be interpreted; "the dictionary gave several senses for the word"; "in the best sense charity is really a duty"; "the signifier is linked to the signified"
    • BSS Nexus Global © 2013April 2013 Agenda Performance Reference Model Outputs and Outcomes Value versus Benefits Critical Success Factors Key Performance Indicators Metrics and Measures
    • BSS Nexus Global © 2013April 2013 Performance Reference Model (PRM) Effectiveness Modified version of - http://agict.gov.au/policy-guides-procurement/australian-government-architecture-aga/aga-rm/2-reference-model-overviewnce Fixed Assets Efficiency Technology Money People Data and Information Projects Ad-hoc Tasks Processes Products Services Service Outcomes Business Outcomes INPUTS WORK OUTPUTS OUTCOMES Facilities Where I am Where I want to go
    • BSS Nexus Global © 2013April 2013 Measurement Domains • The Inputs domain contains fixed assets, technology, people, data and information, and finances (the types of input) • The Work domain contains ad hoc tasks, projects and processes and operations (business as usual) — the types of work • The Outputs domain contains Products and Services (the types of output) • The Outcomes domain contains Service Outcomes and Business Outcomes (the types of outcome) Modified version of - http://agict.gov.au/policy-guides-procurement/australian-government-architecture-aga/aga-rm/2-reference-model-overviewnce
    • BSS Nexus Global © 2013April 2013 Agenda Performance Reference Model Outputs and Outcomes Value versus Benefits Critical Success Factors Key Performance Indicators Metrics and Measures
    • BSS Nexus Global © 2013April 2013 Outputs vs. Outcomes • Outputs are important products, services, profits, and revenues: the What. • Outcomes create meanings, relationships, and differences: the Why. • Outputs, such as revenue and profit, enable us to fund outcomes; but without outcomes, there is no need for outputs. See more at: http://www.innovationexcellence.com/blog/2013/04/02/its-not-just-semantics-managing-outcomes-vs- outputs/#sthash.ZB18O0YY.dpuf
    • BSS Nexus Global © 2013April 2013 What is an Outcome? • Outcome:  "something that follows as a result or consequence“ of some action(s)  An outcome involves an intentional change being imposed on the system (people, processes, technology), with a resulting end state that can be measured • Outcome Management:  the set of activities for the planning, managing, and realizing of the desired outcomes from initiatives • Synonyms for Outcome:  Aftermath, Consequence, Results International Organization for Collaborative Outcome Management http://www.iocomsa.org/node/6 • Answers the question “what difference does it make?”
    • BSS Nexus Global © 2013April 2013 Origin of Outcomes (Management) • The use of the term to describe the planned consequences from activity originated with Florence Nightingale who introduced outcome indicators to healthcare • Later used in “outcomes-based education” and in program management “logic models” • Fundamentally, it is a quality management approach focused on the “customer”  Ian Clayton’s USMBOK® calls this an “outside-in” approach Patient Care Process Outcomes Care Team http://books.google.ca/books?id=a8RftMjbxcQC&pg=PA20&lpg=PA20&dq=%22history+of+outcome+management%22&source= bl&ots=brzKFAXusw&sig=KG6MMhW3krR4LcHB6sYaFNhAaV8&hl=en&sa=X&ei=TVIjUq26FIa5igKe5oCoBQ&ved=0CDsQ6AEwAQ #v=onepage&q=%22history%20of%20outcome%20management%22&f=false
    • BSS Nexus Global © 2013April 2013 PRM to Logic Model Inputs Activities Outputs Outcomes Goals Certain resources and people are needed to do your work IF you have adequate resources and people, THEN you can use them to accomplish your planned activities IF you accomplish your planned activities, THEN you will deliver a certain amount of product and/or services IF you accomplish your planned activities to the extent you intended, THEN your customers will benefit in certain ways IF these benefits to customers are achieved, THEN certain changes in people, organizations, or systems would be expected to occur Benefits
    • BSS Nexus Global © 2013April 2013 Why Outcomes Management? Not how many worms the bird feeds its young, but how well the fledgling flies If we start with the desired ends (i.e. outcomes) in mind Then we can work backwards towards the means to achieve them Inputs Activities Outputs Outcomes Goals Certain resources and people are needed to do your work IF you have adequate resources and people, THEN you can use them to accomplish your planned activities IF you accomplish your planned activities, THEN you will deliver a certain amount of product and/or services IF you accomplish your planned activities to the extent you intended, THEN your customers will benefit in certain ways IF these benefits to customers are achieved, THEN certain changes in people, organizations, or systems would be expected to occur Benefits
    • BSS Nexus Global © 2013April 2013 Outcomes Questions 1. What is the current situation that we want to affect? 2. What will it look like when we achieve the desired solution or outcome? 3. What behaviours need to change for that outcome to be achieved? 4. What knowledge or skills do people need before the behaviour will change? 5. What activities need to be performed to cause the necessary learning? 6. What resources and people will be required to achieve the desired outcome?
    • BSS Nexus Global © 2013April 2013 Incident Management (IM) Example • Goal Statement:  Minimize the effect of outages by restoring service as quickly as possible so that we can maintain our customers • Outcome(s) Statements:  Business Outcome: Minimize Incident Outage Times…by improving the accuracy and completeness of Incident resolution details to create an Incident Knowledge Base thereby increasing the first point of contact (FPOC) resolution rates  Learning Outcome: Improve Service Desk and Support Staff knowledge of how to accurately and completely record Incident Resolution details for use in resolving future Incidents of this type  To achieve business outcomes we always start with people…
    • BSS Nexus Global © 2013April 2013 POro Process Model/Tools Measures Metrics Outcomes KPIs Goal CSFs Business Outcome: Minimize Incident Outage Times Minimize the effect of outages by restoring service as quickly as possible so that we can maintain our customers Learning Outcome: Improve Service Desk and Support Staff knowledge Benefits
    • BSS Nexus Global © 2013April 2013 Agenda Performance Reference Model Outputs and Outcomes Value versus Benefits Critical Success Factors Key Performance Indicators Metrics and Measures Processes and Tooling
    • BSS Nexus Global © 2013April 2013 Value and Benefits • Benefits are measurable improvements perceived to be of value by one or more of the stakeholders (Rajegopal, McGuin, & Waller, 2007, p. 209; Venning, 2007) • Bannister (2001) made a distinction between value and benefit in the following way:  Value is what we perceive; benefit is what we receive  Benefits can be thought of as an operationalization of the value construct • Benefits can be tangible or intangible  Tangible benefits are often classified as financial or non-financial • The benefits realizaiton approach: ‘The Information Paradox’, John Thorpe, 1997 – precursor to ValIT from ISSACA
    • BSS Nexus Global © 2013April 2013 Benefits Statement Qualifiers • Increased • Enhanced • Strengthened • Improved • Reduced • Lowered • Eliminated • Maximized • Minimized • Avoided • …. Implies a directional change
    • BSS Nexus Global © 2013April 2013 IM Example • Outcome:  Minimize Incident Outage Times • Perceived customer/client value:  IT Support Staff are more effective in providing a stable operating environment • Benefits:  Increased Customer Satisfaction (Non-tangible)  Reduced costs of Incidents (Tangible – Financial)  Improved Availability of Business Systems and Applications (Tangible – Financial)
    • BSS Nexus Global © 2013April 2013 POro Process Model/Tools Measures Metrics Outcomes KPIs Goal CSFs Business Outcome: Minimize Incident Outage Times Minimize the effect of outages by restoring service as quickly as possible so that we can maintain our customers Learning Outcome: Improve Service Desk and Support Staff knowledge Benefits Increased Customer Satisfaction Reduced costs of Incidents Improved Availability of Business Systems and Applications
    • BSS Nexus Global © 2013April 2013 Agenda Performance Reference Model Outputs and Outcomes Value versus Benefits Critical Success Factors Key Performance Indicators Metrics and Measures Processes and Tooling
    • BSS Nexus Global © 2013April 2013 Critical Success Factors • Earliest reference: 10 problems that worry presidents. By: Spencer, Lyle M.. Harvard Business Review, Nov/Dec55, Vol. 33 Issue 6, p75-83, 9p  “What are the essential factors that produce success in my company?” • RH Daniel, 1961, HBR article identified need for “success factors” that enable the success of the business – he said there should be at most 3 to 6 such factors to focus on • Expanded by Rockhart, 1979 to “the limited number of areas in which satisfactory results will ensure successful competitive performance for the individual, department or organization. CSFs are the few key areas where ‘things must go right' for the business to flourish and for a manager's goals to be attained”
    • BSS Nexus Global © 2013April 2013 CSF Definition (Rockhart 81) • key areas of activity in which favorable results are absolutely necessary to reach goals • key areas where things must go right for the business to flourish • “factors” that are “critical” to the “success” of the organization • key areas of activities that should receive constant and careful attention from management • a relatively small number of truly important matters on which a manager should focus attention
    • BSS Nexus Global © 2013April 2013 CSF Hierarchy Industry CSFs Organizational CSFs Division-level CSFs Operational Unit-level CSFs Individual CSFs Influence Drive Drive Are supported by
    • BSS Nexus Global © 2013April 2013 Other CSF Influencers CSFs Temporal Environmental The Industry Managerial Position Competitive Strategy
    • BSS Nexus Global © 2013April 2013 Some Sample CSFs? • ITIL® book(s) examples of CSFs:  A good service desk  Integrated support tools to drive and control the process  Manage availability and reliability of IT service • Recent TSO book on Metrics – CSF examples:  Incident management plans, policies  Maintaining customer satisfaction  Resolving incidents within defined timelines  Documentation of suggestions to other processes • In both cases, they did not establish clear linkages to “the Why” question
    • BSS Nexus Global © 2013April 2013 IM Example • Outcome:  Minimize Incident Outage Times • Perceived customer/client value:  IT Support Staff effectiveness in providing a stable operating environment • Benefits:  Increased Customer Satisfaction (Non-tangible)  Reduced costs of Incidents (Tangible – Financial)  Improved Availability of Business Systems and Applications (Tangible – Financial) • CSFs:  IT Service Desk and Support staff trained and knowledgeable in how to accurately and completely record Incident Resolution details for use in resolving future Incidents of this type  Service Desk tool satisfies defined technical and process capabilities for Incident Management and integrated Knowledge Management and is accessible to both Service Desk and other levels of support as needed
    • BSS Nexus Global © 2013April 2013 POro Process Model/Tools Measures Metrics Outcomes KPIs Goal CSFs Business Outcome: Minimize Incident Outage Times Minimize the effect of outages by restoring service as quickly as possible so that we can maintain our customers IT Service Desk and Support staff trained and knowledgeable Learning Outcome: Improve Service Desk and Support Staff knowledge Benefits Increased Customer Satisfaction Reduced costs of Incidents Improved Availability of Business Systems and Applications Service Desk Tool satisfies business and technical capabilities Knowledge base For Incident Management in place
    • BSS Nexus Global © 2013April 2013 Agenda Performance Reference Model Outputs and Outcomes Value versus Benefits Critical Success Factors Key Performance Indicators Metrics and Measures Processes and Tooling
    • BSS Nexus Global © 2013April 2013 Key Performance Indicators • First known reference: BusinessWeek article “Corporate ‘War Rooms’ Plug Into the Computer” 1976 • Rockart, 1979 described a key indicator system based on three concepts: 1. The selection of key indicators to represent the health of the organization 2. Exception reporting or, in other words, the ability to present only those indicators where performance was considerably different from expected results 3. The visual display of that information • A KPI has a lifetime and requires continuous updating. Sometimes, its replacement is also needed (Ghalayini & Noble, 1996) • A KPI is a key part of a measurable objective, which is made up of a direction, the KPI statement, benchmark or target and a timeframe KPIs may need to change over time
    • BSS Nexus Global © 2013April 2013 KPI Types • Lagging Indicator  Measures focusing on results at the end of a time period, normally characterizing historical performance. Also referred to as Key Results Indicator (KRI). The importance of a lagging indicator is its ability to confirm that a pattern is occurring or about to occur. • Leading Indicators:  Measures that “drive” or lead to the performance of lag measures; normally measuring intermediate processes and activities. These types of indicators signal future events. Think of how the amber traffic light indicates the coming of the red light. • Leading indicators are activities that should be trended as they predict the outcomes (i.e. lagging indicators). Quotas or goals should only be placed on lagging indicators and never on leading indicators. Placing a goal on a leading indicator will result in gaming and generate the wrong results.
    • BSS Nexus Global © 2013April 2013 Leading versus Lagging Indicators Leading Lagging Advantages • Predictive nature allowing adjustments based on results so far • Normally easy to identify and capture Challenges • May prove difficult to identify and capture • Historical in nature and not reflective of current activities • Lacks predictive power IM Examples • % of Incidents solved at FPOC • % of Severity 1 incidents not worked on in last four 2 hour blocks • % of incidents dispatched more than 2 times over each of previous five business days • Average backlog of incidents per agent over each of previous five business days • % of Incidents at Severity 1 in last 30 days • % of calls that are Incidents in last quarter • % of open incidents older than 1 day by Severity level http://www.icmi.com/files/ICMILeading_LaggingIndicatorsExplained.pdf
    • BSS Nexus Global © 2013April 2013 IM Example • CSFs: 1. IT Service Desk and Support staff trained and knowledgeable in how to accurately and completely record Incident Resolution details for use in resolving future Incidents of this type 2. Service Desk tool satisfies defined technical and process capabilities for Incident Management and integrated Knowledge Management and is accessible to both Service Desk and other levels of support as needed • KPIs: 1. % of Incidents resolved at First Point of Contact in each of previous 5 reporting periods 2. % of Incidents where prior Incident Resolution details were used to resolve Incident by Level 2 support 3. >70% of Incidents solved at FPOC by end of first year
    • BSS Nexus Global © 2013April 2013 POro Process Model/Tools Measures Metrics Outcomes % of Severity 1 incidents not worked on in last four 2 hour blocks KPIs Goal CSFs >70% of Incidents solved at FPOC Business Outcome: Minimize Incident Outage Times Minimize the effect of outages by restoring service as quickly as possible so that we can maintain our customers >98% of Severity 1 Incidents meet SLA Targets IT Service Desk and Support staff trained and knowledgeable Learning Outcome: Improve Service Desk and Support Staff knowledge Benefits Increased Customer Satisfaction Reduced costs of Incidents Improved Availability of Business Systems and Applications 100% of SD staff trained on Incident Management Processes Service Desk Tool satisfies business and technical capabilities % of Incidents resolved at First Point of Contact in each of previous 5 reporting periods Knowledge base For Incident Management in place
    • BSS Nexus Global © 2013April 2013 Agenda Performance Reference Model Outputs and Outcomes Value versus Benefits Critical Success Factors Key Performance Indicators Metrics and Measures Processes and Tooling
    • BSS Nexus Global © 2013April 2013 Metrics and Measures http://www.metrum.org/measures/metrics.htm First references to measurement are from writings dating to 2900 BC Mesopotamia
    • BSS Nexus Global © 2013April 2013 Danger of “Industry Standards”
    • BSS Nexus Global © 2013April 2013 Metrics versus Measures • Measurement: • Process of experimentally obtaining one or more quantity values that can reasonably be attributed (e.g. # of Incidents) • Metric:  Describe a quality and require a measurement baseline (% of Incidents at Severity 1 as a result of a Change) http://samate.nist.gov/index.php/Metrics_and_Measures.html https://cio.gov/performance-metrics-and-measures/
    • BSS Nexus Global © 2013April 2013 House of Quality Metrics http://www.mit.edu/~hauser/Papers/Hauser-Katz%20Measure%2004-98.pdf
    • BSS Nexus Global © 2013April 2013 Good Metrics • Reinforce desired behaviour • Measure results (i.e. outcomes) – not just output or compliance • Measure trends • Support both leading and lagging indicators • Focus on the vital few • Need to be collectable • Fuel meaningful conversations with customers and employees • May measure value to the customer in the form of Benefits that signal Outcomes achievement or it can be related to the quality of a process • Are of good enough quality • May need to change as we get better
    • BSS Nexus Global © 2013April 2013 POro Process Model/Tools Measures Metrics Outcomes % of Severity 1 incidents not worked on in last four 2 hour blocks # of Level 2 support staff trained KPIs # of Severity 1 Incidents by day Goal % of Incidents at Severity 1 CSFs >70% of Incidents solved at FPOC Business Outcome: Minimize Incident Outage Times Minimize the effect of outages by restoring service as quickly as possible so that we can maintain our customers >98% of Severity 1 Incidents meet SLA Targets IT Service Desk and Support staff trained and knowledgeable Learning Outcome: Improve Service Desk and Support Staff knowledge Benefits Increased Customer Satisfaction Reduced costs of Incidents Improved Availability of Business Systems and Applications 100% of SD staff trained on Incident Management Processes Service Desk Tool satisfies business and technical capabilities % of Incidents resolved at First Point of Contact in each of previous 5 reporting periods % of open Incidents by date % of Service Desk Staff Trained # of Severity 1 Incidents solved at FPOC Knowledge base For Incident Management in place
    • BSS Nexus Global © 2013April 2013 Agenda Performance Reference Model Outputs and Outcomes Value versus Benefits Critical Success Factors Key Performance Indicators Metrics and Measures Processes and Tooling
    • BSS Nexus Global © 2013April 2013 Processes and Tooling • Our processes (the What) need to enable what we are trying to achieve (the Why) • Our tools need to enable our processes (the How) • IF we don’t the know the Why, THEN developing the What and the How are likely must less effective, and may in fact be moot
    • BSS Nexus Global © 2013April 2013 POro Process Model/Tools Measures Metrics Outcomes % of Severity 1 incidents not worked on in last four 2 hour blocks # of Level 2 support staff trained KPIs Knowledge Management Incident Management Escalation Model Service Desk Tool # of Severity 1 Incidents by day Goal % of Incidents at Severity 1 CSFs >70% of Incidents solved at FPOC Business Outcome: Minimize Incident Outage Times Minimize the effect of outages by restoring service as quickly as possible so that we can maintain our customers >98% of Severity 1 Incidents meet SLA Targets IT Service Desk and Support staff trained and knowledgeable Learning Outcome: Improve Service Desk and Support Staff knowledge Benefits Increased Customer Satisfaction Reduced costs of Incidents Improved Availability of Business Systems and Applications 100% of SD staff trained on Incident Management Processes Service Desk Tool satisfies business and technical capabilities % of Incidents resolved at First Point of Contact in each of previous 5 reporting periods % of open Incidents by date % of Service Desk Staff Trained # of Severity 1 Incidents solved at FPOC Knowledge base For Incident Management in place
    • BSS Nexus Global © 2013April 2013 From Measures to Outcomes? Measures Metrics KPIs CSFs Outcomes/ Benefits Data Collection Metrics Computation Metrics Evaluation Indicator Evaluation Outcomes Evaluation/ Benefits Realization Project view In-operations view
    • BSS Nexus Global © 2013April 2013 Outcomes
    • BSS Nexus Global © 2013April 2013 To get to good Outcomes 1. Listen to the voice of the customer (use “outside-in” thinking) 2. Understand the contributors to outcomes and what drives them (voice of the employee in the processes and work) 3. Understand the relationships (from Outcomes to Measures – and back) as well as the linkages between the customer and the employee (1 and 2) 4. Measure actual outcomes and benefits – not just compliance Adapted from http://www.mit.edu/~hauser/Papers/Hauser-Katz%20Measure%2004-98.pdf
    • BSS Nexus Global © 2013April 2013 BSSNexus Model
    • BSS Nexus Global © 2013April 2013 To summarize  We need to think “outside-in”  Outcomes are what we are trying to achieve as indicators of our goal  Value is what our customers perceive; benefit is what they receive  CSFs are the few key areas where ‘things must go right’  KPIs are simply metrics that are tied to a target  A KPI is a key part of a measurable objective, which is made up of a direction, the KPI statement, benchmark or target and a timeframe  Outcomes, CSFs, KPIs, Metrics and Measures may need to change over time!
    • BSS Nexus Global © 2013April 2013 The final word
    • BSS Nexus Global © 2013April 2013
    • BSS Nexus Global © 2013April 2013 For more information Larry.Cooper@BSSNexus.com Larry.Cooper@AthenaNexus.com ca.linkedin.com/lawrencekcooper Twitter: @coopelk99 Skype Id: Larry.Cooper +1 (613) 868-0982 (cell) +1 (888) 316-2745 BSSNexus Global Inc. AthenaNexus Inc.