SlideShare a Scribd company logo
Health care indicator selection processes:
How to select engaging metrics to motivate front-line managers.
Dr. Michael A. Heenan, PhD
McMaster University, eHealth Summer Program
PhD Dissertation Based Presentation
July 26, 2023
Mike Heenan, Ph.D.
• Hospital Executive Vice-President
• Sessional Lecturer, McMaster University – DeGroote School
of Business
• PhD, Business Administration, Health Policy Management,
2023
• Thesis research: how health care organizations selected key
performance metrics and how the process of indicator
selection impacts front line managers motivation to
improve performance.
2
Session Format
3
eHealth – Why this Lecture?
The Literature and Our Problem – Metric Madness
Hospital Executive Perception of Front-Line Manager’s Knowledge
What Indicators Motivate Front-Line Clinical Managers?
What did the pandemic teach us?
The 5-P Indicator Selection Process Framework
The Politics of Improvement: Your Challenge as eHealth Leaders
eHealth – Why this Lecture?
4
eHealth – Why this Lecture?
Your program objectives:
1. Understand eHealth concepts
2. Communicate effectively in eHealth contexts
3. Apply eHealth Knowledge in practical contexts
5
Key Readings We will Reference
• Damschroder LJ, Robinson CH, Francis J, et al. Effects of Performance Measure Implementation on Clinical
Manager and Provider Motivation. J Gen Intern Med. 2014; 29(S4):877-884. DOI: 10.1007/s11606-014-
3020-9.
• Elliot C, Mcullagh C, Brydon M, Zwi K. Developing key performance indicators for a tertiary children’s
hospital network. Australian Health Review. 2018; 42(5):491-500. DOI: 10.1071/AH17263.
• Ginsburg LS. Factors that Influence Line Managers’ Perceptions of Hospital Performance Data. Health
Services Research. 2003; 38(1):261-286. DOI: 10.1111/1475-6773.00115.
• Heenan MA, Randall GE, Evans JM. Selecting Performance Indicators and Targets in Health Care: An
International Scoping Review and Standardized Process Framework. Risk Management and Healthcare
Policy. 2022; 15:747-764. DOI: 10.2147/RMHP.S357561.
• Mannion R, Braithwaite J. Unintended consequences of performance measurement in health care: 20 salutary
lessons from the English National Health Service. Internal Medicine Journal. 2012; 42(5):569–574. DOI:
10.1111/j.1445-5994.2012.02766.x
6
To Start …
Two Key Definitions
Performance Measurement is defined as the collection, use and public
reporting of data for the purpose of quality improvement, accountability,
and transparency.
America Academy of Family Physicians Performance Measurement Criteria. [Internet] 2022. [Cited 27 Nov 2022].
Available from: https://www.aafp.org/about/policies/all/performance-measures.html
Indicators are defined as measurable elements of practice performance that
relate to clinical, population health, financial, or organizational performance.
Lawrence M, Olesen F. Indicators of Quality in Health Care. The European Journal of General Practice.
1997; 3(3):103-108. DOI: 10.3109/13814789709160336
7
To Start …
Indicators are Metrics, Metrics are Indicators
Indicators also known as
• Key Performance Indicators
• Performance Indicators
• Quality Indicators
• Performance Metrics
• Performance Measures
• Quality Metrics
8
To Start …
Donabedian Framework
Framework for evaluating health care service quality.
Information on quality of care can be seen through three types of indicators:
Structure: factors that affect the context in which care is delivered
Process: actions the describe how care is delivered
Outcome: effects of healthcare delivery on patients or populations, including
changes to health status, behavior, or knowledge as well as patient satisfaction
and health-related quality of life
9
Donabedian A. The quality of care: how can it be assessed? JAMA. 1988 Sep 23; 260(12):1743-8. DOI: 10.1001/jama.1988.03410120089033.
The Literature and Our Problem – Metric Madness
10
Needed Measurement Led to Too Much Measurement
• USA, 1999 - Approximately 98,000 Americans lost their lives each year due to medical error.
• Canada, 2004 - Average of 7.5% of hospital admissions resulting in an adverse event.
• Multiple governments, regulatory agencies, and funding bodies have mandated the
collection and monitoring of hundreds of KPIs by hospitals
11
Kohn LT, Corrigan JM, Donaldson MS. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press, 2000.
Baker GR, Norton P, Flintoff V, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. Canadian Medical Association Journal.
2004; 170(11):1678-1686. DOI: 10.1503/cmaj.1040498
Needed Measurement has Led to Too Much Measurement
12
USA:
• Centre for Medicare and Medicaid Services (CMS) monitors over 1700 KPIs
• National Quality Forum (NQF) approved KPIs grew from 200 in 2005 to over 700 in 2011
• Study concluded over 550 distinct KPIs were being collected across 48 states
Canada
Ontario Hospital Association and Health Quality Ontario argue there are over 300 quality KPIs reported
by hospitals.
International
Over-measurement has been echoed in the United Kingdom and Australia (Mannion and Braithwaite 2011).
Panzer RJ 2013; Wilensky G 2018; Meyer GS 2012; Greenburg A 2019; Mannion R 2012;
The Management Problem:
Overabundance of KPIs has resulted in unintended consequences
13
• Data mandated by state bodies do not necessarily reflect every local context and may be
used inappropriately
• Public reporting of KPIs without local input has resulted in a lack of trust between
providers and political bodies, and gaming
• Reporting mandates have also increased financial costs to organizations
• Building of the information technology and data infrastructure required to support
measurement has amplified the amount of data available to health care organizations
Mannion R and Braithwaite 2011; Safavi 2006; Teare 2014; Meyer et al 201, Berwick 2016
Perla: The U Curve
Paralyzed decision making (Cassel et al 2014, Perla 2013).
14
Perla R. Commentary: Health Systems Must Strive for Data Maturity. American Journal of Medical Quality. 2013; 28(3):263-264. DOI:10.1177/1062860612465000.
Call for Balance and Parsimony
1. Measure process quality
2. Measure value
3. Design data systems for internal quality needs and spinoff external
quality measures
4. Use return on investment
5. Establish ongoing process to refine and select core measures
15
Meyer GS, Nelson EC, Pryor DB, et al. More quality measures versus measuring what matters: a call for balance and parsimony. BMJ Qual Saf. 2012; 21(11):964-968.
DOI: 10.1136/bmjqs-2012-001081.
Question # 1
How does eHealth further hinder the phenomenon of over-
measurement?
16
A Call for Balance, but…
Not Directly Linked to Goal Theory
Some agencies have heeded these calls.
• Institute of Medicine (2015) recommends 15 national KPIs that systems should monitor (Blumenthal and
McGinnis 2015).
• National Quality Forum (2019) instituted annual review process that endorses certain KPIs for adoption to
ensure their relevancy to overall system goals. (NQF 2019)
• World Health Organization started examining processes in 2008 and offered criteria to narrow KPI selection
(Smith et al 2008).
• Research papers describe Delphi methods used to select KPIs in clinical areas (Schull et al 2011, Madsen et al
2016, Harvey 2016, Murphy 2016).
These reports describe criteria and methods to select an appropriate number of quality and patient safety
KPIs at the system or clinical service level, but…
a) They do not clarify who should be involved in KPI and Target selection, or,
b) how health service provider organizations like hospitals should use selection processes as an
instrument to motivate leaders to achieve agreed upon goals.
17
Hospital Executive’s Perception
of Front-Line Manager’s Knowledge
18
What processes do hospitals use to select performance indicators and do they align with best
practices? A multiple-case study of four hospitals in Ontario, Canada
Qualitative, multiple-case study of four acute care hospitals and how they select indicators
Results
• Process elements largely absent include:
• adopting evidence-based selection criteria
• considering finance and human resources indicators;
• considering if indicators measure structure, process or outcomes,
• engaging a broader set of end-users in the selection process.
• Revealed senior management teams had little confidence in clinical unit managers’
ability to use indicators to improve performance.
19
What Indicators Motivate Front-Line Managers?
20
Who are and why do we care about
Front-Line Managers?
Who
• Clinical unit / Front-Line managers are employees who are both supervised by an
organization’s top managers and who themselves supervise front-line employees.10
Why
• Managers’ commitment and ability to implement innovative change at the unit level
has been linked to strategy realization, efficiency of operations, cost control, and overall
achievement of quality outcomes.10-12
• If positively motivated and engaged, Managers can effectively support innovation and
change. If managers are disengaged and not motivated, it can impede improvement in
the units they oversee.13-15
• Managers’ motivation and commitment increases when their senior managers support
them with the infrastructure and resources necessary for change.14,16
21
Managers and Goal-Setting Theory of Motivation
Goal-Setting Theory of Motivation emphasizes the relationship between goals and
performance.
Task performance is influenced by the manager’s ability; the stated goal’s content, intensity,
and duration; the manager’s self-efficacy; and the personal goals of the manager.
Key Attributes:
1. Benefit of participating in goal setting processes is cognition on what’s important and why;
2. Specific goals with harder to reach improvements result in a greater effort by managers;
3. Regular feedback increases performance effort by improving manager’s self-efficacy.
4. Managers that feel supported by their leadership have improved self-efficacy;
5. Managers that feel part of a larger team improve effort through both a shared understanding of the
goal’s direction and shared-accountability; and,
6. Managers increased confidence leads to greater motivation to act and perform tasks.
22
Locke EA, Latham GP. Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist. 2002; 57(9): 705–717. DOI:
10.1037//0003-066x.57.9.705
Need to understand base motivations:
Why clinicians become managers
• Clinical managers never thought of becoming a manager when completing
their initial clinical education.
• Front-line clinicians move into management to help improve quality and
the working conditions of their fellow staff.
• When asked what indicators they prefer clinical unit managers referenced
process indicators that help advance clinical practice, and front-line staff
engagement as criteria by which they would choose indicators.
23
Kleinman CS. Leadership Roles, Competencies, and Education: How Prepared Are Our Nurse Managers? Journal of Nursing Administration. 2003; 33(9):451-455.
DOI: 10.1097/00005110-200309000-00005.
White J. Motivation for Seeking a Nurse manager Position. [dissertation]. San Diego, CA: University of San Diego, 2015.
Available from: https://digital.sandiego.edu/dissertations/17
Why do managers use performance data?
24
Ivankovic D, Poldrugovac M, Garel P, Klazinga NS, Kringos DS (2020) Why, what and how do European healthcare managers use performance data? Results of a survey and workshop
among members of the European Hospital and Healthcare Federation. PLoS ONE 15 (4): e0231345. https://doi.org/10.1371/journal.pone.0231345
What barriers do managers face in using data?
• Managers gave three barriers: lack of training
and knowledge on measurement, workload,
and job experience.
• Managers need to obtain is the understanding,
interpretation, and use of performance data for
evidence-based decision making.
• Clinicians are often appointed to managerial
roles without business skills, and they therefore
need to learn them through trial and error.
25
“My education is based on how to be a
nurse, not how to decipher data and know
what it means. So, some of this stuff is
foreign to me and it’s been trial by fire for
me.”
Case B-Manager 3
Birken SA, Lee SYD, Weiner BJ. 2012; DOI: 10.1186/1748-5908-7-28.
Naranjo-Gil D. 2009; DOI: 10.1097/HMR.0b013e31819e8fd0.
Kleinman CS. 2003; DOI: 10.1097/00005110-200309000-00005.
McCallin AM, Frankson C. 2010; 1DOI: 10.1111/j.1365-2834.2010.01067.x
Heenan, MA. 2023. (Doctoral dissertation).
“I have not received any training or
education on measurement here. When I
started in the role, it was kind of just sink or
swim.”
Case A-Manager 3
How clinical unit managers’ roles in selecting hospital indicators impacts their motivation and
self-efficacy to improve performance: A qualitative multiple-case study.
Qualitative multiple-case study of four hospitals involving 22 managers. Semi-structured
Interviews. Thematic Analysis.
Results:
• Managers reported being unaware of the criteria hospitals used to select indicators, are
not trained in measurement, feel they have little control over the indicators they were
accountable for, and were fearful of reporting on performance.
• Managers motivated by process indicators focused on clinical quality and patient
experience vs. outcome or business indicators.
• Managers gained confidence over time and by learning how to communicate the utility
of indicators to front-line staff.
26
Heenan, M. A. (2023). Health Care Indicator Selection Processes and their Impact on Clinical Unit Managers’ Motivation and Self-Efficacy to Improve
Performance (Doctoral dissertation).
Ginsburg: Factors that Influence Line Managers’
Perceptions of Hospital Performance Data
27
Ginsburg LS. Factors that Influence Line Managers’ Perceptions of Hospital Performance Data. Health Services Research. 2003; 38(1):261-286. DOI: 10.1111/1475-6773.00115
“Simply generating performance data will not, in and of itself, lead to improvement.”
Factors Significant Correlation to
Perceived Usefulness of Data
• Data quality
• Data relevance
• Report complexity
• Intensity of dissemination
Damschroder: Effects of Performance Measure
Implementation on Manager Motivation
• Indicators need to be
designed to both incent
individuals and build
coalitions.
• Indicators sit within a QI
foundation but only effective
if seen positively by fully
engaged staff and
management.
28
Damschroder LJ, Robinson CH, Francis J, et al. Effects of Performance Measure Implementation on Clinical Manager and Provider Motivation. J Gen Intern Med. 2014;
29(S4):877-884. DOI: 10.1007/s11606-014-3020-9.
Question # 2
Now knowing the motivations and challenges end-users have
in using data, how can eHealth help the phenomena of over-
measurement?
29
What did the pandemic teach us?
30
What did the pandemic teach us?
Executive Perspectives
• Senior management leaders
mentioned that some clinical
practice, supply chain, and human
resource indicators, once thought
not important to regular
monitoring, had surfaced as
priorities.
31
Heenan MA. Health Care Indicator Selection Processes and their Impact on Clinical Unit Managers’ Motivation and Self-Efficacy to Improve Performance (Doctoral dissertation).
“COVID taught us that after years of trying to get the
organization to make data-driven decisions that often fell
on deaf ears, the organization was thirsty for data about
this new disease. This demand for data was likely a result
of operating in a new unknown environment and the need
to base operational decisions on emerging data compared
to traditional lagging indicators.
COVID also taught us that we likely had been
providing the wrong data before the pandemic. COVID
forced us to prioritize our efforts and stopped us from
chasing too many indicators. It led us to provide more
leading process-based indicators that clinical units found
useful in planning their day versus lagging outcome data
that government wanted.”
Case B-Executive 2
What did the pandemic teach us?
Front-Line Manager Perspectives
Clinical unit managers uniformly shared that the
pandemic:
• created a single priority for their organizations;
• involved greater engagement in, and lead to,
faster decision making;
• Focus on process indicators related to practice
and staff safety.
• Example, new indicator monitored more
closely – personal protective equipment (PPE)
Inventory Levels
32
“COVID has changed everything. I've seen
senior leader sentiment sort of evolve from
when people are being told to do things versus
asking us why we're doing them. Senior leaders
have become clinically focused. We see them
maybe a little bit better for who they are now
because the data is not being shoved down our
throats. We are being asked if certain data
makes sense to us. My VP is just, yep, whatever
you need, so I get that support, which makes a
huge difference.”
Case A-Manager 4.
Heenan MA. Health Care Indicator Selection Processes and their Impact on Clinical Unit Managers’ Motivation and Self-Efficacy to Improve Performance (Doctoral dissertation).
Lessons
1. Crises create opportunities to see new needs and new ways
2. Decision making structures get flatter, relationships gets stronger,
because you engage the end-user deliberately and quickly
3. Process metrics matter and help inform priority actions
4. Managers want to be engaged to better understand their accountabilities
and help select process indicators that measure quality, clinical practice,
patient experience and staff engagement
33
In designing or selecting indicators, what framework
should guide us?
The 5-P Indicator Selection Process Framework
34
Selecting performance indicators and targets in health care:
An international scoping review and standardized process framework
Purpose: Synthesize international approaches to indicator selection and develop a standardized process
framework.
Methods:
• PRISMA-ScR Scoping review using Pub Med & Web of Science.
• English-language papers from 11 countries, 2010-2020.
• Reviewed 33 peer-reviewed papers and 11 grey-literature documents.
• Looked at processes with a lens to: Aim; Governance; Preparation; Process Methods; Validation
Results:
• Some processes used Appraisal of Indicators through Research and Evaluation (AIRE) Instrumentand the
Quality Indicator Critical Appraisal (QICA) tool to guide which individual indicators should be considered.
• No standardized indicator selection process framework exists that focuses on engagement that solicits buy
in and drives motivation.
35
Heenan MA, Randall GE, Evans JM. Selecting Performance Indicators and Targets in Health Care: An International Scoping Review and Standardized Process Framework.
Risk Management and Healthcare Policy. 2022; 15:747-764.
The 5-P Indicator Selection Process Framework
Incomplete indicator selection processes
risk:
o over-measurement,
o lack of prioritizing strategic and
operational goals,
o lack of support by end-users
o paralyzed decision-making ability.
• Standardized framework developed to
assist organizations complete a
comprehensive indicator and target
selection process.
36
Domain Element
Clarify Aim
Develop Guiding Principles
Identify Level of Use
Build Governance Structures
Recruit Participants
Seek End-User Input
Research Evidence-Based Literature
Build an Inventory of Potential Indicators
Categorize Potential Indicators into Strategic
Orient and Train Participants
Utilize a Consensus Building Method
Identify a Facilitator
Indicator Selection Criteria
Analytical Assess Indicators
Set Indicator Targets
Assess Data Quality
Validate with End-Users
5-P Indicator Selection Process Framework
Prepare
Procedure
Prove
Purpose
Polity
Heenan, MA, Randall GE, Evans JA. Selecting performance indicators and targets in health care: An international scoping review and standardized process framework. Risk Management and
Healthcare Policy. 2022; 15:747-764. DOI: 10.2147/RMHP.S357561.
Domain: Purpose
37
“Purpose” sets out the reasons why an indicator selection and target setting process is undertaken
– facilitating a shared understanding of what the organization is setting out to achieve.
Elements
• Clarify Aim
• Set Guiding Principles
• Identify Level of Use
Purpose: Three Elements
38
Watkins: Setting clear aims and guiding principles allows participants to understand the desired goals a process is
aiming to achieve and the actions to which they are expected to contribute.
Watkins MD. Demystifying Strategy: The What, Who How and Why. Harvard Business Review. September 10, 2007. Available from: https://hbr.org/2007/09/demystifying-strategy-the-what
Elliot, 2018
39
Elliot C, Mcullagh C, Brydon M, Zwi K. Developing key performance indicators for a tertiary children’s hospital network. Australian Health Review. 2018; 42(5):491-500. DOI: 10.1071/AH17263.
Aim:
Following merger ad strategic plan, to address executive level
performance indicators for pediatric tertiary care services across two
campus
Guiding Principles:
KPIs must address the mission statement, “working in partnership to
improve heath and well being of children through clinical care, research
education and advocacy”.
Drive quality improvement in child health outcomes
Level of Use:
Executive Level / Board
Domain: Polity
Elements
• Build Governance Structures
• Recruit Participants
40
“Polity” identifies the governance structures that manage the selection process, how the process
will be resourced, and who participates.
Polity: Two Elements
41
Aktaa: Expert panels have inherent bias. Therefore, expansion of participants is important mitigation. Indicator
selection processes that have broader participation are more likely than others to have a more inclusive view of
front-line operations
Aktaa S, Batra G, Wallentin L, et al. European Society of Cardiology methodology for the development of quality indicators for the quantification of cardiovascular care and outcomes. European
Heart Journal Quality of Care and Clinical Outcomes. 2022; 8(1):4-13. DOI: 10.1093/ehjqcco/qcaa069.
Example: Elliot, 2018
42
Elliot C, Mcullagh C, Brydon M, Zwi K. Developing key performance indicators for a tertiary children’s hospital network. Australian Health Review. 2018; 42(5):491-500. DOI: 10.1071/AH17263.
Governance Structure
Single body - Executive Group
Participants / Inputs:
Network Executives;
Clinical Directors;
Medical, Allied Health and Nurse Managers.
Domain: Prepare
Elements
• Seek End-User Input
• Research Evidence Base Literature
• Build an Inventory of Indicators
• Categorize Indicators into Strategic Themes
• Orient and Train Participants
43
“Prepare” addresses how to plan for selection.
Prepare: Five Elements
44
Heenan
• Final list of indicators built from comprehensive sources improves their relevancy to end-users while enabling future
comparability and benchmarking.
• End-user input upfront on indicator knowledge and user requirements and issued orientation materials increased
participant engagement and improved understanding of the process among participants.
Example: Elliot, 2018
45
Elliot, 2018
46
Elliot, 2018
47
Elliot, 2018
48
Domain: Procedure
Elements
• Utilize a Consensus Building Methodology
• Identify a Facilitator
• Set Indicator Selection Criteria
• Analytically Assess Indicators
• Set Indicator Targets
49
“Procedure” describes the how to steps used to assess indicators and targets and gain consensus.
Procedure: Five Elements
50
Stetler: Internal facilitation may bias processes whereas external facilitators have an ability to play both a problem
solving and supportive role that might otherwise be difficult when only internal parties are engaged. External
facilitation is valued when a process is focused on an interactive problem or distinct activity.
Stetler CB, Legro MW, Rycroft-Malone J, Bowman C, Curran G, Guihan M, Hagedorn H, Pineros S, Wallace CM. Role of external facilitation in implementation of research findings: a qualitative
evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science. 2006; 1(1):1-5. DOI: 10.1186/1748-5908-1-23.
Example: Elliot, 2018
51
Domain: Prove
Elements
• Assess Data Quality (Quantitatively)
• Validate with End-Users (Qualitatively)
52
“Prove” describes the validation processes used to test any final set of indicators for data quality
and relevance with end-users.
Prove: Two Elements
53
Heenan
• Processes that statistically test indicators increase scientific soundness and better informs target setting.
• Processes that validate a final list of indicators with end-users improve relevance and usability by users, especially in
cases where the expert panels did not include front-line directors, managers, or patients.
Heenan, MA, Randall GE, Evans JA. Selecting performance indicators and targets in health care: An international scoping review and standardized process framework. Risk Management and
Healthcare Policy. 2022; 15:747-764. DOI: 10.2147/RMHP.S357561.
Examples: Elliot, 2018 and Heenan, 2023
54
QUALITATIVE: Noted Limitation (Elliot)
“There was little or no consultation with less senior staff members, which risks losing the
support of staff who will be relied upon to provide data and ensure accurate reporting”
QUANTITATIVE: 2022 Case Study of 4 Ontario Hospitals (Heenan)
• None quantitatively validated KPIs - reasoned most of the indicators they monitor are
mandated by external funding agencies, and it is therefore assumed these indicators
were previously validated.
• This is a false assumption.
• While the technical formula of an indicator may have been validated by an external
agency, the data that is generated from the different local hospital information systems
may be of different quality than the data by which an indicator was tested by the
external agency. (e-heath)
Key Conclusion
Recall
• Overabundance of indicators has paralyzed decision making, and eroded trust
between those who ask for indicators and those who are expected to use them to
make change.
• Many policy institutes and academics have called for a more appropriate, lower
number of indicators.
Therefore
• Indicator selection or reduction processes cannot occur by happenstance.
• The 5-P Indicator Selection Process Framework to ensure a systematic, evidence-
based, and inclusive approach that engages measurement experts and those who
use indicators to monitor and improve performance in both selection and
validation. 55
Question # 3
What domain and element of the 5-P Indicator Selection
Process Framework can eHealth most help with? Why?
56
The politics of improvement:
Your challenge as eHealth Leaders
57
Process improvement is not only about adopting evidence but includes the process of
engaging those responsible for implementing the prescribed change
Beyond the evidence (or technology):
You’re not just an eHealth Leader but a Change Leader
58
• Quality Improvement initiatives are not simple
• Evidence is insufficient in itself to ensure implementation
Distributed Nature of Costs
and Benefits
Variety of Value Systems
Underlying Interventions
What’s in it ?
For Whom ?
What ideologies and
ethics are involved ?
Langley A, Denis J. Beyond evidence: the micropolitics of improvement. BMJ Quality and Safety. 2014; 20:i43-i46. DOI: 10.1136/bmjqs.2010.046482.
Hard Core and Soft Periphery
Hard core = Evidence that is
irreducible and carries key benefit.
(The Technical What)
Soft Periphery = Complex delivery
systems that take different forms and
if poorly organized may destroy the
benefit of the hard core.
(The Cultural Why & People)
59
Navigate:
The Micro-Political / Change Leader Check List
1. Understand the evidence
2. Identify who it will impact
3. Understand how it will change practice
4. Don’t make change alone: find a champion(s)
5. Solicit input on how to adopt locally
6. Sell the why (involve end-users)
7. Teach: Be a coach not a consultant
60
Final Conclusions
61
Your Program Objectives:
You hold knowledge and power, Use it Wisely
1. Understand eHealth Concepts
• The technology and systems you build and promote will be used to
make change by producing information and data.
• This information and data (indicators) can overwhelm the end-
users not aware of eHealth concepts.
2. Communicate Effectively in eHealth Contexts
• Ask why are we producing the information?
3. Apply eHealth Knowledge in practical contexts
• What will it be used for? And by Whom?
62
Final Conclusions
Answering … “How to select engaging performance metrics?”
1. Have a Structured, Open, Transparent Process
2. Include End-Users as Participants
3. Educate and Train participants on concepts
4. Allow Input on Past Experience before Selection
5. People Process Things to get to Outcomes so Process Metrics Matter
6. Validate Data in Real Environments
63
Abd then don’t forget …The U Curve
64
Perla R. Commentary: Health Systems Must Strive for Data Maturity. American Journal of Medical Quality. 2013; 28(3):263-264. DOI:10.1177/1062860612465000.
Thank you
Questions?
65
Dissertation Reference
66
Heenan, M. A. (2023). Health Care Indicator Selection Processes and their Impact on Clinical Unit Managers’
Motivation and Self-Efficacy to Improve Performance (Doctoral dissertation).
https://macsphere.mcmaster.ca/bitstream/11375/28348/2/Heenan_Michael_A_20
2302_PhD.pdf

More Related Content

What's hot

Borang senarai semak pemantauan kantin sekolah
Borang senarai semak pemantauan kantin sekolahBorang senarai semak pemantauan kantin sekolah
Borang senarai semak pemantauan kantin sekolah
penaabadi1979
 
Carta gantt unit kurikulum
Carta gantt unit kurikulumCarta gantt unit kurikulum
Carta gantt unit kurikulum
maizaKSEL
 
Latihan matematik dlm cuti (LINUS)
Latihan matematik dlm cuti (LINUS)Latihan matematik dlm cuti (LINUS)
Latihan matematik dlm cuti (LINUS)
PAKLONG CIKGU
 
Diet pada penyakit hati
Diet pada penyakit hatiDiet pada penyakit hati
Diet pada penyakit hatiwokwok
 
TAKLIMAT PBPPP 2021.pdf
TAKLIMAT PBPPP 2021.pdfTAKLIMAT PBPPP 2021.pdf
TAKLIMAT PBPPP 2021.pdf
SEKKEBSUNGAISUMUN
 
Taklimat SEGAK Tahun 6.pptx
Taklimat SEGAK Tahun 6.pptxTaklimat SEGAK Tahun 6.pptx
Taklimat SEGAK Tahun 6.pptx
ssuser4ecc97
 
Motivasi pelajar slide share
Motivasi pelajar slide shareMotivasi pelajar slide share
Motivasi pelajar slide share
Dr. Rus Sumariyanti Haji Surin
 
Bengkel cdl mei 2017
Bengkel cdl mei 2017Bengkel cdl mei 2017
Bengkel cdl mei 2017
SMK RPI
 
Analisis item dan headcount sains tingkatan 5 2021
Analisis item dan headcount sains tingkatan 5 2021Analisis item dan headcount sains tingkatan 5 2021
Analisis item dan headcount sains tingkatan 5 2021
TRACYVIANNEYANOMoe
 
intrumen fail panitia
intrumen fail panitiaintrumen fail panitia
intrumen fail panitia
NANCYJOSEPHKUMBIRMoe
 
Evaluasi diri pgsd stkip bina bangsa meulaboh
Evaluasi diri pgsd stkip bina bangsa meulabohEvaluasi diri pgsd stkip bina bangsa meulaboh
Evaluasi diri pgsd stkip bina bangsa meulaboh
Henra Saputra Tanjung
 
BULATAN TS25.pptx
BULATAN TS25.pptxBULATAN TS25.pptx
BULATAN TS25.pptx
Muslihah Md Desa
 
Teknik menjawab math spm
Teknik menjawab math spmTeknik menjawab math spm
Teknik menjawab math spm
Tuan Azahan
 
Pengurusan panitia
Pengurusan panitiaPengurusan panitia
Pengurusan panitiaimanmutiara
 
(34)bidang tugas ketua panitia
(34)bidang tugas ketua panitia(34)bidang tugas ketua panitia
(34)bidang tugas ketua panitiaJojo Micheal
 
Dialog-Prestasi-Sekolah-pptx.pptx
Dialog-Prestasi-Sekolah-pptx.pptxDialog-Prestasi-Sekolah-pptx.pptx
Dialog-Prestasi-Sekolah-pptx.pptx
mohdsofi10
 
Seorang Anak Laki-laki dengan Diare Akut Dehidrasi Sedang, Anemia Defisiensi ...
Seorang Anak Laki-laki dengan Diare Akut Dehidrasi Sedang, Anemia Defisiensi ...Seorang Anak Laki-laki dengan Diare Akut Dehidrasi Sedang, Anemia Defisiensi ...
Seorang Anak Laki-laki dengan Diare Akut Dehidrasi Sedang, Anemia Defisiensi ...
Argo Dio
 
Garis panduan penilaian prestasi tahunan
Garis panduan penilaian prestasi tahunanGaris panduan penilaian prestasi tahunan
Garis panduan penilaian prestasi tahunan
INTAN Bukit Kiara
 

What's hot (20)

Borang senarai semak pemantauan kantin sekolah
Borang senarai semak pemantauan kantin sekolahBorang senarai semak pemantauan kantin sekolah
Borang senarai semak pemantauan kantin sekolah
 
Borang borang headcount
Borang borang headcountBorang borang headcount
Borang borang headcount
 
Carta gantt unit kurikulum
Carta gantt unit kurikulumCarta gantt unit kurikulum
Carta gantt unit kurikulum
 
Latihan matematik dlm cuti (LINUS)
Latihan matematik dlm cuti (LINUS)Latihan matematik dlm cuti (LINUS)
Latihan matematik dlm cuti (LINUS)
 
Diet pada penyakit hati
Diet pada penyakit hatiDiet pada penyakit hati
Diet pada penyakit hati
 
TAKLIMAT PBPPP 2021.pdf
TAKLIMAT PBPPP 2021.pdfTAKLIMAT PBPPP 2021.pdf
TAKLIMAT PBPPP 2021.pdf
 
Taklimat SEGAK Tahun 6.pptx
Taklimat SEGAK Tahun 6.pptxTaklimat SEGAK Tahun 6.pptx
Taklimat SEGAK Tahun 6.pptx
 
Motivasi pelajar slide share
Motivasi pelajar slide shareMotivasi pelajar slide share
Motivasi pelajar slide share
 
Bengkel cdl mei 2017
Bengkel cdl mei 2017Bengkel cdl mei 2017
Bengkel cdl mei 2017
 
Borang penilaian kegiatan kokurikulum
Borang penilaian kegiatan kokurikulumBorang penilaian kegiatan kokurikulum
Borang penilaian kegiatan kokurikulum
 
Analisis item dan headcount sains tingkatan 5 2021
Analisis item dan headcount sains tingkatan 5 2021Analisis item dan headcount sains tingkatan 5 2021
Analisis item dan headcount sains tingkatan 5 2021
 
intrumen fail panitia
intrumen fail panitiaintrumen fail panitia
intrumen fail panitia
 
Evaluasi diri pgsd stkip bina bangsa meulaboh
Evaluasi diri pgsd stkip bina bangsa meulabohEvaluasi diri pgsd stkip bina bangsa meulaboh
Evaluasi diri pgsd stkip bina bangsa meulaboh
 
BULATAN TS25.pptx
BULATAN TS25.pptxBULATAN TS25.pptx
BULATAN TS25.pptx
 
Teknik menjawab math spm
Teknik menjawab math spmTeknik menjawab math spm
Teknik menjawab math spm
 
Pengurusan panitia
Pengurusan panitiaPengurusan panitia
Pengurusan panitia
 
(34)bidang tugas ketua panitia
(34)bidang tugas ketua panitia(34)bidang tugas ketua panitia
(34)bidang tugas ketua panitia
 
Dialog-Prestasi-Sekolah-pptx.pptx
Dialog-Prestasi-Sekolah-pptx.pptxDialog-Prestasi-Sekolah-pptx.pptx
Dialog-Prestasi-Sekolah-pptx.pptx
 
Seorang Anak Laki-laki dengan Diare Akut Dehidrasi Sedang, Anemia Defisiensi ...
Seorang Anak Laki-laki dengan Diare Akut Dehidrasi Sedang, Anemia Defisiensi ...Seorang Anak Laki-laki dengan Diare Akut Dehidrasi Sedang, Anemia Defisiensi ...
Seorang Anak Laki-laki dengan Diare Akut Dehidrasi Sedang, Anemia Defisiensi ...
 
Garis panduan penilaian prestasi tahunan
Garis panduan penilaian prestasi tahunanGaris panduan penilaian prestasi tahunan
Garis panduan penilaian prestasi tahunan
 

Similar to M Heenan_PhD Dissertation Lecture_eHealth Lecture_Engaging Leaders in KPI Selection_Final_July 2023

Healthcare quality improvement for meaningful use
Healthcare quality improvement for meaningful useHealthcare quality improvement for meaningful use
Healthcare quality improvement for meaningful use
Samantha Haas
 
NTI2010 Increasing Quality And Safety
NTI2010 Increasing Quality And SafetyNTI2010 Increasing Quality And Safety
NTI2010 Increasing Quality And Safety
Todd_Grivetti
 
Healthcare Quality Improvement Advice by Dr.Mahboob ali khan Phd
Healthcare Quality Improvement Advice by Dr.Mahboob ali khan Phd Healthcare Quality Improvement Advice by Dr.Mahboob ali khan Phd
Healthcare Quality Improvement Advice by Dr.Mahboob ali khan Phd
Healthcare consultant
 
Quality assurance
Quality assuranceQuality assurance
Quality assurance
harshraman1989
 
Governance and management in healthcare
Governance and management in healthcareGovernance and management in healthcare
Governance and management in healthcare
Rick Jones
 
1545 eilish mc auliffe nhc 28-5-15 emc
1545 eilish mc auliffe nhc   28-5-15 emc1545 eilish mc auliffe nhc   28-5-15 emc
1545 eilish mc auliffe nhc 28-5-15 emc
investnethealthcare
 
Measure For Improvement
Measure For ImprovementMeasure For Improvement
Measure For Improvement
Musa Abu Sbeih
 
Measure the Improvement
Measure the  ImprovementMeasure the  Improvement
Measure the Improvement
Musa Abu Sbeih
 
QUALITY ASSURANCE IN HEALTH CARE.ppt
QUALITY ASSURANCE IN HEALTH CARE.pptQUALITY ASSURANCE IN HEALTH CARE.ppt
QUALITY ASSURANCE IN HEALTH CARE.ppt
S A Tabish
 
QUALITY ASSURANCE IN HEALTH CARE.ppt
QUALITY ASSURANCE IN HEALTH CARE.pptQUALITY ASSURANCE IN HEALTH CARE.ppt
QUALITY ASSURANCE IN HEALTH CARE.ppt
S A Tabish
 
HS410 Unit 6 Quality Management - DiscussionDiscussionThi.docx
HS410 Unit 6 Quality Management - DiscussionDiscussionThi.docxHS410 Unit 6 Quality Management - DiscussionDiscussionThi.docx
HS410 Unit 6 Quality Management - DiscussionDiscussionThi.docx
AlysonDuongtw
 
PHF nphpsp webinar pm framework 12.20.11
PHF nphpsp webinar pm framework 12.20.11PHF nphpsp webinar pm framework 12.20.11
PHF nphpsp webinar pm framework 12.20.11
PublicHealthFoundation
 
Quality improvement healthcare final
Quality improvement healthcare finalQuality improvement healthcare final
Quality improvement healthcare final
Evanvs
 
Clingov5understandingaudit2003
Clingov5understandingaudit2003Clingov5understandingaudit2003
Clingov5understandingaudit2003
Papri Sarkar
 
Paying for performance to improve the delivery of health interventions in LMICs
Paying for performance to improve the delivery of health interventions in LMICsPaying for performance to improve the delivery of health interventions in LMICs
Paying for performance to improve the delivery of health interventions in LMICs
ReBUILD for Resilience
 
Medical audit process
Medical audit processMedical audit process
Medical audit process
sopi_1234
 
Improving Public Health Outcomes W Epm
Improving Public Health Outcomes W EpmImproving Public Health Outcomes W Epm
Improving Public Health Outcomes W Epm
reidblock
 
Knowledge Translation: Practical Strategies for Success v1
Knowledge Translation: Practical Strategies for Success v1Knowledge Translation: Practical Strategies for Success v1
Knowledge Translation: Practical Strategies for Success v1
Imad Hassan
 
Quality improvement and patient safety in anesthesia
Quality improvement and patient safety in anesthesiaQuality improvement and patient safety in anesthesia
Quality improvement and patient safety in anesthesia
Dr. Ravikiran H M Gowda
 
Standards Compliance And Quality Management
Standards Compliance And Quality ManagementStandards Compliance And Quality Management
Standards Compliance And Quality Management
Rick Jones
 

Similar to M Heenan_PhD Dissertation Lecture_eHealth Lecture_Engaging Leaders in KPI Selection_Final_July 2023 (20)

Healthcare quality improvement for meaningful use
Healthcare quality improvement for meaningful useHealthcare quality improvement for meaningful use
Healthcare quality improvement for meaningful use
 
NTI2010 Increasing Quality And Safety
NTI2010 Increasing Quality And SafetyNTI2010 Increasing Quality And Safety
NTI2010 Increasing Quality And Safety
 
Healthcare Quality Improvement Advice by Dr.Mahboob ali khan Phd
Healthcare Quality Improvement Advice by Dr.Mahboob ali khan Phd Healthcare Quality Improvement Advice by Dr.Mahboob ali khan Phd
Healthcare Quality Improvement Advice by Dr.Mahboob ali khan Phd
 
Quality assurance
Quality assuranceQuality assurance
Quality assurance
 
Governance and management in healthcare
Governance and management in healthcareGovernance and management in healthcare
Governance and management in healthcare
 
1545 eilish mc auliffe nhc 28-5-15 emc
1545 eilish mc auliffe nhc   28-5-15 emc1545 eilish mc auliffe nhc   28-5-15 emc
1545 eilish mc auliffe nhc 28-5-15 emc
 
Measure For Improvement
Measure For ImprovementMeasure For Improvement
Measure For Improvement
 
Measure the Improvement
Measure the  ImprovementMeasure the  Improvement
Measure the Improvement
 
QUALITY ASSURANCE IN HEALTH CARE.ppt
QUALITY ASSURANCE IN HEALTH CARE.pptQUALITY ASSURANCE IN HEALTH CARE.ppt
QUALITY ASSURANCE IN HEALTH CARE.ppt
 
QUALITY ASSURANCE IN HEALTH CARE.ppt
QUALITY ASSURANCE IN HEALTH CARE.pptQUALITY ASSURANCE IN HEALTH CARE.ppt
QUALITY ASSURANCE IN HEALTH CARE.ppt
 
HS410 Unit 6 Quality Management - DiscussionDiscussionThi.docx
HS410 Unit 6 Quality Management - DiscussionDiscussionThi.docxHS410 Unit 6 Quality Management - DiscussionDiscussionThi.docx
HS410 Unit 6 Quality Management - DiscussionDiscussionThi.docx
 
PHF nphpsp webinar pm framework 12.20.11
PHF nphpsp webinar pm framework 12.20.11PHF nphpsp webinar pm framework 12.20.11
PHF nphpsp webinar pm framework 12.20.11
 
Quality improvement healthcare final
Quality improvement healthcare finalQuality improvement healthcare final
Quality improvement healthcare final
 
Clingov5understandingaudit2003
Clingov5understandingaudit2003Clingov5understandingaudit2003
Clingov5understandingaudit2003
 
Paying for performance to improve the delivery of health interventions in LMICs
Paying for performance to improve the delivery of health interventions in LMICsPaying for performance to improve the delivery of health interventions in LMICs
Paying for performance to improve the delivery of health interventions in LMICs
 
Medical audit process
Medical audit processMedical audit process
Medical audit process
 
Improving Public Health Outcomes W Epm
Improving Public Health Outcomes W EpmImproving Public Health Outcomes W Epm
Improving Public Health Outcomes W Epm
 
Knowledge Translation: Practical Strategies for Success v1
Knowledge Translation: Practical Strategies for Success v1Knowledge Translation: Practical Strategies for Success v1
Knowledge Translation: Practical Strategies for Success v1
 
Quality improvement and patient safety in anesthesia
Quality improvement and patient safety in anesthesiaQuality improvement and patient safety in anesthesia
Quality improvement and patient safety in anesthesia
 
Standards Compliance And Quality Management
Standards Compliance And Quality ManagementStandards Compliance And Quality Management
Standards Compliance And Quality Management
 

Recently uploaded

About CentiUP - Product Information Slide.pdf
About CentiUP - Product Information Slide.pdfAbout CentiUP - Product Information Slide.pdf
About CentiUP - Product Information Slide.pdf
CentiUP
 
2024 Media Preferences of Older Adults: Consumer Survey and Marketing Implica...
2024 Media Preferences of Older Adults: Consumer Survey and Marketing Implica...2024 Media Preferences of Older Adults: Consumer Survey and Marketing Implica...
2024 Media Preferences of Older Adults: Consumer Survey and Marketing Implica...
Media Logic
 
Medicard presentation for companies 2024
Medicard presentation for companies 2024Medicard presentation for companies 2024
Medicard presentation for companies 2024
FrancescaAlainaDeGuz
 
contraindications in asana movement - during yoga practice
contraindications in asana movement - during yoga practicecontraindications in asana movement - during yoga practice
contraindications in asana movement - during yoga practice
Karuna Yoga Vidya Peetham
 
Mohali Call Girls 7742996321 Call Girls Mohali
Mohali Call Girls  7742996321 Call Girls  MohaliMohali Call Girls  7742996321 Call Girls  Mohali
Mohali Call Girls 7742996321 Call Girls Mohali
Digital Marketing
 
About CentiUP - Introduction and Products.pdf
About CentiUP - Introduction and Products.pdfAbout CentiUP - Introduction and Products.pdf
About CentiUP - Introduction and Products.pdf
CentiUP
 
Kolkata Call Girls 🔝 7374876321 🔝 Top Escorts Service Experiences With Fore...
Kolkata Call Girls 🔝 7374876321 🔝   Top Escorts Service Experiences With Fore...Kolkata Call Girls 🔝 7374876321 🔝   Top Escorts Service Experiences With Fore...
Kolkata Call Girls 🔝 7374876321 🔝 Top Escorts Service Experiences With Fore...
aadeshkumar4448
 
GORDON'S 11 FUNCTIONAL PATTERN-Health Assessment.pptx
GORDON'S 11 FUNCTIONAL PATTERN-Health Assessment.pptxGORDON'S 11 FUNCTIONAL PATTERN-Health Assessment.pptx
GORDON'S 11 FUNCTIONAL PATTERN-Health Assessment.pptx
Rommel Luis III Israel
 
Bashundhara Toiletries Logo Guideline 2024
Bashundhara Toiletries Logo Guideline 2024Bashundhara Toiletries Logo Guideline 2024
Bashundhara Toiletries Logo Guideline 2024
khabri85
 
ASSESSMENT OF THE SKIN, HAIR, AND NAILS.pptx
ASSESSMENT OF THE SKIN, HAIR, AND NAILS.pptxASSESSMENT OF THE SKIN, HAIR, AND NAILS.pptx
ASSESSMENT OF THE SKIN, HAIR, AND NAILS.pptx
Rommel Luis III Israel
 
Columbia毕业证书退学办理
Columbia毕业证书退学办理Columbia毕业证书退学办理
Columbia毕业证书退学办理
ozcot
 
Call Girls Goa 7023059433 Celebrity Escorts Service in Goa
Call Girls Goa 7023059433 Celebrity Escorts Service in GoaCall Girls Goa 7023059433 Celebrity Escorts Service in Goa
Call Girls Goa 7023059433 Celebrity Escorts Service in Goa
rajni kaurn06
 
Simple Steps to Make Her Choose You Every Day
Simple Steps to Make Her Choose You Every DaySimple Steps to Make Her Choose You Every Day
Simple Steps to Make Her Choose You Every Day
Lucas Smith
 
Luxury Massage Experience at Affordable Rate - Malayali Kerala Spa Ajman
Luxury Massage Experience at Affordable Rate - Malayali Kerala Spa AjmanLuxury Massage Experience at Affordable Rate - Malayali Kerala Spa Ajman
Luxury Massage Experience at Affordable Rate - Malayali Kerala Spa Ajman
Malayali Kerala Spa Ajman
 
Effect of Repeated low-level red light therapy in.pptx
Effect of Repeated low-level red light therapy in.pptxEffect of Repeated low-level red light therapy in.pptx
Effect of Repeated low-level red light therapy in.pptx
Manojitbasak4
 
Dr. Sherman Lai, MD — Guelph's Dedicated Medical Professional
Dr. Sherman Lai, MD — Guelph's Dedicated Medical ProfessionalDr. Sherman Lai, MD — Guelph's Dedicated Medical Professional
Dr. Sherman Lai, MD — Guelph's Dedicated Medical Professional
Sherman Lai Guelph
 
Unlimited Short Call Girls Thane ✅ 9833325238 FULL CASH PAYMENT
Unlimited Short Call Girls Thane ✅ 9833325238 FULL CASH PAYMENTUnlimited Short Call Girls Thane ✅ 9833325238 FULL CASH PAYMENT
Unlimited Short Call Girls Thane ✅ 9833325238 FULL CASH PAYMENT
rajesh344555
 
Faridkot ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 7742996321 ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 Faridkot
Faridkot ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 7742996321 ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 FaridkotFaridkot ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 7742996321 ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 Faridkot
Faridkot ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 7742996321 ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 Faridkot
varun0kumar00
 
Assessment of ear, Eye, Nose, and-Throat.pptx
Assessment of ear, Eye, Nose, and-Throat.pptxAssessment of ear, Eye, Nose, and-Throat.pptx
Assessment of ear, Eye, Nose, and-Throat.pptx
Rommel Luis III Israel
 
一比一原版布里斯托大学毕业证(Bristol毕业证书)学历如何办理
一比一原版布里斯托大学毕业证(Bristol毕业证书)学历如何办理一比一原版布里斯托大学毕业证(Bristol毕业证书)学历如何办理
一比一原版布里斯托大学毕业证(Bristol毕业证书)学历如何办理
obowu
 

Recently uploaded (20)

About CentiUP - Product Information Slide.pdf
About CentiUP - Product Information Slide.pdfAbout CentiUP - Product Information Slide.pdf
About CentiUP - Product Information Slide.pdf
 
2024 Media Preferences of Older Adults: Consumer Survey and Marketing Implica...
2024 Media Preferences of Older Adults: Consumer Survey and Marketing Implica...2024 Media Preferences of Older Adults: Consumer Survey and Marketing Implica...
2024 Media Preferences of Older Adults: Consumer Survey and Marketing Implica...
 
Medicard presentation for companies 2024
Medicard presentation for companies 2024Medicard presentation for companies 2024
Medicard presentation for companies 2024
 
contraindications in asana movement - during yoga practice
contraindications in asana movement - during yoga practicecontraindications in asana movement - during yoga practice
contraindications in asana movement - during yoga practice
 
Mohali Call Girls 7742996321 Call Girls Mohali
Mohali Call Girls  7742996321 Call Girls  MohaliMohali Call Girls  7742996321 Call Girls  Mohali
Mohali Call Girls 7742996321 Call Girls Mohali
 
About CentiUP - Introduction and Products.pdf
About CentiUP - Introduction and Products.pdfAbout CentiUP - Introduction and Products.pdf
About CentiUP - Introduction and Products.pdf
 
Kolkata Call Girls 🔝 7374876321 🔝 Top Escorts Service Experiences With Fore...
Kolkata Call Girls 🔝 7374876321 🔝   Top Escorts Service Experiences With Fore...Kolkata Call Girls 🔝 7374876321 🔝   Top Escorts Service Experiences With Fore...
Kolkata Call Girls 🔝 7374876321 🔝 Top Escorts Service Experiences With Fore...
 
GORDON'S 11 FUNCTIONAL PATTERN-Health Assessment.pptx
GORDON'S 11 FUNCTIONAL PATTERN-Health Assessment.pptxGORDON'S 11 FUNCTIONAL PATTERN-Health Assessment.pptx
GORDON'S 11 FUNCTIONAL PATTERN-Health Assessment.pptx
 
Bashundhara Toiletries Logo Guideline 2024
Bashundhara Toiletries Logo Guideline 2024Bashundhara Toiletries Logo Guideline 2024
Bashundhara Toiletries Logo Guideline 2024
 
ASSESSMENT OF THE SKIN, HAIR, AND NAILS.pptx
ASSESSMENT OF THE SKIN, HAIR, AND NAILS.pptxASSESSMENT OF THE SKIN, HAIR, AND NAILS.pptx
ASSESSMENT OF THE SKIN, HAIR, AND NAILS.pptx
 
Columbia毕业证书退学办理
Columbia毕业证书退学办理Columbia毕业证书退学办理
Columbia毕业证书退学办理
 
Call Girls Goa 7023059433 Celebrity Escorts Service in Goa
Call Girls Goa 7023059433 Celebrity Escorts Service in GoaCall Girls Goa 7023059433 Celebrity Escorts Service in Goa
Call Girls Goa 7023059433 Celebrity Escorts Service in Goa
 
Simple Steps to Make Her Choose You Every Day
Simple Steps to Make Her Choose You Every DaySimple Steps to Make Her Choose You Every Day
Simple Steps to Make Her Choose You Every Day
 
Luxury Massage Experience at Affordable Rate - Malayali Kerala Spa Ajman
Luxury Massage Experience at Affordable Rate - Malayali Kerala Spa AjmanLuxury Massage Experience at Affordable Rate - Malayali Kerala Spa Ajman
Luxury Massage Experience at Affordable Rate - Malayali Kerala Spa Ajman
 
Effect of Repeated low-level red light therapy in.pptx
Effect of Repeated low-level red light therapy in.pptxEffect of Repeated low-level red light therapy in.pptx
Effect of Repeated low-level red light therapy in.pptx
 
Dr. Sherman Lai, MD — Guelph's Dedicated Medical Professional
Dr. Sherman Lai, MD — Guelph's Dedicated Medical ProfessionalDr. Sherman Lai, MD — Guelph's Dedicated Medical Professional
Dr. Sherman Lai, MD — Guelph's Dedicated Medical Professional
 
Unlimited Short Call Girls Thane ✅ 9833325238 FULL CASH PAYMENT
Unlimited Short Call Girls Thane ✅ 9833325238 FULL CASH PAYMENTUnlimited Short Call Girls Thane ✅ 9833325238 FULL CASH PAYMENT
Unlimited Short Call Girls Thane ✅ 9833325238 FULL CASH PAYMENT
 
Faridkot ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 7742996321 ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 Faridkot
Faridkot ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 7742996321 ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 FaridkotFaridkot ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 7742996321 ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 Faridkot
Faridkot ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 7742996321 ℂ𝕒𝕝𝕝 𝔾𝕚𝕣𝕝𝕤 Faridkot
 
Assessment of ear, Eye, Nose, and-Throat.pptx
Assessment of ear, Eye, Nose, and-Throat.pptxAssessment of ear, Eye, Nose, and-Throat.pptx
Assessment of ear, Eye, Nose, and-Throat.pptx
 
一比一原版布里斯托大学毕业证(Bristol毕业证书)学历如何办理
一比一原版布里斯托大学毕业证(Bristol毕业证书)学历如何办理一比一原版布里斯托大学毕业证(Bristol毕业证书)学历如何办理
一比一原版布里斯托大学毕业证(Bristol毕业证书)学历如何办理
 

M Heenan_PhD Dissertation Lecture_eHealth Lecture_Engaging Leaders in KPI Selection_Final_July 2023

  • 1. Health care indicator selection processes: How to select engaging metrics to motivate front-line managers. Dr. Michael A. Heenan, PhD McMaster University, eHealth Summer Program PhD Dissertation Based Presentation July 26, 2023
  • 2. Mike Heenan, Ph.D. • Hospital Executive Vice-President • Sessional Lecturer, McMaster University – DeGroote School of Business • PhD, Business Administration, Health Policy Management, 2023 • Thesis research: how health care organizations selected key performance metrics and how the process of indicator selection impacts front line managers motivation to improve performance. 2
  • 3. Session Format 3 eHealth – Why this Lecture? The Literature and Our Problem – Metric Madness Hospital Executive Perception of Front-Line Manager’s Knowledge What Indicators Motivate Front-Line Clinical Managers? What did the pandemic teach us? The 5-P Indicator Selection Process Framework The Politics of Improvement: Your Challenge as eHealth Leaders
  • 4. eHealth – Why this Lecture? 4
  • 5. eHealth – Why this Lecture? Your program objectives: 1. Understand eHealth concepts 2. Communicate effectively in eHealth contexts 3. Apply eHealth Knowledge in practical contexts 5
  • 6. Key Readings We will Reference • Damschroder LJ, Robinson CH, Francis J, et al. Effects of Performance Measure Implementation on Clinical Manager and Provider Motivation. J Gen Intern Med. 2014; 29(S4):877-884. DOI: 10.1007/s11606-014- 3020-9. • Elliot C, Mcullagh C, Brydon M, Zwi K. Developing key performance indicators for a tertiary children’s hospital network. Australian Health Review. 2018; 42(5):491-500. DOI: 10.1071/AH17263. • Ginsburg LS. Factors that Influence Line Managers’ Perceptions of Hospital Performance Data. Health Services Research. 2003; 38(1):261-286. DOI: 10.1111/1475-6773.00115. • Heenan MA, Randall GE, Evans JM. Selecting Performance Indicators and Targets in Health Care: An International Scoping Review and Standardized Process Framework. Risk Management and Healthcare Policy. 2022; 15:747-764. DOI: 10.2147/RMHP.S357561. • Mannion R, Braithwaite J. Unintended consequences of performance measurement in health care: 20 salutary lessons from the English National Health Service. Internal Medicine Journal. 2012; 42(5):569–574. DOI: 10.1111/j.1445-5994.2012.02766.x 6
  • 7. To Start … Two Key Definitions Performance Measurement is defined as the collection, use and public reporting of data for the purpose of quality improvement, accountability, and transparency. America Academy of Family Physicians Performance Measurement Criteria. [Internet] 2022. [Cited 27 Nov 2022]. Available from: https://www.aafp.org/about/policies/all/performance-measures.html Indicators are defined as measurable elements of practice performance that relate to clinical, population health, financial, or organizational performance. Lawrence M, Olesen F. Indicators of Quality in Health Care. The European Journal of General Practice. 1997; 3(3):103-108. DOI: 10.3109/13814789709160336 7
  • 8. To Start … Indicators are Metrics, Metrics are Indicators Indicators also known as • Key Performance Indicators • Performance Indicators • Quality Indicators • Performance Metrics • Performance Measures • Quality Metrics 8
  • 9. To Start … Donabedian Framework Framework for evaluating health care service quality. Information on quality of care can be seen through three types of indicators: Structure: factors that affect the context in which care is delivered Process: actions the describe how care is delivered Outcome: effects of healthcare delivery on patients or populations, including changes to health status, behavior, or knowledge as well as patient satisfaction and health-related quality of life 9 Donabedian A. The quality of care: how can it be assessed? JAMA. 1988 Sep 23; 260(12):1743-8. DOI: 10.1001/jama.1988.03410120089033.
  • 10. The Literature and Our Problem – Metric Madness 10
  • 11. Needed Measurement Led to Too Much Measurement • USA, 1999 - Approximately 98,000 Americans lost their lives each year due to medical error. • Canada, 2004 - Average of 7.5% of hospital admissions resulting in an adverse event. • Multiple governments, regulatory agencies, and funding bodies have mandated the collection and monitoring of hundreds of KPIs by hospitals 11 Kohn LT, Corrigan JM, Donaldson MS. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press, 2000. Baker GR, Norton P, Flintoff V, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. Canadian Medical Association Journal. 2004; 170(11):1678-1686. DOI: 10.1503/cmaj.1040498
  • 12. Needed Measurement has Led to Too Much Measurement 12 USA: • Centre for Medicare and Medicaid Services (CMS) monitors over 1700 KPIs • National Quality Forum (NQF) approved KPIs grew from 200 in 2005 to over 700 in 2011 • Study concluded over 550 distinct KPIs were being collected across 48 states Canada Ontario Hospital Association and Health Quality Ontario argue there are over 300 quality KPIs reported by hospitals. International Over-measurement has been echoed in the United Kingdom and Australia (Mannion and Braithwaite 2011). Panzer RJ 2013; Wilensky G 2018; Meyer GS 2012; Greenburg A 2019; Mannion R 2012;
  • 13. The Management Problem: Overabundance of KPIs has resulted in unintended consequences 13 • Data mandated by state bodies do not necessarily reflect every local context and may be used inappropriately • Public reporting of KPIs without local input has resulted in a lack of trust between providers and political bodies, and gaming • Reporting mandates have also increased financial costs to organizations • Building of the information technology and data infrastructure required to support measurement has amplified the amount of data available to health care organizations Mannion R and Braithwaite 2011; Safavi 2006; Teare 2014; Meyer et al 201, Berwick 2016
  • 14. Perla: The U Curve Paralyzed decision making (Cassel et al 2014, Perla 2013). 14 Perla R. Commentary: Health Systems Must Strive for Data Maturity. American Journal of Medical Quality. 2013; 28(3):263-264. DOI:10.1177/1062860612465000.
  • 15. Call for Balance and Parsimony 1. Measure process quality 2. Measure value 3. Design data systems for internal quality needs and spinoff external quality measures 4. Use return on investment 5. Establish ongoing process to refine and select core measures 15 Meyer GS, Nelson EC, Pryor DB, et al. More quality measures versus measuring what matters: a call for balance and parsimony. BMJ Qual Saf. 2012; 21(11):964-968. DOI: 10.1136/bmjqs-2012-001081.
  • 16. Question # 1 How does eHealth further hinder the phenomenon of over- measurement? 16
  • 17. A Call for Balance, but… Not Directly Linked to Goal Theory Some agencies have heeded these calls. • Institute of Medicine (2015) recommends 15 national KPIs that systems should monitor (Blumenthal and McGinnis 2015). • National Quality Forum (2019) instituted annual review process that endorses certain KPIs for adoption to ensure their relevancy to overall system goals. (NQF 2019) • World Health Organization started examining processes in 2008 and offered criteria to narrow KPI selection (Smith et al 2008). • Research papers describe Delphi methods used to select KPIs in clinical areas (Schull et al 2011, Madsen et al 2016, Harvey 2016, Murphy 2016). These reports describe criteria and methods to select an appropriate number of quality and patient safety KPIs at the system or clinical service level, but… a) They do not clarify who should be involved in KPI and Target selection, or, b) how health service provider organizations like hospitals should use selection processes as an instrument to motivate leaders to achieve agreed upon goals. 17
  • 18. Hospital Executive’s Perception of Front-Line Manager’s Knowledge 18
  • 19. What processes do hospitals use to select performance indicators and do they align with best practices? A multiple-case study of four hospitals in Ontario, Canada Qualitative, multiple-case study of four acute care hospitals and how they select indicators Results • Process elements largely absent include: • adopting evidence-based selection criteria • considering finance and human resources indicators; • considering if indicators measure structure, process or outcomes, • engaging a broader set of end-users in the selection process. • Revealed senior management teams had little confidence in clinical unit managers’ ability to use indicators to improve performance. 19
  • 20. What Indicators Motivate Front-Line Managers? 20
  • 21. Who are and why do we care about Front-Line Managers? Who • Clinical unit / Front-Line managers are employees who are both supervised by an organization’s top managers and who themselves supervise front-line employees.10 Why • Managers’ commitment and ability to implement innovative change at the unit level has been linked to strategy realization, efficiency of operations, cost control, and overall achievement of quality outcomes.10-12 • If positively motivated and engaged, Managers can effectively support innovation and change. If managers are disengaged and not motivated, it can impede improvement in the units they oversee.13-15 • Managers’ motivation and commitment increases when their senior managers support them with the infrastructure and resources necessary for change.14,16 21
  • 22. Managers and Goal-Setting Theory of Motivation Goal-Setting Theory of Motivation emphasizes the relationship between goals and performance. Task performance is influenced by the manager’s ability; the stated goal’s content, intensity, and duration; the manager’s self-efficacy; and the personal goals of the manager. Key Attributes: 1. Benefit of participating in goal setting processes is cognition on what’s important and why; 2. Specific goals with harder to reach improvements result in a greater effort by managers; 3. Regular feedback increases performance effort by improving manager’s self-efficacy. 4. Managers that feel supported by their leadership have improved self-efficacy; 5. Managers that feel part of a larger team improve effort through both a shared understanding of the goal’s direction and shared-accountability; and, 6. Managers increased confidence leads to greater motivation to act and perform tasks. 22 Locke EA, Latham GP. Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist. 2002; 57(9): 705–717. DOI: 10.1037//0003-066x.57.9.705
  • 23. Need to understand base motivations: Why clinicians become managers • Clinical managers never thought of becoming a manager when completing their initial clinical education. • Front-line clinicians move into management to help improve quality and the working conditions of their fellow staff. • When asked what indicators they prefer clinical unit managers referenced process indicators that help advance clinical practice, and front-line staff engagement as criteria by which they would choose indicators. 23 Kleinman CS. Leadership Roles, Competencies, and Education: How Prepared Are Our Nurse Managers? Journal of Nursing Administration. 2003; 33(9):451-455. DOI: 10.1097/00005110-200309000-00005. White J. Motivation for Seeking a Nurse manager Position. [dissertation]. San Diego, CA: University of San Diego, 2015. Available from: https://digital.sandiego.edu/dissertations/17
  • 24. Why do managers use performance data? 24 Ivankovic D, Poldrugovac M, Garel P, Klazinga NS, Kringos DS (2020) Why, what and how do European healthcare managers use performance data? Results of a survey and workshop among members of the European Hospital and Healthcare Federation. PLoS ONE 15 (4): e0231345. https://doi.org/10.1371/journal.pone.0231345
  • 25. What barriers do managers face in using data? • Managers gave three barriers: lack of training and knowledge on measurement, workload, and job experience. • Managers need to obtain is the understanding, interpretation, and use of performance data for evidence-based decision making. • Clinicians are often appointed to managerial roles without business skills, and they therefore need to learn them through trial and error. 25 “My education is based on how to be a nurse, not how to decipher data and know what it means. So, some of this stuff is foreign to me and it’s been trial by fire for me.” Case B-Manager 3 Birken SA, Lee SYD, Weiner BJ. 2012; DOI: 10.1186/1748-5908-7-28. Naranjo-Gil D. 2009; DOI: 10.1097/HMR.0b013e31819e8fd0. Kleinman CS. 2003; DOI: 10.1097/00005110-200309000-00005. McCallin AM, Frankson C. 2010; 1DOI: 10.1111/j.1365-2834.2010.01067.x Heenan, MA. 2023. (Doctoral dissertation). “I have not received any training or education on measurement here. When I started in the role, it was kind of just sink or swim.” Case A-Manager 3
  • 26. How clinical unit managers’ roles in selecting hospital indicators impacts their motivation and self-efficacy to improve performance: A qualitative multiple-case study. Qualitative multiple-case study of four hospitals involving 22 managers. Semi-structured Interviews. Thematic Analysis. Results: • Managers reported being unaware of the criteria hospitals used to select indicators, are not trained in measurement, feel they have little control over the indicators they were accountable for, and were fearful of reporting on performance. • Managers motivated by process indicators focused on clinical quality and patient experience vs. outcome or business indicators. • Managers gained confidence over time and by learning how to communicate the utility of indicators to front-line staff. 26 Heenan, M. A. (2023). Health Care Indicator Selection Processes and their Impact on Clinical Unit Managers’ Motivation and Self-Efficacy to Improve Performance (Doctoral dissertation).
  • 27. Ginsburg: Factors that Influence Line Managers’ Perceptions of Hospital Performance Data 27 Ginsburg LS. Factors that Influence Line Managers’ Perceptions of Hospital Performance Data. Health Services Research. 2003; 38(1):261-286. DOI: 10.1111/1475-6773.00115 “Simply generating performance data will not, in and of itself, lead to improvement.” Factors Significant Correlation to Perceived Usefulness of Data • Data quality • Data relevance • Report complexity • Intensity of dissemination
  • 28. Damschroder: Effects of Performance Measure Implementation on Manager Motivation • Indicators need to be designed to both incent individuals and build coalitions. • Indicators sit within a QI foundation but only effective if seen positively by fully engaged staff and management. 28 Damschroder LJ, Robinson CH, Francis J, et al. Effects of Performance Measure Implementation on Clinical Manager and Provider Motivation. J Gen Intern Med. 2014; 29(S4):877-884. DOI: 10.1007/s11606-014-3020-9.
  • 29. Question # 2 Now knowing the motivations and challenges end-users have in using data, how can eHealth help the phenomena of over- measurement? 29
  • 30. What did the pandemic teach us? 30
  • 31. What did the pandemic teach us? Executive Perspectives • Senior management leaders mentioned that some clinical practice, supply chain, and human resource indicators, once thought not important to regular monitoring, had surfaced as priorities. 31 Heenan MA. Health Care Indicator Selection Processes and their Impact on Clinical Unit Managers’ Motivation and Self-Efficacy to Improve Performance (Doctoral dissertation). “COVID taught us that after years of trying to get the organization to make data-driven decisions that often fell on deaf ears, the organization was thirsty for data about this new disease. This demand for data was likely a result of operating in a new unknown environment and the need to base operational decisions on emerging data compared to traditional lagging indicators. COVID also taught us that we likely had been providing the wrong data before the pandemic. COVID forced us to prioritize our efforts and stopped us from chasing too many indicators. It led us to provide more leading process-based indicators that clinical units found useful in planning their day versus lagging outcome data that government wanted.” Case B-Executive 2
  • 32. What did the pandemic teach us? Front-Line Manager Perspectives Clinical unit managers uniformly shared that the pandemic: • created a single priority for their organizations; • involved greater engagement in, and lead to, faster decision making; • Focus on process indicators related to practice and staff safety. • Example, new indicator monitored more closely – personal protective equipment (PPE) Inventory Levels 32 “COVID has changed everything. I've seen senior leader sentiment sort of evolve from when people are being told to do things versus asking us why we're doing them. Senior leaders have become clinically focused. We see them maybe a little bit better for who they are now because the data is not being shoved down our throats. We are being asked if certain data makes sense to us. My VP is just, yep, whatever you need, so I get that support, which makes a huge difference.” Case A-Manager 4. Heenan MA. Health Care Indicator Selection Processes and their Impact on Clinical Unit Managers’ Motivation and Self-Efficacy to Improve Performance (Doctoral dissertation).
  • 33. Lessons 1. Crises create opportunities to see new needs and new ways 2. Decision making structures get flatter, relationships gets stronger, because you engage the end-user deliberately and quickly 3. Process metrics matter and help inform priority actions 4. Managers want to be engaged to better understand their accountabilities and help select process indicators that measure quality, clinical practice, patient experience and staff engagement 33
  • 34. In designing or selecting indicators, what framework should guide us? The 5-P Indicator Selection Process Framework 34
  • 35. Selecting performance indicators and targets in health care: An international scoping review and standardized process framework Purpose: Synthesize international approaches to indicator selection and develop a standardized process framework. Methods: • PRISMA-ScR Scoping review using Pub Med & Web of Science. • English-language papers from 11 countries, 2010-2020. • Reviewed 33 peer-reviewed papers and 11 grey-literature documents. • Looked at processes with a lens to: Aim; Governance; Preparation; Process Methods; Validation Results: • Some processes used Appraisal of Indicators through Research and Evaluation (AIRE) Instrumentand the Quality Indicator Critical Appraisal (QICA) tool to guide which individual indicators should be considered. • No standardized indicator selection process framework exists that focuses on engagement that solicits buy in and drives motivation. 35 Heenan MA, Randall GE, Evans JM. Selecting Performance Indicators and Targets in Health Care: An International Scoping Review and Standardized Process Framework. Risk Management and Healthcare Policy. 2022; 15:747-764.
  • 36. The 5-P Indicator Selection Process Framework Incomplete indicator selection processes risk: o over-measurement, o lack of prioritizing strategic and operational goals, o lack of support by end-users o paralyzed decision-making ability. • Standardized framework developed to assist organizations complete a comprehensive indicator and target selection process. 36 Domain Element Clarify Aim Develop Guiding Principles Identify Level of Use Build Governance Structures Recruit Participants Seek End-User Input Research Evidence-Based Literature Build an Inventory of Potential Indicators Categorize Potential Indicators into Strategic Orient and Train Participants Utilize a Consensus Building Method Identify a Facilitator Indicator Selection Criteria Analytical Assess Indicators Set Indicator Targets Assess Data Quality Validate with End-Users 5-P Indicator Selection Process Framework Prepare Procedure Prove Purpose Polity Heenan, MA, Randall GE, Evans JA. Selecting performance indicators and targets in health care: An international scoping review and standardized process framework. Risk Management and Healthcare Policy. 2022; 15:747-764. DOI: 10.2147/RMHP.S357561.
  • 37. Domain: Purpose 37 “Purpose” sets out the reasons why an indicator selection and target setting process is undertaken – facilitating a shared understanding of what the organization is setting out to achieve. Elements • Clarify Aim • Set Guiding Principles • Identify Level of Use
  • 38. Purpose: Three Elements 38 Watkins: Setting clear aims and guiding principles allows participants to understand the desired goals a process is aiming to achieve and the actions to which they are expected to contribute. Watkins MD. Demystifying Strategy: The What, Who How and Why. Harvard Business Review. September 10, 2007. Available from: https://hbr.org/2007/09/demystifying-strategy-the-what
  • 39. Elliot, 2018 39 Elliot C, Mcullagh C, Brydon M, Zwi K. Developing key performance indicators for a tertiary children’s hospital network. Australian Health Review. 2018; 42(5):491-500. DOI: 10.1071/AH17263. Aim: Following merger ad strategic plan, to address executive level performance indicators for pediatric tertiary care services across two campus Guiding Principles: KPIs must address the mission statement, “working in partnership to improve heath and well being of children through clinical care, research education and advocacy”. Drive quality improvement in child health outcomes Level of Use: Executive Level / Board
  • 40. Domain: Polity Elements • Build Governance Structures • Recruit Participants 40 “Polity” identifies the governance structures that manage the selection process, how the process will be resourced, and who participates.
  • 41. Polity: Two Elements 41 Aktaa: Expert panels have inherent bias. Therefore, expansion of participants is important mitigation. Indicator selection processes that have broader participation are more likely than others to have a more inclusive view of front-line operations Aktaa S, Batra G, Wallentin L, et al. European Society of Cardiology methodology for the development of quality indicators for the quantification of cardiovascular care and outcomes. European Heart Journal Quality of Care and Clinical Outcomes. 2022; 8(1):4-13. DOI: 10.1093/ehjqcco/qcaa069.
  • 42. Example: Elliot, 2018 42 Elliot C, Mcullagh C, Brydon M, Zwi K. Developing key performance indicators for a tertiary children’s hospital network. Australian Health Review. 2018; 42(5):491-500. DOI: 10.1071/AH17263. Governance Structure Single body - Executive Group Participants / Inputs: Network Executives; Clinical Directors; Medical, Allied Health and Nurse Managers.
  • 43. Domain: Prepare Elements • Seek End-User Input • Research Evidence Base Literature • Build an Inventory of Indicators • Categorize Indicators into Strategic Themes • Orient and Train Participants 43 “Prepare” addresses how to plan for selection.
  • 44. Prepare: Five Elements 44 Heenan • Final list of indicators built from comprehensive sources improves their relevancy to end-users while enabling future comparability and benchmarking. • End-user input upfront on indicator knowledge and user requirements and issued orientation materials increased participant engagement and improved understanding of the process among participants.
  • 49. Domain: Procedure Elements • Utilize a Consensus Building Methodology • Identify a Facilitator • Set Indicator Selection Criteria • Analytically Assess Indicators • Set Indicator Targets 49 “Procedure” describes the how to steps used to assess indicators and targets and gain consensus.
  • 50. Procedure: Five Elements 50 Stetler: Internal facilitation may bias processes whereas external facilitators have an ability to play both a problem solving and supportive role that might otherwise be difficult when only internal parties are engaged. External facilitation is valued when a process is focused on an interactive problem or distinct activity. Stetler CB, Legro MW, Rycroft-Malone J, Bowman C, Curran G, Guihan M, Hagedorn H, Pineros S, Wallace CM. Role of external facilitation in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science. 2006; 1(1):1-5. DOI: 10.1186/1748-5908-1-23.
  • 52. Domain: Prove Elements • Assess Data Quality (Quantitatively) • Validate with End-Users (Qualitatively) 52 “Prove” describes the validation processes used to test any final set of indicators for data quality and relevance with end-users.
  • 53. Prove: Two Elements 53 Heenan • Processes that statistically test indicators increase scientific soundness and better informs target setting. • Processes that validate a final list of indicators with end-users improve relevance and usability by users, especially in cases where the expert panels did not include front-line directors, managers, or patients. Heenan, MA, Randall GE, Evans JA. Selecting performance indicators and targets in health care: An international scoping review and standardized process framework. Risk Management and Healthcare Policy. 2022; 15:747-764. DOI: 10.2147/RMHP.S357561.
  • 54. Examples: Elliot, 2018 and Heenan, 2023 54 QUALITATIVE: Noted Limitation (Elliot) “There was little or no consultation with less senior staff members, which risks losing the support of staff who will be relied upon to provide data and ensure accurate reporting” QUANTITATIVE: 2022 Case Study of 4 Ontario Hospitals (Heenan) • None quantitatively validated KPIs - reasoned most of the indicators they monitor are mandated by external funding agencies, and it is therefore assumed these indicators were previously validated. • This is a false assumption. • While the technical formula of an indicator may have been validated by an external agency, the data that is generated from the different local hospital information systems may be of different quality than the data by which an indicator was tested by the external agency. (e-heath)
  • 55. Key Conclusion Recall • Overabundance of indicators has paralyzed decision making, and eroded trust between those who ask for indicators and those who are expected to use them to make change. • Many policy institutes and academics have called for a more appropriate, lower number of indicators. Therefore • Indicator selection or reduction processes cannot occur by happenstance. • The 5-P Indicator Selection Process Framework to ensure a systematic, evidence- based, and inclusive approach that engages measurement experts and those who use indicators to monitor and improve performance in both selection and validation. 55
  • 56. Question # 3 What domain and element of the 5-P Indicator Selection Process Framework can eHealth most help with? Why? 56
  • 57. The politics of improvement: Your challenge as eHealth Leaders 57 Process improvement is not only about adopting evidence but includes the process of engaging those responsible for implementing the prescribed change
  • 58. Beyond the evidence (or technology): You’re not just an eHealth Leader but a Change Leader 58 • Quality Improvement initiatives are not simple • Evidence is insufficient in itself to ensure implementation Distributed Nature of Costs and Benefits Variety of Value Systems Underlying Interventions What’s in it ? For Whom ? What ideologies and ethics are involved ? Langley A, Denis J. Beyond evidence: the micropolitics of improvement. BMJ Quality and Safety. 2014; 20:i43-i46. DOI: 10.1136/bmjqs.2010.046482.
  • 59. Hard Core and Soft Periphery Hard core = Evidence that is irreducible and carries key benefit. (The Technical What) Soft Periphery = Complex delivery systems that take different forms and if poorly organized may destroy the benefit of the hard core. (The Cultural Why & People) 59
  • 60. Navigate: The Micro-Political / Change Leader Check List 1. Understand the evidence 2. Identify who it will impact 3. Understand how it will change practice 4. Don’t make change alone: find a champion(s) 5. Solicit input on how to adopt locally 6. Sell the why (involve end-users) 7. Teach: Be a coach not a consultant 60
  • 62. Your Program Objectives: You hold knowledge and power, Use it Wisely 1. Understand eHealth Concepts • The technology and systems you build and promote will be used to make change by producing information and data. • This information and data (indicators) can overwhelm the end- users not aware of eHealth concepts. 2. Communicate Effectively in eHealth Contexts • Ask why are we producing the information? 3. Apply eHealth Knowledge in practical contexts • What will it be used for? And by Whom? 62
  • 63. Final Conclusions Answering … “How to select engaging performance metrics?” 1. Have a Structured, Open, Transparent Process 2. Include End-Users as Participants 3. Educate and Train participants on concepts 4. Allow Input on Past Experience before Selection 5. People Process Things to get to Outcomes so Process Metrics Matter 6. Validate Data in Real Environments 63
  • 64. Abd then don’t forget …The U Curve 64 Perla R. Commentary: Health Systems Must Strive for Data Maturity. American Journal of Medical Quality. 2013; 28(3):263-264. DOI:10.1177/1062860612465000.
  • 66. Dissertation Reference 66 Heenan, M. A. (2023). Health Care Indicator Selection Processes and their Impact on Clinical Unit Managers’ Motivation and Self-Efficacy to Improve Performance (Doctoral dissertation). https://macsphere.mcmaster.ca/bitstream/11375/28348/2/Heenan_Michael_A_20 2302_PhD.pdf