This document discusses public expectations of regulatory revalidation of clinicians and whether revalidation can deliver on those expectations. It notes that public expectations are modest, focusing more on access to care than quality or outcomes. It also discusses different definitions of competency and the complexity of problems in healthcare. While revalidation aims to maintain competency, it has limitations as practice is continuous but revalidation is periodic. The document suggests that for revalidation to meet public expectations, it would need to take a systems approach and include organizational assessments in addition to individual assessments. It also raises the possibility of alternative approaches to evaluation that focus more on intrinsic motivation and attitudes rather than just knowledge and skills.

1. Can Revalidation Deliver
What the Public Expects?
IAMRA Revalidation Symposium
Montreal, October 29, 2015
1
Steven Lewis
Access Consulting Ltd., Saskatoon
Adjunct Professor of Health Policy
Simon Fraser University
slewistoon1@gmail.com

2. Public Expectations Are Modest
 No research on public expectations of regulation
 Public tends to assume that:
Almost all clinicians are competent
Their clinicians are competent even if others’ aren’t
Incompetence is revealed by adverse events
Access and relationships are primary
 We are cognitively unprepared to deal with the safety and
quality problems that plague health care
2

3. Framing Competency
3
 Competency is a continuum and there is no universally
standardized definition
 If competency is ends-driven, it depends on the ends
 Basic safety
 Reasonable command of evidence-based practice
 Ability to meet typical needs
 Ability to meet complex needs
 Self-management
 A minimalist version of competency reflects regulatory history
 A more robust version of competency is a game-changer

4. So How Competent Is Health Care?
 It’s excellent
The public tends to revere doctors
High degree of trust in local institutions
Main concern is access, not quality or outcomes
 It’s terrible
Every major study identifies huge toll of harm and death
Polypharmacy and practice variations are rampant
Chronic disease management and mental health are disaster
zones
4

5. 5
Complexity of Problems and Needs
Regulatory
Intensity
Needed to
Ensure
Competency

6. The Limits of Revalidation
 Practice is continuous; revalidation is periodic
 Difficult to capture the full range of performance
 Invariably small sample sizes if chart reviews
 Role of individual competencies varies depending on complexity
of needs and care plans
 Confirmation bias and implicitly high bar for taking action
6

7. What the Public (Should) Expect
of Revalidation
 A systems approach to maintaining excellence
 Effective early warning mechanisms
 Standards linked to ability to meet more complex needs
 Combination of individual and organizational assessments
 Strategy for communicating areas of deficiency with patients
 Self-rating mechanisms for clinicians
 Don’t recertify anyone you wouldn’t send your family member to
7

8. The Revalidation Paradox
 If revalidation is virtually automatic, it will do little to improve
practice
 If revalidation is too onerous, there will be tremendous pushback
from physicians and patients
 It is difficult to reconcile a summary judgment (however
qualified) with a process designed to solve problems
 However, the mere existence of revalidation may result in some
people self-selecting out of the pool and others taking
continuing competency more seriously
8

9. Thinking Outside the Evaluative Box
 The holy grail of evaluation is to find low burden, simple, easily
available, and cheap proxies that correlate with performance
 What measure proved to predict the recent Canadian election
results most accurately?
 How do social media assessments of practice correlate with
formal review assessments?
 What can we learn retrospectively from poor performance
debacles – was it failure to see; failure to infer; failure to act?
9

10. Intrinsic vs. Extrinsic Motivation
 Performance assessment as we know it focuses on what doctors know
and do
 Alternate hypothesis: performance is at least as much a function of
what doctors are
 Humble and healthy dose of self-doubt
 Willingness to subordinate tradition and comfort level to evidence
 Motivated knowledge-seeker
 Good medical citizen who views improvement as a collective and
collaborative responsibility
10

11. Implications for Revalidation
 Consider testing for risky psychological profiles (narcissism,
over-confidence, hierarchical predisposition, etc.)
 Measure attitudes toward care pathways, practice variation,
standardization of protocols
 Less emphasis on quiz performance – in today’s world “just in
time” knowledge is easily accessible (to patients too)
 Test for collegiality, comfort level with interdependence,
sensitivity to heightened risks
 Interview on responses to own experience of failure - do they
deny/explain away/learn/adapt
11

12. The Desired Future State
 Health science education would be redesigned to focus on chronic
disease management, frailty, and mental health
 Competency would be layered and clinicians would be certified for
different needs just as mechanics are certified to work on different cars
 Regulation would be more about aspiration and influence and less
about formal processes and accounting
 Regulation would be less about individuals and more about
environments
 Blunt instruments would give way to multi-party interactions and
processes that invest in clinicians’ continuous well-being
12

13. Regulating vs. Influencing:
Success Is A Collective Effort
13
EXCELLENCE
Organizational
Culture
Clinical Policy Accredita-
tion
Health IT/
Analytics
Peer Review
Accountability
Legislation
Regulation

14. Revalidation: An Interim Approach?
 Imagine a health care world where
Health information technology was fully developed
Real-time measurement was valid and automated
Organizations routinely implemented QI theory and practice
Clinical autonomy was viewed as risk factor
 The role of initial and recertified credentials would diminish
 The need for revalidation may simply be proof that others are
not yet doing what needs to be done
 The sociology of medicine may be the single most important
factor in system-wide improvement and collective responsibility
14

15. 15
rPperformanp[eeREVAL