Closing the Gap Between Clinician Education and Quality Improvement Through an Evidence-Based Taxonomy That Links Terms and Interventions: A Two-Part Brainstorming Session for the Alliance for CEhp Quality Improvement Education (QIE) Initiative

1. Closing the Gap Between Clinician Education and
Quality Improvement Through an Evidence-Based
Taxonomy That Links Terms and Interventions:
A Two-Part Brainstorming Session for the Alliance for
CEhp Quality Improvement Education (QIE) Initiative
August 20 - 21, 2014
Sandra Haas Binford, M.A.Ed.
Kathleen Geissel, Pharm.D.
Co-Leaders of the ACEhp QIE Domain on
Taxonomy, Terms, & Interventions

2. Closing the Gap Between
Clinician Education and Quality
Improvement Through an
Evidence-Based Taxonomy
That Links Terms and
Interventions
Agenda
Introductions
Overview of Project
Brainstorm Taxonomy
● Define scope
● Identify resources
● Recommendations
Brainstorm Interventions
● Define scope
● Identify resources
● Recommendations

3. Overview of Project
Alliance for Continuing Education in the Health Professions (ACEhp)
Quality Improvement Education (QIE) Initiative

4. What is QIE? Incorporation of education tools
and techniques into quality improvement
activities across the U.S. healthcare delivery
system
Why is the Alliance doing this? We are the
education network helping to change inter-
professional behavior that results in
measurable systems-change & QI in care
delivery and population health…
...and we uncovered a major gap between
Educators and QI Leaders
Where are we going? The QIE “Vision” is that
education will be best used by both teams and
individual practitioners to drive QI and system
efficiencies in an ongoing paradigm shift
towards quality outcomes
Overall Goals of the Alliance Quality Improvement
Education (QIE) Initiative

5. Knowing (Education)
Doing (QI)
Current
Situation
Not a
Desirable
Direction -
Don’t forget
the individual
Yes:
Implement
the plan for
Individuals
through
Organizations
Individuals Teams Organizations
Paradigm Shift: Transition to QIE

6. ● Expand the focus of the Alliance beyond certified CME;
● Broaden the focus of the Alliance to include healthcare related continuing
education (CE) and continuing professional development (CPD);
● Be visionary and proactive in strategically aligning the Alliance with the
emerging healthcare environment while remaining sensitive to current
needs;
● Model innovative leadership to shape the future;
● Actively develop strategic relationships with other healthcare
organizations;
● Build a strategy to be the recognized voice for CE in the health
professions;
● Be the professional home for health professionals in continuing
education; and
● Advance the field of CEhp through research and other scholarly
activities.
8 Alliance for CEhp Strategic Initiatives

7. Strategy: Develop a Roadmap Report for QIE Initiative
● Advisory Panel (AP) established to author a Roadmap Report for 10
Domains
● AP is supported by a Commentary Group who provide guidance and
feedback on the strategy laid out in the Report. Commenters will “bring the
outside in” with healthcare leaders, adding input through a structured
interview and document review process.
Our role: Domain Leadership
● Advisory Panel (AP) is also supported by a group of Domain Leaders
who are Alliance experts to help validate that the Recommendations are
articulated in an actionable manner for educators.
Overall Strategy and Domain Action Items

8. Development of Roadmap Report
● Domain Leaders submit draft Domain descriptions and recommendations
on August 25, 2014.
● Draft QIE Roadmap Report at Alliance Quality Symposium in Baltimore,
September 23 - 25, 2014.
• Presented by Lou Diamond as Keynote, with additional Keynotes by Patrick Conway of
CMMS and Chris Cassel of the NQF.
• Domain Leaders to run Workshops to gather additional feedback from QI leaders and
ACEhp members.
● Final QIE Roadmap Report at Alliance Annual Conference in Dallas,
January 14 - 17, 2015, with peer-review publication submission.
Overall Strategy and Domain Action Items
Request:
Recommend others to whom we should reach out for
attendance at Taxonomy & Intervention Domain’s
concurrent session/workshop on September 23, 2014

9. Domain action items to complete before January 2015 (QIE Phase 1)
● Co-leaders add perspective to domain, and lead the charge to capture
broad perspectives by engaging stakeholders
(anyone you would recommend we to add to this Domain’s network?)
● Description of Domain, and relevant bibliography
(First Draft due Monday, 8/25/14)
● Identification of recommendations and actionable next steps
(First Draft due Monday, 8/25/14)
Future QIE Phases
● Operationalize the AP Recommendations, mapped to the 10 Domains
● Demonstration projects (case studies)
Your Ideas Wanted:
We welcome you to submit changes to descriptive language for the domain in
this slide deck, recommendations, and additional works for the bibliography …
by Sunday, August 24, 2014, at 1:00 PM Eastern Time.
Overall Strategy and Domain Action Items

10. Competencies &
Professionalism
Financial
Considerations
Research
Agenda
Taxonomy, Terms,
& Interventions
Professional
Evaluation
Quality Metrics
& Tools
Policy & Ethics
QIE Roadmap:
10 Domains for Education Embedded in QI
Certification &
Accreditation
Health Sector--CE
Integration
TrendsH
Health
Information
Technology

11. Brainstorm Taxonomy
Define scope, identify resources, write
recommendations

12. Talking points:
How do we validate, organize, and report
the data we now have for
SUSTAINED IMPACT and RELEVANCE
for CME/CE, (IPE) and QI stakeholders?
Domain Description
● The goal is to have a single set of “concepts,” “terms,” and “definitions” that
are integral to quality improvement, healthcare quality, and clinical
education (as they relate to, or from the perspective of, QI and healthcare
quality).
● The taxonomy should focus on “operational definitions” that cross
professional domains.
● Where there are currently multiple definitions of specific terms, we will
work to integrate them into a single definition.
● The final product will be a document of terms that includes definitions,
strategies, tools, and mechanisms to measure the terms, where applicable.

13. Purpose of the Taxonomy
● Need to determine whether the taxonomy should be
“prescriptive” vs. “descriptive”
• Descriptive: example, Webster’s 3rd International “described” the current use of words,
rather than “prescribe” how words should be used (as in the 2nd International)
• Prescriptive: example, MeSH terms belong to a “structured vocabulary” with nested tree of
terms
• Should the taxonomy reflect professional competencies and practice patterns, the system
and reimbursement patterns? How does it define terms without inappropriately
constraining or prescribing use of terms in existing systems?
● Lexicons are more common than taxonomies. Consult but do not duplicate.
● Define “commonly used” terms to make sure everyone in on the same
page. Do not assume that a term is what you think it is!
The taxonomy must have
SUSTAINABILITY Across STAKEHOLDERS
Describe or Prescribe?
How do we know it’s relevant and useful?

14. We see Moore’s Levels 6 and 7 as the fulcrum between
CME/CPD (PI) and QI. Where do taxonomy and
interventions play a role at the fulcrum?
Purpose of the Taxonomy
Start with answering, “What is quality?” and through which lens it is assessed?
Should we use the lens of the health-system at the population-health level?
● Quality = an evidence base exists to support the practice and a well-
established belief supporting the approach to care. The approach to care
or expected outcome should be fully endorsed, consistent with prevailing
beliefs and consensus, and not easily dismissed.
● Differentiate quality improvement from performance improvement
● A population health-management approach would focus on delivering
interventions and services that address the needs of all members of a
population across the care continuum (individuals, teams, organizations)
● For CME/CE providers, this would mean Level 7, where currently Level 6
is a challenge.
Intervention

15. ● Determine scope of the taxonomy
•Conduct a project to define the terms, definitions, and taxonomy focused on all aspects of QI
activities and interventions, covering the educational enterprise and the broader QI
enterprise
•Determine commonly used MeSH terms in relevant journals: Consider a scientific survey of
QI, Health Affairs, interprofessional education (IPE), policy, CME, guideline- and quality-
based, and “core clinical” journals
● Seek collaboration with organizations already engaged
•Review the current literature and activities/outputs
•SACME and MedBiquitous
•SACME has already worked with NLM librarian
•Collaborate with a health care librarian at National Library of Medicine
● Align taxonomy with existing competencies
•ACCME QI information
•ACGME domains and competencies
•Updated competencies for CCMEPs
Taxonomy Recommendations

16. Scope of the Taxonomy Project: Known Resources
MedBiquitous
Procedure of one project reveals a paradigm: “One of the major issues in working with any type of a
national system is the concept of creating meaningful aggregate data. [...] The upload to the …
Inventory will include a process that matches local vocabulary to standardized vocabulary.”
● Vocabulary for instructional methods: http://medbiq.org/curriculum/vocabularies.pdf
● Competency/Performance related definitions:
http://groups.medbiq.org/medbiq/display/CWG/Performance+Framework+-+Definitions
● Educational achievement specification:
http://medbiq.org/sites/default/files/files/EducationalAchievementSpecification.pdf
● Recently investigated Competency, Proficiency, Professionalism …
● Participation Modality - defines the learner’s mode of participation, dictated by the activity
medium. Valid values are conference/workshop, technology based, on the job, print.
● Activity Delivery - indicates the temporal nature of the activity. Valid values are live and not
live.
● Where is the best place to add to their existing resources? - Set-up a GoggleDocs project
folder?

17. Scope of the Taxonomy Project: Known Resources
SACME has a taxonomy project, now referred to as a “definitions” project,
designed to describe the major terms and concepts in the QI/QIE space.
● SACME found 56 terms to pursue first - manuscript currently under
embargo.
● They have “edges” of research into scope to expand, recommending
that other organization complement their work.
Existing work relates to NLM and MeSH taxonomy (see next)
● “A Bibliometric Analysis of Evaluative Medical Education Studies:
Characteristics and Indexing Accuracy” (Margaret Sampson, MLIS,
PhD, Tanya Horsley, PhD, and Asif Doja, MD, MEd. Acad Med.
2013;88:421–427.)
Other vocabularies that might be relevant to consider from our healthcare LOM
specification: P

18. National Library of Medicine develops the Medical Subject Heading (MeSH)
controlled vocabulary or meta-thesaurus.
It’s greatest benefit? It closely matches indexed articles to MeSH terms that are
directly relevant, so that the researcher does not depend on keyword searches.
Some terms are labeled as “major” for greatest relevance to article.
● Tutorial: http://www.nlm.nih.gov/bsd/disted/pubmedtutorial/015_010.html
● “MeSH … consists of sets of terms naming descriptors in a [twelve-level]
hierarchical structure that permits searching at various levels of specificity”
The new taxonomy needs to relate to literature indexed in PubMed.
It is also a Medicare & Medicaid (CMS) issue for reimbursement: if the ACEhp
taxonomy can raise the bar, then QI will occur in all domains, including from
perspective of CMS.
Scope of the Taxonomy Project: Known Resources:
MeSH: Controlled Vocabulary by NLM

19. Scope of the Taxonomy Project: Known Resources:
MeSH: Structured Vocabulary by NLM
Here’s a glimpse of an medical education editor’s unpublished compendium of
useful Medical Subject Headings (MeSH terms) that are relevant to educational
design in continuing education. How do we ...
● Avoid scaring people off before they try using the taxonomy we develop?
● Make a taxonomy developed through this Domain accessible and useful?
● Catalog terms and how they are used without developing massive
spreadsheets of terms from other known resources?

20. After seeing just a glimpse of the MeSH taxonomy, the relevance of
this quotation to our marching orders is clear:
“The term “taxonomy” can be a slippery slope. I find many do not
understand it and more importantly do not know how to apply it and improve
their ability to manage the amount of information we are privy to. Defining
the process for enhancing the current taxonomy and establishing
governance are areas of focus ....”
~ Someone from industry (emphasis added)
Perspectives:
I Overheard You Talking About Taxonomy ...

21. Random comments from various points of view (POVs) indicate larger
issues and a potentially oversized scope for the project:
● All stakeholders:
How do we know what we know when we’re not using the same language?
● Educational designer POVs:
No consistency in how we present content (or describe how we present
content) in medical/continuing education.
How do you define an intervention? an assessment?
● Patient and patient educator POV:
Health literacy - what is the common denominator needed to describe
something - what terms can be grasped?
● Educational outcomes researchers POV:
We've overlooked defining terms commonly used in CME (making
research challenging), I'm hoping this means you're working to standardize
some language.
Do you have more comments from your network or scope of practice?
Submit quotations and perspectives to consider.
Perspectives:
I Overheard You Talking About Taxonomy ...

22. Scope of the Taxonomy Project: Known Resources
ACEhp is carrying out QIE communications on these topics, which are relevant
to the scope of our work on the Taxonomy, Terms, and Interventions Domain:
● Baseline understanding of role of Education in QI (June)
● Patient Engagement in QI (July)
● Inter-professional Education in QI (August)
● QIE Roadmap Recommendations (September thru January)
GAME and IPE projects [brainstorming point: move this to Interventions?]

23. Brainstorm “Interventions”
Define scope, identify resources, write
recommendations

24. Questions to Guide Intervention Description
● What resources can we use?
● Should we take the points on the following slides as an
appropriate scope for the Interventions component?
● Do the following slides provide appropriate definitions?

25. Intervention Description
● Professional norms and peers’ influence: Every system has a culture
that employees work in and adherence to the culture is appropriate,
legitimate, expected, normative; Just as non adherence is improper and
unacceptable. The culture may be more influential on a clinician's
approach to care than their own beliefs. A QI intervention set should
include engagement with the local advocates to understand professional
norms and design the intervention to work within these norms to drive
change.
● External pressure, incentives and expectations for improvement:
Leadership/management and other influential entities (e.g. CMS) should
be available to apply pressure at a level adequate to overcome competing
demands and to focus attention and interest of the target audience for the
QI project. The pressure should be as broad and comprehensive/pervasive
as possible (including external stakeholders, institutional senior leadership,
supervisors, peers, patients). Ideally projects include meaningful
consequences for non-compliance. Projects require the ability to measure
and report compliance.

26. Intervention Description
● Patient concurrence: Patients are an integral aspect to quality care; their
knowledge, attitude, beliefs can affect acceptability of the recommended
care plan and the feasibility of a successfully executed quality
improvement initiative.
• The QI intervention set needs to include interventions to increase patient acceptability.
• Look at PCORI terminology and definitions.
● Evidence of deviations from recommended practices: A QI intervention
should include a method to measure and demonstrate to the target
audience (at the individual and system level) they are not adherent to the
evidence. The evidence/data needs to be valid, accurate, credible, timely,
relevant and include an appropriate benchmark as the comparator.

27. Intervention Description (continued)
● Etiology of deviations: A QI intervention should include an analysis to
understand the causes, influences, barriers and facilitators of why quality
gaps exists at the intervention site. A thorough diagnosis of the
multifaceted influences on current practice and the causes of quality gaps
is necessary. This diagnostic should be location specific as each practice
site works differently.
● Feasibility to change operations: A QI project needs to have the ability
to logistically implement and utilize new practices to address the etiology of
deviations identified. This may include elimination of financial,
organizational and operational constraints (staffing, workflow, time,
technology). To be effective, each solution set should be specific to only
one local implementation site. Reliable broad spectrum practice change
interventions do not exists and therefore should not be developed.
However, resources can be shared and modified for future site
implementations that increase the sustainability of early work.

28. Intervention Description (continued)
● Quality Improvement strategies/"Change packages": Implementation
strategies will vary based on all the point mentioned previously. They could
be pulled from several categories:
• Education strategies: Awareness campaigns/communications, training on how to
implement new care approaches at the local level, train-the-trainer programs, create
learning collaboratives, shadowing opportunities, programs to enhance knowledge and
competence skills. Would include CME and non-CME projects.
• Finance strategies: These would come from the system to support change and include
such things like penalties, incentives, reduced patient co-pays, etc.
• Restructure strategies: Revise roles/responsibilities, create new clinical teams, change
physical structure and equipment, facilitate data clinical data exchange, change
recordkeeping processes
• Quality management strategies: Develop system for ongoing quality monitoring and
reporting, audit and provide feedback, reminders, obtain and use patient feedback,
interventions to enhance update, capture and share local knowledge, reexamine the entire
process for additional change requirements

29. Random comments from various points of view (POVs) indicate larger
issues and a potentially oversized scope for the project:
● Information technology POV: IT design for unique needs relies on
applying an existing taxonomy to IT platforms
● Medical affairs and pharmaceutical R&D need a defined language for
engagement with biotech
● Traditional CME POV:
“Provider” and “sponsor” are also consistently misused
Another angle: EHR (chart) data are not considered part of hospital CME
● Quality Improvement POV:
Quality improvement occurs every day, in a location/setting, without
awareness or recognition of the role of CME.
Do you have more comments from your network or scope of practice?
Submit quotations and perspectives to consider.
Perspectives:
I Overheard You Talking About Interventions...

30. Intervention Recommendations
● Align interventions with established competencies for professionals in each
stakeholder arena
● Move toward evidence-based (and research- or statistically based??)
practice for CME/CEhp professionals
● Engage top perceived stakeholders for Interventions component of Domain
•QI organizations and LinkedIn groups, PCORI, CMS
•ACEhp QIE Commentary Group will review for relevance
● Engage with Instructional Design and E-Learning Groups
● Engage with B. J. Fogg of Stanford Persuasive Technology Lab professor,
on behavioral change
● Engage with business faculty in Organizational Behavior
● Work in context of ABMS MOC requirements and perspectives
● Add a hospital-system pro to inform the intervention and determine
whether the taxonomy is actionable and relevant - is it sustainable?

31. Intervention Recommendations (continued)
What else would you recommend for the Roadmap?
How do we make this a well-rounded but not duplicative project?

32. Bibliographies
Taxonomies, Terms, & Interventions

33. • Van Hoof, T., Miller, N. Consequences of a Lack of Standardization of
Continuing Education Terminology: The Case of Practice Facilitation and
Educational Outreach. Journal of Continuing Education in the Health
Professions. J. Contin. Educ. Health Prof., 34: 83–86.
• Quality Improvement in Healthcare: A Lexicon. Medscape Center for the
Advancement of Healthcare Quality Through Education.
http://img.medscape.com/pi/global/logos/mscp/edu/vision/Lexicon_Draft_010314_bb.pdf
• Raymond Fabius, MD, CPE, FACPE, Linda MacCracken, MBA, Jill Pritts,
MBS. Vocabulary of Healthcare Reform: A white paper; Thomson Reuters,
January 2012
• MeSH guides from NLM: Training slides (link); fact sheets (overview links)
• Lexicon for Behavioral Health and Primary Care Integration: Concepts and
Definitions Developed by Expert Consensus. Prepared for: Agency for
Healthcare Research and Quality.
http://integrationacademy.ahrq.gov/sites/default/files/Lexicon.pdf
Draft Bibliography for Taxonomy and Terms

34. • Patient Engagement: A Lexicon. A guide to terminology, technology,
legislation, and information resources
• Guide to Reference in Medicine and Health, edited by Christa Modschiedler,
Denise Beaubien Bennett
• Introduction to Health Sciences Librarianship, M. Sandra Wood
• The Global Guide III: A CME Reference Compendium.
http://www.globaleducationgroup.com/cme-resources/cme-publications/
• Heather Hedden. The Accidental Taxonomist.
(Has been described as “somewhat helpful”)
Draft Bibliography for Taxonomy and Terms
And do we need to distinguish between “Taxonomy” and “Terms”???

35. Draft Bibliography for Intervention
• SQUIRE:
Davidoff F, Batalden P, Stevens D, et al. Publication guidelines for quality
improvement in health care: evolution of the SQUIRE project. Qual Saf
Health Care. 2008 Oct;17(Suppl 1):i3–9. PMC free article
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2773518/
• Ovretveit J. A framework for quality improvement translation: Understanding
the conditionality of interventions. Jt Comm J Qual Patient Saf. 2004;Global
Supplement:15–24.
• Jay, S. J. and Anderson, J. G. (1993), Continuing medical education and
public policy in an era of health care reform. J. Contin. Educ. Health Prof.,
13: 195–209.
• Grol, R. (2002), Changing physicians' competence and performance:
Finding the balance between the individual and the organization. J. Contin.
Educ. Health Prof., 22: 244–251
• Adelson, R., Vanloy, W. J. and Hepburn, K. (1997), Performance change in
an organizational setting: A conceptual model. J. Contin. Educ. Health Prof.,
17: 69–80

36. Draft Bibliography for Intervention
● Tamblyn R., Battista R.. Changing clinical practice: which interventions
work? J Cont Educ Health Prof 1993; 13: 273–288.
● The Cochrane Collaboration. Cochrane Effective Practice and
Organisation of Care Group. 2012. [September 24, 2012]. http://epoc​.cochrane.org/
● Through the Quality Kaleidoscope: Reflections on the Science and
Practice of Improving Health Care Quality: Closing the Quality Gap:
Revisiting the State of the Science [Internet]. http://www.ncbi.nlm.nih.gov/books/NBK126722/
(Has a treasure trove of references)

37. ● Recommend others to engage in the work of this Domain
● Contribute to the First Draft due Monday, 8/25/14:
• Description of Domain
• Identification of recommendations and actionable next steps
• Relevant bibliography
What else would you recommend for the Roadmap?
How do we make this a well-rounded but not duplicative project?
Next Steps … for those who are able
Do you have suggestions?
Please submit changes to descriptive language for the
domain in this slide deck, recommendations, and additional
references for the bibliography …
to kgeissel@medscape.net or sandra.binford@gmail.com
by Sunday, August 24, 2014, at 1:00 PM Eastern Time

38. Additional Call Notes
Many thanks to everyone who joined
the calls!

39. Recommendations (continued)
Notes from Call #1, August 20, 2014:
● Refine the target and scope, and define how each Domain can/should
team up with leadership’s higher-level work for linkage to QI space
● Expand the bibliography of relevant materials through investigations of the
other 9 QIE Domains’ bibliographies as insurance that the taxonomy is
relevant
● Engage with pros in quality, performance measures, institutional CME
departments, educational designers, interprofessional education for teams,
outcomes researchers/analysts, CME/CE supporters, patient-centered
research (PCORI definitions), health economists, evidence-based
medicine, clinical trials, medical/nursing/mental health schools, human and
organizational behavior scientists, taxonomists for information technology,
CMS, patient advocacy groups, … (prioritize)
● Determine needs for ongoing staff/internship support for project.
MedBiquitous approach to taxonomy project was to establish a task force
of 6 people who met every 2 weeks, with a staff person who researched
the literature, wrote the document, etc.

40. Recommendations (continued)
Notes from Call #2, August 21, 2014:
● [notes]