The American Board of Anesthesiology is piloting a new model of post-certification assessment called MOCA 2.0 that focuses on continuous learning rather than high-stakes episodic exams. MOCA 2.0 uses an interactive tool called MOCA Minute that provides questions to diplomats on a quarterly basis to continuously assess knowledge and identify gaps. This longitudinal assessment better supports diplomats in maintaining up-to-date knowledge throughout their careers compared to the previous exam model done every 10 years. The goal is to promote lifelong learning and improve patient care.

1. The American Board of Anesthesiology
The American Board of
Anesthesiology
Changing Models of
Post-Certification Assessment
in the Professions
Ting Wang, Ph.D.
February 2017

2. The American Board of AnesthesiologyThe American Board of Anesthesiology
Purpose
• The purpose of this presentation is to preview for
you how post-certification assessment models within
the American Board of Medical Specialties (ABMS)
member-board community are beginning to change
– 24 ABMS specialty boards in the US
– Over half have begun exploring new models for
assessing our diplomates’ continuing knowledge,
judgement and skills
2

3. The American Board of AnesthesiologyThe American Board of Anesthesiology
What We Will Cover
• The current model ABMS member boards use to
assess diplomates’ knowledge, judgment and skills
• The changing philosophy about the purpose and value
of post-certification assessment
• The new model the American Board of Anesthesiology
(ABA) is piloting for its Maintenance of Certification in
Anesthesiology Program (MOCA®)
3

4. The American Board of AnesthesiologyThe American Board of Anesthesiology
The Current Model
• Requires a cognitive examination every 6 to
10 years
– Mimic the boards’ initial certification exams in
structure, i.e., SBA MCQ, high-stakes, testing
center administration
– Knowledge being tested is the same as initial
certification exam
4

5. The American Board of AnesthesiologyThe American Board of Anesthesiology
The Current Model
• Boards continue in a “gate-keeping” role
• Rationale:
– Boards primarily serve the public to protect
the public
• Criticisms:
– Diplomates view it as burdensome, costly and
irrelevant to their current practice
– Supports “binge-and-purge learning” or “just-
in-time-forgetting”
5

6. The American Board of AnesthesiologyThe American Board of Anesthesiology
The Changing Philosophy
• Some new thinking is emerging:
– Recertification is episodic and transactional;
– MOC programs should be continuous and
relationship-based;
– MOC programs should continuously support
diplomates in keeping their walking around
knowledge, judgement and skills up-to-date over
the course of their career
6

7. The American Board of AnesthesiologyThe American Board of Anesthesiology
The Changing Philosophy
• Some new thinking is emerging:
– Initial certification assessments provide the
public and the profession a dependable
mechanism for identifying practitioners who
have met particular specialty standards
– Post-certification assessments should focus
on supporting diplomates who have already
met the standard for certification to remain “at
the top of their game” throughout their career
7

8. The American Board of AnesthesiologyThe American Board of Anesthesiology
The Changing Philosophy
• The ABA believes:
– we serve the public with our initial certification
assessments; the public is our primary
“customer” at this stage
– once certified, the diplomate becomes our
primary customer; the public becomes a
beneficiary of the relationship between the board
and the diplomate
– Post-certification assessment models should
support and advance this relationship
8

9. The American Board of AnesthesiologyThe American Board of Anesthesiology
Evolution to MOCA 2.0®
Background
• 2011: Developed Strategic Plan
• 2012: Hosted Technology Summit
• 2013: Conceptualized MOCA 2.0 and MOCA Minute®
• 2014: Facilitated Collaborative Concept Development
• 2016: Launched MOCA 2.0 and expanded MOCA
Minute pilot
9

10. The American Board of AnesthesiologyThe American Board of Anesthesiology
The Goal
Build a learning platform that promotes and supports
personalized knowledge acquisition, assessment,
and demonstration of proficiencies
10

11. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA 2.0® Strategy
• Use an intensive longitudinal assessment model
to…
– Identify knowledge gaps among diplomates, as a
group and as individuals
– Connect diplomates to targeted educational
opportunities based upon their knowledge gaps
11

12. The American Board of AnesthesiologyThe American Board of Anesthesiology 12
MOCA 2.0®
Design Principles
Promote continuous lifelong learning
Relevant to diplomates’ practices
Professionally and publicly credible
Ongoing continuous assessment
Encourage group discussion
Incorporate principles of adult learning theory
Facilitate Quality Improvement and Safety
Integrate Parts I, II, III and IV

13. The American Board of AnesthesiologyThe American Board of Anesthesiology
Traditional MOCA® vs. MOCA 2.0®
Traditional MOCA MOCA 2.0
Part 1 Professional Standing (unrestricted
license)
Professional Standing – No change
Part 2 Lifelong Learning & Self-
Assessment
• 250 Category 1 CME credits,
including:
• 90 Self-Assessment
• 20 Patient Safety
Lifelong Learning & Self-Assessment
• 250 Category 1 CME Credits
(including 20 Patient Safety)
• Self-Assessment CMEs no longer
required
• Diplomates who previously
completed Self-Assessment CMEs
will get credit for them in MOCA 2.0
Part 3 Assessment of Knowledge,
Judgment, and Skills
• 200-question MOCA Exam
• Taken once every 10 years
Assessment of Knowledge, Judgment,
and Skills
• Longitudinal assessment using MOCA
Minute™ pilot
• MOCA Minute® is being piloted to replace
the MOCA Exam
13

14. The American Board of AnesthesiologyThe American Board of Anesthesiology
Traditional MOCA® vs. MOCA 2.0®
Traditional MOCA MOCA 2.0
Part 4 Improvement in Medical
Practice
• Attestation
• Simulation at an ASA-
endorsed simulation center
• Case Evaluation
Part 4 requirements vary based
on year of certification
Improvement in Medical Practice
• Variety of options provide greater
flexibility to complete relevant
activities
• Point System weights activities
based on the time and effort
• Simulation as an option; not a
requirement, although the ABA
strongly encourages participation
Fee: $2,100 every 10 years $210 annually
14

15. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA Minute® Pilot
• MOCA Minute is an interactive learning tool that helps
diplomates continuously assess and refresh their
knowledge
– More relevant and personalized approach to lifelong learning
– Can inform diplomates’ learning choices for MOCA Part 2
component
15

16. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA Minute® Pilot
• Longitudinal assessment data from MOCA Minute will
allow us to identify knowledge gaps for individual
diplomates and across the diplomate corps, which we
can strategically help address
– Positively impact patient care across the anesthesia practice
spectrum
– Not possible with once-every-10-years MOCA Exam
16

17. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA Minute® Pilot
• The MOCA Minute pilot is available to all diplomates
enrolled in MOCA in 2016; this is the core of the
MOCA 2.0 program
17

18. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA Minute™ Pilot
Requirements
• MOCA 2.0 participants:
– Must answer 30 MOCA Minute questions per
calendar quarter
– Must tell us how confident they are about their
answer
– May tell us how relevant the questions are to
their practice
18

19. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA Minute® Question Types
• Core – covers content that forms the general
anesthesia knowledge base that all ABA diplomates
are expected to know
• New knowledge – covers new information that all
ABA diplomates need to learn quickly (i.e., Ebola,
Zika Virus)
• Practice-specific – covers content based on the
diplomate’s practice profile
• Subspecialty – covers content based on the
diplomate’s subspecialty certification(s), if any
19

20. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA Minute® Application
20

21. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA Minute® Application
21

22. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA Minute® Application
22

23. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA Minute® Pilot
evaluating Diplomate Performance
• MOCA Minute® is an interactive longitudinal
assessment model we are piloting to evaluate
whether diplomates are maintaining their
specialty-specific knowledge
• Given this innovative approach, we’re using a
new method to analyze diplomates’ performance
– Measurement Decision Theory or MDT
23

24. The American Board of AnesthesiologyThe American Board of Anesthesiology
MOCA Minute® Pilot
evaluating Diplomate Performance
• More detailed information about how the ABA is
using MDT to evaluate diplomate performance
as part of our MOCA Minute pilot can be found
on the ABA’s website at:
www.theABA.org/MOCA/MOCA-Minute
24

25. The American Board of Anesthesiology
The American Board of
Anesthesiology
The American Board of Anesthesiology
Questions?