Debra Sibbald, Executive Director, Assessment Programs BScPhm RPh ACPR MA (Adult Education) PhD (Curriculum Teaching & Learning) Presented at Perspectives in Competency Assessment A Symposium by Touchstone Institute www.touchstoneinstitute.ca

1. PERSPECTIVES IN
COMPETENCYASSESSMENT
Debra Sibbald, Executive Director, Assessment Programs
BScPhm RPh ACPR MA (Adult Education) PhD (Curriculum Teaching & Learning)
February 2015

2. KEY DECISION-MAKING
IN ASSESSMENT DESIGN:
EXPERT LENSES

3. Thus far you have heard opinions from
expert authoritative sources on competency
assessments:
how best practices should inform key decision
making...

4. Best Practices: Identify Important Design Principles
• Candidates
• Outcomes
• Competencies
• Levels
• ‘Fit for purpose’ aligned blueprint
• Methods
• formats
• Scales
• Subject matter experts
• Writing
• Validating
• Standard setting
• Iterative reflection and refinements

5. HARMONIZATION OF
COMPETENCYASSESSMENTS:

6. Collective
CommunityIndividual
Agents
E
X
P
E
R
T
S

7. INDIVIDUAL
AGENTS
likely perceive the problem relative to their context

8. COLLECTIVE COMMUNITY
A Collaboration

10. IndividualAgents:
• Relevant decision-makers /organizations
• Major Stakeholders (directly affected and held accountable)
• Experts in their contexts
• Others (groups/individuals who provide commentaries)
• HARMONIZATION PROCESS:
• EXTERNAL EXPERTS (TOUCHSTONE)
• Consider each agent’s vested interest and what that group wants
• ‘Fit-for-purpose’ alignment
• Acceptable to all
• Understood and acted upon by all

11. EXAMPLE:
OPTOMETRYEVALUATING EXAM FOR
INTERNATIONALOPTOMETRYGRADUATES

12. Major Stakeholders
1. Candidates – International Optometry Graduates
2. College of Optometrists of Ontario (Registration Committee)
3. International Optometric Bridging Program (IOBP)
4. CEO / ECO (Canadian Examiners Optometry) / (Examinateurs
Canadiens en Optométrie)
5. Practicing Optometrists (from Canadian schools or from
schools in North America accredited by the Accreditation
Council on Optometric Education (ACOE)
6. Practicing Optometrists who are graduates of IOBP
7. University of Waterloo

13. Major Stakeholders
8. Pan Canadian Provincial Jurisdictions (FORAC – CORA)
9. Office of the Fairness Commissioner
10. TOUCHSTONE
• Executive Director of Assessment
• Director Health Professional Affairs
• Psychometricians
11. TOUCHSTONE: Board of Directors
• Chief Executive Officer
• Directors Finance/Resources
• Managers – Operations and Standardized Patients
12. Blueprint Subject Matter Experts:
• Participants for EE from Optometric Practice

14. Other Stakeholders
13. Evaluating Exam Chief Examiners
14. Subject Matter Experts Writers
15. Evaluating Exam Examiners

15. Major Stakeholder Major Stakeholder Minor Stakeholder
Candidates from IEO Pan Canadian Provincial
Jurisdictions (FORAC – CORA)
Evaluating Exam
Chief Examiners
College of Optometrists of
Ontario
(Registration Committee)
Office of the Fairness
Commissioner
Subject Matter Experts Writers
International Optometric
Bridging Program (IOBP)
TOUCHSTONE
-Director of Assessment
-Director Health Professional
Affairs
-Psychometricians
Evaluating Exam Examiners
CEO / ECO
(Canadian Examiners Optometry) /
(Examinateurs Canadiens en
Optométrie)
TOUCHSTONE Board of Directors
- Chief Executive Officer
- Directors - Finance/Resources
- Managers – Operations/SPs
Practicing Optometrists from
ACOC schools
Blueprint Subject Matter Experts:
Participants for EE from Optometric
Practice
Practicing Optometrists who are
graduates of IOBP

18. WHOARE THEAGENTS IN OUR
AUDIENCE?
Turn on your clickers!

20. What is your main focus of interest?
A. Regulatory
B. Education
C. Clinical
D. Assessment
E. Administrative

21. From what viewpoint is the domain of your concern?
A. Economic forces
B. Social/sociological structures and practices
C. Political structures and policies
D. Educational affordances and constraints
E. Cultural beliefs or supports

22. TODAY YOU HAVE BEEN
PRESENTED WITHAGREAT DEAL
OF INFORMATION FROM VARIOUS
DIMENSIONS.

23. i-Clicker Polling -
CommunityMicrophone -
Agents
It’s time to pause and engage you in a guided reflection

24. Recall the issues and perspectives presented(s)
How did you react during the presentation?
Why did you feel this way?
Future changes based on these insights?
Reflection & Feedback on This Morning

25. Identify
Change
Analyze
Reflect

26. Identify:
The most surprising piece of information that you learned
today
Or
The perspective or view which is not aligned with your
current practices or beliefs

27. What was your primary Reaction?

28. Why do you have this reaction or concern?
The domain your view arises from is :
A. Economic
forces
B. Social/sociological
structures and practices
C. Political
structures and policies
D. Educational
affordances and constraints
E. Cultural
beliefs or supports

29. What are your main issues?

30. Considering the reasons for these surprises or
differences, will you change your beliefs or
practices?
A. YES
B. No

31. If yes, what will be your main goal?

32. What will be likely implications of this decision ?

33. You identify your primary challenge as
A. Problems created by your own decisions and behaviours
B. Problems created by external forces

34. What is the primary characteristic of your
challenges?
A. Problems can be solved in whole or in part
B. Problems beyond our controls

35. On what step towards change has today’s discussion had
impact, if any?
A. You recognize you now face an important decision
B. You now more clearly or accurately can identify alternatives
C. You now are better equipped to evaluate alternatives
D. You are now ready to act on the best alternative
E. No impact on steps toward change

37. Audience has the floor!
You Say…..
I Say….