Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
|||
© 2016 MCC | CMC
André De Champlain, PhD
Director, Psychometrics and
Assessment Services
Presentation at the MCC’s
Ann...
||
© 2016 MCC | CMC
Rethinking the Nuts and Bolts of Assessment
• Reconceptualization of assessment over the past 2 decade...
||
© 2016 MCC | CMC
Rethinking the Nuts and Bolts of Assessment
• Reconceptualizition of assessment over the past 2
decade...
||
© 2016 MCC | CMC
4
Education and Assessment As Willing Partners
• Mechanisms through which learning
occurs have shifted...
||
© 2016 MCC | CMC
5
Education and Assessment As Willing Partners
• From a traditional view of education
• Teacher-center...
||
© 2016 MCC | CMC
6
Education and Assessment As Willing Partners
• Evolution of learning models & modalities not
complet...
||
© 2016 MCC | CMC
7
Education and Assessment As Willing Partners
• Evolution of learning models & modalities not
complet...
||
© 2016 MCC | CMC
Assessment Paradigm Shift
• Increasing dissatisfaction with
established educational assessment
models
...
||
© 2016 MCC | CMC
Assessment Paradigm Shift
• Concerns
• Lack of overarching framework (program) to guide the design of ...
||
© 2016 MCC | CMC
Assessment Paradigm Shift
• Programmatic assessment
• Calls for a deliberate, arranged set of longitud...
||
© 2016 MCC | CMC
Assessment Paradigm Shift
• Programmatic assessment
• Application of a program evaluation framework to...
||
© 2016 MCC | CMC
Wading Into the Programmatic Assessment Waters –
The MCC Experience
Assessment Review Task Force (ARTF...
||
© 2016 MCC | CMC
Tacit Sub-recommendation – The Need for a Macro-analysis
An intimated challenge outlined in the ARTF
r...
||
© 2016 MCC | CMC
Wading Into the Programmatic Assessment Waters in Canada
Assessment Continuum for Canada (ACC)
• Purpo...
||
© 2016 MCC | CMC
Practical Implications of a Pan-Canadian Program of
Assessment for Medical Licensure
• Programmatic as...
||
© 2016 MCC | CMC
Practical Implications of a Pan-Canadian Program of
Assessment for Medical Licensure
• Programmatic as...
|||
© 2016 MCC | CMC
THANK YOU!
André De Champlain, PhD adechamplain@mcc.ca
|||
© 2016 MCC | CMC
PANEL Q&A
Medical Council of Canada
2016 Annual Meeting
#MCCagm
#AssembléeCMC
Upcoming SlideShare
Loading in …5
×

Bridging the Transition: Setting the Stage for a More Continuous and Integrated Model of Assessment for Medical Licensure in Canada

215 views

Published on

André De Champlain, PhD
Director, Psychometrics and Assessment Services

Presentation at the MCC’s Annual General Meeting
Sept. 11, 2016

Published in: Science
  • Be the first to comment

  • Be the first to like this

Bridging the Transition: Setting the Stage for a More Continuous and Integrated Model of Assessment for Medical Licensure in Canada

  1. 1. ||| © 2016 MCC | CMC André De Champlain, PhD Director, Psychometrics and Assessment Services Presentation at the MCC’s Annual General Meeting Sept. 11, 2016 Setting the Stage for a More Continuous and Integrated Model of Assessment for Medical Licensure in Canada Bridging the Transition #MCCagm #AssembléeCMC
  2. 2. || © 2016 MCC | CMC Rethinking the Nuts and Bolts of Assessment • Reconceptualization of assessment over the past 2 decades • Over the past two decades, tremendous amount of thought & activity aimed at proposing models of assessment & related processes that are: ◦ More transparent & flexible ◦ Better linked to learning activities ◦ More informative from an educational standpoint • Concurrently, effort has been devoted to improving processes necessary to support these re-envisioned assessments • Revisitation of assessment’s epistemological core
  3. 3. || © 2016 MCC | CMC Rethinking the Nuts and Bolts of Assessment • Reconceptualizition of assessment over the past 2 decades • Assessment paradigm shift • Programmatic assessment (van der Vleuten et al., 2012) • Post-modern test theory (Mislevy, 1997) • Cognitively-based assessment of, for & as learning (CBAL [Bennett, 2010]) • Use of technology to support this shift o Enhancement of test development practices (e.g. - automated item generation [AIG]) o For more frequent and flexible assessments o Marking of open-ended responses & narrative text o To accommodate a broadening of measurement strategies
  4. 4. || © 2016 MCC | CMC 4 Education and Assessment As Willing Partners • Mechanisms through which learning occurs have shifted • From traditional (paper-based) to electronic media ◦ Tablet & mobile device-based learning is ubiquitous (e.g., MedPage Today, QuantiaMD, etc.) • Exponential growth of knowledge in medicine
  5. 5. || © 2016 MCC | CMC 5 Education and Assessment As Willing Partners • From a traditional view of education • Teacher-centered with high exam scores as main goal • To alternate models which stress learning, retention & integration of knowledge & skills using a host of assessment modalities (e.g., PBL, competency-based education, etc.)
  6. 6. || © 2016 MCC | CMC 6 Education and Assessment As Willing Partners • Evolution of learning models & modalities not completely mirrored by similar changes in educational assessment • Educational assessment must evolve alongside learning models or risk fostering an antagonistic relationship • Educational assessment must (Bennett, 2002): • Provide meaningful information • Satisfy multiple purposes • Use modern conceptions of competency as a design basis • Design for positive impact & engagement • Use technology to achieve substantive goals
  7. 7. || © 2016 MCC | CMC 7 Education and Assessment As Willing Partners • Evolution of learning models & modalities not completely mirrored by similar changes in educational assessment • Educational assessment must evolve alongside learning models or risk fostering an antagonistic relationship • Educational assessment must (Bennett, 2002): • Provide meaningful information • Satisfy multiple purposes • Use modern conceptions of competency as a design basis • Design for positive impact & engagement • Use technology to achieve substantive goals
  8. 8. || © 2016 MCC | CMC Assessment Paradigm Shift • Increasing dissatisfaction with established educational assessment models • Unidirectional relationship between learning & assessment ◦ Discrete, episodic hurdles to “overcome” ◦ Decay/minimal transfer of knowledge and skills • Unlinked assessments
  9. 9. || © 2016 MCC | CMC Assessment Paradigm Shift • Concerns • Lack of overarching framework (program) to guide the design of the assessment tools along an educational/professional training continuum ◦ Plea for a macroscopic rather than microscopic view of assessment (de Rosnay, 1979) • Reductionist lens that is applied to what is a complex, adaptive system with interconnected components & dynamic relationships • Missed opportunity to view learning & assessment in a rich, recursive relationship ◦ Both activities can dynamically inform each other ◦ Critical to feed forwarding information or handover
  10. 10. || © 2016 MCC | CMC Assessment Paradigm Shift • Programmatic assessment • Calls for a deliberate, arranged set of longitudinal assessment activities • Joint attestation of all data points for decision & remediation purposes • Input of expert professional judgment is a cornerstone of this model • Purposeful link between assessment & learning/remediation • Dynamic, recursive relationships between assessment & learning points
  11. 11. || © 2016 MCC | CMC Assessment Paradigm Shift • Programmatic assessment • Application of a program evaluation framework to assessment • Systematic collection of data to answer specific questions about a program • Gaining popularity within several medical education settings • Competency-based workplace learning • Medical schools (e.g., Dalhousie, U of T, UBC, etc.) • Etc.
  12. 12. || © 2016 MCC | CMC Wading Into the Programmatic Assessment Waters – The MCC Experience Assessment Review Task Force (ARTF) • As the MCC approached it’s 100th anniversary (2012), a task force of eminent Canadian medical educators was convened to undertake a reflective & strategic review of the MCC’s assessment purposes and objectives, their structure & their alignment with MCC’s major stakeholder requirements • Report published in 2011 which contained six recommendations to fulfill including validating & updating the blueprint for MCC examinations, offering exams with greater flexibility, enhancing & standardizing IMG assessments & engaging in in- practice assessment
  13. 13. || © 2016 MCC | CMC Tacit Sub-recommendation – The Need for a Macro-analysis An intimated challenge outlined in the ARTF report pertains to the need to conduct a macro- analysis & review of the MCC Qualifying Examination (MCCQE) • Applying a systemic (macroscopic) lens to the MCCQE as an integrated examination system • How are the components of the MCCQE interconnected & how do they inform key markers along a physician’s educational & professional continuum? • How can the MCCQE continue to evolve towards embodying an integrated, logically planned & sequenced system of assessments that mirrors the Canadian physician’s journey?
  14. 14. || © 2016 MCC | CMC Wading Into the Programmatic Assessment Waters in Canada Assessment Continuum for Canada (ACC) • Purpose • To define the ‘life of a physician’ from the beginning of medical school to retirement in terms of assessments • To propose a common, national framework of assessment using a programmatic approach • ACC composition includes representation from the MCC, UGME/PGME, CPD, certification colleges, FMRAC and MRAs • First outcomes include a definition of a CMG physician pathway with partners as well as broad agreement on key principles of assessment • Next step – National summit (spring of 2017)
  15. 15. || © 2016 MCC | CMC Practical Implications of a Pan-Canadian Program of Assessment for Medical Licensure • Programmatic assessment is predicated on more frequent & flexible assessment via a combination of tools: • Traditional exam formats • Lower-stakes, in-practice and other observations • Ancillary sources (e.g.: narratives) • This shift impacts core assessment tasks including test development & scoring activities • Assessments need to be developed, administered & scored more frequently
  16. 16. || © 2016 MCC | CMC Practical Implications of a Pan-Canadian Program of Assessment for Medical Licensure • Programmatic assessment rests on a concurrent development of a linked training and assessment continuum • Both activities are equally important in informing each other • Assessment in and of itself is viewed as a critical learning activity • Decisions are based not only on a sum of discrete elements but on a carefully developed composite of measures which reflects key learning outcomes in the trajectory of a physician’s medical education • What are the key data points that inform a decision in regard to entry into supervised/independent/continuous practice and what learning outcomes do they reflect? • How can key handoff (milestone) points be identified for all stakeholders that make sense within a licensing/certification context?
  17. 17. ||| © 2016 MCC | CMC THANK YOU! André De Champlain, PhD adechamplain@mcc.ca
  18. 18. ||| © 2016 MCC | CMC PANEL Q&A Medical Council of Canada 2016 Annual Meeting #MCCagm #AssembléeCMC

×