1ExamSoft Assessment Conference | #ASSESS2015
ExamSoft Assessment Conference
#ASSESS2015
2ExamSoft Assessment Conference | #ASSESS2015
Utilizing Multiple Grader Rubrics for Practical
Assessment of Student Performance
Sherry Jimenez, Ed.D. Sarah Zahl, Ph.D.
Assistant Dean for Educational Development Director of Educational Assessment
3ExamSoft Assessment Conference | #ASSESS2015
Agenda
• Audience Response
• MUCOM Brief Overview
• Multiple Grader Rubrics
– Assessment Types and Examples
– Tracking outcomes in rubrics
• Recommendations
• Small Group- Pair/Share
• Q&A
Audience Response
7ExamSoft Assessment Conference | #ASSESS2015
Audience Response
8ExamSoft Assessment Conference | #ASSESS2015
MUCOM Overview
• Competency Based Curricular Design
• Systems based/integrated curriculum
– Course Director, multiple internal faculty and external
clinicians
– e.g. Cardiovascular course + OSCE
• iPad exclusive
• Building a culture of evidence among
faculty/students/administrators
9ExamSoft Assessment Conference | #ASSESS2015
Tracking Student Performance
• Competencies
• COMLEX blueprint
• System
• Discipline
• Learning Domains
• Course specific outcomes
10ExamSoft Assessment Conference | #ASSESS2015
National Board of Osteopathic Medical Examiners
Fundamental Osteopathic Medical Competencies
COMLEX Blueprint
11ExamSoft Assessment Conference | #ASSESS2015
Use of Rubrics
• Lab Practicals: Osteopathic Manipulative Medicine
(OMM) Lab
• Simulations
• Objective Structured Clinical Examinations (OSCE)
• Small Group Evaluations: Clinical Medicine
12ExamSoft Assessment Conference | #ASSESS2015
Small Group Evaluations
13ExamSoft Assessment Conference | #ASSESS2015
Small Group Evaluations – Grading Process
14ExamSoft Assessment Conference | #ASSESS2015
Small Group Evaluations – Grading Process
• Complete the administrative planning process
externally before using ExamSoft
• Pre-assign faculty graders and students
• Pre-assign assessment area (region)
• Prepare extra materials and guides
• Communication
15ExamSoft Assessment Conference | #ASSESS2015
OSCE
• Humanistic rubric, SOAP note, SP evaluation,
self-assessment
16ExamSoft Assessment Conference | #ASSESS2015
Lab Practicals
• Paper  Very complex online process  streamlined
process in ExamSoft
Old process: multiple
stations, partner flip,
multiple regions,
multiple graders and
students at each
station
New process: one
station, partner flip
only, one region (pre-
determined but
surprise to student)
17ExamSoft Assessment Conference | #ASSESS2015
Outcomes
• Culture shift –modifying structure
• Positive feedback – ease of evaluation
• Immediate faculty feedback – inter-rater
• Timeliness of student feedback
• Pairing rubric data with written exam data to develop
longitudinal reports
• Continuous refinement
18ExamSoft Assessment Conference | #ASSESS2015
Group/Pair Share
• How are you currently assessing student
performance in practical settings?
• How could you incorporate multiple grader
rubrics in your curriculum?
19ExamSoft Assessment Conference | #ASSESS2015
Discussion
• Debrief and share
• Questions?
20ExamSoft Assessment Conference | #ASSESS2015
Thank You
sjimenez@marian.edu
szahl@marian.edu

Utilizing Multiple Grader Rubrics for Practical Assessment of Student Performance

  • 1.
    1ExamSoft Assessment Conference| #ASSESS2015 ExamSoft Assessment Conference #ASSESS2015
  • 2.
    2ExamSoft Assessment Conference| #ASSESS2015 Utilizing Multiple Grader Rubrics for Practical Assessment of Student Performance Sherry Jimenez, Ed.D. Sarah Zahl, Ph.D. Assistant Dean for Educational Development Director of Educational Assessment
  • 3.
    3ExamSoft Assessment Conference| #ASSESS2015 Agenda • Audience Response • MUCOM Brief Overview • Multiple Grader Rubrics – Assessment Types and Examples – Tracking outcomes in rubrics • Recommendations • Small Group- Pair/Share • Q&A
  • 5.
  • 7.
    7ExamSoft Assessment Conference| #ASSESS2015 Audience Response
  • 8.
    8ExamSoft Assessment Conference| #ASSESS2015 MUCOM Overview • Competency Based Curricular Design • Systems based/integrated curriculum – Course Director, multiple internal faculty and external clinicians – e.g. Cardiovascular course + OSCE • iPad exclusive • Building a culture of evidence among faculty/students/administrators
  • 9.
    9ExamSoft Assessment Conference| #ASSESS2015 Tracking Student Performance • Competencies • COMLEX blueprint • System • Discipline • Learning Domains • Course specific outcomes
  • 10.
    10ExamSoft Assessment Conference| #ASSESS2015 National Board of Osteopathic Medical Examiners Fundamental Osteopathic Medical Competencies COMLEX Blueprint
  • 11.
    11ExamSoft Assessment Conference| #ASSESS2015 Use of Rubrics • Lab Practicals: Osteopathic Manipulative Medicine (OMM) Lab • Simulations • Objective Structured Clinical Examinations (OSCE) • Small Group Evaluations: Clinical Medicine
  • 12.
    12ExamSoft Assessment Conference| #ASSESS2015 Small Group Evaluations
  • 13.
    13ExamSoft Assessment Conference| #ASSESS2015 Small Group Evaluations – Grading Process
  • 14.
    14ExamSoft Assessment Conference| #ASSESS2015 Small Group Evaluations – Grading Process • Complete the administrative planning process externally before using ExamSoft • Pre-assign faculty graders and students • Pre-assign assessment area (region) • Prepare extra materials and guides • Communication
  • 15.
    15ExamSoft Assessment Conference| #ASSESS2015 OSCE • Humanistic rubric, SOAP note, SP evaluation, self-assessment
  • 16.
    16ExamSoft Assessment Conference| #ASSESS2015 Lab Practicals • Paper  Very complex online process  streamlined process in ExamSoft Old process: multiple stations, partner flip, multiple regions, multiple graders and students at each station New process: one station, partner flip only, one region (pre- determined but surprise to student)
  • 17.
    17ExamSoft Assessment Conference| #ASSESS2015 Outcomes • Culture shift –modifying structure • Positive feedback – ease of evaluation • Immediate faculty feedback – inter-rater • Timeliness of student feedback • Pairing rubric data with written exam data to develop longitudinal reports • Continuous refinement
  • 18.
    18ExamSoft Assessment Conference| #ASSESS2015 Group/Pair Share • How are you currently assessing student performance in practical settings? • How could you incorporate multiple grader rubrics in your curriculum?
  • 19.
    19ExamSoft Assessment Conference| #ASSESS2015 Discussion • Debrief and share • Questions?
  • 20.
    20ExamSoft Assessment Conference| #ASSESS2015 Thank You sjimenez@marian.edu szahl@marian.edu

Editor's Notes

  • #5 How is your institution using ExamSoft? Select all that apply. https://www.polleverywhere.com/multiple_choice_polls/cOnt95DSsqojr4B
  • #6 How is your institution using ExamSoft? Select all that apply. https://www.polleverywhere.com/multiple_choice_polls/cOnt95DSsqojr4B
  • #7 Please choose your discipline/subject area/specialty. https://www.polleverywhere.com/multiple_choice_polls/A9iSpUgcMrOZzrX
  • #8 Please choose your discipline/subject area/specialty. https://www.polleverywhere.com/multiple_choice_polls/A9iSpUgcMrOZzrX
  • #12 sz