Utilizing Multiple Grader Rubrics for Practical Assessment of Student Performance

•Download as PPTX, PDF•

1 like•720 views

Presented by Sherry Jimenez and Sarah Zahl Vertical- Medical This interactive session will discuss methods for utilizing ExamSoft multiple grader rubrics for practical assessment activities. Building a multiple grader rubric and assigning graders to student groups will be demonstrated. Procedures to track and categorize student performance data will be discussed and strategies for faculty collaboration will be shared. Physical and technological logistics associated with utilizing multiple grader rubrics for practical assessments will be outlined for replication. Utilizing audience response technology and small group discussion, participants will be actively engaged in an exchange surrounding the topics presented and how they can be applied in their academic setting. A discussion surrounding ways multiple grader rubrics can be incorporated at attendees’ institutions to support practical aspects of student assessment will ensue. At the end of the session, participants will be able to: 1) Describe methods for utilizing ExamSoft multiple grader rubrics for practical assessment activities 2) Identify logistical considerations associated with administering practical assessments 3) Develop a process for replicating the use of multiple grader rubrics

1ExamSoft Assessment Conference | #ASSESS2015
ExamSoft Assessment Conference
#ASSESS2015

2ExamSoft Assessment Conference | #ASSESS2015
Utilizing Multiple Grader Rubrics for Practical
Assessment of Student Performance
Sherry Jimenez, Ed.D. Sarah Zahl, Ph.D.
Assistant Dean for Educational Development Director of Educational Assessment

3ExamSoft Assessment Conference | #ASSESS2015
Agenda
• Audience Response
• MUCOM Brief Overview
• Multiple Grader Rubrics
– Assessment Types and Examples
– Tracking outcomes in rubrics
• Recommendations
• Small Group- Pair/Share
• Q&A

7ExamSoft Assessment Conference | #ASSESS2015
Audience Response

8ExamSoft Assessment Conference | #ASSESS2015
MUCOM Overview
• Competency Based Curricular Design
• Systems based/integrated curriculum
– Course Director, multiple internal faculty and external
clinicians
– e.g. Cardiovascular course + OSCE
• iPad exclusive
• Building a culture of evidence among
faculty/students/administrators

9ExamSoft Assessment Conference | #ASSESS2015
Tracking Student Performance
• Competencies
• COMLEX blueprint
• System
• Discipline
• Learning Domains
• Course specific outcomes

10ExamSoft Assessment Conference | #ASSESS2015
National Board of Osteopathic Medical Examiners
Fundamental Osteopathic Medical Competencies
COMLEX Blueprint

11ExamSoft Assessment Conference | #ASSESS2015
Use of Rubrics
• Lab Practicals: Osteopathic Manipulative Medicine
(OMM) Lab
• Simulations
• Objective Structured Clinical Examinations (OSCE)
• Small Group Evaluations: Clinical Medicine

12ExamSoft Assessment Conference | #ASSESS2015
Small Group Evaluations

13ExamSoft Assessment Conference | #ASSESS2015
Small Group Evaluations – Grading Process

14ExamSoft Assessment Conference | #ASSESS2015
Small Group Evaluations – Grading Process
• Complete the administrative planning process
externally before using ExamSoft
• Pre-assign faculty graders and students
• Pre-assign assessment area (region)
• Prepare extra materials and guides
• Communication

15ExamSoft Assessment Conference | #ASSESS2015
OSCE
• Humanistic rubric, SOAP note, SP evaluation,
self-assessment

16ExamSoft Assessment Conference | #ASSESS2015
Lab Practicals
• Paper  Very complex online process  streamlined
process in ExamSoft
Old process: multiple
stations, partner flip,
multiple regions,
multiple graders and
students at each
station
New process: one
station, partner flip
only, one region (pre-
determined but
surprise to student)

17ExamSoft Assessment Conference | #ASSESS2015
Outcomes
• Culture shift –modifying structure
• Positive feedback – ease of evaluation
• Immediate faculty feedback – inter-rater
• Timeliness of student feedback
• Pairing rubric data with written exam data to develop
longitudinal reports
• Continuous refinement

18ExamSoft Assessment Conference | #ASSESS2015
Group/Pair Share
• How are you currently assessing student
performance in practical settings?
• How could you incorporate multiple grader
rubrics in your curriculum?

19ExamSoft Assessment Conference | #ASSESS2015
Discussion
• Debrief and share
• Questions?

20ExamSoft Assessment Conference | #ASSESS2015
Thank You
sjimenez@marian.edu
szahl@marian.edu

Presented by Sally-Ann Clement, Teah Cummings, Avril Best, Leisha Griffith-Mark, and Wayne Radix Vertical- Medical In an effort to find a user friendly and cost effective solution to administering examinations St. George’s University (SGU) researched several computer based options. After careful consideration, ExamSoft was identified as the best solution for its needs as it allowed for the easy handling of all major aspects of the exam process. The decision to transition was also based on the fact that computer based examinations have become the standard format for administering exams in higher education. The rollout of the computer based exams at SGU was phased, with an initial pilot in three School of Medicine (SOM) courses. Through our ExamSoft Pilot, the University was able to identify key resources needed as well as resolve major issues in preparation for a university wide rollout of the software. The Course Directors/Pilot Users were not only instrumental in this process but also helped boost the confidence of their peers some of whom had reservations about the transition process. After successful completion of the Pilot, the university felt confident about extending the use of ExamSoft to other schools specifically the School to Veterinary Medicine (SVM) and later the School of Arts and Science (SAS). Exam Managers found the unique capabilities of the software such as the use of categories, reports and the availability of certain security features for example exam password, snapshot etc. to be very useful. Additionally, the availability of the highlight features as well as the question review pane provides an enhanced exam experience. The development of student policies and guidelines for Exam Managers were beneficial as they provide clear procedures for the use of ExamSoft at SGU. This proposal will not only focus on both the successes and challenges SGU faced during the implementation of ExamSoft but will also focus on how faculty and students were engaged throughout the process. At the end of the session, participants will be: Equipped with guidelines for the successful implementation of ExamSoft at an educational institution.

LRT Talks 20161012 Jisc Student Experience Experts Group

Mark Stubbs

School management system neeschashokkumar_rdr

TLC2016 - KULeuven's new learning portal and css/JS tweaks for course design

BlackboardEMEA

Presenter: Bert Coenen Organisation: KU Leuven Description: At KU Leuven, we developed a new portal to replace our tabs-and-modules-based portal. The portal runs outside / on top of / alongside Blackboard. It aggregates information from different sources - Enrollments - Updates and messages - User-targeted tools, links and information Key features: - Course list - Inline access to updates, messages, settings - Filter, search and customization tools: - Favorites - Colours - Banners We're also introducing a new design for course pages. Key features: - Based on the latest Learn theme - Enhancements - Simplified, inline content creation - ""Intelligent"" learning modules - ""Productivity"" enhancements - Availability of courses and items - Drag and drop - ... - Responsive design

TLC2016 - A showcase of using BB LEARN in large courses

BlackboardEMEA

Presenter: Robert Rozier Organisation: Hanze University Groningen Description: At the School of Communication Media and IT eduction is structured in integrated domains. Web programming is one of these domains. Students have to attend a number of different courses. Courses are subdivided into lectures and practicals, demand that online instructional videos are to be viewed and have formative tests at regular intervals. The ramifications are that heavy burden is placed on the LMS since it has to both incorporate all the different courses, projects, assignments, and has to allow for attendance registration. The number of users and of roles is high. We will present a show case of how to structure a course in BB thus reducing this complexity especially for instructors.

Data and Trust - Norway

EduSkills OECD

The role of assessment data in Austria

EduSkills OECD

LRT Talks 20150415 Talis Insight Birmingham

Mark Stubbs

Presented by Connie Remsberg and Danielle Birch Vertical- Pharmacy Faculty at Washington State University have implemented ExamSoft® rubrics within their lab-based courses for activities such as patient counseling, SOAP note writing, and grading of compounding lab products, simulation exercises, and informal patient case presentations. This has helped to increase collaboration by allowing multiple faculty members to take part in the grading process. Presenters will share Washington State University’s experience with rubrics including: Multiple rubrics for one assessment Multiple graders for one assessment Utilizing the detailed grader report Rubrics for essays Rubrics for simulations Rubrics for compounding lab products and recordsPresenters will end with a discussion on the challenges and successes with rubrics followed by a Q & A with participants. At the end of the session, participants will be able to: 1) Develop and use rubrics for essay grading 2) Assign multiple rubrics to one assessment 3) Assign multiple graders to specific students 4) Utilize checklist in rubric format to evaluate skills based activities 5) Identify challenges and benefits of adopting computerized rubric technology to assess skills-based assessments

Learning Analytics - Sweden

EduSkills OECD

Building Sakaigers: Students as QA Teams

Dede Hourican

Naace Strategic Conference 2009: The Next Generation of assessment - GL Asses...Naace Naace

What are e portfolios inquiry project

aries_shine

Governing Complex Education Systems: The Use of Data, Tracey Burns, OECD

EduSkills OECD

Developing Data Systems - Portugal

EduSkills OECD

Gba project

Lawton Milanovich

EdVIE - Features - Collaborate without boundaries

srikanthreddy7979

DegreeWorks Project - Board Presentation - FinalCalvin B. Madlock

6230slides

ReeseyT1

New Interactive Career and College Planning Activities for Middle School

North Carolina Association for Middle Level Education

New Interactive Career and College Planning Activities for Middle School This new set of 21 online activities (7 per grade) includes guided exercises, readings, inventories, videos, and games to help students learn about themselves and options for their future. Please join us to see activities, student work products, and to consider how these free activities from CFNC.org can fit into your school plan. Presenters: Mark Wiles & Lisa Sommerfeldt - University of North Carolina General Administration - Greensboro, NC

Successfully Delivering Secure Online Exams at Scale

Ashley Wright

In Semester 1 of this academic year (2013-14) Newcastle University successfully delivered 71 exams, with more than 9000 student sittings across a multitude of venues across campus. The only additional software cost has been a site-wide Respondus Licence used to convert existing formatted questions and produce backup paper exam copies. By building on Blackboard's existing functionality, we have provided a streamlined process for building and running secure, locked down online exams with additional reporting and feedback functionality. This presentation outlines how the OLAF (Online Assessment and Feedback) project has evolved since 2007 to its present state today. The stakeholders involved in a delivering a complex cross-institutional service and the tools we have developed to assist us to keep track of multiple exam delivery across multiple PC venues. Our technical support team have developed a free in-house solution for providing a locked down browser, using Firefox portable and a compatible extension. During the session we will explore the pros and cons of Blackboard as an assessment system, outlining what can be achieved and where we are hitting limitations and whether we can do anything about these limitations. The existing assessment tools in Blackboard can meet the majority of needs for online examinations at present. We can use exported data from Blackboard to produce additional reports and student feedback. Yet there are limitations to the testing that can be conducted through Blackboard that will stop some users from fulfilling their requirements. There are still features that other software offers that staff would like to use, such as breaking tests into sections, more advanced text marking and so on. Should institutions be aiming for more online assessments with more sophisticated features, or should we stick with what the majority need to use? During the session we will encourage debate about short term/long term needs and strategies

PDD 2019 Developing an in-house self directed CV support e-learning resourc...

uopces

SAM to the Rescue - Course Technology Computing Conference

Cengage Learning

SAM to the Rescue - Course Technology Computing Conference Presenter: Leslie Martin & Teresa Worthy, Gaston College Want to provide online students with immediate feedback to ensure they are learning skills? Want to know the students have grasped the knowledge and skills needed for the workplace? Want to see Student Outcome Success? Want to prepare students to take the Microsoft Office Certification exams? If you answer yes to any of these questions, this session is for you. Gaston College is utilizing SAM in the Online and Seated Office Software Applications course to accomplish all of these. We will discuss and show how we set up the course using the Blackboard System and incorporate SAM as a vital component for our virtual or seated classroom success. We utilize SAM to ensure that our students know the material that is necessary for success on the final exam. We will discuss how we use the trainings, skills test, projects and reader in SAM to help students succeed. This course is one of our measurement courses for our Medical Office, Legal Office and Office Administration programs for our SACS and ACBSP accreditation. The goals for the session are: Discuss “Why use SAM?”; Show attendees how you can verify that students are learning material; Show how students can receive immediate feedback and know what they need to remediate; Discuss how our students use SAM to prepare for the Microsoft Office Certification Exams. Attendees will walk away with the knowledge of how to utilize SAM in their course to help their students be successful using the Microsoft Office products.

SLTCC2017 What makes a Virtual Learning Environment good? (Shane McMordie)

Solent Learning and Teaching Institute

Data Warehouse Tube Map Update

Mark Stubbs

Student services regional introduction to redesign

cccscoetc

InstructureCon 2014: Using the Canvas API for Faculty Feedback

Seth Battis

New USQ assessment functionality: Out with the old in with the new

Charles Darwin University

RubricsS Kennedy

Rubrics