Building a scalable ePortfolio based assessment strategy to connect clinical learning experiences – preliminary lessons Helen Wozniak Shari Bowker

•Download as PPTX, PDF•

1 like•159 views

The development of professional expertise in medicine requires a lifelong commitment to education. During university training medical students encounter a wide variety of practical experiences making it difficult for them to monitor and evaluate their development towards clinical competence. The use of an eportfolio to collate feedback about their assessed performance can assist in making clearer connections between these experiences and aid in the transition to novice practitioners. Over the past year the University of Queensland’s medical program has been refining their approach to assessment in the clinical learning environment and has trialled the use of an eportfolio to manage the delivery of workplace-based assessments (WBAs). This paper outlines the design stages undertaken as well as the roadblocks that have been navigated concluding with an overview of the key lessons that emerged during the early stages of eportfolio adoption.

Helen Wozniak
Shari Bowker
Office of Medical Education
Building a scalable ePortfolio based
assessment strategy to connect clinical
learning experiences: Preliminary lessons

Audience context
pollev.com/ntmp
(Not case sensitive)

1. Assessment
design
2.
Assessment
processes
3. Translation
to ePortfolio
Iterative process

Outcomes-based and Competency-based medical education
Programmatic assessment
1. Assessment design
Low stakes High stakes
One data point
- Focused on
information
- Feedback oriented
- Not decision oriented
Intermediate program
decision
- More data points
needed
- Focus on diagnosis,
remediation, prediction
Final decision on progression
- Many data points needed
- Focusing on an important decision
(non-surprising)
Van der Vleuten, 2017

Components of the workplace learning portfolio
1. History
2. Examination
3. Clinical reasoning
4. Clinical management & procedural skills
5. Clinical communication
6. Professional practice
7. Social/cultural competence and safety
8. Reflective practice

Assessment of short clinical activity
- Mini clinical evaluation exercise - Mini-CEX
- 3 types (History, Examination, Management)
- Direct observation of procedural skills – DOPS
Global assessment across placement
- Clinical Participation Assessment (CPA)
Workplace-based assessments in medicine

Mini-CEX - History
Unsatisfactory Borderline Competent Proficient
Mapping to WLP
components
History taking
technique
History
Clinical reasoning
during history
taking
Clinical reasoning
Professionalism –
manner with
patient
Clinical communication
Global rating
Overall results

2017 – small
pilot
2018 – wider
implementation
3. Translation to
ePortfolio
1

Student Assessor
Course
Coordinator
Student
Coordinator
Us (OME)
Use Case Development
Pilot testing of 2019 WBA plan with students
• Evaluate acceptability

Monitoring of
learning
CPA:
Professionalism

Importance of Learning Design
WBAs
Preparation
Rationale
Guides
Sequence
over time
Learning Outcome
Oliver et al 2002, Goodyear, 2010
Learning Activity
Rubrics
Assessment
Roadmap
ePortfolio

Next steps / Questions
pollev.com/ntmp
(Not case sensitive)

Goodyear, P. (2010) Teaching, technology and educational
design: The architecture of productive learning environments. Final
report Senior Fellowship, ALTC
Oliver, R. & Harper, B., Hedberg, J., Wills, S., & Agostinho, S.
(2002). Exploring strategies to formalise the description of learning
designs. In A. Goody, J. Herrington & M. Northcote (Eds.), Quality
conversations: Research and Development in Higher Education,
Volume 25 (pp. 496-504). Jamison, ACT: HERDSA.
Van der Vleuten, C. & Fuller, R. (2017). Advanced Assessment
Course, Singapore, April 2017

This document discusses using multiple grader rubrics to assess student performance. It provides an overview of Marshall University College of Osteopathic Medicine (MU-COM), which uses a competency-based curriculum and iPad-exclusive assessments. The presentation discusses using rubrics to evaluate lab practicals, simulations, objective structured clinical examinations, and small group evaluations. Rubrics allow tracking student performance across competencies and disciplines. Benefits of multiple grader rubrics include immediate faculty feedback, timelier student feedback, and generating longitudinal reports. The presentation concludes with questions about assessing practical student performance and incorporating multiple grader rubrics.

Effective course evaluation

surveyresults

The document discusses improving course evaluation methods at universities. It notes low survey response rates are a key issue and that in-class administration can help increase participation. Students want feedback that directly benefits them and surveys should engage students and clearly communicate the actions taken in response. A consistent, centralized approach to surveys is recommended to allow benchmarking, but individual departments should have flexibility to include bespoke questions. Effective course evaluation provides evidence of a university's value to students.

Course evaluation questionarie example

Merve Özdemir

The document appears to be a course evaluation questionnaire that asks students to rate their agreement with 15 statements about their lecturer and course. Students are asked to choose one response on a scale from "I totally agree" to "I totally disagree" or "I have no idea" for statements about the lecturer announcing course goals, lessons being planned, highlighting important points, the course's usefulness, using effective tools/materials, providing active participation opportunities, understandable speech, taking time outside class for topics, content relating to topics and goals, measuring success relating to content and goals, informing about exams/assignments/projects, regular attendance, classroom management, being a good role model, and effective student presentations.

$J:\The Evaluation$ $J:\The Evaluation$

J:\The Evaluation

gmjthomas

The document discusses the evaluation/assessment process in nursing education. It describes the process as the systematic collection and analysis of data to determine merit and render judgment. Evaluation is used to determine the effectiveness of learning processes, student progress, and an institution's ability to prepare nurses. Evaluation can be formative, providing ongoing feedback, or summative, assessing achievement at the end. Several models and methods of evaluation are presented, including tests, written work, observations, and rubrics.

Leigh Davis And Moira Cordiner 2008

Diana Quinn

The document describes the process of designing and implementing the first iteration of an evidence-based practice (EBP) curriculum for undergraduate nursing students. A team developed a major assessment task around a clinical scenario instead of an essay. They evaluated student and teacher feedback which indicated that while most found the subject relevant, only 62% found the difficulty level satisfactory. The main problem was insufficient development of EBP skills before applying them in assessments. Future iterations will provide more scaffolding and practice of EBP skills and remove the policy document requirement from the major task.

Course Evaluation

Annasta Tastha

This document discusses different approaches to evaluating educational courses and programs. It describes product-oriented, static-characteristic, process-oriented, and decision-facilitation approaches. It also discusses the dimensions of evaluation, including formative vs summative, process vs product, and quantitative vs qualitative. Formative evaluation aims to improve a curriculum during development, while summative evaluates success after completion. Process evaluation focuses on program workings, and product evaluates goal achievement. Quantitative data are numbers, while qualitative are more holistic observations.

Understanding assessment

reneeaitken

The document discusses Northcentral University's approach to assessment, which has three tiers: program assessment, specialization assessment, and student learning assessment. The assessments are systematic, iterative, and use data to demonstrate student learning and identify areas for improvement. Program assessment evaluates overall program results every two years. Specialization assessment examines learning in each specialization. Student learning assessment evaluates individual student performance at the course level using rubrics to measure outcomes. Together these assessments form a data-driven system to ensure student learning and program quality.

This document proposes a model for programmatic assessment that optimizes assessment for learning while arriving at robust decisions about learner progress. The model distinguishes between learning activities, assessment activities, and learner support activities throughout an ongoing curriculum. Individual assessments are designed to be maximally informative for learning, while a longitudinal program of various assessment methods contributes to certification decisions. The principles discussed include ensuring validity in standardized and non-standardized assessments, using both quantitative and qualitative data, and relying on expert judgement at various evaluation points. An example is provided of how this model could be applied to a blended TeleGeriatrics Nurse Training Course.

Language course evaluation

Carlos Mayora

Prium Vienna 2008 Eval 2 * PRIUM Project : What is next ?

unica.projects

The document outlines the quality assurance policies and procedures at the University of Vienna. It discusses the mission, aims, and organization of quality assurance efforts, which include ensuring international standards, promoting innovation, and providing accountability. It describes the comprehensive peer-review evaluation process conducted every 5 years for faculties and service units, including a self-assessment report, site visit, and follow-up measures. Student course evaluations are also used to improve teaching quality.

Referring for Evaluation

Sherry Copley

This document provides guidance for referring students for a multidisciplinary evaluation after attempting research-based classroom interventions. It lists the documentation that should be included in the referral packet, such as the student's educational history, benchmark data, targeted areas of concern, details of classroom instruction and interventions provided, diagnostic assessment results, target goals, peer achievement levels, and progress monitoring results. It also outlines the process for scheduling an evaluation meeting, providing parents with procedural safeguards and notice of the referral decision.

$D:\Ug Flex\Project Presentations\Ug Flex Pdg Presentation 25 5 10$ $D:\Ug Flex\Project Presentations\Ug Flex Pdg Presentation 25 5 10$

D:\Ug Flex\Project Presentations\Ug Flex Pdg Presentation 25 5 10

Claire Eustance

The UG-Flex project aims to transform university systems and processes over four years to better support flexible curricula. This will help reduce inefficiencies, improve guidance for staff, and better serve students in non-standard programs. The project was prompted by goals to offer more flexible learning options and changing student and industry demands. Stakeholder consultation found diversity in flexibility approaches and that current systems can discourage innovation. The project will redesign systems and processes through piloting and then full implementation.

1.1.geiger

afacct

The study aimed to determine which student readiness factors most influence success in well-designed online courses. Researchers hypothesized that individual attributes, life factors, comprehension, general knowledge, reading rate, and typing speed impact success. Data from 200 students in 11 online courses at a community college was analyzed. Statistical analysis found typing speed, reading rate, and recall significantly impacted grades, while other factors did not. The findings suggest colleges should assess these skills and provide remedial tutorials to improve student readiness for online learning.

Reporting to parents

Learning Media, NZ

Assessments for Programs and Learning

Lisa MacLeod

Overall, assessments are used either as a Programmatic Assessment or as a Learning Assessment. One of the most familiar learning assessments is the multiple choice assessment that reflects the typical pen and paper traditional classroom test (Popham, 2006). However, these tests are not very easy to construct to ensure validity due to unclear directions, ambiguous statements, unintended clues, complicated syntax and difficult vocabulary (Popham, 2006). Other learning assessments with construct validity, such as the essay and the reflective journal, tend to focus on student-centered pedagogy. These assessments are ideal for assessing the learning outcomes of the individual and increase the student’s personal responsibility for their own learning. This reading document provides a brief summary of assessment tools that are available for both programmatic and learning.

Anja cedefop prelim infos

Ecuisine

This document discusses approaches to establishing a quality culture within vocational education and training (VET) institutions. It outlines the methodology used in a study of 20 VET cases across 13 European countries. Common elements found include use of self-assessment and PDCA cycles to collect stakeholder feedback. Successful self-assessment requires staff commitment, involvement in identifying improvement actions, and using results to inform changes. External support structures can help by providing quality frameworks, indicators, data collection tools, and training for VET staff. The goal is a handbook and tools to help VET institutions strengthen internal quality management.

The hidden achievement factor - SXSW

emergenttree

This document discusses the effects of behavior systems on academic outcomes. It outlines key components of multi-tiered behavior support systems including understanding behavior science, team-based problem solving, data analysis, aligned behavior curriculum, acknowledgement of appropriate behaviors, consistent responses to violations, and individualized interventions. It shows how implementing these components fully at all tiers leads to improved student progress and campus culture, while limited or inconsistent implementation is associated with reduced student progress, staff buy-in, and instructional effectiveness. The document provides recommendations for districts to develop their knowledge and implementation of behavior support systems from limited to high levels.

Achieve Webinar Slides: Student Assessment Inventory Tool for School Districts

Achieve, Inc.

June 11, 2014 Achieve Webinar: Student Assessment Inventory Tool for School Districts On a webinar, Achieve provided an overview of the newly released Student Assessment Inventory Tool for School Districts, designed to guide district administrators and instructional leaders through a process to ensure that students are only taking the minimum number of tests necessary to serve essential instructional, diagnostic and accountability purposes. Beyond teacher-developed classroom assessments that are core to the learning process, it is clear that many students are required to take a host of additional assessments that have too little value. This tool is designed to provide an opportunity for district leaders to take stock and take action. Achieve was joined by state leaders who have been sharing the inventory tool with districts as a framework to guide reflection and decision-making, including Abe Krisst, Technology Readiness Coordinator, Connecticut State Department of Education and Nancy DePalma, Assistant Superintendent for Curriculum, Instruction & Assessment, West Hartford Public Schools. The Student Assessment Inventory for School Districts can be access on the Achieve website at http://www.achieve.org/meetings-webinars

John McInerney 2014 - Training assessors

Building a scalable ePortfolio based assessment strategy to connect clinical learning experiences – preliminary lessons Helen Wozniak Shari Bowker

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Building a scalable ePortfolio based assessment strategy to connect clinical learning experiences – preliminary lessons Helen Wozniak Shari Bowker

Similar to Building a scalable ePortfolio based assessment strategy to connect clinical learning experiences – preliminary lessons Helen Wozniak Shari Bowker (20)

More from ePortfolios Australia

More from ePortfolios Australia (20)

Recently uploaded

Recently uploaded (20)

Building a scalable ePortfolio based assessment strategy to connect clinical learning experiences – preliminary lessons Helen Wozniak Shari Bowker

Editor's Notes