Download to read offline
Uploaded byLACE Project
LACE Spring Briefing - WatchMe project overview
The document discusses the WatchMe project, which aims to develop a mobile electronic portfolio system to improve feedback and assessment for competency-based higher education. The system will integrate workplace-based feedback and assessment using modules for feedback, visualization, and e-assessment of entrustable professional activities (EPAs). It will combine the EPA approach with learning analytics to provide tailored feedback based on assessments. The portfolio system, called ePASS, will offer just-in-time feedback to learners before assessments and aggregate assessment data in visualizations.
More Related Content
Similar to LACE Spring Briefing - WatchMe project overview
LACE Spring Briefing - WatchMe project overview
- 1. www.project-watchme.eu @Project_WatchMe Workplace-based e-Assessment Technologyfor Competency- based Higher Multi-professional Education Marieke van der Schaaf, PC, Utrecht University, The Neth Denise Janssen, Mateum, The Neth.
- 2. 1. Utrecht University,NL 2. University Medical Centre Utrecht, NL 3. Szent Istvan University, Hungary 4. University of Tartu, Estonia 5. Universitätsmedizin Charité Berlin, Germany 6. University of California San Francisco, USA 7. Maastricht University, NL 8. Mateum, NL 9. University of Reading, UK 10.Jayway, Denmark 11.NetRom, Rumania/NL
- 3. WATCHME’S AIM Solve efficiencyand quality problems of workplace- based feedback and assessment by means of a mobile electronic portfolio system that is enhanced with a feedback module and a visualization module.
- 4. - Develop complexcompetences - Integrated in context - Demands long learning trajectories in workplace - Deliberate practice: feedback and reflection
- 5. Medical expert Communicator Collaborator Manager Health advocate Scholar Professional CRITICSCOMPETENCY-BASED ASSESSMENT MOST CRITICS COME DOWN TO THE ANALYTIC APPROACH (E.G. CANMEDS) Source: Ten Cate
- 6. Medical expert Communicator Collaborator Manager Health advocate Scholar Professional THEANALYTIC APPROACH With nursing staff With family With patients With colleagues With trainees … Source: Ten Cate
- 7. Medical expert Communicator Collaborator Manager Health advocate Scholar Professional THEANALYTIC APPROACH With nursing staff With family With patients With colleagues With trainees … Consultation Breaking bad news Explain medication With children With elderly … Source: Ten Cate
- 8. Medical expert Communicator Collaborator Manager Health advocate Scholar Professional THESYNTHETIC FRAMEWORK APPROACH EPA1 EPA2 EPA3 EPA4 EPA5 Source: Ten Cate
- 9. ENTRUSTABLE PROFESSIONAL ACTIVITY Taskbased instead of construct based approach Crucial question: would I entrust this candidate unsupervised with this task? (with my sick mother, animal or teach my daughter/son…) An EPA is a task that an individual can be trusted to perform unsupervised, in a given professional context, once sufficient competence has been demonstrated. International Competency-Based Medical Education Collaborators, March 18, 2014 Source: Ten Cate
- 10. COMBINE EPA APPROACHWITH LEARNING ANALYTICS Application of probabilistic student models that enable tailored visualised feedback based on multi sorted assessments Bayesian models provide detailed model of trainees’ entrustability
- 11. ELECTRONIC PORTFOLIO E AR L I S I G 1 M A D R I D A U G 2 0 1 4
- 12. E A RL I S I G 1 M A D R I D A U G 2 0 1 4
- 13. E A RL I S I G 1 M A D R I D A U G 2 0 1 4
- 14. E A RL I S I G 1 M A D R I D A U G 2 0 1 4
- 15. E A RL I S I G 1 M A D R I D A U G 2 0 1 4
- 16. E A RL I S I G 1 M A D R I D A U G 2 0 1 4
- 17. E A RL I S I G 1 M A D R I D A U G 2 0 1 4
- 18. E A RL I S I G 1 M A D R I D A U G 2 0 1 4
- 19. E A RL I S I G 1 M A D R I D A U G 2 0 1 4
- 20. E A RL I S I G 1 M A D R I D A U G 2 0 1 4
- 21. EXAMPLE JUST-IN-TIME FEEDBACK Beforeworkplace-based assessment is performed EPASS displays just-in-time feedback concerning learner. E.g.: “remind your communication skills towards patient. Explain treatment in simple words”.
- 22.
- 23. QUESTIONS OR WOULDYOU LIKE TO HAVE A LOOK AT EPASS? CONTACT ME DURING LUNCH BREAK! THANK YOU FOR YOUR ATTENTION!
- 24. Architecture database Mobile device Student Model Database LearningAnalytics Engine - Online Learning Justin time Feedback Aggregated Visualisation Portfolio system (ePASS) WPB AssessmentGAME EPA Matrix ARCHITECTURE
- 25. COMBINING THE COMPETENCY-FRAMEWORKWITH ENTRUSTABLE PROFESSIONAL ACTIVITIES Medical expert Communicator Collaborator Manager Health advocate Scholar Professional + + + + + ++ ++ + + + + ++ ++ + ++ ++ + ++ + ++ EPA1 EPA2 EPA3 EPA4 EPA5 AssessmentSource: Ten Cate
Editor's Notes
- #4 Portfolios contain selected evidence of performance and products accompanied by reflections. Underuse of potential data and not very well tailored to worksituations (not timely). Drawbacks: Authentic, personal unique character of evidence, difficults scoring. Full standardization impossible, asks for interpretation of assessor (no golden standard or external criterion). Impacts reliability and validity of the assessment. Limited impact on learning Low motivation of students Aim: improve feedback and assessment with an eportfolio; Continuing education at the workplace has seen increasing demand as a crucial means of acquiring the requisite professional knowledge and skills. Electronic Portfolios (e-portfolios) for workplace training provide a global view of each trainee’s progress but offer no dynamic feedback to exploit the rich learning assessment data that could be analysed to support responsive adaptation for more efficient and rewarding training. De basis is een bestaand digitaal portfoliosysteem genaamd Epass, waarover zometeen meer.
- #5 Leg uit waarom 3 disciplines
- #6 Canmeds toelichten
- #8 Subdivisions Risk of checklist approach 142 milestones PBLI on Evalue
- #9 When we use epas a combined task and construct approach occurs. Instead of analytic approach; combine
- #10 Acting is not allowed: more observation Acting is allowed under direct supervision: supervision physically present; proactive supervision Acting is allowed under indirect supervision: supervision not physically present, but quickly available if needed; reactive supervision Acting is allowed “unsupervised”:under clinical oversight; distant/backstage supervision; post hoc report Supervision may be provided to juniors (to junior residents, to medical students at level 2 or 3) A formal acknowledgment happens at level 4 To improve possibilities for feedback and assessment, bring it near to dialogue Crucial question for supervisors clinical practice medical graduates have to cope with unfamiliar clinical situations and tasks. These are situations that. It is clinicians/supervisors’ daily practice to determine whether or not a medical trainne is ready to cope with unfamiliar situations and tasks in a clinical environment. They observe adspeak a trainee at work and assess how well the act in a clinical setting. ?next, they determin whehter they would trust them with tasks and situations that are not shown. When is a medical trainee ready for independent practice? This question bothers medical educators for decades and underlies the decision to entrust clinical tasks. Features: Independently executable within a time frame Observable and measurable in process and outcome Suitable for entrustment decisions. Assessment-result always framed as supervision level Typically, EPAs can be scheduled and allocated to individuals. EPA concept was designed to make CBME work
- #11 Recent technological ddevelopmetns enable the colelciton and use of behaviorual data on massive scale. Todays’ cmputional poser, and high quality interactive gapi user interfaces make it possible to bringsophisticated analysis to end users quickl Milan, 2010: Bayesian student models aloow for a sound and detaile evaluation of each student, according to the granularity level defined by teachers In stead of having only a final grade to measure students’ performance, the system will be ble to provide a more detialed model of trainees entrustability which contains informaiton about which parts the crrciulum the student is strggling wiht and whih parts he/she has already mastered. This information is essential to provide feeback, remediation and personlized instruction node
- #12 Basis of project which we expand with student models and visualization modules
- #22 Providing detailed view on progress & dynamic just-in-time feedback VIA VIZ tools
- #23 Providing detailed view on progress & dynamic just-in-time feedback VIA VIZ tools
- #24 Providing detailed view on progress & dynamic just-in-time feedback VIA VIZ tools
- #26 curriclum