Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Assessing students and giving feedback


Published on

Published in: Education, Business, Technology
  • Feedback videos can be found at;
    Are you sure you want to  Yes  No
    Your message goes here

Assessing students and giving feedback

  1. 1. Assessing Students & Giving Feedback Sean Polreis Educational Support & Development Council of Health Science Deans Office, University of Saskatchewan
  2. 2. Objectives <ul><li>Distinguish between formative and summative assessment </li></ul><ul><li>Discriminate between various assessment instruments </li></ul><ul><li>Decide on appropriate ways to assess students. </li></ul>
  3. 3. Objectives Learning Activity Goal is to improve performance Formative Feedback Games & Puzzles Summative Final mark Grade Pass/Fail Assessment
  4. 4. CanMEDS Assessment Tools <ul><li>Handout </li></ul>
  5. 5. 1. Multi-Source Assessment <ul><li>Who should give it? </li></ul><ul><li>What are the benefits? </li></ul><ul><li>Why are residents so negative about peer and patient (and other) assessment? </li></ul><ul><li>How can we overcome this fear? </li></ul>
  6. 6. 2. Direct Observation <ul><li>The learner is observed performing </li></ul><ul><li>work which occurs naturally in real </li></ul><ul><li>clinical settings during their training </li></ul><ul><li>period. </li></ul>
  7. 7. Direct Observation Advantages <ul><li>Authentic assessment </li></ul><ul><li>Assess wide variety of skills – nearly all key competencies of CanMEDS roles </li></ul><ul><li>Variety of perspectives/assessors is possible </li></ul><ul><li>Assess higher order behaviours – may be difficult to assess in exams </li></ul><ul><li>Very useful as formative assessment – immediate feedback usually possible </li></ul>
  8. 8. Direct Observation Disadvantages <ul><li>Standardization difficult (validity suffers) </li></ul><ul><li>Multiple perspectives – different standards of performance </li></ul><ul><li>Behaviour varies with little control over the situations observed </li></ul>
  9. 9. Observation Instruments <ul><li>Field Notes </li></ul><ul><li>Checklists </li></ul><ul><li>In-Training Evaluation Reports </li></ul>
  10. 10. Selectivity □ Focused/appropriate □ Establish priorities □ Urgent vs. non-urgent □ Complete, thorough Clinical Reasoning Clinical Reasoning □ Hypotheses / Diff. Dx □ Gather data (Hx & Px) □ Interpret data □ Make Decisions □ Set goals / Objectives Professionalism □ Responsible/Reliable/Trustworthy □ Know Limits □ Flexible/Resourceful □ Evokes Confidence □ Caring/Compassionate □ Maintains Boundaries □ Respectful □ Ethical/Honest □ Evidence Influenced □ Community Responsive □ Good Balance □ Mindful Approach  History  Physical  Hypothesis  Investigation  Diagnosis  Procedure  Management/Treatment  Referral  Follow-up Date: __________________ Resident ______________________ Supervisor ________________________ Patient: M F AGE______ Diagnosis/Procedure ____________________ Directly Observed □Yes □ NO Patient centered approach □ Explores illness /disease □ Understand context □ Common ground □ Build relationship □ Be realistic □ Health promotion / prevention Procedural skills □ Decision to Act □ Informed Consent □ Preparation □ During procedure (comfort/safety) □ If problems reevaluate □ After Care/Follow-up Physical Exam □ Appropriate Exam / Technique □ Competent Performance Communication □ Verbal □ Written & Charting □ Listening □ Non-Verbal □ Culture/Gender/Age Appropriate □ Attitudinal Based on Dalhousie University’s work Field Notes
  11. 11. Continue (Strengths): Do More (Area for Development): Consider (Learning Objectives): Stop (or do less): Reporter  Interpreter  Manager  Educator  Procedure competence achieved  Resident’s Initials ____________ Supervisor’s Initials ______ Field Notes (Back)
  12. 12. 3. Rubrics <ul><li>Rubrics are explanations of </li></ul><ul><li>assessment which reveal the “scoring </li></ul><ul><li>rules” – the criteria against which work </li></ul><ul><li>will be judged. If shared with the </li></ul><ul><li>learner it gives guidance learners can </li></ul><ul><li>use to develop, revise, and judge their </li></ul><ul><li>own work. </li></ul>
  13. 13. Advantages of Rubrics <ul><li>Gives learners information on how to improve – formative </li></ul><ul><li>Describes what constitutes good or excellent performance – real life standards of the profession </li></ul><ul><li>Reinforce intended learning outcomes </li></ul><ul><li>Guides instructor feedback to be specific </li></ul><ul><li>Summative – reliability enhanced </li></ul>
  14. 14. Questions to Consider <ul><li>What would constitute high quality work? </li></ul><ul><li>What components contribute to quality? </li></ul><ul><li>What is substantial vs. superficial? </li></ul><ul><li>What are the consequences of poor performance? </li></ul>
  15. 15. Other Considerations <ul><li>Use objective descriptions to help give guidance </li></ul><ul><li>Use words that describe qualitative differences – not comparative words (like “more” or “less” of a certain quality) </li></ul><ul><li>Example: Exemplary – Proficient – Acceptable – Unacceptable </li></ul>
  16. 16. Rubric Steps <ul><li>Essential criteria required for high quality work </li></ul><ul><ul><li>General abilities and skills </li></ul></ul><ul><ul><li>Specific knowledge/content </li></ul></ul><ul><li>Levels of achievement – number and descriptor </li></ul><ul><li>Performance consequences at each level </li></ul><ul><li>Rating scheme – scale and weighting </li></ul>
  17. 17. 4. Portfolios <ul><li>A portfolio is collection of materials and documentation which provides evidence of learning - skills and knowledge. It offers proof that competencies have been attained. </li></ul>Supports Learning Demonstrates Competence
  18. 18. What’s in a portfolio? <ul><li>Case histories </li></ul><ul><li>Questionnaires used </li></ul><ul><li>Images and video </li></ul><ul><li>Projects </li></ul><ul><li>Teaching sessions attended </li></ul><ul><li>Presentations </li></ul><ul><li>References/testimonials </li></ul>
  19. 19. Types of Portfolios <ul><li>1. Unstructured </li></ul><ul><ul><li>Only learning goals are defined </li></ul></ul><ul><li>2. Structured </li></ul><ul><ul><li>Type of activity needed to achieve the competency and documentation required are also predetermined </li></ul></ul><ul><li>3. Semi-Structured </li></ul><ul><ul><li>Balanced approach combining #1 & #2 (likely best) </li></ul></ul><ul><ul><li>Provides some freedom </li></ul></ul>
  20. 20. Portfolio Purpose <ul><li>Formative </li></ul><ul><ul><li>Reflection </li></ul></ul><ul><ul><li>Identify areas requiring work </li></ul></ul><ul><li>Summative </li></ul><ul><li>- Certifying predetermined </li></ul><ul><li>competencies </li></ul><ul><li>Mixed </li></ul><ul><li>- Useful for both instructor and </li></ul><ul><li>learner. </li></ul>
  21. 21. Steps Involved <ul><li>Define competencies – list required skills and knowledge </li></ul><ul><li>Create timeline </li></ul><ul><li>Describe actions required to acquire competencies </li></ul><ul><li>Assessment criteria </li></ul><ul><ul><li>- Expectations and standards </li></ul></ul><ul><ul><li>- % for various areas </li></ul></ul>
  22. 22. Portfolio Advantages <ul><li>Authentic – real activities of residents </li></ul><ul><li>Incremental/Growth – progress over time </li></ul><ul><li>Learner-centered – multiple ways to demonstrate achievement </li></ul><ul><li>Focus and direction provided at outset </li></ul><ul><li>Encourage learner responsibility </li></ul><ul><li>Encourage reflection </li></ul><ul><li>Broad and comprehensive – beyond medical expert role </li></ul>
  23. 23. Portfolio Disadvantages <ul><li>Complexity – content and competencies must be formulated at outset </li></ul><ul><li>Time consuming – planning, compiling, and assessing </li></ul><ul><li>Content overload – selectivity important – only include materials that facilitate assessment and demonstrate competencies </li></ul><ul><li>Difficulty converting qualitative information into quantitative data </li></ul>
  24. 24. Assessment Summary <ul><li>It is important to choose an assessment method that is appropriate for the role/competency or objective </li></ul><ul><li>Multi-source, direct observation, rubrics, and portfolios have unique ways of assessing a variety of skills and knowledge </li></ul><ul><li>These methods can be used in both formative and summative assessment </li></ul>
  25. 25. Giving Effective Feedback <ul><li>Objectives: </li></ul><ul><li>Describe the benefits and importance of effective feedback </li></ul><ul><li>Using the mnemonic THANCS, describe the important factors involved in giving positive feedback </li></ul><ul><li>Describe how feedback changes based on the learner’s location in the learning cycle. </li></ul>
  26. 26. Goal of Feedback? <ul><li>The goal of feedback is to improve learner performance. </li></ul><ul><li>Receptivity of the learner is important. </li></ul><ul><li>Supportive learning environment </li></ul><ul><li>Trusting relationship </li></ul><ul><li>Separate behaviour from the person </li></ul><ul><li>Use descriptive, non-judgmental language </li></ul>
  27. 27. Barriers? <ul><li>Complex skill </li></ul><ul><li>Giving negative feedback </li></ul><ul><li>Insufficient time </li></ul><ul><li>Insufficient data </li></ul><ul><li>Using secondary data </li></ul><ul><ul><li>Less time available to observe learner </li></ul></ul>
  28. 28. Types of Feedback <ul><li>Minimal – Emotional </li></ul><ul><li>Behavioural – One Way </li></ul><ul><li>Interactive – Goal Based </li></ul>
  29. 29. Feedback (THANCS) <ul><li>T imely </li></ul><ul><li>H elpful </li></ul><ul><li>A ppropriate </li></ul><ul><li>N ever labelling or demoralizing </li></ul><ul><li>C ollaborative and culturally sensitive </li></ul><ul><li>S pecific </li></ul>
  30. 30. Feedback Videos <ul><li>Discuss effective and </li></ul><ul><li>ineffective feedback methods </li></ul><ul><li>based on these videos . </li></ul><ul><li> </li></ul>
  31. 31. Theoretical Underpinning Depending on how the student evaluates themselves, they are … Unconsciously Incompetent Video observation 360 Feedback Stop Start Consciously Incompetent Identify Behaviour Start Do more Stop Consciously Competent Encourage Continue Do more Teach others Unconsciously Competent
  32. 32. Feedback Videos <ul><li>Identify strengths and weaknesses </li></ul><ul><li>in the following videos. </li></ul><ul><li> </li></ul>
  33. 33. Feedback Steps <ul><li>Mutually set goals and expectations – prepare for feedback at the start of a rotation </li></ul><ul><li>Select time to meet </li></ul><ul><li>Have learner self assess – listen carefully – probe and respond </li></ul><ul><li>Collaborate on solutions </li></ul><ul><li>Assess understanding/acceptance </li></ul>
  34. 34. Feedback Summary <ul><li>Since the goal of feedback is to improve learner performance , what factors are important: </li></ul><ul><ul><li>Collaborate - interactive </li></ul></ul><ul><ul><li>Be specific – helpful </li></ul></ul><ul><ul><li>Give in a timely manner </li></ul></ul><ul><ul><li>Supportive environment – not personal </li></ul></ul><ul><ul><li>Identify learner weaknesses and strengths </li></ul></ul><ul><ul><ul><li>Learning cycle awareness </li></ul></ul></ul>