Assessing Students & Giving Feedback Sean Polreis Educational Support & Development Council of Health Science Deans Office...
Objectives <ul><li>Distinguish between formative and summative assessment </li></ul><ul><li>Discriminate between various a...
Objectives Learning Activity Goal is to improve performance Formative Feedback Games & Puzzles Summative Final mark Grade ...
CanMEDS Assessment Tools <ul><li>Handout </li></ul>
1. Multi-Source Assessment <ul><li>Who should give it? </li></ul><ul><li>What are the benefits? </li></ul><ul><li>Why are ...
2. Direct Observation <ul><li>The learner is observed performing </li></ul><ul><li>work which occurs naturally in real </l...
Direct Observation Advantages <ul><li>Authentic assessment </li></ul><ul><li>Assess wide variety of skills – nearly all ke...
Direct Observation Disadvantages <ul><li>Standardization difficult (validity suffers)  </li></ul><ul><li>Multiple perspect...
Observation Instruments <ul><li>Field Notes </li></ul><ul><li>Checklists </li></ul><ul><li>In-Training Evaluation Reports ...
Selectivity □ Focused/appropriate □ Establish priorities □ Urgent vs. non-urgent □ Complete, thorough Clinical Reasoning C...
Continue (Strengths): Do More (Area for Development): Consider (Learning Objectives): Stop (or do less): Reporter   Inter...
3. Rubrics <ul><li>Rubrics are explanations of  </li></ul><ul><li>assessment which reveal the “scoring </li></ul><ul><li>r...
Advantages of Rubrics <ul><li>Gives learners information on  how  to improve –  formative </li></ul><ul><li>Describes what...
Questions to Consider <ul><li>What would constitute high quality work? </li></ul><ul><li>What components contribute to qua...
Other Considerations <ul><li>Use objective descriptions to help give guidance </li></ul><ul><li>Use words that describe qu...
Rubric Steps <ul><li>Essential criteria required for high quality work </li></ul><ul><ul><li>General abilities and skills ...
4. Portfolios <ul><li>A portfolio is collection of materials and documentation which provides evidence of learning - skill...
What’s in a portfolio? <ul><li>Case histories </li></ul><ul><li>Questionnaires used </li></ul><ul><li>Images and video </l...
Types of Portfolios <ul><li>1.  Unstructured </li></ul><ul><ul><li>Only learning goals are defined </li></ul></ul><ul><li>...
Portfolio Purpose <ul><li>Formative </li></ul><ul><ul><li>Reflection </li></ul></ul><ul><ul><li>Identify areas requiring w...
Steps Involved <ul><li>Define competencies – list required skills and knowledge </li></ul><ul><li>Create timeline </li></u...
Portfolio Advantages <ul><li>Authentic – real activities of residents </li></ul><ul><li>Incremental/Growth – progress over...
Portfolio Disadvantages <ul><li>Complexity – content and competencies must be formulated at outset </li></ul><ul><li>Time ...
Assessment Summary <ul><li>It is important to choose an assessment method that is appropriate for the  role/competency or ...
Giving Effective Feedback <ul><li>Objectives: </li></ul><ul><li>Describe the benefits and importance of effective feedback...
Goal of Feedback? <ul><li>The goal of feedback is to improve learner performance. </li></ul><ul><li>Receptivity of the lea...
Barriers? <ul><li>Complex skill </li></ul><ul><li>Giving negative feedback </li></ul><ul><li>Insufficient time </li></ul><...
Types of Feedback <ul><li>Minimal – Emotional  </li></ul><ul><li>Behavioural – One Way </li></ul><ul><li>Interactive – Goa...
Feedback (THANCS) <ul><li>T  imely </li></ul><ul><li>H  elpful </li></ul><ul><li>A  ppropriate </li></ul><ul><li>N  ever l...
Feedback Videos <ul><li>Discuss effective and  </li></ul><ul><li>ineffective feedback methods </li></ul><ul><li>based on t...
Theoretical Underpinning Depending on how the student evaluates themselves, they are … Unconsciously Incompetent Video obs...
Feedback Videos <ul><li>Identify strengths and weaknesses </li></ul><ul><li>in the following videos. </li></ul><ul><li>htt...
Feedback Steps <ul><li>Mutually set goals and expectations – prepare for feedback at the start of a rotation </li></ul><ul...
Feedback Summary <ul><li>Since the goal of feedback is to  improve learner performance , what factors are important: </li>...
Upcoming SlideShare
Loading in …5
×

Assessing students and giving feedback

1,758
-1

Published on

Published in: Education, Business, Technology
1 Comment
0 Likes
Statistics
Notes
  • Feedback videos can be found at;
    http://www.practicalprof.ab.ca/observation_feedback/giving_feedback.html
    http://medicine.usask.ca/tips/giving_feedback.php
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total Views
1,758
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
65
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide
  • Ask for definition of formative and summative.
  • Handouts; “Observation Checklist”; “U of T ITER”.
  • 3 Handouts; “Scientific Report”; “Problem-Solving”; “Service Learning”.
  • Anything that helps to demonstrate acquisition of a competency. It will also show what is lacking in terms of competencies.
  • 1 Handout.
  • Clinical skills –diagnostic and therapeutic as well as communication, professionalism, collaboration, etc. Facilitates exchange between learner and instructor.
  • Portfolios offer a rare opportunity to use workplace-based assessment (unlike an OSCE) for learners. Discuss the role/purpose/benefits of portfolios with learners to help with buy-in. Portfolios are excellent ways of encouraging independent continuing professional development and encourage learner responsibility for their own learning. Ensure confidentiality with personal reflective pieces of portfolios.
  • Note this is similar to patient encounters.
  • Assessing students and giving feedback

    1. 1. Assessing Students & Giving Feedback Sean Polreis Educational Support & Development Council of Health Science Deans Office, University of Saskatchewan
    2. 2. Objectives <ul><li>Distinguish between formative and summative assessment </li></ul><ul><li>Discriminate between various assessment instruments </li></ul><ul><li>Decide on appropriate ways to assess students. </li></ul>
    3. 3. Objectives Learning Activity Goal is to improve performance Formative Feedback Games & Puzzles Summative Final mark Grade Pass/Fail Assessment
    4. 4. CanMEDS Assessment Tools <ul><li>Handout </li></ul>
    5. 5. 1. Multi-Source Assessment <ul><li>Who should give it? </li></ul><ul><li>What are the benefits? </li></ul><ul><li>Why are residents so negative about peer and patient (and other) assessment? </li></ul><ul><li>How can we overcome this fear? </li></ul>
    6. 6. 2. Direct Observation <ul><li>The learner is observed performing </li></ul><ul><li>work which occurs naturally in real </li></ul><ul><li>clinical settings during their training </li></ul><ul><li>period. </li></ul>
    7. 7. Direct Observation Advantages <ul><li>Authentic assessment </li></ul><ul><li>Assess wide variety of skills – nearly all key competencies of CanMEDS roles </li></ul><ul><li>Variety of perspectives/assessors is possible </li></ul><ul><li>Assess higher order behaviours – may be difficult to assess in exams </li></ul><ul><li>Very useful as formative assessment – immediate feedback usually possible </li></ul>
    8. 8. Direct Observation Disadvantages <ul><li>Standardization difficult (validity suffers) </li></ul><ul><li>Multiple perspectives – different standards of performance </li></ul><ul><li>Behaviour varies with little control over the situations observed </li></ul>
    9. 9. Observation Instruments <ul><li>Field Notes </li></ul><ul><li>Checklists </li></ul><ul><li>In-Training Evaluation Reports </li></ul>
    10. 10. Selectivity □ Focused/appropriate □ Establish priorities □ Urgent vs. non-urgent □ Complete, thorough Clinical Reasoning Clinical Reasoning □ Hypotheses / Diff. Dx □ Gather data (Hx & Px) □ Interpret data □ Make Decisions □ Set goals / Objectives Professionalism □ Responsible/Reliable/Trustworthy □ Know Limits □ Flexible/Resourceful □ Evokes Confidence □ Caring/Compassionate □ Maintains Boundaries □ Respectful □ Ethical/Honest □ Evidence Influenced □ Community Responsive □ Good Balance □ Mindful Approach  History  Physical  Hypothesis  Investigation  Diagnosis  Procedure  Management/Treatment  Referral  Follow-up Date: __________________ Resident ______________________ Supervisor ________________________ Patient: M F AGE______ Diagnosis/Procedure ____________________ Directly Observed □Yes □ NO Patient centered approach □ Explores illness /disease □ Understand context □ Common ground □ Build relationship □ Be realistic □ Health promotion / prevention Procedural skills □ Decision to Act □ Informed Consent □ Preparation □ During procedure (comfort/safety) □ If problems reevaluate □ After Care/Follow-up Physical Exam □ Appropriate Exam / Technique □ Competent Performance Communication □ Verbal □ Written & Charting □ Listening □ Non-Verbal □ Culture/Gender/Age Appropriate □ Attitudinal Based on Dalhousie University’s work Field Notes
    11. 11. Continue (Strengths): Do More (Area for Development): Consider (Learning Objectives): Stop (or do less): Reporter  Interpreter  Manager  Educator  Procedure competence achieved  Resident’s Initials ____________ Supervisor’s Initials ______ Field Notes (Back)
    12. 12. 3. Rubrics <ul><li>Rubrics are explanations of </li></ul><ul><li>assessment which reveal the “scoring </li></ul><ul><li>rules” – the criteria against which work </li></ul><ul><li>will be judged. If shared with the </li></ul><ul><li>learner it gives guidance learners can </li></ul><ul><li>use to develop, revise, and judge their </li></ul><ul><li>own work. </li></ul>
    13. 13. Advantages of Rubrics <ul><li>Gives learners information on how to improve – formative </li></ul><ul><li>Describes what constitutes good or excellent performance – real life standards of the profession </li></ul><ul><li>Reinforce intended learning outcomes </li></ul><ul><li>Guides instructor feedback to be specific </li></ul><ul><li>Summative – reliability enhanced </li></ul>
    14. 14. Questions to Consider <ul><li>What would constitute high quality work? </li></ul><ul><li>What components contribute to quality? </li></ul><ul><li>What is substantial vs. superficial? </li></ul><ul><li>What are the consequences of poor performance? </li></ul>
    15. 15. Other Considerations <ul><li>Use objective descriptions to help give guidance </li></ul><ul><li>Use words that describe qualitative differences – not comparative words (like “more” or “less” of a certain quality) </li></ul><ul><li>Example: Exemplary – Proficient – Acceptable – Unacceptable </li></ul>
    16. 16. Rubric Steps <ul><li>Essential criteria required for high quality work </li></ul><ul><ul><li>General abilities and skills </li></ul></ul><ul><ul><li>Specific knowledge/content </li></ul></ul><ul><li>Levels of achievement – number and descriptor </li></ul><ul><li>Performance consequences at each level </li></ul><ul><li>Rating scheme – scale and weighting </li></ul>
    17. 17. 4. Portfolios <ul><li>A portfolio is collection of materials and documentation which provides evidence of learning - skills and knowledge. It offers proof that competencies have been attained. </li></ul>Supports Learning Demonstrates Competence
    18. 18. What’s in a portfolio? <ul><li>Case histories </li></ul><ul><li>Questionnaires used </li></ul><ul><li>Images and video </li></ul><ul><li>Projects </li></ul><ul><li>Teaching sessions attended </li></ul><ul><li>Presentations </li></ul><ul><li>References/testimonials </li></ul>
    19. 19. Types of Portfolios <ul><li>1. Unstructured </li></ul><ul><ul><li>Only learning goals are defined </li></ul></ul><ul><li>2. Structured </li></ul><ul><ul><li>Type of activity needed to achieve the competency and documentation required are also predetermined </li></ul></ul><ul><li>3. Semi-Structured </li></ul><ul><ul><li>Balanced approach combining #1 & #2 (likely best) </li></ul></ul><ul><ul><li>Provides some freedom </li></ul></ul>
    20. 20. Portfolio Purpose <ul><li>Formative </li></ul><ul><ul><li>Reflection </li></ul></ul><ul><ul><li>Identify areas requiring work </li></ul></ul><ul><li>Summative </li></ul><ul><li>- Certifying predetermined </li></ul><ul><li>competencies </li></ul><ul><li>Mixed </li></ul><ul><li>- Useful for both instructor and </li></ul><ul><li>learner. </li></ul>
    21. 21. Steps Involved <ul><li>Define competencies – list required skills and knowledge </li></ul><ul><li>Create timeline </li></ul><ul><li>Describe actions required to acquire competencies </li></ul><ul><li>Assessment criteria </li></ul><ul><ul><li>- Expectations and standards </li></ul></ul><ul><ul><li>- % for various areas </li></ul></ul>
    22. 22. Portfolio Advantages <ul><li>Authentic – real activities of residents </li></ul><ul><li>Incremental/Growth – progress over time </li></ul><ul><li>Learner-centered – multiple ways to demonstrate achievement </li></ul><ul><li>Focus and direction provided at outset </li></ul><ul><li>Encourage learner responsibility </li></ul><ul><li>Encourage reflection </li></ul><ul><li>Broad and comprehensive – beyond medical expert role </li></ul>
    23. 23. Portfolio Disadvantages <ul><li>Complexity – content and competencies must be formulated at outset </li></ul><ul><li>Time consuming – planning, compiling, and assessing </li></ul><ul><li>Content overload – selectivity important – only include materials that facilitate assessment and demonstrate competencies </li></ul><ul><li>Difficulty converting qualitative information into quantitative data </li></ul>
    24. 24. Assessment Summary <ul><li>It is important to choose an assessment method that is appropriate for the role/competency or objective </li></ul><ul><li>Multi-source, direct observation, rubrics, and portfolios have unique ways of assessing a variety of skills and knowledge </li></ul><ul><li>These methods can be used in both formative and summative assessment </li></ul>
    25. 25. Giving Effective Feedback <ul><li>Objectives: </li></ul><ul><li>Describe the benefits and importance of effective feedback </li></ul><ul><li>Using the mnemonic THANCS, describe the important factors involved in giving positive feedback </li></ul><ul><li>Describe how feedback changes based on the learner’s location in the learning cycle. </li></ul>
    26. 26. Goal of Feedback? <ul><li>The goal of feedback is to improve learner performance. </li></ul><ul><li>Receptivity of the learner is important. </li></ul><ul><li>Supportive learning environment </li></ul><ul><li>Trusting relationship </li></ul><ul><li>Separate behaviour from the person </li></ul><ul><li>Use descriptive, non-judgmental language </li></ul>
    27. 27. Barriers? <ul><li>Complex skill </li></ul><ul><li>Giving negative feedback </li></ul><ul><li>Insufficient time </li></ul><ul><li>Insufficient data </li></ul><ul><li>Using secondary data </li></ul><ul><ul><li>Less time available to observe learner </li></ul></ul>
    28. 28. Types of Feedback <ul><li>Minimal – Emotional </li></ul><ul><li>Behavioural – One Way </li></ul><ul><li>Interactive – Goal Based </li></ul>
    29. 29. Feedback (THANCS) <ul><li>T imely </li></ul><ul><li>H elpful </li></ul><ul><li>A ppropriate </li></ul><ul><li>N ever labelling or demoralizing </li></ul><ul><li>C ollaborative and culturally sensitive </li></ul><ul><li>S pecific </li></ul>
    30. 30. Feedback Videos <ul><li>Discuss effective and </li></ul><ul><li>ineffective feedback methods </li></ul><ul><li>based on these videos . </li></ul><ul><li>http://www.practicalprof.ab.ca/observation_feedback/giving_feedback.html </li></ul>
    31. 31. Theoretical Underpinning Depending on how the student evaluates themselves, they are … Unconsciously Incompetent Video observation 360 Feedback Stop Start Consciously Incompetent Identify Behaviour Start Do more Stop Consciously Competent Encourage Continue Do more Teach others Unconsciously Competent
    32. 32. Feedback Videos <ul><li>Identify strengths and weaknesses </li></ul><ul><li>in the following videos. </li></ul><ul><li>http://medicine.usask.ca/tips/giving_feedback.php </li></ul>
    33. 33. Feedback Steps <ul><li>Mutually set goals and expectations – prepare for feedback at the start of a rotation </li></ul><ul><li>Select time to meet </li></ul><ul><li>Have learner self assess – listen carefully – probe and respond </li></ul><ul><li>Collaborate on solutions </li></ul><ul><li>Assess understanding/acceptance </li></ul>
    34. 34. Feedback Summary <ul><li>Since the goal of feedback is to improve learner performance , what factors are important: </li></ul><ul><ul><li>Collaborate - interactive </li></ul></ul><ul><ul><li>Be specific – helpful </li></ul></ul><ul><ul><li>Give in a timely manner </li></ul></ul><ul><ul><li>Supportive environment – not personal </li></ul></ul><ul><ul><li>Identify learner weaknesses and strengths </li></ul></ul><ul><ul><ul><li>Learning cycle awareness </li></ul></ul></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×