• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Appraising my Teaching Skills using MSF
 

Appraising my Teaching Skills using MSF

on

  • 2,208 views

 

Statistics

Views

Total Views
2,208
Views on SlideShare
1,744
Embed Views
464

Actions

Likes
0
Downloads
20
Comments
0

13 Embeds 464

http://colinsmededblog.blogspot.com 304
http://colinsmededblog.blogspot.co.uk 121
http://www.colinsmededblog.blogspot.co.uk 13
http://colinsmededblog.blogspot.in 9
http://www.slideshare.net 4
http://colinsmededblog.blogspot.de 3
http://colinsmededblog.blogspot.ie 2
http://colinsmededblog.blogspot.com.au 2
http://medicineedu.blogspot.in 2
http://colinsmededblog.blogspot.ca 1
http://colinsmededblog.blogspot.sg 1
http://colinsmededblog.blogspot.jp 1
http://colinsmededblog.blogspot.kr 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-NonCommercial LicenseCC Attribution-NonCommercial License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Appraising my Teaching Skills using MSF Appraising my Teaching Skills using MSF Presentation Transcript

    • Appraising my ‘IMPROMPTU WARD-BASED Teaching’
      Dr Colin Mitchell
      MSc Geriatric Medicine
      June 2009
    • Choosing a Topic
      I know some of my strengths as a teacher:
      Can explain complex concepts
      Keen to teach
      And I’m aware of my weaknesses
      Prone to long-windedness
      Can over-elaborate
      I already know to concentrate on the known weaknesses
      But are there any issues that I’m not fully aware of?
    • Is this right?
      Previous 360o feedback
      ‘Can be intimidating’
      Really?
      Jars with my self-image
      Could interfere with a conducive learning environment
    • The MSF - Keeping it simple
      Aim to focus on the learning environment
      Is the environment conducive to learning?
      Relaxed / Intimidating
      Is some pressure a good thing?
      Further focus on one skill: ward-based teaching (WBT)
      Multiple exposures, multiple sources of feedback
      Just F1s
    • Assessing Educational Environment
      DREEM - Dundee Ready Educational Environment Measure*
      50 Questions – agree / disagree
      Based on instruments in use for 40 years
      Designed by 80+ experts using the Delphi process
      Statistically validated to focus on reliable / discriminatory elements
      Used worldwide to assess medical school teaching
      (PHEEM)
      *Roff et al, 1997. “Development and validation of the DREEM. Medical Teacher 19:4, 205-209
    • Refining the Quantitative
      Principles of choosing the items*:
      Relevant
      Clear, concise, focused and simple
      Some balance of +ve and –ve (DREEM 82% +ve)
      Chose 7 from DREEM (slightly adapted)
      Added 4 of my own
      Some concordance, not repetition
      8 +ve, 3 -ve
      *Adapted from Berk (2006), 13 Strategies to Measure College Teaching. Stylus: Virginia
    • Choosing the scale
      DREEM uses 5-point bipolar ‘intensity’ anchors:
      Strongly Agree
      Agree
      Uncertain
      Disagree
      Strongly disagree
      Simple but allows a breadth of response
      Still sufficiently granular & simple
      Balanced (✔) and has a midpoint (?✔)
    • Defining the Qualitative
      Opportunity to explore particular issues more fully
      Correlate with quantitative themes
    • Defining the Qualitative
      Opportunity to explore particular issues more fully
      Correlate with quantitative themes
    • Making it anonymous - online
      MSF in UK training – anonymous. In industry – 97% anonymous
      “Maintaining the anonymity of the rater’s feedback is key to the process. Just as a tie to compensation can lead raters to soften their ratings, so can a lack of anonymity”*
      “…when peer assessment is used in a high stakes setting, it results in inflated estimates of performance... There is no way to completely avoid this issue but it may help to ensure the anonymity of evaluators”**
      Online forms preserve anonymity - This can be made clear in the MSF
      Rogers E et al, 2002. “Improving the Payoff from 360-Degree Feedback”. Human Resource Development. 25: 44-54
      **Norcini J, 2003. “Peer assessment of competence”. Medical Education. 37(6): 539-543
    • Validation
      Cannot empirically validate without significant resources
      Using 7 items from the DREEM – pre-validated
      My own questions must be logically validated
      Representative pilot
      Sent to 5 SHOs. 4 replied with comments (unstructured)
      Minor adjustments made to wording, stressing anonymity
    • Results
    • Rater Scores (% of Maximum)
    • Question Scores (Fractions of 100%)
    • Scoring
      For negative aspects the score is reversed
      Therefore: Higher score is a good thing
    • Academic Perceptions
    • Perception of the Teacher
    • Teaching Atmosphere
    • Free Responses (1)
      What are the strengths of my ward-based teaching?
      When a situation arises you take time to discuss the case (23/44*)
      Enthusiastic and approachable… make[s] the learner think the problem through in a logical manner (30/40)
      …instead of just giving an answer, you get us to think through the problem and help us work out the solution ourselves (24/44)
      *Commenter’s overall score for quantitative elements shown in parentheses
    • Free Responses (2)
      What do you like LEAST about my style of ward-based teaching? (Only 5/11 replied)
      Sometimes assume too much which can be uncomfortable (20/40)
      Sometimes can go into a lot of depth for too long (26/44)
      … on some occasions it would be easier to get an answer, especially when it gets busy, but I would not learn as well… (24/44)
      Sometimes it becomes a little didactic (25/44)
    • Free Responses (3)
      How does the pressure of on-the-spot quizzing affect your learning (+vely or –vely)?
      (In the US, “Pimping”generally viewed positively*)
      8/11 were generally positive, 2 didn’t answer
      Only negative response:
      Too pressurizing and can be intimidating and lead to not wanting to ask for teaching (29/44)
      *Wear D et al, 2005. “Pimping: perspectives of 4th year medical students”. Teaching and Learning in Medicine. 17(2) 184-191
    • Free Responses (4)
      Every teacher can improve his skills – how would you suggest I improve my ward-based teaching?
      Be a bit more aware of when people have got lost in your explanations (20/40)
      You are a good teacher and have a lot of knowledge, but could improve by talking more succinctly and not rambling (26/44)
      There is very little to improve on, your teaching is excellent (25/44)
      I think sometimes you loose [sic] the balance and focus too much on teaching rather than seeing the patients… and thus even though your teaching is good, you are not setting a good example for juniors (28/40)
    • Analysis
    • Robust?
      Some basic statistical analysis of average total score:
      Median = average (59%, around a 2.5)
      95% CI: ±19%
      Some observations can be made:
      Lower scorers more likely to give a critique in the weaknesses question
      Poor correlation between ridicule / intimidating / relaxed questions.
      The lowest scorer commented: “the relaxed manner of teaching is definitely a good approach”
      He/she also disagreed with the ‘relaxed atmosphere’ item (?)
    • Findings – My Teaching
      Quantitative results
      Overall positive (59% compares with DREEM results)
      I can be intimidating
      Yet the atmosphere is generally relaxed (=3rd highest scoring item)
      3 comments specifically mentioned friendly / approachable
      Themes emerging from qualitative results
      Rambling / long-windedness
      Correlates with ‘teacher-centred’
      Few negative comments about atmosphere
      Only one comment directly mentioned ‘intimidating’ or synonyms.
      Also note comment about assuming too much = ‘uncomfortable’
      This rater also agreed with both the intimidate and ridicule questions
    • Findings – The Process
      Getting good feedback is more difficult than it seems
      Despite:
      Limiting scope
      Using validated questions
      Piloting
      Anonymous data entry
      The results didn’t tell me much I didn’t already know
      Partially corroborated ‘intimidating’ problem
      Confirmed ‘long-winded’ problem
    • assimilation
      Borg Cube by Martin Teufel - http://www.infosun.fim.uni-passau.de/br/lehrstuhl/Sommercamp/virtualworld/2005/galerie/
    • How I see myself
      Image from http://www.theinsider.com/photos/1079058_TOM_CRUISE_FOR_TOP_GUN_2. Top Gun - Paramount Pictures.
    • How I see myself
    • YOU BOY!
      What I need to avoid:
      Val-de-ree!
      Val-de ha ha ha ha ha ha!
    • Further Actions
      I may not be intimidating to most…
      But clearly a few may find me so
      How do I know who they are?
      What do I do about it?
      Design my own feedback forms for my teaching sessions
      Add demographics
      Push for free-text responses
      Don’t be afraid of constructive criticism
      Criticism is worst when I don’t know how to fix the problem
    • Questions?
      Links:
      Email: drcolinmitchell@gmail.com
      http://colinsmededblog.blogspot.com
      Twitter: @drcolinmitchell
      http://obamicon.pastemagazine.com