interACT:
Interactive
Assessment and
Collaboration via
Technology
Dr Sean McAleer – Programme Director   Grant Murray – Learning
                                             Technologist CME
Embarked on a literature review…
And synthesised some principles:
1. Supports personalised learning
2. Ensures feedback leads to learning
3. Focuses on student self-development
4. Stimulates dialogue
5. Inducts students in the educational culture
and assessment and feedback practices
6. Is acceptable to all stakeholders
Baseline activities
• Document analysis: external examiner reports
  (2006-2011); end of course evaluations (2006-
  2011); additional evaluation surveys
  conducted in 2010 and 2011 as part of our
  curriculum review; and the HEA postgraduate
  student experience survey
• Focus groups with staff including part-time
  tutors
• Feedback audit
Baseline activities revealed
1. Inconsistency in the quality and quantity of
     feedback provided
2. Assessment design (e.g. over-assessment,
     over-reliance on essays, lack of formative
     assessment)
3. Timeliness of the feedback
4. Lack of assessment and feedback dialogue
Reason for Feedback
Focus of Feedback
Re-engineered our assessment
          • Modular approach to
            sequencing of assessment
          • Explicating feedback times
            in a student-tutor charter
          • Development of
            assessment rubrics for all
            summative
          • Increased use of formative
            assessments
PG Certificate in Medical Education


   Equivalent 2   Equivalent 2       Equivalent 2       Equivalent 2   Equivalent 2   Equivalent 2   Equivalent 2   Equivalent 2   Equivalent 2   Equivalent 2
   chapters       chapters           chapters           chapters       chapters       chapters       chapters       chapters       chapters       chapters




  Equivalent 2    Equivalent 2       Equivalent 2   Equivalent 2       Equivalent 2   Equivalent 2   Equivalent 2   Equivalent 2   Equivalent 2   Equivalent 2
  chapters        chapters           chapters       chapters           chapters       chapters       chapters       chapters       chapters       chapters




   Revised PG Certificate in Medical Education

                                                                                                                       Formative assessment
Chapters    2                    4                  6                   8              10
                                                                                                                       Medium stakes assessment
                                                                                                                       High stakes assessment
Chapters   2                     4                  6                   8             10




Chapters   2                     4                  6                   8              10


                                                                                                                          ©Mark Russell 2010
Chapters    2                    4                  6                   8              10                           University of Hertfordshire
                                                                                                                                ESCAPE project
Faculty development
     • Workshops about feedback
       and importantly agreement
       on how to give feedback
       (process)
Include self-review
   • Self-review is a scenario where
     students make evaluative
     judgements about the own work
     and produce a written feedback
     commentary

   Nicol, D. (2012). Assessment and feedback - in the hands
   of the student [Online]. JISC. Available:
   http://jiscdesignstudio.pbworks.com/w/page/50118521
   /Assessment%20and%20feedback%20-
   %20in%20the%20hands%20of%20the%20student
   [Accessed 01/02/12].
Content (understanding of theory / principles and          Self-evaluation:
application to own context)

- Understanding of learning theory

- Evidence of critical reflection on learning theory and
key learning and teaching principles

- Evidence of application of learning theory to own
practice                                                   Tutor feedback:




Style, format and language (e.g. structure, coherence,     Self-evaluation:
flow, formatting, use of language)




                                                           Tutor feedback:




Sources and references (e.g. range of references cited,    Self-evaluation:
relevance, consistency, accuracy and completeness of
referencing)



                                                           Tutor feedback:




Which aspect(s) of your assignment would you               Student comment:
specifically like feedback on?




                                                           Tutor feedback:




How did previous feedback inform this assignment?          Student comment:


Name of Tutor:                                                       Date:
Engaging students in processing of
   feedback through the wiki
         When you receive your assignment
         with feedback please upload a copy to
         your wiki and include a reflection on
         the following four questions:
         1. How well does the tutor feedback
             match with your self-evaluation?
         2. What did you learn from the
             feedback process?
         3. What actions, if any, will you take in
             response to the feedback process?
         4. What if anything is unclear about
             the tutor feedback?
Aspects I would like feedback on:
• Ideas of not preparing too rigidly in order to be flexible
  within sessions – practical advice would be welcomed!
• As the first essay I have written in nearly 20 years, I would
  like to know whether the standard overall was acceptable
• Please advise me how I can enter a specific page number in
  a reference when using Endnote (I wanted to add “p. 58” to
  the first reference used in the text since this is an exact
  quote, but failed to find out how I can do it, despite using
  the help option of the software).
• Feedback on whether my peers have had similar thoughts
  for their own teaching, or other ideas that have been
  commonly developed would be beneficial in case I have not
  thought or considered them.
• any part of it
How did previous feedback inform this
            assignment
• It made me realise that instead of focusing on a
  single or a few key teaching principles, I focused
  on many of them without going into much detail.
  Also I had used bullet points in the text.
• Feedback that my writing style was agreeable
  was reassuring. I appreciated knowing my use of
  literature was valid and supportive in the previous
  assignment, so have tried to continue applying
  the literature to my work.
• I tried to be careful to define and reference jargon
• It was really helpful in writing present assignment
I really like the cover sheet and request for
self assessment and what you would like
feedback about ... consistent with principles of
feedback!! (M&L end of module feedback)
Added challenges of:



                       For sharing feedback
              •   Relationship and trust
              •   Asynchronous dialogue and
                  shared understanding
              •   Reliance on written feedback
              •   Technology
Future developments
• Develop a toolkit of e-activities designed to
  improve self, peer and tutor dialogic feedback
  through the use of available technologies such
  as blogs, Friendfeed, Skype/webinars, or
  synchronous chat
• Introduce a patchwork assignment at the end
  of the Certificate and
  Diploma
• Work to refine technology
Project website
http://blog.dundee.ac.uk/interact

InterACT JISC Case Study

  • 1.
  • 2.
    Dr Sean McAleer– Programme Director Grant Murray – Learning Technologist CME
  • 4.
    Embarked on aliterature review… And synthesised some principles: 1. Supports personalised learning 2. Ensures feedback leads to learning 3. Focuses on student self-development 4. Stimulates dialogue 5. Inducts students in the educational culture and assessment and feedback practices 6. Is acceptable to all stakeholders
  • 5.
    Baseline activities • Documentanalysis: external examiner reports (2006-2011); end of course evaluations (2006- 2011); additional evaluation surveys conducted in 2010 and 2011 as part of our curriculum review; and the HEA postgraduate student experience survey • Focus groups with staff including part-time tutors • Feedback audit
  • 6.
    Baseline activities revealed 1.Inconsistency in the quality and quantity of feedback provided 2. Assessment design (e.g. over-assessment, over-reliance on essays, lack of formative assessment) 3. Timeliness of the feedback 4. Lack of assessment and feedback dialogue
  • 8.
  • 9.
  • 12.
    Re-engineered our assessment • Modular approach to sequencing of assessment • Explicating feedback times in a student-tutor charter • Development of assessment rubrics for all summative • Increased use of formative assessments
  • 13.
    PG Certificate inMedical Education Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 chapters chapters chapters chapters chapters chapters chapters chapters chapters chapters Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 chapters chapters chapters chapters chapters chapters chapters chapters chapters chapters Revised PG Certificate in Medical Education Formative assessment Chapters 2 4 6 8 10 Medium stakes assessment High stakes assessment Chapters 2 4 6 8 10 Chapters 2 4 6 8 10 ©Mark Russell 2010 Chapters 2 4 6 8 10 University of Hertfordshire ESCAPE project
  • 14.
    Faculty development • Workshops about feedback and importantly agreement on how to give feedback (process)
  • 15.
    Include self-review • Self-review is a scenario where students make evaluative judgements about the own work and produce a written feedback commentary Nicol, D. (2012). Assessment and feedback - in the hands of the student [Online]. JISC. Available: http://jiscdesignstudio.pbworks.com/w/page/50118521 /Assessment%20and%20feedback%20- %20in%20the%20hands%20of%20the%20student [Accessed 01/02/12].
  • 16.
    Content (understanding oftheory / principles and Self-evaluation: application to own context) - Understanding of learning theory - Evidence of critical reflection on learning theory and key learning and teaching principles - Evidence of application of learning theory to own practice Tutor feedback: Style, format and language (e.g. structure, coherence, Self-evaluation: flow, formatting, use of language) Tutor feedback: Sources and references (e.g. range of references cited, Self-evaluation: relevance, consistency, accuracy and completeness of referencing) Tutor feedback: Which aspect(s) of your assignment would you Student comment: specifically like feedback on? Tutor feedback: How did previous feedback inform this assignment? Student comment: Name of Tutor: Date:
  • 17.
    Engaging students inprocessing of feedback through the wiki When you receive your assignment with feedback please upload a copy to your wiki and include a reflection on the following four questions: 1. How well does the tutor feedback match with your self-evaluation? 2. What did you learn from the feedback process? 3. What actions, if any, will you take in response to the feedback process? 4. What if anything is unclear about the tutor feedback?
  • 18.
    Aspects I wouldlike feedback on: • Ideas of not preparing too rigidly in order to be flexible within sessions – practical advice would be welcomed! • As the first essay I have written in nearly 20 years, I would like to know whether the standard overall was acceptable • Please advise me how I can enter a specific page number in a reference when using Endnote (I wanted to add “p. 58” to the first reference used in the text since this is an exact quote, but failed to find out how I can do it, despite using the help option of the software). • Feedback on whether my peers have had similar thoughts for their own teaching, or other ideas that have been commonly developed would be beneficial in case I have not thought or considered them. • any part of it
  • 19.
    How did previousfeedback inform this assignment • It made me realise that instead of focusing on a single or a few key teaching principles, I focused on many of them without going into much detail. Also I had used bullet points in the text. • Feedback that my writing style was agreeable was reassuring. I appreciated knowing my use of literature was valid and supportive in the previous assignment, so have tried to continue applying the literature to my work. • I tried to be careful to define and reference jargon • It was really helpful in writing present assignment
  • 20.
    I really likethe cover sheet and request for self assessment and what you would like feedback about ... consistent with principles of feedback!! (M&L end of module feedback)
  • 21.
    Added challenges of: For sharing feedback • Relationship and trust • Asynchronous dialogue and shared understanding • Reliance on written feedback • Technology
  • 22.
    Future developments • Developa toolkit of e-activities designed to improve self, peer and tutor dialogic feedback through the use of available technologies such as blogs, Friendfeed, Skype/webinars, or synchronous chat • Introduce a patchwork assignment at the end of the Certificate and Diploma • Work to refine technology
  • 23.