Slides from eAssessment Conference 05Dec 2012

1. interACT: Interactive
Assessment and
Collaboration via
Technology

2. Outline
• Background to project
• What we did
• Where we are up to
• Preliminary evaluation data
• Challenges and future
development

3. The Project Team
Rola Ajjawi Karen Barton Grant Murray Susie Schofield
Project Director Project Officer Learning Technologist Project Manager
Natalie Lafferty Sean McAleer David Walker
e-learning Advisor Assessment Advisor Learning Technology
Advisor

5. Recognition of a need for change….
• External examiner reports (2006-2011)
• End of course evaluations (2006-2011)
• Additional evaluation surveys conducted in 2010
and 2011 as part of curriculum review
• HEA postgraduate student experience survey
• Staff discussions
• Feedback audit (5% random sample of all
assignments submitted in 2011 audited)
Outlined that…

6. Specific issues raised:
1. Inconsistency in the quality and quantity of
feedback provided
2. Assessment design (e.g. over-assessment,
over-reliance on essays, lack of formative
assessment)
3. Timeliness of the feedback
4. Lack of assessment and feedback dialogue

7. A critque of monologic feedback1-6
• Lack of learner engagement with feedback
• Lack of understanding of feedback
• Transmitted feedback creates dependency on
teacher
• Not utilising self/peer feedback
• Lack of a shared context for assessment for
teacher and learner
• High teacher effort— low efficiency
• Reduced staff satisfaction as evidence of
feedforward not seen

8. Shift conceptions of feedback
• Feedback should develop the students’
capacity to make evaluative judgements about
their own and others work1,2
• Feedback should serve the function of
progressively enabling students to better
monitor, evaluate and regulate their own
learning, independently of the teacher3

9. So what did we do…
http://youtu.be/S5bBFEbXDD0

10. Re-engineered our assessment
• Modular approach to
sequencing of assessment
• Explicating feedback times
in a student-tutor charter
• Development of
assessment rubrics for all
summative
• Increased use of formative
assessments

11. PG Certificate in Medical Education
Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2
chapters chapters chapters chapters chapters chapters chapters chapters chapters chapters
Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2
chapters chapters chapters chapters chapters chapters chapters chapters chapters chapters
Revised PG Certificate in Medical Education
Formative assessment
Chapters 2 4 6 8 10
Medium stakes assessment
High stakes assessment
Chapters 2 4 6 8 10
Chapters 2 4 6 8 10
©Mark Russell 2010
Chapters 2 4 6 8 10 University of Hertfordshire
ESCAPE project

12. Faculty development
• Workshops about feedback
and importantly agreement
on how to give feedback
(process)

13. Include self-review
• Self-review is a scenario where
students make evaluative
judgements about the own work
and produce a written feedback
commentary4

14. Content (understanding of theory / principles and Self-evaluation:
application to own context)
- Understanding of learning theory
- Evidence of critical reflection on learning theory and
key learning and teaching principles
- Evidence of application of learning theory to own
practice Tutor feedback:
Style, format and language (e.g. structure, coherence, Self-evaluation:
flow, formatting, use of language)
Tutor feedback:
Sources and references (e.g. range of references cited, Self-evaluation:
relevance, consistency, accuracy and completeness of
referencing)
Tutor feedback:
Which aspect(s) of your assignment would you Student comment:
specifically like feedback on?
Tutor feedback:
How did previous feedback inform this assignment? Student comment:
Name of Tutor: Date:

15. Engaging students in processing of
feedback through the wiki
When you receive your assignment
with feedback please upload a copy to
your wiki and include a reflection on
the following four questions:
1. How well does the tutor feedback
match with your self-evaluation?
2. What did you learn from the
feedback process?
3. What actions, if any, will you take in
response to the feedback process?
4. What if anything is unclear about
the tutor feedback?

27. Student engagement
• interACT launched 30th April 2012
• 100% completion of the cover page
• Wiki engagement varies from 65-20%

28. Evaluation interviews to date
• Conducted interviews with 5 students
• They find the instructions to be clear (questions
to technical team have dropped)
• They value the opportunity to ask for specific
feedback on aspects of their work and ability to
dialogue with tutors
• The cover page self-evaluation is repetitive for
references and style sections
• Building on this with a survey to be launched in
the next week

29. Aspects I would like feedback on:
• Ideas of not preparing too rigidly in order to be flexible
within sessions – practical advice would be welcomed!
• As the first essay I have written in nearly 20 years, I would
like to know whether the standard overall was acceptable
• Please advise me how I can enter a specific page number in
a reference when using Endnote (I wanted to add “p. 58” to
the first reference used in the text since this is an exact
quote, but failed to find out how I can do it, despite using
the help option of the software).
• Feedback on whether my peers have had similar thoughts
for their own teaching, or other ideas that have been
commonly developed would be beneficial in case I have not
thought or considered them.
• any part of it

30. How did previous feedback inform this
assignment
• It made me realise that instead of focusing on a
single or a few key teaching principles, I focused
on many of them without going into much detail.
Also I had used bullet points in the text.
• Feedback that my writing style was agreeable
was reassuring. I appreciated knowing my use of
literature was valid and supportive in the previous
assignment, so have tried to continue applying
the literature to my work.
• I tried to be careful to define and reference jargon
• It was really helpful in writing present assignment

31. I really like the cover sheet and request for self
assessment and what you would like feedback
about ... consistent with principles of feedback!!
(M&L end of module feedback)
I like this better, as there is an element of
expectation that there will be some dialogue …
What I really like about it, is that it gives me a
chance to request, to maybe direct a bit the
specificity of what kind of information I am going to
get (student interview #3)

32. Added challenges of:
For sharing feedback
• Relationship and trust
• Asynchronous dialogue and
shared understanding
• Reliance on written feedback
• Technology

33. Benefits of doing the audit
• Raises awareness of quantity and quality of
feedback provided individually and across the
centre
• Opportunity for faculty development
• Objective measure of changes in feedback
culture
– The process will be repeated on 2012 assignments
at the beginning of 2013

34. Future developments
• Develop a toolkit of e-activities designed to
improve self, peer and tutor dialogic feedback
through the use of available technologies such
as blogs, Friendfeed, Skype/webinars, or
synchronous chat
• Introduce a patchwork assignment at the end
of the Certificate and
Diploma
• Work to refine process

35. Project website
http://blog.dundee.ac.uk/interact

36. Contact Details
Rola Ajjawi
Centre for Medical Education
University of Dundee
Email: r.ajjawi@dundee.ac.uk
Twitter: @r_ajjawi
http://blog.dundee.ac.uk/interact/
Karen Barton
Centre for Medical Education
University of Dundee
Email: interact@dundee.ac.uk

37. References
1. Sadler, D. R. (2010) Beyond feedback: developing student capability in
complex appraisal. Assessment & Evaluation in Higher
Education, 35:5, 535-550.
2. Boud, D., & Associates. (2010). Assessment 2010: Seven propositions for
assessment reform in higher education. Sydney: Australian Learning and
Teaching Council
3. Nicol, D. J., & Macfarlane‐Dick, D. (2006). Formative assessment and
self‐regulated learning: a model and seven principles of good feedback
practice. Studies in Higher Education, 31(2), 199-218.
4. Nicol, D. (2012). Assessment and feedback - in the hands of the student
[Online]. JISC. Available:
http://jiscdesignstudio.pbworks.com/w/page/50118521/Assessment%2
0and%20feedback%20-
%20in%20the%20hands%20of%20the%20student [Accessed 01/02/12].
5. Brown, E. & Glover, C. (2006) Evaluating written feedback. in: B. C. & K.
Klegg (Eds) Innovative assessment in higher education.
London, Routledge), 81-91.
6. Hattie, J. & Timperley, H. (2007) The Power of Feedback. Review of
Educational Research, 77, 81-112.