1. Dr Rola Ajjawi
interACT: Interactive Assessment and
Collaboration via Technology
Centre for Medical Education, Tay Park House, 484 Perth Road, Dundee DD2 1LR, Scotland, UK

2. The Project Team
Rola Ajjawi Karen Barton Grant Murray Susie Schofield
Project Director Project Officer Learning Technologist Project Manager
Natalie Lafferty Sean McAleer David Walker
e-learning Advisor Assessment Advisor Learning Technology
Advisor

3. http://www.psy.gla.ac.uk/~simon/DIALOGUE.html

5. A critique of monologic
feedback1-6
• Lack of learner engagement with feedback
• Lack of understanding of feedback (and acting on)
• Transmitted feedback creates dependency on
teacher
• Not utilising self-evaluation or peer-feedback
• High teacher effort — low efficiency
• Lack of a shared context for assessment between
teacher and learner
• Reduced staff satisfaction as evidence of
feedforward not seen

6. Theoretical shift in conceptions
of feedback that:
 is a dialogue not a product to be delivered (relational
rather than transmissive)7
 develops the students’ capacity to make evaluative
judgements about their own and others’ work1,2
 serves the function of progressively enabling students to
better monitor, evaluate and regulate their own learning,
independently of the teacher3

7. Aims & Research questions
Aim
 We sought to re-engineer our assessment and feedback
processes in line with good practice principles from the
literature and the use of technology
Research questions
 How can technology be used to promote feedback dialogue in
online distance learning?
 What are tutors’ and students’ current conceptions of
feedback and barriers to change?
 What is the impact of dialogic feedback processes on students’
self-assessment ability, on tutors’ feedback profiles and key
stakeholder satisfaction?

8. Research approach
 Action research8
 Data collection methods
 Student and staff interviews
 Questionnaire
 Content analysis of cover pages and wikis
 Quantitative data about engagement rates and time to
progress through modules
 Routinely collected data from module evaluations, external
examiner reports
 Feedback audit

10. PG Certificate in Medical Education
Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2
chapters chapters chapters chapters chapters chapters chapters chapters chapters chapters
Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2 Equivalent 2
chapters chapters chapters chapters chapters chapters chapters chapters chapters chapters
Revised PG Certificate in Medical
Education
Formative assessment
Chapters 2 4 6 8 10
Medium stakes assessment
High stakes assessment
Chapters 2 4 6 8 10
Chapters 2 4 6 8 10
©Mark Russell 2010
Chapters 2 4 6 8 10 University of Hertfordshire
ESCAPE project

12. Student engagement
 interACT launched 30th April 2012
 100% completion of the cover page
 Wiki engagement varies from 65-20%
 85.2%(n=46) of the students who responded to the
survey thought that the instructions provided about the
assignment submission process were clear

13. 40
35
Disagree/Strongly Disagree
30
Not sure
25
Number of Students
Agree/Strongly Agree
20
15
10
5
0
Valuable for my Valuable for my Promoted Promoted Self Beneficial to
own learning own teaching dialogue about Evaluation future
feedback assignment
tasks

14. Majority agree the cover page is
valuable (CP)
 Ability to request feedback about specific issues
 Allows last review of work before submission and focuses
the self assessment
 Gives you a chance to tell the tutor what your concerns
about the assignment are 'up front‘
 It is a chance to think critically about how you
approached the assignment and the assignment
assessment criteria
 There is an element of expectation that there will be
some dialogue

15. Aspects I would like feedback on:
 Ideas of not preparing too rigidly in order to be flexible
within sessions – practical advice would be welcomed!
 As the first essay I have written in nearly 20 years, I would
like to know whether the standard overall was acceptable
 Please advise me how I can enter a specific page number
in a reference when using Endnote
 Feedback on whether my peers have had similar thoughts
for their own teaching, or other ideas that have been
commonly developed would be beneficial in case I have
not thought or considered them
 any part of it

16. How did previous feedback
inform this assignment
 It made me realise that instead of focusing on a single or
a few key teaching principles, I focused on many of them
without going into much detail
 Feedback that my writing style was agreeable was
reassuring. I appreciated knowing my use of literature
was valid so have tried to continue applying the literature
to my work.
 I tried to be careful to define and reference jargon
 It was really helpful in writing present assignment

17. Which aspect(s) of your assignment would you specifically like
feedback on?
 Student comment (CP): I originally had a paragraph around
learning principles and theories relating to small group
teaching. I unfortunately had to delete it due to the word
restriction. Is that something I should have included?
 Tutor feedback (CP): Would you if you were the new tutor have
appreciated this?
 Student comment (wiki): I do think the tutor would have
appreciated a small paragraph around educational principles
and theories to help put small group teaching into context. I
think that 5 pages though is plenty for the tutor to
read, anymore than this and they may not have time to read it
all. I have also learned how to feedback to the tutor based on
the feedback given.

18. Which aspect(s) of your assignment would you specifically like
feedback on?
 Student comment: My choice of the Angoff method for this
exam. I heavily considered the borderline group method, but
decided against it as there was relatively little evidence for its
use in exams with smaller examinee numbers such as this one.
Thanks.
 Tutor feedback: Yes, when borderline group method is applied
to an exam with a small group of candidates there is a risk of
not finding any borderline candidates! However, if you use
borderline regression method, in which marks of all candidates
are considered for setting the pass mark, this risk can be
overcome.

19. Conclusions & implications
 Creating assessment and feedback dialogue in online
distance learning is possible
 Majority of students find it valuable for their learning and
that it promotes self-evaluation and dialogue
 Feedback audit has been a useful tool for faculty
development initiatives and measuring change in quantity
and quality of feedback
 Challenges have included streamlining the
process, dealing with tokenism, and improving the quality
and timeliness of the feedback

20. Research &Development
 Continue to refine and streamline the process
 More work is needed to help students better understand
the pedagogic rationale and how to optimise their use of
feedback
 Embed into programmatic assessment approach through
introduction of a personal development plan and
patchwork/capstone assessment
 Future research needs to explore how the online
environment mediates learning from feedback

21. Project website
http://blog.dundee.ac.uk/interact

22. So what did we do…
http://youtu.be/S5bBFEbXDD0

23. Contact Details
Rola Ajjawi
Centre for Medical Education
University of Dundee
Email: r.ajjawi@dundee.ac.uk
Twitter: @r_ajjawi
http://blog.dundee.ac.uk/interact/
http://youtu.be/S5bBFEbXDD0
Karen Barton
Centre for Medical Education
University of Dundee
Email: interact@dundee.ac.uk

24. References
1. Sadler, D. R. (2010) Beyond feedback: developing student capability in complex
appraisal. Assessment & Evaluation in Higher Education, 35:5, 535-550.
2. Boud, D., & Associates. (2010). Assessment 2010: Seven propositions for
assessment reform in higher education. Sydney: Australian Learning and
Teaching Council
3. Nicol, D. J., & Macfarlane‐Dick, D. (2006). Formative assessment and
self‐regulated learning: a model and seven principles of good feedback practice.
Studies in Higher Education, 31(2), 199-218.
4. Nicol, D. (2012). Assessment and feedback - in the hands of the student [Online].
JISC. Available:
http://jiscdesignstudio.pbworks.com/w/page/50118521/Assessment%20and%20
feedback%20-%20in%20the%20hands%20of%20the%20student [Accessed
01/02/12].
5. Brown, E. & Glover, C. (2006) Evaluating written feedback. in: B. C. & K. Klegg
(Eds) Innovative assessment in higher education. London, Routledge), 81-91.
6. Hattie, J. & Timperley, H. (2007) The Power of Feedback. Review of Educational
Research, 77, 81-112.
7. Ajjawi, R. (2012). Going beyond ‘received and understood’ as a way of
conceptualising feedback. Medical Education, 46(10), 1018-1019.
8. Carr, W. & Kemmis, S. 1986. Becoming critical: education, knowledge, and action
research, Lewes, Falmer Press.