Keynote presentation from David Nicol at sheffield Hallam University's Assessment Exchange conference 2015

1. Principles as discourse: A blueprint for
transformational change in assessment
Emeritus Professor of Higher Education
University of Strathclyde, Scotland
Visiting Professor, University of Ulster
Adjunct Professor, University of Swinburne, Australia
Expert Consultant to JISC: Assessment and Feedback Programme
Sheffield Hallam University: 15 September 2015

2.  Barriers to enhancing the quality of teaching,
learning and assessment institution-wide
 Principles-led change
 REAP project (University of Strathclyde)
 Viewpoints project (University of Ulster)
 Discourse-centred change – learning from REAP
and Viewpoints
 Value at Sheffield Hallam University
Plan for session

3. Barriers to transformational educational
change institution-wide
1. Lack of shared educational frame of reference to
guide innovations in practice
2. Difficulty defining good educational practice
3. Accessibility of educational research
4. Disciplinary differences in teaching and learning
5. Weak links between local practices and educational
policies, strategies and procedures
6. Challenge of getting multi-stakeholder buy-in
7. Cultures and micro-cultures in institution
See Nicol & Draper (2009)

4. Shifting paradigm for assessment & feedback
Teacher-centred approach
Assessment of learning
Experts make judgements
Focus on leaning outcomes
Transmission of criteria
Individual assessment tasks
Teacher as feedback source
Quality of feedback message
Feedback as monologue
Teacher-feedback reviews
Externally provided feedback
Teacher responsibility
Delivery of feedback
Learning-centred approach
Assessment for learning
Students (learn) to judge
Focus on process and outcome
Co-construction of criteria
Collaborative tasks
Multiple sources – peers, others
Quality of feedback interaction
Feedback as dialogue
Student feedback-reviews
Internally generated feedback
Shared responsibility
Use of feedback

5. The challenge
How do we embed new thinking in policies and
in educational practices across a whole
institution in ways that are informed and
enhancing and not constraining?

6. Approaches to quality enhancement
1. Improve teachers’ skills – workshops, courses
2. Introduce new teaching method institution-wide (e.g.
problem based learning)
3. Facilitate changes in teachers’ conceptions
(reflection)
4. Institutional plans to define and support change
5. Disciplinary focus – foster scholarly discussion
amongst colleagues
6. Action research – teachers investigate things of
interest to them
Amundsen and Wilson (2012)

7. Re-engineering Assessment Practices
(REAP) project (www.reap.ac.uk)
 Scottish Funding Council (£1m): 2005-2007
 Enhance learning quality, evidence teaching
efficiencies (technology) and embed changes.
 Strathclyde, Glasgow and Glasgow Caledonian
 Large 1st year classes (160-900 students)
 Range of disciplines (19 modules ~6000 students)
 Many technologies: online tests, simulations, discussion
boards, e-portfolios, e-voting, peer/feedback software,
VLE, online-offline
 Assessment for learner self-regulation
 Set of assessment and feedback principles

8. Original 7 principles of good
assessment and feedback
Good assessment and feedback should:
1. Clarify what good performance is (goals, criteria,
standards).
2. Facilitate the development of reflection and self-
assessment in learning
3. Deliver high quality feedback to students: that enables
them to self-correct
4. Encourage peer and student-teacher dialogue around
learning
5. Encourage positive motivational beliefs & self esteem
through assessment
6. Provide opportunities to act on feedback
7. Provide information to teachers that can be used to help
shape their teaching (making learning visible)
Nicol and Macfarlane-Dick (2006)

9. Thinking behind REAP
 The vision – Assessment and Feedback should support
the development of learner self-regulation
 A set of assessment & feedback principles drawn from
research – to operationalise this vision
 Principles seen as translation device – to make the
research easily accessible
 A common frame of reference across disciplines
 Examples of implementation in different disciplines

10. Good assessment and feedback practice should
1. Clarify what good performance is (goals, criteria,
standards).
 Provide students with list of criteria before task.
 Get students to write out criteria in own words
 Groups rank samples of work according to criteria
 Students derive criteria from exemplars before a task
 Students peer review others’ work (with/without criteria)
 Groups create problems (e.g. MCQs) for others to solve
 Groups create criteria for an assignment
Students must develop own ‘concept of quality’

11. Good assessment and feedback should:
2. Facilitate the development of reflection and self-
assessment (and peer assessment) in learning.
 Provide an abstract with an essay (reflection)
 Students identify what is strong and weak when they
hand in an assignment
 Students request the type of feedback they want
 Provide written explanation of concepts underpinning a
set of problems they are working on (deep reflection)
 Or evaluate the elegance of different solution pathways
 Implement peer review: e.g. students’ comment on
each other’s work (see Nicol, 2015: Nicol et al, 2014)
then review their own work
Give students practice in making evaluative judgements

12. Good assessment and feedback practice should:
3. Delver high quality information to students: that helps
them to self-correct
 Students request feedback they wish (cover sheet)
 Feedback on processes and skills – maximise transfer
 Teacher provides ‘feed forward’ rather than feedback
 Feedback on students’ self-assessments and/or peer
reviews
 Don’t give feedback – point to resources where
answer/issue can be elaborated
 Ask students to review peer work but insert a sample
of your work to be reviewed as well
Calibrate students’ ability to make evaluative judgements
(see Hattie and Timperley, 2007)

13. Good assessment and feedback should:
4. Encourage teacher-student and peer dialogue around
learning
 Discussions of feedback in tutorials or scheduled
feedback events (e.g. Bring and discuss)
 Collaborative assignments
 Electronic voting methods: polling and peer discussion
 Students reviewing each other’s work
 Wrap dialogue around any or all assessment processes –
before, during and after (Nicol,2010)
Attenuates teachers’ voice and strengthens students’
voice (shifts responsibility towards students)

14. Good assessment and feedback should:
5. Encourage positive motivational beliefs and self-esteem
 Encourage climate of respect and accountability
 Emphasise mistakes are part of learning
 Focus students on learning rather than marks
 Sequence tasks for progressive level of difficulty
 Align formative and summative tasks
 Use reader-responsive feedback (non-evaluative)
 Use real life (authentic) learning and group tasks
 Give learners choice in topic, methods, criteria
 Implement other principles (group working etc)
About self-esteem and giving students a sense of control over their
learning: balancing structure with increasing responsibility

15. Good assessment and feedback should:
6. Provide opportunities to act on (respond to) feedback
 Align your feedback to goals/criteria
 Provide feedback as action points
 Linked assignments so feedback can be used
 Reward actual use of feedback in a new task (Gunn,
2010)
 Get students to respond to teacher feedback – say what
it means
 Get groups to discuss feedback and create action plan
 Get students to say how used feedback when submit
next assignment [proforma]
Ensures feedback is processed and leads to knowledge building. Key
principle if your goal is to enhance NSS results

16. Good feedbacks:
7. Provides information to teachers that helps them shape
their teaching
 Requested feedback
 Just-in-time teaching –online tests before lecture
delivery
 Electronic voting methods allow dynamic adaptation
 One-minute papers
 Discussion boards

17. The REAP project

18. Implementation
Local redesigns
 19 module redesigns - principles guided implementations
 Success: learning gains in exams (11 out of 19 modules)
improved quality w/o extra costs, high levels of student
satisfaction, efficiency gains.
Institutional developments
 Sharing of good practice using principles as reference
 Dep. Principal T&L set up working group: new A&F policy
(Strathclyde) grounded in principles agreed by Senate
 Principles embedded in QA procedures
 Many departmental/university initiatives referencing
REAP and using the principles
 Feedback as dialogue campaign for students (see leaflet)
 Widespread use of principles, nationally/internationally

21. In summary: learning from REAP
 Principles-based approach has great potential
 People talked about and used the principles – an
emerging discourse
 Widespread take-up of principles in UK and,
surprisingly, internationally – translation/pragmatic
 Nicol & Draper (2009) talked about the principles
acting as a kind of rhetorical resource – their format
and use seemed to persuade many people to come on
board.

22. Question
How to take things forward so that the ideas spread
more rapidly and deeply across the whole institution?

23. The Viewpoints project
 Built on the REAP assessment and feedback principles
and examples
 Funded by Jisc UK under its Curriculum Design and
Delivery programme (2008-12)
 Took the REAP principles and examples turned them
into artefacts.
 Tools to be used in face-to-face course redesign
workshops

24. Learner timeline worksheet & resources

28.  Working round table participants storyboard a sequence of
teaching and learning activities while engaging in dialogue
using the assessment and feedback cards as prompts.
 Each participant has a set of cards
 The main phases of the process......are
Course design: the workshop process

29. Activity 1: Agree objective/challenge
 The team agree the challenge/objective for their
session and write it at the top of the module
worksheet.
 EG. encourage greater use of feedback, enhance
learner engagement.

30. Activity 2 – Select principles
 Examine the front of the ‘Assessment and
Feedback’ cards and initially choose those that
might help you address your objective(s).

31. Activity 3: Map principles to timeline
 Place the cards on the timeline, where relevant,
considering the student perspective.
 At the induction phase, during the first few
weeks
 Create a conceptual map
Note: cards can be repeated on the timeline.

32. Activity 4: Consider specific
implementation ideas
 Turn over the most important card(s) and identify
any ideas that might help your group address
your challenge/objective(s).
 Adapt or formulate new examples
 Record by ticking examples and/or writing post-it
notes with examples

33. Activity 5 - Tailoring a solution
 Discuss how your selected ideas could be used
in practice to address your challenge/objective(s)
 As a group write a brief plan and make notes on
the worksheet using post-its or markers in the
‘your plan’ area.

34. Activity 6: Record Outputs & share
 Record outputs (e.g. Take photo of design
plan)
 Share outputs with others

35. Output examples

36. The principles/cards

37. Participants’ experience of workshop
 Sociability of a board game –tactile, principles put in ‘the
hands of the users’
 Sharing, problem-solving, solution focused, exploratory,
creative
 Structured by timeline – storyboarding the learning
 Discussion shaped/facilitated by principles (research-
informed) – adapted to their discipline
 Participants construct, reconstruct, co-construct
meanings of principles again and again
 Learning-focused rather than content-focused

38. Evaluation of Viewpoints
 Workshop process highly successful - ideas had spread
beyond the workshop and across the institution.
 As in REAP, principles established in policy, embedded in
quality procedures, used to support revalidation, taken
up by lifelong learning unit, used by Students’ Union in
leaflet etc. Departments ran workshops themselves.
 Seeded new conversations
 Like REAP, the emergence of a new discourse about
assessment and feedback across the institution
 Model of change extrapolated – ‘principles-based
discourse’ model

39. Key criteria in constructing principles
1. Embody compelling vision – principles a way of operationalising that
vision
2. Informed by best available research – each captures a poweful
idea/aspiration from that research
3. Succinct (6-13 words) and written in plain English – accessible,
immediate impact, face validity
4. Action-oriented (verb-phrase): call to action
5. Tight-loose in formulation: applicable to any disciplinary context,
users construct/interpret
6. Complemented by other texts that enrich meaning – disciplinary
examples, research arguments, solutions to problems
7. Relevant to wider educational agendas/concerns
8. Number of principles important
 See: http://www.reap.ac.uk/TheoryPractice/Principles.aspx

40. What educational principles are NOT
 Procedures (e.g. ensure faster turnaround time for
assignments)
 Vague statements with no obvious action (e.g. use every
opportunity to develop students as learners)
 Things that are worthwhile but are taken for granted in
assessment (e.g. ensure assessment is valid or reliable or
fair)
 Contentious issues (e.g. encourage competition in
learning)
 Ideas with no clear educational basis (e.g. maximise use
of available classroom/learning space)
 Efficiency directives (e.g. reduce time spent marking)

41. Changing the discourse: research
Grant and Marshak (2011)
A different way of thinking about and planning for change
 Discourse plays a central role in the construction of
social reality
 It shapes how people think about things and how they
act: and how people think and act shapes their
discourse
 Changing the existing and dominant discourse will
support and lead to organisational change
 Grant and Marshak (2011) identify criteria for
planning and evaluating change in discourse
Ref: Grant and Marshak (2011) Towards a discourse centred understanding of
organisational change, Journal of Applied Behavioural Science, 47(2), 204-235

42. Discourse is not just about conversations but also about
written texts, official documents, emails, memos,
stories, narratives, metaphors, slogans etc.
(Grant and Marshak, 2011)
Definition

43. New focus in organisational development
The central task of the change agent in an organization
is to identify what the prevailing discourses are, how
they serve to maintain the status quo and how
alternative discourses supportive of an intended
change might be established and maintained.
(Grant and Marshak, 2011)

44. Changing discourse: Grant & Marshak criteria
1. Establish the discourse for the intended change: principles
2. Facilitate change in discourse at multiple levels – projects focused
on interpersonal discourse but influenced organisational discourse
3. Conversations are the key discursive practice in seeding change –
focal point for REAP/Viewpoints conversations of disciplinary groups
4. Influence the discourse of those with power – DVPs put principles in
policy, also in validation and review procedures etc.
5. Harness alternative discourses to support change – assessment
problems, technology applications, graduate attributes all relevant
6. Keep the discourse on message: embed in documentation,
disseminate successes using principles
7. Maintain openness/self-reflective stance – advice to change agents
Ref: Grant and Marshak (2011) Towards a discourse centred understanding of
organisational change, Journal of Applied Behavioural Science, 47(2), 204-235

45. JISC-funded projects: 2011-2014
 Other ways of spreading the principles-based discourse
 Queens’ University Belfast: principles to baseline current
practice and envision future (Appreciative Enquiry) – planned
conversations about strategy, values and mission
 Bath-Spa and Winchester – Student Fellows facilitated
conversations with academics about principles
 Exeter and Hertfordshire – principles used in toolkits to support
decisions about technology application – discourse of technology
and discourse of principles brought together
 Exeter – student employability agenda linked to assessment
principles, seen as a way of realising it

46.  Way of addressing change in complex organisations
 Focuses on meaning-making and the social construction
of reality
 Gives a single focus for all educational change
activities
 About back-stage processes as much as front-stage
events
 Identifies discourse as a driver for change not just as a
symptom
There are already discourses about assessment and
feedback but are these research-informed?
Why important?

47. Addressing the barriers to change
 Lack of shared frame of reference to guide
innovations: the big idea and principles
 Difficulty defining good educational practice
 Isolation of academics from educational research:
research synthesised into key ideas for action
 Disciplinary differences – tight-loose structure of
principles
 Weak links between local practices and educational
policies and strategies – single set of educational
ideas, tying each course design to policy framework
 Challenge of getting multi-stakeholder buy-in
 Cultures and micro-cultures

48. The End – thank you.

49. Further resources
Nicol, D (2009) Transforming assessment and feedback: enhancing
integration and empowerment in the first year. Scottish
Enhancement themes publication http://tinyurl.com/ca7dygx
Nicol, D. (2012) Transformational change in teaching and learning:
recasting the educational discourse, Evaluation of the Viewpoints
project at the University of Ulster. Funded by JISC UK. July 22nd
[also provides a detailed description of workshop with variations]
http://www.reap.ac.uk/TheoryPractice/Principles.aspx
Viewpoints Resources: provides presentation template, workshop plan,
principles cards, timeline etc that can be downloaded.
http://wiki.ulster.ac.uk/display/VPR/Home
Grant and Marshak (2011) Towards a discourse centred understanding of
organisational change, Journal of Applied Behavioural Science, 47(2),
204-235
See also other resources on REAP website at www.reap.ac.uk especially
page at www.reap.ac.uk/TheoryPractice/Principles.aspx

50. References
 Nicol, D (2012) Transformational change in teaching and learning: recasting
the educational discourse, Evaluation of the Viewpoints project at the
University of Ulster. Funded by JISC UK. July 22nd [This document shows how
the principles developed in Nicol, 2009 were turned into a toolkit and used to
spread a new discourse across a whole HE institution.]
 Nicol and Draper (2009) A blueprint for transformational organisational change
in higher education: REAP as a case study available at:
http://www.reap.ac.uk/TheoryPractice/Principles.aspx
 Nicol, D. (2009) Transforming assessment and feedback: enhancing integration
and empowerment in the first year. Published by the Quality Assurance
Agency for Higher Education. [This document provides a list of 12 principles,
the research rationale for each, an explanation , some examples of how they
could be implemented in different disciplinary contexts]
 Nicol, D, (2009), Assessment for Learner Self-regulation: Enhancing
achievement in the first year using learning technologies, Assessment and
Evaluation in Higher Education, 34(3),335-352
 Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-
regulated learning: A model and seven principles of good feedback practice,
Studies in Higher Education, 31(2), 199-218.