Emergent evaluation

1. Emergent
Evaluation
Phil Wood and Palitha Edirisingha
An initial exploration of a formative framework for evaluating
distance learning modules

2. Guiding Considerations:
• Many students don’t fill in distance learning evaluations
• Evaluations often too generalised
• Evaluations are inherently retrospective
• Evaluations often focus on activities, tutoring, resources, environments, but
rarely learning and student action.
• Analysis often gets reduced to simplistic reflections on numbers

3. Are there alternatives?
• Ellery (2006) – multidimensional evaluation on a campus-based course
(evaluation throughout the module, with both student and lecturer
perspectives)
• Benson et al (2009) - Participatory evaluation, again multi-modal. Based on
Jackson and Kassam 1998
a process of self-assessment, collective knowledge production, and cooperative action
in which the stakeholders in a development intervention participate substantively in
the identification of the evaluation issues, the design of the evaluation, the collection
and analysis of the data, and the action taken as a result of the evaluation findings.
(1998, 3)

4. We wanted to develop an approach which allowed for:
• Formative module evaluation
• Linked to curriculum development
• To make distance learning review more than a ‘performative’ activity
• Putting pedagogy (interpenetration of teaching, learning, curriculum and
assessment and their interaction with teachers and students) at the centre of
the process
• Emergence and trialling of new approaches as a standard element of our work

5. Basic outline of process

6. Possible Advantages
• Allows us to develop elements of the curriculum in real time, with student
response and reflection helping shape the content and approach
• Students value the opportunity to give deeper, more critical views
• Deeper understanding of the complexities of pedagogies, with opportunity to
respond and use formatively
• Rolling programme of curriculum renewal which is holistic – measured
innovation and emergent understanding of student experiences/learning
• Synergy with research opportunities
• Opportunity to develop a model for campus-based use (perhaps reflecting
Jackson and Kassam 1998)

7. Issues on Which to Reflect
• Time
• Workload at points of development – intensive process
• Making time for ‘deep work’ (Newport, 2016) to draw out insights/reflections
and emerging models of pedagogy.
• How does the data gained fit into increasingly simplistic and performative
evaluation frameworks at university level? Does it matter?

8. References
Jackson, E.T., and Y. Kassam. 1998. Knowledge shared: Participatory evaluation in development
cooperation. West Harford, CT: Kumarian Press
Newport, C. (2016) Deep Work, Rules for focused success in a distracted world. London,
Little Brown.
Benson, R.; Samarawickrema, G. & O’Connell, M. (2009) ‘Participatory evaluation:
implications for improving electronic learning and teaching approaches.’ Assessment &
Evaluation in Higher Education, 34:6, 709-720.
Ellery, K. (2006) ‘Multi‐dimensional evaluation for module improvement: a
mathematics‐based case study.’ Assessment & Evaluation in Higher Education, 31:1, 135-
149.