Ursula Rutherford is developing evaluation instruments to assess student learning in an evolving undergraduate engineering curriculum at Coventry University that is increasingly adopting activity-led learning (ALL). The research has three interconnected strands to identify teaching quality indicators, investigate how they can be recognized, and evaluate their effectiveness. However, the rapid changes in modules and student cohorts pose a challenge for validating new instruments. Rutherford is seeking ideas from others on how to demonstrate rigor in the research and reliability of instruments developed in this environment of constant change and shorter feedback cycles.

1. Coventry University: Department of the Built Environment
How to hit a moving target:
developing evaluation instruments in
an evolving curriculum
Research design by Ursula Rutherford
Director of Studies: John Davies
Supervisors: Sarah Wilson-Medhurst and Tim Davis

2. Outline
Introduction
The research task
The research design
Work in progress
The problem
The solution ?

3. Introduction
The Engineering and Computing faculty at Coventry University
is increasingly adopting activity led learning (ALL) as an
effective teaching strategy to enable students to gain
knowledge, experience and skills in their chosen engineering
or computing professions, together with additional life-long
learning skills which equip them to understand the
requirements and constraints of working in industry.
activity led learning

4. Introduction
Activity led learning
“Engaging students through challenges requiring them to
develop and apply their technical and scientific knowledge,
whilst simultaneously developing their team working,
leadership, problem solving and life-long learning skills.”
(Wilson-Medhurst & Glendinning 2009)
Problem solving Experiential
Real world challenges Applying knowledge Student seeks learning
Group work Communicating Leadership Project work

5. The research task
Overall aim: develop appropriate instruments to evaluate learning
Scope:
• undergraduate engineering modules
• activity led learning based on real world industrial examples
Steps:
• identifying appropriately aligned teaching quality indicators
• investigating how to recognise them in action
• evaluating their effectiveness as useful predictors
• proposing new applications
Additional focus:
• characteristics and experience of part-time students

6. The research design
ideas from
other
institutions
There are three
inter-related strands student
mentor
of research which will scheme
proceed through
integrated
several phases to: project
• engender information and ideas
• provide test beds for instruments under development
• enable cross-referencing of findings

7. Work in progress
Data collection began in the first year of the study
with several objectives:
• provide a baseline
• enable the researcher to gain skills
• explore the research area
• identify teaching quality indicators

8. A problem ?
ongoing change:
• improve quality
• move toward activity led learning
rapid change:
• demonstrate responsiveness
• customised changes
• immediate benefit

9. The impact
Conventional Accelerated
module module
development development

10. The problem stated
changed module presentation
different student cohort
© Ivan Chuyev | Dreamstime.com
In this environment of shorter feedback cycles and constant
change, how can new evaluation instruments be validated?
How can I demonstrate rigour in my research and reliability of
the instruments I develop?

11. The solution
I am searching the literature for ideas.
This presentation is an opportunity for me to collect ideas
from you!
Please let me know your thoughts.
References
Wilson-Medhurst, S. and Glendinning, I. (2009), “Winning Hearts and Minds: Implementing Activity Led
Learning (ALL)”, in proceedings of the 2009 conference Learning by Developing – New Ways to
Learn, Laurea Publications, Helsinki