This document discusses evaluating coursework from a student perspective. It outlines common purposes of evaluation like quality assurance and improving teaching. The focus of evaluation should be on both teaching/teacher and learning/outcomes. Strategies mentioned include surveys, observations, interviews, pre-/post-testing. Drawbacks include low response rates and students not always accurately assessing teaching. The document advocates using a variety of quantitative and qualitative methods like those in Kirkpatrick's four-level model to gain a holistic view of a course's effectiveness and opportunities for improvement from both a student learning and teaching practice perspective.

1. Interrogating Course Evaluation
UWC Induction 20 March 2015
Rita Kizito - kizitorita@gmail.com

2. Aim of this part of the session
• Interrogate ‘evaluation’ from own
perspective
• Come up with an evaluation
strategy

3. Common Purpose of evaluation
• Required by universities near end of
each course *
• Can be used by
administration/departments as an
important element in making
decisions about promotion
• Could be a source of great pride or
trepidation
(Cashin, 1999; Clayson, 2009)

4. Purposes of evaluation
audit
development
“quality assurance”
(Biggs, 2003; Edström, 2008; Patton, 1997)
appraising
teachers
developing/ improving
courses and teaching
effectiveness
“quality enhancement”

5. What should the focus of the
evaluation be?

6. What should the focus of the
evaluation be?
• To check that something is working
What is that something ?(e.g. course,
degree programme, activity)
What do we mean by ‘working’ ? (what goal
should be achieved?)
• To figure out how ‘it’ can be improved

7. What should the focus of the
evaluation be?
Teaching
( teacher)
Learning
(outcomes)
Teaching
(process)
Learning
(process)
“Rather than ratings, teachers
should be asked to include
their course analyses in their
teaching portfolio in order to
show their ability to both
analyze the student learning
experience and the quality of
the student learning outcomes ,
and to improve these with
adequate course development
measures”
(Edström, 2008, p.104).

8. “Evaluation is often viewed as a test of
effectiveness – of materials, teaching
methods or whatnot - but this is the least
important aspect of it. The most important
is to provide intelligence on how to
improve these things”
What should the purpose of
evaluation be ?
( Bruner, 1966, as cited in Ramsden, 2003, p. 233).

9. Typical questions
• Have student attitudes changed?
• Has the lecturer approaches changed?
• Have student learnt something in class?
• What is happening in class?
• Have classroom practices changed?
• Are students engaged?
• Am I meeting students’ needs?

10. Evaluation strategies – student
attitudes & perceptions
Have student attitudes changed?
• Likert-scale questions & surveys
• Could be used to measure the effect of a
course, a degree or a long term change
over a number of years

11. Evaluation strategies – Lecturer
approaches
Have lecturer approaches to teaching
changed?
• Teaching Practices Inventories
• Can help departments reflect on teaching,
by allowing comparisons between different
courses/departments

12. Evaluation strategies – Student
learning
Have students learned anything?
• Measure student performance
• Important to standardise testing (using
locally developed or already developed
tools)
• Important to conduct pre and post-testing,
before and after a learning intervention
Is there evidence that the Graduate Attributes
have been adopted by the students?

13. Evaluation strategies – classroom
practices
What is going on in the class?
• Video recordings
• Peer observations (fellow lecturers) using
observation protocols
• This data can be used in your Teaching
Portfolio
Have classroom practices changed?
• Use of innovative teaching & learning
practices
• Providing useful feedback etc.

14. Evaluation strategies – student
engagement
Are students engaged?
• Observation of about 10 students to see if
they are engaged by recording student
activity
• What teaching & learning activities result in
high levels of student engagement?

15. Evaluation strategies – student
learning needs
Are students learning needs being met?
• Focus groups or Interviews
• Anonymous surveys/midterm surveys
(online, or on paper) – letter to the
facilitator
• Assignment by assignment survey (time it
took, approach, resources used etc. )
• Keep, Start, Stop

16. Kirkpatrick’s (1994) four level model
Level What is measured Examples
1 Reaction - changes in
perception ,
satisfaction levels
How students feel about
the learning experience
Feedback forms
2 Learning – changes in
knowledge, skills,
attitudes
Increase in student
knowledge and skills
Informal/informal assessment
before and after learning
interventions
3 Changes in behavior/
practices
How far learning is applied
in practice resulting in
personal changes
Observations and interviews
of students over time
4 Results - noticeable
changes in results or
conditions.
How far the module/course
impacts on program or
institutional factors (
student performance,
retention, throughput).
Use institutional data to
identify whether the
program/module shifts the
nature of student
participation/
performance/engagement.
Student surveys.

17. Some Drawbacks
• Students are not always good at evaluating
teaching effectiveness. Popularity is usually
mistaken for good teaching
• Response rate is always less than 100%
• Susceptible to bias - voluntary
participation/polarization
• Questions are usually general –missing the
finer details of practice

18. The UWC Evaluation Guideline document
Instruments
GENERIC
EVALUATION
DISCIPLINE
SPECIFIC
Question Bank
A standard set of questions that
students are invited to answer
anonymously at the end of each
course. ( often, the results are given
weight in promotion and tenure
decisions)
Run centrally and completed by all
students to provide the institution with
data on how students are performing
with regards to institutional strategies
and plans

19. Your evaluation strategy
Which evaluation methods could you consider
implementing in your teaching?

20. References
• Biggs, J. (2003). Teaching for quality learning at university: What the
student does, Buckingham, UK: SRHE and Open University Press.
• Cashin, W. E. (1999). Student ratings of teaching: uses and misuses. Ch 2,
In P. Seldin and Associates (Eds). Changing practices in Evaluating
Teaching: A practical guide to improved faculty performance and
promotion/tenure decisions. Pp 25-44. Bolton, MA. Anker Publishing.
• Clayson, D. E. (2009). Student evaluations of teaching: Are they related to
what students learn? A meta-analysis and review of the literature. Journal
of Marketing Education. 31 (1), 16-30.
• Edström, K (2008), Doing course evaluation as if it matters most. Higher
Education Research and Development, 27 :2: 95-106
• Kirkpatrick, D.L. (1994). Evaluating Training Programs. The Four Levels.
San Francisco: Berrett- Koehler.
• Ramsden, P (2003). Learning to teach in higher education ( 2nd edition).
London: Routledge Falmer.
• Schimpf, N (2015) Evidence-based Active Learning:
Evaluating Classroom Practices. Presentation made at the University of the
Western Cape

21. Thank you