This document discusses Iowa State University's use of online course evaluations through their Class Climate system. It provides an overview of the benefits of online evaluations compared to paper evaluations, the process of administering the evaluations, who has access to the reports, and how the Animal Science department utilizes the evaluation results. The presentation aims to help faculty understand how to interpret and apply the student feedback to improve their teaching.

1. Center for Excellence in Learning and Technology
Putting the Results from the
Student Evaluation of Teaching
into Practice
Jodi Sterle, Ph.D. Laura Bestler, Ph.D.
Harman Endowed Professor Program Coordinator
In Teaching & Learning and CELT
Teaching Section Leader bestler@iastate.edu
Animal Science 294-5357
jsterle@iastate.edu http://www.celt.iastate.edu/
http://www.ans.iastate.edu/

2. Agenda
• About Online Course Evaluations
• The Process
• Types of Reports
• Who has access? Who views the reports?
• Big Picture
• How Animal Science Uses Course Evaluations
• Moving Forward
• Student Participation
• Other Uses for Class Climate

3. About Online Course
Evaluations

4. In 2002, a national study found that 98%
of the nation's "most-wired"
colleges, still used a paper-based for
the student evaluation of teaching
(Hoffman, 2003).

5. Paper Course Evaluations

6. Paper Course Evaluations
Pros Cons
Done in-class - good response rates Very time consuming (preparation before and
after administration)
Students feel anonymous May take 4-12 weeks for faculty to get results
Resource intensive to type up answers to
open ended questions
Lots of room for error (scanning
errors, student errors-using pen, etc., illegible
comments, handling errors: students can
tamper with data or forget to return, people
often put forms in a packet for the wrong
class, etc)
Students may be apathetic and just fill in
anything
Costly (cost of
forms, pencils, bins, envelopes, work
hours, scanner)
Bad for the environment (uses lots of paper)
Students may not take time on open ended
questions because of time

7. Online Course Evaluations

8. Online Course Evaluations
Pros Cons
Immediate results Lower response rate?
Far less room for error (no lost forms, Has to be done on the students‟time
scanning issues) (unless technology allows for in-class)
Far less time consuming Anonymity concerns
Far less costly (no scanner, paper Students who have not attended
forms, much less work hours,etc.) class are now able to give opinions
Green- no need for paper
More student comments
Flexibility for questions/scales
Students who take the time to do
them have an opinion
Students are able to fill them out 24
hours a day
Less time intensive for administrators

9. Iowa State University
ISU started using Scantron's Class Climate online course
evaluation system on a pilot basis in Spring 2011.
S2011 F2011 S2012 F2012
DEPTS 56 92 103 115
COURSES/SECTI
2071 4680 4576 5574
ONS
INSTRUCTORS 717 1540 1667 1901
EVAL INVITES 55,359 142,568 159,420 165,950

10. Response Rates
F2012 S2012 F2011 S2011
OVERALL AVERAGE 59.74 62.30 63.50 74.40
WEIGHTED AVERAGE 60.17 59.65 63.18 71.57
Weighted Average: is taking the difference of the
number of students into account
Formula:
=sumproduct(enrolled students in the course,response
rate
/sum(enrolled students in the course)

11. The Process

12. The Process: Departments
Each department/college has local control of:
• evaluation questions
• email texts
• when email invitations are sent out
• number/frequency of email reminders
• when evaluations starts and ends
• when instructors receive course reports

13. The Process: Email Invitation
SUBJECT: ISU Online Course Evaluation for [COURSENAME] [COURSETYPE]
Iowa State Student [PARTICIPANT_EMAIL],
This email asks you to respond to an anonymous online course evaluation
survey for
[COURSENAME] [COURSETYPE].
Please follow the link to open the course evaluation survey, fill-out the
questionnaire, and click “submit” when completed.
[DIRECT_ONLINE_LINK]
Please complete this evaluation by 12:00 p.m. (Noon) on Friday, Month XX,
2012. This survey will not be reopened.
You will receive this reminder email periodically until the survey is complete.
Please complete all online surveys you receive. Links to your course surveys
will also appear in your Blackboard Learn page under “My Course Evaluations.”
Thank you for your participation,
- [SUBUNIT] Course Evaluation Administrator

14. The Process: Email Reminder
EMAIL HEADER: Course Evaluation Reminder for [COURSENAME]
[COURSETYPE]
Iowa State Student [PARTICIPANT_EMAIL],
This email is to remind you that you have not yet completed the survey for:
[COURSENAME] [COURSETYPE].
Please follow the link to open the course evaluation survey, fill-out the
questionnaire, and click “submit” when completed.
[DIRECT_ONLINE_LINK]
Please complete this evaluation by 12:00 p.m. (noon) on Friday, Month XX,
2012. This survey will not be reopened.
You will receive this reminder email periodically until the survey is complete.
Please complete all online surveys you receive. Links to your course surveys will
also appear in your Blackboard Learn page under “My Course Evaluations.”
Thank you for your participation,
- [SUBUNIT] Course Evaluation Administrator

15. The Process: Students fill out the evaluation

16. The Process: Departments/College run and
send Instructor/Course Reports

17. Types of Reports

18. Example Course Report

19. Example Instructor Profile
Instructor Profile:
a combined report of all
courses taught during a
term.

20. Report Creator: Any Compilation (Tree structure)
Example of Department Overall report

21. Example Course Report:
Open Ended Questions

22. Example of Profile Line

23. Report for Comparison on Questionnaire
The “Hook” Profile line
into standard Report

24. Who has access?
Who views the reports?

25. Who has access to Class Climate?
• Departments/Colleges:
• Designated administrator(s) (Merit, P&S, or
Faculty member)
• Only their respective department(s)
• System-wide:
• CELT administrator for Class Climate
(2 FT P&S staff members)
• IT administrator for Class Climate
(1 FT P&S staff member)

26. Who sees instructor reports?
• Departmental designated administrator(s)
(Merit, P&S, or Faculty member)
http://bit.ly/ccadmins
• Instructor (may share with whomever they wish)
• Department Chair and/or College Administrator
• Teaching Coordinator or other departmental
designee

27. Big Picture

28. Big Picture: What is your
Position Responsibility Statement (PRS)?
• Research, Teaching, Extension and
Outreach
• Each area needs to be documented and
then evaluated
VanDerZanden, A.M.. (2012, January 30). You have course evaluations: Now what? Department of Animal Science [Presentation].
Ames, IA: Iowa State University.

29. Big Picture: Teaching
Portfolio Teaching
of teaching philosophy
Peer Student
review of evaluation
teaching Evaluation of teaching
of
Teaching
VanDerZanden, A.M.. (2012, January 30). You have course evaluations: Now what? Department of Animal Science [Presentation].
Ames, IA: Iowa State University.

30. Promotion & Tenure
http://www.provost.iastate.edu/help/promotion-and-tenure

31. How Animal Science Uses
Online Course Evaluations

32. Animal Science
• 969 undergraduates
• ~200 grad students
• >80 courses/sections per semester
• Every course and section evaluated using
ClassClimate since Spring „11
• Senior exit surveys
• Advising surveys

33. Using the Results
• Evaluation results and departmental averages
sent to instructors
• Teaching coordinator and Department Head
review
• Teaching coordinator assists in interpretation
of results and development of improvement
plan
• Department Head addresses results in annual
review

34. Using the Results
• Teaching coordinator and Department Head
also pass along outstanding teaching faculty
names to awards committee
• Use results (with benchmarks, over time) to
show improvement
• P&T packets
• Used with Peer Evaluation of Teaching
evaluations

35. What I’ve Seen
• Lots of initial anxiety over move to online (and
mandatory) course evals
• Love getting results early
• Open and transparent
• Departmental trends
• Focus on improvement

36. Moving Forward

37. Moving Forward: Reflecting on Fall 2012
• Snap shot in time
• Don‟t take everything at face value
• Concentrate on the 80%; not the 20%
• Discuss with a trusted mentor
• What can you do with the constructive
feedback?
VanDerZanden, A.M.. (2012, January 30). You have course evaluations: Now what? Department of Animal Science [Presentation].
Ames, IA: Iowa State University.

38. Moving Forward:
Strategies for Analyzing Feedback
• Control your defense mechanisms.
• Analyze the source of your students’
reactions in a way that sheds light on any
issues and problems that have been identified.
• Work hard not to under-react or over-react to
information that you receive via evaluation
feedback.
• Divide the issues raised by students into
actionable and non-actionable categories.
Moore, S., & Kuol, N. (2005). A punitive tool or a valuable resource? Using student evaluations to enhance your teaching. In G.
O‟Neill, S. Moore, & B. McMulline (Eds)., Emerging issues in the practice of university learning and teaching (pp. 141-148).
Dublin: All Ireland Society for Higher Education.

39. Moving Forward:
Strategies for Analyzing Feedback
• Communicate with students before and after
their provision of feedback.
• Do not make the simplistic assumption that
all positive responses are related to good
teaching and all negative responses are
related to bad teaching.
• Remember that small changes can have big
effects.
• Develop a teaching enhancement strategy
that takes into account the evaluation feedback
Moore, S., & Kuol, N. (2005). A punitive tool or a valuable resource? Using student evaluations to enhance your teaching. In G.
O‟Neill, S. Moore, & B. McMulline (Eds)., Emerging issues in the practice of university learning and teaching (pp. 141-148).
Dublin: All Ireland Society for Higher Education.

40. Comments Analysis
Worksheet

41. Comments Analysis Worksheet
• Often multiple comments are related to the
same category; for example:
10 students may all make comments about
the assignments being unclear. This is not
really 10 different comments but rather one
comment 10 times.
• The multiple mentions give it weight, but it is
only one area that needs to be addressed for
improvement.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Canada License.
Winer, L., Di Genova, L., Vungoc, P.-A., & Talsma, S. (2012). Comments analysis worksheet. Excerpted from Interpreting end-of-
course evaluation results. Montreal: Teaching and Learning Services, McGill University available at:
http://www.mcgill.ca/tls/teaching/course-evaluations/interpretation.

42. Comments Analysis Worksheet
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Canada License.
Winer, L., Di Genova, L., Vungoc, P.-A., & Talsma, S. (2012). Comments analysis worksheet. Excerpted from Interpreting end-of-course evaluation results.
Montreal: Teaching and Learning Services, McGill University available at: http://www.mcgill.ca/tls/teaching/course-evaluations/interpretation.

43. Comments Analysis Worksheet
The worksheet has been organized alphabetically
in sections according to most frequently
commented categories.
• Note any student comments that will help in
interpretation.
• Indicate positive and negative comments.
• Record the frequency of comments
surrounding each theme to help identify the
areas where students felt most strongly.
• Add any personal notes that will help in the
process of building on the feedback received.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Canada License.
Winer, L., Di Genova, L., Vungoc, P.-A., & Talsma, S. (2012). Comments analysis worksheet. Excerpted from Interpreting end-of-
course evaluation results. Montreal: Teaching and Learning Services, McGill University available at:
http://www.mcgill.ca/tls/teaching/course-evaluations/interpretation.

44. How Can You Help with
Student Participation?

45. Increasing Student Participation
• Give students examples on how the results enhance
your course and teaching.
• Include the course evaluation start/end date and/or
information in your course syllabus.
• Add the “My Course Evaluation” module into your
BbLearn course.
• Request that your departmental administrator send
you the response rate notification prior to the end
of the evaluation period. You will then receive an
automatic email that your course has a low response
rate (set at the local level).
Contact your departmental administrator:
http://bit.ly/ccadmins

46. Should there be incentives for
completing online course evaluations?
6% Don’t Care
24% No
70% Yes
2012 Comets survey
5000 random ISU students
763 students responded
15% response rate

47. What would be considered good incentives?

48. Additional Uses for
Class Climate

49. What else can Class Climate be used for?
• Mid-semester course/instructor
evaluations
• senior exit surveys
• advising surveys
• program evaluations
• research projects (STEM, etc.)
Contact your departmental administrator:
http://bit.ly/ccadmins