More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (8)
Similar to Class eval and incentives talk
Similar to Class eval and incentives talk (20)
Recently uploaded
Recently uploaded (20)
Class eval and incentives talk
- 1. Using Incentives to Increase Student Response Rates
- 3. “Do it for the Good of Future Students”
- 4. And, College Students’ Frontal Lobes
- 5. So, Can We Provide Incentives, and will it change behavior?
- 6. What Happened? Question 1 (Cross Sectional Analysis): Did instructors who used incentives have higher student response rates than instructors who did not? • Mean rate without incentives: 47.3% of students completed part or all of on-line ClassEval • Mean rate with incentives: 79.6% of students completed part or all of on- line ClassEval • This difference was significant by both - t-test: t (df = 44) = 7.54, p < .0001 (percentages require arcsin transform) - Non-parametric test (Krukall-Wallis) p< .0001 • So, instructors that used class-based incentives had dramatically higher student participation in ClassEval -- But maybe those instructors would have had higher participation rates anyway
- 7. What Happened? Question 2 (LongitudinalAnalysis): Did instructors who used incentives in Spring ‘10 have higher evaluations than for same course in Spring ‘09 (when no one used incentives)? (Note: Not all instructors taught the same course in both ‘09 and ‘10) • So, even the same instructors had higher student participation when they used class-based incentives % response in Spring ‘09 % response in Spring ‘10 No incentives in Spring ‘10 49.4% 46.6% Used incentives in Spring ‘10 59.0% 77.6%
- 8. What Happened? Question 3 (one more interesting question): Do evaluations scores correlate with the response rates? • Here is the scary part – without incentives the fewer the students who participate, the lower the mean ratings of the class • But, if you use incentives, the number of students participating has no relationship with mean ratings of class • Why? Did not use Incentives Used Incentives Q. 9 (Instructor) .39 -.01 Q. 14 (Course) .49 .00
- 10. Conclusion
Editor's Notes
- Students evaluate their classes from very early on. In Alaska, this student evaluation would not be counted.
- As a psychologist, I believe in using incentives to get people to behave in desired ways
- Telling students to do evaluations for the common good may not be enough incentive. This might be called the “Broccoli” approach. Broccoli may be better for future Doug, but present Doug may find the bacon cheeseburger to be more rewarding to the sensory system.
- The frontal and prefrontal cortex plays an important role in decision making. They aren’t well developed in college students. So thinking about the future is not as likely.
- In my department last spring, we did a naturalistic experiment – I suggested that instructors might try a class-based incentive, such as if 80% of the class does the on-line student evaluation, then the final exam will include an extra credit question. About half of the instructors used class-based incentives and half did not (they used exhortation to do the right thing)
- Any class has a number of dissatisfied students – they have an incentive to respond to the ClassEval survey. So, their numbers stay about the same. With low response rates, the evaluations will be sensitive to the number of the reasonably satisfied students that participate. But once you have a larger number of respondents, the weight of the dissatisfied students doesn’t vary much as the overall number of respondents changes.
- Thinking that students will participate in ClassEval without incentives is the stupidest idea in the history of stupid ideas. It is so stupid that I suspect the real intent was to undermine the system, to make sure that it failed. The only problem is that ClassEval data are used for promotion and tenure, for yearly evaluations, by faculty to revise classes. So now we are stuck making all of those decisions using unrepresentative or bad data.