ASHE 2010 - Course Evaluation Nonresponse

599 views

Published on

A powerpoint summarizing a scholarly paper presented at the Association for the Study of Higher Education (ASHE) on nonresponse to online course evaluations. Uploaded 16 Nov 2010. Indianapolis, IN.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
599
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Realistic Majors =
  • To what extent did course salience increase the likelihood of response? To what extent did between-level interactions influence these results? To what extent did survey fatigue influence response?
  • INSTRUMENT: Lab questions then the lab is surveyed too. ADMIN: Cut down on opportunity costs (off-campus students, athletes) ENVIRONMENT: Smart et al., 2000 (academic disciplines book)
  • ASHE 2010 - Course Evaluation Nonresponse

    1. 1. WHO DOESN’T RESPOND AND WHY? AN ANALYSIS OF NONRESPONSE TO ONLINE STUDENT EVALUATIONS OF TEACHING Meredith J.D. Adams & Paul D. Umbach
    2. 2. CONTEXT • Participation decline while survey requests are increasing. (Nonresponse increases the risk for error.) • Ongoing change from paper to web-based course evaluations. • Results from course evaluations are used for research and decision-making processes about personnel.
    3. 3. FRAMEWORK • Survey Theories – Salience (Dillman et al., 2002; Groves et al., 2004, 2009) – Survey Fatigue (Groves et al., 2004; Porter & Whitcomb, 2005) • Academic Disciplines framework (Holland, 1966; Rosen et al., 1997; Smart et al., 2000) • Previous survey and course evaluation research at postsecondary institutions
    4. 4. PURPOSE OF STUDY AND RQs • Purpose of study is to investigate factors of participation in course evaluations • Research questions – To what extent did course salience increase the likelihood of response? To what extent did between- level interactions influence these results? – To what extent did survey fatigue influence response?
    5. 5. METHODS OF ANALYSIS • Study measured likelihood of response • Hierarchical modeling techniques • 30 variables – Students (demographics, housing, number of surveys students completed) – Course (if department of course and student’s major matched; grade earned)
    6. 6. RESULTS Statistically significant factors associated with nonrepsonse: – Gender (Male) – Ethnicity (African-American, Asian) – Housing (off- campus/commuters) – Athletes – Grades (Ds, Fs, ungraded students/courses) – Age (traditional age) – Nontransfers – Class rank (sophomores & juniors) – Students with more than 10 SETs to complete
    7. 7. 48.9% response rate (overall)
    8. 8. RESULTS (continued) • Using Holland’s six major academic types – Realistic majors were more likely to respond than social, artistic, conventional, enterprising, and investigative disciplines. – Social majors were less likely to respond than all other students if the course was in the same department as the student’s major. • Most variables were no longer statistically significant when the course was in the same department as their major.
    9. 9. DISCUSSION • Mostly aligned with previous research and theories of survey participation* – Exceptions = Environment of major/course, class rank • Introduced new potential influences on participation – Campus housing, athletes, transfer status, etc. * Avery et al., 2006; Cohen, 1981; Clarksberg, et al., 2008; Dey, 1997; Dillman et al., 2002, 2009; Fidelman, 2007; Groves et al., 2004, 2009; Johnson et al., 2002; Jones, 2009; Kaplowitz et al., 2004; Lepkowski & Couper, 2002; Marsh, 2007; Moore & Tarnai, 2002; Porter et al., 2004; Porter & Umbach, 2006a; Porter & Whitcomb, 2005; Sax et al., 2003, 2008
    10. 10. DISCUSSION • Nonresponse Bias • Salience – Course Location – Grades – Environment of Student’s Major • Survey Fatigue
    11. 11. LIMITATIONS • Only one institution • Only undergraduates • Limited information about the course, the faculty, class size, etc.
    12. 12. IMPLICATIONS • Evaluate the major’s environment/culture – How to increase salience? • Encourage students unlikely to respond • Examine how we utilize course evaluations
    13. 13. FOR MORE INFORMATION meredith_adams@ncsu.edu This paper can be found at: 1. https://cedupload.ncsu.edu/dropbox/uploads/Adams%20Um bach%20Course%20Evaluation%20Nonresponse%20ASHE %202010.pdf 2. http://bit.ly/9zjMAY 3. http://goo.gl/ByiZo 4. This PowerPoint Presentation is available at http://www.slideshare.net/meredithNCSU/ashe-2010-course- eval-nonresponse

    ×