The invisible student – the visible feedback:The impact of lecture attendance and online  administration on Student Evalua...
Attendance crisis in Universities?            How many of you                experience           attendance issues in    ...
Student                           Online SET   feedback                          (student volunteers,                     ...
Factors impacting on SET• SET ratings often influenced by factors outside  of the teaching effectiveness scale (Hackett,  ...
How to read SET?• Students don’t tend to blame themselves for  their problems with a module or the fact that  they are not...
The Dilemma• How can we rely on validity of SET  systems if they are completed by  absent students?
SET and the captive audience  Online Vs In class evaluations
Paper – in class                 Online• Cumbersome                     • Flexibility in administration• Longer turnaround...
Method of administration – impact           on results?   Potential         Lower response negative bias;     rates but mi...
Need for further study because…• These studies refer to small and cross  sectional samples• This is a longitudinal study b...
Research methodology(Controlling for other contextual variables) itis proposed that…  class attendance and online SET  adm...
Survey Instrument• Survey with 10 Lecturer items, 8 Module items and 7  Student items, 5 point Likert scale• Item “I atten...
Sample• 88,686 student responses collected over a period of  11 years• All evaluations requested by their lecturer• Repres...
Findings and results           Variables            R2      Sig.     Beta Coefficients           controlledLecturer   Facu...
Findings and results• Lecturer Scale: attendance accounts for an  increase of 0.182 in average scores, paper SET  administ...
Class attendance does have some      (positive) impact over SET scores• On its own, class           • The combination of  ...
But online administration is not such             a devil…            • (In line with previous findings)              onli...
Limitations• Results must be generalised with care• Sampling method is an issue both in paper  based and online SET admini...
Implications• “Invisible” student provides (slightly) less  positive feedback  – why is this the case when they don’t atte...
Thanks!• Any questions???  – Dr. Mary Fitzpatrick, mary.fitzpatrick@ul.ie  – Dr. Angelica Risquez, angelica.risquez@ul.ie
Upcoming SlideShare
Loading in …5
×

The impact of lecture attendance and online administration on Student Evaluation of Teaching (SET) Feedback

3,413 views

Published on

Fitzpatrick, M.; Risquez, A. (2012): The invisible student – the visible feedback: The impact of lecture attendance and online administration on Student Evaluation of Teaching (SET) Feedback. ISFAM 2012, 26th - 29th June 2012. University of Limerick, Ireland. Kemmy Business School.

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,413
On SlideShare
0
From Embeds
0
Number of Embeds
2,789
Actions
Shares
0
Downloads
4
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

The impact of lecture attendance and online administration on Student Evaluation of Teaching (SET) Feedback

  1. 1. The invisible student – the visible feedback:The impact of lecture attendance and online administration on Student Evaluation of Teaching (SET) Feedback Dr Mary Fitzpatrick & Dr Angelica Risquez Centre for Teaching and Learning, University of Limerick
  2. 2. Attendance crisis in Universities? How many of you experience attendance issues in your classes?Student make informed choices in terms of not attending class – this is the new reality
  3. 3. Student Online SET feedback (student volunteers, lower response rates and other factors)- Studentcommitment andinterest in subject Level of Feedback attendance? scores- Student level ofengagement-Performance
  4. 4. Factors impacting on SET• SET ratings often influenced by factors outside of the teaching effectiveness scale (Hackett, 2006)• These impact on the level of student engagement and attendance to class• Should these invisible students (who have not attended) provide feedback on the entirety of the module?
  5. 5. How to read SET?• Students don’t tend to blame themselves for their problems with a module or the fact that they are not attending – they attribute this to their teachers and the actual module (Davidovitch & Doen, 2006)• Academics are left reeling with evaluations despite the fact that many of the students have not attended
  6. 6. The Dilemma• How can we rely on validity of SET systems if they are completed by absent students?
  7. 7. SET and the captive audience Online Vs In class evaluations
  8. 8. Paper – in class Online• Cumbersome • Flexibility in administration• Longer turnaround times • Rapid archiving of data – constrains the ability of • Minimise demand on teaching staff to take on students and class-time board the feedback • Eliminates paper and the during the semester or time-consuming optical to use the data scanning process and paper effectively handling • Improves turnaround time (Kiesler & Sproull, 1986)
  9. 9. Method of administration – impact on results? Potential Lower response negative bias; rates but minimal Convenient but impact on results unreliable
  10. 10. Need for further study because…• These studies refer to small and cross sectional samples• This is a longitudinal study based on a very large sample
  11. 11. Research methodology(Controlling for other contextual variables) itis proposed that… class attendance and online SET administration do not impact on average SET results
  12. 12. Survey Instrument• Survey with 10 Lecturer items, 8 Module items and 7 Student items, 5 point Likert scale• Item “I attended most or all of the required contact hours for this module” to infer attendance• We use average scores in the Lecturer and Module scales• Multivariate regression analysis SET average score = attendance * paper Vs online * other controlling factors
  13. 13. Sample• 88,686 student responses collected over a period of 11 years• All evaluations requested by their lecturer• Representation – 83% in the traditional age group (18-23) – 87% undergraduates – 42% Business, 28% Science and Engineering, 17% Humanities and 12% Education & Health Science – 36% first year students, 22% second year, 19% third year and 22% final year
  14. 14. Findings and results Variables R2 Sig. Beta Coefficients controlledLecturer Faculty of S&E, 0.102 p=<.001 Attendance=0.182scale class size, years Paper admin=0.086average teachingModule Faculty of S&E, 0.098 p=<.001 Attendance=0.24average class size, years Paper admin=0.066scale teaching, year of evaluation
  15. 15. Findings and results• Lecturer Scale: attendance accounts for an increase of 0.182 in average scores, paper SET administration accounts for an increase of 0.086 in average scores• Module Scale: attendance accounts for an increase of 0.24 in average scores, but paper SET administration accounts for an increase of 0.066 in average scores
  16. 16. Class attendance does have some (positive) impact over SET scores• On its own, class • The combination of attendance accounts class size, faculty, year for 4% of the variance of evaluation, years of of scores in the lecturer teaching experience, scale averages and attendance and method almost 6% of the of administration can module scale averages explain up to 10% variance of SET scores in lecturer and module items
  17. 17. But online administration is not such a devil… • (In line with previous findings) online administration has not been shown to have such a decisive impact on SET scores
  18. 18. Limitations• Results must be generalised with care• Sampling method is an issue both in paper based and online SET administration – attendance is still an issue!
  19. 19. Implications• “Invisible” student provides (slightly) less positive feedback – why is this the case when they don’t attend the classes?• SET offers many possibilities to access the thoughts of students who seem to be falling though the cracks – certainly worth considering…..
  20. 20. Thanks!• Any questions??? – Dr. Mary Fitzpatrick, mary.fitzpatrick@ul.ie – Dr. Angelica Risquez, angelica.risquez@ul.ie

×