1. Attendance and Learning Gains for LESLLA Students Larry Condelli American Institutes for Research, USA LESLLA Seventh Annual Symposium Minneapolis, MN September 29, 2011 1
2. Overview of Presentation Deconstructing the meaning of attendance Attendance measures used Study designs Findings Discussion 2
3. Why Study Attendance? Why is it important? More attendance = more instruction = more learning? Mixed research evidence No or small relationship Why? Ineffective instruction? Poor research design? Other factors affect attendance and learning? 3
4. Deconstructing Attendance What does attendance reflect? Motivation Time Attention and engagement Instruction (dosage model) Do all affect learning? Which are more important? 4
5. Research Questions How many hours do LESLLA students attend class? Does amount of attendance affect student outcomes (test scores)? What other measures of attendance affect outcomes? 5
6. Measuring Attendance Four measures to reflect components: Total hours of instruction attended Intensity of instruction- average hours per week attended Total weeks of instruction attended Rate of Instruction-proportion of hour attended out of all possible hours Note: Attendance measures (except rate) are constrained by class schedule That is, you only can attend as much as the class is scheduled 6
7. Attendance Measures: Examples A student attends a class that meets 9 hours/week for 18 weeks. She misses one full week of class and 3 additional days (misses 18 hours total): Total Hours: 144 Intensity: 8.5 (144/17) Total weeks: 17 Rate: 0.89 (144/162) 7
8. Meaning of Attendance Measures Total Hours: Total time exposed to instruction, unadjusted for class schedule or time (mostly instruction) Intensity: concentrated attendance –dosage of instruction Total weeks: Mostly time, does not directly measure amount of instruction, but may include motivation Rate: Not affected by class schedule, most directly reflects motivation 8
9. Research Studies: Secondary Data Analysis Study 1: What Works Study (Condelli, Wrigley & Cronen, 2003) 495 LESLLA students from 38 classes , 13 sites, seven USA states Study 2: Impact of a Reading Intervention on Low-Literate Students (Condelli, Cronen & Bos, 2010) 1,344 low-literate (not all LESLLA) students from 33 classes, 10 sites, four USA states 9
10. Study 1: Design Correlational study, followed two cohorts of LESLLA students for one year Measures of basic literacy, reading comprehension and oral language Assessed at start of instruction, 3 and 9 months later Used latent growth model analysis Student age, L1, years of schooling, teacher variables and English literacy (pretest scores) held constant Addressed all three research questions, using all four attendance measures 10
12. Study 1: Analytic Findings Attendance Measures Total Hours: No effect on test measures Intensity: No effect on test measures Total weeks: No effect on test measures Rate: Significant relationship for reading comprehension and oral language Reading comprehension (Woodcock Johnson) and oral Language (BEST Oral) Small growth in skills over nine months with higher rate of attendance Independent of amount of instruction – motivation measure 12
14. Study 1: Oral BEST and Rate of Attendance (p=.02) 14
15. Study 2: Design Experimental study of a literacy intervention for low literates, random assignment Measures of basic literacy, comprehension (same as study 1), vocabulary, listening and oral expression Assessed at start and 12 weeks later Multiple regression analysis Student demographics, teacher variables, L1, years of schooling and English literacy (pretest scores) held constant 15
16. Study 2: Descriptive Findings Data collection constrained by research design–stopped after 12 weeks Mean total hours: 75.2 Rate of attendance: 0.61 Other attendance measures could not be studied due to insufficient variation 16
18. Study 2: Attendance and Test Scores Instruction and outcomes correlated Instruction appears to have an effect on learning Relationship is weak Larger effect on reading outcomes May mean literacy gains more sensitive to gain, may be testing artifact 18
19. Summary and Conclusions Attendance hours had no (Study 1) or little (Study 2) relationship to learning, as measured by standardized tests. Attendance rate was related to oral language and reading comprehension, regardless of amount of attendance Attendance rate may reflect motivation Motivation seems to trump instruction 19
20. Discussion of Implications Attendance hours relate to instruction, but very weakly. Why? Is it because of research design or insenstive tests, or is this a real effect? Is attendance worth studying in this way? Does the weak relationship imply other means of delivering instruction (e.g., online courses) may be better Are there better ways to look a the effects of participation? For example, longitudinal persistence studies 20
No impacts represent the null hypothesis, and the most frequent scenario we see in these large impact studies. It means S&P is just as good as other instruction.
