AERA TICA Instrument Validation

Validity and Reliability of Measures of Online Reading Achievement J. Gregory McVerry & W. Ian O’Byrne University of Connecticut Katherine Robbins Clemson University Instrument : Measured online reading comprehension performance using online quiz interface. A total of 15 items were scored using a rubric. ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Results : Recoding items increased the reliability of the instrument, but also threatened construct validity of the item. For example, collapsing items 2c from a 3, 2, 1, 0 scale to a 1, 0 scale would ensure almost any response would be scored correctly. Therefore we decided to check reliability of scales collapsing only the items that did not threaten construct validity (5a, 5b, and 4e). Reliability was highest with only items 5a and 5b collapsed. Instrument : Determined which students were at the greatest risk for school dropout. Data gathered helped to determine the degree to which the Internet Reciprocal Teaching intervention yielded greater student engagement with, and attitudes towards school. ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Results : The three scale solution explained the greatest amount of variance (64.609%) and each scale had adequate reliability. The composite score was also comprised of the three scales the intervention could most affect. Instrument : Identified frequency of Internet use both inside and outside school. Assessed knowledge and skill of Internet-specific reading and writing activities. ,[object Object],[object Object],[object Object],[object Object],[object Object],Results : Explained 56.245% of variance with a marvelous KMO (.906). ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Results : Explained 50.89% of variance with a marvelous KMO (0.876). Instrument : This instrument identified and measured the attitudes and aptitudes necessary for online reading comprehension. ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],The five factors were shown to explain 38.68% of the variance, with an achieved KMO of 0.939. In future iterations of the DORC, additional items need to be written to match the constructs and improve reliability. Results : Scale Items Collapsed item in bold Initial Reliability Pre/Post  Collapsed Reliability Pre/Post  Locating 2a, 2b, 2c .758/.784 .762/.778 Critical Evaluation 3a, 3b .905/.916 .905/.922 Critical Evaluation 4a, 4b, 4c,4d, 4e, 4f, 4g .631/.619 .631/.608 Synthesis and Communication 5a, 5b , 5c .622/.605 .710/.655 Overall Pre-Test Reliability  Overall Post-Test Reliability  No Collapsed Items .705 .705 all collapsed items .736 .718 items 5a and 5b collapsed .793 .725 Items 5a,5b, and 4e collapsed .737 .720 Scale Items α School Engagement 10 .857 Learning Climate 3 .925 Teacher Supprt 6 .931 Scale Items α Out of school Internet Leisure Use 11 .932 In school online content area reading 7 .902 Out of school content area reading 8 .927 Internet Self-Efficacy 9 .926 Pop culture communication in school 7 .771 In school Internet leisure use 5 .793 Discussion boards in school 2 .713 Discussion boards out of school 4 .875 Scale Items α Reading Online 6 .812 School Literacy 5 .756 Accuracy & Reliability 3 .705 Scale Items α Reflective Thinking 14 .907 Critical Stance 4 .686 Collaboration 3 .754 Flexibility 4 .623 Persistence 2 .700

AERA TICA Instrument Validation

Recommended

Recommended

More Related Content

What's hot

What's hot (9)

Similar to AERA TICA Instrument Validation

Similar to AERA TICA Instrument Validation (20)

More from Greg Mcverry

More from Greg Mcverry (10)

AERA TICA Instrument Validation