Diagnosis the missing ingredient in RTI assessment


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Diagnosis the missing ingredient in RTI assessment

  1. 1. 204The Reading Teacher Vol. 65 Issue 3 pp. 204–208 DOI:10.1002/TRTR.01031 © 2011 International Reading AssociationR TP E R S P E C T I V E S O N R T IDIAGNOSISDescriptions and advice about Responseto Intervention (RTI) implementationsoften focus on assessment. In an earliercolumn, Wixson and Valencia (2011)described the multiple purposes for assessment in anRTI system:■ Screening■ Diagnostics■ Formative progress monitoring■ Benchmark progress monitoring■ Summative outcome assessmentBecause RTI is an approach to identifying studentsas learning disabled, it is important to attend to thequality of the measurements used to make high-stakes decisions about student placements. However,RTI is also intended to reduce the number ofstudents who become identified as having a learningdisability by preventing reading difficulties. This is aninstructional problem and, as Scanlon and Anderson(2010) noted, too little attention is given to “the natureand qualities of the instruction that is offered” (p. 21).All students deserve high-quality first instruction (wewill address this topic in an upcoming column).However, good teaching requires thoughtfulassessment. Furthermore, some students continue tostruggle, even when they receive very good classroominstruction. These students need more tailoredinstruction that is responsive to their specific strengthsand areas of need. Screening measures designed toidentify struggling students are usually quite general orsample only a snippet of skilled reading performance.They rarely provide the specific information neededto determine the most appropriate intervention orinstruction.By itself, this would not be problematic. However,many schools and districts have created RTI systemsthat move straight from screening to instructionwithout looking more closely at the individualstudent, an approach called the direct route (Johnson,Jenkins, & Petscher, 2010). Of course, when thedirect route decision is based on only one screeninginstrument, there is quite a grave possibility of errorin classification (false negatives or false positives). Aswell, these single measures do not provide enoughinformation to make good instructional decisionsabout individuals or small groups of students(Valencia et al., 2010).This problem is exacerbated when the school adoptsa one-size-fits-all intervention, a practice sometimescalled standard protocol (Fuchs et al., 2003). AlthoughFuchs and his colleagues were clear that a standardprotocol was to be employed with groups of studentswith similar reading difficulties, a single screeningmeasure often invites only one possible response, and,Marjorie Y. Lipson and Pam Chomsky-Higgins provide research andprofessional development support to schools through the Vermont ReadsInstitute at the University of Vermont, Montpelier, USA; e-mail mlipson@vriuvm.org and pamela.chomsky-higgins@uvm.edu.Jane Kanfer is a reading specialist at Milton Elementary School inVermont; e-mail JKanfer@mtsd-vt.org.The department editors welcome reader comments. Karen K. Wixsonis dean of the school of education at the University of North Carolinaat Greensboro, USA; e-mail kwixson@uncg.edu. Marjorie Y. Lipson isProfessor Emerita at the University of Vermont, Burlington, USA; e-mailmlipson@uvm.edu.The Missing Ingredient in RTI AssessmentMarjorieY. Lipsonwith Pam Chomsky-Higgins ■ Jane KanferTRTR_1031.indd 204TRTR_1031.indd 204 11/7/2011 4:05:47 PM11/7/2011 4:05:47 PM
  2. 2. DIAGNOSIS: THE MISSING INGREDIENT IN RTI ASSESSMENTwww.reading.org R T205for many schools, the direct route leads tojust one instructional option.A very large body of research isemerging to confirm what good teachersand specialists have always known: theunderlying roots of students’ readingdifficulties are diverse (Aaron et al.,2008; Valencia & Buly, 2004). In addition,it is becoming quite clear that instructionfocused on the wrong thing not onlydoes not help students, but it mayactually be harmful (Connor, Morrison,& Katch, 2004). It should be a very highpriority for teachers and specialiststo gather specific information aboutindividual students to make appropriateinstructional decisions. So, we turnour attention to an essential, but oftenneglected, type of assessment: diagnosis.Using Assessmentfor Diagnostic PurposesBecause the word diagnosis is so oftenassociated with identifying disease orits symptoms, we want to be clear thatwe do not assume a medical model ofreading difficulties. Instead, we wantto promote the original meaning ofthe word diagnosis, from the Greek todiscern the nature and cause of anything.According to Merriam-Webster, adiagnosis is “an investigation or analysisof the cause or nature of a condition,situation or problem” (www.aolsvc.merriam-webster.aol.com/dictionary/diagnosis). Without very good diagnosticinformation and/or a flexible formativeassessment system, our instructionalprograms and student performance willnot improve, and RTI will simply bean alternate route to special educationplacement or to permanent membershipin Title I classrooms.We have been collaborating withmany Vermont schools to improveliteracy success for all students and haverecently turned our attention to helpingschools adopt a systemic approach toRTI. Pam and Marge started to workat Jane’s school 2 years ago. This largeVermont school was entering year 3 ofcorrective action. Despite several years ofattention to literacy, too many studentswere identified for special education,and too few were benefitting from theliteracy instruction they received.The reading teachers in the buildingwere knowledgeable and committed.However, they were relying on aconventional approach to services. Usinguniversal assessment data, they identifiedstudents near the beginning of the year,created a schedule for the students on“their caseload,” and then pulled thosestudents from the classroom for theremainder of the year. The nature of theinstruction provided varied from teacherto teacher and was not well documented.More troubling were the longitudinaldata suggesting that students whoparticipated in reading services (providedby reading teachers or special educators)were not gaining ground against theirpeers. Although there were exceptions,students typically continued to qualify forreading services from year to year, andtheir overall performance was below thestate standards.During year 1 of our collaboration,Jane and her colleagues began toexamine their available assessment datamuch more closely. They moved fromglobal reading levels to looking at thecomponent areas of reading (e.g., wordrecognition, fluency, comprehension),and they constructed “typical profiles”of student difficulties (see Figure for theframework). The first profiles focused notFigure Template for Analyzing Student Data and Creating ProfilesIntervention Plan for Literacy—TemplateStudent Profile: [Describe or name the type of profile—for example, Struggling Grade 1 Student orIntermediate Comprehension Difficulty]Intervention Focus: [Describe or name primary focus for intervention]Area(s) of Primary Concern Evidence and Data SourcePhonological awarenessLetter IDSight vocabularyFluencyDecodingVocabularyComprehensionMotivationStaminaWriting about readingText levelOtherGoal(s) of the Intervention:Describe the Intervention: [Include Information about:• Duration• Frequency• Materials to be used• Strategies, techniques, approaches, etc.• Instructional framework• Person responsible• Relationship to classroom instruction]Progress Monitoring: [How will instructional impact and effectiveness beassessed?]TRTR_1031.indd 205TRTR_1031.indd 205 11/7/2011 4:05:48 PM11/7/2011 4:05:48 PM
  3. 3. DIAGNOSIS: THE MISSING INGREDIENT IN RTI ASSESSMENT206The Reading Teacher Vol. 65 Issue 3 November 2011R Ton the students with the most severe (andleast frequent) types of difficulties, but onthe most common types of difficulties—arguing that these would have thebroadest impact and might also provideinsights into classroom practices thatcould be strengthened. These studentsare not necessarily the so-called “bubblekids,” those whose test results suggestthat they are very nearly proficient (or “onthe bubble”). Cynical practices that directteachers to focus on those students sothat schools can realize “improvement”have been criticized roundly elsewhere(Koretz, 2008). In some schools, the mostcommon profile might be students whoare close to proficiency; in others, it couldbe other students.The most common profile whenwe started involved struggling grade1 students who showed very limitedproficiency. We called this profile“beginning struggling word learner,”and the intervention focus was strategicdecoding. These were students who had,with good instruction in kindergarten,come to grade 1 knowing all or most ofthe letter names and many/most lettersounds. They typically knew a smallcollection of high-frequency sight words.However, they had not met the leveledtext standard for end of kindergarten(Level B, guided reading level) and weremaking very slow progress in letter-sound blending and in problem solving.These students were an early focus forus because we knew that helping themwas pivotal in preventing reading failurelater on, especially because anothercommon profile in the school involvedstudents in grades 3 through 6 who hadacquired good decoding skills but whosecomprehension was weak. These studentswere especially worrisome because theirperformance actually declined as theymoved through the grades.As we created diagnostic profiles,Jane and her colleagues detailed theresearch-basedinterventionplansappropriate foreach student.The resultwas similarto what Dornand Henderson(2010) calleda “portfolio ofinterventions.”AlthoughJane’s schoolhas not fullyimplementedthe portfolio of interventions, the resultsof the early interventions have beenexcellent. By the middle of the secondyear of collaboration, every student ingrade 1 who had been identified as moststruggling met or exceeded the standardon the periodic benchmark assessmentused by the school, and most werereleased from intervention. Importantly,these students did not receive moreintervention than they had in the past;they simply received more tailored andfocused instruction based on carefuland comprehensive assessment andthe research-based interventions,confirming what others have observed:More intervention is not necessarilybetter (Wanzek & Vaughn, 2008).Diagnostic AssessmentPracticesClassroom teachers and readingspecialists may already have excellentassessment information available tothem collected for screening, periodicbenchmarking, or outcomes evaluation.In these cases, the challenge is toadopt systems that permit effective useof that information for instructionaldecision making. Designing data-teamformats and/or profiles can facilitate thatconsiderably. Other schools and districtseither have few data or have restrictedteachers’ access to the information.There are two actions that may beneeded: (1) disaggregating existingdata to mine it for insights and/or (2) planning to gather additionalinformation. In either event, it isimportant to start with a matrix of themost important component areas ofreading (and writing). These are visibleon the profile planner (see Figure). Ofcourse, not all information is equallyrelevant for all students or all ages/grades. One of the major difficultieswith a standard protocol approach isthat it may not recognize that there“The challenge is to adopt systems thatpermit effective use of that informationfor instructional decision making.”TRTR_1031.indd 206TRTR_1031.indd 206 11/7/2011 4:05:49 PM11/7/2011 4:05:49 PM
  4. 4. DIAGNOSIS: THE MISSING INGREDIENT IN RTI ASSESSMENTwww.reading.org R T207are developmental changes in therelationships between and amongvarious factors and reading performance(Valencia et al., 2010).Whenever possible, it is efficientto gather observations about multipletypes of reading performance with onetool. At the very least, this can helpnarrow the need for further assessment.For example, Jane’s school, like manywe work with, uses one of the manystructured inventories or benchmarkassessment systems available (i.e., DRA,Benchmark Assessment System, QRI-4),which yields a text level for each student.Over the past 2 years, the readingprofessionals in that building have begunto examine the results more closely forevidence of word recognition accuracy,fluency, and comprehension. Whereasthey had previously used only the textlevel to make decisions about studentplacements and programs, they nowlook more closely at their data. They nowexamine (and keep track of) differencesbetween students’ word recognitionaccuracy as distinct from fluency. Theyalso consider whether students’ accuracyis affected by phonics decoding, sightword recognition, or both. Finally, ofcourse (and especially for older students),they examine differences betweenand among resultsin comprehension,fluency, and wordidentification. They havedifferentiated instructionfor those students whowere struggling withcomprehension eventhough they were quitefluent and vice versa.It is important toexamine assessmentresults in more than oneway because differentmeasures can providedifferent insights. AsValencia and her colleagues (2010)demonstrated, we might get quite adifferent picture of a student’s strengthand areas of need with an array ofinformation than if we relied on onlyone score. The student profile form wasintended to encourage teachers to lookat multiple measures to identify areas ofconcern that were not at all evident withonly word-level assessment data. Thiscomprehensive range of information isimportant for all struggling readers, butit is especially important to make goodinstructional decisions for older studentsand English learners (Klingner, Soltero-Gonzales, & Lesaux, 2010; Vaughn &Fletcher, 2010).Every bit of this diagnosticinformation can be gathered duringcomprehensive benchmarking events.Once we have looked at the data, wecan make additional decisions aboutboth assessment and instruction.Good diagnosis can sometimes leadto more assessment. For example, if astudent has made many miscues duringreading, further examination usinga modified miscue analysis can behelpful (Lipson & Wixson, 2009). If thestudent is exceptionally accurate, thereis no need to explore that component.Similarly, if the student’s comprehensionis strong, we probably do not needto look to vocabulary. This mightchange, of course, if we have evidencethat the student performs less well onexpository text than narrative. Well-planned diagnostic teaching episodescan be helpful as well (Fuchs et al., 2007;Lipson & Wixson, 1986, 2009).Using Assessmentto DifferentiateWe want to emphasize that this typeof closer look, engaging in diagnosticassessment, is not necessary foreveryone. One of the advantages of aneffective overall assessment system isthat it can identify the students who arebenefitting from classroom instructionand who are doing well. If we test thosechildren just a bit less (or use good,formative assessment effectively), then itfrees up time to assess the students whoreally have us worried.Of course, diagnostic assessmentwill not make any difference if it doesnot lead to action. Matching students’needs with instruction by using “if-then”thinking (Strickland, 2005) is at the heartof effective and efficient assessmentinstruction and is essential if RTI is goingto make a difference. Students who arestruggling for success in the classroomdeserve differentiated instruction in theclassroom and tailored interventions insupplemental settings. Fortunately, thereis an increasingly strong body of researchthat points to the types of instruction“Of course, diagnosticassessment will notmake any differenceif it does not leadto action.”TRTR_1031.indd 207TRTR_1031.indd 207 11/7/2011 4:05:50 PM11/7/2011 4:05:50 PM
  5. 5. DIAGNOSIS: THE MISSING INGREDIENT IN RTI ASSESSMENT208The Reading Teacher Vol. 65 Issue 3 November 2011R Tand intervention that can help to preventreading disabilities and/or improvestudents’ abilities during later years.REFERENCESAaron, P.G., Joshi, R.M., Gooden, R., &Bentum, K.W. (2008). Diagnosis andtreatment of reading disabilities basedon the component model of reading: Analternative to the discrepancy model of LD.Journal of Learning Disabilities, 41(1), 67–84.doi:10.1177/0022219407310838Connor, C.M., Morrison, F.J., & Katch, L.E.(2004). Beyond the reading wars: Exploringthe effect of child-instruction interactionson growth in early reading. Scientific Studiesof Reading, 8(4), 305–336. doi:10.1207/s1532799xssr0804_1Dorn, L. J., & Henderson, S. C. (2010). Thecomprehensive intervention model: Asystems approach to RTI. In M.Y. Lipson &K.W. Wixson (Eds.), Successful approaches toRTI (pp. 88-120). Newark, DE: InternationalReading AssociationFuchs, D., Fuchs, L.S., Compton, D.L.,Bouton, B., Caffrey, E., & Hill, L. (2007).Dynamic assessment as responsiveness tointervention. Teaching Exceptional Children,39(5), 58–63.Fuchs, D., Mock, D., Morgan, P.L., & Young,C. (2003). Responsiveness to intervention:Definitions, evidence, and implications forthe learning disabilities construct. LearningDisabilities Research & Practice, 18(3), 157–171. doi:10.1111/1540-5826.00072Johnson, E.S., Jenkins, J.R., & Petscher, Y.(2010). Improving the accuracy of a directroute screening process. Assessment forEffective Intervention, 35(3), 131–140.doi:10.1177/1534508409348375Klingner, J.K., Soltero-Gonzales, L., & Lesaux,N. (2010). RTI for English-language learners.In M.Y. Lipson & K.K. Wixson (Eds.),Successful approaches to RTI: Collaborativepractices for improving K-12 literacy (pp. 134–162). Newark, DE: International ReadingAssociation.IRA Books■ RTI and the Adolescent Reader by William G.Brozo (copublished with Teachers CollegePress)■ RTI in Literacy—Responsive and Comprehensiveedited by Peter H. Johnston■ Successful Approaches to RTI: CollaborativePractices for Improving K–12 Literacyedited by Marjorie Y. Lipson andKaren K. WixsonIRA Journal Articles■ “Response to Intervention (RTI):What Teachers of Reading Need to Know”by Eric M. Mesmer and Heidi AnneE. Mesmer, The Reading Teacher,December 2008■ “Responsiveness to Intervention: MultilevelAssessment and Instruction as EarlyIntervention and Disability Identification” byDouglas Fuchs and Lynn S. Fuchs,The Reading Teacher,November 2009MORE TO EXPLOREKoretz, D. (2008). Measuring up: Whateducational testing really tells us. Boston:Harvard University Press.Lipson, M.Y., & Wixson, K.K. (1986). Readingdisability research: An interactionist perspective.Review of Educational Research, 56(1), 111–136.Lipson, M.Y., & Wixson, K.K. (2009). Assessmentand instruction of reading and writingdifficulties: An interactive approach (4th ed.).New York: Pearson Education.Scanlon, D.M., & Anderson, K.L. (2010).Using the interactive strategies approach toprevent reading difficulties in an RTI context.In M.Y. Lipson & K.W. Wixson (Eds.), Successfulapproaches to RTI (pp. 20–65). Newark, DE:International Reading Association.Strickland, K. (2005). What’s after assessment?Follow-up instruction for phonics, fluency,and comprehension. Portsmouth, NH:Heinemann.Valencia, S.W., & Buly, M.R. (2004). Behind testscores: What struggling readers really need.The Reading Teacher, 57(6), 520–531.Valencia, S.W., Smith, A.T., Reece, A.M., Li,M., Wixson, K.K., & Newman, H. (2010).Oral reading fluency assessment: Issuesof construct, criterion, and consequentialvalidity. Reading Research Quarterly, 45(3),270–291. doi:10.1598/RRQ.45.3.1Vaughn, S., & Fletcher, J. (2010). Thoughts onrethinking response to intervention withsecondary students. School PsychologyReview, 39(2), 296–299.Wanzek, J., &Vaughn, S. (2008). Responseto varying amounts of time in readingintervention for students with lowresponse to intervention. Journal ofLearning Disabilities, 41(2), 126–142.doi:10.1177/0022219407313426Wixson, K.K., & Valencia, S.W. (2011).Assessment in RTI: What teachers andspecialists need to know. The ReadingTeacher, 64(6), 466–469.TRTR_1031.indd 208TRTR_1031.indd 208 11/7/2011 4:05:52 PM11/7/2011 4:05:52 PM
  6. 6. Copyright of Reading Teacher is the property of International Reading Association and its content may not becopied or emailed to multiple sites or posted to a listserv without the copyright holders express writtenpermission. However, users may print, download, or email articles for individual use.