AIR Presentation: OPAPP and FAMS: Two Studies of Interim Assessments


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

AIR Presentation: OPAPP and FAMS: Two Studies of Interim Assessments

  1. 1. OPAPP and FAMS: Two Studies of Interim Assessments Shazia R. MillerCopyright © 2011 Managing DirectorAmerican Institutesfor ResearchAll rights reserved. June 2012
  2. 2. Interim AssessmentsIntroduction ASSESSOPAPPFAMSWrap Up ADJUST INTERPRET INSTRUCTION 2
  3. 3. IntroOPAPP Make no little plans: theyFAMS have no magic to stirWrap Up men’s souls. Daniel Burnham3
  4. 4. Study 1: Ohio Performance Assessment Pilot Program (OPAPP)Intro Learning Assessment Task Dyad-SystemOPAPP  Rich, lengthy complex learning tasks  Shorter but still complex onlineFAMS assessmentsWrap Up  Scoring of online assessments by trained teachers  Re-engagement as appropriate4
  5. 5. Video of a Learning Task5
  6. 6. Summary of Research Questions 1.How is OPAPP implemented?Intro 2.Does the OPAPP assessment allow students to demonstrate their skills?OPAPP 3.What valid inferences do school staff believe can be made from students’ performance on online assessment tasks?FAMS 4.Do schools have the required technology? 5.How do students’, teachers’, administrators’ and coaches’Wrap Up perceptions change? 6.What policy or cultural changes are implemented in schools (i.e., extra prep time, PD, support, communities of practice)? Can these be sustained? 7.To what extent do the programs result in improved student performance?6
  7. 7. OPAPP Data CollectionIntro • Surveys • Extant DataOPAPP • Program ParticipationFAMS • Online server logs • Case Study InterviewsWrap Up • Documents • Video 7
  8. 8. OPAPP Classroom Video FootageIntroOPAPPFAMSWrap Up A stationary A hand-held , Interview with camera. mobile camera the teacher 8
  9. 9. Details of VideographyIntro Videography conducted in spring 2012: • 4 science and 1 math classroomOPAPP • At 1 high school • A range of beginning and veteran teachersFAMS • One inclusive, special education classroom withWrap Up co-teaching • 1 class period of a learning task (Learning tasks took pilot teachers anywhere from 3 to 15 class periods to complete.) 9
  10. 10. Paired Group Discussion Find a partner near you and discuss how a research could make use of this rich, but unsystematic data.10
  11. 11. How We Used the Videos: Modified Cognitive InterviewsIntro • 2 researchers reviewed all clipsOPAPP • Created sequences focusing on teacher instructional practice.FAMS • Reviewed videos with the teachersWrap Up  What are you doing now (and why)?  What are your students doing now (and why)?  How typical is this for your class?  Questions specific to video. 11
  12. 12. Study 2: Formative Assessments for Middle School (FAMS)Intro • Goal for teachers to developOPAPP individual portfolio of methodsFAMS  Assistance - Professional developmentWrap Up - Intensive coaching - Communities of practice 12
  13. 13. Summary of Research Questions 1.What changes are there in perceptions of the program?Intro 2.What changes are there in policy or culture? 3.To what extent do the programs result in improved studentOPAPP performance? 4.What factors are involved in developing and maintainingFAMS communities of practice? 5.What are the facilitators and barriers to developing andWrap Up maintaining communities of practice? 6.What are the key elements of the coaching program that make it effective, or prevent it from being effective? 7.How can the coaching component be scaled up? 13
  14. 14. Data CollectionIntro •SurveysOPAPP •Coach logs •Extant documentationFAMS •InterviewsWrap Up •Student performance on state assessments 14
  15. 15. FAMS Preliminary Interview Findings from Two Case Study SchoolsIntro Successes • Teacher growthOPAPP • Student intellectual risk takingFAMS • Self AssessmentWrap Up 15
  16. 16. FAMS Preliminary Interview Findings from Two Case Study SchoolsIntro ChallengesOPAPP • Program designed to be individualized, schools take ownershipFAMS • Teachers and coaches sometimes feltIndiana there was not enough guidanceWrap Up 16
  17. 17. IntroOPAPP Discussion: What are the risksFAMS and benefits of a individualizedWrap Up program? 17
  18. 18. Interim AssessmentsIntroOPAPPFAMSWrap Up 18
  19. 19. Shazia R. Miller P: 312-288-7616 E-Mail: 20 N. Wacker Dr, Suite 1231 Chicago, IL 60606 General Information: 800-356-2735 Website: www.air.org19
  20. 20. OPAPP: Who is in the Pilots? Fall Spring Fall Spring Fall Spring 2011 2012 2012 2013 2013 2014 23 Continuing Cohort High Schools 1 Grades 11-12 ELA, math, science 10 New High Schools Cohort Grades 9-12 2 ELA, math, science, social studies, career tech 10 New Elementary Schools Cohort Grades: 3-5 3 ELA, math, science, social studies20
  21. 21. FAMS: Who is in the Pilots? Fall Spring Fall Spring Fall Spring 2011 2012 2012 2013 2013 2014 Cohort 6 Schools 1 Grades 5-8 ELA and Math Cohort 7 Schools 2 Grades 5-8 ELA and Math Cohort 9 Schools 3 Grades 5-8 ELA and Math21