The Do's and Dont's of Administering High Stakes Tests in Schools Final 121217
•Download as PPSX, PDF•
0 likes•207 views
There is a great deal of advice available about giving high stakes tests securely in school settings. States run annual training sessions and provide test administration manuals. Major vendors serving schools provide training and guidelines of varying types. Sometimes the different sources disagree and the emphases vary by the nature of the helping agency. What is a test administrator to do? This webinar focuses on administering tests in schools and identifies ten "best practices" that apply to all high stakes testing. The content is drawn from careful analyses of current testing practices by states, districts, and testing vendors. To be an effective test administrator, you will need to read the background materials about each testing program and attend any training that is provided. If you also follow the guidelines presented in this webinar, you will be in a very good position to promote fairness and validity in each of the programs for which you share responsibility. In this webinar, you will learn: * Ten Best Practices that apply to all high stakes testing * What is required to be an effective test administrator * How to promote fairness and validity in your testing programs Sponsored by the National Association of Assessment Directors and Caveon Consulting Services, Caveon Test Security
Recommended
Recommended
More Related Content
Similar to The Do's and Dont's of Administering High Stakes Tests in Schools Final 121217
Similar to The Do's and Dont's of Administering High Stakes Tests in Schools Final 121217 (20)
More from Caveon Test Security
More from Caveon Test Security (20)
Recently uploaded
Recently uploaded (20)
The Do's and Dont's of Administering High Stakes Tests in Schools Final 121217
- 1. The Do’s and Don'ts of Administering High Stakes Tests in Schools NAAD & Caveon December 2017 © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission.
- 2. • Formerly known as the National Association of Test Directors • A diverse group of practitioners in the field of measurement and assessment in education, preK-16. • Our membership includes: • Testing directors from school districts and post-secondary testing programs • University faculty • Representatives from non-profit educational groups and foundations • Testing companies. • Join Today! http://nationalassociationoftestdirectors.org © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 2
- 3. • Founded by a small team of test security visionaries who shared a fundamental belief in quality testing and trustworthy test results • Full service company providing test security and development services to enhance the security of many of the largest testing programs, including: • Assessment programs • States and school districts • Other entities in the certification, higher education, and national testing domains • Follow Caveon on Twitter @caveon • Check out our blog @ www.caveon.com/blog © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 3
- 4. Leigh Bennett Assessment Supervisor Loudoun County Public Schools Ashburn, Virginia USA © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 4 Today’s Moderator
- 5. Today’s Presenters © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. John Fremer Ph.D. President Caveon Consulting Services @TestSecurityGuy Lou Woodruff Ed.D. Senior Security Director Caveon Consulting Services
- 6. Agenda A. The Test Security Situation B. Ten Best Practices That Apply to All High Stakes Testing C. What is Required to Effectively Administer Assessments © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 6
- 7. A. The Test Security Situation •Who cheats? •How many of them are there? •How often do they cheat? •Why do they do it? © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 7
- 8. Josephson Institute Survey Research •95% admitted to cheating in high school •51% believed they must cheat to get ahead •90% were pleased with their own morality © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 8 40,000 College Students Surveyed in 2010
- 9. Survey of Adults Over 50 & Youth Under 17 7,000 Survey Respondents Under 17 Over 50 Believe cheating is necessary 50% 10% OK to deceive your boss 31% 8% OK to keep incorrect change 49% 15% Ok to lie to your spouse 45% 22% © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 9
- 10. Michigan Educator Survey Detroit Free Press, July 26, 2011 •29% felt pressure to cheat on standardized tests •34% felt pressure to change students’ scores for the better •21% knew educators that changed scores on students’ tests •8% admitted to changing students’ scores due to pressure © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 10
- 11. B. Ten Best Practices That Apply to All High Stakes Testing © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 11
- 12. #1 – Communication •Raise test security awareness •Include security references in communications •Issue periodic security focused communications © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 12
- 13. #2 – Expectations •Develop a system-wide Student Honor Code •Emphasize the importance of honest test taking •Issue clear expectations to test staff for professional behavior in test administration © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 13
- 14. #3 – Procedures •Review all procedures for security related content •Emphasize procedures for handling test materials •Review / update security procedures annually © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 14
- 15. #4 – Training •Review your test security training materials •Track who completes the training •Provide refresher training annually © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 15
- 16. #5 – Observations •Perform announced pre-test session site visits •Perform unannounced test session observations •Follow up on all observation outcomes © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 16
- 17. #6 – Feedback •Seek feedback from test administrators / proctors •Review procedures for reporting test irregularities •Provide an anonymous tip line © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 17
- 18. #7 – Study Your Test Data Carefully •Very unusual gains •Other indices • Similarity • Erasures • Individual patterns • Timing (in computer testing) © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 18
- 19. #8 – Preparing for Possible Security Incidents •Develop a Security Incident Response Plan (SIRP) •Specify procedures, roles and responsibilities •List potential outcomes for security infractions © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 19
- 20. #9 – Learning From Others •Learn about developments in prevention and detection • CCSSO TILSA SCASS “Guidebook on Test Security” • Wollack & Fremer “Handbook of Test Security” • Caveon Webinars & Security Insights Newsletter © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 20
- 21. #10 – If You Develop Your Own Tests •Reduce exposure rate of test questions •Randomize question locations •Rotate answer options © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 21
- 22. C. What is Required to Effectively Administer Assessments •Professional Commitment •Supporting Organizational Structure •Comprehensive Training •Diligence – Details – Determination © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 22
- 23. Test Administration Challenges •Maintaining a standardized environment •Avoiding conflicts of interest •Preserving fairness and security •Protecting secure test materials © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 23
- 24. A School District Test Security Matrix •District Superintendent •District Assessment Coordinator •Building Test Coordinator •Testing Room Proctor © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 24
- 25. Common Sense Steps for Secure Testing •No one should access secure test materials alone •Prohibit jackets, hats & belongings during testing •No electronic devices allowed in the testing room •Maintain a secure distance between test takers •Be actively engaged in proctoring during testing •Commit to established policies and procedures •Create a consistently positive testing environment © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 25
- 26. Security Incident Response Planning • Communicate procedures for responding to incidents • Establish thresholds for staff actions • Define how evidence should be secured and reported • Set communication & decision making protocols • Develop strategies for public relations management • Maintain confidentiality of all involved parties • Publish the appeals process and potential sanctions © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 26
- 27. Promoting Fair and Honest Testing •Model the importance of the assessment process •Emphasize the value of honesty in test taking •Avoid giving any student an unfair advantage •Have students sign formal no-cheat agreements •Also use agreements with staff administering tests •Display a commitment to following the rules © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 27
- 28. Thank You! John Fremer President Caveon Consulting Services John.Fremer@caveon.com © 2017, Caveon, LLC. All rights reserved. Do not copy or distribute without permission. 28 Lou Woodruff Senior Security Director Caveon Consulting Services Lou.Woodruff@caveon.com Leigh Bennett Assessment Supervisor Loudoun County Public Schools Ashburn, Virginia USA
Editor's Notes
- Be sure to insert polls and poll placeholder slides. (See John’s Email) Also insert Q&A slides between parts 1,2,3 and at end
- Recent Headlines: Paid Test taker proxies on the SAT Student caught hacking the College Bowl quiz questions
- Remember John’s guidance: This webinar must provide participants with suggested steps they can take themselves.
- However, it could be much, much worse. A couple of months ago, results of a Michigan educator survey were published. Of the 3,000 or so teachers who responded: 34% felt pressure to change grades 1 in 3 felt pressure to cheat on standardized tests 21%, 1 in 5 said they know of an educator who changed scores, and most alarming… 8% admitted to changing students’ grades due to outside pressure. Holy smokes—if that’s the case, we just jumped from 1,700 to 14,000 classrooms we’re concerned about!
- This is a great slide to acknowledge the unique demands and challenges for assessment in school districts. Be prepared to mention a few additional points that reveal your long experience in schools. Also the notion that the teachers, if engaged in inappropriate assistance to students, are doing so in the misguided notion that their actions represent student advocacy rather than self-imposed pressure to use test results to validate their teaching effectiveness. Potential Conflicts of Interest Teacher Accountability & Bonuses tied to Score Gains School Accountability or exit from “Under Improvement” Status Non-Standardized Environment Making do in potentially compromised test situations with word walls, other cues, students in close proximity, lax proctor monitoring, cell phones Clustering distractible or poor students with good models in small groups Testing in short chunks to avoid fatigue, or maximize effort Examinee Familiarity Teacher identification with, advocacy for, students with whom they have invested considerable effort Student Halo effect in the eyes of teacher who also their proctor, scorer Transporting and storing test materials Large schools/long hallways, and concessions by building test coordinator to teachers on storage, access, collection Shortcuts by teacher proctors and building test coordinators when crises or fatigue arises, or typical last minute dilemmas such as teacher absence, time crunch toward end of testing window LOST secure materials, unauthorized prep guides, teacher copies items from screen
- I’ll touch on some specifics for each key role across the segmented time frame of state assessment programs, but I invite you to review the entire Test Security Matrix located on the Caveon website at the link shown in the next slide. The District leader sets the tone for the overall administration of all phases of the district’s assessment program- including state as well as local assessments. Positive, highly visible, and well-organized interactions by the school leader with test proctors, building assessment coordinators, student groups, the teacher association, and the community serve the district well by signaling the importance and the overall monitoring of a sound program to measure and report on student growth. All local stakeholders will be reassured by these activities- while potential mischief makers will be put on alert that their efforts to subvert the process will quickly be brought to light. Prompt investigation of suspected test irregularities will further reinforce this positive stance. In the absence of a strong consistent leader stance on the importance of adhering to the established assessment procedures – before, during, and after the testing window, teachers, administrators, and students often come to believe they are on their own- each may have, as we have commented, different perspectives on the importance of both the testing, and their own rule compliance. In instances when school leaders emphasis score gains at all costs, threatening discipline, demotion, or dismissal for poor results, the potential rises for score erasure activity, deliberate coaching before and during test sessions, or actions such as illegally removing low scoring students from attendance rolls .
- General comments here about the bulletted items.
- General comments here about the bulletted items.
- General comments here about the bulletted items.