In recent years, test security issues have received greater attention, almost to the point of distraction, by school system administrators. Motivations to cheat on state assessments appear to be higher than ever. The number of test security violations, the severity of breaches, and risks to state assessments have been increasing. Members of the PARCC and SBAC consortia are using the same tests, increasing the likelihood that actual content of state assessments will be illicitly distributed on the Internet. Unless action is taken soon, we may experience the perfect test security storm in state assessments. This storm is likely to result in more educators being forced to contend with security issues, more revelations of test security breaches, and more emergency funding requests to deal with the aftermath. Presenters of this webinar will explain why this has happened and what state departments of education and school districts can do to handle and mitigate test security breaches. Dennis Maynes and Jennifer Miller from the Caveon Data Forensics team will present this informative session. They will be joined by special guest Michelle Croft, Principal Research Associate at ACT, Inc. Prior to joining ACT, Michelle worked in the assessment division at the District of Columbia Office of State Superintendent of Education. Important, timely topics to be covered in the presentation are: • Best practices that can be used to prevent, detect, and respond to breaches. • Recommendations for formalizing processes and adopting a quality improvement approach to test security. • Suggestions for how to measure, monitor, and manage test security risks.

1. Jennifer Miller, CaveonTest Security
Dennis Maynes, CaveonTest Security
Michelle Croft,ACT
Weathering the PerfectTest Security
Storm in Educational Assessments
February 25, 2015
Caveon Webinar Series presents:

2. Agenda for Today
• Elements of the Perfect Storm
(Jennifer Miller)
• Fallout from the storm
(Michelle Croft)
• Weathering the storm
(Dennis Maynes)

3. Introduction

4. The Perfect Storm
• Test scores are tied to school funding.
• Performance evaluations are tied to test
scores.
• The responsibility of test security is placed in
the hands of those who are motivated to
cheat.
• Security threats and vulnerabilities are
increasing.

5. NCLB (2001)
“Adequate yearly progress”
Federal funding goes to struggling
schools, but…..
…with funding come sanctions
NCLB & Funding

6. • Each state sets timeline and definition for AYP.
• Schools held accountable for performance of all
students, not just the average performance.
• Schools that do not meet AYP for two years in a
row are subject to immediate interventions.
– Technical assistance
– Severe corrective actions
Adequate Yearly Progress

7. ESEA Flexibility Waiver
• Announced in 2011 to allow flexibility from
some specific NCLB requirements.
• State-developed plans where turnaround
interventions are implemented at “priority”
schools.
• Requires student growth data as part of
educator evaluations.

8. Parks Middle School in Atlanta
“school in need of improvement” 2001-2005
In 2006, “Unless fifty-eight per cent of students passed the
math portion of the test and sixty-seven per cent passed in
language arts, the state could shut down the school.”
State Assessments and School Funding
The NewYorker, 21 July 2014
Governor’s investigation in Georgia found cheating in 44
APS schools, with almost 200 educators potentially
involved.

9. “Investigators found ‘organized and systemic
misconduct’ …and said administrators created a
‘culture of fear, intimidation and retaliation.’
The administrators used data as an ‘abusive and
cruel weapon’ to coax employees into crossing
ethical lines”
Los Angeles Times, 6 September 2014
State Assessments and School Funding

10. Performance Evaluations and Test Scores
States are moving to performance-based compensation and
using test scores to evaluate teachers
Administrators
(Superintendents, etc.)
Teachers

11. • 2012 El Paso Independent School District.
• Lorenzo Garcia, Superintendent.
• Discouraged and/or prevented low-
performing students from taking the TAKS.
• 381 students 170 students
• Garcia received almost $60K in bonuses.
New York Times, 13 October 2012
Use of Test Scores for Personal Gain

12. Test Security in the Hands of Teachers
"If the IRS notified you that your tax return had irregularities,
and they wanted you to recheck it yourself, everybody will
write back and say, 'I checked my taxes, and they're fine.‘”
– Greg Cizek

13. Test Security in the Hands of Teachers
Teachers feeling the pressure:
Survey of Educators in Michigan
• 29% felt pressure to cheat on standardized
tests.
• 34% felt pressure to help students.
• 21% knew other educators who helped
students.
• 8% admitted to changing answer sheets.
“Survey: Nearly 30% of Michigan Teachers Report Pressure to
Cheat,” Detroit Free Press, July 27, 2011

14. Threats are Increasing
• Cheating technology is
becoming more sophisticated.
• Threats and CBT.
• Threats associated with
consortia membership (PARCC
and SBAC).
• Threats with Social Media.
thegreenhead.com

15. • Harm to Students
• Tarnished Reputations
• Lost Credibility
• Negative Media Attention
•$$$$$
Fallout from the Storm

16. Harm to Students
“As a graduating senior from a tarnished school
system, I have to worry every time I apply to a
college…good grades isn’t considered a reward in
itself anymore because those grades come from a
school system of perceived cheaters…it’s no joke for
students who have to overcome the perception of a
morally bankrupt school system.”
Fallout from the Storm
Jenn Steckl
APS School District
youthradio.org
27 October 2014

17. Negative Media Attention
• APS Scandal, Atlanta Journal Constitution,
2008.
• Cheating in 44 APS schools.
• Prosecution of 12 educators under RICO
• Trial ongoing…
Fallout from the Storm

18. Pa Schools, The Notebook, 2009
• Anomalies in 89 Schools - Cheating?
• As school starts, no action against Philly principals where cheating is suspected,
suspected, 8/24/2012.
• Two District administrators surrender licenses in cheating scandal, 4/3/2013.
• District to release report on cheating investigation at 19 schools, 7/22/2013.
• 138 Philadelphia educators implicated in cheating probes, 2/5/2014.
• SRC to vote on terminating three principals related to cheating investigation,
1/16/2014.
• Two former principals charged in state cheating investigation, 9/25/2014.
• Former principal now eighth educator charged in cheating probe, 1/7/2015.
Fallout from the Storm
thenotebook.org

19. Fallout from the Storm
District of Columbia
• USAToday article (2011).
• Entire year of items released.
• Solicited bids for external
investigators to increase
transparency.

20. • Budget
• Strategy
• Implement a Comprehensive Program
Weathering the Storm

21. • Strategy
• Create Mission andVision Statements
• AssessThreats andVulnerabilities
• Review and Allocate Resources
• Establish a Communications Plan
Weathering the Storm

22. • Implement a Comprehensive Program
• Protection
• Detection
• Response
• Improvement
Weathering the Storm

23. Weathering the Storm
Comprehensive Program
Protect
Detect
Respond
Improve

24. “Never before have so many had so much
reason to cheat. Students’ scores are now used
to determine whether teachers and principals
are good or bad, whether teachers should get a
bonus or be fired, whether a school is a success
or failure…”
NewYorkTimes, 31 July 2011

25. Audience Questions
25

26. Thank you!
- Follow Caveon on twitter @caveon
- Check out our blog…www.caveon.com/blog
- LinkedIn Group – “Caveon Test Security”
Jennifer Miller
Data Forensics Coordinator,
Caveon
jennifer.miller@caveon.com
Michelle Croft
Principal Research Associate,
ACT
michelle.croft@act.org
Dennis Maynes
Chief Scientist, Caveon
dennis.maynes@caveon.com