In the coming weeks, schools all over the country will be administering standardized exams to millions of students. And inevitably, test security incidents will arise, many of which may directly impact test score validity. Is your team prepared to answer the following tough questions? • What will you do if you find yourself in a position of having to respond to an incident or breach in your state or district? • What process will you follow? • What is your incident escalation plan? • How will you communicate with internal and external stakeholders? • Most importantly, how will you discover the truth of what did or did not occur, and its impact on test scores? Join Caveon’s test security experts for an important, hour-long webinar to help you understand the steps to take when challenging situations arise. We will share: • Recent experiences other districts have had with possible cheating, and what they have done to resolve their concerns • Information and tools for you to arm yourself before an issue arises, and to help you be better equipped to deal effectively and efficiently • Essential tips you need to know when invoking a Security Incident Response Plan, and further conducting a security investigation

1. www.caveon.com1
Caveon Webinar Series
Conducting Test Security
Investigations in Districts and
LEAs
May 25, 2016

2. www.caveon.com2
Today’s Presenters
Steve Addicott Marc Weinstein

3. Agenda for Today
www.caveon.com3
• What’s going on in K-12 testing
• The impact of a security incident
• Preparing for an incident, before one
occurs
• Building your Security Incident Response
Plan
• Conducting an internal investigation
• Summary
• Q&A

4. www.caveon.com4
What’s Happening Out There?
• Recent, real-life situations some districts
are facing:
o “Disappearance” of Austin High Students
Sparked Questions
o 50 Students Face Grade Tampering
Allegations
o Two Colorado Springs Teachers May Have
Shared Test Information with Students
o U. of Iowa Investigates Possible Cheating in
Online Courses
o 3 Face Expulsion in Coast Guard Academy
Cheating Probe

5. www.caveon.com5
What is at stake?
Consider what you are protecting
 Validity of assessments
 Accountability determinations
 Educational resource decisions
 Professional evaluations
 Exam content
 Funding
 Public trust and confidence
 Public Welfare

6. SECURITY = TEST SCORE VALIDITY
Test Security = Exam Integrity
Exam Integrity = Test Score
Validity
6 www.caveon.com

7. www.caveon.com7
Test Security is an Ongoing Process
Investigations Serve an Important Function In Every Step of the
Process
Protect
Detect
Respond
Improve

8. www.caveon.com8
Before an Incident Occurs
• Know your test security processes and
procedures
• All programs have vulnerabilities. Know yours.
• Ensure all testing staff is thoroughly trained
• Identify all stakeholders in the testing process
• Create your Security Incident Response Plan

9. www.caveon.com9
What is an SIRP?
Security Incident Response Plan
• Dictates what to do when a testing irregularity
occurs
• Creates a consistent methodology for treating
infractions and breaches
• Establishes protocols for penalties imposed
when incidents occur
o Decision-making
o Escalation Path
o Communications

10. Security Incident Response Plan
www.caveon.com10
Key Components of an SIRP
• Agreements in place
• Policies in place
• Practices in place for analysis and monitoring
• Incident response matrix
• Investigative strategies
• Communications plan

11. www.caveon.com11
Key Functions
• Management
• Budgeting
• Analysis – Statistical analysis; monitoring of
the internet; etc.
• Training– Staff training and awareness
• Investigations– First line investigation
processes

12. Seven Step Incident
Investigation Process
www.caveon.com12

13. www.caveon.com13
Incident Investigation Process
• Identify and document triggering
incident
o Tip line
o Proctor reports
o Statistical analyses
o Media
o Web monitoring

14. www.caveon.com14
Incident Investigation Process
• Determine scope of potential harm
o What is the impact
o Who is involved
o Effect on test scores?

15. www.caveon.com15
Incident Investigation Process
• Complete initial incident report as
required by applicable policies and
procedures
o Leveraging SIRP
o Disseminate it appropriately
o Escalation and communication

16. www.caveon.com16
Incident Investigation Process
• Identify evidence available
o Varies by the nature of the incident
o May include myriad physical and electronic
documents and correspondences

17. www.caveon.com17
Incident Investigation Process
• Initial report/allegation
• Documentary evidence
• Forensic data analysis
• Comparison data
• Physical evidence
• Electronic evidence
• Witness statements

18. www.caveon.com18
Incident Investigation Process
• Instruct administrators, educators, students,
vendors and others to preserve relevant
evidence
• Know document/data retention policies
• Collect all relevant evidence
• Preserve chain of custody for all documents and
electronic evidence
• Review/analyze all evidence
• Conduct interviews after reviewing all relevant
evidence

19. www.caveon.com19
Incident Investigation Process
• Gather and preserve evidence
o Leverage case management software solutions
o Role based access
o Collaboration

20. www.caveon.com20
Incident Investigation Process
• Interviews and Witness statements
o Nature of incident will help identify witnesses
to interview
 Person who reported triggering incident
 Test participants
School Assessment Coordinator
Students
Test Administrator
Proctor
 Administrators
 Educators

21. www.caveon.com21
Incident Investigation Process
• Analyze evidence
o Decision-making criteria
o Outputs and reporting

22. www.caveon.com22
Incident Investigation Process
• Report findings in accordance with
applicable policies and procedures
o Consider the audience: courts, appeal boards,
board of education, licensure committee, etc.
o Objective, never subjective

23. Resources
www.caveon.com23
• Handbook of Test Security
• TILSA Guidebooks
• Caveon Webinar Series
• NCME Whitepapers
• Operational Best Practices
• Caveon District Investigations Training
Workshop
• www.cespcert.org

24. Key Points Summarized
www.caveon.com24
• SEAs & LEAs are operating in a crucible
• Test security incidents may impact the
validity of test scores
• An ounce of prevention is worth a pound of
cure
• Importance of both formal, approved SIRP
and investigation process

25. www.caveon.com25
Q&A

26. Thank You!
www.caveon.com26
Follow Caveon on twitter @caveon
Check out our blog
www.caveon.com/blog
LinkedIn Group “Caveon Test Security”
Steve Addicott
Steve.addicott@caveon.com
Marc Weinstein
Marc.weinstein@caveon.com