Your SlideShare is downloading. ×
Caveon Webinar - "Preventing & Detecting Cheating On High Stakes Tests"
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Caveon Webinar - "Preventing & Detecting Cheating On High Stakes Tests"

340
views

Published on

Published in: Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
340
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Fix size of text to fit on slide.
  • Insert Poll Question #2 Here! For the last three years has your testing program conducted cheating detection analyses (data forensics)?Yes, for every administrationYes, but only occasionallyNo
  • Insert Poll Question #1 Here! Has there been media coverage of a cheating incident within your testing program in the last three years? Yes/No
  • Increase font size- skyler
  • Transcript

    • 1. Hashtag #HStakes Caveon Webinar Series “Preventing and Detecting Cheating on High Stakes Tests”Presented by:John Fremer, President, Caveon Consulting ServicesVickie Cartwright, Sr. Director, Accountability, Research, and Assessment, Orange County School District
    • 2. Presentation Overview Overall Summary External Assistance  Training  Security Audit  Data Forensics  Investigations Dealing with Media
    • 3. Overall Summary Provide Appropriate Training Evaluate the Adequacy of Your Security Procedures Perform Cheating Analyses Routinely Follow Up Thoroughly on All Suspicions Take Appropriate and Public Action
    • 4. Providing Appropriate Training Focus on Test Administration  Greatest vulnerability for all programs  Biggest “bang for the buck” Build on State and Vendor Materials Make Sure Everyone is Trained Have Building Administrators Participate
    • 5. Issues for which ExternalAssistance is Recommended External Test Security Audit Data Forensics Web Patrol (Monitoring Internet) Security Investigations
    • 6. Test Security Audit: The Importance of Knowing Validates your Security Practices Identifies Vulnerabilities Recommends and Prioritizes Actions to Correct Shows Your Seriousness about Security
    • 7. Test Security Audit: What is it? Systematic Review by Experienced Professionals Objective Third Party Evaluation Use Formal Standards Seek Practical and Specific Advice
    • 8. Need External Security Auditors Do Your Own Security Reviews Regularly? Have Such Reviews as an Assignment for a Key Staff Member Can’t do a Full Security Review Internally  It’s like evaluating your own children  Are they more beautiful or talented or smart?  Need objective and well-informed perspective
    • 9. Caveon Test Security Audit Topics1. Security Plan 8. Test Administration2. Roles & Responsibilities 9. Test Scores & Results3. Budget & Funding 10. Physical security4. Legal Precautions & 11. Information SecurityAgreements 12. Web & Media Monitoring5. Test & Item Design 13. Security Awareness &6. Test & Item Development & TrainingMaintenance 14. Security Incidence7. Test Publication Response Plan 15. Conducting Security Investigations [New]
    • 10. Questions?
    • 11. Psychometric Analyses -Detecting CheatingData Forensics Looking at Your Test Data Has a Test been Compromised? When and Where was it Compromised?“Traditional” Procedures No Longer Good Enough
    • 12. Data Forensics – School Districts Type of Data Available to Districts Usually Limited  Gains Analyses  Erasure Analyses in Paper and Pencil - by Inspection  Not practical except when there is a tip or something observed More Sophisticated Analyses Require More Data and are Very Desirable
    • 13. Data Forensics Benefits Use Data to make Informed Decisions Train from Known Threats Detect and Deter Cheaters Stay Ahead of Cheating Methods Evaluate Test Admin Security Procedures Evidence of Strong Action – Bringing in Independent Experts
    • 14. Other Data Forensics Analyses Similarity of Responses for Pairs and Groups of Students Aberrance – Failure to Have a Logical Pattern, e.g., Answering More Hard Items Correctly than Easy Ones Complete Erasure or Answer Changing Analyses – What % Wrong to Right
    • 15. Success Story - State Assessment Reduction in "Irregularities" 72% reduction – 2 years 6.0% Test admin monitoring 5.0% Conditional status 4.0% Districts follow through 3.0% “Students confess” 2.0% Coordinator training 1.0% 0.0% Report 1 Report 2 Report 3 Report 4 Report 5 Report 6 Report 7 Report 8 Report 9
    • 16. Web Patrol Are “Secure” Items on the Internet? Occurs in High Stakes Testing Long Testing Windows Tend to Lead to “Leakage” (Here comes the consortia) Involves Searching Tools, Expert Sleuthing, and Quick Follow-Up Very Common with Certification Programs
    • 17. Any Questions?
    • 18. Atlanta Public Schools Story – Persistent Accusations of Cheating in APS Schools Consistent Denials  Reports are wrong  Newspaper is prejudiced Data Revealed Unmistakable Evidence of Improprieties  Extreme cases very pronounced – one over 10 to 52nd power
    • 19. Media Lessons from High Profile Incidents  Expect Media Attention  Get Out Front with Your Own Story  Cultivate Local Media  Most do not understand test results  Looking for interesting story – a no problem administration is not a good story  Learn Who You Can Trust  Don’t Expect a Cheating Story to Go Away – You Cannot Control It
    • 20. FURTHER INFORMATION Slides & recording available at: www.caveon.com/resources/webinars Caveon Security Insights Blog  http://www.caveon.com/blog/ Twitter  @Caveon LinkedIn  Caveon Company Page  Caveon Test Security Group  Please contribute!

    ×