Caveon Webinar Series - Lessons Learned at the 2015 National Conference on Student Assessment - July 2015
•Download as PPSX, PDF•
1 like•286 views
The National Conference on Student Assessment (NCSA) was held last month in San Diego, and Caveon was there. This month's webinar will focus on lessons learned at the conference regarding test security, and what's happening in the state assessment arena in terms of test security right now. Caveon's Steve Addicott and Jamie Mulkey will be joined by special guest Walt Drane, State Assessment Director, Mississippi Department of Education. The panelists will summarize the test security trends and strategies that they drew from the conference, and also share key points from sessions they presented.
Recommended
Recommended
More Related Content
What's hot
What's hot (13)
Viewers also liked
Viewers also liked (13)
Similar to Caveon Webinar Series - Lessons Learned at the 2015 National Conference on Student Assessment - July 2015
Similar to Caveon Webinar Series - Lessons Learned at the 2015 National Conference on Student Assessment - July 2015 (20)
More from Caveon Test Security
More from Caveon Test Security (20)
Recently uploaded
Recently uploaded (20)
Caveon Webinar Series - Lessons Learned at the 2015 National Conference on Student Assessment - July 2015
- 1. Caveon Webinar Series www.caveon.com 1 July 22, 2015 Walt Drane, MS DOE Jamie Mulkey, Caveon Test Security Steve Addicott, Caveon Test Security Lessons Learned at the National Conference on Student Assessment
- 2. Agenda for Today • Important themes • Keynote • Security sessions – Walt – Jamie • New Resources • Q&A www.caveon.com 2
- 3. State Assessment Director Breakfast • Wow! • State Assessments = Politics • Common Core or No Common Sense? • Exodus from Consortia www.caveon.com 3
- 4. Impact of Technology on Security • Devices • Technology Enhanced Items • Automated Scoring • Computer Adaptive Testing • Others? www.caveon.com 4
- 5. Keynote – Dr. Andreas SchleicherHow the U.S. and Other Countries Compare: Lessons Learned from International Education Data What is the OECD? Thirty-four democracies work together promoting economic growth, prosperity, & sustainable development What is PISA? Test that evaluates 15-year-old students in reading, mathematics, and science, worldwide www.caveon.com 5
- 6. Keynote Key Messages • “Math skills” vs math skills • Problem solving skills = Career success • Great teachers require development www.caveon.com 6
- 7. Developing Teaching as a profession Recruit top candidates into the profession Support teachers in continued development of practice Retain and recognise effective teachers – path for growth Improve the societal view of teaching as a profession Mean mathematics performance, by school location, after acc ounting for socio-economic status77 Capacity at the point of delivery
- 8. Test Security Sessions… www.caveon.com 8 We noticed….there weren’t many!!
- 9. www.caveon.com 9 Essential Tips for Leveraging Your Vendor’s Expertise to Strengthen the Security of Statewide Assessments
- 10. State Assessment Needs • Valid and reliable score results • If results aren’t valid, then actions may be taken – or not taken – that actually harm rather than help www.caveon.com 10
- 11. Holistic Test Security Framework www.caveon.com 11 Dete r DetectDecide (now and for the future)
- 12. Vendor’s Role in DETERRING Negative Activiti www.caveon.com 12 Security Audits • Use the Vendor to “Audit” the Security Audit Process • Thinking Outside the Box Support/Consulting on policies and procedures • Use vendor to “audit” current policies and procedures • Student Assessment Handbook, Appendix F, and Legislation Invalidations as a deterrent • Use the Vendor during the appeals process
- 13. Vendor’s Role in DETECTING? www.caveon.com 13 Data Forensics • Systems that utilize sophisticated statistical analyses of test-response data to identify patterns indicative of test fraud (e.g. cheating and piracy). • Erasure Analysis for paper and pencil assessments • Analysis of test responses of on-line Assessments
- 14. State’s Expectation of Vendor When DECIDING To inform our decisions, we expect from our vendors • Statistical analyses for invalidations oConsistently use the data you have oPlan for future challenges (more than data) • Information to support go/no go decisions for investigations oPlay to win - don't pick a fight you are not reasonably sure you can win • Deep-dive data analysis and interpretation www.caveon.com 14
- 15. www.caveon.com 15 Test Security’s Changing Landscape in Computerized Testing Environments
- 16. Security in Computerized Assessments Why should we still be concerned about test security with the move to computerized assessments? www.caveon.com 16
- 17. When Threats Occur Before During After Unauthorized access Computer hacking Keystroke logging Storing outside delivery system Testing interruptions Testing window Memorable items Student workstations Electronic devices Social media
- 18. Policies and Procedures – How do state assessment programs implement computerized assessments? What policies, procedures, and practices have they put in place? – Focus on Exam Administration: • Assessment monitoring • Proctor training • PITA Hotline • Online training modules (25% of which deal with test security) • Resources for staff/teachers/coordinators www.caveon.com 18
- 19. Test Security Investigations How can you conduct legally defensible investigations that deliver appropriate consequences? • Having the right documentation in place • Collecting all related information • Conducting interviews as needed • What are some of the consequences www.caveon.com 19
- 20. New Resources Made Available at NCSA www.caveon.com 20
- 21. New Resources Available www.caveon.com 21 http://bit.ly/1CHUvsi
- 22. New Resources Available www.caveon.com 22 http://bit.ly/1J71f8q
- 23. New Resources Available www.caveon.com 23 http://bit.ly/1VlyARX
- 24. New Resources Available www.caveon.com 24 http://bit.ly/1RBCKWm
- 25. Thank You! www.caveon.com 25 Walt Drane State Assessment Director, Mississippi DOE WDrane@mde.k12.ms.us Steve Addicott Vice President, Caveon Steve.addicott@caveon.com Follow Caveon on twitter @caveon Check out our blog www.caveon.com/blog LinkedIn Group “Caveon Test Security” Jamie Mulkey Vice President, Caveon Jamie.mulkey@caveon.com
Editor's Notes
- Always a highlight for me. 40 or SADs at a round table…rectangle table..discussing the state of their state. This year? Wow! Director after director describing very challenging situations. New test programs with no $$, super tight timelines—test programs implemented in weeks, not years. Legislatures driving assessments like never before— Opt-out, privacy legislators clearly focused on re-election, not Common core“our standards will not contain any common core standards!” creating state standards without DOE involvement Walt—Education and Assessments are a profession—when public loses confidence, legislature must step in.
- Many sessions on this…consider covering this quickly…
- Andreas Schleicher leads work on education and skills at the Organisation for Economic Co-operation and Development (OECD), including the Programme for International Student Assessment (PISA). PISA evaluates education systems worldwide by testing the skills and knowledge of 15-year-old students in reading, mathematics, and science. The Organization for Economic Cooperation and Development (OECD) is a unique forum where the governments of 34 democracies with market economies work with each other, as well as with more than 70 non-member economies to promote economic growth, prosperity, and sustainable development. Why should we care about problem solving skills? Learning Math doesn’t always equal learning Math Gender differences in Math What are the lessons from high performers? What can we do to help capacity at the point of delivery?
- Andreas Schleicher leads work on education and skills at the Organisation for Economic Co-operation and Development (OECD), including the Programme for International Student Assessment (PISA). PISA evaluates education systems worldwide by testing the skills and knowledge of 15-year-old students in reading, mathematics, and science. The Organization for Economic Cooperation and Development (OECD) is a unique forum where the governments of 34 democracies with market economies work with each other, as well as with more than 70 non-member economies to promote economic growth, prosperity, and sustainable development. Why should we care about problem solving skills? Learning Math doesn’t always equal learning Math Gender differences in Math What are the lessons from high performers? What can we do to help capacity at the point of delivery?
- The other great thing about this session were our two presenters from state assessment departments. One had been delivering computer-delivered assessments for awhile, while the other had more recent experiences. Potential Inappropriate Testing Activity - PITA
- The last section of the presentation focused on conducting legally defensible investigations when a security incident does occur.