Download to read offline
More Related Content
Similar to Confronting Data Manipulation
Confronting Data Manipulation
- 1. Confronting Data Manipulation onYour Campus Dr. Will Miller Director of Institutional Research and Effectiveness Flagler College
- 3. Context • We relyon others for data • IPEDS • Outcomes reports • Closing the Loop • Course evaluations • Technology has eliminated the paper trail
- 4. Data Manipulation • Comesin a variety of flavors • Different than data error! • Survey responses • Course evaluations • Course outcomes • Closing the Loop reports • Or more troubling forms
- 5. Flagler College • Foundedin 1968 • Baccalaureate only • Approximately 2700 students • U.S. News Top Regional • The unicorn of admissions • Increase applicant pool • Increase cohort size • Decrease merit scholarships • Increase class profile
- 6.
- 7. The Swift Fallof the Unicorn • Background of the offender • New Director of IRE and New Director of College Writing in Summer 2013 • During a random audit of student placements from fall semester, we notice a vast majority of students were under-placed in writing courses according to information in Student Management System • Initially blame advisors • But, test scores in the system do not match hardcopy test scores in student files • The two cases that sealed his fate
- 8. Internal Process • Notificationof VPAA (February 7) • More analysis (Weekend of February 8-9) • Audit of data entry changes shows one user affiliated with all changes on weekends in September for numerous years (beginning in 2004) • Update of VPAA (February 9 around 2am) • Notification of President (February 10 at 8am) • More analysis • Final notification of President (February 12 at 8am) • Trip with VPEM (February 13) • Removal (February 14) • Measure Impact (Weekend of February 15-16) • Notify Cabinet on February 17 at 8am
- 9.
- 10.
- 11. Contributing Factors • Relationships •Whistle blower • Access to databases • Test score entry • Policies on paper v. policies in practice • Silos
- 12. The Public Process •External investigators • External data auditors • Crisis communication consultants • Openness and transparency • Stakeholders • Campus community
- 13.
- 14.
- 15. Reporting and Response •Open and transparent • Easier to tell all at the beginning • No sugar coating • Responses from community • Responses from stakeholders • Accrediting bodies • Government • Publications
- 16. Impact • Faculty • Students •Staff • Institution as a whole
- 17. And the OtherShoe Falls • Students aren’t impacted…if they take math and English first semester • Which some students don’t • 223 students end up misplaced in math, English, or in a few cases both due to test score fraud • A whole new process • Pull transcripts, courses taken, time of registration v. time of data manipulation, determine remediation while consulting with faculty, presenting remediation, and in some cases scrub transcripts • Interests DoE
- 18. Lessons Learned Any pressdoes not necessarily make for good press Be open and transparent Be selective in choosing investigators Do not worry about cost…you’ll be remembered by how you respond
- 19. Lessons Learned Do notsugarcoat Policies exist to be followed Do not hold press conferences Some people don’t need access to some things
- 20. Lessons Learned Do notblindly trust
- 21. Lessons Learned It canbe a positive impetus for change • We’re actually a unicorn • Decreased applications, increased yield, made the class (and then some), and increased profile • Re-established relationships between enrollment management and faculty • Personnel changes
- 22.