Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Establishing an Ethics
Framework for Predictive
Analytics in Higher Education
Cyber Summit 2016, Banff
Stephen Childs, Ins...
Disclamer
 The content of this presentation represents my views only.
and not that of my employer, the University of Calg...
Data Abundance in Higher Education
3
Big Data, Big Problems
 Advancing technology
—Better data collection
—Handle more data
—Apply algorithms to data
 We kno...
Solutions
 Develop an ethics framework around student data.
 Build on existing guidelines.
 Build on the norms of servi...
Outline
 Introduction
 Students and Student Data
 Predictive Analytics
 Existing Frameworks
 Next Steps
About Me
About My Office
 Office of Institutional Analysis
 https://oia.ucalgary.ca/
What OIA Does
About the University
Students
Student Data
 Application
 Student Information System
 LMS
 Unicard
 Surveys
 Residence
 Facilities
 Awarding Degr...
Student Data
 Students can opt out of some data collection, but not all
 Student give us their data because they trust u...
Privacy
Access to Data
Transparency and Accountability
 Internalize norms is not enough!
 How Universities use data should be known
—We aren’t ...
Consultation
Consider the Consequences
 Moving from institutional decision
making to acting on individual data
 Lathe of Heaven – a m...
Predictive Analytics
Best Practices using Predictive Analytics
 Have to carefully present information to students
—Present a positive outlook
...
Cathy O’Neil
 @mathbabe, mathbabe.org
 Mathematician, former hedge-fund
quant
Weapons of Math Destruction
 Three factors make a model a WMD:
—Is the participant aware of the model? Is the model
opaqu...
Student Data Principles
 http://studentdataprinciples.org/
 Purpose and use of student data
 Timely access to data
 Da...
Student Data Pledge
 http://www.edtechmagazine.com/k12/article/2015/03/prote
ct-personal-student-information-pair-organiz...
uCalgary Data Rules
 Freedom of Information and Privacy Act (1999)
—Students must be able to correct own info
—University...
Financial Modeler’s Manifesto
 https://www.wilmott.com/financial-modelers-manifesto/
 Emanuel Derman and Paul Wilmott – ...
Responsible Use of Student Data in Higher Education
 http://gsd.su.domains/
 Opportunity to understand student learning ...
Maciej Cegłowski
 https://pinboard.in and @pinboard
 http://idlewords.com/talks/
 Two talks on Data in particular:
—htt...
Basic Framework
 Safeguard Student Privacy
—Vendors; Monetizing Data
 Strong internal norms around data
 Consider and M...
Next Steps
 Write down your norms/expectations for working with
Student data
 Set up a discussion with your co-workers a...
Continue the Conversation
 Follow me on twitter: @sechilds
 Stephen.Childs@ucalgary.ca or sechilds@gmail.com
 https://o...
Upcoming SlideShare
Loading in …5
×

Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics in Higher Education

348 views

Published on

Stephen Childs was hired by the University of Calgary to develop an individual-level predictive model mapping students' decisions to attend the University. In his experience, the higher education sector was slow to use all the data it has available, but this is now changing.

As interest in making use of organizational data grows, staff must consider how these models will be used, and any problems that could arise. When individual predictions become the basis for decisions, how do we ensure our algorithms don't make existing problems worse? A framework for handling these issues now will let organizations handle these issues in a way that is consistent with their values.

Given the culture of today's institutions, and the success of predictive analytics in other fields, there is no doubt that these tools will be used. These techniques can improve student success and the competitiveness of educational organizations, but the benefits should not be gained at the expense of individuals within the system. This talk will propose a set of best practices for using institutional data for predictive modelling to address equity, privacy and other concerns. We must start thinking of this now, before other practices become entrenched.

Published in: Technology
  • Be the first to comment

Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics in Higher Education

  1. 1. Establishing an Ethics Framework for Predictive Analytics in Higher Education Cyber Summit 2016, Banff Stephen Childs, Institutional Analyst October 27, 2016
  2. 2. Disclamer  The content of this presentation represents my views only. and not that of my employer, the University of Calgary.  I am not qualified to accurately describe University of Calgary policy in the areas discussed in this talk.  Please contact the University if you have policy questions.
  3. 3. Data Abundance in Higher Education 3
  4. 4. Big Data, Big Problems  Advancing technology —Better data collection —Handle more data —Apply algorithms to data  We know more about our students  Can make predictions about their behavior  Very few guidelines about this practice
  5. 5. Solutions  Develop an ethics framework around student data.  Build on existing guidelines.  Build on the norms of service to students  Do this now while these practices are new.
  6. 6. Outline  Introduction  Students and Student Data  Predictive Analytics  Existing Frameworks  Next Steps
  7. 7. About Me
  8. 8. About My Office  Office of Institutional Analysis  https://oia.ucalgary.ca/
  9. 9. What OIA Does
  10. 10. About the University
  11. 11. Students
  12. 12. Student Data  Application  Student Information System  LMS  Unicard  Surveys  Residence  Facilities  Awarding Degrees  Grades  USRI  IT usage  Others…
  13. 13. Student Data  Students can opt out of some data collection, but not all  Student give us their data because they trust us  We need to deserve that trust! —Respect student privacy —Transparency about how data is used —Accountability —Consultation —Consider the Consequences
  14. 14. Privacy
  15. 15. Access to Data
  16. 16. Transparency and Accountability  Internalize norms is not enough!  How Universities use data should be known —We aren’t corporations with competitive secrets —We need to set up ways to report and share  We need to be able describe what happened!  Log events  Version control your software  Develop reporting methods
  17. 17. Consultation
  18. 18. Consider the Consequences  Moving from institutional decision making to acting on individual data  Lathe of Heaven – a mad social scientist
  19. 19. Predictive Analytics
  20. 20. Best Practices using Predictive Analytics  Have to carefully present information to students —Present a positive outlook —Don’t personalize it – talk about a group of similar students.  The factors in the model may be less deterministic than unobserved factors.  Difference between causality and correlation.  Beware the self-fulfilling prophecy
  21. 21. Cathy O’Neil  @mathbabe, mathbabe.org  Mathematician, former hedge-fund quant
  22. 22. Weapons of Math Destruction  Three factors make a model a WMD: —Is the participant aware of the model? Is the model opaque or invisible? —Does the model work against the participant’s interest? Is it unfair? Does it create feedback loops? —Can the model scale?
  23. 23. Student Data Principles  http://studentdataprinciples.org/  Purpose and use of student data  Timely access to data  Data should not replace professional judgement.  Data governance, security, breach notification
  24. 24. Student Data Pledge  http://www.edtechmagazine.com/k12/article/2015/03/prote ct-personal-student-information-pair-organizations- recommends-commitment  Don’t sell student data, use data to target ads, or profile students for non-educational purposes  Don’t collect more information or retain information longer than necessary.  Do disclose how, what and why
  25. 25. uCalgary Data Rules  Freedom of Information and Privacy Act (1999) —Students must be able to correct own info —University must provide own info upon confirmation of ID  Categories of Data Confidentiality  Research Ethics Boards —Data collection for University operations does not generally fall under REB jurisdiction.
  26. 26. Financial Modeler’s Manifesto  https://www.wilmott.com/financial-modelers-manifesto/  Emanuel Derman and Paul Wilmott – January 7, 2009  The Modelers’ Hippocratic Oath — I will remember that I didn’t make the world, and it doesn’t satisfy my equations. — Though I will use models boldly to estimate value, I will not be overly impressed by mathematics. — I will never sacrifice reality for elegance without explaining why I have done so. — Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights. — I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.
  27. 27. Responsible Use of Student Data in Higher Education  http://gsd.su.domains/  Opportunity to understand student learning and enhance educational attainment.  New questions about the ethical collection, use, and sharing of information.  Commitments to honor the integrity, discretion, and humanity of students.  Improve practice in light of accumulating information and knowledge.
  28. 28. Maciej Cegłowski  https://pinboard.in and @pinboard  http://idlewords.com/talks/  Two talks on Data in particular: —http://idlewords.com/talks/deep_fried_data.htm —http://idlewords.com/talks/haunted_by_data.htm
  29. 29. Basic Framework  Safeguard Student Privacy —Vendors; Monetizing Data  Strong internal norms around data  Consider and Measure Outcomes  Work with Data Owners and Stewards  Responsibility to Educate  Consult with Students and Stakeholders  Data should have a clear purpose
  30. 30. Next Steps  Write down your norms/expectations for working with Student data  Set up a discussion with your co-workers about it.  Seek out others who perform a similar role and discuss it.  Discuss with the Student Data Steward at your institution.  Send me your comments!
  31. 31. Continue the Conversation  Follow me on twitter: @sechilds  Stephen.Childs@ucalgary.ca or sechilds@gmail.com  https://oia.ucalgary.ca/Contact  https://www.meetup.com/PyData-Calgary/

×