Successfully reported this slideshow.
Your SlideShare is downloading. ×

Zombie categories, broken data and biased algorithms: What else can go wrong? Ethics in the collection, analysis and use of student data

Ad

Imagecredit:https://pixabay.com/en/art-sculpture-scrap-sculpture-human-1699977/
Zombie categories, broken data and biased
...

Ad

Imagecredit:https://pixabay.com/en/travel-sculpture-man-stone-move-3034459/
• Re-assess/redefine/destabilise our categorie...

Ad

Under what conditions can the
collection, analysis and use of
student data be just and ethical?

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Check these out next

1 of 67 Ad
1 of 67 Ad

Zombie categories, broken data and biased algorithms: What else can go wrong? Ethics in the collection, analysis and use of student data

Download to read offline

Keynote: Datafication of Higher Education: Considering an ethical approach to enhancing the student experience
28 August, University of Stirling

Keynote: Datafication of Higher Education: Considering an ethical approach to enhancing the student experience
28 August, University of Stirling

Advertisement
Advertisement

More Related Content

Similar to Zombie categories, broken data and biased algorithms: What else can go wrong? Ethics in the collection, analysis and use of student data (20)

More from University of South Africa (Unisa) (20)

Advertisement

Zombie categories, broken data and biased algorithms: What else can go wrong? Ethics in the collection, analysis and use of student data

  1. 1. Imagecredit:https://pixabay.com/en/art-sculpture-scrap-sculpture-human-1699977/ Zombie categories, broken data and biased algorithms: What else can go wrong? Ethics in the collection, analysis and use of student data Paul Prinsloo University of South Africa (Unisa) @14prinsp Keynote: Datafication of Higher Education: Considering an ethical approach to enhancing the student experience 28 August, 2018 University of Stirling)
  2. 2. Imagecredit:https://pixabay.com/en/travel-sculpture-man-stone-move-3034459/ • Re-assess/redefine/destabilise our categories of analysis • Acknowledge the brokenness of data and epistemic arrogance • Carefully and ethically optimise the potential of algorithmic decision-making systems We also have to consider what can go right when we…
  3. 3. Under what conditions can the collection, analysis and use of student data be just and ethical?
  4. 4. Acknowledgement I do not own the copyright of any of the images in this presentation. I therefore acknowledge the original copyright and licensing regime of every image used. This presentation (excluding the images) is licensed under a Creative Commons Attribution 4.0 International License.
  5. 5. Overview of presentation • The data revolution and how it unfolds/impacts on our lives • The data revolution in higher education – the collection, analysis and use of student data • (Broken) data • (Zombie) categories • (Biased) algorithms • Data justice • The ethical collection, analysis and use of student data • Some guiding principles • (In)conclusions
  6. 6. The data revolution • More data (volume) • Greater variety of data from a range of sources • Real-time collection, analysis and use (velocity) • Exhaustive (n=all) • “Fine-grained in resolution and uniquely indexical in identification” • Relational • “flexible, holding the traits of extensionality (can add new fields easily) and scaleability (can expand in size rapidly)” Kitchin, R., & Lauriault, T. (2014). Towards critical data studies: Charting and unpacking data assemblages and their work. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2474112
  7. 7. Image credit: https://pixabay.com/en/puzzle-jigsaw-jigsaw-puzzle-1487340/ Data assemblages arise from a variety of intersecting and often mutually constitutive technological, political, social and economic processes and apparatuses that frame the nature, operation and work of these assemblages
  8. 8. Image credit: https://pixabay.com/en/puzzle-cooperation-together-1020002/ Central to these assemblages are, inter alia, (broken) data, (zombie) categories and (biased) algorithms
  9. 9. Source credit: http://uk.businessinsider.com/how-to-see-everything-google- knows-about-you-2016-6?r=US&IR=T Source credit: https://medium.com/productivity-in-the-cloud/6-links-that-will-show-you-what-google-knows-about-you- f39b8af9decc
  10. 10. Source credit: https://www.theguardian.com/news/2018/mar/17/cambridge-analytica-facebook-influence-us- election “We exploited Facebook to harvest millions of people’s profiles. And built models to exploit what we knew about them and target their inner demons. That was the basis the entire company was built on.”
  11. 11. Source credit: https://www.washingtonpost.com/blogs/post-partisan/wp/2018/07/17/how-your-data-is- used-by-police-and-where-it-goes-wrong/?noredirect=on&utm_term=.61ec3d7e7206
  12. 12. “Algorithms that may conceal hidden biases are already routinely used to make vital financial and legal decisions. Proprietary algorithms are used to decide, for instance, who gets a job interview, who gets granted parole, and who gets a loan.” Source credit: https://www.technologyreview.com/s/608248/biased-algorithms-are-everywhere-and-no-one-seems-to-care/
  13. 13. The world of (student) data Academic Analytics Learning Analytics (Higher) Education • Individuals • Corporates • Governments • Data brokers • Fusion centers • Directed • Automated • Gifted
  14. 14. Source credit: https://tekri.athabascau.ca/analytics/ “Learning institutions and corporations make little use of the data learners ‘throw off’ in the process of accessing learning materials, interacting with educators and peers, and creating new content. In an age where educational institutions are under growing pressure to reduce costs and increase efficiency, analytics promises to be an important lens through which to view and plan for change at course and institution levels” (emphasis added).
  15. 15. Source credit: https://tekri.athabascau.ca/analytics/ “Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.”
  16. 16. Source credit: http://timoelliott.com/blog/2013/02/gartnerbi-emea-2013-part-1-analytics-moves-to-the-core.html Learning analytics in action
  17. 17. Source credit: https://www.theguardian.com/higher-education-network/2016/aug/03/learning-analytics-universities-data-track-students 2016
  18. 18. Page credit: http://www.npr.org/sections/ed/2017/01/11/506361845/the-higher-ed-learning-revolution-tracking-each-students-every-move 2017
  19. 19. Source credit: https://mobile.edweek.org/c.jsp?cid=25919761&bcid=25919761&rssid=25919751&item=http%3a%2f%2fapi.edweek.org%2fv1 %2few%2f%3fuuid%3dC08929D8-6E6F-11E8-BE8B-7F0EB4743667 2018
  20. 20. 2018
  21. 21. Source credit: https://campustechnology.com/articles/2018/05/02/when-learning-analytics-violate-student-privacy.aspx 2018
  22. 22. Source credit: https://er.educause.edu/articles/2018/5/setting-the- table-responsible-use-of-student-data-in-higher- education 2018
  23. 23. Kitto, K., Shum, S. B., & Gibson, A. (2018, March). Embracing imperfection in learning analytics. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (pp. 451-460). ACM.
  24. 24. Pink, S., Ruckenstein, M., Willim, R., & Duque, M. (2018). Broken data: Conceptualising data in an emerging world. Big Data & Society, 5(1), 2053951717753228. We have to recognize that our data are broken
  25. 25. “Data is not necessarily accurate, complete or full aggregated representations of what individuals or societal groups have done, or able to predict what they will do” (p. 10) Pink, S., Ruckenstein, M., Willim, R., & Duque, M. (2018). Broken data: Conceptualising data in an emerging world. Big Data & Society, 5(1), 2053951717753228.
  26. 26. Image credit: https://www.flickr.com/photos/itsnitram/32843850295
  27. 27. Image credit: https://pixabay.com/en/zombie-horror-creepy-night-531116/ Zombie Categories “categories from the past that we continue to use even though they have outlived their usefulness and even though they mask a different reality” (Plummer, 2011, p. 195)
  28. 28. Beck, U. (2001). Interview with Ulrich Beck. Journal of Consumer Culture, 1(2), 261-277. Zombie categories are ‘living dead’ categories which govern our thinking but are not really able to capture the contemporary milieu.
  29. 29. “…what is a ‘household’ nowadays? […] it is quite difficult to figure out how to define the contemporary household. […].. So producing data about consumption or voting behaviour on the basis of ‘households’ is part of a zombie sociology. They just don’t exist anymore.” Beck, U. (2001). Interview with Ulrich Beck. Journal of Consumer Culture, 1(2), 261-277.
  30. 30. (a) Indicators name things “To claim ‘naming rights’, indigenous peoples need to replace indicators that have been constructed according to hegemonic categories and motivated by Global North normative assumptions with indicators that reflect their own local understandings of their social world” (Morphy, 2016, p. 104) Morphy, F.(2016). Research as solidarity. In T. Kukutai and J. Taylor. (Eds), Indigenous data sovereignty. Toward an agenda (pp. 99-116). Canberra, Australia: Australian National University Press. Retrieved from https://press.anu.edu.au/publications/series/centre-aboriginal-economic-policy-research-caepr/indigenous-data- sovereignty
  31. 31. (b) Indicators compare and rank “Encapsulated indigenous minorities within settler states constantly find themselves being compared, as a ‘population’, with the ‘mainstream population’—and found wanting. […] They have ‘gaps’ that need to be ‘closed’, and improvement is defined in terms of the indicators that measure the gaps” (Morphy, 2016, p. 105) Morphy, F.(2016). Research as solidarity. In T. Kukutai and J. Taylor. (Eds), Indigenous data sovereignty. Toward an agenda (pp. 99-116). Canberra, Australia: Australian National University Press. Retrieved from https://press.anu.edu.au/publications/series/centre-aboriginal-economic-policy-research-caepr/indigenous-data- sovereignty
  32. 32. (c) Indicators simplify complex phenomena “…categorisation is used as a tool of simplification with respect to complex phenomena such as the ‘family’ and the ‘household’” (Morphy, 2016; p. 106). Morphy, F.(2016). Research as solidarity. In T. Kukutai and J. Taylor. (Eds), Indigenous data sovereignty. Toward an agenda (pp. 99-116). Canberra, Australia: Australian National University Press. Retrieved from https://press.anu.edu.au/publications/series/centre-aboriginal-economic-policy-research-caepr/indigenous-data- sovereignty
  33. 33. To what extent do our categories of analysis dismember and dislocate our students, alienate them from what they know and how they see the world and their learning journeys?
  34. 34. Gullion (2018) does not propose that we stop using categories, but we need to recognise the ontologies and epistemologies that informed our categories of analysis and the effects of their normalisation Image credit: Amazon
  35. 35. Source credit: https://www.technologyreview.com/s/610026/algorithms-are-making-american-inequality-worse/
  36. 36. Image credits: Amazon
  37. 37. Image credit: https://www.flickr.com/photos/donpezzano/4903384635 What happens when data assemblages depend on broken data, zombie categories and biased algorithms?
  38. 38. Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The Current Landscape of Learning Analytics in Higher Education. Computers in Human Behavior. 2018
  39. 39. Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The Current Landscape of Learning Analytics in Higher Education. Computers in Human Behavior. Four propositions of learning analytics (Ferguson & Clow, 2017) 1)improve learning outcomes 2)support learning and teaching? 3) deployed widely; and 4) used ethically? To what extent does learning analytics To what extent is learning analytics
  40. 40. Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The Current Landscape of Learning Analytics in Higher Education. Computers in Human Behavior. “It is worrying that more than 80% of the papers do not mention ethics at all. Moreover, there are only few studies that approach ethical issues (e.g., data privacy and security, informed consent) in a systematic way. However, we should not jump to the conclusion that most studies are done in an unethical way, but we call for more explicit reflection on ethics to rise in the coming years. The increase of the studies that reflect on the ethical issues for the year 2017 (36%) might indicate that there is already a positive move in this direction” (emphasis added).
  41. 41. Source credit: https://www.technologyreview.com/s/526401/laws-and-ethics-cant-keep-pace-with-technology/
  42. 42. Heeks, R., & Renken, J. (2018). Data justice for development: What would it mean?. Information Development, 34(1), 90-102.
  43. 43. Data justice 1. Instrumental data justice – fair use – will the outcome of the use of data be fair? 2. Procedural data justice – the handling of data – the sampling, the ‘cleaning’, the analysis, the storage and governance of data. Individuals feel that a process is fair if they are in control of the process. Issues surrounding consent – opting in/out. Individual control/input ito consistency, correctness and correctability 3. Distributive data justice – who has access to what data – issues surrounding privacy 4. Agentic justice – “the personal choice to exercise rights-as- capabilities and convert them into rights-as-achieved functionings” 5. Structural justice – based on the above, what frameworks/ structures/laws will support, enable, enact Heeks, R., & Renken, J. (2018). Data justice for development: What would it mean?. Information Development, 34(1), 90-102.
  44. 44. “Ethics are the mirror in which we evaluate ourselves and hold ourselves accountable” (emphasis added). Holding actors and humans accountable still works “better than every single other system ever tried” (Brin, 2016) Ethics and accountability
  45. 45. Ethics in learning analytics: Selected examples 2013-2017
  46. 46. Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510-1529. 2013
  47. 47. Source credit: https://www.open.ac.uk/students/charter/sites/www.open.ac.uk.students.charter/files/files/ecms/web-content/ethical- use-of-student-data-policy.pdf 2014
  48. 48. Source credit: https://www.open.ac.uk/students/charter/sites/www.open.ac.uk.students.charter/files/files/ecms/web-content/ethical- use-of-student-data-policy.pdf Principle 1: Learning analytics is an ethical practice that should align with core organisational principles, such as open entry to undergraduate level study. Principle 2: The OU has a responsibility to all stakeholders to use and extract meaning from student data for the benefit of students where feasible. Principle 3: Students should not be wholly defined by their visible data or our interpretation of that data. Principle 4: The purpose and the boundaries regarding the use of learning analytics should be well defined and visible.
  49. 49. Source credit: https://www.open.ac.uk/students/charter/sites/www.open.ac.uk.students.charter/files/files/ecms/web-content/ethical- use-of-student-data-policy.pdf Principle 5: The University is transparent regarding data collection, and will provide students with the opportunity to update their own data and consent agreements at regular intervals. Principle 6: Students should be engaged as active agents in the implementation of learning analytics (e.g. informed consent, personalised learning paths, interventions). Principle 7: Modelling and interventions based on analysis of data should be sound and free from bias. Principle 8: Adoption of learning analytics within the OU requires broad acceptance of the values and benefits (organisational culture) and the development of appropriate skills across the organisation.
  50. 50. Willis, J. E., Slade, S., & Prinsloo, P. (2016). Ethical oversight of student data in learning analytics: A typology derived from a cross-continental, cross-institutional perspective. Educational Technology Research and Development, 64, 881-901. DOI: 10.1007/s11423-016-9463-4 http://link.springer.com/article/10.1007/s11423- 016-9463-4 Who will provide oversight over the ethical issues in learning analytics? An interpretative multiple-case study: Indiana University, Open University (UK) and the University of South Africa (Unisa) 2016
  51. 51. (1) Humans perform the task (2) Task is shared with algorithms (3) Algorithms perform task: human supervision (4) Algorithms perform task: no human input Seeing Yes or No? Yes or No? Yes or No? Yes or No? Processing Yes or No? Yes or No? Yes or No? Yes or No? Acting Yes or No? Yes or No? Yes or No? Yes or No? Learning Yes or No? Yes or No? Yes or No? Yes or No? Danaher, J. (2015). How might algorithms rule our lives? Mapping the logical space of algocracy. [Web log post]. Retrieved from http://philosophicaldisquisitions.blogspot.com/2015/06/how-might-algorithms-rule-our- lives.html Human-algorithm interaction in the collection, analysis and use of student data: What are the (ethical) issues?
  52. 52. Source credit: https://www.siyaphumelela.org.za/documents/5a61c7b737ff5.pdf
  53. 53. Imagecredit:https://pixabay.com/en/binary-code-man-display-dummy-face-1327512/ Guiding principles for an ethics of care: Principle 1: The moral, relational duty of learning analytics Principle 2: Defining student success in the nexus of student, institution and macro-societal agencies and context Principle 3: Understanding data as framed and framing Principle 4: Student data sovereignty Principle 5: Accountability Principle 6: Transparency Principle 7: Co-responsibility
  54. 54. “If you have come to help us, you can go home. If you have come to accompany us, please come. We can talk” Glesne, C. (2016). Research as solidarity. In T. Kukutai and J. Taylor. (Eds), Indigenous data sovereignty. Toward an agenda (pp. 169-178). Canberra, Australia: Australian National University Press. Retrieved from https://press.anu.edu.au/publications/series/centre-aboriginal-economic-policy- research-caepr/indigenous-data-sovereignty Principle 1: The moral, relational duty of learning analytics Image credit: https://pixabay.com/en/sculpture-bronze-2196139/
  55. 55. Principle 2: Student success as entangled Image credit: https://pixabay.com/en/rope-knot-string-strength-cordage-3052477/
  56. 56. THE STUDENT AS AGENT IDENTITY, ATTRIBUTES, HABITUS Success THE INSTITUTION AS AGENT IDENTITY, ATTRIBUTES, HABITUS SHAPING CONDITIONS: (predictable as well as uncertain) SHAPING CONDITIONS: (predictable as well as uncertain) THE STUDENT WALK Multiple, mutually constitutive interactions between student, institution & networks FIT FIT F I T F I T F I T F I T F I T F I T F I T F I T Subotzky, G., & Prinsloo, P. (2011). Turning the tide: a socio-critical model and framework for improving student success in open distance learning at the University of South Africa. Distance Education, 32(2): 177-19.
  57. 57. Principle 3: Understanding data as broken and framed but also as acting and framing Image credit: https://pixabay.com/en/eyeglasses-broken-glasses-sight-366446/
  58. 58. Student data are not something separate from students’ identities, their histories, their beings. Data are an integral, albeit informational part of students being. In the light of the view that data are not something students own but rather who they are; what are we assuming when we say we ‘collect’ their data? E.g. Floridi, L. (2005). The ontological interpretation of informational privacy. Ethics and Information Technology, 7(4), 185-200. Principle 4: Student data sovereignty
  59. 59. Ctrl Alt Del • How much control do students have to determine what data institutions harvest; to challenge the meaning of their data and our categories of analysis? • Can students (re)define/alter interpretations of data and definitions/categories? Can they offer us counter- narratives to our understanding of their learning and their life-worlds? • Can students opt out of personalised data collection, analysis and use and have their data deleted?
  60. 60. Willis, J. E., Slade, S., & Prinsloo, P. (2016). Ethical oversight of student data in learning analytics: A typology derived from a cross- continental, cross-institutional perspective. Educational Technology Research and Development, 64, 881-901. DOI: 10.1007/s11423- 016-9463-4 http://link.springer.com/article/10.1007/s11423-016-9463-4 Principle 5: Accountability An interpretative multiple-case study: Indiana University, Open University (UK) and the University of South Africa (Unisa)
  61. 61. Principle 6: Transparency If they don’t know that we collect their data, the scope and purpose of the collection, how we will use their data and how it will impact on their learning journeys, how is this ethical? Image credit: https://pixabay.com/en/sculpture-bronze-the-listening-2209152/
  62. 62. Principle 7: Co-responsibility Our students’ journeys are intimately weaved into our (institutional) stories. What happens in their lives, impact ours. And vice versa. Image credit: https://pixabay.com/en/weave-hand-labor-samoa-exotic-55/
  63. 63. Student data are an invitation to start a conversation
  64. 64. You call me a misfit, a risk, a dropout and stop-out Your research indicates that ‘students like me’ may not make it You ask me questions regarding my financial status, where I live, how many dependents I have, and I know that once I tell you, I will become a number on a spreadsheet I will be color-coded I will become part of a structural equation model that re-affirms that People like me Don’t belong here Somehow I don’t fit in you spreadsheet But I want you to know that I am so much more I am so much more than how you define me I am so much more than my home address (the one I lied about to get access to funding or to get a place in residence) I am also a brother, a sister, a mother, a dependent, a carer I don’t fit in your spreadsheets I am not a dropout, I am a refugee, a migrant I am in exile Talk to me
  65. 65. THANK YOU Paul Prinsloo (Prof) Research Professor in Open Distance Learning (ODL), College of Economic and Management Sciences, Samuel Pauw Building, Office 5-21, P.O. Box 392, Unisa, 0003, Republic of South Africa T: +27 (0) 12 433 4719 (office) prinsp@unisa.ac.za Skype: paul.prinsloo59 Personal blog: http://opendistanceteachingandlearning.wordpress.com Twitter profile: @14prinsp
  66. 66. Imagecredit:https://pixabay.com/en/sculpture-bronze-child-boy-1392529/ Thank you Paul Prinsloo (Prof) Research Professor in Open Distance Learning (ODL), University of South Africa, Unisa, T: +27 (0) 12 433 4719 (office) prinsp@unisa.ac.za Skype: paul.prinsloo59 Personal blog: http://opendistanceteachingandlearning.wordpress.com Twitter profile: @14prinsp

×