Buckingham Shum, S. (2021). Deliberative Democracy as a strategy for co-designing university ethics around analytics and AI in education. AARE2021: Australian Association for Research in Education, 28 Nov. – 2 Dec. 2021 Deliberative Democracy as a Strategy for Co-designing University Ethics Around Analytics and AI in Education Simon Buckingham Shum Connected Intelligence Centre, University of Technology Sydney Universities can see an increasing range of student and staff activity as it becomes digitally visible in their platform ecosystems. The fields of Learning Analytics and AI in Education have demonstrated the significant benefits that ethically responsible, pedagogically informed analysis of student activity data can bring, but such services are only possible because they are undeniably a form of “surveillance”, raising legitimate questions about how the use of such tools should be governed. Our prior work has drawn on the rich concepts and methods developed in human-centred system design, and participatory/co-design, to design, deploy and validate practical tools that give a voice to non-technical stakeholders (e.g. educators; students) in shaping such systems. We are now expanding the depth and breadth of engagement that we seek, looking to the Deliberative Democracy movement for inspiration. This is a response to the crisis in confidence in how typical democratic systems engage citizens in decision making. A hallmark is the convening of a Deliberative Mini-Public (DMP) which may work at different scales (organisation; community; region; nation) and can take diverse forms (e.g. Citizens’ Juries; Citizens’ Assemblies; Consensus Conferences; Planning Cells; Deliberative Polls). DMP’s combination of stratified random sampling to ensure authentic representation, neutrally facilitated workshops, balanced expert briefings, and real support from organisational leaders, has been shown to cultivate high quality dialogue in sometimes highly conflicted settings, leading to a strong sense of ownership of the DMP's final outputs (e.g. policy recommendations). This symposium contribution will describe how the DMP model is informing university-wide consultation on the ethical principles that should govern the use of analytics and AI around teaching and learning data.

1. UTS CRICOS 00099F
Deliberative Democracy as a strategy
for co-designing university ethics
around analytics and AI in education
Simon Buckingham Shum
Director, Connected Intelligence Centre
Professor of Learning Informatics
Simon.BuckinghamShum.net
cic.uts.edu.au
@sbuckshum
#AARE2021 Annual Conference • 28 Nov. – 2 Dec. 2021 • https://www.aare.edu.au/events/2021-conference/
PPIE SIG 1 Symposium: Educational datafication and automated decision-making:
Concepts, tools, and encounters for technical democracy and data justice

2. We need to know how to design “Technical Democracy”
Michel Callon, Pierre Lascoumes, Yannick Barthe (2001). Acting in an Uncertain World: An Essay on
Technical Democracy. (Translation by Graham Burchell. 2009). The MIT Press: Cambridge, Massachusetts
“hybrid forums” (p.154)

3. Introducing myself and my context
• Psychology à Ergonomics à Human-Computer Interaction à
Educational Technology à Learning Analytics/AI in Education
• CIC is researching, inventing, deploying and evaluating tools to
make better use of data to enhance the UTS learning and
teaching experience
• Not in a faculty, reporting to the DVC (Education & Students)
• Seeking theoretically grounded, empirically validated, but
intensely practical insights into the effective, ethical
deployment of Analytics/AI-powered EdTech

4. UTS invents, researches and deploys
Analytics/AI-powered EdTech
• Analysing faculty/university outcomes data to provide
strategic insights for leaders
e.g. survey data • employment data • subject pathways
• Analysing student activity traces to identify significant
patterns, to improve feedback to instructors and students
• Validating new ways to bring stakeholders into the ed-tech
design process to empower them to shape the tools we hope
they’ll use

5. ProctorU online exam invigilation
https://www.proctoru.com

6. AI to analyse academic writing à instant feedback
UTS AcaWriter: https://uts.edu.au/acawriter
Archetypal ‘moves’
in academic writing
(research abstract / introduction)

7. Student feedback can be differentiated by the
academic, tailored to hundreds of students at a time
OnTask Project: https://cic.uts.edu.au/tools/ontask/
Feedback message
to students who
attended at least 1
class
Feedback message
to students who
missed all classes
weeks 1-3

8. Giving timely feedback to instructors and students
on face-to-face teamwork in the simulation ward

9. UTS news story and video
Feedback to instructors
on their positioning and
movement around the
ward during a session
Feedback to students
on their nursing teamwork
(e.g. actions, movement,
formation, speed)
Giving timely feedback to instructors and students
on face-to-face teamwork in the simulation ward

10. Adopting a Deliberative Democracy approach
https://www.amacad.org/daedalus/prospects-limits-deliberative-democracy

11. Adopting a Deliberative Democracy approach
https://www.newdemocracy.com.au
https://liminalbydesign.com.au/deliberative-processes

12. Introducing a Deliberative Democracy approach at UTS
Recruit a
Deliberative Mini-
Public (DMP)

13. https://cic.uts.edu.au/projects/edtech-ethics/
DVC(ES) request for a student/staff consultation to build
the trust needed for the ethical use of such tools

14. Recruiting the DMP: stratified randomised sampling from 130 applicants

15. About to complete a series of 5 (Zoom) workshops
following a Deliberative Democracy process
Recruit a
Deliberative Mini-
Public (DMP)
DMP commits to
learn from expert
witnesses, and may
call their own
witnesses
External,
professional
facilitators
Identify,
discuss and
prioritise key
ethical
principles from
the examples
and dilemmas
that they are
given
Present
principles to
senior leaders for
feedback
Formal review for
alignment/
adoption as UTS
policy (2022)

16. Informing the DMP’s thinking
One of several Cool or Creepy vignettes presenting the student/educator
experience of current/near-future ed-tech...

17. Informing the DMP’s thinking
...illustrated by a videoconferencing product automatically
classifying “student engagement”
https://www.minervaproject.com/solutions/forum-learning-environment/

18. Expert briefings of the DMP:
research evidence and practitioner experiences
Lecturer in Eng/IT
Learning Analytics Researcher

19. Deliberative Mini-Public collaboratively drafting principles
The facilitation team could monitor
the progress of student/staff
teams as they drafted principles in
Zoom breakout rooms, in the
Google Doc

20. Zoom voting on the acceptability of draft principles
Participants uncover their camera
when they want to indicate their
approval of a draft principle
Loathe it
Lament it
Live with it
Like it
Love it

21. Ethical themes emerging...

22. Template for a principle
Examples of this in action at UTS...
Rationale
Principle

23. Illustrative principle
Examples of this in action at UTS...
1. Use of natural language processing to analyse student writing. All users of AcaWriter are
clearly informed about what the tool does and doesn’t do, with assurance that the tool offers
feedback on drafts, and their final grade will be determined by the teaching team, not AI.
2. Use of image processing to validate a student’s identity for an online exam. Students will
be informed which exams use ProctorU, and which functions are automated, and who will act on
alerts, when, and how the student will be notified.
Rationale: Students, academics and tutors have the right to be
clearly informed how data is being analysed by AI, and who/what
acts on its outputs. (Potentially appeal to established ethical principles)
Principle: UTS will be transparent about the use of AI in a course

24. A set of draft principles for review by
UTS Analytics & Data Governance

25. Deliberative Democracy is one way to implement Technical Democracy
Technical Democracy Deliberative Democracy
Investigates sociotechnical
controversies
Focuses on controversial dilemmas requiring multiple stakeholders to own the process, and create
legitimate outcomes (DD may operate from national à regional à community à organisational scale)
Expands capacity of
institutions to deal with
sociotechnical
controversies
DD offers a process toolkit for an institution to recruit and facilitate a representative “mini-public”,
building their knowledge, skills and dispositions to reach consensual, workable agreements, avoiding
the polarisation that comes when unfacilitated (often poor) argumentation is dominated by opposing
activists
Dialogic processes
The mini-public commits to extended, informed, respectful listening and deliberation (case studies show
that DD can be effective in diverse cultures in which talk takes diverse forms)
Collective learning
The mini-public commits to learning about the topic from expert witnesses, appointed not only by subject
matter experts supporting the process, but also by the mini-public. The M-P is also coached on their
critical thinking and teamworking
Distributes expertise Expertise is cultivated first in the mini-public, and among those with whom they engage
Experiments with ideas
Opening up the decision-making process to diverse stakeholders increases the diversity of ideas
generated
Co-produces knowledge
The high quality of deliberation that is accomplished when the process works well cultivates new ideas,
and a strong sense of ownership to see the insights and their rationale understood and applied
“hybrid forums” (p.154)
Michel Callon, Pierre Lascoumes, Yannick Barthe (2001). Acting in an Uncertain World:
An Essay on Technical Democracy. (Translation by Graham Burchell. 2009). The MIT
Press: Cambridge, Massachusetts

26. Deliberative Democracy is one way to implement Technical Democracy
Technical Democracy Deliberative Democracy (when effectively implemented)
Investigates sociotechnical
controversies
Focuses on controversial dilemmas requiring multiple stakeholders to own the process, and create
legitimate outcomes (DD may operate from national à regional à community à organisational scale)
Expands capacity of
institutions to deal with
sociotechnical
controversies
DD offers a process toolkit for an institution to recruit and facilitate a representative “mini-public”,
building their knowledge, skills and dispositions to reach consensual, workable agreements, avoiding
the polarisation that comes when unfacilitated (often poor) argumentation is dominated by opposing
activists
Dialogic processes
The mini-public commits to extended, informed, respectful listening and deliberation (case studies show
that DD can be effective in diverse cultures in which talk takes diverse forms)
Collective learning
The mini-public commits to learning about the topic from expert witnesses, appointed not only by subject
matter experts supporting the process, but also by the mini-public. The M-P is also coached on their
critical thinking and teamworking
Distributes expertise Expertise is cultivated first in the mini-public, and among those with whom they engage
Experiments with ideas
Opening up the decision-making process to diverse stakeholders increases the diversity of perspectives,
and range of ideas
Co-produces knowledge
The high quality of deliberation that is accomplished when the process works well cultivates new ideas,
and a strong sense of ownership to see the insights and their rationale understood and applied

27. Summary and next steps
A university DMP
can be formed with
a high level of
ownership of the
process and
product
The DMP produced
a coherent proposal
for university review
DD is a practical
means of
implementing a
more open,
informed
accountable
process for
2022 will track the
next chapter: the
process by which
the proposed
principles are
engaged by UTS
policy governance
bodies

28. UTS community
Deliberative Mini-Public (DMP)
à Proposed Principles
UTS data/analytics/AI
governance
Ongoing dialogue as principles are
applied, and tech evolves...