Talk given to GIST / HCI group at University of Glasgow. About paper "implementing ethics" accepted for OzCHI 2016.

1. IMPLEMENTING ETHICS FOR A
MOBILE APP DEPLOYMENT
JOHN ROOKSBY, PARVIN ASADZADEH, ALISTAIR
MORRISON, CLAIRE MCCALLUM, CINDY GRAY,
MATTHEW CHALMERS

2. These are slides for a “GIST” seminar at the
University of Glasgow (17 / 11 / 2016).
Pre-conference talk about OzCHI paper.
Full paper here: johnrooksby.org/publications.html

3. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016

4. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
IN THIS TALK
▸ Case study of “implementing” or “operationalising” ethics in a
deployment-based study.
▸ No claim that our work is perfect. Instead, an exploration of the
problem of ethics:
▸ There is uncertainty and complexity in research ethics for
deployment studies.
▸ Research ethics are enmeshed with system design in deployment
studies.
▸ I will conclude that a “in action” view of research ethics is appropriate.

5. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
IN-ACTION ETHICS
▸ “The realities of conducting the study can unexpectedly differ from
what the researchers have planned for”
Munteanu et al. Situational Ethics: Re-thinking Approaches to Formal Ethics Requirements for Human-
Computer Interaction. Proc. CHI ’15.
▸ “While technology design has become explorative, situated and
responsive, the accompanying ethics processes largely remain static
and anticipatory.”
Frauenberger et al. In Action Ethics. Interacting With Computers 2016.
▸ Responsible HCI research “is not an absolute virtue but an attitude and
practice”
Grimpe et al. Towards a closer dialogue between policy and practice: responsible design in HCI. Proc.
CHI ’14.

6. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
DEPLOYMENT BASED RESEARCH
▸ Characteristics:
▸ A technology released “in the wild” (outside of a
controlled environment), over a prolonged period.
▸ Beta/advanced prototype.
▸ Uptake is not necessarily via formal recruitment, and
often there is little interaction between participants and
researchers.
▸ Data is collected during use of the system.

7. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
DEPLOYMENT BASED RESEARCH
▸ Problems:
▸ People may be unaware they are participating in a study.
▸ Limited opportunity for communication between
participants and researchers.
▸ It is difficult to anticipate and manage problems outside
of a controlled environment.

8. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016

9. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
QUPED
▸ Designed to encourage
walking. Uses three
behavioural change
strategies:
▸ Self monitoring (step
counts)
▸ Automated goal setting
▸ Social comparison

10. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
ETHICS OF QUPED
▸ Formal review processes
▸ Institutional review and Apple review
▸ Supporting autonomy
▸ Informed consent and withdrawal
▸ There are other ethical issues
▸ Risk - covered in the paper but not the talk
▸ Multiple other issues in ethics and responsible research

11. FORMAL REVIEWS

12. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016TEXT
INSTITUTIONAL REVIEWS
▸ Until recently, adherence to ‘codes of ethics’ was the norm.
▸ Now, many institutions and funding bodies mandate a
formal ethical review for any project involving human
subjects.
▸ “Ethical Creep”
Brown et al. Five Provocations for Ethical HCI Research. Proc. CHI ’16.

13. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016TEXT
INSTITUTIONAL REVIEWS
▸ ACM has long had a Code of Ethics
▸ ACM Conferences (such as CHI) moving towards
mandating formal, institutional reviews:

14. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
INSTITUTIONAL REVIEW
▸ At the University of Glasgow, research with (adult, non-vulnerable) human
participants is reviewed by a college ethics committee. There are four
colleges, including:
▸ College of Science and Engineering.
▸ College of Social Science.
▸ The application forms/processes for the colleges are different.
▸ Science and Engineering has an orientation to lab studies, Social Science
to field studies. Neither directly accommodates deployment research.
▸ Neither specifies what the ethical issues are.

15. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
INSTITUTIONAL REVIEW
▸ For Quped, as a multi-disciplinary team we had a choice of
committee. We chose Social Science:
▸ Deployments more akin to fieldwork.
▸ Previous experience with applications showed an
application to social science would be easier.
▸ We told Claire it would be a “good experience” for her
to apply.

16. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
APPLE REVIEW
▸ App Store Apps are also reviewed by Apple, with their own criteria,
e.g.:
▸ “Apps conducting health-related human subject research must
obtain consent from participants or, in the case of minors, their
parent or guardian. Such consent must include the (a) nature,
purpose, and duration of the research; (b) procedures, risks, and
benefits to the participant; (c) information about confidentiality
and handling of data (including any sharing with third parties); (d)
a point of contact for participant questions; and (e) the
withdrawal process.”
▸ Two separate, independent and overlapping review processes.

17. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
FORMAL REVIEW: SUMMARY
▸ Multiple, partially conflicting guidelines and processes.
▸ Guidelines must be translated to specific study.
▸ Reviews are late in the day - the objective is to “pass” the
review, not to consider ethical issues at this point.
▸ Different organisations have jurisdiction over ethics.

18. AUTONOMY

19. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
AUTONOMY
▸ The term autonomy comes from an ancient Greek term translating as
“self law” (i.e. to be a law to oneself).
▸ In the context of research ethics, it means that an individual should be
able to make an informed and free decision to participate in a study.
▸ Often seen as key for research ethics, however:
▸ Consent as severance from data.
Luger and Rodden Luger, E. and Rodden, T. An Informed View on Consent for UbiComp. UbiComp ’13.
▸ Consent as limitation of legal liability.
Cairns, P. and Thimbleby, H. The diversity and ethics of HCI. Computer and Information Science ’03.

20. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
CONSENT
▸ For Quped, main recruitment mechanism is the app store.
It is not possible to ensure people are informed about the
study before the install.
▸ Cannot frame app as a research study in the app store.
▸ Cannot specify the app is for adults.
▸ Installation cannot be considered consent.
▸ Therefore, we need in-app mechanisms to gain consent.

21. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
CONSENT
▸ Consent integrated with on-boarding process

22. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
CONSENT
▸ In app information sheet appears after
description.
▸ No data uploaded until after ‘accept’
▸ A ‘decline’ button (just sends you back)
▸ Scrollable in textarea to log how far people
view the text
▸ Under 18 switch
▸ Changed in an update to be ‘switch off is you
are under 18’.

23. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
CONSENT
▸ In app information sheet appears after
description.
▸ No data uploaded until after ‘accept’
▸ A ‘decline’ button (just sends you back)
▸ Scrollable in textarea to log how far people
view the text
▸ Under 18 switch
▸ Changed in an update to be ‘switch off is you
are under 18’.

24. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
CONSENT
▸ Apple also requires the app to ask for
permissions.
▸ Pop up, ‘just in time’ permission
requests.

25. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
CONSENT
▸ Information
sheet also
available in-app.

26. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
CONSENT
▸ Scaffolded approach to
collecting demographic data
▸ This data not required to
use the app, only for
social comparison.
▸ Second opportunity for
identifying non-adults
▸ If a user specifies under
18, they are excluded
from the study.

27. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
WITHDRAWAL
▸ People should also be able to freely withdraw from a
research study
▸ Withdrawing from a deployment somewhat different from
with drawing from a lab study.
▸ Stopping using the app: people can do this whenever
they wish.
▸ Pulling out of the study (i.e. data not to be included): we
need to support this in-app.

28. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
WITHDRAWAL

29. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
DEBRIEFING
▸ Debriefing is seen as good practice in research studies
▸ Another opportunity to give information and ask questions
▸ However, with a deployment such as Quped there is no end.
▸ If a user deletes the app, there is no way to contact them (unless we
collected extra data such as email address).
▸ Left debriefing as a design problem to address later.
▸ We can turn off logging after some point (12 months?)
▸ We could create a strategy based on whether the intervention does/
doesn’t work.

30. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
AUTONOMY: SUMMARY
▸ Standard approaches to autonomy problematised in
deployment research.
▸ Automation, with limited opportunity for discussion
▸ Changing nature of withdrawal and debriefing
▸ Consent and withdrawal part of the app design.
▸ Design changes were made after release, and some
decisions put off until future.

31. DISCUSSION

32. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
UNCERTAINTY IN ETHICS
▸ No clear set of ethical rules and procedures, but multiple, diverging
review processes and guidelines for us to navigate.
▸ The deployment is built upon opaque infrastructure that sits between
the participants and us, which constrains and to an extent dictates what
we can do.
▸ As HCI researchers we have a concern for naturalistic, ecologically valid
data. This produces tensions with ethical concerns for informing people
about the research.
▸ For long term deployments, some issues are clarified during the
deployment. Therefore it can be appropriate to revisit ethical decisions.

33. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
ETHICS AS DESIGN
▸ Research ethics had to be worked out in anticipation of the
ethical reviews during the design process. Problems with
reviews may have entailed major changes to our design.
▸ We needed to create mechanisms for consent, withdrawal
and data management that were a coherent part of an
overall user experience. These needed to be present in the
app, easy to understand and not off-putting.
▸ App design is iterative, and updates and changes can have
implications for our ethics procedures.

34. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
CONCLUSION
▸ “While technology design has become explorative, situated and responsive,
the accompanying ethics processes largely remain static and anticipatory.”
Frauenberger et al. In Action Ethics. Interacting With Computers 2016.
▸ Implications
▸ HCI should share and discuss design knowledge relevant to research
ethics
▸ Ongoing support research ethics in the planning/design phases and post-
release
▸ Reviews are valuable (in principle) but better support needed for
deployment based research