The document discusses ethical considerations in online experiments, focusing on informed consent, risk assessment, and debriefing processes. It emphasizes the importance of ensuring participants understand the studies they are involved in and highlights the lower risk of online studies compared to traditional ones. Additionally, the document outlines methods for managing extraneous variables and controlling biases in experimental design.

1. Debriefing in Online Experiment
Harryadin Mahardika, PhD

2. Harryadin Mahardika
• Pop Economist
• FEUI & laporsuap.com
• Research objective:
– “to liberate and empower consumer...”
• Current research:
– Consumer empowerment
– Consumer intervention/engineering
– Mobile advertising
• Contact:
– harryadin.mahardika@ui.ac.id / harryadin@gmail.com
– @HarrySastro
2

3. Discussion Agenda
• Informed consent
• Debriefing
• Randomization
• Participant incentives

4. Risk of experiment on human subject
• Deception, manipulation, priming, scenario.
– May harm participants’ mind.
– Mental fatigue.
• Examples?

5. Informed consent
• Information about the experiment and its risk.
– As a basis for participants to decide after they
understand what the research involves (risks and
benefits ).
• Written consent vs unwritten consent.
– Written:
• Investigators must typically obtain and document voluntary
informed consent from research subjects.
– Unwritten:
• button on an online form to indicate they have read and
understood the consent form.

6. Informed consent in online experiment
• Limited interaction with participants
– investigator often cannot tell whether a subject understood the
informed consent statement.
• Online form:
– Researchers can increase the likelihood that subjects are granting truly
informed consent by requiring feedback from subjects about their
level of understanding,
• Example:
– by requiring a “click to accept” for each element in an informed
consent statement or even administering short quizzes to establish
that a subject understood.
• Reduce response rate:
– Increase nonresponse to sensitive items (Singer, 1978)
– Possibly produce biased data (Trice, 1987).

7. Risk in online experiment
• It exposes subjects to innocuous questions and benign
or transient experiences with little lasting impact.
• In general, online experiments is no more risky than
any of their offline counterparts.
• In some respects, they may be less risky:
– The reduced social pressure (Sproull & Kiesler, 1991) in
online surveys or experiments makes it easier for subjects
to quit whenever they feel discomfort.
– This freedom to withdraw is no trivial benefit, given the
strong pressures to continue in face-to-face studies (e.g.,
Milgram, 1963) and even telephone calls.

8. Risk in online experiment
• Although risk in online settings is typically low,
the actual risk depends on the specifics of the
study.
• For example:
– Some questions in a survey or feedback from an
experiment may cause subjects to reflect on
unpleasant experiences or to learn something
unpleasant about themselves
– e.g., Nosek et al.’s, 2002b, research on automatic
stereotyping.

9. Risk in online experiment
• Experiments that deliberately manipulate a:
– subject’s sense of self-worth,
– reveal a lack of cognitive ability,
– challenge deeply held beliefs or attitudes, or
– disclose some other real or perceived
characteristic
..... may result in mental or emotional harm to some
subjects.

10. Debriefing
• American Psychological Association (2002)
ethical guidelines call for debriefing subjects:
– “Providing an explanation of the nature, results,
and conclusions of the research—as soon after
their participation as practical.“
• If deception was involved:
– Researcher needs to explain the value of the
research results and why deception was
necessary.

11. Debriefing in online experiment
• When conducting research online:
– Researchers can post debriefing materials at a
Web site,
– Provide debriefing materials to those who leave
before completing the research (Nosek, Banaji, &
Greenwald, 2002a).
– For example, researchers can deliver debriefing
material through a link to a “leave the study”
button or through a pop-up window, which
executes when a subject leaves a defined Web.

12. Debriefing in online experiment
• Appropriate debriefing in online research may
be difficult:
– The absence of a researcher in the online setting
makes it difficult to assess a subject’s state.
– Difficult to determine whether an individual has
been upset by an experimental procedure or
understands feedback received.

13. Extraneous Variables cont.
Selection Bias
Improper assignment of test units to
treatment conditions
[sampling error]

14. Extraneous Variables cont.
Mortality
Loss of test units while the experiment is in
progress
[respondents selected no longer wish to
participate]

15. Controlling Extraneous Variables
Randomisation
 Randomly assigning test units to experimental groups by using random
numbers
Matching
 Comparing test units on a set of key background variables before assigning
them to the treatment
Statistical Control
 Measuring the extraneous variables and adjusting for their effects through
statistical analysis
Design Control
 Use of experiments designed to control specific extraneous variables

16. Case: Beauty is in the eye of the beholder
Scientists have found a link between drinking
alcohol and perceptions of beauty
80 students were shown colour photographs of
120 male and female students and were asked to
rate the aesthetic properties on a 7-point scale
from high unattractive to highly attractive
Half the students had drunk up to four units of
alcohol, the other half had no alcohol.
The students who had consumed alcohol rated
the people in the photographs as more attractive
than the student who did not consume alcohol.
Source: http://www.theage.com.au/articles/2002/09/091031115991721.html