This document provides an overview of key concepts for data gathering and analysis in interaction design. It discusses techniques for interviews, questionnaires, observations, and the analysis of both qualitative and quantitative data. The goal is to understand users and inform the design process. Techniques covered include interviews, questionnaires, observations, analysis frameworks like grounded theory, and presenting findings.

1. IM2044 – Week 4: Lecture
Dr. Andres Baravalle
1

2. Interaction design
• The next slides are based on the
companion slides for the textbook
• By the end of this week, you should have
studied all chapters of the textbook up to
chapter 8
2

3. Overview
• Data gathering
– Interviews
– Questionnaires
– Ethnographic observations and observations
in a controlled environment
• Qualitative and quantitative analysis
3

4. Key issues in data gathering
• Setting goals
– Decide how to analyze data once collected
• Identifying participants
– Decide who to gather data from
• Relationship with participants
– Clear and professional
– Informed consent when appropriate
• Triangulation
– Look at data from more than one perspective
• Pilot studies
– Small trial of main study
4

5. Data recording
• Typically supported by a combination of:
– Notes
– Audio recording
– Video recording
– Photography
5

6. Interviews
• Unstructured - not directed by a script.
Rich but not replicable.
• Structured - tightly scripted, often like a
questionnaire. Replicable but may lack
richness.
• Semi-structured - guided by a script but
interesting issues can be explored in more
depth. Can provide a good balance
between richness and replicability.
6

7. Closed vs. open questions
• ‘Closed questions’ have a predetermined
answer format, e.g., ‘yes’ or ‘no’
– Easier to analyse
• ‘Open questions’ do not have a
predetermined format
– Can allow to better explore research topics
7

8. Questions to avoid
• Long questions
• Compound sentences - split them into two
• Jargon and language that the interviewee
may not understand
• Leading questions that make assumptions
– e.g., why do you like …? Or
– Asking a question that the respondent is not
qualified to answer
• Unconscious biases, e.g. gender
stereotypes
8

9. Running the interview
• Introduction – introduce yourself, explain the goals of the
interview, reassure about the ethical issues, ask to
record, present any informed consent form.
• Warm-up – make first questions easy and nonthreatening.
• Main body – present questions in a logical order
• A cool-off period – include a few easy questions to
defuse tension at the end
• Closure – thank interviewee and signal the end,
e.g. switch recorder off.
9

10. Enriching the interview process
• Props - devices for prompting interviewee, e.g., a
prototype, scenario
10

11. Questionnaires
• Questions can be closed or open
– Closed questions are easier to analyze, and
may be done by computer
• Can be administered to large populations
– Paper, email and the web used for
dissemination
• Sampling can be a problem when the size
of a population is unknown as is common
online
11

12. Questionnaire design
• Provide clear instructions on how to complete
the questionnaire.
• Decide on whether phrases will all be positive,
all negative or mixed.
• Different versions of the questionnaire might be
needed for different populations.
• The impact of a question can be influenced by
question order.
• Strike a balance between using white space and
keeping the questionnaire compact.
12

13. Question and response format
• ‘Yes’ and ‘No’ checkboxes
• Checkboxes that offer many options
• Rating scales
– Likert scales
– Semantic scales
– 3, 5, 7 or more points
• Open-ended responses
13

14. Encouraging a good response
• Make sure purpose of study is clear
• Ensure questionnaire is well designed
– Consider offering a short version for those who do not
have time to complete a long questionnaire
•
•
•
•
Promise anonymity
Follow-up with emails, phone calls, letters
Provide an incentive
40% response rate is high, 20% is often
acceptable
14

15. On-line questionnaires
• Responses are usually
received quickly
• No copying and/or
postage costs
• Data can be easily
collected in database for
analysis
• Time required for data
analysis is reduced
• Errors can be corrected
easily
15

16. Problems with online
questionnaires
• Sampling is problematic if population size
is unknown
• Preventing individuals from responding
more than once
16

17. Observation
• Direct observation in the
field
– Structuring frameworks
– Degree of participation
(insider or outsider)
– Ethnography
• Direct observation in
controlled environments
• Indirect observation:
tracking users’ activities
– Diaries
– Interaction logging
17

18. Structuring frameworks to guide
observation
• The Goetz and LeCompte (1984)
framework:
- Who is present?
- What is their role?
- What is happening?
- When does the activity occur?
- Where is it happening?
- Why is it happening?
- How is the activity organized?
18

19. Ethnographic observations (1)
• Ethnography is a is a qualitative research
method aimed to understand human
behaviour
– Techniques used include participant observation,
interviews and questionnaires
• Ethnographers immerse themselves in the
culture that they study
– Collections of comments, incidents, and artifacts are
made
– A researcher’s degree of participation can vary along
a scale from ‘outside’ to ‘inside’
19

20. Ethnographic observations (2)
• Co-operation of people
being observed is
required
– Informants are central
– Questions get refined as
understanding grows
• Data analysis is
continuous and typically
time consuming
• Reports usually contain
examples
20

21. Online ethnography
• On-line interaction differs from face-toface
• Virtual worlds have a persistence that
physical worlds do not have
• Ethical considerations and presentation
issues are different
21

22. Observation in a controlled
environment
• Direct observation:
– Think-aloud technique
• Indirect observation:
– Diaries
– Interaction logs
– Web analytics
22

23. Choosing and combining data
gathering techniques
• Depends on:
– The focus of the study
– The participants involved
– The nature of the techniques selected
– The resources available
23

24. Quantitative and qualitative data
• Quantitative data – expressed as numbers
– Quantitative analysis – numerical methods to
ascertain size, magnitude, amount
• Qualitative data – difficult to measure sensibly
as numbers e.g. dissatisfaction
– Qualitative analysis – expresses the nature of
elements and is represented as themes, patterns,
stories
• Be very careful on how you manipulate data and
numbers!
24

25. Simple quantitative analysis
• Averages
– Mean: add up values and divide by number of data
points
– Median: middle value of data when ranked
– Mode: figure that appears most often in the data
• Percentages
• Graphical representations give overview of data
Number of errors made
Internet use
10
< once a day
8
once a day
6
4
once a week
2
2 or 3 times a week
0
0
5
10
15
20
once a month
User
25
Number of errors made
Number of errors made
Number of errors made
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
1
3
5
7
9
User
11
13
15
17

26. Visualising log data
Interaction
profiles of players
in online game
Log of web page
activity
26

27. Visualising log data
• Web analytics
27

28. Simple qualitative analysis
• Recurring patterns or themes
– Emergent from data, dependent on
observation framework if used
• Categorizing data
– Categorization scheme may be emergent or
pre-specified
• Looking for critical incidents
– Helps to focus in on key events
28

29. Tools to support data analysis
• Spreadsheet – simple to use, basic
graphs
• Statistical packages - e.g. SPSS
• Specialist qualitative data analysis tools
– Categorization and theme-based analysis,
e.g. N6
– Quantitative analysis of text-based data
29

30. Theoretical frameworks for
qualitative analysis
• Basing data analysis around theoretical
frameworks provides further insight
• Three such frameworks are:
– Grounded Theory
– Distributed Cognition
– Activity Theory
30

31. Grounded Theory
• Aims to derive theory from systematic analysis
of data
• Based on categorization approach (called here
‘coding’)
• Three levels of ‘coding’
– Open: identify categories
– Axial: flesh out and link to subcategories
– Selective: form theoretical scheme
• Researchers are encouraged to draw on own
theoretical backgrounds to inform analysis
31

32. Distributed Cognition
• The people, environment & artefacts are
regarded as one cognitive system
• Used for analyzing collaborative work
• Focuses on information propagation &
transformation
32

33. Activity Theory
• Explains human behavior in terms of our
practical activity with the world
• Provides a framework that focuses
analysis around the concept of an ‘activity’
and helps to identify tensions between the
different elements of the system
• Two key models: one outlines what
constitutes an ‘activity’; one models the
mediating role of artifacts
33

34. Presenting the findings
• Only make claims that your data can support
• The best way to present your findings depends
on the audience, the purpose, and the data
gathering and analysis undertaken
• Graphical representations may be appropriate
for presentation
• Other techniques are:
– Rigorous notations, e.g. UML
– Using stories, e.g. to create scenarios
34