In this paper, the authors draw on methods used in the field of interaction design, emphasizing a user-centred design approach including methods such as usability testing, design metaphors, interview with users, video observations, focus groups, and think aloud sessions. However, a challenge of these methods is that they are designed for adults and are not necessarily appropriate to investigations including children. The guiding questions for this systematic literature review are (1) the motivation for conducting usability tests with children, and (2) the kind of methodological, practical, and ethical considerations that should be considered when involving children in usability studies. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist and PRISMA flow diagram are applied in order to assure the quality of the process of this investigation. Nine articles are analyzed and then synthesized by applying the constant comparative method. The synthesis of the literature review is based on the identified thematic priorities, which are categorized as follows: 1) the motivation for involving children as test persons in design processes, 2) definitions of usability, 3) practical considerations, 4) methodological considerations, and 5) ethical considerations.
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Considerations and Methods for Usability Testing with School Children
1. CONSIDERATIONS AND METHODS FOR
USABILITY TESTING WITH CHILDREN
M A L E N E H J O R T B O E A N D E R S E N , M D . S A I F U D D I N K H A L I D , A N D E V A I R E N E B R O O K S
D E P A R T M E N T A N D L E A R N I N G A N D P H I L O S O P H Y , F A C U L T Y O F H U M A N I T I E S ,
A A L B O R G U N I V E R S I T Y
1st EAI International Conference on Design, Learning & Innovation
MAY 2–3, 2016 | ESBJERG, DENMARK
2. Outline
• Scope and Goal
• Methods
• Data Collection
• Data Analysis and Synthesis
• Findings
• Discussion and Conclusion
3. Scope and Goals
• Scope
• User-centered design approach including methods are designed for
adults and are not necessarily appropriate to investigations including
children.
• The needs, skills, terminologies, and desires of children are essentially
different from those of adults.
• Goals: A systematic literature review to synthesize:
• (1) the motivation for conducting usability tests with children
• (2) the methodological, practical, and ethical considerations when
involving children in usability studies (taking point of departure from
Read, 2015).
4. Methods
• Preferred Reporting Items for Systematic Reviews and Meta-Analyses
(PRISMA) checklist and PRISMA flow diagram
• Databases - Educational Resources Information Center (ERIC) and Scopus
• Keywords usability testing, usability evaluation, children, interaction design,
methods, and guidelines
• exclusion criteria
• if (1) the article is not about children aged 5-17 years of age, or
• (2) the primary focus of the article is on usability tests of a specific
product, and not on the methods or considerations focusing on the
involvement of children in the testing process.
• Analysis and Synthesis. constant comparative method (Glaser, 1965).
5. Methods cont.
Fig. 1. PRISMA flow diagram
9 articles, published between
1997 and 2015.
6.
7. Qualitative Analysis and Synthesis
• Categories
• 1) the motivation for involving children as test persons in design processes,
• 2) definitions of usability,
• 3) Considerations/concerns (Read, 2015):
• 3.1 Practical considerations around arranging studies and recruiting
children,
• 3.2 Methodological concerns in terms of ensuring that children can
contribute in meaningful ways,
• 3.3 Ethical considerations around the meaning of the childrens
participation.
• ”In practical terms, the classic workaround is to carry out evaluations in schools
or in afterschool clubs.” (p.64)
8. Usability Testing with Children: Methods & Theories
According to Khanum and Trivedi’s (2013) review:
• The theory of behaviour settings (Roger Garlock Barker in late 1940s):
states that ”there are specific, identifiable units of the environment, the
physical and social elements, which are combined into one unit, which
have very powerful influences on human behaviour”
Inspection Methods
(without end users)
Test Methods
(with end users)
Heuristic Evaluation or
Expert evaluations
Think Aloud
Cognitive Walkthrough Field observation
Action Analysis Questionnaire
9. The Motivation to Involve Children as Participants in
Design Processes
• Roles:
• User
• Tester
• Informant
• Design partner
• Underlying dimensions:
• relationship to adults (indirect,
including feedback, dialogue, and
elaboration)
• relationship to technology (ideas,
prototypes and products)
• goals for the inquiry (questioning
impact of technology, and
improved usability/design)
• (Druin, 2002; Guha et al., 2004)
10. Practical Considerations
• Guidelines for usability testing with children (Hanna, 1997)
• Usability engineers
• Laboratories
• How to set up child-friendly test environments
• Age groups
• “Children under 12 years of age are not able to think aloud”
• Children, interactive technology change (Markopoulos & Bekker, 2003; Read et
al., 2008)
• Mobile technologies
• Time span
• Additional guidelines needed
11. Methodological Considerations
• Think aloud test is mostly used
• Think aloud
• Constructive interaction
• Selection of participants
• Evaluations beyond the restrictions of locations and time
• Transformation of power, confidence, motivation and feeling of comfort
• (Read et al., 2008)
12. Ethical considerations
Meaningful participation
In the IDC [Interaction Design and Children] literature very few researchers
have documented how they have concerned themselves with the rights and
feelings of children within the context of research using participatory design.
(Read et al., 2014)
13. Discussion and conclusion
• Not many studies include younger children as a participant group
• Cooperative Inquiry
• Bags of Stuff technique
• Mixing Ideas technique
(e.g. Guha et al., 2004; Borum et al., 2015)
Aalborg University: PhD course on CCI addressing methodologies,
methods and techniques for younger children
Editor's Notes
Qualitative analysis where we identified themes:
the motivation for involving children as test persons in design processes,
definitions of usability,
practical considerations,
methodological considerations, and
ethical considerations.
The child’s role as a test person was first discussed in the literature of the late 1980s and early 1990s by Allison Druin
Papert: children as builders, scientists and learners that led to the early inclusion of children in the technology design process
child computer interaction community (CCI)
Hanna et al. set some guidelines for usability testing with children, based on their work as usability engineers at Microsoft
Several of the studies have applied the guidelines developed by Hanna and her colleagues.
preschool (2-5 years), elementary school (6-10 years) and middle-school (11-14 year)
Extroversion and verbalization skills
Markopoulos and Bekker question some of the advice or guidelines described by Hanna et al.
nothing stays the same, and in the dynamic area of interactive technology and children, change is inevitable and rapid
children have shorter attention spans than they used to, but modern young children can often concentrate for only very short periods—as short as 10 minutes—and even older children find sessions beyond 30 minutes problematic
constructive interaction is the variation that identifies the most problems
process of selecting the participants: “Constructive interaction with pairs of children knowing each other identified more problems (on all severities) and specifically more critical problems
Children are becoming independent and gaining increasing autonomy in accessing, personalizing and owning interactive digital systems
to acknowledge the children for their participation
know the purpose of their involvement
how their ideas will be used and what happens if some of their ideas contribute to the success of the company or the investigation
that the more meaningful participation is for children the more beneficial the activity is across all aspects