HANNA, RISDEN & ALEXANDER
Guidelines for Usability
Testing with Children
by Selena M.
Although user-centered design is a well-supported concept in the literature on adult
computer products, not until recently have publications begun to appear addressing the
need to include the user in the design process of children’s computer products. Good
examples are a recent panel discussion in interactions on the importance of understanding
the perspectives and needs of children, and the energizing work of Allison Druin and Cynthia
Solomon [1, 2]. Growth has also occurred in evaluation research in both the industrial
and academic communities, assessing the effectiveness or appeal of various types of
interactions...september + october 1997 9
children’s software and hardware products [5, 8]. (6 to 10 years), and middle-school-aged children
As the body of literature on children’s use of com- (11 to 14 years). These age divisions are arbi-
puter products grows, a necessary step is to flesh trary, and many behaviors will overlap. In our
out the details of exactly how to include children experience, most children younger than 2 1/2
in computer product design. The goal of this years of age are not proficient enough with stan-
column is to provide examples and guidelines dard input devices (e.g., mouse, trackball, or
from one research group working to incorporate keyboard) to interact with the technology and
user data from children in the design process. provide useful data. Children older than 14 years
As usability engineers at Microsoft, we apply of age will likely behave as adults in a testing sit-
various research methods to improve the design uation and should be treated accordingly.
of children’s computer products, both software
and hardware. We review design specifications Preschool (ages 2 to 5 years)
for common usability issues and age-appropriate- Preschoolers require the most extensive adapta-
ness for the target population. We also visit tions of usability testing because their attention
homes and schools to gather observa- span, their motivation to please adults, and their
tional, interview, and questionnaire ability to adjust to strange surroundings and new
data on children’s use and liking of people may change from one moment to the
computer products. We conduct lon- next. In general, children in this age range should
gitudinal studies in homes or in the lab be allowed to explore the computer according to
to assess ease of use, learning, and appeal over their own interests and pacing instead of per-
time. However, the majority of our research con- forming a series of directed tasks. They will often
sists of usability lab tests, which offer the advan- be happy to show you what they know, and what
tages of iterative testing of design changes, team they can do on the computer independently.
involvement in observing children, and rapid dis- When assessing appeal or engagement, testers
covery of usability issues. Applying our develop- will need to closely observe children’s behavior
mental psychology backgrounds and professional such as sighing, smiling, or sliding under the
experience, we have developed the following table. Children this age often have difficulty
guidelines for how to tailor traditional software expressing their likes and dislikes in words.
usability testing to allow children to participate.
This column is geared towards the computer Elementary School (ages 6 to 10 years)
professional who is suddenly presented with the Children in this age range are relatively easy to
need to assess the usability of a computer prod- include in software usability testing. Their
uct with children. We will assume that you have experience in school makes them ready to sit at
access to basic usability lab equipment (e.g., a a task and follow directions from an adult, and
quiet room for testing that is set up with com- they are generally not self-conscious about
puter equipment as well as a camera to record being observed as they play on the computer.
the interaction, and a one-way mirror into an They will answer questions and try new things
observation room next door) and a general with ease. In this age range, children will devel-
knowledge of usability testing. A few resources op more sophistication about how they can
for usability methods that we have found helpful describe the things they see and do. Six- and
can be found in the reference list [3, 4, 6, 7]. In seven-year-old children will be more hands-
the rest of this column, we will focus our infor- on—ready to work on the computer but a lit-
mation on how to adapt testing to maximize the tle shy or inarticulate when talking about the
benefit of using children. computer. Ten-year-old children may have
extensive computer experience and be ready to
Descriptions of Common Target Age critique your software.
Following are general descriptions of three com- Middle School (ages 11 to 14 years)
mon target age ranges: preschool-aged children Children in this age range are very easy to
(2 to 5 years), elementary-school–aged children include in usability testing. Most will be com-
10 interactions...september + october 1997
fortable with computers and with unfamiliar will find yourself more tired than usual after
adults. Children this age can be asked to per- testing children, especially when you are in the
form, and actually enjoy, specific tasks after a room with them observing and providing
period of free exploration. Some older children encouragement and feedback.
in this age range may be able to “think aloud” • When planning a series of tasks, switch the
during the session, while others may be self- order around for different children so that the
conscious about having people watch them and same tasks do not always come at the end of the
listen to what they say. These children may test when children are tired. We have had the
bring a very high level of computer expertise, or experience of children’s beginning to chafe at
distinct expectations for what they will be being told what to do towards the end of the
doing, to a usability session. test and inventing novel ways of responding,
such as picking all the left-hand items when
Guidelines for Testing Children asked to show preferences in a paired compari-
Set-up and planning • Screen children for testing that have at
• Make the lab a little more child-friendly by least some experience with the computer. You
placing a couple of colorful posters on the don’t want to have to spend your testing time
walls, but avoid going overboard. Strike a bal- teaching a child how to use a mouse. We usu-
ance between an adult-oriented lab environ- ally require that children have 6 months’ expe-
ment and an inviting play space that may rience with mouse-driven software at home
distract children from the computer. For exam- before they can participate in usability testing.
ple, one of us added giant floor pillows to the • Do not include children in testing who
lab until one child decided to spend her time have too much computer expertise (unless they
there having a pillow fight. are your target audience). Children with
• Preschool-aged children may have difficul- advanced computer skills (programming their
ty switching from the input device they use at own games, scripting Web pages for their
home (trackball versus mouse, one-button ver- school, and so on) will not provide the kind of
sus two-button mouse) to a different one in the data necessary to build products that are usable
lab. If possible, find out what they regularly use for children at large.
and set it up for them before the test. Also, set • In general it is not a good idea to use your
the cursor or pointer speed to its slowest setting own or colleagues’ children as usability
for children in this age range. participants. Even if children whose
• Use laboratory equipment as effectively yet parents work for software companies
as unobtrusively as possible. Make sure to place do not have the aforementioned
microphones close to children to pick up their advanced skills, mentioned above,
soft voices but opt for smaller ones such as pres- they probably have above-average
sure zone microphones (PZM) over large ones. exposure to the products their parents
Avoid furniture arrangements that face children develop and to terms used in the industry.
directly toward the video camera or a one-way This knowledge may bias the study. It also
mirror. In one poorly designed arrangement, puts children in an awkward position if they
two 11-year-olds faced the one-way mirror as truly do not like your product.
they interacted with a paper prototype. This
resulted in a lot of off-task grooming behavior. Introductions
• Schedule children for an hour of lab time. • Establish a relationship with children
Preschoolers will last on average about 30 min- when you first meet them by engaging in
utes, but will need extra time for play and some small talk to find out more about one
exploration. Even older children will become another. Children are often happy to talk
fatigued after an hour of concentrated comput- about their birthdays, their favorite computer
er use. Give yourself plenty of time in between games, or their favorite subjects or sports at
children, and don’t over-schedule the day. You school.
interactions...september + october 1997 11
• Explain confidentiality agreements by be play-testing games refused to participate
telling children that designs are “top-secret.” when he was presented with a paper prototype
Parents should sign the agreements because for an educational title.
they are the legal guardians and because they • Show children and parents around the lab,
will also see or hear about the design. Older including behind one-way mirrors. Letting chil-
children may be dropped off by parents or dren see the workings of the lab gives them a
come by bus. In that case, confidentiality agree- better sense of control and trust in you. Explain
ments can be sent home with the child and to that team members like to watch children use
Libby Hanna, Ph.D. be returned by mail. their designs, and when in another room, they
IMG Usability • Have a script for introducing children to can talk to one another about how to fix things
Microsoft Corporation the testing situation. A possible script might be without bothering you. Explain cameras and
One Microsoft Way (after introducing yourself): microphones, saying that you use them so you
Redmond, WA 98052- “I call this a test, but I’m not testing you at all. don’t have to remember everything children tell
6399 I’m asking you to help us test our software you—you can look at the tape later.
+1-425-882-8080 design. I need to see what’s too easy or what’s • Younger children (up to 7- or 8-year-olds)
x26684 too hard for children your age so we can fix it will need to have the tester in the room with
Fax: +1-425-936-7329 and make it better. I’ll ask you to figure out them—they will need reassurance and encour-
email@example.com things on your own most of the time, but I’m agement and may be agitated by being alone or
here if you get stuck.” following directions from a loudspeaker.
Kirsten Risden, Ph.D. Because these children will probably have their
IMG Usability For preschool-aged children, this dialogue is parents in the room as well, your presence is
Microsoft Corporation also directed to parents, and could be altered as essential to diminish parents’ interference.
One Microsoft Way follows: Older children can try being by themselves,
Redmond, WA 98052- “I want to make it clear that I’m testing the which will enable you to take notes in the obser-
6399 software, not your child. We want the software vation room. When necessary to give help or
+1-425-703-4088 to be fun and easy for your child to use on her instructions through an intercom, speak softly
Fax: +1-425-936-7329 own, so I will be asking you to sit back and and give children some kind of warning before
firstname.lastname@example.org allow your child to try things out. I’m right launching into what you need to communicate.
here if she gets stuck, and I will help her out • Younger or shyer children may be uncom-
Kristin J. Alexander, by giving some hints and asking her to make fortable alone with the tester. In our labs, all
Ph.D. some guesses.” children under 5 have their parents remain with
Hardware Ergonomics them throughout testing. If needed, make pro-
and Usability • Motivate older children by emphasizing visions for the parent to stay in the testing room
Microsoft Corporation the importance of their role. For example, tell with his or her child (ideally out of the child’s
One Microsoft Way them that you have forgotten what it is like to line of sight), and explain to the parent that he
Redmond, WA 98052- be a child, and that you need their help to make or she they may observe but should interact
6399 a good product for children all around the with their child as little as possible during the
+1-425-703-4462 world. We have seen this kind of statement test. Especially shy children may need addition-
Fax: +1-425-936-7329 make the difference between rowdy, uncooper- al support from their parents, and can sit on
email@example.com ative and willing, helpful junior high school their parents’ laps if they would like to do so. If
classrooms. there is a separate observation area, the parents
• Set children’s expectations appropriately for of older children should be allowed to watch the
what they will be doing during the usability ses- test from there.
sion. Many children will expect to see a finished • If siblings accompany children to a test,
piece of software when they enter the lab and make sure they stay in the observation area or
may be disappointed when they are presented another separate room for the duration. We
with a paper prototype. Explain to them why it each have had the experience of the older sib-
is important to get their feedback at the current lings in the corner of the room who finally
stage in development. One of us had a disas- can’t contain themselves and start to shout out
trous session in which a child who expected to directions.
12 interactions...september + october 1997
During the Test random conversation, they should be gently
• Preschool-aged children may need a little reminded to pay attention to the computer.
warm-up with the computer at the beginning of They can be encouraged to keep working with
the test. One way to help children warm up is comments like “We need to keep trying this for
to play a game of placing your finger somewhere 5 more minutes—then we can try something
on the computer screen and asking the child, different.” “I want to see just how much you can
“Can you make the pointer touch my finger?” do—let’s try some more.”
Move to a few other locations on screen and ask, • Another way to encourage young children
“How about over here?” “What about in this to try an activity they aren’t immediately attract-
corner?” This game also gives you a good indi- ed to is to pretend that you need help doing it.
cation of their competence with the mouse. One girl was immediately drawn in after the
• Older children can be expected to perform tester gave several incorrect answers in a row and
specific tasks in addition to free exploration of then offered her a turn to get it right.
the product. However, it will be necessary to • If your test will run longer than about
break down the tasks into smaller segments 45 minutes, children should be asked if
than for adults, particularly for complex they want to take a short break at some
activities. Check to make sure children point. This should be something brief,
understand what is being asked of them like using the restroom if needed or
and restate the task if you see signs that getting a drink of water. It’s definite-
a child has forgotten it. It is also a good ly time to take a break when a
idea to prepare a script of hints that offer preschool-aged child begins to
varying levels of support for performing the bounce or wriggle uncontrollably, even
task. Tracking which hints are needed and if he states he doesn’t want to stop (check
when can give you a better understanding of with the parents for their child’s cues for
the help a child will need to progress in the needing to use the restroom).
program. • For children who are struggling to
• Children in general are often used to read words or numbers, you may have to
working or playing with new computer pro- read items for them (if that’s an expected
grams with others—their parents, siblings, part of the interaction), for example, METHODS & TOOLS
friends, or classmates. They are used to ask- e.g., “That says 3 plus 2 equals what? COLUMN EDITORS
ing for help if they aren’t sure what to do. You’ll What’s 3 plus 2?” If a child says, “I don’t Michael Muller
need to redirect their questions by questions of know—tell me” you can say, “Try to figure it Microsoft Corporation
your own. A sample exchange might run: out” or “Make a guess—what do you think?” One Microsoft Way
Child: What do I click on to start? • Keep children feeling encouraged by offer- Redmond, WA 98052
Tester: What does it look like you click on? ing generic positive feedback, in case they may firstname.lastname@example.org
Child: I don’t know. feel they are failing at figuring out the software.
Tester: Look around the screen. Do you see Statements like, “You really worked at that!” or Finn Kensing
anything that looks like it might start the “You did that all on your own!” can help keep Computer Science
game? children motivated without giving them infor- Roskilde University
Child: Is this it? mation about success or failure. P.O. Box 260
Tester: What do you think? DK-4000 Roskilde
Finishing Up Denmark
• Do not ask children if they want to play the • Gauge how much children like a program email@example.com
game or do a task—that gives them the option by observing signs of engagement such as smiles +45-4675-7781-2548
to say no. Instead use phrases such as “Now I and laughs or leaning forward to try things, and fax: +45-4675-4201
need you to…” or “Let’s do this…” or “It’s time signs of disengagement such as frowns, sighs,
to…” yawns, or turning away from the computer.
• If children begin to stray from the comput- These behavioral signs are much more reliable
er, swinging around to look at the mirror or pic- than children’s responses to questions about
tures on the wall or trying to engage you in whether or not they like something, particular-
interactions...september + october 1997 13