Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. HANNA, RISDEN & ALEXANDER methods & tools Guidelines for Usability Testing with Children Bizzaro World by Selena M. A Although user-centered design is a well-supported concept in the literature on adult computer products, not until recently have publications begun to appear addressing the need to include the user in the design process of children’s computer products. Good examples are a recent panel discussion in interactions on the importance of understanding the perspectives and needs of children, and the energizing work of Allison Druin and Cynthia Solomon [1, 2]. Growth has also occurred in evaluation research in both the industrial and academic communities, assessing the effectiveness or appeal of various types of interactions...september + october 1997 9
  2. 2. children’s software and hardware products [5, 8]. (6 to 10 years), and middle-school-aged children As the body of literature on children’s use of com- (11 to 14 years). These age divisions are arbi- puter products grows, a necessary step is to flesh trary, and many behaviors will overlap. In our out the details of exactly how to include children experience, most children younger than 2 1/2 in computer product design. The goal of this years of age are not proficient enough with stan- column is to provide examples and guidelines dard input devices (e.g., mouse, trackball, or from one research group working to incorporate keyboard) to interact with the technology and user data from children in the design process. provide useful data. Children older than 14 years As usability engineers at Microsoft, we apply of age will likely behave as adults in a testing sit- various research methods to improve the design uation and should be treated accordingly. of children’s computer products, both software and hardware. We review design specifications Preschool (ages 2 to 5 years) for common usability issues and age-appropriate- Preschoolers require the most extensive adapta- ness for the target population. We also visit tions of usability testing because their attention homes and schools to gather observa- span, their motivation to please adults, and their tional, interview, and questionnaire ability to adjust to strange surroundings and new data on children’s use and liking of people may change from one moment to the computer products. We conduct lon- next. In general, children in this age range should gitudinal studies in homes or in the lab be allowed to explore the computer according to to assess ease of use, learning, and appeal over their own interests and pacing instead of per- time. However, the majority of our research con- forming a series of directed tasks. They will often sists of usability lab tests, which offer the advan- be happy to show you what they know, and what tages of iterative testing of design changes, team they can do on the computer independently. involvement in observing children, and rapid dis- When assessing appeal or engagement, testers covery of usability issues. Applying our develop- will need to closely observe children’s behavior mental psychology backgrounds and professional such as sighing, smiling, or sliding under the experience, we have developed the following table. Children this age often have difficulty guidelines for how to tailor traditional software expressing their likes and dislikes in words. usability testing to allow children to participate. This column is geared towards the computer Elementary School (ages 6 to 10 years) professional who is suddenly presented with the Children in this age range are relatively easy to need to assess the usability of a computer prod- include in software usability testing. Their uct with children. We will assume that you have experience in school makes them ready to sit at access to basic usability lab equipment (e.g., a a task and follow directions from an adult, and quiet room for testing that is set up with com- they are generally not self-conscious about puter equipment as well as a camera to record being observed as they play on the computer. the interaction, and a one-way mirror into an They will answer questions and try new things observation room next door) and a general with ease. In this age range, children will devel- knowledge of usability testing. A few resources op more sophistication about how they can for usability methods that we have found helpful describe the things they see and do. Six- and can be found in the reference list [3, 4, 6, 7]. In seven-year-old children will be more hands- the rest of this column, we will focus our infor- on—ready to work on the computer but a lit- mation on how to adapt testing to maximize the tle shy or inarticulate when talking about the benefit of using children. computer. Ten-year-old children may have extensive computer experience and be ready to Descriptions of Common Target Age critique your software. Ranges Following are general descriptions of three com- Middle School (ages 11 to 14 years) mon target age ranges: preschool-aged children Children in this age range are very easy to (2 to 5 years), elementary-school–aged children include in usability testing. Most will be com- 10 interactions...september + october 1997
  3. 3. fortable with computers and with unfamiliar will find yourself more tired than usual after adults. Children this age can be asked to per- testing children, especially when you are in the form, and actually enjoy, specific tasks after a room with them observing and providing period of free exploration. Some older children encouragement and feedback. in this age range may be able to “think aloud” • When planning a series of tasks, switch the during the session, while others may be self- order around for different children so that the conscious about having people watch them and same tasks do not always come at the end of the listen to what they say. These children may test when children are tired. We have had the bring a very high level of computer expertise, or experience of children’s beginning to chafe at distinct expectations for what they will be being told what to do towards the end of the doing, to a usability session. test and inventing novel ways of responding, such as picking all the left-hand items when Guidelines for Testing Children asked to show preferences in a paired compari- son task. Set-up and planning • Screen children for testing that have at • Make the lab a little more child-friendly by least some experience with the computer. You placing a couple of colorful posters on the don’t want to have to spend your testing time walls, but avoid going overboard. Strike a bal- teaching a child how to use a mouse. We usu- ance between an adult-oriented lab environ- ally require that children have 6 months’ expe- ment and an inviting play space that may rience with mouse-driven software at home distract children from the computer. For exam- before they can participate in usability testing. ple, one of us added giant floor pillows to the • Do not include children in testing who lab until one child decided to spend her time have too much computer expertise (unless they there having a pillow fight. are your target audience). Children with • Preschool-aged children may have difficul- advanced computer skills (programming their ty switching from the input device they use at own games, scripting Web pages for their home (trackball versus mouse, one-button ver- school, and so on) will not provide the kind of sus two-button mouse) to a different one in the data necessary to build products that are usable lab. If possible, find out what they regularly use for children at large. and set it up for them before the test. Also, set • In general it is not a good idea to use your the cursor or pointer speed to its slowest setting own or colleagues’ children as usability for children in this age range. participants. Even if children whose • Use laboratory equipment as effectively yet parents work for software companies as unobtrusively as possible. Make sure to place do not have the aforementioned microphones close to children to pick up their advanced skills, mentioned above, soft voices but opt for smaller ones such as pres- they probably have above-average sure zone microphones (PZM) over large ones. exposure to the products their parents Avoid furniture arrangements that face children develop and to terms used in the industry. directly toward the video camera or a one-way This knowledge may bias the study. It also mirror. In one poorly designed arrangement, puts children in an awkward position if they two 11-year-olds faced the one-way mirror as truly do not like your product. they interacted with a paper prototype. This resulted in a lot of off-task grooming behavior. Introductions • Schedule children for an hour of lab time. • Establish a relationship with children Preschoolers will last on average about 30 min- when you first meet them by engaging in utes, but will need extra time for play and some small talk to find out more about one exploration. Even older children will become another. Children are often happy to talk fatigued after an hour of concentrated comput- about their birthdays, their favorite computer er use. Give yourself plenty of time in between games, or their favorite subjects or sports at children, and don’t over-schedule the day. You school. interactions...september + october 1997 11
  4. 4. • Explain confidentiality agreements by be play-testing games refused to participate telling children that designs are “top-secret.” when he was presented with a paper prototype Parents should sign the agreements because for an educational title. they are the legal guardians and because they • Show children and parents around the lab, will also see or hear about the design. Older including behind one-way mirrors. Letting chil- children may be dropped off by parents or dren see the workings of the lab gives them a come by bus. In that case, confidentiality agree- better sense of control and trust in you. Explain ments can be sent home with the child and to that team members like to watch children use Libby Hanna, Ph.D. be returned by mail. their designs, and when in another room, they IMG Usability • Have a script for introducing children to can talk to one another about how to fix things Microsoft Corporation the testing situation. A possible script might be without bothering you. Explain cameras and One Microsoft Way (after introducing yourself): microphones, saying that you use them so you Redmond, WA 98052- “I call this a test, but I’m not testing you at all. don’t have to remember everything children tell 6399 I’m asking you to help us test our software you—you can look at the tape later. +1-425-882-8080 design. I need to see what’s too easy or what’s • Younger children (up to 7- or 8-year-olds) x26684 too hard for children your age so we can fix it will need to have the tester in the room with Fax: +1-425-936-7329 and make it better. I’ll ask you to figure out them—they will need reassurance and encour- things on your own most of the time, but I’m agement and may be agitated by being alone or here if you get stuck.” following directions from a loudspeaker. Kirsten Risden, Ph.D. Because these children will probably have their IMG Usability For preschool-aged children, this dialogue is parents in the room as well, your presence is Microsoft Corporation also directed to parents, and could be altered as essential to diminish parents’ interference. One Microsoft Way follows: Older children can try being by themselves, Redmond, WA 98052- “I want to make it clear that I’m testing the which will enable you to take notes in the obser- 6399 software, not your child. We want the software vation room. When necessary to give help or +1-425-703-4088 to be fun and easy for your child to use on her instructions through an intercom, speak softly Fax: +1-425-936-7329 own, so I will be asking you to sit back and and give children some kind of warning before allow your child to try things out. I’m right launching into what you need to communicate. here if she gets stuck, and I will help her out • Younger or shyer children may be uncom- Kristin J. Alexander, by giving some hints and asking her to make fortable alone with the tester. In our labs, all Ph.D. some guesses.” children under 5 have their parents remain with Hardware Ergonomics them throughout testing. If needed, make pro- and Usability • Motivate older children by emphasizing visions for the parent to stay in the testing room Microsoft Corporation the importance of their role. For example, tell with his or her child (ideally out of the child’s One Microsoft Way them that you have forgotten what it is like to line of sight), and explain to the parent that he Redmond, WA 98052- be a child, and that you need their help to make or she they may observe but should interact 6399 a good product for children all around the with their child as little as possible during the +1-425-703-4462 world. We have seen this kind of statement test. Especially shy children may need addition- Fax: +1-425-936-7329 make the difference between rowdy, uncooper- al support from their parents, and can sit on ative and willing, helpful junior high school their parents’ laps if they would like to do so. If classrooms. there is a separate observation area, the parents • Set children’s expectations appropriately for of older children should be allowed to watch the what they will be doing during the usability ses- test from there. sion. Many children will expect to see a finished • If siblings accompany children to a test, piece of software when they enter the lab and make sure they stay in the observation area or may be disappointed when they are presented another separate room for the duration. We with a paper prototype. Explain to them why it each have had the experience of the older sib- is important to get their feedback at the current lings in the corner of the room who finally stage in development. One of us had a disas- can’t contain themselves and start to shout out trous session in which a child who expected to directions. 12 interactions...september + october 1997
  5. 5. During the Test random conversation, they should be gently • Preschool-aged children may need a little reminded to pay attention to the computer. warm-up with the computer at the beginning of They can be encouraged to keep working with the test. One way to help children warm up is comments like “We need to keep trying this for to play a game of placing your finger somewhere 5 more minutes—then we can try something on the computer screen and asking the child, different.” “I want to see just how much you can “Can you make the pointer touch my finger?” do—let’s try some more.” Move to a few other locations on screen and ask, • Another way to encourage young children “How about over here?” “What about in this to try an activity they aren’t immediately attract- corner?” This game also gives you a good indi- ed to is to pretend that you need help doing it. cation of their competence with the mouse. One girl was immediately drawn in after the • Older children can be expected to perform tester gave several incorrect answers in a row and specific tasks in addition to free exploration of then offered her a turn to get it right. the product. However, it will be necessary to • If your test will run longer than about break down the tasks into smaller segments 45 minutes, children should be asked if than for adults, particularly for complex they want to take a short break at some activities. Check to make sure children point. This should be something brief, understand what is being asked of them like using the restroom if needed or and restate the task if you see signs that getting a drink of water. It’s definite- a child has forgotten it. It is also a good ly time to take a break when a idea to prepare a script of hints that offer preschool-aged child begins to varying levels of support for performing the bounce or wriggle uncontrollably, even task. Tracking which hints are needed and if he states he doesn’t want to stop (check when can give you a better understanding of with the parents for their child’s cues for the help a child will need to progress in the needing to use the restroom). program. • For children who are struggling to • Children in general are often used to read words or numbers, you may have to working or playing with new computer pro- read items for them (if that’s an expected grams with others—their parents, siblings, part of the interaction), for example, METHODS & TOOLS friends, or classmates. They are used to ask- e.g., “That says 3 plus 2 equals what? COLUMN EDITORS ing for help if they aren’t sure what to do. You’ll What’s 3 plus 2?” If a child says, “I don’t Michael Muller need to redirect their questions by questions of know—tell me” you can say, “Try to figure it Microsoft Corporation your own. A sample exchange might run: out” or “Make a guess—what do you think?” One Microsoft Way Child: What do I click on to start? • Keep children feeling encouraged by offer- Redmond, WA 98052 Tester: What does it look like you click on? ing generic positive feedback, in case they may Child: I don’t know. feel they are failing at figuring out the software. Tester: Look around the screen. Do you see Statements like, “You really worked at that!” or Finn Kensing anything that looks like it might start the “You did that all on your own!” can help keep Computer Science game? children motivated without giving them infor- Roskilde University Child: Is this it? mation about success or failure. P.O. Box 260 Tester: What do you think? DK-4000 Roskilde Finishing Up Denmark • Do not ask children if they want to play the • Gauge how much children like a program game or do a task—that gives them the option by observing signs of engagement such as smiles +45-4675-7781-2548 to say no. Instead use phrases such as “Now I and laughs or leaning forward to try things, and fax: +45-4675-4201 need you to…” or “Let’s do this…” or “It’s time signs of disengagement such as frowns, sighs, to…” yawns, or turning away from the computer. • If children begin to stray from the comput- These behavioral signs are much more reliable er, swinging around to look at the mirror or pic- than children’s responses to questions about tures on the wall or trying to engage you in whether or not they like something, particular- interactions...september + october 1997 13
  6. 6. ly for younger children. Children are eager to found that they placed items over the top of the please adults, and may tell you they like your bowl in order to drop them in. Another example program just to make you happy. was the use of a funny noise as an error message • Older children may be able to give reliable that children enjoyed so much that they stopped ratings about aspects of the software. We have progressing and simply made the error again and found it helpful to use a vertical scale with a again. smile face on the top end and a frown face on Like other products, computer products for the bottom end to make the end markers clear. children should be iteratively tested by children. These children also enjoy being asked to give Many teams do play-testing with children at the ideas about how to make things better. end of a product cycle, assessing general appeal • After testing, reward children by comment- and playability when the product can be seen in ing on how helpful they were. Explain to them all its glory. However, the cost of making changes that all of their hard work helped you to see at such a late stage severely curtails the amount exactly which things need to be fixed. of fixing that can be done. Adding usability test- • For a gratuity, children and parents ing to interim stages of design allows for child often appreciate a choice of a gift certifi- input at crucial stages. Children’s usability test- cate to a local toy store or movie theater in ing can resolve design debates, give quantitative addition to standard software or payment information about effectiveness of educational options. approaches, refine user interface design so it’s accessible for all ages—all before the design is Conclusions too difficult and costly to change. A complete In general, the same rules about planning, lab program of children’s usability testing is the arrangement, and interactions with participants, essential component to creating products that etc. that are used in usability testing with adults are child-friendly and child-approved. also apply to usability testing with children. Adapting the testing situation for children essen- References tially means taking a few steps to tailor the envi- [1] Brouwer-Janse, M. D., Suri, J. F., Yawitz, M., de ronment to their developmental level. Vries, G., Fozard, J. L., and Coleman, R. User interfaces Simplifying instructions so that they are clear to for young and old. interactions (March–April 1997), 34- those with more limited vocabularies, providing 46. PERMISSION TO MAKE DIGITAL/ HARD additional comfort levels for those who are less [2] Druin, A., and Solomon, C. Designing Multimedia COPY OF PART OR ALL OF THIS WORK able to regulate their emotional arousal, pacing Environments for Children. John Wiley, New York, 1996. FOR PERSONAL OR CLASSROOM USE IS the test to accommodate differing attention [3] Dumas, J. S., and Redish, J. C. A Practical Guide to GRANTED WITHOUT FEE PROVIDED spans and energy reserves—these are all part of Usability Testing. Ablex, Norwood, NJ, 1993. THAT COPIES ARE NOT MADE OR respectfully testing children. [4] Hix, D., and Hartson, H. R. Developing User DISTRIBUTED FOR PROFIT OR COM- The benefits to be gained from gathering data Interfaces: Ensuring Usability through Product and Process. MERCIAL ADVANTAGE, THE COPYRIGHT from children as users are unquestionable. John Wiley, New York, 1993. NOTICE, THE TITLE OF THE PUBLICA- Usability testing with children gives insight into [5] Lieberman, D. A. Learning to learn revisited: TION AND ITS DATE APPEAR, AND the different perspectives children bring to com- Computers and the development of self-directed learning NOTICE IS GIVEN THAT COPYING IS BY puter products and such basic elements as navi- skills. Journal of Research on Computing in Education, PERMISSION OF ACM, INC. TO COPY gation and productivity. Some teams assume 23,3 (1991), 373-395. OTHERWISE, TO REPUBLISH, TO POST that because they were once children, or because [6] Nielsen, J. Usability Engineering. Academic Press ON SERVERS, OR TO REDISTRIBUTE TO they have children at home, they can evaluate Professional, Chestnut Hill, MA, 1993. LISTS REQUIRES PRIOR SPECIFIC PERMIS- the usability of a product for children. This [7] Rubin, J. Handbook of Usability Testing: How to Plan, SION AND/OR A FEE. rarely results in a good outcome. No amount of Design, and Conduct Effective Tests. John Wiley, New © ACM 1072-5220/97/0900 $3.50 adult evaluation will find all the issues that chil- York, 1994. dren will stumble into as a matter of course. For [8] Strommen, E. F., and Revelle, G. L. Research in example, a drag-and-drop interactivity of placing interactive technologies at the Children’s Television food items in a mixing bowl got through several Workshop. Educational Technology Research & reviews until it was tested with children and Development, 38,4, 65-80. 14 interactions...september + october 1997