Usability Testing Bootcamp

How to Plan and Moderate a Usability Test Dr David Travis, Userfocus ltd #uxlx #ut

I currently run a consultancyfirm called Userfocus. We workwith clients like Yahoo!, theWorld Health Organisation andthe UK Tax Office. Yes, even theyare using these techniques tomake sure you pay what you owe.Who am I? ?

You can still buy mybook on amazon!

Design and plan usability tests. Recruit the right test participants. Apply usability testing checklists to make sure your test runs without a hitch. Develop test tasks that are relevant to your customers and your business. Organise and prepare usability testing sessions. Moderate thinking aloud usability tests in an unbiased and balanced manner. Code, collect and interpret behavioural data.What you’ll learn

Where do you work and what do you do? What one thing do you already know about usability testing? What do you want to be able to do differently as a result of the training?Interview your partner

How to Plan and Moderate aUsability TestWhat so special about a usability test?

When people are askedto justify theirchoices, they do soreadily — even whenthey never made thechoice. We need to beaware of thisconfabulation whenrunning any kind ofuser research. Johansson, P., Halls, L., Sikström, S & Olsson, A. (2005) Failure to Detect Mismatches Between Intention and Outcome in a Simple Decision Task. Science, 310: 116-119.

Lichfield Dakota 3 Man Confidence Holiday 6 Lichfield Crow 2 Man Lichfield Cherokee 3 Tent Denim/SkyBlue Man 2 Room Tent Tent Denim/Navy Man Tent Red The first tent presented wasLichfield Crow 2 Man Lichfield Cherokee 3 Confidence Holiday 6 chosen 2.5 times Lichfield Dakota 3 Man more often than any Tent Denim/Navy Man Tent Red Man 2 Room Tent Tent Denim/SkyBlue other. Yet people ‘explained’ their choices using logic and rationality. Lichfield Cherokee 3 Lichfield Crow 2 Man Lichfield Dakota 3 Man Confidence Holiday 6 Man Tent Red Tent Denim/Navy Tent Denim/SkyBlue Man 2 Room Tent Felfernig, A. et al. (2007) Persuasive Recommendation: Serial Position Effects in Knowledge-Based Recommender Systems. Lecture Notes in Computer Science, 4744: 283-294.

“The accuracy of subjectivereports is so poor as tosuggest that any introspectiveaccess that may exist is notsufﬁcient to produce generallycorrect or reliable reports.”- Nisbett, R.E., and Wilson, T.D. “Telling more than we can know: Verbal reports on mental processes.” Psychological Review, 84 (1977), pp. 231-259.

Behaviour is observed — what users do (rather than what users say they do). Participants perform real tasks — tasks that are meaningful to them and to the business. Involves real users — not just someone at the next desk. Observations and data are recorded and the data are analysed.What is a usability test?

If we’re going to test for usability, we better define it first. Listen to the definitions of usability and call out the missing words.Quick Activity: Usability Blankety Blank

“Usability means that the people who use theproduct can do so quickly and easily toaccomplish their own tasks. This deﬁnitionrests on four points: (1) Usability meansfocusing on USERS; (2) people use productsto be productive; (3) users are busy peopletrying to accomplish tasks; and (4) usersdecide when a product is easy to use.”- Janice (Ginny) Redish and Joseph Dumas (1999). A Practical Guide to Usability Testing.

“Usability is a term used todenote the ease with whichpeople can employ a particulartool or other human-madeobject in order to achieve aparticular GOAL.”- Wikipedia.

“The capability in human functionalterms to be used easily andEFFECTIVELY by the specified range ofusers, given specified training and usersupport, to fulfil the specified range oftasks, within the specified range ofenvironmental scenarios.”- Brian Shackel (1991). ‘Usability—context, framework, definition, design and evaluation’. In B. Shackel and S. J. Richardson (eds) Human factors for informatics usability.

“At ﬁrst, using words that all startedwith “E” was a word game, but I wasalso looking for a way to make thedimensions of usability easy toremember and the 5Es were born. Idecided on: Effective; EFFICIENT;Engaging; Error Tolerant; Easy to Learn.”- Whitney Quesenbery (2004) ‘Balancing the 5Es: Usability’. Cutter IT Journal, Feb, 2004.

“It is important to realize that usabilityis not a single, one-dimensional propertyof a user interface. Usability has multiplecomponents and is traditionallyassociated with these ﬁve usabilityattributes: learnability, efﬁciency,memorability, errors, SATISFACTION.”- Jakob Nielsen (1993), Usability Engineering.

“Usability is a function of the easeof use (including learnability whenrelevant) and the acceptability ofthe product and will determine theactual usage by a particular userfor a particular task in a particularCONTEXT.”- Nigel Bevan, Jurek Kirakowski and Jonathan Maissel (1991) ‘What is Usability?’ Proceedings of the 4th International Conference on HCI, Stuttgart, September 1991.

So based on thesequotations, it seems that usersusability has something todo with: goals • USERS effectiveness efﬁciency • GOALS satisfaction • EFFECTIVENESS • • EFFICIENCY SATISFACTION contextand • CONTEXTtHere’s a definition we canuse that brings all of thesetogether…

“Extent to which a product, system or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” - Effectiveness The accuracy and completeness with which users achieve specified goals. - Efficiency The accuracy and completeness of goals achieved in relation to resources. - Satisfaction Freedom from discomfort, and positive attitudes towards the use of the product, system or service.ISO 9241 definition of usability (from ISO 9241-11 and ISO 9241-210)

S T U “Extent to which a product, Jbe E O UT S system or service can used by specified users to N O achieve specified goals with IS T effectiveness, efficiency and Y SY IT A satisfaction in a specified IL E context of use.” B- S Effectiveness SA ING U H - The accuracy and completeness with which users achieve specified goals. T T A G Efficiency H N T I The accuracy and completeness of goals achieved in relation to resources. S K N A - Satisfaction EA T M Freedom from discomfort, and positive attitudes towards the use of the product, system or service. M U IS O H B T AISO 9241 definition of usability (from ISO 9241-11 and ISO 9241-210)

1With usability, we ask if 2people can actually use aproduct.You don’t need many peopleto identify a problem that 3 4needs to be fixed.For example, which knobwould you use to light thegas under the back righthob?

1answer: 4 2 3 4

Usability testingthen gives youthese kinds ofinsights to improvedesigns.

“Testing one userearly in the project isbetter than testing50 near the end.”- Steve Krug

Usability tests don’t require: - A usability lab - Observers - A moderator - Screen recording - Video recording All these are ‘nice to have’. Focusing on these alone puts you at risk of running a ‘cargo cult’ usability test.So what deﬁnes a usability test?

How to Plan and Moderate aUsability TestA live test

Try to deconstruct the usability test. How would you describe what happened when? Are there different phases?Deconstruct a usability test

Q. Would you like to take partin a usability evaluation?As a thank-you, you’ll get afree copy of E-CommerceUsability!

YES: Remain standingNO: Sit down

Q. In the last 6 months, haveyou bought any electronicgoods online, such as acomputer, a camera or amobile phone?

Q. Do you use a colour printerat home?

Q. Is your printer a laserprinter or an ink-jet printer?

LASER: Sit downINK JET: Remain standingDON’T KNOW: Remainstanding

Q. Have you done any ofthese activities in the last 2weeks? Give yourself 1 pointfor each ‘YES’ answer.- Edited the html code on a web page- Sold something on eBay - Created or commented on a blog - Installed software on your computer - Signed into your online bank account - Played an online game - Downloaded and listened to podcasts - Shared photographs or videos on web sites like ﬂickr, Twitpic or YouTube - Added a page to a social bookmarking site like del.icio.us, reddit or Digg - Subscribed to a syndicated ("RSS") web feed

7 or more: Sit down6 or fewer: Remain standing

Oh no! Your home printer isbroken.You can spend up to £100 ona new one.The printer must be wi-ﬁenabled and it must work witha Mac.Find the best printer withinyour budget.

Deconstruct the usability testParticipantRecruitment ? ? ? ? ? ?

Identify the test goalsScreen and recruitparticipantsIdentify the test tasksGreet the participantGet the video consent form /NDA signedDemonstrate and practicethinking aloudProbe with open questions

How to Plan and Moderate aUsability TestPlanning a usability test

Why are you running the test? Where will it take place? When will it take place? Who will be the test participants? What system (and what functionality) will you be testing? How will you collect and analyse the data?The Five W’s (and one H)

!"#$%&()*+,-*.&/0001*/*2(/3&)4*)#() 5*6##%(*3#+,-#*)"#*)#()* :"#*;,-001*,+*)"#*)#()* ! "#$%%!&%&!()*%+&,-%! ! H2&!KD.&$2+&,(.!0($!()%$-%$LA!&/%!4,&!(0!+%.1$,(!1.5! ! "#$%%!(.!&/%!0%1&2$%!%&!&/1&!3,44!)%!&%&%5! 1!+(6<!(0!&/%!&%&!641.!,.!&/%!()%$-1&,(.!$((9! ! "#$%%!61$&,+,61.&!6$(0,4%! ! 7/%+;!&/%!6$(&(&<6%!3($;! ! 7$%1&%!61$&,+,61.&!+$%%.%$! ! 7/%+;!&/%!$%+($5,.#!%=2,69%.&!R!+$%%.!$%+($5,.#! ! 8.#1#%!$%+$2,&9%.&!0,$9! (0&31$%!3($;!?125,(!1.5!-,5%(B! ! :((;!(2&!1.<!%=2,69%.&!<(2>44!.%%5!?%@#@!$((9A! ! 7/%+;!&/%!-,5%(!1.5!125,(!0%%5!,.!&/%!()%$-1&,(.! +(962&%$A!-,5%(!+19%$1B! $((9! ! ! 7*6##%(*3#+,-#*)"#*)#()* <#+,-#*#/$"*./-)$./0)*/--=#(* ! C$,&%!&1;!+%.1$,(! ! M%S%&!&/%!3%)!,&%!&(!,&!5%0124&!&1&%T!+4%1$!&/%!)$(3%$! ! C$,&%!&%&!641.!1.5!%.5!&(!&1;%/(45%$!0($!$%-,%3! /,&($<A!5%4%&%!+((;,%A!4(15!1!)41.;!)$(3%$!61#%! The test plan toolkit contains guidance and checklists for running a ! D.-,&%!()%$-%$! ! O2$.!(00!&/%!+$%%.!?)2&!;%%6!&/%!+(962&%$!$2..,.#B! ! ! F&1$&!&/%!$%+($5,.#!%=2,69%.&! usability test. 8*6##%*3#+,-#*)"#*)#()* ! M1.5(9,%!&/%!+%.1$,(!+1$5! ! You! 7/%+;!$%+$2,&9%.&!,!(.!+/%524%! in Microsoft get an editable version Word >"#0*)"#*./-)$./0)*/--=#(* when you sign up for refresher training ! E,.14,%!1.5!,2%!&%&!641.!! (free!) ! F%.5!(2&!166(,.&9%.&!&,9%A!916!1.5!-,5%(!+(.%.&! ! U1.1#%!%I6%+&1&,(.!G!%I641,.!&/%!%&&,.#! 0($9!&(!61$&,+,61.&! ! ";!&/%!61$&,+,61.&!&(!+(964%&%!1!-,5%(!+(.%.&!0($9! ! 7/%+;!&/%!6$(&(&<6%!266($&!&/%!+%.1$,(!G!9(5,0<! ! +%.1$,(!,0!.%+%1$<! >"#0*)"#*./-)$./0)*(*+0("#9* ! D0!1.<&/,.#!<(2>$%!2,.#!.%%5!)1&&%$,%A!#%&!61$%! ! ! ";!&/%!61$&,+,61.&!&(!+(964%&%!1!6(&S&%&! =2%&,(..1,$%! 8*9/4*3#+,-#*)"#*)#()* ! 7/%+;!,0!()%$-%$!/1-%!1.<!0(44(3S26!=2%&,(.! ! H$,.&!+%.1$,(!(.!,.5%I!+1$5!0($!%1,%$!/1.54,.#! ! H1<!&/%!,.+%.&,-%!1.5!#%&!1!,#.%5!$%+%,6&! ! H$,.&!&%&!+$,6&! ! F&(6!&/%!$%+($5,.#!G!.19%!&/%!$%+($5,.#!166$(6$,1&%4<!Usability !Test Plan Toolkit H$,.&!61$&,+,61.&!=2%&,(..1,$%!?J!6%$!61$&,+,61.&B! ! H$%61$%!0($!&/%!.%I&!61$&,+,61.&! ! H$,.&!-,5%(!+(.%.&!0($9!?61$&,+,61.&!91<!0($#%&!&(! ! )$,.#!&/%!(.%!<(2!%.&B! ?)*)"#*#09*,+*#/$"*9/4@(*)#()01* ! H$,.&!KD.&$2+&,(.!0($!()%$-%$L!

Purpose of usability test Product under test Participants & responsibilities Evaluation procedure Data collection Appendices - Screener, consent form, checklists, etc.Contents of the Test Plan

Step 1: Use personas - If you don’t have any, create assumption personas based on low / high technological expertise and low / high domain knowledge Step 2: Go where your users congregate - Friends & family - Customer lists - Professional societies - LinkedIn, Twitter - Recruitment agencies Step 3: Filter with an online screener Step 4: Follow up by phoneRecruiting participants

Portable test lab The moderator takes the test to the participant and sits at the participant’s desk or cubicle. If observers are present, they squeeze in and sit where they can.A taxonomy of tests

Single room setup The moderator and participant sit at a desk in a small ofﬁce. Observers sit at the back of the room and either crane their necks or watch the test on a separate computer running screen sharing software. Observers are close enough to hear what the moderator and participant say to each other.A taxonomy of tests

Classic testing lab setup The moderator and participant sit in one room. Observers can see the moderator and participant through a one-way mirror and hear their conversation through the lab’s built-in audio system. Observers get a close-up view of the participant’s screen via screen sharing software.A taxonomy of tests

Classic benchmark test The participant sits in one room and works alone. The moderator and observers can see the participant through a one-way mirror and get a close-up view of the participant’s screen via screen sharing software.A taxonomy of tests

Multi-room setup The moderator and participant sit in one room. Observers sit in another room elsewhere in the building. Observers watch the participant’s screen via screen sharing software projected on the wall and hear the participant through the computer’s speakers.A taxonomy of tests

Remote moderated test The participant carries out the test from his or her home or ofﬁce desk. The moderator uses screen sharing software to see the participant’s screen and presents the tasks to the participant over the phone. Observers use conferencing software to view the test over the Internet.A taxonomy of tests

Remote unmoderated test The participant carries out the test from his or her home or ofﬁce desk. The computer presents the tasks to the participant and the participant signals when he or she has completed the task. Nobody observes or moderates the test.A taxonomy of tests

How to Plan and Moderate aUsability TestWriting Test Tasks

Roads in London with red lines on them are known as red routes: these are the key road arteries in London. Transport for London does everything it can to keep these routes clear. Your product or web site has “red routes” too. They are the critical “user journeys” with your product.Red routes

What are the Red Routes for a university web site? - Can I study [subject]? - What does it cost? - What qualiﬁcation will I earn? - How will this help me in my job or career? - Where do I go to take this course? - Can I study part time, while working?Red Route Examples (from Gerry McGovern (2010), The Stranger’s Long Neck).

What are the Red Routes for a government web site that aims to help small business? - What grants are available to my company? - How do I apply for funding? - What funding is available to a start-up? - How do I apply for funding? - How can I reduce my tax liability?Red Route Examples (from Gerry McGovern (2010), The Stranger’s Long Neck).

What are the Red Routes for a health web site? - Based on my symptoms, what disease/ condition do I have? - What are the basic fact about [condition / disease]? - How do I book an appointment online? - How can I get advice from a health professional? - Based on my symptoms, what should I do next?Red Route Examples (from Gerry McGovern (2010), The Stranger’s Long Neck).

You have 300 seconds. Think about a product, system or service that you are working on at the moment. Brainstorm a handful of red routes — the ‘headline’ tasks.Brainstorm red routes

People approach tasks differently based on the context of use. - Imagine you need to book a flight and hotel in Paris for a 2-day trip with your partner. - Imagine you need to book a flight and hotel in Paris for a 2-day business trip. The red routes — finding a hotel, booking a flight — are the same but the context of use alters the way you would carry out these tasks. You build context into your test tasks with scenarios.The context of use

Red route Scenario “You’re fed up with the long journey to “Search for jobs by geographic work. Find all the suitable job area” openings within a reasonable commute of your home.” What are the implications of this scenario for the design? We can offer a better experience if we show “commuting time” as an option in the search results. And note that this may not be the same as distance, it depends on transport connections. You wouldn’t get this from the original red route which doesn’t have the contextual information.Expand your red routes into scenarios

Red route Scenario “Darren wants to buy a Sony PlayStation Portable and a suitable game for his 12 “Buy a PSP” year-old son for his birthday this Monday. Because of the short notice, he won’t buy the product unless he can collect it from a store before his son’s birthday.” What are the implications of this scenario? • We need to offer cross-channel support (buy online, pick up in store) • We need to be able to show shop stock levels • We need to show store locations and directions relative to the customer’s location • We need to offer a reservation system so the PSP can be put aside • We need to cross sell suitable games by age group, or at least classify games according to age group andExpand your red routes into scenarios gender

Does it describe a Is it speciﬁc and complete activity Is it really a red route? measurable? (integrated, not simple tasks)? Does it describe what Does it include Is the task “portable” the user wants to do enough information to to competitor (not how the user will complete the task yet products? do it)? avoid hidden clues?Six-step scenario checklist

You have 300 seconds. Take one of the red routes you created earlier. Use it to create a scenario — build in the ‘context’ information.Create a scenario

How to Plan and Moderate aUsability TestHow to moderate a usability test

Flight Attendant - Safeguard the physical, psychological and emotional wellbeing of the participant. • Greet participants, hang up their coats, offer a drink, engage in small-talk • Monitor participants for signs of stress • Provide reassurance and assistance as needed • Obtain informed consentHats moderators wear in a usability test (based on Carolyn Snyder’s Paper Prototyping)

Before starting — let participants know they are going to be listened to and that their opinions are valued. Observe and read the participant’s non-verbal behaviour — movement, facial expressions, etc. Reinforce the participant by nodding or paraphrasing Take some notes, but don’t write all the time Reﬂect back what you think you heard and check your understanding. Ask open-ended questions Pace your talking speed to match the participant’s Listen to inconsistencies that may need to be challengedHow to be a good moderator

To what degree does my non- verbal behaviour show that I value the participant (e.g. am I shufﬂing papers)? What attitudes am I expressing non-verbally (e.g. am I focusing on the recording technology)? What attitudes am I expressing verbally (e.g. am I making the person feel like a “subject” rather than a “participant”)? What is stopping me from giving my full attention to this participant (e.g. noise, tiredness, boredom)?To be an effective listener, listen to yourself

What am I doing to handle these distractions? How intently am I listening to what the participant is saying? What am I doing to demonstrate my willingness to listen to the participant? How well am I reading the participant’s non-verbal behaviours? How careful am I not to over- interpret non-verbal behaviour?To be an effective listener, listen to yourself

- Saying, “Remember, we’re not testing you,” more than three times. - “Don’t worry, the last participant couldn’t do it either.” - “No-one’s ever done that before.” - “HA! HA! HA!” - “That’s amazing! I didn’t even know it could go in upside down!” - “Could we stop for a moment? Watching you struggle is making me tired.” - “I didn’t really mean you should press any button.” - “Yes, it’s normal for observers to laugh during a test.” - “Don’t feel bad, many people take 15 or 16 tries.” - “Are you sure you’ve used a computer before?”What NOT to say to participants (slightly adapted from Rubin & Chisnell’s Handbook of Usability Testing)

Sports Commentator - Ensure that observers get as much useful information from the test as possible.Hats moderators wear in a usability test (based on Carolyn Snyder’s Paper Prototyping)

Getting participants to think aloud Phase 1 — Give instructionsGetting participants to think aloud

Getting participants to think aloud Phase 2 — Demonstrate and let them practiceGetting participants to think aloud

Demonstrate thinking aloud with an example: Turn your phone to silent Get the participant to practice with a different example: Adjust the height of his or her chair Make sure your practice task is easy: you want the user to practice the technique and feel successful!Demonstrate and let them practice

Practise welcoming a participant - Set expectations and reassure your participant. - Ask for your participant’s consent to record the session. - Demonstrate thinking aloud. - Get your participant to practise thinking aloud.Project Activity: Quick activity

Getting participants to think aloud Phase 3 — Probe with open questionsGetting participants to think aloud

Keep talking… Tell me more about that What are you thinking right now? PHASE 3: probe with open questionsProbe with open questions

Tell me more about that What are you thinking right now? PHASE 3: probe with open questions it was easy! what was easy?Probe with open questions

it was easy! what was easy? is that how is that how its supposed you expected to work? it to work?Probe with open questions

dont ask users to extrapolate… Mmm, do I think that someone, somewhere is this a will find some use useful for this feature? feature? would this feature, as its currently presented, be valuable to the way you choose…instead, products today?focus onthe presentinstead of generic questions…

products today?focus onthe presentinstead of generic questions… Mmm, there are some good things and some bad things, so what do Is this a i think overall? good idea? was there anything you particularly liked or disliked about the what specifically?…focus on prototype?the specifics

dont be judgemental crikey, he thinks this dont you think this option should appear option would be on the home page and better if it was he wont like it if i say available on the otherwise. home page? is there any othera better place youd like to seequestion a feature like this?would be…donT encourage confabulation

a better place youd like to seequestion a feature like this?would be…donT encourage confabulation do i have my surprised face on again? i better you seem think of something surprised, surprising now. were you? was that what you expected? what did youbut keep the expect?participanttalking

Scientist - Responsible for avoiding test bias and recording the data. • Representative users. • Appropriate task scenarios. • Balanced design • Neutral questioning. • Where bias can’t be avoided, make a note. • Decide when to intervene to help a participant • Log the data • Record audio and video of the session. • Collect accurate measures of task completion, time on task and satisfaction.Hats moderators wear in a usability test (based on Carolyn Snyder’s Paper Prototyping)

! -#($!$*!8*49!7(+5.(5#! 0"&-8)*(98&?*()&-8*(98-&)*5"++6F& ;$54&(&*3&2)-$5$23-&>*1?&/239(29"&*,&,)(+-)2-$*3&*)& ! D,2!3*/!,&#)3)!#%(@&7#,!G!#+*./(5#!,$*/9$#77)+5! 233*?235"&231&21@(+-&$3-")25-$*3&255*)1$39/?6& B21M&DP*&?*(&82#"&-)*(>/"&%$-8&+"2)58GF&O**1M&DC"//&0"& ! :*+;$!4#3#+4!$0#!4#,)5+! 2>*(-&2&-$0"&%8"3&?*(&5*(/13=-&,$31&%82-&?*(&%")"& :5-&2+&2&3"(-)2/&2)-?&*(-+$1"&*,&-8"&1"+$93&-"20&A&?*(&2)"& +"2)58$39&,*)6F& -8")"&,*)&)"+"2)58.&3*-&-*&1","31&1"+$93+6& ! D>*)4!7*+5!(+4!*>#/79!*@&7#%!E.#,$)*+,! ! <($0!3*/!,#73=#+,*/)+5! B21M&DK8"3&-8$+&+$-"&%2+&1"+$93"1.&-8")"&%2+&1$+5(++$*3& ;)*>"&,()-8")&%8"3&2)-$5$23-+&5"3+*)&-8"$)&%*)1+&*)&>/20"& %8"-8")IF&O**1M&DN*%&5*(/1&-8$+&+$-"&>"+-&0""-&?*()& -8"0+"/#"+6& 3""1+GF& ! :#>#7*&!3/)#+479!/(&&*/$! ! :*+;$!(,2!&(/$))&(+$,!$*!/#=4#,)5+!$0#!)+$#/3(#! B"&%"/5*0$39&231&8"/,(/&-*&-8"&2)-$5$23-&%$-8*(-&>"$39& B21M&DK82-&%*(/1&>"&-8"&>"+-&%2?&-*&1"+$93&-8$+GF&O**1M&DH3& -**&"02-8"-$5&$,&2)-$5$23-&$+&3"92-$#"&-*%2)1&2&1"+$936& ?*()&"R")$"35".&%82-&*-8")&%">&+$-"+&1*&-8$+&%"//GF& ! ?/*>)4#!+#.$/(7!#+*./(5#@#+$! ! D,2!./)*.,!E.#,$)*+,!$*!#%&7*/#H!8.$!(>*)4! C)"2-&*+$-$#"&231&3"92-$#"&5*00"3-+&-8"&+20"&%2?6&DC8$+&$+& *+3/*+$($)*+! #")?&8"/,(/6&E*(=)"&1*$39&@(+-&%82-&%"&3""1&?*(&-*&1*F6& B21M&DP*&?*(&82#"&23?-8$39&"/+"&-*&+2?&2>*(-IGF&O**1M& DS23&?*(&-"//&0"&0*)"&2>*(-IGF& ! -#&#($! !"$-")2-"&5*00"3-+&#")>2-$0&-*&"35*()29"&"/2>*)2-$*36& ! I#$!8#0)+4!$0#!8#0(>)*./J! Use a checklist to remind you how DE*(&+2$1&?*(&(+(2//?&+-2)-&-8$+&2/$52-$*3&,$)+-GF& to phrase unbiassed questions. B21M&DN*%&%*(/1&?*(&)2-"&?*()&"R")$"35"&%$-8&-8"&+$-"GF& O**1M&DN*%&%*(/1&?*(&)2-"&?*()&"R")$"35"&%$-8&-8"&+$-"G& You’ll receive a copy of this with your second refresher training e-mail. ! 10#2!9*./!)+$#/&/#$($)*+! K8?GF& !"8)2+"&-8"&2)-$5$23-=+&+-2-"0"3-&$3&?*()&*%3&%*)1+&-*& "3+()"&255()25?6&DH&20&8"2)$39&?*(&+2?IF& ! ".@@(/),#!(+4!4/(K!(!*+7.,)*+! DT*&%82-&H&20&8"2)$39&?*(&+2?&$+&-82-&-8"&%2?&-8"&LB**4&2& ! A)+)@)B#!@*4#/($*/!4),$/($)*+,! ,25$/$-?=&$+&$0/"0"3-"1&8")"&$+&5*3,(+$39.&>(-&-8"&$1"2&$+& B"&2%2)"&*,&23?&233*?$39&*)&)""-$-$#"&9"+-()"+&231& +$0/"6F& ")21$52-"&-8"06& ! C+*./(5#!$0)+2=(7*.4! 6-07712*37* !"0$31&-8"&2)-$5$23-&-*&-8$34J2/*(16&DK82-=+&9*$39& ! D,2!3*/!(+!*>#/(77!(,,#,,@#+$! -8)*(98&?*()&8"21&2-&-8"&0*0"3-GF& DS23&?*(&+(002)$U"&?*()&"R")$"35"&%$-8&Q&-*12?GF&*)& ! :*+;$!3*/#!*&)+)*+,! DK82-&%*(/1&?*(&+2?&%")"&-8"&-*&-%*&+-)"39-8+&231& :+4&2>*(-&2)-$5$23-&*$3$*3+&%8")"&2)*)$2-".&>(-&1*3=-& %"243"++"+6F& ,*)5"&23&*$3$*3&$,&-8"&2)-$5$23-&1*"+3=-&,""/&+-)*39/?6&Usability Test Plan Toolkit ! -#E.#,$!*@@#+$,!/#5(/4)+5!$*&),!+*$!/(),#4!89! ! 17(/)39!(!$#/@!*/!*+#&$! @*4#/($*/! DK82-&1*"+&L+?+-"0=&0"23&8")"GF&DK82-&1*&?*(&52//&-8*+"& DK82-&5*00"3-+&1*&?*(&82#"&*3&-*$5+&%"&1$13=-&5*#")& -*12?GF&*)&DK82-&"/+"&+8*(/1&H&82#"&2+4"1GF&

For example: - Closed question: Don’t you think you should try the Back button? - Open question: How else might you return to the home page? Are you looking for search? Did you just click on the Submit button? Didn’t you see the link to the site map? Don’t you think you should look in the pull-down menu? Do the radio buttons make your options clear? Do you think the design of this page is good? Do you think people will ﬁnd this easy to use?Quick Exercise: Improve these closed questions

Ask the people observing the test to write a short description of each usability problem they see on a sticky note. One problem per sticky note. Identify the participant associated with the observation (e.g. P5). At the end of the day’s testing, work with the observers to group, prioritise and label the usability issues.Getting clients and the development team to log usability issues

As you watch the test, you should note down the participant’s behaviours and code your observations. Good note taking saves hours in analysis time.Why data logging is important

- Data logging ensures you note all behaviours, not just the ones that stand out (this helps reduce bias in your observations). - It helps you concentrate and focus on the session. - When lots of observations come at once, you will be able to just note the observation code — you can then review it later on the videotape.Why data logging is important

- When scanning your notes, the observation codes make it easy to distinguish one class of observation (e.g. the usability issues) from other observations. - It’s one of those things you’ll be glad you did when there are problems with the video recording (e.g. when the sound is poor or when the recording is corrupted).Why data logging is important

S Start task F Facial reaction (e.g. surprise) E End task A Assist from moderator X Usability problem Q Gives up or wrongly thinks ﬁnished * Video highlight — an “Ah-ha!” moment H Help or documentation accessed G Comment from P (general) M Misc (general observation by P Positive opinion logger) N Negative opinion C Clickstream (optional) B BugHow to classify your observation

S Start task B Bug E End task F Facial reaction (e.g. surprise) G General comment A Assist from moderator As a rule of thumb, you P N X Positive opinion Negative opinion Usability problem Q H M Gives up or wrongly thinks finished Help or documentation accessed Misc (general observation by logger) should average about one * Video highlight — an “Ah-ha!” moment C Clickstream observation per minute. Time Code Description But remember this is an average - Observations are a bit like buses (none for ages, then three come along at once). What you should write down: - The time - The class of observation - A short descriptionHow to log usability problems

USERFOCUS 180 Piccadilly, London, W1J 9HF Dr David Travis Managing Director Work 020 7917 9535 Mobile 07747 016132 Email david.travis@userfocus.co.uk Twitter @userfocusThank You!

http://www.ifixux.pl	144
http://lanyrd.com	51
http://uxlabs.pl	49
http://paper.li	14
http://user-experience-specialist.blogspot.in	7
http://user-experience-specialist.blogspot.com	4
http://user-experience-specialist.blogspot.se	4
http://user-experience-specialist.blogspot.de	2
http://twitter.com	1

Usability Testing Bootcamp

by David Travis

Accessibility

Categories

Upload Details

Usage Rights

9 Embeds 276

Statistics