Usability Testing for Usable Accessibility
Upcoming SlideShare
Loading in...5
×
 

Usability Testing for Usable Accessibility

on

  • 2,001 views

 

Statistics

Views

Total Views
2,001
Views on SlideShare
2,001
Embed Views
0

Actions

Likes
2
Downloads
42
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Examples – dynamic updates on forms, images / proximity and things showing in the sidebars that aren’t necessarily easy to reach
  • Pick a volunteer to be the participantKate moderates.Keep it short
  • Whitney does the debrief.
  • Tell them aboutthe toolkit

Usability Testing for Usable Accessibility Usability Testing for Usable Accessibility Presentation Transcript

  • 1Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 1Usability Testing forUsable AccessibilityUKate WalserWhitney Quesenbery
  • 2Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 2HiKateInteraction design, usabilitytesting and slippingaccessibility under the radar.WhitneyUsers’ stories, plain language,discovered accessibility as aright in electionsWhat about you?
  • 3Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 3What do you want to learn today?What do you think usabilitytesting is?Does anyone in yourorganization do any sort ofusability testing?What stands in the way ofdoing usability?
  • 4Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 4Here’s what we plannedIntroduction to usability testingDemo of a usability testPlanning a usability testTake a deep breathFirst practice testDebrief and moderating skillsBreakTesting with people with disabilitiesPractice testsDebrief and applying what you learned
  • 5Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 5What is usability (testing)?1
  • 6Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 6Usability is about people
  • 7Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 7Different types ofpeople…doing differenttasks…have differentusability needsdepending ontheir goalsand tasksFor this person efficiency and effectiveness arethe most important dimensions.For this person, easy to learn and errortolerant are the most important dimensions.
  • 8Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 8Helpful models for thinking about usabilityA’s Ability, aptitude, attitude(what users bring to the site)E’s Efficient, effective, engaging, error-tolerant, easy to learn(dimensions of usability)Use Useful, usable (operable), desirable(how we define success)
  • 9Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 9Usability testing lets you see behaviorObserve People like the audience using thesite rather than asking them about it How easily they meet their goals What causes confusion or problemsInform design Lets you confirm or challengeassumptions about the design. Gives you the opportunity to fixproblems.Informal usability testing of votingmaterials at the Farmers Market inOlympia, Washington
  • 10Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 10How easily can we learn from users?We learn from usersby watching them use thesite or other materials. Don’t explain or demo. Watch what they do. Listen to their comments. Take their problemsseriously.Dona Vitale conducting a usability testin Chicago, 2005
  • 11Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 11A usability test labA formal lab includes a room wherethe participants work, and aseparate room for observers towatch.Sessions are often recorded onvideoMichigan State University Usability & Accessibility Centerhttp://usability.msu.edu/default.asp
  • 12Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 12Many optionsUsability testingcan happen indifferent forums,even farmersmarkets……and hallways, usingdevices or even paperand post-it notes asshown here at GeneralServicesAdministration andOpen University
  • 13Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 13Usability testing with people with disabilitiesHow will a person with disabilities use the product? Assistive technology use and settings Other resources they useHow usable is the product by people with disabilities? Content, forms without surrounding visual context, proximity Navigation Truly accessible vs. just meets the standardsHow engaging and effective is your content strategy? Images only vs. engaging context + images Search engine readiness (headings, link labels) Great content flow
  • 14Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 14Demo2
  • 15Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 15Your goalYou want to a group from Access to gofishing at a local park.Find a park that has a wheelchairaccessible fishing pier.What are the admission fees?
  • 16Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 16Planning a usability test3
  • 17Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 17Begin with your goalsSuccess − for youand your usersYour site visitorsgoals (and their reality)• Answer peoples questions aboutyour topic• Have people completea transaction correctlywithout calling• Get more people to fill outa form correctly without callingYourorganizationsgoals
  • 18Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 18Location and context: formal informalRecruiting: defined opportunisticActivities: instructed tasks free tasksQuestions: structured unstructuredData collection: observation task/data onlyResults quantitative qualitativeThe purpose of the test influences the plan
  • 19Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 19Think about what senses are most involved in the UXExample Component Senses Include peoplewho are…Website,Mobile- Visual interface- Links, menus- Forms- Vision- Way to navigate(mobility, speech)- Blind- Vision impaired- Mobility impairedPhone - Speech andaudio feedback- Hearing- Speech (or way toindicate option)- Deaf- Hearing impaired- Speech challengesMultimedia - Visual interface- Sound- Way to selectoptions- Vision- Hearing- Way to navigate(mobility, speech)- Blind- Vision impaired- Deaf- Hearing impaired- Speech challenges
  • 20Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 20What you need for a usability testWhat The material you want to test Scenarios, if it’s not just an exploratory “try it out” type testWhere A quiet room (Maybe. We know of very successfulusability testing in an open marketplace, a hangarat an air show, the chemotherapy center at the NIHClinical Center.)Who Moderator Observer/note-taker Users: 3 − 6 people, one at a time
  • 21Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 21The moderator runs the sessionImpartial, unbiased, observingNo teaching!Listen and watchOpen-ended questions: Why?How? What were you doing?Moderator roles:Flight attendant:Ensuring safety and comfortScientist:Planning,maintaining objectivity,managing data
  • 22Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 22Observers and note-takersWatch (and listen) quietly for whether the user Is confused by anything, or has to re-read information Has to correct mistakes, or backtrack to correct naviationTake good notes Write down what you see and hear. Don’t “translate” or put words inthe user’s mouth Be specific. Not "hes confused." But "he said he doesnt know whatAPR means.” Dont infer the users reasons for doing something. Dont solve problems while taking notes. Thats for later.It will take all your concentration to note what is happening.
  • 23Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 23Briefing the userThanks for trying out this […]. Your doing this will help improve thismaterial. Note: If you never use the words "test" or "evaluate" with the user,you will not have to say "We are not testing you."You can stop anytime.Your involvement will be confidential.If you get stuck or confused, say so.Please let me know what you are thinking as you use this …
  • 24Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 24Different ways of having people try out your siteAsk participants for their own scenarios. Have you used this site? What for? Please show me how you did thatwith this site. or Would you be likely to use this site? What for? Pleaseshow me how you would do that with this site.Give participants tasks you have written. Write scenarios that will have people use parts of the site you areworried about.Customize scenarios for the participants. Write general scenarios, but adapt them to what you know about theparticipant, letting them fill in details to make the task more relevant.Use both of these with their own scenario first.
  • 25Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 25Use think aloud during − or after the taskConsider asking people to “think aloud” as they work What they’re doing Why they’re doing itAssistive technology may make think aloud harder. If so, you can goback over the material (“retrospective”) Ask the participant to walk you through what they didand why. Use the material as a guide for the discussion. Ask if anything was confusing or frustrating.
  • 26Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 26Use simple techniques to help users keep goingIf the user says, “hmmm” or “oops” or “I wonder…” Say, “What questions do you have right now?”If you are doing "think aloud" and the user is silent for10 or 20 seconds (count!) Say, “What are you thinking?”If users stop because they think theyre doneor they are stuck (and you think there’s a problem) Summarize what you saw the user do. Ask "What would you do now?"
  • 27Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 27How easily can we learn from users?Usability testingdoes not have to be formal,lengthy, or expensive.You don’t need a formal laboratory 100s of participants special equipment (except foryour voting system) special recording systemsPoster from Washington State
  • 28Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 28Summary: When to do usability testsEarlyUnderstand usersand their goalsProjectInitiationDesign andDevelopmentProductReleaseIn UseMiddleEvaluate the sitethroughout thedevelopmentprocessEndEvaluate thefinished site tomeasure itssuccessOngoingWatch siteusage andtechnicalsupport forevidence ofproblems
  • 29Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 29Take any opportunity for a quick usability observationThe next time someone asksyou a question about your site,ask them to use it instead.Watch as they try to find theinformation.
  • 30Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 30First practice test25 minutes4
  • 31Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 31Plan your practice testPick something you want people to do.Review the script for briefing the participant.Write the task instructions.Starting point (desktop, a URL?)Phrase it in their languageWhat do you want to watch for?Remember to thank the participant at the end
  • 32Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 32Ready? Three roundsGet into a group of three.Take turns, so everyone tries each role.We’ll call time.
  • 33Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 33How did that go?What felt awkward?What worked well?Anything surprise you?
  • 34Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 34Moderating skills5
  • 35Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 35From earlier: The moderator runs the sessionImpartial, unbiased, observingNo teaching!Listen and watchOpen-ended questions: Why?How? What were you doing?Moderator rolesFlight attendantEnsuring safety and comfortScientistPlanning,maintaining objectivity,managing data
  • 36Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 36Moderator ResponsibilitiesOften, but not always, create the test plan and testing scenariosWelcome and support the participant Informative but not revealingAvoid accidentally influencing results Explain the objectives succinctly Helpful but not too helpful Try not to answer questions that affect outcomesObserve quietly and avoid distracting participants Thoughtful about taking notes Ask questions only when they realize participant is at a stoppingpoint or is ready to talk
  • 37Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 37Managing the customerOften, moderators also manage the customerteam whose products will be used in thesessions.
  • 38Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 38Prepare the customer teamDefine and prioritize objectives Address these in scenarios, probe questions, questionnaire Determine which are “must haves” and which are optionalDescribe to the customer the process, logistics Where they will sit (if they will attend) What you expect from them (no interruptions, other room, etc.)Explain how you will handle “edge cases” Late or no-show participants Technology woes Questions that arise during the sessions (and how to get moderator’sattention to ask)
  • 39Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 39A good moderator uses a test planCreates a test plan to useduring the session Manage the usability testinglike a mini-project Provide clear schedule withtasks and deadlines –moderator, customer, etc. Follow up to make sureeverything’s setCreating a test plan and schedulefor usability testing helps avoidmistakes and oversights.
  • 40Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 40A good moderatorPrepares Create a test script Practice on real participants Verify that everything is ready and there are plan B’sArrive early – “on time” is “late”Listen and watch with an open mind If you wonder, ask without bias – don’t jump to conclusions Story – boy with the black crayonsRespect the participant Watch for “trouble” indicators Stop as needed due to time, tech issues, or other factors
  • 41Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 41Expect most participants to be…Hesitant or nervous Are thier skills being tested? What if they don’t give the“right” answer?Ready to throw their friends andcolleagues under a bus beforethemselves! Give them an easy way toprovide their feedbackwithout worrying abouthurting feelings Ask what their friends orcolleagues will thinkSmiling man points to friend who is covering his face.Photo credit: “accuse,” Grey Worldhttp://www.flickr.com/photos/greyworld/
  • 42Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 42Great moderators make participants feel valued andimportantIf possible, confirm the sessionyourself Call or email the participant toconfirm, introduce themselves Make sure participant knowswhere to go Give participant a friendly faceto expectMemorize the scriptLearn negotiation tactics Maintain an open mind Manage challenges that arise(Handle surprises withoutdisruptionA little girl sitting in dentist’s chair is smilingfrom ear to ear after learning she has nocavities.Photo credit: “No Cavities!” daveparkerhttp://www.flickr.com/photos/daveparker/
  • 43Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 43Have a backup planPerson uses paper printout of screens to describe interactionand pathPhoto credit: “Usability Testing our Paper Prototypes” Jason de Runahttp://www.flickr.com/photos/jderuna/
  • 44Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 44Summary: ModeratorsPrepare and anticipate problemsObserve without influencing outcomesMake participants feel valued
  • 45Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 45Break15 minutes*
  • 46Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 46Practice tests45minutes6
  • 47Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 47Moderating and observing Conduct sessions the same way. Ask the participant to “pause” the AT or lower thevolume as they may have thoughts they want toshare. Remember that the AT becomes a large componentof the UX Avoid interrupting the user as they use the AT Remember to watch the cursor, not the mouse.
  • 48Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 48Get into teamsIf you are the moderator, use your practice test.If you are an observer, think aboutWhat you will watch forWhere to sit so you can see,but won’t distract the participant.Challenges observing this person’sAT or accessibility setup.Remember: focus on what happens, not analysisor recommendations
  • 49Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 49Planning a test with people withdisabilities15 minutes7
  • 50Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 50How did that go?What felt awkward?What worked well?Anything surprise you?
  • 51Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 51Consider how different AT and disabilities affectusability test logistics Recruiting Participants Scenarios and handouts Incentives A place to tests
  • 52Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 52RecruitingAllow more time to find participants Screener + disabilitiesLook for community groups and ask Meetup, Facebook, TwitterUse specialized recruiting services KnowbilityAccessWorks portalWork with outreach groups Advocacy groups Employment non-profit groups Colleges and universities
  • 53Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 53Outreach GroupsGroup Focus WebsiteAmerican Foundation for theBlindBlind, Vision loss http://www.afb.orgColumbia Lighthouse for theBlindBlind, Vision loss http://www.clb.org/Gallaudet University Deaf, Hearing loss http://www.gallaudet.edu/Governor’s Committee onEmployment of People withDisabilitiesVarious Search for your state name +“governor’s committee onemployment…”National Federation of the Blind Blind, Vision loss https://nfb.org/Rehabilitation centers Mobility impairments Search for rehabilitationcenters or programs in yourarea
  • 54Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 54KnowbilityAccessWorks Portalhttp://www.knowbility.org/v/service-detail/AccessWorks-Usability-Accessibility-Testing-Portal/3k/Knowbility’sAccessWorks portal gives users with disabilities a way to sign up to participate in onlinetests and explains to usability and marketing professionals that they can use Loop11’s onlinetesting and recruit users with disabilities.
  • 55Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 55Decide where, when to conduct testingRemoteConduct via online tools Accessible screen sharingsoftware (or) accessible onlineusability testingModerator may facilitate ormay just review resultslaterUser uses their own, native,familiar assistivetechnology and setupIn PersonConduct in a physical location Traditional lab Some other placeModerator sits with participantBe prepared to provide theassistive technologies orask users to bring theirs(including anypersonalized style sheetsfor web pages)
  • 56Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 56In-person: Environment & SetupLocation becomes even more critical Is it reachable by public transportation? Is the building itself accessible? Are the signs to find the room accessible?Room setup matters Is there space to navigate with wheelchairs or walking devices? Isthere space for a service animal to lie down near its human? Are tables or desks wheelchair accessible? If visually impaired, can the person sit close enough to the monitor toview the screen? Can you mirror the screen on another monitor forthe moderator to view? How close are other workstations if screen readers will be used?
  • 57Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 57For in-person tests,explain setup and ask about accommodationsExplain what will happen and ask about specifics “We usually provide handouts for users to read that describe thescenario. What’s the best way to provide that to you?”Describe the setup, and include accessibility specifics in yourconfirmation emails “We’re on the 3rd floor. There’s a ramp to the right of the front doors.The doors automatically open. There’s an elevator bay in the lobby toget to our floor.”Ask if they will bring anything Service animal (would they like a rest or water stop for the animal?)If refreshments will be served, ask about dietary restrictions
  • 58Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 58Remote / Online: Setup & UseEnsure that the testing and/or screen sharing application are trulyaccessible Can the user download / access them ahead of time to install anyplugins and test with their chosen assistive technologies? Can they turn on / off screen sharing? Can they access and use their assistive technology while using thescreen sharing capabilities? Can the remote moderator hear and see what they’re doing?Ask about the best way to interact with the participant Explain what will happen and ask about specifics “We usually provide the scenarios via the tool. What’s the best way toprovide that to you?”
  • 59Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 59ScheduleAdd time between sessions for Setup time Moderator breakDry runs are critical, preferrably with a representative user or else,with someone trying to reflect the different considerations Online: Try downloading / opening the tool, going to the scenario,talking with the moderator with AT on In-person: Try going from closest transportation spot to location, intobuilding to designated location, taking / using handouts
  • 60Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 60Tips for Moderating and ObservingAct like you normally would Treat them like you would any other colleague or friend Ask first before assuming they need helpLearn the AT Practice listening to JAWS or other assistive technology whilesomeone is talkingAsk first before assuming they need help Know when the AT will affect how you interact with the participantLook for general usability problems as well as those made worse bypoor accessibility or interaction with the AT Missing information, problems navigating can reveal problems foreveryone Look for issues caused by the presentation
  • 61Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 61Summary: Testing with People with DisabilitiesConsider logistics, location, and setupAllow more time for recruiting and between sessionsDo a dry run and practice observing
  • 62Usability Testing for Usable Accessibility – Whitney Quesenbery, Kate Walser 62Handbook of UsabilityTesting by Jeff Rubinand Dana Chisnellusabilitytestinghowto.blogspot.comwww.peachpit.com/promotions/promotion.aspx?promo=137602Rocket Surgery demonstration videowww.usability.govA guide to developing usable and useful web sites (from US Dept of HHS)www.howto.gov/web-content/usability/first-fridaysFirst Fridays Product Testing ProgramRocket SurgeryMade Easyby Steve Krugwww.sensible.comLearn more about usability testing
  • Whitney QuesenberyWQusability.comwhitney@wqusability.com908-617-1122Kate WalserCX Insightskwalser@cxinsights.com571-281-2626