• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
User Research & Usability Testing
 

User Research & Usability Testing

on

  • 1,255 views

 

Statistics

Views

Total Views
1,255
Views on SlideShare
1,254
Embed Views
1

Actions

Likes
1
Downloads
0
Comments
0

1 Embed 1

http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    User Research & Usability Testing User Research & Usability Testing Presentation Transcript

    • User Researchofand Experience Architects the User Usability Testing: Techniques and Tips for Optimizing Government Web Sites Toral Contractor, Director of User Research Kirsten Miller, Information Architect Presented to Potomac Forum Symposium Building Better Government Web Sites: From Strategy to Fulfillment March 30, 2010
    • Potomac Forum | March 30, 2010 Agenda User Experience Design User Research • Methods • Case Studies & Tips Usability Testing • Methods • Case Studies & Tips Project Lifecycle Wrap-Up NavigationArts, LLC 2
    • Potomac Forum | March 30, 2010 User Experience Design NavigationArts, LLC 3
    • User Experience Design Potomac Forum | March 30, 2010 User Experience (UX) Design User-centered design, which puts users rather than organizational or system-related considerations at the center of the design process, is the foundation of our UX practice. Information Architecture • Structure information and functionality for easy access • Design effective navigational systems and taxonomies Interaction Design • Design complex online operations so they are intuitive for users • Establish clear priorities for user action Creative Interface Design • Create an engaging, attractive and brand-building visual interface • Create a visual hierarchy to facilitate browsing and scannability NavigationArts, LLC 4
    • User Experience Design Potomac Forum | March 30, 2010 UX Design Inputs Usability Expertise • Expert heuristic assessment Business Goals • Stakeholder interviews Our focus today: User Research • Direct vs. indirect Usability Testing • Live site vs. prototype • Moderated vs. unmoderated • In-person vs. remote NavigationArts, LLC 5
    • Potomac Forum | March 30, 2010 User Research NavigationArts, LLC 6
    • User Research Potomac Forum | March 30, 2010 Direct Methods Contextual inquiry In a nutshell: • Interview and observe users in their own environment as they use the site and perform relevant tasks What do Intercept interviews YOU think? • “Hi. Would you like to help us improve your web site by answering a few questions?” Focus groups • Structured group interviews used to collect opinions and insights into user needs and perceptions User surveys • Collect a range of qualitative and quantitative data on user demographics, behavior and opinions Card sorts • Users sort, group and label site topics NavigationArts, LLC 7
    • User Research: Direct Methods Potomac Forum | March 30, 2010 Case Study: Contextual Inquiry Department of State – Bureau of Consular Affairs Intranet redesign • Shadowed employees at work • Inventoried common user tasks both on and off web site • Asked about browser bookmarks and paper documents kept close at hand • Observed ways of browsing and using information • Inventoried likes and dislikes about current site, wishlist for redesign Benefits: • Understand the environment they work in. Are they under a lot of stress while working? Do they have a short amount of time to search for something? • Opportunity to observe documents they have printed out at their desks for easy access and reference for common tasks. DEVELOPMENT UX DESIGN DEPLOYMENT SUPPORT INTEGRATION NavigationArts, LLC 8
    • Potomac Forum | March 30, 2010 Contextual Inquiry Tip: Pay Attention to the Sticky Notes! NavigationArts, LLC 9
    • User Research: Direct Methods Potomac Forum | March 30, 2010 Case Studies: Intercept Interviews M.D. Anderson Cancer Center • Interviewed patients and family members in the Learning Center who were seeking information or passing through on their way to New Patient Orientation. • Discovered two modes: Research mode (e.g., seeking information about a disease, the organization, or a doctor) and Task mode (e.g., quickly print a map or schedule). George Washington University • Asked students about their primary reasons for applying, attending or leaving GW • Gained understanding of current student perspectives and expectations of GW Benefits • Quick and economical way to sample a target population DEVELOPMENT UX DESIGN DEPLOYMENT SUPPORT INTEGRATION NavigationArts, LLC 10
    • Potomac Forum | March 30, 2010 NavigationArts, LLC 11
    • User Research: Direct Methods Potomac Forum | March 30, 2010 Case Study: Focus Group Nemours • Interviewed parents of child patients • Learned of two main types of needs with patient families: Long-term (when a child has cancer), and Short-term (when a child has a minor condition). • Learned that families weren’t expecting the hospital to provide information about diseases, but did want hospital-recommended or approved sources of information. Benefits • Able to interview a large group at once for their opinions, evaluation or formulation of a concept. • Group discussion triggers conversations and topics that may not occur in one-on-one interviews. • Pick up on common user language, terminology DEVELOPMENT UX DESIGN DEPLOYMENT SUPPORT INTEGRATION NavigationArts, LLC 12
    • Potomac Forum | March 30, 2010 Focus Group Tip: Get written feedback from participants before starting group discussion. NavigationArts, LLC 13
    • User Research: Direct Methods Potomac Forum | March 30, 2010 Case Study: User Survey Department of State – Bureau of Consular Affairs • ForeSee Survey implemented • Questions about user roles, goals, satisfaction, problems encountered, and suggestions for improvement • ACSI model questions allow customer satisfaction benchmarking against other ForeSee clients Benefits • Large sample of quantitative data and qualitative feedback to guide and support design decisions • Customer segmentation helps refine and prioritize user profiles NavigationArts, LLC 14
    • Potomac Forum | March 30, 2010 User Survey Tips: • Form questions around user tasks • Use survey logic to ask targeted questions to various audiences • Include some open-ended questions NavigationArts, LLC 15
    • User Research: Direct Methods Potomac Forum | March 30, 2010 Case Study: Card Sort The George Washington University • Open sort: High school students and prospective undergraduate students grouped and labeled undergraduate majors • Closed sort: Freshman evaluated main navigation of the website Nemours • Open sort: Groups of patient families categorized Nemours content to build the main navigation. • Learned that all the participants wanted a “My Account” area for easy access to payment options, history of patient activity, appointment information, and an easy way to communicate with the doctor. Benefits • Formulate and evaluate navigation that aligns to users’ mental model of the information DEVELOPMENT UX DESIGN DEPLOYMENT SUPPORT INTEGRATION NavigationArts, LLC 16
    • Potomac Forum | March 30, 2010 Card Sort Tips: • Conduct open card sort first • Then conduct closed sort to validate design NavigationArts, LLC 17
    • User Research Potomac Forum | March 30, 2010 Indirect Methods Web Analytics In a nutshell: • Analyze site traffic and user behavior Call center interviews and log analysis What are • Interview call center agents and review call THEY doing? logs, if available Social media sites • Explore groups that form around common user interests; review user behavior on your own social media sites/pages • Can also use for direct user research NavigationArts, LLC 18
    • User Research: Indirect Methods Potomac Forum | March 30, 2010 Case Study: Web Analytics Department of State – Bureau of Consular Affairs intranet redesign • Number of visitors • Most popular pages and directories • Most popular entry and exit pages • Top referrers/inlinks • Most popular search terms • Time on site • Pages per visit Benefits • Understand user behaviors and priorities • Set success metrics/benchmarks to evaluate post- launch • Identify areas for future qualitative research DEVELOPMENT UX DESIGN DEPLOYMENT SUPPORT INTEGRATION NavigationArts, LLC 19
    • Potomac Forum | March 30, 2010 Web Analytics Tip: Look at exit pages for areas to focus on for further qualitative research NavigationArts, LLC 20
    • User Research: Indirect Methods Potomac Forum | March 30, 2010 Case Study: Call Center Analytics NHTSA • Interviewed call center personnel about common questions and emails from the public • Gained insight into customer needs not currently being met by the web site Benefits • Call center agents are the front line for direct interaction with the public • Agents often walk customers through the web site and know where the pain points are DEVELOPMENT UX DESIGN DEPLOYMENT SUPPORT INTEGRATION NavigationArts, LLC 21
    • Potomac Forum | March 30, 2010 Call Center Analytics Tip: Ask agents about browser bookmarks and FAQ lists. NavigationArts, LLC 22
    • User Research: Direct & Indirect Methods Potomac Forum | March 30, 2010 Case Studies: Social Media Sites Indirect: Council of Residential Specialists • Networking is very important to realtors and therefore, social networking sites was a key source for information • Researched social networking sites to see what users are discussing, gain insight into their needs and interests Direct: Smithsonian • Help imagine a completely new and more engaging way for people from around the world to interact with the Smithsonian Institute online • Used YouTube to solicit feedback in forms of videos to understand what users wanted to see from Smithsonian DEVELOPMENT UX DESIGN DEPLOYMENT SUPPORT INTEGRATION NavigationArts, LLC 23
    • Potomac Forum | March 30, 2010 Social Media Tip: Sign up for Google alerts so you know what people are saying online about your organization. NavigationArts, LLC 24
    • Potomac Forum | March 30, 2010 Usability Testing NavigationArts, LLC 25
    • Usability Testing Potomac Forum | March 30, 2010 Usability Testing Techniques Live site vs. Prototype testing • Live site: Test participants interact with currently existing site • Prototype: Test participants interact with proposed design; can be low or high fidelity, paper or electronic Moderated vs. Unmoderated • Moderated: Test participants interact with system/prototype and with researcher/facilitator • Unmoderated: Test participants interact only with Mix and match! system/prototype; researcher may observe behavior and/or analyze data collected by system Remote vs. In-person • Remote: Test participant and researcher/facilitator in separate physical locations • In-person: Test participant and researcher/facilitator in same physical location NavigationArts, LLC 26
    • Usability Testing Potomac Forum | March 30, 2010 Case Study: Unmoderated, Remote, Prototype Test Department of State – travel.state.gov • Testing navigation menu prototypes using an online heatmap generator Benefits • Participants complete test in their own environment and whenever they have time • No need for travel or appointments • Inexpensive • Many tools available now for creating remote, unmoderated usability tests and analyzing data generated PLANNING & DEVELOPMENT DEPLOYMENT SUPPORT DISCOVERY INTEGRATION NavigationArts, LLC 27
    • Usability Testing Potomac Forum | March 30, 2010 Case Studies: Moderated, Live Site Test Remote: World Bank • Tested search feature on live public site with domestic and international users within the international development community and media • Conducted free exploration test in which users formulated their own scenarios true to the way they use the site In-person: Department of State - travel.state.gov • Set specific tasks with correct and incorrect outcomes; scored and analyzed results Benefits • Directly observe user behavior on the site; identify trouble spots • Draw out user opinions and insights about performing tasks while they are doing it DEVELOPMENT UX DESIGN DEPLOYMENT SUPPORT INTEGRATION NavigationArts, LLC 28
    • Potomac Forum | March 30, 2010 Moderated Live Site Test Tip: International users? Plan to work odd hours! NavigationArts, LLC 29
    • Usability Testing Potomac Forum | March 30, 2010 Case Study: Moderated, Remote, Prototype Test KidsHealth.org • Tested a low-fidelity, clickable prototype with children, teens, parents and teachers to evaluate design at wireframe stage Benefits • Users test in their environment or the context in which they would use the site • No need to travel • Lower participant incentives • Many tools on the market now to facilitate remote user testing PLANNING & DEVELOPMENT DEPLOYMENT SUPPORT DISCOVERY INTEGRATION NavigationArts, LLC 30
    • Potomac Forum | March 30, 2010 Moderated Remote Test Tip: Be prepared for distractions NavigationArts, LLC 31
    • Potomac Forum | March 30, 2010 Project Lifecycle NavigationArts, LLC 32
    • Project Lifecycle Potomac Forum | March 30, 2010 When? User Research Contextual inquiry Intercept interviews Focus groups User surveys Card sorts Web analytics Call center interviews, log analysis Social media sites Usability Testing Live site testing Prototype testing NavigationArts, LLC 33
    • Project Lifecycle Potomac Forum | March 30, 2010 When? User Research Contextual inquiry  Intercept interviews  Focus groups  User surveys      Card sorts   Web analytics      Call center interviews, log analysis   Social media sites   Usability Testing Live site testing   Prototype testing   NavigationArts, LLC 34
    • User Researchofand Experience Architects the User Usability Testing: Questions? Toral Contractor, Director of User Research Kirsten Miller, Information Architect Presented to Potomac Forum Symposium Building Better Government Web Sites: From Strategy to Fulfillment March 30, 2010