SlideShare a Scribd company logo
1 of 49
Download to read offline
Case study from a Webinar presentation:

Combining Lab and Online Usability Testing:
                          Lessons Learned


                                                   May 2010
Overview




                   1.  Brief introduction to both research methods

                   2.  The power of combining methods

                   3.  Case study: Parallel studies

                   4.  Conclusion: Looking at the ‘big picture’




www.userzoom.com
Brief introduction to both research methods
Usability testing in the Lab




www.userzoom.com
Brief introduction to both research methods

Usability testing in the Lab  Benefits/pros & limitations/cons


                   +  The face-to-face  Catching the very details

                   +  Ability to ask questions/interact with the participant live

                   +  The ability to invite others to attend the sessions



                   −  Small sample size

                   −  Lack of natural environment/context

                   −  Cost +++




www.userzoom.com
Brief introduction to both research methods




                   Online Usability Testing

                   (a.k.a. Unmoderated Remote Usability Testing)




www.userzoom.com
Brief introduction to both research methods

Online usability testing




                                                  •  Hundreds of users can be tested
                                             •  Participation in the natural context…
                                           •  …from geographically spread locations
                                                    •  No human moderation needed
                           •  Our browser bar connects users with our secure servers
www.userzoom.com
Brief introduction to both research methods

Online usability testing




www.userzoom.com
Brief introduction to both research methods

Online usability testing  Benefits/pros and limitations/cons


                   +  Quantifying usability  Objective, statistically significant data

                   +  Automation of results, cost-effectiveness

                   +  Participation in the natural context



                   −  Lack of face-to-face

                   −  Can’t invite to observe sessions

                   −  Must anticipate participant’s questions in advance




www.userzoom.com
Overview




                   1.  Brief introduction to both research methods

                   2.  The power of combining methods

                   3.  Case study: Parallel studies

                   4.  Conclusion: Looking at the ‘big picture’




www.userzoom.com
Our UX Toolkit




www.userzoom.com                    10
Power of Combining Methods

                     Every research method has unique strengths and limitations

                     Gather new insights with each method

                     Greater confidence when observing similar findings through multiple methods


                   Which one goes first?
                    Lab first, then Online                    Online first, then Lab
                    Identify/fix “low hanging fruit”, then    Identify the most significant issues online
                    focus on remaining tasks with large       through metrics, then use lab study to
                    sample size                               gather deeper qualitative understanding
                                                              of those issues
                    Generate new concepts, ideas, questions   Collect video clips or more quotes of
                    through lab testing, then test/validate   users to help bring metrics to life
                    online
                    Validate attitudes/preferences observed   Gather all the metrics to validate design
                    in lab testing                            – if it tests well, then no need to bring
                                                              users into the lab

www.userzoom.com                                                                                   11
Overview




                   1.  Brief introduction to both research methods

                   2.  The power of combining methods

                   3.  Case study: Parallel studies

                   4.  Conclusion: Looking at the ‘big picture’




www.userzoom.com
Case study: Parallel studies


We conducted 2 parallel studies using each method. Basic details:




       •     Website tested: Amazon.com’s Grocery and Gourmet Food Store

       •     Lab study conducted by the DUC

       •     Online Usability Study conducted by UZ

       •     Date of the study: April 2010




www.userzoom.com
Case study: Parallel studies

Basic details of the studies (II)
       •     Objectives:

              •    What is the unique contribution of each method?

              •    What do we learn by combining methods?

       •     Participants: 10 for the lab study, 100 for the online study

       •     Study design:

              •    Initial questionnaire

              •    3 tasks

              •    Follow up questions

              •    Final questionnaire




www.userzoom.com
Case study: Parallel studies

Participant demographics from the Lab Usability Study:	
  
       •  Average age 29

       •  Age range between 19 and 51

       •  6 Males, 4 Females

       •  All had previously used Amazon.com website

       •  None had ordered groceries on-line

       •  Visited Amazon.com approximately every 2 weeks, on average

       •  Previously purchased books, electronics, toys, and more.

       •  Expected packaged, non-perishable grocery items to be sold on-line

       •  Most said they hadn’t bought groceries on-line because they, “like the experience of picking
           up the produce and meat.”

              •  Expressed concern about products being fresh

       •  A few worried about exorbitant service or shipping charges making it not worthwhile.

www.userzoom.com
Case study: Parallel studies



                   Task 1:


                      “You just ran out of dishwasher detergent. See if Amazon has a pack
                      of 6 Cascade Complete Dishwasher Detergent Powder, in the 45
                      ounce size.”




www.userzoom.com
Results of Locating Dishwashing Detergent	
  
                   Case study: Parallel studies

Search vs. Menu Navigation
      •  Approximately half of the participants wanted to immediately use SEARCH

Category Confusion
      •  Most likely categories were “Home & Garden”, “Grocery, Health & Beauty”




          •  Chose “Grocery, Health & Beauty” through process of elimination
          •  Further confusion within “Grocery, Health & Beauty”
          •  “I don’t know if ‘Grocery and Gourmet food’ will have detergent”

www.userzoom.com
Results of Locating Dishwashing Detergent	
  
                    Case study: Parallel studies
Confusion within “Grocery & Gourmet Food” menu
      •  First menu is 4 screens long
      •  Participants were confused where to look for dishwasher detergent




                                                                      Dishwasher Detergent
                                                                      found under here




www.userzoom.com
Results of Locating Dishwashing Detergent	
  
                    Case study: Parallel studies
Nesting Confusion
•  All participants were confused by path to find dishwashing detergent:
          •  Amazon.com > Grocery & Gourmet Food > All Household (page is titled ‘Health & Personal
             Care ‘) > All Household Supplies > All Dishwashing > Scroll to locate Cascade Dishwashing
             Detergent




          •  Participants scrolled up and down navigation menus
          •  A few gave up mid-way & used SEARCH
          “There are a lot of clicks to go through: In Household Cleaners, Kitchen Cleaners…I don’t see
            dishes…maybe by brand? I don’t see Cascade here. I would do SEARCH.”
          “That took probably 5-6 minutes. For me, that’s really long.”
www.userzoom.com
Case study: Parallel studies



                   Task 2:


                      “You want to buy Skippy Peanut Butter. Find out if there are any
                      discounts or coupons for it.”




www.userzoom.com
Results of Locating Skippy Peanut Butter Coupon	
  
                   Case study: Parallel studies
Product Search vs. Coupon Search
•  A few looked on Amazon.com home page for Today’s Deals
•  Most looked for the product in the “Grocery” section, and then looked for a coupon
•  Some looked at “Special Offers” in the “Grocery” screen
      •  Special Offers led to a coupon and New and Used food, which disturbed participants.




www.userzoom.com
Results of Locating Skippy Peanut Butter Coupon
                      Case study: Parallel studies
Participants that looked for the product complained about the length of the menu on
  the left side of the grocery page
      •  Peanut Butter, under “Sauces and Dips,” was four screens down plus 2 clicks. “It was tedious.”




www.userzoom.com
Results of Locating Skippy Coupon
                   Case study: Parallel studies

Users had strong feelings about coupons for grocery items

“I would expect they’d show me the price they have, it doesn’t matter if it is cheaper.”
“If I am purchasing a big purchase, I would do it from Google. I would look for coupons [only] for big
   items.”
“I would have preferred to see a link from the home page– specials, coupons, etc…I don’t want to hunt
   down each thing.”
“It was not clear that a discount was available until you went to the [item] description page”
“I had to look through a lot of things."




www.userzoom.com
Case study: Parallel studies



                   Task 3:


                      “Find out how you would schedule the same groceries to be
                      delivered every 3 months.”




www.userzoom.com
Scheduling Groceries for Repeated Delivery	
  
                    Case study: Parallel studies

All found the correct starting place: the “Grocery & Gourmet Food” page
      •  Most noticed the “Shop Subscribe and Save” link
      •  Others tried to schedule each item from the item’s page
              “I think you will have to do it for every item.”
              “I would probably go to the product and check if I can schedule it.”
              “[a product page] might say, “do you want this delivered?”




www.userzoom.com
Scheduling Groceries for Repeated Delivery
                   Case study: Parallel studies
Participants were confused on how to schedule grocery delivery
      •  Most missed the delivery scheduling link at the top of the page




                                                                           Delivery
                                                                           Scheduling
                                                                           Information
                                                                           Link




      “The text was small, and I wasn’t sure I was in the right place.”
      “The grayed-out part doesn’t jump out”
www.userzoom.com
Conclusions from the Lab Study	
  
                    Case study: Parallel studies


Confusing IA
    “Dishwasher detergent was under “Health [& Personal Care], which is not the first place I’d look.””
    “There were too many categories, it confuses you…”
    “It is not as well organized as other parts of Amazon.”
    “I don’t know why ‘All Household Supplies’ is under ‘Health and Personal Care’”


Difficult navigation
    “Navigation wasn’t intuitive. The left-hand-side menu was so far down the page and not duplicated
    in the banner.”
    “The browse list was too long. I don’t want to go through 10 chocolates before going to the pasta.”
    “It didn’t feel intuitive.”
    “I didn’t like how many levels I had to go through.”
    “It was hard to find things. There were long lists to find food items.”




www.userzoom.com
Case study: Parallel studies




                   Highlighted findings from the Online Usability Study:




www.userzoom.com
Study Background


                   - 71% Male, 29% Female
                   -  Age range distributed between 18–55; 88% fell between 26-45
                   - All have purchased from Amazon.com and visit Amazon once a week,
  Profile          month or every few months.
                   -  Eighty-eight percents of participants purchase groceries from in the
                   store only, 6% online only and 6% both online and in a store.
                   -  Main reasons for considering purchasing groceries online is for
                   convenience and time.




                    - April 17th to April 26, 2010
  Detail            -  Unmoderated remote usability testing method
  s                 -  100 participants
                    -  Recruited through participant panel




www.userzoom.com                                                                             29
Overview of tasks and validation criteria

                   1.  Locate specific item: You just ran out of dishwasher detergent. Find Cascade
                       Complete Dishwasher Detergent Powder, 45 oz (pack of 6).


                       Validation by reaching the correct page with the detergent (URL validation)


                   2.  Finding a discount code: You want to buy Skippy Peanut Butter with a discount
                       you heard about on Amazon.com. Please write down the discount code.


                       Validation by choosing the correct discount code (multiple choice)


                   3.  Schedule grocery delivery: Find out how often you can schedule a delivery to your
                       home.


                       Validation by how often you can set up a delivery (multiple choice)




www.userzoom.com                                                                                 30
Study Design

      Initial Questionnaire:                                      Post Task:

      How frequently do you visit Amazon.com?                     Validation by url or question (Success, Error, Abandon
                                                                  ratios collected).
      Where do you usually buy your groceries?
                                                                  Ease of Use.
      Why have you considered purchasing groceries online?
                                                                  How intuitive links and menus were and how easy it was to
                                                                  start the search.

                                                                  Issues and problems experienced while completing tasks.



      Final Questionaire:

      Overall satisfaction

      Satisfaction with navigation, product search, information
      offered and look and feel of the site.

      Likelihood to purchase groceries from Amazon.com.

      Likelihood to recomment to a friend.




www.userzoom.com                                                                                                      31
T1: Locating Cascade Complete




                                        Success
                                        Non-Success
                                                      I never felt lost on the site
                                                                                                   77%
                                                            while searching


                                                      The menus and links were
                                                                                      72%
                                                           intuitive to use

                                                        It was clear how to start
                                                            searching for the           73%
  Participants did feel particularly lost on the          dishwasher detergent
  site (most likely due to experience using
  Amazon.com.

  However they did not feel the menus and
  links were extremely intuitive and weren’t
  always sure were to start their search.




www.userzoom.com                                                                              32
T1: Locating Cascade Complete




                                                                 Highlights	
  dominant	
  path	
  

    Over half (51%) of the participants defaulted to using    *3%	
  insignificance	
  removed	
  
    search indicating the menus and navigation of the site
    were not very intuitive.

    Fifteen percent started in Health and Personal while 5%
    looked in promotional deals.

www.userzoom.com                                                                        33
T1: Locating Cascade Complete

     What participants found difficult when attempting the task:




     Exact Quotes:

     “ Hard to navigate to find desired product. Too long and was getting frustrating.”

     “ There were a lot of results so you have to type it in exactly or spend a couple minutes searching.”

     “ I didn’t see a soap category and didn’t realize how the search function worked.”

     “ I didn’t know where to look.”




www.userzoom.com                                                                                   34
T2: Finding a discount code

                                                Success
                                                Non-Success




         Extremely Difficult	

                                  Extremely Easy 	

              Forty-seven percent of the participants thought it was very difficult to locate the
              discount code. They thought it was very unclear how to start searching, it was not
              where they expected it to be and were somewhat lost during the search process.

www.userzoom.com                                                                                    35
T2: Finding a discount code




www.userzoom.com                                 36
T2: Finding a discount code




          Error participants (55%) could not locate the code, thought it was too difficult to find
          the code or the peanut butter and stated:

          “ I was expecting the discount in the results page, or the product page. I ended up
          looking for it under ’specials’, but I don’t know if that’s the best place for it.”

          “ Too much searching and clicking.”

          “ Looked under special offers having firstly used search, had to notice the offer code
          above product within the offer description.”


www.userzoom.com                                                                                     37
T3: Schedule Grocery Delivery

                                              Success
                                              Non-Success




                             	

           Extremely Difficult                                 Extremely Easy 	

     Seventy-five percent of participants could not complete the task. They did not think it was clear
     how to start searching and felt lost during the search process. Forty percent of participants who
     were successful, felt it was very difficult to complete the task.



www.userzoom.com                                                                                  38
T3: Schedule Grocery Delivery




     Abandon participants (38%) could not find the information, thought the menus were difficult to
     understand and the task was taking too long.

     “ It was not obvious to me that I needed to click on the link at the top right of the screen to get
     this information until I had spent quite a bit of time looking all over the rest of the screen.”

     “ I wasn’t certain where to look for the information.”

     “ Nothing about this process has been user friendly.”


www.userzoom.com                                                                                    39
Final Questionnaire




                    Fifty-three percent of participants thought the navigation
                    was very difficult, 39% thought it was difficult to search for
                    a product, forty-seven percent rated the overall look and
                    feel of the site as very poor.



www.userzoom.com                                                                     40
Final Questionnaire

                                                         Fifty-five percent of participants stated
                                                         they would not purchase their groceries
                                                         in the future from Amazon.com.




                        Net Promoter Score (NPS) calculation:
                        0 to 6 = Detractors
                        7 to 8 = Passive
                        9 to 10 = Promoters
                        Net Promoter Score (NPS) = % of Promoters - % of Detractors


www.userzoom.com                                                                                 41
Overview




                   1.  Brief introduction to both research methods

                   2.  The power of combining methods

                   3.  Case study: Parallel studies

                   4.  Conclusion: Looking at the ‘big picture’




www.userzoom.com
Conclusions: Looking at the ‘big picture’


                   •    Main, obvious issue:



                                  Gourmet store was built like the book store


                   •    IA + navigation + labeling + content problems/issues encountered



                   •    Both research methods prove this:

                         1.  By observing & listening at the Lab we learned a lot, but also…

                         2.  …we quantified how big the issues were using UserZoom




www.userzoom.com
Conclusions: Looking at the ‘big picture’




www.userzoom.com
The introductions…




                   Alfonso de la Nuez        Kim Oslob              Bill Albert
                     Partner & Chief    Research & Product   Director of the Design &
                    Marketing Officer    Strategy Director       Usability Center,
                      at UserZoom	

       at UserZoom	

      Bentley University	





www.userzoom.com
The introductions

                     Leading online user experience research software company
                     Develops proprietary on-demand software app for online research

                     Built for & by UX & marketing professionals

                     Offers a cost-effective, time-saving methodology

                     Has 9 years of experience in UX research & consulting

                     Has offices in Sunnyvale (CA), London (UK) & Barcelona (Spain)	





www.userzoom.com
The introductions


                     The Design and Usability Center at Bentley University was founded in 1999

                     Provides UX consulting services to corporate clients

                     Supports the MS program in Human Factors in Information Design

                     Focuses on user experience research and evaluation




www.userzoom.com
Acknowledgements

                     Special thanks to Cynthia Kamishlian
                   and Dharmesh Mistry from Bentley
                   University   for   carrying   out   this
                   research



                     Based on our recent book “Beyond
                   the Usability Lab”, co-authored with
                   Tom Tullis and Donna Tedesco




www.userzoom.com                                              48
º
                                  Thanks so much for your time!

                                          Oh! Don’t forget:

       The slides and the video of this webinar will be available in a few days in our blog

                                          www.userzoom.com




        USA                                UK                            SPAIN
        440 N. Wolfe Rd.                   Aylesbury House, 17 – 18      Av. Diagonal 419 3º 2ª
        Sunnyvale, CA 94085 USA            Aylesbury Street, London      08008 Barcelona, Spain
                                           ECIR 0DB, UK
        Phone: +1 (408) 524 – 74 45                                      Phone: +34 93 414 7554
        Contact: Alfonso de la Nuez        Phone: +44 (0) 20 7193 2171   Cell: +34 93 368 4295
        alfonso@userzoom.com               Cell: +44 (0) 7900 472 920    Contact: Javier Darriba
                                           Contact: Arthur Moan          jdarriba@userzoom.com
www.userzoom.com                           amoan@userzoom.com                                      49

More Related Content

What's hot

Crafting Product Strategy Blueprint for Success by Atlassian PM.pdf
Crafting Product Strategy Blueprint for Success by Atlassian PM.pdfCrafting Product Strategy Blueprint for Success by Atlassian PM.pdf
Crafting Product Strategy Blueprint for Success by Atlassian PM.pdfProduct School
 
UX Research by Lisandra Maioli
UX Research by Lisandra MaioliUX Research by Lisandra Maioli
UX Research by Lisandra MaioliLisandra Maioli
 
Recommender Engines
Recommender EnginesRecommender Engines
Recommender EnginesThomas Hess
 
Case Study 3 - Portfolio Project Final - Google UX Design Certificate
Case Study 3 - Portfolio Project Final - Google UX Design CertificateCase Study 3 - Portfolio Project Final - Google UX Design Certificate
Case Study 3 - Portfolio Project Final - Google UX Design CertificateAbelKCS
 
Práticas Ágeis para bons Product Owners
Práticas Ágeis para bons Product OwnersPráticas Ágeis para bons Product Owners
Práticas Ágeis para bons Product OwnersPedro Paulo Oliveira
 
UX Research within an Agile Design and Development Sprint Cycle
UX Research within an Agile Design and Development Sprint CycleUX Research within an Agile Design and Development Sprint Cycle
UX Research within an Agile Design and Development Sprint CycleUXPA International
 
Conducting User Research
Conducting User ResearchConducting User Research
Conducting User ResearchJeremy Horn
 
The Elements Of User Experience
The Elements Of User ExperienceThe Elements Of User Experience
The Elements Of User ExperienceJohn Chen, Jun
 
Heuristic Evaluation - UX Approach
Heuristic Evaluation - UX ApproachHeuristic Evaluation - UX Approach
Heuristic Evaluation - UX ApproachGanesan Arjunan
 
Fundamentals and practices of UX research
Fundamentals and practices of UX research Fundamentals and practices of UX research
Fundamentals and practices of UX research Lucia Trezova
 
Psychology for UX and Human Experience
Psychology for UX and Human ExperiencePsychology for UX and Human Experience
Psychology for UX and Human ExperienceDave Hogue
 
UX Design - Client presentation
UX Design - Client presentationUX Design - Client presentation
UX Design - Client presentationMarta Fioni
 
Recommender systems for E-commerce
Recommender systems for E-commerceRecommender systems for E-commerce
Recommender systems for E-commerceAlexander Konduforov
 
User story slicing
User story slicing User story slicing
User story slicing Ankit Tandon
 
How to improve your product sense?
How to improve your product sense?How to improve your product sense?
How to improve your product sense?manjeetjakhar
 
Agile UX Research – Ben Ralph
Agile UX Research – Ben RalphAgile UX Research – Ben Ralph
Agile UX Research – Ben RalphBen Ralph
 
Movie lens recommender systems
Movie lens recommender systemsMovie lens recommender systems
Movie lens recommender systemsKapil Garg
 

What's hot (20)

Crafting Product Strategy Blueprint for Success by Atlassian PM.pdf
Crafting Product Strategy Blueprint for Success by Atlassian PM.pdfCrafting Product Strategy Blueprint for Success by Atlassian PM.pdf
Crafting Product Strategy Blueprint for Success by Atlassian PM.pdf
 
UX Research by Lisandra Maioli
UX Research by Lisandra MaioliUX Research by Lisandra Maioli
UX Research by Lisandra Maioli
 
Recommender Engines
Recommender EnginesRecommender Engines
Recommender Engines
 
Case Study 3 - Portfolio Project Final - Google UX Design Certificate
Case Study 3 - Portfolio Project Final - Google UX Design CertificateCase Study 3 - Portfolio Project Final - Google UX Design Certificate
Case Study 3 - Portfolio Project Final - Google UX Design Certificate
 
Práticas Ágeis para bons Product Owners
Práticas Ágeis para bons Product OwnersPráticas Ágeis para bons Product Owners
Práticas Ágeis para bons Product Owners
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
 
UX Research within an Agile Design and Development Sprint Cycle
UX Research within an Agile Design and Development Sprint CycleUX Research within an Agile Design and Development Sprint Cycle
UX Research within an Agile Design and Development Sprint Cycle
 
Conducting User Research
Conducting User ResearchConducting User Research
Conducting User Research
 
The Elements Of User Experience
The Elements Of User ExperienceThe Elements Of User Experience
The Elements Of User Experience
 
Basics of UX Research
Basics of UX ResearchBasics of UX Research
Basics of UX Research
 
User Stories explained
User Stories explainedUser Stories explained
User Stories explained
 
Heuristic Evaluation - UX Approach
Heuristic Evaluation - UX ApproachHeuristic Evaluation - UX Approach
Heuristic Evaluation - UX Approach
 
Fundamentals and practices of UX research
Fundamentals and practices of UX research Fundamentals and practices of UX research
Fundamentals and practices of UX research
 
Psychology for UX and Human Experience
Psychology for UX and Human ExperiencePsychology for UX and Human Experience
Psychology for UX and Human Experience
 
UX Design - Client presentation
UX Design - Client presentationUX Design - Client presentation
UX Design - Client presentation
 
Recommender systems for E-commerce
Recommender systems for E-commerceRecommender systems for E-commerce
Recommender systems for E-commerce
 
User story slicing
User story slicing User story slicing
User story slicing
 
How to improve your product sense?
How to improve your product sense?How to improve your product sense?
How to improve your product sense?
 
Agile UX Research – Ben Ralph
Agile UX Research – Ben RalphAgile UX Research – Ben Ralph
Agile UX Research – Ben Ralph
 
Movie lens recommender systems
Movie lens recommender systemsMovie lens recommender systems
Movie lens recommender systems
 

Viewers also liked

Usability Study: Gogobeans.com
Usability Study: Gogobeans.comUsability Study: Gogobeans.com
Usability Study: Gogobeans.comPuja Parakh
 
Last.fm Usability Presentation
Last.fm Usability PresentationLast.fm Usability Presentation
Last.fm Usability PresentationRobert Sherron
 
Our Usability Study Process
Our Usability Study ProcessOur Usability Study Process
Our Usability Study ProcessJune UX
 
MOHAI Usability Study Presentation
MOHAI Usability Study PresentationMOHAI Usability Study Presentation
MOHAI Usability Study PresentationSusan Oldham
 
UX Cambridge 2014 - Usability testing with young children
UX Cambridge 2014 - Usability testing with young childrenUX Cambridge 2014 - Usability testing with young children
UX Cambridge 2014 - Usability testing with young childrenMonica Ferraro
 
Amazon's User Experience
Amazon's User ExperienceAmazon's User Experience
Amazon's User ExperienceJorge Franco
 
Why UX #FAILS (with notes)
Why UX #FAILS (with notes)Why UX #FAILS (with notes)
Why UX #FAILS (with notes)Chris Feix
 
Lapcodex Aviones 1
Lapcodex Aviones 1Lapcodex Aviones 1
Lapcodex Aviones 1guest671e5e0
 
Reputation snapshot for the banking industry, 2012, final
Reputation snapshot for the banking industry, 2012, finalReputation snapshot for the banking industry, 2012, final
Reputation snapshot for the banking industry, 2012, finalDamjana Kocjanc
 
Eu Kyoto Prototcol Class Presentation Sumiko
Eu Kyoto Prototcol Class Presentation SumikoEu Kyoto Prototcol Class Presentation Sumiko
Eu Kyoto Prototcol Class Presentation SumikoKeith Dickson
 
New Directions In Russian Private Equity
New Directions In Russian Private EquityNew Directions In Russian Private Equity
New Directions In Russian Private EquityThomas Nastas
 
20121119 Csusm Business Br
20121119 Csusm Business Br20121119 Csusm Business Br
20121119 Csusm Business BrSteveScheibe
 
Dokumentacia pechatni materiali
Dokumentacia pechatni materialiDokumentacia pechatni materiali
Dokumentacia pechatni materialiNural Tataoglu
 

Viewers also liked (20)

Usability Study: Gogobeans.com
Usability Study: Gogobeans.comUsability Study: Gogobeans.com
Usability Study: Gogobeans.com
 
Last.fm Usability Presentation
Last.fm Usability PresentationLast.fm Usability Presentation
Last.fm Usability Presentation
 
Our Usability Study Process
Our Usability Study ProcessOur Usability Study Process
Our Usability Study Process
 
Mobile Healthcare App
Mobile Healthcare AppMobile Healthcare App
Mobile Healthcare App
 
MOHAI Usability Study Presentation
MOHAI Usability Study PresentationMOHAI Usability Study Presentation
MOHAI Usability Study Presentation
 
E-GROCERY
E-GROCERY E-GROCERY
E-GROCERY
 
MatBOT
MatBOTMatBOT
MatBOT
 
UX Cambridge 2014 - Usability testing with young children
UX Cambridge 2014 - Usability testing with young childrenUX Cambridge 2014 - Usability testing with young children
UX Cambridge 2014 - Usability testing with young children
 
Amazon's User Experience
Amazon's User ExperienceAmazon's User Experience
Amazon's User Experience
 
UI/UX Design
UI/UX DesignUI/UX Design
UI/UX Design
 
Why UX #FAILS (with notes)
Why UX #FAILS (with notes)Why UX #FAILS (with notes)
Why UX #FAILS (with notes)
 
From Paths to Sandboxes
From Paths to SandboxesFrom Paths to Sandboxes
From Paths to Sandboxes
 
Lourenza
LourenzaLourenza
Lourenza
 
Lapcodex Aviones 1
Lapcodex Aviones 1Lapcodex Aviones 1
Lapcodex Aviones 1
 
Reputation snapshot for the banking industry, 2012, final
Reputation snapshot for the banking industry, 2012, finalReputation snapshot for the banking industry, 2012, final
Reputation snapshot for the banking industry, 2012, final
 
Pdf Drawings New
Pdf Drawings NewPdf Drawings New
Pdf Drawings New
 
Eu Kyoto Prototcol Class Presentation Sumiko
Eu Kyoto Prototcol Class Presentation SumikoEu Kyoto Prototcol Class Presentation Sumiko
Eu Kyoto Prototcol Class Presentation Sumiko
 
New Directions In Russian Private Equity
New Directions In Russian Private EquityNew Directions In Russian Private Equity
New Directions In Russian Private Equity
 
20121119 Csusm Business Br
20121119 Csusm Business Br20121119 Csusm Business Br
20121119 Csusm Business Br
 
Dokumentacia pechatni materiali
Dokumentacia pechatni materialiDokumentacia pechatni materiali
Dokumentacia pechatni materiali
 

Similar to Case study: Lab + Online Usability Testing

Webinar lab + ous presentation may10
Webinar lab + ous presentation may10Webinar lab + ous presentation may10
Webinar lab + ous presentation may10guestbb908d
 
Webinar: Combining Lab and Online Usability Testing: Lessons Learned
Webinar: Combining Lab and Online Usability Testing: Lessons LearnedWebinar: Combining Lab and Online Usability Testing: Lessons Learned
Webinar: Combining Lab and Online Usability Testing: Lessons LearnedUserZoom
 
CIS375 Interaction Designs Chapter13
CIS375 Interaction Designs Chapter13CIS375 Interaction Designs Chapter13
CIS375 Interaction Designs Chapter13Dr. Ahmed Al Zaidy
 
Evaluation methods
Evaluation methodsEvaluation methods
Evaluation methodsEva Durall
 
Advocating for Usability: When, Why, and How to Improve User Experiences
Advocating for Usability: When, Why, and How to Improve User ExperiencesAdvocating for Usability: When, Why, and How to Improve User Experiences
Advocating for Usability: When, Why, and How to Improve User Experiencesnclatechandtrends
 
Usability testing.pdf
Usability testing.pdfUsability testing.pdf
Usability testing.pdfsairaazeem3
 
Analytic emperical Mehods
Analytic emperical MehodsAnalytic emperical Mehods
Analytic emperical MehodsM Surendar
 
Userzoom Webinar Monster Aug09
Userzoom Webinar Monster Aug09Userzoom Webinar Monster Aug09
Userzoom Webinar Monster Aug09Alfonso de la Nuez
 
Usability Testing On A Digital Product
Usability Testing On A Digital ProductUsability Testing On A Digital Product
Usability Testing On A Digital ProductKyle Soucy
 
Usability Testing Methods
Usability Testing MethodsUsability Testing Methods
Usability Testing Methodsdillarja
 
Demystifying the FDA's Human Factors Guidance
Demystifying the FDA's Human Factors GuidanceDemystifying the FDA's Human Factors Guidance
Demystifying the FDA's Human Factors GuidanceGreenlight Guru
 
Vittles - Quantified Self for Personal Health
Vittles - Quantified Self for Personal Health Vittles - Quantified Self for Personal Health
Vittles - Quantified Self for Personal Health Anne Everars
 
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.Carol Smith
 
Lightweight and ‘guerrilla’ usability testing for digital humanities projects
Lightweight and ‘guerrilla’ usability testing for digital humanities projectsLightweight and ‘guerrilla’ usability testing for digital humanities projects
Lightweight and ‘guerrilla’ usability testing for digital humanities projectsMia
 
Lean User Research: Why and How
Lean User Research: Why and HowLean User Research: Why and How
Lean User Research: Why and HowParul Soi
 
Best Practices on Recruiting Participants for Remote User Testing
Best Practices on Recruiting Participants for Remote User TestingBest Practices on Recruiting Participants for Remote User Testing
Best Practices on Recruiting Participants for Remote User TestingUserZoom
 
Research comparison pro forma (2) (1) (1) (1)
Research comparison pro forma (2) (1) (1) (1)Research comparison pro forma (2) (1) (1) (1)
Research comparison pro forma (2) (1) (1) (1)jackedwards46
 
Sampling techniques market research
Sampling techniques market researchSampling techniques market research
Sampling techniques market researchKrishna Ramakrishnan
 

Similar to Case study: Lab + Online Usability Testing (20)

Webinar lab + ous presentation may10
Webinar lab + ous presentation may10Webinar lab + ous presentation may10
Webinar lab + ous presentation may10
 
Webinar: Combining Lab and Online Usability Testing: Lessons Learned
Webinar: Combining Lab and Online Usability Testing: Lessons LearnedWebinar: Combining Lab and Online Usability Testing: Lessons Learned
Webinar: Combining Lab and Online Usability Testing: Lessons Learned
 
CIS375 Interaction Designs Chapter13
CIS375 Interaction Designs Chapter13CIS375 Interaction Designs Chapter13
CIS375 Interaction Designs Chapter13
 
Evaluation methods
Evaluation methodsEvaluation methods
Evaluation methods
 
Advocating for Usability: When, Why, and How to Improve User Experiences
Advocating for Usability: When, Why, and How to Improve User ExperiencesAdvocating for Usability: When, Why, and How to Improve User Experiences
Advocating for Usability: When, Why, and How to Improve User Experiences
 
Usability testing.pdf
Usability testing.pdfUsability testing.pdf
Usability testing.pdf
 
Analytic emperical Mehods
Analytic emperical MehodsAnalytic emperical Mehods
Analytic emperical Mehods
 
Userzoom Webinar Monster Aug09
Userzoom Webinar Monster Aug09Userzoom Webinar Monster Aug09
Userzoom Webinar Monster Aug09
 
Don’t make me think!
Don’t make me think!Don’t make me think!
Don’t make me think!
 
Usability Testing On A Digital Product
Usability Testing On A Digital ProductUsability Testing On A Digital Product
Usability Testing On A Digital Product
 
Neverstop
NeverstopNeverstop
Neverstop
 
Usability Testing Methods
Usability Testing MethodsUsability Testing Methods
Usability Testing Methods
 
Demystifying the FDA's Human Factors Guidance
Demystifying the FDA's Human Factors GuidanceDemystifying the FDA's Human Factors Guidance
Demystifying the FDA's Human Factors Guidance
 
Vittles - Quantified Self for Personal Health
Vittles - Quantified Self for Personal Health Vittles - Quantified Self for Personal Health
Vittles - Quantified Self for Personal Health
 
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.
 
Lightweight and ‘guerrilla’ usability testing for digital humanities projects
Lightweight and ‘guerrilla’ usability testing for digital humanities projectsLightweight and ‘guerrilla’ usability testing for digital humanities projects
Lightweight and ‘guerrilla’ usability testing for digital humanities projects
 
Lean User Research: Why and How
Lean User Research: Why and HowLean User Research: Why and How
Lean User Research: Why and How
 
Best Practices on Recruiting Participants for Remote User Testing
Best Practices on Recruiting Participants for Remote User TestingBest Practices on Recruiting Participants for Remote User Testing
Best Practices on Recruiting Participants for Remote User Testing
 
Research comparison pro forma (2) (1) (1) (1)
Research comparison pro forma (2) (1) (1) (1)Research comparison pro forma (2) (1) (1) (1)
Research comparison pro forma (2) (1) (1) (1)
 
Sampling techniques market research
Sampling techniques market researchSampling techniques market research
Sampling techniques market research
 

More from UserZoom

Retail UX in 2020: How to stay on top of changing customer behaviors
Retail UX in 2020: How to stay on top of changing customer behaviorsRetail UX in 2020: How to stay on top of changing customer behaviors
Retail UX in 2020: How to stay on top of changing customer behaviorsUserZoom
 
[Webinar] Transitioning to Remote Research
[Webinar] Transitioning to Remote Research[Webinar] Transitioning to Remote Research
[Webinar] Transitioning to Remote ResearchUserZoom
 
Moderated vs Unmoderated Research: It’s time to say ELMO (Enough, let’s move ...
Moderated vs Unmoderated Research: It’s time to say ELMO (Enough, let’s move ...Moderated vs Unmoderated Research: It’s time to say ELMO (Enough, let’s move ...
Moderated vs Unmoderated Research: It’s time to say ELMO (Enough, let’s move ...UserZoom
 
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...UserZoom
 
[19.2 UserZoom Spring Release Webinar] Get Card Sort Insights with Confidence
[19.2 UserZoom Spring Release Webinar] Get Card Sort Insights with Confidence[19.2 UserZoom Spring Release Webinar] Get Card Sort Insights with Confidence
[19.2 UserZoom Spring Release Webinar] Get Card Sort Insights with ConfidenceUserZoom
 
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...UserZoom
 
How to Work Collaboratively on the User Experience
How to Work Collaboratively on the User ExperienceHow to Work Collaboratively on the User Experience
How to Work Collaboratively on the User ExperienceUserZoom
 
UX Research for the Masses
UX Research for the MassesUX Research for the Masses
UX Research for the MassesUserZoom
 
Don’t Guess It, Test It!
Don’t Guess It, Test It!Don’t Guess It, Test It!
Don’t Guess It, Test It!UserZoom
 
Sharing the Love: Democratising Research at MoneySuperMarket
Sharing the Love: Democratising Research at MoneySuperMarketSharing the Love: Democratising Research at MoneySuperMarket
Sharing the Love: Democratising Research at MoneySuperMarketUserZoom
 
Making Research a UX Team Sport
Making Research a UX Team SportMaking Research a UX Team Sport
Making Research a UX Team SportUserZoom
 
How to Help Newbies Run UX Tests
How to Help Newbies Run UX TestsHow to Help Newbies Run UX Tests
How to Help Newbies Run UX TestsUserZoom
 
Ready to Democratise UX Insights?
Ready to Democratise UX Insights?Ready to Democratise UX Insights?
Ready to Democratise UX Insights?UserZoom
 
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...UserZoom
 
How to Master UserZoom’s Latest Study Builder & Extract Meaningful UX Insights
How to Master UserZoom’s Latest Study Builder & Extract Meaningful UX InsightsHow to Master UserZoom’s Latest Study Builder & Extract Meaningful UX Insights
How to Master UserZoom’s Latest Study Builder & Extract Meaningful UX InsightsUserZoom
 
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...UserZoom
 
How to Run Research in Agile Sprints by Democratizing It Across Teams
How to Run Research in Agile Sprints by Democratizing It Across TeamsHow to Run Research in Agile Sprints by Democratizing It Across Teams
How to Run Research in Agile Sprints by Democratizing It Across TeamsUserZoom
 
[Product Release Highlight] Seamlessly Discover and Share Your User Experienc...
[Product Release Highlight] Seamlessly Discover and Share Your User Experienc...[Product Release Highlight] Seamlessly Discover and Share Your User Experienc...
[Product Release Highlight] Seamlessly Discover and Share Your User Experienc...UserZoom
 
Hold on to Your Hats: The Scaled Agile Framework (SAFe) Might Actually Be a G...
Hold on to Your Hats: The Scaled Agile Framework (SAFe) Might Actually Be a G...Hold on to Your Hats: The Scaled Agile Framework (SAFe) Might Actually Be a G...
Hold on to Your Hats: The Scaled Agile Framework (SAFe) Might Actually Be a G...UserZoom
 
Telling Your UX Metrics Story - The 21st Century Metrics Model
Telling Your UX Metrics Story - The 21st Century Metrics ModelTelling Your UX Metrics Story - The 21st Century Metrics Model
Telling Your UX Metrics Story - The 21st Century Metrics ModelUserZoom
 

More from UserZoom (20)

Retail UX in 2020: How to stay on top of changing customer behaviors
Retail UX in 2020: How to stay on top of changing customer behaviorsRetail UX in 2020: How to stay on top of changing customer behaviors
Retail UX in 2020: How to stay on top of changing customer behaviors
 
[Webinar] Transitioning to Remote Research
[Webinar] Transitioning to Remote Research[Webinar] Transitioning to Remote Research
[Webinar] Transitioning to Remote Research
 
Moderated vs Unmoderated Research: It’s time to say ELMO (Enough, let’s move ...
Moderated vs Unmoderated Research: It’s time to say ELMO (Enough, let’s move ...Moderated vs Unmoderated Research: It’s time to say ELMO (Enough, let’s move ...
Moderated vs Unmoderated Research: It’s time to say ELMO (Enough, let’s move ...
 
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
 
[19.2 UserZoom Spring Release Webinar] Get Card Sort Insights with Confidence
[19.2 UserZoom Spring Release Webinar] Get Card Sort Insights with Confidence[19.2 UserZoom Spring Release Webinar] Get Card Sort Insights with Confidence
[19.2 UserZoom Spring Release Webinar] Get Card Sort Insights with Confidence
 
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
 
How to Work Collaboratively on the User Experience
How to Work Collaboratively on the User ExperienceHow to Work Collaboratively on the User Experience
How to Work Collaboratively on the User Experience
 
UX Research for the Masses
UX Research for the MassesUX Research for the Masses
UX Research for the Masses
 
Don’t Guess It, Test It!
Don’t Guess It, Test It!Don’t Guess It, Test It!
Don’t Guess It, Test It!
 
Sharing the Love: Democratising Research at MoneySuperMarket
Sharing the Love: Democratising Research at MoneySuperMarketSharing the Love: Democratising Research at MoneySuperMarket
Sharing the Love: Democratising Research at MoneySuperMarket
 
Making Research a UX Team Sport
Making Research a UX Team SportMaking Research a UX Team Sport
Making Research a UX Team Sport
 
How to Help Newbies Run UX Tests
How to Help Newbies Run UX TestsHow to Help Newbies Run UX Tests
How to Help Newbies Run UX Tests
 
Ready to Democratise UX Insights?
Ready to Democratise UX Insights?Ready to Democratise UX Insights?
Ready to Democratise UX Insights?
 
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
 
How to Master UserZoom’s Latest Study Builder & Extract Meaningful UX Insights
How to Master UserZoom’s Latest Study Builder & Extract Meaningful UX InsightsHow to Master UserZoom’s Latest Study Builder & Extract Meaningful UX Insights
How to Master UserZoom’s Latest Study Builder & Extract Meaningful UX Insights
 
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
 
How to Run Research in Agile Sprints by Democratizing It Across Teams
How to Run Research in Agile Sprints by Democratizing It Across TeamsHow to Run Research in Agile Sprints by Democratizing It Across Teams
How to Run Research in Agile Sprints by Democratizing It Across Teams
 
[Product Release Highlight] Seamlessly Discover and Share Your User Experienc...
[Product Release Highlight] Seamlessly Discover and Share Your User Experienc...[Product Release Highlight] Seamlessly Discover and Share Your User Experienc...
[Product Release Highlight] Seamlessly Discover and Share Your User Experienc...
 
Hold on to Your Hats: The Scaled Agile Framework (SAFe) Might Actually Be a G...
Hold on to Your Hats: The Scaled Agile Framework (SAFe) Might Actually Be a G...Hold on to Your Hats: The Scaled Agile Framework (SAFe) Might Actually Be a G...
Hold on to Your Hats: The Scaled Agile Framework (SAFe) Might Actually Be a G...
 
Telling Your UX Metrics Story - The 21st Century Metrics Model
Telling Your UX Metrics Story - The 21st Century Metrics ModelTelling Your UX Metrics Story - The 21st Century Metrics Model
Telling Your UX Metrics Story - The 21st Century Metrics Model
 

Recently uploaded

Unlock Canva Pro for Free Today - Canva Pro for Free.pdf
Unlock Canva Pro for Free Today - Canva Pro for Free.pdfUnlock Canva Pro for Free Today - Canva Pro for Free.pdf
Unlock Canva Pro for Free Today - Canva Pro for Free.pdfLewis John
 
Karim apartment ideas 02 ppppppppppppppp
Karim apartment ideas 02 pppppppppppppppKarim apartment ideas 02 ppppppppppppppp
Karim apartment ideas 02 pppppppppppppppNadaMohammed714321
 
General Simple Guide About AI in Design By: A.L. Samar Hossam ElDin
General Simple Guide About AI in Design By: A.L. Samar Hossam ElDinGeneral Simple Guide About AI in Design By: A.L. Samar Hossam ElDin
General Simple Guide About AI in Design By: A.L. Samar Hossam ElDinSamar Hossam ElDin Ahmed
 
City Hall London, Norman Foster building description with building details.pptx
City Hall London, Norman Foster building description with building details.pptxCity Hall London, Norman Foster building description with building details.pptx
City Hall London, Norman Foster building description with building details.pptxYaminiDabbara
 
Exploring Tehran's Architectural Marvels: A Glimpse into Vilaas Studio's Dyna...
Exploring Tehran's Architectural Marvels: A Glimpse into Vilaas Studio's Dyna...Exploring Tehran's Architectural Marvels: A Glimpse into Vilaas Studio's Dyna...
Exploring Tehran's Architectural Marvels: A Glimpse into Vilaas Studio's Dyna...Yantram Animation Studio Corporation
 
Map of St. Louis Parks
Map of St. Louis Parks                              Map of St. Louis Parks
Map of St. Louis Parks CharlottePulte
 
ALISIA: HOW MIGHT WE ACHIEVE HIGH ENVIRONMENTAL PERFORMANCE WHILE MAINTAINING...
ALISIA: HOW MIGHT WE ACHIEVE HIGH ENVIRONMENTAL PERFORMANCE WHILE MAINTAINING...ALISIA: HOW MIGHT WE ACHIEVE HIGH ENVIRONMENTAL PERFORMANCE WHILE MAINTAINING...
ALISIA: HOW MIGHT WE ACHIEVE HIGH ENVIRONMENTAL PERFORMANCE WHILE MAINTAINING...Pranav Subramanian
 
Karim apartment ideas 01 ppppppppppppppp
Karim apartment ideas 01 pppppppppppppppKarim apartment ideas 01 ppppppppppppppp
Karim apartment ideas 01 pppppppppppppppNadaMohammed714321
 
eCultify brand guidelines- Shamika Dukle.pdf
eCultify brand guidelines- Shamika Dukle.pdfeCultify brand guidelines- Shamika Dukle.pdf
eCultify brand guidelines- Shamika Dukle.pdfswdukle
 
LIGHTSCAPES: HOW MIGHT WE DESIGN AN INCLUSIVE AND ACCESSIBLE CLASSICAL CONCER...
LIGHTSCAPES: HOW MIGHT WE DESIGN AN INCLUSIVE AND ACCESSIBLE CLASSICAL CONCER...LIGHTSCAPES: HOW MIGHT WE DESIGN AN INCLUSIVE AND ACCESSIBLE CLASSICAL CONCER...
LIGHTSCAPES: HOW MIGHT WE DESIGN AN INCLUSIVE AND ACCESSIBLE CLASSICAL CONCER...Pranav Subramanian
 
The spirit of digital place - game worlds and architectural phenomenology
The spirit of digital place - game worlds and architectural phenomenologyThe spirit of digital place - game worlds and architectural phenomenology
The spirit of digital place - game worlds and architectural phenomenologyChristopher Totten
 
PORTFOLIO 2024 ANASTASIYA KUDINOVA
PORTFOLIO 2024       ANASTASIYA KUDINOVAPORTFOLIO 2024       ANASTASIYA KUDINOVA
PORTFOLIO 2024 ANASTASIYA KUDINOVAAnastasiya Kudinova
 
Interior Design for Office a cura di RMG Project Studio
Interior Design for Office a cura di RMG Project StudioInterior Design for Office a cura di RMG Project Studio
Interior Design for Office a cura di RMG Project StudioRMG Project Studio
 
Piece by Piece Magazine
Piece by Piece Magazine                      Piece by Piece Magazine
Piece by Piece Magazine CharlottePulte
 
simpson-lee_house_dt20ajshsjsjsjsjj15.pdf
simpson-lee_house_dt20ajshsjsjsjsjj15.pdfsimpson-lee_house_dt20ajshsjsjsjsjj15.pdf
simpson-lee_house_dt20ajshsjsjsjsjj15.pdfLucyBonelli
 
TIMBRE: HOW MIGHT WE REMEDY MUSIC DESERTS AND FACILITATE GROWTH OF A MUSICAL ...
TIMBRE: HOW MIGHT WE REMEDY MUSIC DESERTS AND FACILITATE GROWTH OF A MUSICAL ...TIMBRE: HOW MIGHT WE REMEDY MUSIC DESERTS AND FACILITATE GROWTH OF A MUSICAL ...
TIMBRE: HOW MIGHT WE REMEDY MUSIC DESERTS AND FACILITATE GROWTH OF A MUSICAL ...Pranav Subramanian
 
CERTIFICATE TEMPLATES FOR PORTFOLIO DAYS
CERTIFICATE TEMPLATES FOR PORTFOLIO DAYSCERTIFICATE TEMPLATES FOR PORTFOLIO DAYS
CERTIFICATE TEMPLATES FOR PORTFOLIO DAYSCarloJamesSablan1
 
Highway LOS - Freeway-- Multilane -- highway class 2 & 3 Housam Siyoufi
Highway LOS - Freeway-- Multilane -- highway class 2 & 3 Housam SiyoufiHighway LOS - Freeway-- Multilane -- highway class 2 & 3 Housam Siyoufi
Highway LOS - Freeway-- Multilane -- highway class 2 & 3 Housam Siyoufisiyoufihoussam
 
FW25-26 Knit Cut & Sew Trend Book Peclers Paris
FW25-26 Knit Cut & Sew Trend Book Peclers ParisFW25-26 Knit Cut & Sew Trend Book Peclers Paris
FW25-26 Knit Cut & Sew Trend Book Peclers ParisPeclers Paris
 
ArtWaves 2024 - embracing Curves in Modern Homes
ArtWaves 2024 - embracing Curves in Modern HomesArtWaves 2024 - embracing Curves in Modern Homes
ArtWaves 2024 - embracing Curves in Modern HomesVellyslav Petrov
 

Recently uploaded (20)

Unlock Canva Pro for Free Today - Canva Pro for Free.pdf
Unlock Canva Pro for Free Today - Canva Pro for Free.pdfUnlock Canva Pro for Free Today - Canva Pro for Free.pdf
Unlock Canva Pro for Free Today - Canva Pro for Free.pdf
 
Karim apartment ideas 02 ppppppppppppppp
Karim apartment ideas 02 pppppppppppppppKarim apartment ideas 02 ppppppppppppppp
Karim apartment ideas 02 ppppppppppppppp
 
General Simple Guide About AI in Design By: A.L. Samar Hossam ElDin
General Simple Guide About AI in Design By: A.L. Samar Hossam ElDinGeneral Simple Guide About AI in Design By: A.L. Samar Hossam ElDin
General Simple Guide About AI in Design By: A.L. Samar Hossam ElDin
 
City Hall London, Norman Foster building description with building details.pptx
City Hall London, Norman Foster building description with building details.pptxCity Hall London, Norman Foster building description with building details.pptx
City Hall London, Norman Foster building description with building details.pptx
 
Exploring Tehran's Architectural Marvels: A Glimpse into Vilaas Studio's Dyna...
Exploring Tehran's Architectural Marvels: A Glimpse into Vilaas Studio's Dyna...Exploring Tehran's Architectural Marvels: A Glimpse into Vilaas Studio's Dyna...
Exploring Tehran's Architectural Marvels: A Glimpse into Vilaas Studio's Dyna...
 
Map of St. Louis Parks
Map of St. Louis Parks                              Map of St. Louis Parks
Map of St. Louis Parks
 
ALISIA: HOW MIGHT WE ACHIEVE HIGH ENVIRONMENTAL PERFORMANCE WHILE MAINTAINING...
ALISIA: HOW MIGHT WE ACHIEVE HIGH ENVIRONMENTAL PERFORMANCE WHILE MAINTAINING...ALISIA: HOW MIGHT WE ACHIEVE HIGH ENVIRONMENTAL PERFORMANCE WHILE MAINTAINING...
ALISIA: HOW MIGHT WE ACHIEVE HIGH ENVIRONMENTAL PERFORMANCE WHILE MAINTAINING...
 
Karim apartment ideas 01 ppppppppppppppp
Karim apartment ideas 01 pppppppppppppppKarim apartment ideas 01 ppppppppppppppp
Karim apartment ideas 01 ppppppppppppppp
 
eCultify brand guidelines- Shamika Dukle.pdf
eCultify brand guidelines- Shamika Dukle.pdfeCultify brand guidelines- Shamika Dukle.pdf
eCultify brand guidelines- Shamika Dukle.pdf
 
LIGHTSCAPES: HOW MIGHT WE DESIGN AN INCLUSIVE AND ACCESSIBLE CLASSICAL CONCER...
LIGHTSCAPES: HOW MIGHT WE DESIGN AN INCLUSIVE AND ACCESSIBLE CLASSICAL CONCER...LIGHTSCAPES: HOW MIGHT WE DESIGN AN INCLUSIVE AND ACCESSIBLE CLASSICAL CONCER...
LIGHTSCAPES: HOW MIGHT WE DESIGN AN INCLUSIVE AND ACCESSIBLE CLASSICAL CONCER...
 
The spirit of digital place - game worlds and architectural phenomenology
The spirit of digital place - game worlds and architectural phenomenologyThe spirit of digital place - game worlds and architectural phenomenology
The spirit of digital place - game worlds and architectural phenomenology
 
PORTFOLIO 2024 ANASTASIYA KUDINOVA
PORTFOLIO 2024       ANASTASIYA KUDINOVAPORTFOLIO 2024       ANASTASIYA KUDINOVA
PORTFOLIO 2024 ANASTASIYA KUDINOVA
 
Interior Design for Office a cura di RMG Project Studio
Interior Design for Office a cura di RMG Project StudioInterior Design for Office a cura di RMG Project Studio
Interior Design for Office a cura di RMG Project Studio
 
Piece by Piece Magazine
Piece by Piece Magazine                      Piece by Piece Magazine
Piece by Piece Magazine
 
simpson-lee_house_dt20ajshsjsjsjsjj15.pdf
simpson-lee_house_dt20ajshsjsjsjsjj15.pdfsimpson-lee_house_dt20ajshsjsjsjsjj15.pdf
simpson-lee_house_dt20ajshsjsjsjsjj15.pdf
 
TIMBRE: HOW MIGHT WE REMEDY MUSIC DESERTS AND FACILITATE GROWTH OF A MUSICAL ...
TIMBRE: HOW MIGHT WE REMEDY MUSIC DESERTS AND FACILITATE GROWTH OF A MUSICAL ...TIMBRE: HOW MIGHT WE REMEDY MUSIC DESERTS AND FACILITATE GROWTH OF A MUSICAL ...
TIMBRE: HOW MIGHT WE REMEDY MUSIC DESERTS AND FACILITATE GROWTH OF A MUSICAL ...
 
CERTIFICATE TEMPLATES FOR PORTFOLIO DAYS
CERTIFICATE TEMPLATES FOR PORTFOLIO DAYSCERTIFICATE TEMPLATES FOR PORTFOLIO DAYS
CERTIFICATE TEMPLATES FOR PORTFOLIO DAYS
 
Highway LOS - Freeway-- Multilane -- highway class 2 & 3 Housam Siyoufi
Highway LOS - Freeway-- Multilane -- highway class 2 & 3 Housam SiyoufiHighway LOS - Freeway-- Multilane -- highway class 2 & 3 Housam Siyoufi
Highway LOS - Freeway-- Multilane -- highway class 2 & 3 Housam Siyoufi
 
FW25-26 Knit Cut & Sew Trend Book Peclers Paris
FW25-26 Knit Cut & Sew Trend Book Peclers ParisFW25-26 Knit Cut & Sew Trend Book Peclers Paris
FW25-26 Knit Cut & Sew Trend Book Peclers Paris
 
ArtWaves 2024 - embracing Curves in Modern Homes
ArtWaves 2024 - embracing Curves in Modern HomesArtWaves 2024 - embracing Curves in Modern Homes
ArtWaves 2024 - embracing Curves in Modern Homes
 

Case study: Lab + Online Usability Testing

  • 1. Case study from a Webinar presentation: Combining Lab and Online Usability Testing: Lessons Learned May 2010
  • 2. Overview 1.  Brief introduction to both research methods 2.  The power of combining methods 3.  Case study: Parallel studies 4.  Conclusion: Looking at the ‘big picture’ www.userzoom.com
  • 3. Brief introduction to both research methods Usability testing in the Lab www.userzoom.com
  • 4. Brief introduction to both research methods Usability testing in the Lab  Benefits/pros & limitations/cons +  The face-to-face  Catching the very details +  Ability to ask questions/interact with the participant live +  The ability to invite others to attend the sessions −  Small sample size −  Lack of natural environment/context −  Cost +++ www.userzoom.com
  • 5. Brief introduction to both research methods Online Usability Testing (a.k.a. Unmoderated Remote Usability Testing) www.userzoom.com
  • 6. Brief introduction to both research methods Online usability testing •  Hundreds of users can be tested •  Participation in the natural context… •  …from geographically spread locations •  No human moderation needed •  Our browser bar connects users with our secure servers www.userzoom.com
  • 7. Brief introduction to both research methods Online usability testing www.userzoom.com
  • 8. Brief introduction to both research methods Online usability testing  Benefits/pros and limitations/cons +  Quantifying usability  Objective, statistically significant data +  Automation of results, cost-effectiveness +  Participation in the natural context −  Lack of face-to-face −  Can’t invite to observe sessions −  Must anticipate participant’s questions in advance www.userzoom.com
  • 9. Overview 1.  Brief introduction to both research methods 2.  The power of combining methods 3.  Case study: Parallel studies 4.  Conclusion: Looking at the ‘big picture’ www.userzoom.com
  • 11. Power of Combining Methods   Every research method has unique strengths and limitations   Gather new insights with each method   Greater confidence when observing similar findings through multiple methods Which one goes first? Lab first, then Online Online first, then Lab Identify/fix “low hanging fruit”, then Identify the most significant issues online focus on remaining tasks with large through metrics, then use lab study to sample size gather deeper qualitative understanding of those issues Generate new concepts, ideas, questions Collect video clips or more quotes of through lab testing, then test/validate users to help bring metrics to life online Validate attitudes/preferences observed Gather all the metrics to validate design in lab testing – if it tests well, then no need to bring users into the lab www.userzoom.com 11
  • 12. Overview 1.  Brief introduction to both research methods 2.  The power of combining methods 3.  Case study: Parallel studies 4.  Conclusion: Looking at the ‘big picture’ www.userzoom.com
  • 13. Case study: Parallel studies We conducted 2 parallel studies using each method. Basic details: •  Website tested: Amazon.com’s Grocery and Gourmet Food Store •  Lab study conducted by the DUC •  Online Usability Study conducted by UZ •  Date of the study: April 2010 www.userzoom.com
  • 14. Case study: Parallel studies Basic details of the studies (II) •  Objectives: •  What is the unique contribution of each method? •  What do we learn by combining methods? •  Participants: 10 for the lab study, 100 for the online study •  Study design: •  Initial questionnaire •  3 tasks •  Follow up questions •  Final questionnaire www.userzoom.com
  • 15. Case study: Parallel studies Participant demographics from the Lab Usability Study:   •  Average age 29 •  Age range between 19 and 51 •  6 Males, 4 Females •  All had previously used Amazon.com website •  None had ordered groceries on-line •  Visited Amazon.com approximately every 2 weeks, on average •  Previously purchased books, electronics, toys, and more. •  Expected packaged, non-perishable grocery items to be sold on-line •  Most said they hadn’t bought groceries on-line because they, “like the experience of picking up the produce and meat.” •  Expressed concern about products being fresh •  A few worried about exorbitant service or shipping charges making it not worthwhile. www.userzoom.com
  • 16. Case study: Parallel studies Task 1: “You just ran out of dishwasher detergent. See if Amazon has a pack of 6 Cascade Complete Dishwasher Detergent Powder, in the 45 ounce size.” www.userzoom.com
  • 17. Results of Locating Dishwashing Detergent   Case study: Parallel studies Search vs. Menu Navigation •  Approximately half of the participants wanted to immediately use SEARCH Category Confusion •  Most likely categories were “Home & Garden”, “Grocery, Health & Beauty” •  Chose “Grocery, Health & Beauty” through process of elimination •  Further confusion within “Grocery, Health & Beauty” •  “I don’t know if ‘Grocery and Gourmet food’ will have detergent” www.userzoom.com
  • 18. Results of Locating Dishwashing Detergent   Case study: Parallel studies Confusion within “Grocery & Gourmet Food” menu •  First menu is 4 screens long •  Participants were confused where to look for dishwasher detergent Dishwasher Detergent found under here www.userzoom.com
  • 19. Results of Locating Dishwashing Detergent   Case study: Parallel studies Nesting Confusion •  All participants were confused by path to find dishwashing detergent: •  Amazon.com > Grocery & Gourmet Food > All Household (page is titled ‘Health & Personal Care ‘) > All Household Supplies > All Dishwashing > Scroll to locate Cascade Dishwashing Detergent •  Participants scrolled up and down navigation menus •  A few gave up mid-way & used SEARCH “There are a lot of clicks to go through: In Household Cleaners, Kitchen Cleaners…I don’t see dishes…maybe by brand? I don’t see Cascade here. I would do SEARCH.” “That took probably 5-6 minutes. For me, that’s really long.” www.userzoom.com
  • 20. Case study: Parallel studies Task 2: “You want to buy Skippy Peanut Butter. Find out if there are any discounts or coupons for it.” www.userzoom.com
  • 21. Results of Locating Skippy Peanut Butter Coupon   Case study: Parallel studies Product Search vs. Coupon Search •  A few looked on Amazon.com home page for Today’s Deals •  Most looked for the product in the “Grocery” section, and then looked for a coupon •  Some looked at “Special Offers” in the “Grocery” screen •  Special Offers led to a coupon and New and Used food, which disturbed participants. www.userzoom.com
  • 22. Results of Locating Skippy Peanut Butter Coupon Case study: Parallel studies Participants that looked for the product complained about the length of the menu on the left side of the grocery page •  Peanut Butter, under “Sauces and Dips,” was four screens down plus 2 clicks. “It was tedious.” www.userzoom.com
  • 23. Results of Locating Skippy Coupon Case study: Parallel studies Users had strong feelings about coupons for grocery items “I would expect they’d show me the price they have, it doesn’t matter if it is cheaper.” “If I am purchasing a big purchase, I would do it from Google. I would look for coupons [only] for big items.” “I would have preferred to see a link from the home page– specials, coupons, etc…I don’t want to hunt down each thing.” “It was not clear that a discount was available until you went to the [item] description page” “I had to look through a lot of things." www.userzoom.com
  • 24. Case study: Parallel studies Task 3: “Find out how you would schedule the same groceries to be delivered every 3 months.” www.userzoom.com
  • 25. Scheduling Groceries for Repeated Delivery   Case study: Parallel studies All found the correct starting place: the “Grocery & Gourmet Food” page •  Most noticed the “Shop Subscribe and Save” link •  Others tried to schedule each item from the item’s page “I think you will have to do it for every item.” “I would probably go to the product and check if I can schedule it.” “[a product page] might say, “do you want this delivered?” www.userzoom.com
  • 26. Scheduling Groceries for Repeated Delivery Case study: Parallel studies Participants were confused on how to schedule grocery delivery •  Most missed the delivery scheduling link at the top of the page Delivery Scheduling Information Link “The text was small, and I wasn’t sure I was in the right place.” “The grayed-out part doesn’t jump out” www.userzoom.com
  • 27. Conclusions from the Lab Study   Case study: Parallel studies Confusing IA “Dishwasher detergent was under “Health [& Personal Care], which is not the first place I’d look.”” “There were too many categories, it confuses you…” “It is not as well organized as other parts of Amazon.” “I don’t know why ‘All Household Supplies’ is under ‘Health and Personal Care’” Difficult navigation “Navigation wasn’t intuitive. The left-hand-side menu was so far down the page and not duplicated in the banner.” “The browse list was too long. I don’t want to go through 10 chocolates before going to the pasta.” “It didn’t feel intuitive.” “I didn’t like how many levels I had to go through.” “It was hard to find things. There were long lists to find food items.” www.userzoom.com
  • 28. Case study: Parallel studies Highlighted findings from the Online Usability Study: www.userzoom.com
  • 29. Study Background - 71% Male, 29% Female -  Age range distributed between 18–55; 88% fell between 26-45 - All have purchased from Amazon.com and visit Amazon once a week, Profile month or every few months. -  Eighty-eight percents of participants purchase groceries from in the store only, 6% online only and 6% both online and in a store. -  Main reasons for considering purchasing groceries online is for convenience and time. - April 17th to April 26, 2010 Detail -  Unmoderated remote usability testing method s -  100 participants -  Recruited through participant panel www.userzoom.com 29
  • 30. Overview of tasks and validation criteria 1.  Locate specific item: You just ran out of dishwasher detergent. Find Cascade Complete Dishwasher Detergent Powder, 45 oz (pack of 6). Validation by reaching the correct page with the detergent (URL validation) 2.  Finding a discount code: You want to buy Skippy Peanut Butter with a discount you heard about on Amazon.com. Please write down the discount code. Validation by choosing the correct discount code (multiple choice) 3.  Schedule grocery delivery: Find out how often you can schedule a delivery to your home. Validation by how often you can set up a delivery (multiple choice) www.userzoom.com 30
  • 31. Study Design Initial Questionnaire: Post Task: How frequently do you visit Amazon.com? Validation by url or question (Success, Error, Abandon ratios collected). Where do you usually buy your groceries? Ease of Use. Why have you considered purchasing groceries online? How intuitive links and menus were and how easy it was to start the search. Issues and problems experienced while completing tasks. Final Questionaire: Overall satisfaction Satisfaction with navigation, product search, information offered and look and feel of the site. Likelihood to purchase groceries from Amazon.com. Likelihood to recomment to a friend. www.userzoom.com 31
  • 32. T1: Locating Cascade Complete Success Non-Success I never felt lost on the site 77% while searching The menus and links were 72% intuitive to use It was clear how to start searching for the 73% Participants did feel particularly lost on the dishwasher detergent site (most likely due to experience using Amazon.com. However they did not feel the menus and links were extremely intuitive and weren’t always sure were to start their search. www.userzoom.com 32
  • 33. T1: Locating Cascade Complete Highlights  dominant  path   Over half (51%) of the participants defaulted to using *3%  insignificance  removed   search indicating the menus and navigation of the site were not very intuitive. Fifteen percent started in Health and Personal while 5% looked in promotional deals. www.userzoom.com 33
  • 34. T1: Locating Cascade Complete What participants found difficult when attempting the task: Exact Quotes: “ Hard to navigate to find desired product. Too long and was getting frustrating.” “ There were a lot of results so you have to type it in exactly or spend a couple minutes searching.” “ I didn’t see a soap category and didn’t realize how the search function worked.” “ I didn’t know where to look.” www.userzoom.com 34
  • 35. T2: Finding a discount code Success Non-Success Extremely Difficult Extremely Easy Forty-seven percent of the participants thought it was very difficult to locate the discount code. They thought it was very unclear how to start searching, it was not where they expected it to be and were somewhat lost during the search process. www.userzoom.com 35
  • 36. T2: Finding a discount code www.userzoom.com 36
  • 37. T2: Finding a discount code Error participants (55%) could not locate the code, thought it was too difficult to find the code or the peanut butter and stated: “ I was expecting the discount in the results page, or the product page. I ended up looking for it under ’specials’, but I don’t know if that’s the best place for it.” “ Too much searching and clicking.” “ Looked under special offers having firstly used search, had to notice the offer code above product within the offer description.” www.userzoom.com 37
  • 38. T3: Schedule Grocery Delivery Success Non-Success Extremely Difficult Extremely Easy Seventy-five percent of participants could not complete the task. They did not think it was clear how to start searching and felt lost during the search process. Forty percent of participants who were successful, felt it was very difficult to complete the task. www.userzoom.com 38
  • 39. T3: Schedule Grocery Delivery Abandon participants (38%) could not find the information, thought the menus were difficult to understand and the task was taking too long. “ It was not obvious to me that I needed to click on the link at the top right of the screen to get this information until I had spent quite a bit of time looking all over the rest of the screen.” “ I wasn’t certain where to look for the information.” “ Nothing about this process has been user friendly.” www.userzoom.com 39
  • 40. Final Questionnaire Fifty-three percent of participants thought the navigation was very difficult, 39% thought it was difficult to search for a product, forty-seven percent rated the overall look and feel of the site as very poor. www.userzoom.com 40
  • 41. Final Questionnaire Fifty-five percent of participants stated they would not purchase their groceries in the future from Amazon.com. Net Promoter Score (NPS) calculation: 0 to 6 = Detractors 7 to 8 = Passive 9 to 10 = Promoters Net Promoter Score (NPS) = % of Promoters - % of Detractors www.userzoom.com 41
  • 42. Overview 1.  Brief introduction to both research methods 2.  The power of combining methods 3.  Case study: Parallel studies 4.  Conclusion: Looking at the ‘big picture’ www.userzoom.com
  • 43. Conclusions: Looking at the ‘big picture’ •  Main, obvious issue: Gourmet store was built like the book store •  IA + navigation + labeling + content problems/issues encountered •  Both research methods prove this: 1.  By observing & listening at the Lab we learned a lot, but also… 2.  …we quantified how big the issues were using UserZoom www.userzoom.com
  • 44. Conclusions: Looking at the ‘big picture’ www.userzoom.com
  • 45. The introductions… Alfonso de la Nuez Kim Oslob Bill Albert Partner & Chief Research & Product Director of the Design & Marketing Officer Strategy Director Usability Center, at UserZoom at UserZoom Bentley University www.userzoom.com
  • 46. The introductions   Leading online user experience research software company   Develops proprietary on-demand software app for online research   Built for & by UX & marketing professionals   Offers a cost-effective, time-saving methodology   Has 9 years of experience in UX research & consulting   Has offices in Sunnyvale (CA), London (UK) & Barcelona (Spain) www.userzoom.com
  • 47. The introductions   The Design and Usability Center at Bentley University was founded in 1999   Provides UX consulting services to corporate clients   Supports the MS program in Human Factors in Information Design   Focuses on user experience research and evaluation www.userzoom.com
  • 48. Acknowledgements   Special thanks to Cynthia Kamishlian and Dharmesh Mistry from Bentley University for carrying out this research   Based on our recent book “Beyond the Usability Lab”, co-authored with Tom Tullis and Donna Tedesco www.userzoom.com 48
  • 49. º Thanks so much for your time! Oh! Don’t forget: The slides and the video of this webinar will be available in a few days in our blog www.userzoom.com USA UK SPAIN 440 N. Wolfe Rd. Aylesbury House, 17 – 18 Av. Diagonal 419 3º 2ª Sunnyvale, CA 94085 USA Aylesbury Street, London 08008 Barcelona, Spain ECIR 0DB, UK Phone: +1 (408) 524 – 74 45 Phone: +34 93 414 7554 Contact: Alfonso de la Nuez Phone: +44 (0) 20 7193 2171 Cell: +34 93 368 4295 alfonso@userzoom.com Cell: +44 (0) 7900 472 920 Contact: Javier Darriba Contact: Arthur Moan jdarriba@userzoom.com www.userzoom.com amoan@userzoom.com 49