Your SlideShare is downloading. ×
Find The Balance Between Qualitative And Quantitative Web Site Usability Measures
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Find The Balance Between Qualitative And Quantitative Web Site Usability Measures

2,128
views

Published on

Published in: Business, Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,128
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
53
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Transcript

    • 1. Find the Balance Between Qualitative and Quantitative Web Site Usability Measures John Lovett Senior Analyst Jupiter Research [email_address] Jeff Schueler President Usability Sciences [email_address]
    • 2. Usability Sciences Overview
      • A leading provider of qualitative usability testing research since 1988
        • We were doing usability before it was cool
      • A leading provider of quantitative user experience research since 2000
        • Our focus is the measurement and improvement of the user experience and effectiveness of web sites
        • Patented technology WebIQ ®
      • Our unique combination of qualitative and quantitative capabilities and methodologies sets us apart from our competitors
      • We provide our research services across the country and around the world
    • 3. Balancing Qualitative and Quantitative Web Site Usability Measures        John Lovett May 21, 2008
    • 4. Usability is Persistently a Top Challenge for Web Site Operators Question: Indicate your top 3 challenges in operating your company’s primary, externally-facing Web site? JupiterResearch Web site Decision-Makers Survey 2004 Improving Usability of the Site (# 1 answer) JupiterResearch/ERI Executive Survey, 12/04, n=235 (US only) 2005 Improving Usability of the Site (#2 answer) JupiterResearch/ERI Executive Survey, 12/05, n=251 (US only) 2006 Improving Usability of the Site (#1 answer) JupiterResearch/ERI Executive Survey, 12/06, n=250 (US only)
    • 5. Usability Challenges are Universal Percentage of Web Decision Makers Question: Please indicate your top 3 challenges in operating your company’s primary, externally-facing Web site? (Please select up to three.) Source: JupiterResearch/ERI Executive Survey, 12/06, n=250 (US only)
    • 6. Understanding Behavior, Satisfaction and Success Requires Different Tools Usability, Behavior, Intent, Satisfaction, Success
    • 7. Technologies Counterbalance Expertise and Substantiate Qualitative Assumptions Percentage of Web Decision Makers Question: Which of the following tactics do you currently use to measure website usability? (Please select all that apply.) Source: JupiterResearch/ERI Executive Survey (12/06), n=250 (US only) Qualitative Usability Resources Quantitative Usability Measures
    • 8. Polling Question:
      • Which of the following tools do you currently use to measure website usability? (Please select all that apply.)
        • Web analytics
        • Satisfaction Surveys
        • Multivariate or A/B testing
        • Exit surveys
        • In-house expertise
        • Focus groups
        • Lab testing
        • E xternal usability consultants
        • Eye Tracking
        • None of the above
    • 9. Highly Effective Tactics Require a Mix of Qualitative & Quantitative Methods Percentage of Web Decision Makers Question: Which of the following tactics have been most effective for improving the usability for your website in the last 12 months? (Please select up to 3.) Source: JupiterResearch/ERI Executive Survey (12/06), n=250 (US only) Qualitative Measures
    • 10. JupiterResearch’s Usability Framework Balances Methods Source: JupiterResearch (5/08) Automated (A/B) testing framework to test hypotheses while expediting time to market Traffic and performance analyses to mine clickstream and monitor for performance impact Voice of Customer Capture to tie feedback to clickstream and split-path testing Traditional usability principles to ground framework in established “best practices”
    • 11. Traditional Usability Principles
      • Develop a usability process
      • Understand, Prioritize, Implement, Optimize
      • Cultivate internally – Fortify externally
      • Adhere to best practices
      • Resist throwing technology at the problem
    • 12. Traffic & Performance Analyses
      • Ensure clickstream visibility
      • Find the holes in your conversion funnel
      • Capture the What of onsite behavior
      • Seek to understand intent
      • Delivery reliability and consistency
      • Usability exceeds design
    • 13. Voice of Customer Capture
      • Quantify Satisfaction
      • Extend further to gauge Success
      • Manage unsolicited feedback
      • Amass social awareness
      • Provide a vehicle for feedback
      • Perform damage control
    • 14. Increased Visibility into Attitudes & Behavior Drives Satisfaction Customer Satisfaction Usability Measurement Technologies (CEM, VOC, Analytics) Lack of visibility into customer interactions Capture of implicit and explicit customer experience data Source: JupiterResearch (11/07)
    • 15. Automated Testing Framework
      • Deploy a Control
      • Test Alternative Hypotheses
      • Use Testing as Segmentation Tool
      • Be Cognizant of Demo/Psycho-graphics
      • Address Improvement not Problems
    • 16. A Balanced Approach Provides a Holistic Perspective Source: JupiterResearch 5/08 Qualitative Quantitative Paper Designs Focus groups Participatory design Card Sorting Diary studies Eye Tracking Performance Analytics A/B or MVT Satisfaction Indexes Online Testing
    • 17. Source: JupiterResearch 5/08 Problem Identification Clickstream Activity Customer Feedback Behavioral Analysis Focus Groups /Labs Visual & Iterative Design Satisfaction Scoring Performance Monitoring (Gomez, Keynote) Web Analytics (Coremetrics, Omniture) VOC technologies (ForeSee, iPerceptions) Design & Development (BitGroup, Allurent) Agencies & Usability Specialists (Usability Sciences, Razorfish) Usability Technologies (Morae, Eye Tracking) CEM technologies (Coradiant, Tealeaf) Usability Measures Usability Solutions
    • 18.
      • Competitive Benchmarking
      • Survey existing site visitors
      • Persona Development
      • jcp.com Scorecard
        • Scorecard of end-to-end experience vs. competitors
      • Site Survey
        • Collect survey and behavioral data to best understand the customer experience
      • Customer feedback
        • Town hall meetings
        • Call monitoring
      Ideas for Understanding the Shopper Understanding the jcp.com Shopper
    • 19.
      • Design Walk-Through
      • Rapid Iterative Testing
      • Information architecture
      • A/B testing in design
      Ideas for Understanding the Shopper Understanding the jcp.com Shopper
      • User needs
        • Use cases and workflows
        • Design concepts and wireframes
        • Prototyping
      • Information architecture
      • Usability Panel
      • Usability Testing
      • Benchmark Usability study
        • Existing vs. new functionality
      • Competitive Benchmark Analysis
    • 20.
      • Usability testing
      • A/B Testing in production
      Ideas for Understanding the Shopper Understanding the jcp.com Shopper
      • A/B and Multivariate Testing
        • A/B testing - test a factor, such as a graphic, button, or image, against one or more variations to see which is most persuasive.
        • Multivariate testing - enables testing many changes simultaneously.
          • Evaluating the impact of combinations of factors and variations often reveals significant interaction effects that can have a dramatic impact on your conversion goals.
    • 21.
      • Survey existing site visitors
      • Usability test incremental enhancements
      Ideas for Understanding the Shopper Understanding the jcp.com Shopper
      • Continue site surveys
        • Continuous measurement of the customer experience
      • Detail Clickstream Analysis
        • Post implementation of major releases
    • 22. Questions?? John Lovett Senior Analyst Jupiter Research [email_address] Jeff Schueler President Usability Sciences [email_address]