Find the Balance Between Qualitative and Quantitative Web Site Usability Measures John Lovett Senior Analyst Jupiter Resea...
Usability Sciences Overview <ul><li>A leading provider of qualitative usability testing research since 1988 </li></ul><ul>...
Balancing Qualitative and Quantitative Web Site Usability Measures         John Lovett May 21, 2008
Usability is Persistently a Top Challenge for Web Site Operators Question: Indicate your top 3 challenges in operating you...
Usability Challenges are Universal Percentage of Web Decision Makers Question: Please indicate your top 3 challenges in op...
Understanding Behavior, Satisfaction and Success Requires Different Tools Usability, Behavior, Intent, Satisfaction,  Succ...
Technologies Counterbalance Expertise and Substantiate Qualitative Assumptions Percentage of Web Decision Makers Question:...
Polling Question: <ul><li>Which of the following tools do you currently use to measure website usability?  (Please select ...
Highly Effective Tactics Require a Mix of Qualitative & Quantitative Methods Percentage of Web Decision Makers Question:  ...
JupiterResearch’s Usability Framework Balances Methods Source: JupiterResearch (5/08) Automated (A/B) testing framework  t...
Traditional Usability Principles <ul><li>Develop a usability process  </li></ul><ul><li>Understand, Prioritize, Implement,...
Traffic & Performance Analyses <ul><li>Ensure clickstream visibility </li></ul><ul><li>Find the holes in your conversion f...
Voice of Customer Capture <ul><li>Quantify Satisfaction </li></ul><ul><li>Extend further to gauge Success </li></ul><ul><l...
Increased Visibility into Attitudes & Behavior Drives Satisfaction Customer Satisfaction Usability Measurement Technologie...
Automated Testing Framework <ul><li>Deploy a Control </li></ul><ul><li>Test Alternative Hypotheses  </li></ul><ul><li>Use ...
A Balanced Approach Provides a Holistic Perspective Source: JupiterResearch 5/08 Qualitative Quantitative Paper Designs Fo...
Source: JupiterResearch 5/08 Problem  Identification Clickstream  Activity Customer  Feedback Behavioral  Analysis Focus G...
<ul><li>Competitive Benchmarking </li></ul><ul><li>Survey existing site visitors </li></ul><ul><li>Persona Development </l...
<ul><li>Design Walk-Through </li></ul><ul><li>Rapid Iterative Testing </li></ul><ul><li>Information architecture </li></ul...
<ul><li>Usability testing </li></ul><ul><li>A/B Testing in production </li></ul>Ideas for Understanding the Shopper Unders...
<ul><li>Survey existing site visitors </li></ul><ul><li>Usability test incremental  enhancements </li></ul>Ideas for Under...
Questions?? John Lovett Senior Analyst Jupiter Research [email_address] Jeff Schueler President Usability Sciences [email_...
Upcoming SlideShare
Loading in...5
×

Find The Balance Between Qualitative And Quantitative Web Site Usability Measures

2,164

Published on

Published in: Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,164
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
53
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Transcript of "Find The Balance Between Qualitative And Quantitative Web Site Usability Measures"

    1. 1. Find the Balance Between Qualitative and Quantitative Web Site Usability Measures John Lovett Senior Analyst Jupiter Research [email_address] Jeff Schueler President Usability Sciences [email_address]
    2. 2. Usability Sciences Overview <ul><li>A leading provider of qualitative usability testing research since 1988 </li></ul><ul><ul><li>We were doing usability before it was cool </li></ul></ul><ul><li>A leading provider of quantitative user experience research since 2000 </li></ul><ul><ul><li>Our focus is the measurement and improvement of the user experience and effectiveness of web sites </li></ul></ul><ul><ul><li>Patented technology WebIQ ® </li></ul></ul><ul><li>Our unique combination of qualitative and quantitative capabilities and methodologies sets us apart from our competitors </li></ul><ul><li>We provide our research services across the country and around the world </li></ul>
    3. 3. Balancing Qualitative and Quantitative Web Site Usability Measures        John Lovett May 21, 2008
    4. 4. Usability is Persistently a Top Challenge for Web Site Operators Question: Indicate your top 3 challenges in operating your company’s primary, externally-facing Web site? JupiterResearch Web site Decision-Makers Survey 2004 Improving Usability of the Site (# 1 answer) JupiterResearch/ERI Executive Survey, 12/04, n=235 (US only) 2005 Improving Usability of the Site (#2 answer) JupiterResearch/ERI Executive Survey, 12/05, n=251 (US only) 2006 Improving Usability of the Site (#1 answer) JupiterResearch/ERI Executive Survey, 12/06, n=250 (US only)
    5. 5. Usability Challenges are Universal Percentage of Web Decision Makers Question: Please indicate your top 3 challenges in operating your company’s primary, externally-facing Web site? (Please select up to three.) Source: JupiterResearch/ERI Executive Survey, 12/06, n=250 (US only)
    6. 6. Understanding Behavior, Satisfaction and Success Requires Different Tools Usability, Behavior, Intent, Satisfaction, Success
    7. 7. Technologies Counterbalance Expertise and Substantiate Qualitative Assumptions Percentage of Web Decision Makers Question: Which of the following tactics do you currently use to measure website usability? (Please select all that apply.) Source: JupiterResearch/ERI Executive Survey (12/06), n=250 (US only) Qualitative Usability Resources Quantitative Usability Measures
    8. 8. Polling Question: <ul><li>Which of the following tools do you currently use to measure website usability? (Please select all that apply.) </li></ul><ul><ul><li>Web analytics </li></ul></ul><ul><ul><li>Satisfaction Surveys </li></ul></ul><ul><ul><li>Multivariate or A/B testing </li></ul></ul><ul><ul><li>Exit surveys </li></ul></ul><ul><ul><li>In-house expertise </li></ul></ul><ul><ul><li>Focus groups </li></ul></ul><ul><ul><li>Lab testing </li></ul></ul><ul><ul><li>E xternal usability consultants </li></ul></ul><ul><ul><li>Eye Tracking </li></ul></ul><ul><ul><li>None of the above </li></ul></ul>
    9. 9. Highly Effective Tactics Require a Mix of Qualitative & Quantitative Methods Percentage of Web Decision Makers Question: Which of the following tactics have been most effective for improving the usability for your website in the last 12 months? (Please select up to 3.) Source: JupiterResearch/ERI Executive Survey (12/06), n=250 (US only) Qualitative Measures
    10. 10. JupiterResearch’s Usability Framework Balances Methods Source: JupiterResearch (5/08) Automated (A/B) testing framework to test hypotheses while expediting time to market Traffic and performance analyses to mine clickstream and monitor for performance impact Voice of Customer Capture to tie feedback to clickstream and split-path testing Traditional usability principles to ground framework in established “best practices”
    11. 11. Traditional Usability Principles <ul><li>Develop a usability process </li></ul><ul><li>Understand, Prioritize, Implement, Optimize </li></ul><ul><li>Cultivate internally – Fortify externally </li></ul><ul><li>Adhere to best practices </li></ul><ul><li>Resist throwing technology at the problem </li></ul>
    12. 12. Traffic & Performance Analyses <ul><li>Ensure clickstream visibility </li></ul><ul><li>Find the holes in your conversion funnel </li></ul><ul><li>Capture the What of onsite behavior </li></ul><ul><li>Seek to understand intent </li></ul><ul><li>Delivery reliability and consistency </li></ul><ul><li>Usability exceeds design </li></ul>
    13. 13. Voice of Customer Capture <ul><li>Quantify Satisfaction </li></ul><ul><li>Extend further to gauge Success </li></ul><ul><li>Manage unsolicited feedback </li></ul><ul><li>Amass social awareness </li></ul><ul><li>Provide a vehicle for feedback </li></ul><ul><li>Perform damage control </li></ul>
    14. 14. Increased Visibility into Attitudes & Behavior Drives Satisfaction Customer Satisfaction Usability Measurement Technologies (CEM, VOC, Analytics) Lack of visibility into customer interactions Capture of implicit and explicit customer experience data Source: JupiterResearch (11/07)
    15. 15. Automated Testing Framework <ul><li>Deploy a Control </li></ul><ul><li>Test Alternative Hypotheses </li></ul><ul><li>Use Testing as Segmentation Tool </li></ul><ul><li>Be Cognizant of Demo/Psycho-graphics </li></ul><ul><li>Address Improvement not Problems </li></ul>
    16. 16. A Balanced Approach Provides a Holistic Perspective Source: JupiterResearch 5/08 Qualitative Quantitative Paper Designs Focus groups Participatory design Card Sorting Diary studies Eye Tracking Performance Analytics A/B or MVT Satisfaction Indexes Online Testing
    17. 17. Source: JupiterResearch 5/08 Problem Identification Clickstream Activity Customer Feedback Behavioral Analysis Focus Groups /Labs Visual & Iterative Design Satisfaction Scoring Performance Monitoring (Gomez, Keynote) Web Analytics (Coremetrics, Omniture) VOC technologies (ForeSee, iPerceptions) Design & Development (BitGroup, Allurent) Agencies & Usability Specialists (Usability Sciences, Razorfish) Usability Technologies (Morae, Eye Tracking) CEM technologies (Coradiant, Tealeaf) Usability Measures Usability Solutions
    18. 18. <ul><li>Competitive Benchmarking </li></ul><ul><li>Survey existing site visitors </li></ul><ul><li>Persona Development </li></ul><ul><li>jcp.com Scorecard </li></ul><ul><ul><li>Scorecard of end-to-end experience vs. competitors </li></ul></ul><ul><li>Site Survey </li></ul><ul><ul><li>Collect survey and behavioral data to best understand the customer experience </li></ul></ul><ul><li>Customer feedback </li></ul><ul><ul><li>Town hall meetings </li></ul></ul><ul><ul><li>Call monitoring </li></ul></ul>Ideas for Understanding the Shopper Understanding the jcp.com Shopper
    19. 19. <ul><li>Design Walk-Through </li></ul><ul><li>Rapid Iterative Testing </li></ul><ul><li>Information architecture </li></ul><ul><li>A/B testing in design </li></ul>Ideas for Understanding the Shopper Understanding the jcp.com Shopper <ul><li>User needs </li></ul><ul><ul><li>Use cases and workflows </li></ul></ul><ul><ul><li>Design concepts and wireframes </li></ul></ul><ul><ul><li>Prototyping </li></ul></ul><ul><li>Information architecture </li></ul><ul><li>Usability Panel </li></ul><ul><li>Usability Testing </li></ul><ul><li>Benchmark Usability study </li></ul><ul><ul><li>Existing vs. new functionality </li></ul></ul><ul><li>Competitive Benchmark Analysis </li></ul>
    20. 20. <ul><li>Usability testing </li></ul><ul><li>A/B Testing in production </li></ul>Ideas for Understanding the Shopper Understanding the jcp.com Shopper <ul><li>A/B and Multivariate Testing </li></ul><ul><ul><li>A/B testing - test a factor, such as a graphic, button, or image, against one or more variations to see which is most persuasive. </li></ul></ul><ul><ul><li>Multivariate testing - enables testing many changes simultaneously. </li></ul></ul><ul><ul><ul><li>Evaluating the impact of combinations of factors and variations often reveals significant interaction effects that can have a dramatic impact on your conversion goals. </li></ul></ul></ul>
    21. 21. <ul><li>Survey existing site visitors </li></ul><ul><li>Usability test incremental enhancements </li></ul>Ideas for Understanding the Shopper Understanding the jcp.com Shopper <ul><li>Continue site surveys </li></ul><ul><ul><li>Continuous measurement of the customer experience </li></ul></ul><ul><li>Detail Clickstream Analysis </li></ul><ul><ul><li>Post implementation of major releases </li></ul></ul>
    22. 22. Questions?? John Lovett Senior Analyst Jupiter Research [email_address] Jeff Schueler President Usability Sciences [email_address]
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×