Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Benchmark Your Online Customer Experience Against Competition

9,963 views

Published on

Benchmarking your site against competitors is a terrific way to improve the user experience. Join this webinar to learn about benchmarking best practices and to see the results of a study benchmarking the website experience of: Macy’s, Nordstrom, Lands’ End & Urban Outfitters.

  • I liked that! LOOK, if you're doing MLM then you should try using MLMRC.com it automates your downline building. It's the only service that does the recruting for you. Works with any mlm company.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

How to Benchmark Your Online Customer Experience Against Competition

  1. 1. How to Benchmark Your Online Customer Experience Against Competition 1
  2. 2. 2 Conducting Remote Unmoderated Usability Studies: Part 1 Ann Rochanayon Sr. Director UX/CX Research UserZoom arochanayon@userzoom.com Speaker Meet the Speakers Elijah Degen VP, Head of Digital Marketing Practice Interakt elijahd@interaktco.com Speaker
  3. 3. 3 The #1 Platform for Agile Usability Testing & User Experience Analytics Our all-in-one enterprise solution makes research & testing cost-effective, agile and easy
  4. 4. 4 Quick Housekeeping • Chat box is available if you have any questions • There will be time for Q&A at the end • We will be recording the webinar for future viewing • All attendees will receive a copy of the slides/recording Continue the discussion using: #uzwebinar Continue the discussion using #uzwebinar
  5. 5. 5 UserZoom Retail Benchmark | December 2014 Contents • How CE/UX Benchmarks Add Value to Your Brand • About Remote Unmoderated User Testing • Case Study: Retail Competitive Benchmark Study  Study Methodology  Participant Profile  Summary  Brand, Conversion, and NPS Findings
  6. 6. 6 How can CE/UX benchmarks add to value to your brand?
  7. 7. Online User experience – the defining moment 7 The content and quality of each individual visitor’s interaction — the user experience — collectively defines your digital brand, communicates your value proposition, and creates the foundation for future brand interactions — positive and negative. Perceived Credibility Profitability Intent to Return Intent to Purchase User Satisfaction Word of Mouth A Positive Experience A Negative Experience Expectations User Experience
  8. 8. 8 Lots of macro benchmarks are out there… • Traffic, user demographics, media spend, etc… Why are there few benchmarks for CE/UX? • Customer Experience & User Experience have varying definitions between companies • Usability benchmarks are generally custom • Competitors vary depending on who you ask • Info from UX benchmarking is so actionable, it is rarely shared
  9. 9. Where can UX benchmarking impact decisions? 9 Area of Discovery Example value Best Practices in service/UX design Leverage competitors findings and save costs Service/UX Design Decisions Justify investments/costs via VOC Or test prototypes Know your customer Know what keeps them from converting Quantify to customer feedback Give hard numbers to show impact. Example Question: For those who considered your brand, what are the top 5 reasons why not converting? Online Brand Impact Help brand marketers understand the brand's impact in online experience Eg Brand Perception can impact conversion as much as online task completion.
  10. 10. 10 What can be measured? Web design components • Navigation, content, site performance, interactive elements. • Example Metrics: Task Completion, Ease of Use, Time on Task, etc. Brand components • Voice of Customer, Brand affinity/likeability, Attitudes toward Brand • Example Metrics: NPS, CSAT, Brand Attributes Matrices
  11. 11. 11 What is needed to benchmark properly? Area of Discovery Example value Correct tools • cross device • website independent • In-app • Video & audio & clicks Correct Audiences • Target Audience • Counter Audiences eg Lost Opportunity Audience, Control Group Correct Proctor / Partner • Impartial = Let the numbers speak for themselves • Specialists w/ Broad Experience Correct Type of Testing • Moderated In-Person (powerful exploratory / directional tool, but EXPENSIVE) • Unmoderated Remote Testing (great deep dive analysis and very cost scalable globally)
  12. 12. 12 UserZoom Retail Benchmark | December 2014 Remote Unmoderated User Testing • ‘In-the-wild’ testing (natural context) • Geographic representation • No moderation needed • Software collects data, stores securely in the cloud, accessible real time •Test any web-based UI, live or prototype • Output: Usability + Behavior + Video & Audio + Satisfaction+NPS+Survey Feedback • Fast data collection • Cost efficient
  13. 13. 13 UserZoom Retail Benchmark | December 2014 Methodology • Goals • Determine which retail site presents the best tablet shopping experience • Assess the impact of the tablet shopping experience on brand perception, Net Promoter Score and conversion • Sites evaluated • Participants tested one of four sites in Nov - Dec 2014 on a tablet • N=202 • Ages 18+ • Shopped for clothing and accessories online at least once per year
  14. 14. 14 UserZoom Retail Benchmark | December 2014 Methodology: Study Design • Between-subjects design • N=202 (~50 per site) Screener Introductory Questions Macy’s 1. Find a jacket 2. Return policy 3. Checkout Nordstrom 1. Find a jacket 2. Return policy 3. Checkout Urban Outfitter 1. Find a jacket 2. Return policy 3. Checkout Lands’ End 1. Find a jacket 2. Return policy 3. Checkout Final Questions
  15. 15. 15 UserZoom Retail Benchmark | December 2014 Participant Profile: Purchase experience Please indicate your gender. 84% 16% Female Male 5% 42% 17% 14% 21% 18 - 25 26 - 35 36 - 45 46 - 55 Over 55 Please indicate your age range. Which of the following best describes how frequently you purchase clothes and accessories online? 15% 39% 38% 8% Weekly Monthly Several times per year Once per year N=202
  16. 16. 16 UserZoom Retail Benchmark | December 2014 Participant Profile: Sites Browse / Purchased • Kohl’s and Macy’s were the top two sites participants visited to purchase clothing and accessories Which of the following websites have you browsed for clothes and accessories using your tablet? / Which of the following websites have you purchased clothes and accessories using your tablet? [Check all that apply] 20% 11% 14% 14% 34% 13% 62% 21% 14% 52% 26% 6% 7% 10% 18% 5% 38% 9% 6% 37% 10% 5% Aeropostle Banana Republic Dillard’s GAP J. Crew Kohl’s L.L.Bean Land’sEnd Macy’s Nordstrom Urban Outfitters Browsed Purchased N=202
  17. 17. 17 Summary
  18. 18. 18 UserZoom Retail Benchmark | December 2014 User Experience Rankings User Experience Score* 79 72 70 55 1 2 3 4 To assess the overall user experience ranking of the sites, UserZoom captured usability metrics for three key tasks on each tablet site. Participants were asked to find an item meeting specific criteria, find return policy information and asked to evaluate the checkout process. The usability metrics captured include key performance indicators such as task effectiveness, ease of use, and clarity of information. The rankings were derived according to how each site performed on each measure. *Score based out of 100 possible points
  19. 19. 19 UserZoom Retail Benchmark | December 2014 Summary: Task Effectiveness • Participants were most successful in completing the study tasks on the Lands’ End site • Urban Outfitters had the lowest overall task success rates 96% 98% 98% 98% 98% 88% 86% 90% 94% 79% 69% 85% Task 1: Find a Jacket Task 2: Return Policy Task 3: Checkout Macy's N=50 Nordstrom N=49 Urban Outfitters N=52 Lands' End N=51
  20. 20. 20 UserZoom Retail Benchmark | December 2014 Summary: Task Ease of Use • Lands’ End performed relatively well overall on task ease of use • Urban Outfitters had the low ease of use ratings, consistent with the lower task success rates 72% 48% 68% 65% 67% 78% 37% 38% 62% 69% 61% 86% Task 1: Find a Jacket Task 2: Return Policy Task 3: Checkout Macy's N=50 Nordstrom N=49 Urban Outfitters N=52 Lands' End N=51
  21. 21. 21 UserZoom Retail Benchmark | December 2014 Summary: Lands’ End • Lands’ End led all sites with the highest ratings in the usability metrics • Nearly all participants succeeded in all three tasks, more than any other site • Lands’ End also had relatively high ease of use ratings and was rated highest on post-study brand perception • Participants found the Lands’ End site fast, simple, easy to use and understand; Lands’ End also provided clear return policy information • Although Lands’ End rated highest on task success and fewest issues encountered for the Return Policy task, one area of improvement to consider would be cleaning up the look and feel / organization of the Return policy page
  22. 22. 22 UserZoom Retail Benchmark | December 2014 Summary: Lands’ End • Study participants appreciated the fast, simple, easy to navigation with clear information available on Lands’ End Simple and easy to use “Very easy to navigate” “Overall a very easy pleasant experience” “Very easy, big selection” “A lot of ways to narrow your search. Easy checkout process” “Everything was so simple and easy to do.” Clear Information, Easy to Understand “Easy to navigate site. Information clear and easy to understand. No issues.” “The site was very easy to navigate, all necessary information was given, pictures and descriptions of merchandise was clear” “Very easy to maneuver around, easy to understand return policy” Fast “Easy and fast to find what I was looking for” “Your website does not lag at all which made it an enjoyable experience.”
  23. 23. 23 UserZoom Retail Benchmark | December 2014 Summary: Nordstrom • Nordstrom edged out Macy’s for second place in the user experience rankings • A vast majority of Nordstrom participants succeeded in all tasks with 88% or more succeeding • Task ease of use were fair with 65% of the participants or higher finding the site easy to use while performing the assigned tasks • Participants overall found the Nordstrom site easy to navigate and several mentioned the helpfulness of the filters • However, a sizeable percentage of participants encountered technical issues on the Nordstrom site with 27% reporting site slowness and 24% had problems with pages jumping around when looking for a product • Another opportunity for improvement for Nordstrom would be providing clear return shipping cost information; 27% reported frustration not understanding the cost to return an item by mail
  24. 24. 24 UserZoom Retail Benchmark | December 2014 Summary: Nordstrom • The ease of navigation and product filters were cited as being the most helpful aspects of the Nordstrom site Easy to Navigate “It was easy to use their website and find the information I needed to make a purchase” “Very easy to navigate and checkout...also great return policy with free returns” “How relatively easy it was to make purchases and to be able to ship those items by mail.” “Easy to navigate. Inviting layout. Easy to add items to shopping bag and to checkout.” Easy-to-use Filters “The website was set up for easy purchases. And the filters made my shopping very easy and fast. Very impressed!” “The whole thing was very smooth and easy to understand. The filters, the check out, customer service- it’s all very accessible.” “The site was pretty easy to navigate with lots of filtering options. Most of the information I looked for was easily found” “Easy to use the filters to narrow a search.”
  25. 25. 25 UserZoom Retail Benchmark | December 2014 Summary: Macy’s • Macy’s finished in third place based on the user experience rankings • Nearly all participants in succeeded in finding a jacket meeting specific criteria and finding the return policy while 86% succeeded with the checkout task • Despite the high success rates in finding the return policy, only 48% found the task easy with 36% reporting difficulty understanding the shipping cost and 18% struggling to find return policy information • 68% found it easy to check out on the Macy’s site; technical issues with the site were the primary problems encountered • 16% mentioned the pages were too slow during checkout and 12% had problems with the page jumping around when clicking
  26. 26. 26 UserZoom Retail Benchmark | December 2014 Summary: Macy’s • Participants found Macy’s easy to navigate, but also mentioned that the site was fast and offered a good tablet experience Easy to Navigate “Fast. Easy to find things and search” “Ease of the website, clear self explanatory instructions on how to check out and add things to my bag” “Site was very easy to navigate, simple links and the filter/sort feature was very fast.” Fast Site “Many products, fast loading” “The site was easy to navigate, loading time was fast and almost everything was very easy to locate.” “Very easy and clear to navigate. Loaded quickly and responded in a good amount of time.” Good Tablet Site “It worked really well on my tablet. I do most of my online shopping on a tablet and some sites work better then other. I would say this one was easy to use its not a site that I would have to switch to another computer to use.” “I like the mobile version better than desktop”
  27. 27. 27 UserZoom Retail Benchmark | December 2014 Summary: Urban Outfitters • Urban Outfitters lagged behind the other sites in fourth place • Participants were least successful finding a jacket, finding return policy information and checking out • Ease of use ratings were also lower than the other sites • Only 37% found it easy to find a jacket with 23% struggling with the amount of scrolling, 17% struggling with the filters and 17% having problems narrowing down results among other issues • With the return policy, 37% had problems finding it and another 13% felt there was too much scrolling required
  28. 28. 28 UserZoom Retail Benchmark | December 2014 Summary: Urban Outfitters • Participants had greater difficulty navigating Urban Outfitters due to the scrolling required along with technical problems with the site Difficult to Navigate “Everything so complicated to find. Too much scrolling to find an item. Info text too small to read” “Not easy to navigate thru to see items. Need more groupings etc., men’s coats, women’s coats, shoes men’s or women’s.” “It is hard to maneuver to categories because the pictures are so big for the categories.” “Images were too big, I had to scroll around too much” Unclear Icons “Also need to identify the BAG icon.” “Icons are not easily identifiable.” Technical Issues “The checkout kept asking for my shipping address and then would reload the same page. I did this 4 times.” “The page did not fully load” “The page would not bring up the items; the page kept going blank”
  29. 29. 29 Brand, Conversion, and NPS Findings
  30. 30. 30 UserZoom Retail Benchmark | December 2014 Brand Perception • After interacting with the site, Lands’ End had the biggest increase in positive brand perception 62% 57% 23% 18% 62% 69% 44% 76% N=37 N=50 N=37 N=49 N=31 N=52 N=39 N=51 Macy's Nordstroms Urban Outfitters Lands' End Pre-site evaluation Post-site evaluation Site Delta Macy’s 0% Nordstrom 14% Urban Outfitters 21% Lands’ End 58% * Selected 6-7 on a 7-point scale where 1 = Negative and 7 = Positive
  31. 31. 31 UserZoom Retail Benchmark | December 2014 Call to Action • Participants were most likely to make a purchase from Macy’s or Nordstrom and least likely to to make a purchase on the Urban Outfitters tablet site 52% Based only on your experience today, how likely are you to make a purchase on [SITE] on your tablet in the future? 47% 15% 39% 38% 35% 43% 57% 10% 18% 42% 4% Macy's N=50 Nordstrom N=49 Urban Outfitters N=52 Unlikely (1-2) Neutral (3-5) Likely (6-7) Lands End N=51
  32. 32. 32 UserZoom Retail Benchmark | December 2014 Net Promoter Score • Participants were most likely to recommend Macy’s, followed by Nordstrom • Urban Outfitters had far more detractors than promoters with a -56% NPS NPS 10% -6% -56% - 23% How likely are you to recommend [SITE] to a friend or 36% 43% colleague? 69% 45% 18% 20% 17% 33% 46% 37% 14% 22% Macy's N=50 Nordstrom N=49 Urban Outfitters N=52 Lands End N=51 Detractor (0-6) Passive (7-8) Promoter (9-10)
  33. 33. 33 UserZoom Retail Benchmark | December 2014 Post Site Interaction Brand Attributes* • Nordstrom was associated with ‘Customer-oriented’ and ‘High quality,’ while Macy’s was most associated with ‘Value’ and ‘Variety’ 42% 10% 51% 54% 37% 40% 52% 24% 63% 18% 41% 19% 23% 40% 12% 35% 8% 51% 31% 33% Customer-oriented Exclusive High quality Value Variety Macy's Post N=50 Nordstrom Post N=49 Urban Outfitters Post N=52 Lands End Post N=51 Based only on your experience today, which of the following attributes do you associate with [COMPANY]? * Partial list displayed
  34. 34. 34 UserZoom Retail Benchmark | December 2014 Change in Brand Attributes* • Perceptions of ‘Customer-oriented’ and ‘Variety’ increased for all sites from pre- to post-study • Macy’s and Nordstrom had decreases in brand association with ‘Exclusive’ and ‘High quality’ 10% -10% -26% 6% 4% 27% -15% -17% 0% 21% 19% 6% 11% 13% -3% 23% 0% 12% 15% 25% Macy's N=50 Nordstrom N=49 Urban Outfitters N=50 Lands End N=51 Customer-oriented Exclusive High quality Value Variety Which of the following attributes do you associate with [COMPANY]? * Partial list displayed
  35. 35. Usability Testing • Web & Mobile Task-based Testing • UX & Usability Benchmarking • Competitive Research • Concept Testing IA & UX Design Validation • Navigation Optimization - Card Sorting • Findability - Tree Testing • Design Research - Click Testing • Impression Testing - Timeout Testing Customer Experience Measurement • Voice of Customer- Live Intercept surveys • Customer Feedback & Sat- Online Surveys • Web Analytics Integration 35 Introduction to UserZoom Suite of tools For All stages of your product life cycle
  36. 36. 36 UserZoom Data | Results
  37. 37. 37 Summary • Benchmarking is valuable to investment decisions • Benchmarking can give you unparalleled insight and a competitive edge • Nearly every customer perspective can be measured • Partner, Tool & Audience selection is key to success • Retail is audiences are very measurable for online user experience & offline in-store experience • Task Metrics are key to conversion improvement • Brand Metrics are very measurable via Remote Testing • Lift in Brand Perception by a website can be HUGE • Know if your website is presenting the desired Brand Attributes
  38. 38. 38 Questions

×