What Can Be Learned From a
 One-Minute Online Survey
  How people really feel about Amazon.com
        (and what does this...
Foreword: Web teams need constant feedback
By Gerry McGovern



Living systems get constant feedback from their external e...
What Can Be Learned From a One-Minute Online Survey




The regular customers didn't feel much happier. "45% of all custom...
What Can Be Learned From a One-Minute Online Survey




Executive summary
In order to demonstrate what can be learned from...
What Can Be Learned From a One-Minute Online Survey




Table of contents
Introduction ......................................
What Can Be Learned From a One-Minute Online Survey




                             Introduction
                        ...
What Can Be Learned From a One-Minute Online Survey




Key findings
A very clear trend emerged from the results – the top...
What Can Be Learned From a One-Minute Online Survey




Results
Participants came from a pool of demographically evenly-sp...
What Can Be Learned From a One-Minute Online Survey




                              Analysis
                           ...
What Can Be Learned From a One-Minute Online Survey




                             There are many other examples of how ...
What Can Be Learned From a One-Minute Online Survey




                          Amazon search has customer-oriented feat...
What Can Be Learned From a One-Minute Online Survey




                            The bigger takeaway story on this fact...
What Can Be Learned From a One-Minute Online Survey




                         Sears, a retailer that has announced majo...
What Can Be Learned From a One-Minute Online Survey




                                Visual appearance not of overridin...
What Can Be Learned From a One-Minute Online Survey




Report Summary
Consumers want choice and they want low prices. Tha...
What Can Be Learned From a One-Minute Online Survey




Conclusions
We produced this report to step through a complete pro...
What Can Be Learned From a One-Minute Online Survey




Methodology
We created an online poll and asked participants to te...
What Can Be Learned From a One-Minute Online Survey




The statements as they were presented to participants (randomized ...
USA

                                           The Customer Respect Group
                                           978....
Upcoming SlideShare
Loading in …5
×

What Can Be Learned From A One-Minute Online Survey

863 views

Published on

What do people who use the Amazon website think of their experience?

Results of this Customer Centric Index survey reveal the strengths of the website and changes that might make it even better.

Published in: Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
863
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

What Can Be Learned From A One-Minute Online Survey

  1. 1. What Can Be Learned From a One-Minute Online Survey How people really feel about Amazon.com (and what does this mean)? An Illustrative Case Study Using
  2. 2. Foreword: Web teams need constant feedback By Gerry McGovern Living systems get constant feedback from their external environment. To truly succeed, web teams need constant feedback from their customers. You're a manager in a restaurant. It's raining. A customer walks in and almost slips on the mat in front of the door. You're very busy at this stage, but you make a mental note: "I must change that mat." About 15 minutes later another customer comes in. She, too, almost slips on the mat. You rush up to her, apologize profusely and then change the mat. People are slipping on our websites right now but, because we don't see them slip, we don't change the mat. I'm one of the biggest offenders. Over the years I have left content and applications on my websites that had problems that I was vaguely aware of, but they just didn't seem important enough to warrant any action. Even when I became clearly aware of the issue I didn't react with enough urgency. Why was that? Why was I so complacent? I would like to think that if I was running a restaurant I would have apologized to the customer and changed the mat. Why don't I do that when it comes down to managing a website? I think a core part of the problem is the lack of real feedback. I'm not actually seeing the customer slip. I don't actually see real people use my websites. Customers are hugely impatient on the Web. When they slip, their first impulse is to hit the Back button. Jared Spool wrote an excellent article in 2009 called the "The $300 Million Button." In it he explained how the removal of a registration button from a particular step in a purchase process resulted in a dramatic improvement in sales. The Web team had created the registration button so as to make it easier and faster for regular customers to buy. But people absolutely hate registration. New customers felt they would be spammed if they registered. One potential customer summed up their feelings as follows: "I'm not here to enter into a relationship. I just want to buy something."
  3. 3. What Can Be Learned From a One-Minute Online Survey The regular customers didn't feel much happier. "45% of all customers had multiple registrations in the system, some as many as 10," Jared wrote. "We also analyzed how many people requested passwords, to find out it reached about 160,000 per day." The Web is so important today. And yet many of the web teams I deal with are way down the management hierarchy. Intranet teams, in particular, tend to get negligible resources. That needs to change because the reality is that the Web is central to the present and future success of most organizations. One of the ways we make that change happen is that we start developing much better feedback mechanisms for our websites. At a most basic level, we must find ways to regularly (weekly at minimum) observe our customers carry out top tasks on our websites. That's how Jared Spool discovered there was a problem: by watching customers trying to buy. According to Wikipedia, "Living things are systems that tend to respond to changes in their environment." Let us embrace our customer environment. Let us observe and evolve. The rewards are very substantial. Gerry is the founder and CEO of Customer Carewords. He is widely regarded as the worldwide authority on increasing web satisfaction by managing customer tasks. Gerry has spoken, written and consulted extensively on web content management issues since 1994. His new book, The Stranger's Long Neck: How to Deliver What Your Customers Really Want Online, was published in July 2010. Read the first chapter of the book at http://www.gerrymcgovern.com/sln-ch1.htm 3
  4. 4. What Can Be Learned From a One-Minute Online Survey Executive summary In order to demonstrate what can be learned from a carefully-designed survey that is very quick to complete, we asked a combination of consumers and Web professionals to rate Amazon.com (chosen as it is very familiar to many people). As expected, the site was rated very positively yet there was still much to learn. The most compelling characteristic of the site for most visitors was the ratings and reviews feature. But while ratings and reviews are a strength now, they might also be under threat. The growth of social media has diminished the advantage of ratings and reviews by strangers and Amazon is arguably no longer at the leading edge in this respect. Other findings demonstrate that female visitors like to ‗get things done quickly‘ and are more troubled by the difficulty of ‗contacting a person‘ while men value ‗complete information‘ a little more. Perhaps surprisingly, Web professionals complained more than consumers that Amazon.com had a cluttered layout, hinting that expert opinions cannot be relied on exclusively without the input of regular Web users. 4
  5. 5. What Can Be Learned From a One-Minute Online Survey Table of contents Introduction ....................................................................................... 6 Key findings........................................................................................ 7 Results ............................................................................................... 8 Analysis.............................................................................................. 9 Ratings, reviews and recommendations are popular but under threat ... 9 People like Amazon‘s search but there is no room for complacency .... 10 Clutter and layout were problems, but not for all .............................. 11 Some people find it hard to contact a person ................................... 12 Visual appearance not of overriding importance ............................... 14 Methodology ..................................................................................... 16 5
  6. 6. What Can Be Learned From a One-Minute Online Survey Introduction Consumers have opinions on websites; they express them to friends and family, not in the technical and analytical terms that professionals might use, but in quick summary statements that capture the essence of the sites. Typically, Web surveys ask consumers to rate sites in technical terms, but this is expecting consumers to think the way we experts do, to tell us what and where site issues exist. . We need to simulate being present at hundreds of social gatherings and eavesdropping on conversations about websites. The approach used in this Amazon We instead need to review, tried and trusted over many years, puts the onus on the analyst to interpret collect what people feel consumer sentiment, and makes it easier for them to express their feelings. It was strongly about; what our goal to run through a illustrative example from start to finish to show you the they would say over the level of actionable information can be gleamed from this simple and comparatively barbecue or at the water inexpensive method. You can be the judge of the success of that objective. cooler This technique does not ask questions in company-centric language (e.g. ‗Please rate the options that are available for you to navigate this website?‘), but instead simply asks people to express what they felt strongly about. This ensures that their instinctive reactions are captured. It also helps that the survey was very quick to complete — it genuinely took less than a minute and nobody abandoned it. The success of the survey depends of course on the results. You have your opinion of Amazon, so how well do the findings match your expectations, but also surprise and provide ‗food for thought‘? Ultimately, what value does this survey bring to website analysis? This study was conducted by The Customer Respect Group using the Customer Carewords methodology. It is far from perfect. We would have liked larger participant numbers and we did invite web professionals with their different preconceptions to get involved, which may have skewed overall results a little. But a lot of data and analysis emerged from what was, superficially, a very simple survey. 6
  7. 7. What Can Be Learned From a One-Minute Online Survey Key findings A very clear trend emerged from the results – the top five attributes accounted for over half of the overall impressions. The strong trend showed that the survey successfully identified the aspects of Amazon.com that are most compelling. The positive sentiments were dominant and the site recorded an extremely high rating of 86 out of 100. The general rating for websites tested with this methodology is typically in the 50-60 range. The top five attributes were: 1. Site has ratings, reviews and recommendations 2. Helpful search results 3. Fast to do things 4. Simple layout/easy to read 5. Accurate information An important consequence of the survey is that it highlights site strengths just as much as weaknesses. So while ratings, reviews and recommendations are a much-loved feature this strength can become a threat should competitors neutralize their value. Although the negative sentiments were in the minority, a significant minority is not happy with some aspects of Amazon.com. Even a successful site like Amazon can be improved. Given the volume of traffic, a small improvement could have a significant effect. The three main areas of concern were: 1. Cluttered layout 2. Hard to contact a person 3. The site was visually unattractive 7
  8. 8. What Can Be Learned From a One-Minute Online Survey Results Participants came from a pool of demographically evenly-spread consumers (approx 60% of the total) and web professionals. The graph below reflects the importance of each of the 26 attributes in the survey. A red bar indicates a negative sentiment. Has ratings, reviews, recommendations 16.3% Helpful search results 13.4% Fast to do things 9.3% Simple layout / easy to read 8.4% Accurate information 6.2% Complete information 6.1% Easy to participate / give feedback 5.2% Clear menus and links 5.0% Looks attractive / appealing 4.9% Cluttered layout / hard to read 4.2% Up-to-date information 4.1% Gives me the facts / transparent 3.0% Hard to contact a person 2.3% Plain language 2.2% Looks unattractive / unappealing 2.2% Easy to contact a person 2.1% Slow to do things 1.4% Confusing menus and links 1.2% Poor search results 1.1% Incomplete information 0.4% Hard to participate / give feedback 0.4% Out-of-date information 0.2% Full of jargon / corporate speak 0.2% Has no ratings, reviews, recommendations 0.2% Misleading / not transparent 0.2% Inaccurate information 0.0% 8
  9. 9. What Can Be Learned From a One-Minute Online Survey Analysis The data is a signpost and tells us where to look; the interpretation brings the data to life and leads to actionable tasks and strategies to pursue. With websites so complex, it is critical to garner consumer sentiment continuously to reflect changing expectations as much as changes in functionality. When so many companies focus on customer retention and growth, sentiment towards websites is a critical metric that up to now has been difficult to harness. Ratings, reviews and recommendations are popular but under threat 84% of US customers prefer The presence of ratings, reviews and recommendations was the top-rated attribute. the opinion of other It was once what differentiated Amazon.com but the landscape has changed. Most customers versus experts. retailers have adopted, and even improved, reviews and for very good commercial reasons: —Marketing Sherpa  78% of Internet users rate recommendations as most credible form of advertising (Neilsen)  84% of US customers prefer the opinion of other customers versus experts (Marketing Sherpa)  96% of online retailers rank customer ratings and reviews as an effective or very effective tactic for driving conversion (Forrester)  65% trust ‗friends‘ recommendations while 33% trust company-supplied recommendations (emarketer)  69% of consumers who read reviews then go on to share them with family and friends (Deloitte)  Products that have reviews show a 35% increase in conversion rates (Bazaarvoice) One weakness of the Amazon implementation is that it does not allow for recommendations by friends. We can read that hundreds of people rated this product as 5-star, but it‘s not immediately clear if they share our tastes or preferences. A clear finding of various studies is that consumers want to hear from ―someone like me‖. Best Buy‘s Facebook page, which has almost 1.2 million fans (compared to Amazon‘s 90,000) includes a ‗Shop + Share‘ tab on which consumers can browse products and traditional reviews. A key enhancement, however, is the ‗Ask Friends‘ button which allows fans to collect input on their decision from friends‘. This makes it very easy to get the opinions of trusted peers. Now, friends can offer an opinion before we buy, in a manner similar to bringing a friend to a bricks-and-mortar store. 9
  10. 10. What Can Be Learned From a One-Minute Online Survey There are many other examples of how ratings and reviews have evolved and developed. Once considered a unique feature of Amazon, this could be a game-changing feature and Amazon can‘t afford to fall behind the innovation curve in an area so ingrained in its users‘ experience. If a key strength is perceived as being inferior to systems in use elsewhere, there is a danger that Amazon might lose some of its popularity. Best Buy‘s Facebook page brings reviews and ratings to a new level by allowing people to ask their friends for advice on selected products Best Buy has recruited 15 times more ‗fans‘ on Facebook than Amazon and more likely to benefit from recommendations from friends People like Amazon’s search but there is no room for complacency Good search results was considered to be the next most important attribute. Search is sometimes considered to be ‗Plan B‘ when navigating through menus and links has not resulted in success, but in this case it seems unlikely. Very few people chose ‗confusing menus and links‘, so it seems that Amazon‘s search is actually particularly good – good enough to be a Plan A when arriving on the site. Customers know that searching for that book or DVD will produce really accurate and fast results. 10
  11. 11. What Can Be Learned From a One-Minute Online Survey Amazon search has customer-oriented features that can help them find what they need very quickly. Sub-categories on the left menu allow customers to filter results Amazon cannot if the default results are not quite right, ratings are visible, and related searches are afford to neglect suggested. improving its search Amazon cannot afford to neglect improving its search facility. There are many web facility... 34.5% of shoppers that are not wedded to an Amazon-like one-stop shopping experience. traffic to top retail According to Hitwise, 34.5% of traffic to top retail sites in July 2010 came directly sites in July 2010 from search engines. came directly from Google‘s new instant search introduces the concept of consumers fine-tuning [external] search search terms dynamically. If this proves popular, Amazon will need to respond. There are also examples of reviews being included in search results (Bazaarvoice is supplying reviews to Google that includes content from sites like BestBuy and Macys). As Facebook becomes more and more a destination site for consumers, its search feature might drive traffic to rival sites, particularly if their Facebook presences are more advanced or feature-rich. Negative sentiments were in the minority but can’t be ignored The important thing about negative sentiments in a one-minute survey is that they help companies isolate general issues and provide the roadmap of where to start ‗digging‘. Furthermore, we have found that Web teams welcome some negative findings, which are often seen as a validation of their own opinions. A more typical ratio of positive to negative is 65/35 or 70/30, and in these cases the discussion of how to rectify problems would dominate the analysis. Clutter and layout were problems, but not for all The most significant negative aspect of Amazon.com was ‗cluttered layout, hard to read‘. Clearly, page layout matters - it has direct impact on the ability to complete tasks. Too many long blocks of text, too many links, clutter, not enough white space, lack of a coherent structure; these can all slow people down. According to a consumer for whom this is the most important facet of Amazon.com: I‘ve used the site many times and have found it confusing to use. It has taken me too long in the past to get to the information I need or to find out that I had to do something else to get where I wanted to be. Just a hassle! - Female consumer, 25-34 years old, quite familiar with the site 11
  12. 12. What Can Be Learned From a One-Minute Online Survey The bigger takeaway story on this factor was that this was much more important for professionals than consumers. It was the only factor where professionals differed ... ‗you guys are so from consumers to any degree of significance. It may indicate that web primitive, you are professionals tend to judge layout more than consumers do, years of experience like cavemen. having trained them to look at websites differently. Very few Web professionals Don‘t you have any would not criticize Craigslist yet it remains a very popular site. According to Jim sense of Buckmaster, CEO of Craigslist: aesthetic?‘... ―I hear this all the time, ‗you guys are so primitive, you are like cavemen. Don‘t In all the you have any sense of aesthetic? ‗ But the people I hear it from are invariably complaints and working for firms that want the job of redoing the site. In all the complaints and requests we get from users, this is never one of them. Time spent on the site, the requests we get number of people who post—we‘re the leader. It could be we‘re doing one or two from users, this is things right.‖ never one of them. To reinforce this view, far more people chose ‗simple layout, easy to read‘, with consumers more satisfied with the layout than the professionals. — Jim Buckmaster, CEO of Craigslist Although subjectivity is also a factor in this attribute, the difference of opinion shows the importance of getting the opinions of non-expert consumers, who vastly outnumber the Web professionals in the online space. Without intending to, we were able to show that web professionals may not always be the best predictors of consumer sentiment. Some people find it hard to contact a person Sears offers chat, call At one time, it was extremely difficult to get in touch with Amazon support staff. back and email on Things have improved, but it still is not that easy to engage with a representative. every page to assist The issue is the lack of real-time escalation – being able to contact an agent by the consumer phone or chat without leaving the page. Many competitive sites offer telephone complete the buying numbers, live chat, instant callback and click-to-call to ensure that consumers don‘t process — this is a lose momentum by having to navigate to another page to resolve an issue. According to Art Technology Group, 58% of consumers now look for live help if standard of missing information while online, while 53% will do so if there are problems availability of help checking out. It was interesting that men looked for help less than women, that sets reinforcing the general perception of men having an aversion to asking for expectations for directions. visitors to sites like Amazon‘s. 12
  13. 13. What Can Be Learned From a One-Minute Online Survey Sears, a retailer that has announced major plans to expand its online offering with the adoption of an ‗Amazon-like‘ marketplace that offers products from third parties, Chatters that engage provides prominent options for consumers to reach out. Service agents can view the via proactive contents of the consumer‘s shopping cart and answer any questions. This helps to invitation are 6.3X reduce cart abandonment, a major issue for online retailers. Sears offers chat, call-back and email on every page to assist consumers complete the buying process more likely to convert — this is a standard of availability of help that sets expectations for visitors to sites than visitors who like Amazon‘s. don‘t chat. Live chat with an agent from product pages allows Sears customers to escalate issues or get answers without interrupting their tasks — Bold Software Live Chat Performance Benchmarks October 2009 It would be very revealing to drill down with the participants that expressed concern about contact options to find out, for example:  Why is this an issue – what was the reason they needed to contact Amazon when they found it difficult?  Was the issue they tried to escalate to do with a site issue, or perhaps an offline issue like late delivery or a billing discrepancy?  Did the issue cause them to abandon a purchase and go elsewhere?  What were their expectations of Amazon – a telephone number on every page, or just an easier path to an email contact form? 13
  14. 14. What Can Be Learned From a One-Minute Online Survey Visual appearance not of overriding importance Visual appearance is of course another subjective characteristic, but what is striking is that the ‗look‘ of the site was not important to most people, whether they liked it or not. (There was only a slight difference here between consumers and … the ‗look‘ of the site was professionals.) not important to most Functionality seems to be what matters — with the appearance of Amazon not people, whether they liked getting in the way of people browsing products and buying. This indicates that once it or not the appearance of a website is not strikingly bad (and Amazon‘s certainly isn‘t), it does not really register prominently with visitors. Once customers can do what they need to do, they‘ll not be too concerned about colors, fonts, imagery or other design features. That‘s not to say that any design will do – it still needs a professional to design an interface that is visually coherent and serves the brand well. But once people can get where they need to go quickly, and complete their tasks, the visual appearance is very much secondary to functionality. Consumers are probably too short on time to notice the visual design of site that is designed to get things done. This finding also raises further questions:  What type of site visitor finds Amazon.com unattractive?  Are new visitors more critical of the appearance, perhaps it becomes less important as they find out that the site is functionally excellent?  What exactly about the appearance is causing an issue – is it related to clutter, or is it a purely subjective attribute like the choice of colors and fonts?  Why do they feel strongly enough about appearance to pick this attribute rather than one of the other 24 available? 14
  15. 15. What Can Be Learned From a One-Minute Online Survey Report Summary Consumers want choice and they want low prices. That‘s a big reason for the success of Amazon.com. Alongside its own merchandise, Amazon also offers goods from other retailers and the revenue from this ‗marketplace‘ in the first half of the year was roughly $4.2 billion. Major online competitors including Sears and Wal-Mart are aggressively building their own marketplaces and industry experts say several more large retailers will launch similar efforts in the coming months. So, does our study help understand Amazon‘s strengths, weaknesses, opportunities and threats?  Clearly, ratings and reviews are a critical attraction for Amazon but they need a significant upgrade to exploit new channels of social media, especially Facebook. Sears has already developed a sophisticated online community which looks to be more advanced than Amazon‘s. This is critical to consumers and in a world of diminishing loyalty may serve to weaken Amazon‘s dominant position  Customers increasingly see online retail as a multi-channel experience and it is likely that Amazon will need to provide better options for speaking to a service person in real time. The study indicated a small (but what is likely to be a growing) dissatisfaction in this area. A continuous review process would be able to monitor that. Both Wal-Mart and Sears already offer better capabilities as well as store pick-up options and are experimenting with same-day delivery in major cities  Search is highlighted as critical and while Amazon‘s is good, there are sites which already surpass its capabilities. There is the added threat of Facebook and Google search as the most popular starting points for consumers. In summary, Amazon received a very positive rating so on the surface it has little to put right. But as we have shown significant challenges are emerging. 15
  16. 16. What Can Be Learned From a One-Minute Online Survey Conclusions We produced this report to step through a complete process; we used Amazon simply because many people are familiar with the site and have impressions. Many readers of the report will have provided input so you know that it did take the 1 minute we say. You all have regular visitors to your sites; what are their impressions, what effect does that have on viral recommendations or on repetitive use. The benefit of the methodology is the combination of a unique survey technique in combination with the expert interpretation from a range of partners that have extensive online experience. Already the methodology has been applied to sites across multiple industries and countries, it provides great value for a comparative inexpensive commitment. 16
  17. 17. What Can Be Learned From a One-Minute Online Survey Methodology We created an online poll and asked participants to tell us which three of 26 statements most closely described how they felt about Amazon.com, and allocate ‗3‘ to the strongest sentiment, ‗2‘ to the next, and ‗1‘ to the third (mirroring Gerry McGovern‘s widely-used Customer Centric Index). The statements were displayed in a randomly-ordered list. Although there were 26 statements, they were in fact positive and negative sentiments of 13 core attributes. Visual/Architectural factors Clear menus and links Confusing menus and links Helpful search results Poor search results Simple layout / easy to read Cluttered layout / hard to read Looks attractive / appealing Looks unattractive / unappealing Fast to do things Slow to do things Content factors Accurate information Inaccurate information Complete information Incomplete information Up-to-date information Out-of-date information Plain language Full of jargon / corporate speak Social factors Gives me the facts / transparent Misleading / not transparent Easy to contact a person Hard to contact a person Easy to participate / give feedback Hard to participate / give feedback Has ratings, reviews, Has no ratings, reviews, recommendations recommendations 17
  18. 18. What Can Be Learned From a One-Minute Online Survey The statements as they were presented to participants (randomized for each) 18
  19. 19. USA The Customer Respect Group 978.412.4047 http://www.customerrespect.com Net Reflector 206.462.4240 http://www.netreflector.com/ Bob Johnson Consulting 248.766.6425 http://www.bobjohnsonconsulting.com Canada Customer Carewords is a methodology Neo Insight, Inc that helps you to truly understand what 613.271.3001 the top tasks of your customers are www.neoinsight.com when they come to your website and the critical impressions they take away. Europe By continuously improving the per- Ireland formance of your customers' top tasks, Customer Carewords, LLC www.customercarewords.com you will maximize the performance of +353 87 238.6136 your website. United Kingdom We have been developing customer centric strategies for websites since Barry Hagan 1994, identifying the top tasks of +44 77 1470.0066 thousands of customers and employees www.customercarewords.com for organizations such as Microsoft, Brian Lamb Cisco, Tetra Pak, Thrivent Financial, US +44 79 8070.0075 Internal Revenue Service, NHS Choices, www.customercarewords.com Rolls-Royce, BBC, Innovation Norway, etc. We have partners in the UK, Hol- Sweden land, Sweden, Norway, Canada and Webbrådgivaren Sverige AB United States. +46 040.134.200 http://www.wpr.se Netherlands Sabel Communicatie +31 088.227.2240 http://www.sabelcommunicatie.nl Norway NetLife Research +47 9240.3165 http://www.netliferesearch.no 19

×