A Holiday Review
Performance Lessons for 2014
Ken Harker, Senior Consultant
January 2014
Some of the numbers are still coming
in, but…
• US online and e-commerce holiday retail
increased 9.3%
• UK online holiday...
The Mobile Madness Grows
©Keynote Systems, Inc. 3
Retailers surveyed for Forrester
Research’s “State of Retailing
Online 2...
Real Madness Is Not Being on Mobile At All
©Keynote Systems, Inc. 4
Only 72 of the top 100 e-retailers have fully function...
The Price is Right
©Keynote Systems, Inc. 5
On the web, there is always a
lower price.
High performing mobile web
sites ar...
Expectations
On the desktop: 2 seconds or faster
What Do Retail Site Customers Expect?
©Keynote Systems, Inc. 7
On mobile: 3 seconds or...
• Delays in page load time result in increased
abandonment, decreased loyalty, and loss of revenue
• Even very small delay...
Keynote Online Holiday Shopping Study
During the 2013 holiday season, Keynote measured the
performance and availability of 13 top online retailers
• Measurement...
The Sites in the Study
Amazon
Apple Store
Best Buy
CDW
Dell
HP
Newegg
Office Depot
Office Max
Overstock
Sears
Sony Styles
...
The Keynote Holiday Retail Study measured a typical user
journey through the desktop and mobile sites
1. Start at the Home...
Desktop Results
Online retailers had
their best showing
on desktop this
year
Average time to
conduct the
purchase path
transaction was
22....
Across all measured sites, the
slowest step in the online
purchase path was retrieving the
results for a product search
Se...
• The fastest site on Desktop
was Apple, averaging only
14.0 seconds to complete
the entire transaction
Desktop: Speed
©Ke...
The most available
site on Desktop was
HP, with an overall
transaction uptime of
99.6%
Only one site had a
sustained downt...
Smartphone Results
Smartphone
©Keynote Systems, Inc. 19
• Nearly all pages designed
for the smartphone
screen size and for access
over mobile...
Smartphone: Slow Pages
©Keynote Systems, Inc. 20
The slowest page on the
smartphone
transactions, by far, was
the Home pag...
Smartphone: Fastest Sites
©Keynote Systems, Inc. 21
The fastest mobile site
measured was the Sony Style
store, coming in w...
Tablet Results
Tablet
©Keynote Systems, Inc. 23
None of the sites in
the Keynote holiday
study have pages
optimized for tablet
users over...
Retailers are gambling that tablet users are
always on fast WiFi connections
Tablet
©Keynote Systems, Inc. 24
Consumers using tablets on 3G mobile network connections
during the holiday shopping season saw very poor performance
The ...
Lessons for success in 2014
• Slow page load times continue to be the biggest point of
frustration for mobile web users
• Mobile needs to be a continu...
1. Dress Light
©Keynote Systems, Inc. 28
Be disciplined about
loading as few assets
on the page as possible
Different browsers/phones have different numbers of parallel
threads
2. Avoid Long Lines
©Keynote Systems, Inc. 29
3. Bundle up
©Keynote Systems, Inc. 30
Employ optimization
techniques that
reduce the number
of HTTP requests
needed to fe...
4. Anticipate the Rush
©Keynote Systems, Inc. 31
You know it’s
coming, so why
wait until next year
to prepare for the
holi...
Good marketing can
help to improve
conversion rates, but if
too much leads to
downtime, slow
speeds and site
abandonment y...
No one says that you
need to go at it
alone, and in many
cases you shouldn’t.
6. Partner Up
©Keynote Systems, Inc. 33
‘Stuff’ Happens
6. Expect the Unexpected
©Keynote Systems, Inc. 34
Conclusions
2014 is a New Year!
©Keynote Systems, Inc. 36
Thank You
Ken Harker
Senior Consultant, Keynote
ken.harker@keynote.com
Upcoming SlideShare
Loading in …5
×

A Holiday Retail in Review: Performance Lessons for 2014

636 views

Published on

Join Ken Harker, Senior Consultant with Keynote Systems for a look back at the 2013 Holiday Online Shopping Season with a view into site performance.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
636
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 1) In the US, the National Retail Federation reported that online and e-commerce retail sales during November and December, 2013 increased 9.3%*2) The British Retail Consortium reported that online retail sales in November and December in the UK increased 19.2% from 2012**3) Forrester Research reported that overall online retail sales in the US for 2014 may reach $300 billion, and the online experience will influence an additional $1.3 trillion in retail purchases***1 http://www.nrf.com/modules.php?name=News&op=viewlive&sp_id=1745** http://www.bbc.co.uk/news/business-25671561*** http://blog.shop.org/2014/01/14/forresters-mulpuru-2013-great-year-for-e-commerce-2014-even-better
  • Retailers surveyed for Forrester Research’s “State of Retailing Online 2014” study ranked mobile as their TOP PRIORITYHowever, most retailers are NOT investing in their mobile sites as much as their desktop sitesFor many retailers, mobile still accounts for a small fraction of their effortsVery few retailers are doing any optimization at all specific to the tablet user experienceMobile moves very quickly, and many retailers are hesitant to invest heavily in something that might change out from under themThis is impacting the user experience
  • Retail was one of the first industries to recognize the importance of the mobile web to their businessBut some sites are still not onboardShockingly, even today more than a quarter of all retailers do not have mobile-optimized sitesIn the 2013 Internet Retailer Top 500 Guide, Internet Retailer found that only 72 of the top 100 e-retailers had fully functional m-commerce sites optimized for smartphones28 of the top 100 sites either had a mobile-optimized site that doesn’t enable purchasing, a mobile-optimized home page only, or no mobile site whatsoever
  • On the web, there is always a lower priceThis is a double-edged sword for retailersA third of consumers use their smartphones to comparison shop online while they are shopping in a store20% of consumers are willing to pay a small premium to get a product on the spot rather than deal with an online order and shipping delayHighly performant mobile web sites are the ones that consumers will turn to for their comparison shoppingNobody wants to wait for pages to load while standing in the aisles of a storeConsumers will subconsciously choose to visit sites that give them the answers they need quickly
  • Retail Is Growing and ExpandingThe e-retail marketplace is growing and expanding rapidlyAlthough even some top e-retailers are lagging in mobile today, there’s a strong conviction that mobile should be a top priority for the retail industryThe future of e-retail will play out on desktop, smartphone, and tabletsCustomers will have very high expectations
  • 2 second or faster page load times are expected for retail web sites on desktop browsersIn 2006, Google experiments report that extra delays as short as 1/10 second affected sales and ad revenueIn 2006, Amazon reported that a 1/10 second increase in page delay translates into a 1% revenue lossIn 2009, Forrester Research identified 2 seconds as the threshold for user satisfaction with page loads In 2010, Forrester Research found that over half of site visitors abandon page loads slower than 3 secondsIn 2010, Google announces that it has begun using site speed in web search rankingsIn 2012, the New York Times reported on findings that as small a difference as 250 ms between your site and that of a close competitor can reduce your web site visits3 second or faster page load times are expected for retail sites on mobile browsersIn 2011, Strangeloop found that an extra 500 ms delay for mobile users resulted in fewer return visitorsThe impact was still observed 4 months after the extra delay was removedIn 2011, DoubleClick removed a client-side redirection from their service, saving over 1 second on average on mobile devices, and click-through rates increased 12%In 2012, a Strangeloop study concluded that smartphone users view fewer pages, spend less time on sites, and have higher bounce rates than visitors on desktop browsersYou have less time to captivate and retain mobile site visitors In 2013, Etsy found an increase in the image content on one of their mobile site pages increased the bounce rate by 12%
  • It’s very important to measure not just single pages on a web site, but entire user journeysThe closer your benchmark data matches real user behavior, the more accurate the results
  • Online retailers had their best showing on desktop this yearThe industry has long understood the correlation between better performance and bottom line resultsAverage time to conduct the purchase path transaction was 22.8 secondsThis varied less than 0.5 seconds from week to weekThe slowest week was the third week of December, with the sites taking an average of 23.16 secondsThe fastest week was the first week of December, taking on 22.65 secondsEffectively, visitors saw no meaningful change in performance during the holiday season
  • Long search application calls and a large amount of products showing up in search results leads to a large amount of image files being returnedHeavy page weights led to longer page load times
  • Study average was 22.8 secondsThe slowest site in the study had a large amount of images and third party tags being used, loading in nearly 7 MB of data during the shopping user journey, and some pages exceeded 200 elementsBy comparison, Apple’s site only resulted in 2 MB of transaction data, and never once saw a page with more than 100 elements on it
  • The most available site on Desktop was HP, with an overall transaction uptime of 99.6%Most of the sites in the study had no major availability issues on desktopOnly one site had a sustained downtime periodAn outage from 03:00 to 05:00 on the morning of the 27th, when most of the site was working, but the search function failed to return resultsThis downtime had limited user impact given the time of day and that it was two days after Christmas
  • Nearly all page designs in the Keynote holiday retail study were optimized for the smartphone screen size and for access over 3G mobile network connectionsOne notable exception was AppleOverall, the user experience on smartphones was much slower than on desktop- The average time to complete a transaction was a slow 32.1 seconds- Throughout the holiday study, the average ranged from 28.6 to 35.7 seconds, certainly more variation than seen on the desktop- The slowest site had a painfully slow average of 54.75 seconds
  • The slowest page on the smartphone transactions, by far, was the Home pageThe Home page was the slowest page on nearly 77% of the measured sitesThat may sound surprising, but in most cases the Home page takes the biggest performance hit with DNS lookup and TCP connection delays compared to other pagesRetailers really need to focus on optimizing the Home page experience for mobile site visitorsSearch Results pages were the next slowest on averageOn the slower sites, very heavy search results pages occasionally took over 20 seconds when measured over the mobile network connections
  • The fastestmobile site measured was the Sony Style store, coming in with an average time of 19.7 secondsStudy average was 32.1 secondsThe Sony Style mobile site was optimized for a faster experience, with a very lean element count on all pages, resulting in speedier load timesThe slowest of the smartphone sites actually was not AppleApple was the only site in the study that did not have a site optimized for the smartphone displayDespite being optimized for the mobile screen size, the slowest site in the study came in at an average of 54.8 secondsThe slowdown was primary being driven by the search page occasionally taking over 20 seconds to return results over the cellular connections
  • Surprisingly, all of the sites measured were serving up the desktop site on tabletsNot one offered an optimized tablet version, which like the smartphone sites would be mindful of the amount of images or third-party content being served up and pay attention to the potential cellular connection and its limitations
  • Measurements were made over 3G network connections, which are noticeably slower than a fast Wi-Fi connectionDesktop pages are not optimized for a 3G connectionSome contained hundreds of elements and reached over a megabyte per page
  • The transaction time was 61.4 secondsAlmost three times as slow as the desktop user experienceThis is an unacceptable amount of time for users and will lead to abandonment
  • Retailers are doing a fairly good job at maximizing the desktop user experienceWith over a decade of experience, the industry has learned to focus on performance as a critical business metric In the 2013 holiday shopping season, Keynote measured the typical shopping purchase path on smartphones took 40% longer on average for the smartphone user on 3G mobile network connections than the desktop site visitor* http://www.keynote.com/docs/reports/Keynote-2012-Mobile-User-Survey.pdf
  • When a site loads has to make fewer requests over the network, there are simply fewer opportunities for errors to occur, especially during high traffic events like Black FridayOne site this holiday season achieved good results by limiting the number of third-party assets, such as marketing analytics tags, on its mobile home pageThe site smartly loads these assets only after all critical page content has already been requestedThat way, even if something does go wrong with a third-party tag, the user can still interact with the page and progress with the shopping experienceHigh latency on mobile network connections is a major concern for sites with too many new HTTP requests on each pageAim for 20 or fewer HTTP requests and 200K or fewer bytes on mobile
  • A major sin in page design is to have something which blocks other threadsLoading content in parallel, rather than serial, will improve the user experience
  • Fetching small images over TCP is very inefficientInstead, use CSS sprites or Data URI encoding techniques to speed up the download process
  • Load test your site early and be sure to see how mobile users are impacted by increased load as wellPlan your strategy well in advanceBenchmark the performance of your site and set goals for transactional speed and reliabilityYou shouldn’t be making any major changes to your site when the shopping season kicks into full gear  
  • One site Keynote observed during the holidays seemed to put on some weight, adding four additional elements that slowed down the user experienceA bit of investigation showed that the extra content was nothing more than promotional imagesWhat’s more, the promotional banners and marketing related images accounted for 200K out of the 311K of the total page sizeIn other words, marketing material exceeded the actual content on the page Your goal in mobile is to keep it light“Count every element and make every element count”
  • Many of the most popular sites followed are using Content Delivery Networks (CDNs) to improve their website experienceMoreover, there are very capable platform providers that do a great job in helping retailers create and manage their mobile web presenceIf you have the budget and want to do it right, sometimes it’s better to engage experts who can free you to do what you do best—while they do the same
  • During the holidays, we observed one mobile retailer whose site was down for an extended period of timePlanned or unplanned, this is something that can happenUnfortunately the site displayed a “page not found” error for several hours, potentially a source of great frustration on the part of a userIn this case, however, the outage seemed to coincide with planned site maintenance and a simple Maintenance Page would have been a much friendlier alternative for turning away a prospective shopperBetter yet – try and handle all maintenance well ahead of the peak seasons
  • The 2013 Holiday Shopping Season was a record year for e-retailers, but it is now done and goneNone of the desktopsites in our study had major performance events or availability failures prior to December 25There’s no room to slip, but your competition will not falter The stakes are expected to be even bigger in 2014Mobile site performance continues to lag despite intense interest on making mobile a top prioritySmartphone users saw 40% slower user experiences than desktop usersTablet users on mobile network connections receive horrendously slow user experiencesWill this change in 2014? Will 2014 finally be the Year of Mobile in online retail?
  • A Holiday Retail in Review: Performance Lessons for 2014

    1. 1. A Holiday Review Performance Lessons for 2014 Ken Harker, Senior Consultant January 2014
    2. 2. Some of the numbers are still coming in, but… • US online and e-commerce holiday retail increased 9.3% • UK online holiday retail sales increased 19.2% from 2012 • Overall online retail sales in the US for 2014 may reach $300 billion (Forrester) • Online experience will influence an additional $1.3 trillion in retail purchases (Forrester) Another Record Year for Online Retail ©Keynote Systems, Inc. 2
    3. 3. The Mobile Madness Grows ©Keynote Systems, Inc. 3 Retailers surveyed for Forrester Research’s “State of Retailing Online 2014” study ranked mobile as their TOP PRIORITY And yet, most retailers are NOT investing in their mobile sites as much as their desktop sites
    4. 4. Real Madness Is Not Being on Mobile At All ©Keynote Systems, Inc. 4 Only 72 of the top 100 e-retailers have fully functional m-commerce sites optimized for smartphones. (Internet Retailer) 2013 Internet Retailer Mobile Commerce Top 500 Guide
    5. 5. The Price is Right ©Keynote Systems, Inc. 5 On the web, there is always a lower price. High performing mobile web sites are the ones that consumers will turn to for their comparison shopping.
    6. 6. Expectations
    7. 7. On the desktop: 2 seconds or faster What Do Retail Site Customers Expect? ©Keynote Systems, Inc. 7 On mobile: 3 seconds or faster
    8. 8. • Delays in page load time result in increased abandonment, decreased loyalty, and loss of revenue • Even very small delays will drive your potential customers to the competition or keep them from returning to your site • Success is measured in milliseconds, not seconds The Challenge to Retailers ©Keynote Systems, Inc. 8
    9. 9. Keynote Online Holiday Shopping Study
    10. 10. During the 2013 holiday season, Keynote measured the performance and availability of 13 top online retailers • Measurements were taken on desktop browsers, smartphone browsers, and tablet browsers • Smartphone and tablet measurements were made on 3G mobile network connections in New York and San Francisco • Desktop measurements were taken on high-speed, low-latency network connections using real browsers One Web, Three Screens ©Keynote Systems, Inc. 10
    11. 11. The Sites in the Study Amazon Apple Store Best Buy CDW Dell HP Newegg Office Depot Office Max Overstock Sears Sony Styles Walmart ©Keynote Systems, Inc. 11
    12. 12. The Keynote Holiday Retail Study measured a typical user journey through the desktop and mobile sites 1. Start at the Home Page 2. Search for a product 3. Filter the search results 4. View product details 5. Add the product to a cart What Was Measured ©Keynote Systems, Inc. 12
    13. 13. Desktop Results
    14. 14. Online retailers had their best showing on desktop this year Average time to conduct the purchase path transaction was 22.8 seconds Desktop ©Keynote Systems, Inc. 14
    15. 15. Across all measured sites, the slowest step in the online purchase path was retrieving the results for a product search Search Results was the slowest step on 46% of the sites in the study Desktop: Search ©Keynote Systems, Inc. 15 The Slowest Step was Search Results
    16. 16. • The fastest site on Desktop was Apple, averaging only 14.0 seconds to complete the entire transaction Desktop: Speed ©Keynote Systems, Inc. 16
    17. 17. The most available site on Desktop was HP, with an overall transaction uptime of 99.6% Only one site had a sustained downtime period Desktop: Availability ©Keynote Systems, Inc. 17
    18. 18. Smartphone Results
    19. 19. Smartphone ©Keynote Systems, Inc. 19 • Nearly all pages designed for the smartphone screen size and for access over mobile connections Overall, the user experience on smartphones was much slower than on desktop
    20. 20. Smartphone: Slow Pages ©Keynote Systems, Inc. 20 The slowest page on the smartphone transactions, by far, was the Home page Search Results pages were the next slowest occasionally taking over 20 seconds
    21. 21. Smartphone: Fastest Sites ©Keynote Systems, Inc. 21 The fastest mobile site measured was the Sony Style store, coming in with an average time of 19.7 seconds The slowest of the smartphone sites actually was not Apple
    22. 22. Tablet Results
    23. 23. Tablet ©Keynote Systems, Inc. 23 None of the sites in the Keynote holiday study have pages optimized for tablet users over mobile network connections
    24. 24. Retailers are gambling that tablet users are always on fast WiFi connections Tablet ©Keynote Systems, Inc. 24
    25. 25. Consumers using tablets on 3G mobile network connections during the holiday shopping season saw very poor performance The average transaction time was 61.4 seconds Three times as slow as the desktop or smartphone user experience Tablet ©Keynote Systems, Inc. 25
    26. 26. Lessons for success in 2014
    27. 27. • Slow page load times continue to be the biggest point of frustration for mobile web users • Mobile needs to be a continuing focus for e-retailer to succeed in the marketplace Eliminate frustration with Mobile ©Keynote Systems, Inc. 27 What can I do to improve my mobile user experience?
    28. 28. 1. Dress Light ©Keynote Systems, Inc. 28 Be disciplined about loading as few assets on the page as possible
    29. 29. Different browsers/phones have different numbers of parallel threads 2. Avoid Long Lines ©Keynote Systems, Inc. 29
    30. 30. 3. Bundle up ©Keynote Systems, Inc. 30 Employ optimization techniques that reduce the number of HTTP requests needed to fetch image content
    31. 31. 4. Anticipate the Rush ©Keynote Systems, Inc. 31 You know it’s coming, so why wait until next year to prepare for the holidays?
    32. 32. Good marketing can help to improve conversion rates, but if too much leads to downtime, slow speeds and site abandonment you’ve gone too far. 5. Resist the Marketing ©Keynote Systems, Inc. 32
    33. 33. No one says that you need to go at it alone, and in many cases you shouldn’t. 6. Partner Up ©Keynote Systems, Inc. 33
    34. 34. ‘Stuff’ Happens 6. Expect the Unexpected ©Keynote Systems, Inc. 34
    35. 35. Conclusions
    36. 36. 2014 is a New Year! ©Keynote Systems, Inc. 36
    37. 37. Thank You Ken Harker Senior Consultant, Keynote ken.harker@keynote.com

    ×