MeasureWorks - Online Optimizers & Analytics meetup Stockholm - Need for Speed

  • 485 views
Uploaded on

Slides from session at the Online Optimizers & Analytics meetup Stockholm... About performance, ux & analytics...

Slides from session at the Online Optimizers & Analytics meetup Stockholm... About performance, ux & analytics...

More in: Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
485
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. @jeroentjepkema Founder & CEO
  • 2. Slides: bit.ly/MW-OOnfs13
  • 3. Content vs. Performance
  • 4. How we perceive performance....
  • 5. How we perceive performance.... 15%-20% worse than in reality
  • 6. How we perceive performance.... 15%-20% worse than in reality Slow downs have more impact
  • 7. How we perceive performance.... 15%-20% worse than in reality Slow downs have more impact Task completion has positive impact
  • 8. How we perceive performance.... 15%-20% worse than in reality Slow downs have more impact Task completion has positive impact Always compared to past experiences
  • 9. A simple online business model: Marketing Conversion Optimization New visitors Search Tweets Mentions ADs seen Conversion rate Growth Number of visits Loss Bounce rate Revenue Pages per visit Time on site X Order value
  • 10. Dutch consumers visit 18-23 websites before they finally convert Source: TNS/Nipo
  • 11. Performance influence cycle Search/Orientation phase 1 Perception of performance http://twinkle100.measureworks.nl
  • 12. Performance influence cycle Stimulate content/ conversion 4 Search/Orientation phase 5 1 Perception of performance http://twinkle100.measureworks.nl Delivered performance 3 2 Your website
  • 13. Need for Speed @jeroentjepkema, MeasureWorks @jeroentjepkema, MeasureWorks
  • 14. 0 Source: Jakob Nielsen
  • 15. 1 0,3 0 Interaction: Let’s conversate... Instantaneous: I like it! Source: Jakob Nielsen
  • 16. 3 1 0,3 0 Mmm, shall I click away? Interaction: Let’s conversate... Instantaneous: I like it! Source: Jakob Nielsen
  • 17. 10 Only if the task/content is relevant 3 1 0,3 0 Mmm, shall I click away? Interaction: Let’s conversate... Instantaneous: I like it! Source: Jakob Nielsen
  • 18. 0:00 sec. Conversation starts
  • 19. 0:00 sec. Conversation starts 2.5 sec. First reply
  • 20. 4.5 sec. Finish conversation 0:00 sec. Conversation starts 2.5 sec. First reply
  • 21. 4.5 sec. Finish conversation 0:00 sec. Conversation starts Comfort zone Tolerated 2.5 sec. First reply Frustrated
  • 22. The holy grail in experience is “flow state”
  • 23. “” Flow is an optimal experience that is “intrinsically enjoyable”
  • 24. Provide relevant content to support task completion
  • 25. Deliver it fast, focus on perception
  • 26. http://www.nytimes.com/2012/08/19/opinion/sunday/why-waiting-in-line-is-torture.html?pagewanted=all&_r=0
  • 27. http://answers.yahoo.com/question/index?qid=1005081200005
  • 28. That’s not my website?
  • 29. Over 60% of people that experienced a slow website ranked design 2 points lower compared to a fast experience Slow Average Fast % of respondents that rated the actual website speed 100 75 50 25 0 1 2 3 Design score (1=bad - 5=beautiful) 4 5
  • 30. Experience the difference Tested with webpagetest.org (3G traffic shaping, 2000Kbps Down, 1000 Kbps Up, 150ms latency)
  • 31. Perception vs. Technology
  • 32. Desktop shopping experience setup.... Buy a summer holiday: 100 users ‣ From 5 selected websites ‣ Rank from fastest to slowest after task execution No measurements tools allowed ;-) Desktop only, either via LAN or WiFi
  • 33. Perceived Technology (User Ranking) (Full page download) 1 Sunweb 2 D-reizen 3 Neckermann 4 Arke 5 Globe Technology vs. Perceived Study performed by MeasureWorks for Emerce eTravel, 2013
  • 34. Perceived Technology (User Ranking) (Full page download) 1 Sunweb Sunweb 2 D-reizen Globe 3 Neckermann Neckermann 4 Arke D-reizen Globe Arke 5 Technology vs. Perceived Study performed by MeasureWorks for Emerce eTravel, 2013
  • 35. Perceived Technology (User Ranking) (Full page download) 1 Sunweb Sunweb 2 D-reizen Globe 3 Neckermann Neckermann 4 Arke D-reizen Globe Arke 5 Technology vs. Perceived Study performed by MeasureWorks for Emerce eTravel, 2013
  • 36. Globe Neckermann Sundio D-reizen Arke
  • 37. M-commerce?
  • 38. Mobile shopping experience setup.... 100 mobile users Perform 2 tasks: ‣Buy a book ‣Buy a T-shirt Mobile browsers only via 3G, no WiFi
  • 39. #1: Speed
  • 40. Functional issues reported with Zalando 50 37,5 Round 1 25 45 42,1 12,5 15,4 0 Zalando HM 20 7,7 V&D 24 3,9 Tom Tailor 4 Design 1. Buy a T-shirt Speed Mobile Readiness Other 2. Feedback 40 30 Round 2 20 10 12 0 54 34 Zalando Mobile Only research: - Task completion: Only use smartphone to buy a book - N = 100, users range from 20-65 15,4 HM 8 V&D Tom Tailor 21 Design 18 Speed 6 Mobile Readiness Other
  • 41. Nexus - Android - 3G Tested with webpagetest.org (3G traffic shaping, 2000Kbps Down, 1000 Kbps Up, 150ms latency)
  • 42. Nexus - Android - 3G Tested with webpagetest.org (3G traffic shaping, 2000Kbps Down, 1000 Kbps Up, 150ms latency)
  • 43. #2: Usability
  • 44. Bol.com Selexyz De Boekerij ECI Bruna Cosmox Mobile Readiness
  • 45. 100 75 50 46 44 26 24 25 16 0 Bol.com Selexyz Bruna Other 1. Buy a book Mobile Only research: - Task completion: Only use smartphone to buy a book - N = 100, users range from 20-65 Design Speed 16 Mobile Readiness 2. Feedback bol.com 16 Other Design 12 Speed Mobile Readiness Other 3. Feedback from different stores
  • 46. Mobile Readiness
  • 47. - One touch? +
  • 48. Login or not?
  • 49. Mobile Search
  • 50. Speed matters
  • 51. Complete Web Monitoring
  • 52. Web Analytics Usability Performance (what did they do on the site?) (how did they interact with it?) (could they do what they wanted to?) Complete Web Monitoring VoC Social Media Competition (what were their motivations?) (what were they saying?) (what are they up to?)
  • 53. “Hard” data Web Analytics Usability Performance (what did they do on the site?) (how did they interact with it?) (could they do what they wanted to?) Complete Web Monitoring VoC Social Media Competition (what were their motivations?) (what were they saying?) (what are they up to?) “Soft” data
  • 54. “Hard” data Web Analytics Usability Performance (what did they do on the site?) (how did they interact with it?) (could they do what they wanted to?) Complete Web Monitoring VoC Social Media Competition (what were their motivations?) (what were they saying?) (what are they up to?) “Soft” data
  • 55. Data driven
  • 56. #1: Performance SLA
  • 57. Establish a baseline:
  • 58. Establish a baseline: A pre-defined set of metrics
  • 59. Establish a baseline: A pre-defined set of metrics that describes normal behavior
  • 60. Establish a baseline: A pre-defined set of metrics that describes normal behavior in order to detect variancies
  • 61. Establish a baseline: A pre-defined set of metrics that describes normal behavior in order to detect variancies and to be comparable within historic context
  • 62. A practical (and easy DIY) example:
  • 63. Purchasing a book, must be completed (speed), where every page loads under 3 sec., Customer journey Metric: Speed Target: Sec using IE9 and higher, User scenario from any location in the Netherlands, User locations for 90% of all users, every day between 6am and 12pm, measured with Real User Monitoring. Percentile Window Collection type Read more: Metrics 101, Velocityconf 2010
  • 64. Performance SLA
  • 65. 100% 150% 125% 100% 75% 50% 50% 25% 25% 0% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Sessions Bounce rate 75%
  • 66. LD50: 5.2 sec. 100% 150% 100% 125% 100% 75% 50% 50% 25% 25% 0% 0% 1 2 3 4 5 6 7 8 9 10 11 12 Baseline: 2,7 sec. Poverty Line: 10,6 sec. 13 14 Sessions Bounce rate 75%
  • 67. #2: Collecting Data
  • 68. (CLOUD) DATA CENTER INTERNAL USERS CUSTOMERS INTERNET Third-party/ Cloud Services Storage Major ISP DB Servers Web Servers Network Mainframe Middleware Servers App Servers Load Balancers Local ISP Content Mobile Delivery Networks Carriers
  • 69. (CLOUD) DATA CENTER INTERNAL USERS CUSTOMERS INTERNET Third-party/ Cloud Services Storage Major ISP DB Servers Web Servers Network Mainframe Middleware Servers App Servers Load Balancers This is what you control... Local ISP Content Mobile Delivery Networks Carriers
  • 70. (CLOUD) DATA CENTER INTERNAL USERS CUSTOMERS INTERNET Third-party/ Cloud Services Storage Major ISP DB Servers Web Servers Network Mainframe Middleware Servers App Servers Load Balancers This is what you control... Local ISP Content Mobile Delivery Networks Carriers What you’re blamed for..
  • 71. Measuring performance?
  • 72. Measuring performance? Outside-in, from the browser perspective...
  • 73. Synthetic Monitoring
  • 74. Simulate business transactions
  • 75. Used for error detection & Root Cause Analysis
  • 76. Ultimately, synthetic monitoring shows you if your site’s working or not...
  • 77. Real User Monitoring
  • 78. But, synthetic isn’t enough...
  • 79. Synthetic heartbeat Real Users But, synthetic isn’t enough...
  • 80. Relies on navigation timing API, custom variables can be added Turn on “Site Speed” in your GA account!
  • 81. Or try Soasta “Light”
  • 82. Ultimately, Real User Monitoring shows you how many users are affected by bad performance...
  • 83. #3: Performance analytics
  • 84. Service levels Service Level Report KPI’s 082 084 Performance (avg) Page 1 www.homepage.nl Loadtime. Loadtime Availability (Document. (.fully.loaded.) complete) Competition You 5 97,0% 2,20 3,50 10% 85% Page 2 ge.nl/autoverzekeren 99,0% 2,50 3,30 20% 90% Page 3 overzekeren/afsluiten 95,0% 2,30 3,60 15% 70% 95,0% 2,43 3,57 20% 67% 1 Page 5 homepage.nl/zoeken 94,0% 2,48 3,62 23% 59% Branche 2 Page 4 homepage.nl/contact Competitors 0 Page 6 e.nl/productoverzicht 93,0% 2,53 3,67 25% 92,0% 2,58 3,72 28% 44% 3 33 34 35 ge.nl/ab8test/variant2 Page 8 91,0% 2,63 3,77 30% 37% ge.nl/facebookplugins Page 9 90,0% 2,68 3,82 33% 89,0% 2,73 3,87 35% 22% ww.homepage.nl/faq Page 11 88,0% 2,78 3,92 38% 14% 37 38 39 40 # 6 3,4 sec Is your ranking against competitors is the average performance of your competitors 40% 40% of your compeitors are faster than you 29% page.nl/muziekpagina Page 10 36 Week 52% Page 7 ge.nl/ab8test/variant1 4 Source: Availability: Gomez Loadtime: Soasta Competition: Webpagetest.org
  • 85. Online Satisfaction Report 100 Availability (%) 80 400 0 -400 -800 -1200 60 40 20 Page 1 Page 11 0 Mentions (#) 1200 1200 800 Page 2 Page 3 Page 4 Positive Messages Source: Availability: Gomez Sentiment: Twitter, Facebook Negative Messages Data collected from ING.nl Availability Messages from Twitter & Facebook, collected with Coosto
  • 86. Speed vs. Engagement per month (Non-Bounce) 90 78,75 56,25 Pageviews (#) Non-Bounce rate (%) 67,5 45 33,75 22,5 11,25 0 0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 Non-Bounce CategorienaamPageviews # 6-7 7-8 8-9 9-10 >10
  • 87. Revenue risked 100.000 visits (Performance/Time Period/Percentiles) Optimal Flow (P_25) Bouncerate (%) Potential Conversion 2.1 100 49% 51.000 Page 2 4.0 100 51% 24.990 Page 3 2.0 100 60% 9996 Page 4 3.5 85 25% 6372 Actual Flow (P_85) Speed (sec.) Availability (%) Bouncerate (%) Actual Conversion Page 1 4.1 100 59% 41.000 Page 2 6.3 100 54% 18.860 Page 3 Speed: Soasta Availability: Webpagetest.org Availability (%) Page 1 Source: Speed (sec.) 2.0 100 60% 7544 Page 4 4.7 85 53% 3013 0-2,7 sec. 2,7-3,6 sec. >3,6 sec.
  • 88. If anything else...
  • 89. Deliver fast(er)
  • 90. Measure Real Usage
  • 91. Find out where to improve?
  • 92. Focus on a fast experience …without it, NO time for content
  • 93. Go check your site @ Webpagetest.org
  • 94. Slides: bit.ly/MW-OOnfs13
  • 95. Thanks! More questions? M: jtjepkema@measureworks.nl T: @jeroentjepkema W: www.measureworks.nl
  • 96. Outrun your competitors
  • 97. Optimize all your touch points
  • 98. Regular website Mobile Web Deliver the same experience to all your (mobile) users Native Application
  • 99. Mobile first www.lukew.com
  • 100. Deliver content accordingly Prioritize your business goals Mobile first www.lukew.com
  • 101. Design for (mobile) use cases
  • 102. Focus on task completion
  • 103. With mobile conditions in mind
  • 104. Optimized for the right screen
  • 105. Native app Mobile Web - Portrait Mobile Web - Landscape
  • 106. (Touch) Screen
  • 107. 49% One device, One hand http://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobiledevices.php?goback=.gmp_72842.gde_72842_member_215909354
  • 108. Golden touch http://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobiledevices.php?goback=.gmp_72842.gde_72842_member_215909354
  • 109. Make it (feel) fast!
  • 110. Abandonment"Rate"Across"200+"Sites"/"177+"Million"Page"Views" Over"2"weeks"/"All#Browsers# 30" Abandonment"Rate"(%)" 25" 20" Abandonment*Rate*,* All*Browsers* 15" 10" 5" 0" 0" 1" 2" 3" Source: Gomez real user monitoring 4" 5" 6" 7" 8" 9" Page"Load"Time"Band"(sec.)" 10" 11" 12" 13" 14" 15" It only takes a second...
  • 111. Abandonment"Rate"Across"200+"Sites"/"177+"Million"Page"Views" Abandonment"Rate"Across"200+"Sites"/"177+"Million"Page"Views" Over"2"weeks"/"All#Browsers#vs.#iPhone#Safari# Over"2"weeks"/"All#Browsers# 25" Abandonment"Rate"(%)" 30" 25" Abandonment"Rate"(%)" 30" 20" 20" 15" 15" 10" 10" 5" 5" 0" Abandonment*Rate*,* All*Browsers* Abandonment*Rate*,* All*Browsers* Abandonment*Rate*,* iPhone*Safari* 0" 0" 1" 1" 2" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 0" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" Page"Load"Time"Band"(sec.)" Source: Gomez real user monitoring Page"Load"Time"Band"(sec.)" Source: Gomez real user monitoring ..To click away