Performability, Could they do it?


    Complete	
  Web	
  Monitoring  	
  
       @jeroentjepkema        	
  
          #...
The Internet is getting complex….
                                "
The End-User: 

 Everywhere!
           !
Technology: 

Enterprise/Virtualization/Cloud!
Weʼre getting mobile"
The end-user is in control
                         "
Back to the fundamentals: 

   Quality of Service"
Performance matters!
                   "
Google loses 1% of traffic for every
    100 milliseconds of delay "
Impact of additional delay on business metrics !
Shopzilla had another angle"


Big, High-traffic site"               16 Month re-engineering"
  100M impressions a day!   ...
12% increase in revenue,"
25% increase in page views for"
   every 5 seconds of gain"
As of April 9th 2010:!
Site speed co-determines "
  Google page ranking!"
Google Pagerank"
Customer Experience"
Evolving Customer Expectations"


     8 seconds



     4 seconds



     2 or less



     ?




     Response Time	
  
...
Availability and Stability are!
      KEY ISSUES!!
Performance impacts Customer Loyalty!
Average Impact of 1 second delay in 

   response time for web users"




                           Source: Aberdeen Grou...
Impact of Poor Experience"




                             Source: Gartner, 2008!
A/B Performance Testing!
Objective: Influence visitor engagement"

                               -




                                   Visitor e...
Objective: Influence visitor engagement"

                               -




                                   Visitor e...
Objective: Influence visitor engagement"

                               -




                                   Visitor e...
Objective: Influence visitor engagement"

                               -
            10	
  sec	
  




                  ...
A/B Performance Test Setup"


    Visitor                Strangeloop            Website

                            Decid...
Technical Problem Outlined - Before"




                           32
Technical Problem Outlined - Before"
               HTML (BackEnd)




 63 Round Trips (Front End)




          1.975sec ...
Strangeloop Solution - After
                                            "




Before"                         After"
   ...
So hereʼs what weʼve discovered…."
Traffic Levels"


                            18!
                          9000
                                    8.505!...
Bounce rate"


               14,60%!
               14,40%!                        14,35%!
Bounce rate"




             ...
Average Time on Site"


                  00:36:00!
                                00:30:10!
Time (seconds)"




        ...
Pages per Visit
                                  "


         18!
                 15,64!
         16!
         14!
     ...
Conversion rate and order value"


                                  20,00%!
                                             ...
Customers want more and they want it faster
                                          "
42   Entire contents © 2009 Forrester Research, Inc. All rights reserved.
Time to speed up!"
Compared to the competition!
ABN Amro    1.81	
  sec	
       Bijenkorf   0.76	
  sec	
       D-reizen    1.79	
  sec	
  



  Alex      3.43	
  sec	
  ...
ABN Amro    1.81	
  sec	
       Bijenkorf   0.76	
  sec	
       D-reizen    1.79	
  sec	
  



  Alex      3.43	
  sec	
  ...
Retail                 2.31	
  sec	
  
   Average speed per sector,
                                                      ...
Retail                 2.31	
  sec	
  
   Average speed per sector,
                                                      ...
Performance by region!
The userʼs distance from the content matters"
 	
  	
  6.54	
  sec	
  
             	
  	
  	
  4.89	
  sec	
  


                                                      ...
Forrester	
  research	
  shows	
  that	
  over	
  	
  
40%	
  of	
  consumers	
  will	
  wait	
  no	
  more	
  than	
  	
 ...
Mobile site performance!
58%	
  of	
  mobile	
  phone	
  users	
  expect	
  websites	
  to	
  load	
  as	
  
                        quickly,	
  al...
desktop
          1.17	
  sec	
  




Iphone
desktop
           1.17	
  sec	
  




          20.19	
  sec	
  
Iphone
desktop
                1.17	
  sec	
  




               20.19	
  sec	
  
Iphone




          http://m.bol.com



     ...
desktop
          1.94	
  sec	
  




Iphone
desktop
           1.94	
  sec	
  




          15.60	
  sec	
  
Iphone
desktop
              1.17	
  sec	
  




             20.19	
  sec	
  




          http://m.nu.nl



                  ...
Performance differs per browser
                              "
Each browser acts different"

Firefox	
                        Internet	
  Explorer	
  
Raw page load time"
Raw page load time vs. Perceived Performance"
0.95	
  sec	
     1.82	
  sec	
  




 1.00	
  sec	
     1.19	
  sec	
  




2.637	
  sec	
     2.59	
  sec	
  
Availability & performance issues can also be browser specific	
  
Assembly within the browser: it all has to work
                                              !
?	
  
?	
  
Real User Experience
                   !
Congestion!
Watch out for Peak Traffic!"
Watch out for Peak Traffic!"
Conclusion: Bottlenecks are everywhere!
Bottlenecks are everywhere"

    How	
  fast	
  is	
  fast	
  enough?	
  

     Performance	
  by	
  region	
  

       Mo...
How to make your website faster?"
Introducing:

Web Performance Management!
How does the performance of 

          Infrastructure and Application….


                                          Manag...
…relate to the customer experience



                                      Management Tool!




                         ...
Introducing the synthetic user



                                             Management Tool!



 Active
Monitoring!
   ...
Active Monitoring


 Continous
Measurements
Active Monitoring


 Continous     Check Performance
Measurements     & Availability
Active Monitoring


 Continous     Check Performance   In case of failure:
Measurements     & Availability         Alerting
Active Monitoring


 Continous     Check Performance   In case of failure:   Root Cause Analysis /
Measurements     & Avai...
Simulate Business Transactions"
Response time and Availability"
Page breakdown"
From where your users are
                        "
But synthetic isnʼt enough"
But synthetic isnʼt enough"



                        Gomez Asia




                        Gomez US



                ...
Measure all your users



                           Client!
   Client!                                                   ...
Collected data:                                                                                                 Active
   ...
Business Impact per user


Satisfaction Reports        Business Transaction Funnel




                                   ...
Session Replay

  User SessionTracking"




                                    Session Replay"


Business Service Monitor...
Root Cause Analysis per user
                           "


         KPI configuration, 

         monitoring, & alerting!
...
Active Monitoring:!
Active Monitoring"     Simulate users by scripting pages and/or
                        business tran...
But, how to become the performance leader?!
5 Web Performance Best Practices
                               "
1: Create Shared Performance Objectives"
Business	
  vs.	
  IT	
  
Business	
  
 	
  Invest	
  to	
  aeract	
  more	
  
and	
  new	
  customers	
  
 	
  Quick	
  Time	
  to	
  market	
  
...
…..How to work
   together?"
Key Business Metrics"

AJenKon	
                 Engagement	
                  Conversion	
  


                NEW
      ...
Key Business Metrics"

AJenKon	
                 Engagement	
                                  Conversion	
  


          ...
Business oriented Key Performance objectives
                                           !
Business oriented Key Performance objectives
                                           !



System	
  Management	
     Bu...
Business oriented Key Performance objectives
                                           !



System	
  Management	
       ...
Business oriented Key Performance objectives
                                           !



System	
  Management	
       ...
Business oriented Key Performance objectives
                                           !



System	
  Management	
       ...
Business oriented Key Performance objectives
                                           !



System	
  Management	
       ...
2: Measure from End User Perspective
                                   !
Monitor Key Transactions from where your End-Users are!
Benchmark against the competition!
Measure every detail of your traffic"

Customers: Global!                                   Access: Getting Mobile!




   ...
3. Plan for Peak Crowds"
1.	
  	
  
  Compare	
  
Actual	
  Results	
  
1.	
  	
  
  Compare	
  
Actual	
  Results	
  




     2.	
  
With	
  Actual	
  
Performance	
  
1.	
  	
  
  Compare	
  
Actual	
  Results	
  




     2.	
  
With	
  Actual	
  
Performance	
  




         3.	
  
 To	...
Peak Traffic: Itʼs a Business KPI!
                                                                             "

        ...
4. Test and Simulate performance"
The performance challenge"
Make sure the view your customers are getting
       is the view you want them to see!
Make sure the view your customers are getting
       is the view you want them to see!
Measure performance for normal traffic….
                                      "
…. AND Peak Traffic"
Find your ceilings
                 "
Source: John Allspaw, Flickr.com




                                               what you have left




               ...
Plan for the unexpected

  Measure performance under normal and peak
conditions to find all the problems and optimization
 ...
5. Tying performance to conversion:"
     Web Performance Analytics!
Is it marketing?"
Or the WEB Experience?"

           Why	
  do	
  customers	
  exit?	
  
               Offer?	
  
               Perform...
Online Success requires a complete picture"




        IT	
  OperaKons:	
           ?           Web	
  AnalyKcs:	
  
 	
...
Online Success requires a complete picture"




        IT	
  OperaKons:	
                                                ...
The Web Performance Challenge:

Balancing Cost vs. Performance 





                    Revenue"



                  Con...
The Web Performance Challenge:

Balancing Cost vs. Performance 





                    Revenue"



                  Con...
The Web Performance Challenge:

                    Balancing Cost vs. Performance 





                                 ...
The Web Performance Challenge:

                    Balancing Cost vs. Performance 


                                    ...
Creating the Big Picture:

Break down the Web Application Delivery Chain…!




                                        Man...
Creating the Big Picture:

              ….and relate Customer Experience…..
                                             ...
Creating the Big Picture:

                           …to revenue risked!

  Type of visitors      Browser type        App...
Make your own business case"




  Current

daily orders"    (!
                *!
                      Increased"
      ...
Make your own business case"




          (!                 )!
$10,000! *! 0.1607! +! 0.0551!        $2,158!      23.17
...
Actionable reporting:"
Monetize your performance "




                           Availability
                           ...
Summary"
1. Create Shared Performance Objectives




2. Measure from End User perspective



3. Plan for Peak Crowds




4. Test an...
Questions?!
Analytics
          Usability
          Performability
(What did they do    (How did they         (Could they do
  on the ...
Thanks! More questions?

M: jtjepkema@measureworks.nl
T: @jeroentjepkema
W: www.measureworks.nl
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
02 web performance
Upcoming SlideShare
Loading in …5
×

02 web performance

2,198 views

Published on

Published in: Technology, Business
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,198
On SlideShare
0
From Embeds
0
Number of Embeds
485
Actions
Shares
0
Downloads
0
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

02 web performance

  1. 1. Performability, Could they do it?
 Complete  Web  Monitoring   @jeroentjepkema   #WAC2010  
  2. 2. The Internet is getting complex…. "
  3. 3. The End-User: 
 Everywhere! !
  4. 4. Technology: 
 Enterprise/Virtualization/Cloud!
  5. 5. Weʼre getting mobile"
  6. 6. The end-user is in control "
  7. 7. Back to the fundamentals: 
 Quality of Service"
  8. 8. Performance matters! "
  9. 9. Google loses 1% of traffic for every 100 milliseconds of delay "
  10. 10. Impact of additional delay on business metrics !
  11. 11. Shopzilla had another angle" Big, High-traffic site" 16 Month re-engineering"   100M impressions a day!   Page load from 6 seconds to 1.2!   8000 searches a second!   Uptime from 99.65% to 99.97%!   20-29M unique visitors a month!   10% of previous hardware   100M products! needs!
  12. 12. 12% increase in revenue," 25% increase in page views for" every 5 seconds of gain"
  13. 13. As of April 9th 2010:! Site speed co-determines " Google page ranking!"
  14. 14. Google Pagerank"
  15. 15. Customer Experience"
  16. 16. Evolving Customer Expectations" 8 seconds 4 seconds 2 or less ? Response Time   Source: Forrester Consulting, 2009
  17. 17. Availability and Stability are! KEY ISSUES!!
  18. 18. Performance impacts Customer Loyalty!
  19. 19. Average Impact of 1 second delay in 
 response time for web users" Source: Aberdeen Group, 2009 24
  20. 20. Impact of Poor Experience" Source: Gartner, 2008!
  21. 21. A/B Performance Testing!
  22. 22. Objective: Influence visitor engagement" - Visitor engagement 10  ms   +
  23. 23. Objective: Influence visitor engagement" - Visitor engagement 100  ms   10  ms   +
  24. 24. Objective: Influence visitor engagement" - Visitor engagement 1  sec   100  ms   10  ms   +
  25. 25. Objective: Influence visitor engagement" - 10  sec   Visitor engagement 1  sec   100  ms   10  ms   +
  26. 26. A/B Performance Test Setup" Visitor Strangeloop Website Decide whether to optimize Normal Receive Accelerated content page Insert Process segment scripts Optimize? marker Send analytics Unaccelerated Google Analytics
  27. 27. Technical Problem Outlined - Before" 32
  28. 28. Technical Problem Outlined - Before" HTML (BackEnd) 63 Round Trips (Front End) 1.975sec 9.537sec 769KB 33
  29. 29. Strangeloop Solution - After " Before" After"   Round trips: 63!   Round trips: 8!   Start Render: 1.975 Sec!   Start Render: 1.198 Sec!   Load Time: 9.537 Sec"   Load Time: 3.69 Sec"   Payload : 769 KB!   Payload: 456 KB! 34"
  30. 30. So hereʼs what weʼve discovered…."
  31. 31. Traffic Levels" 18! 9000 8.505! 16! 14! 6750 12! Total number of visits" 4.740! 10! 4500 8! 6! 2250 4! 2! 00! Optimized! Unoptimized!
  32. 32. Bounce rate" 14,60%! 14,40%! 14,35%! Bounce rate" 14,20%! 14,00%! 13,80%! 13,60%! 13,38%! 13,40%! 13,20%! 13,00%! 12,80%! Optimized! Unoptimized! 37
  33. 33. Average Time on Site" 00:36:00! 00:30:10! Time (seconds)" 00:28:48! 00:23:50! 00:21:36! 00:14:24! 00:07:12! 00:00:00! Optimized! Unoptimized! 38
  34. 34. Pages per Visit " 18! 15,64! 16! 14! 12! 11,04! Pages" 10! 8! 6! 4! 2! 0! Optimized! Unoptimized! 39
  35. 35. Conversion rate and order value" 20,00%! 16,07%! Change in shopping cart metric" 15,00%! 10,00%! 5,51%! 5,00%! 0,00%! Conversion rate! Order value! -5,00%! -10,00%! -15,00%! -20,00%! 40
  36. 36. Customers want more and they want it faster "
  37. 37. 42 Entire contents © 2009 Forrester Research, Inc. All rights reserved.
  38. 38. Time to speed up!"
  39. 39. Compared to the competition!
  40. 40. ABN Amro 1.81  sec   Bijenkorf 0.76  sec   D-reizen 1.79  sec   Alex 3.43  sec   Bol.com 1.20  sec   Neckermann 0.61  sec   ING 0.86  sec   Hema 1.19  sec   NS 0.86  sec   Rabobank 1.42  sec   Mediamarkt 2.39  sec   Sunweb 0.39  sec   SNS Reaal 1.64  sec   TomTom 1.46  sec   Vliegwinkel 5.04  sec  
  41. 41. ABN Amro 1.81  sec   Bijenkorf 0.76  sec   D-reizen 1.79  sec   Alex 3.43  sec   Bol.com 1.20  sec   Neckermann 0.61  sec   ING 0.86  sec   Hema 1.19  sec   NS 0.86  sec   Rabobank 1.42  sec   Mediamarkt 2.39  sec   Sunweb 0.39  sec   SNS Reaal 1.64  sec   TomTom 1.46  sec   Vliegwinkel 5.04  sec  
  42. 42. Retail 2.31  sec   Average speed per sector, Finance 3.30  sec   Gomez Benchmark 2009 Travel 3.55  sec   ABN Amro 1.81  sec   Bijenkorf 0.76  sec   D-reizen 1.79  sec   Alex 3.43  sec   Bol.com 1.20  sec   Neckermann 0.61  sec   ING 0.86  sec   Hema 1.19  sec   NS 0.86  sec   Rabobank 1.42  sec   Mediamarkt 2.39  sec   Sunweb 0.39  sec   SNS Reaal 1.64  sec   TomTom 1.46  sec   Vliegwinkel 5.04  sec  
  43. 43. Retail 2.31  sec   Average speed per sector, Finance 3.30  sec   Gomez Benchmark 2009 Travel 3.55  sec   ABN Amro 1.81  sec   Bijenkorf 0.76  sec   D-reizen 1.79  sec   Alex 3.43  sec   Bol.com 1.20  sec   Neckermann 0.61  sec   ING 0.86  sec   Hema 1.19  sec   NS 0.86  sec   Rabobank 1.42  sec   Mediamarkt 2.39  sec   Sunweb 0.39  sec   SNS Reaal 1.64  sec   TomTom 1.46  sec   Vliegwinkel 5.04  sec  
  44. 44. Performance by region!
  45. 45. The userʼs distance from the content matters"
  46. 46.      6.54  sec        4.89  sec        4.07  sec        5.49  sec        6.09  sec        7.19  sec  
  47. 47. Forrester  research  shows  that  over     40%  of  consumers  will  wait  no  more  than     3  seconds  for  a  Web  page  to  load  before   clicking  away  to  a  compeHtor        6.54  sec        4.89  sec        4.07  sec        5.49  sec        6.09  sec        7.19  sec  
  48. 48. Mobile site performance!
  49. 49. 58%  of  mobile  phone  users  expect  websites  to  load  as   quickly,  almost  as  quickly  or  faster  on  their  mobile   phone,  compared  to  the  computer  they  use  at  home   Among adults who accessed the internet with a mobile phone in the past 12 months, Gomez Mobile Web Experience Survey
  50. 50. desktop 1.17  sec   Iphone
  51. 51. desktop 1.17  sec   20.19  sec   Iphone
  52. 52. desktop 1.17  sec   20.19  sec   Iphone http://m.bol.com 5.02  sec  
  53. 53. desktop 1.94  sec   Iphone
  54. 54. desktop 1.94  sec   15.60  sec   Iphone
  55. 55. desktop 1.17  sec   20.19  sec   http://m.nu.nl 4.64  sec  
  56. 56. Performance differs per browser "
  57. 57. Each browser acts different" Firefox   Internet  Explorer  
  58. 58. Raw page load time"
  59. 59. Raw page load time vs. Perceived Performance"
  60. 60. 0.95  sec   1.82  sec   1.00  sec   1.19  sec   2.637  sec   2.59  sec  
  61. 61. Availability & performance issues can also be browser specific  
  62. 62. Assembly within the browser: it all has to work !
  63. 63. ?  
  64. 64. ?  
  65. 65. Real User Experience !
  66. 66. Congestion!
  67. 67. Watch out for Peak Traffic!"
  68. 68. Watch out for Peak Traffic!"
  69. 69. Conclusion: Bottlenecks are everywhere!
  70. 70. Bottlenecks are everywhere" How  fast  is  fast  enough?   Performance  by  region   Mobile  Performance   Browser  Experience   3d  Party  Performance   Real  User  Experience   CongesHon  
  71. 71. How to make your website faster?"
  72. 72. Introducing:
 Web Performance Management!
  73. 73. How does the performance of 
 Infrastructure and Application….
 Management Tool! Load
 Web App DB! balancer! server! server! Internet! Client!
  74. 74. …relate to the customer experience
 Management Tool! Load
 Web App DB! balancer! server! server! Internet! Client!
  75. 75. Introducing the synthetic user
 Management Tool! Active Monitoring! Load
 Web App DB! balancer! server! server! DB! Internet! Client!
  76. 76. Active Monitoring
 Continous Measurements
  77. 77. Active Monitoring
 Continous Check Performance Measurements & Availability
  78. 78. Active Monitoring
 Continous Check Performance In case of failure: Measurements & Availability Alerting
  79. 79. Active Monitoring
 Continous Check Performance In case of failure: Root Cause Analysis / Measurements & Availability Alerting SLA reporting
  80. 80. Simulate Business Transactions"
  81. 81. Response time and Availability"
  82. 82. Page breakdown"
  83. 83. From where your users are "
  84. 84. But synthetic isnʼt enough"
  85. 85. But synthetic isnʼt enough" Gomez Asia Gomez US Gomez UK
  86. 86. Measure all your users
 Client! Client! Management Tool! Active Monitoring! Load
 Web App DB! balancer! server! server! DB! Internet! Real User! Monitoring! Client! Client! Client!
  87. 87. Collected data: Active Client" Monitoring!  Client-­‐ip,  Server-­‐ip,  Gateway-­‐ip   Page  Requested  URL  (Page,Object)   request  Cookie/Session-­‐id   Network Response Time  Referrer   Networkresponse time  GET  &  POST  requests    User  agents  (phone  type)   Real User! Load
 Monitoring! balancer" Server Response Time Web server"  Response?    Server  error,  web  site  error,  Hit  OK    Content  of  page  (i.e  order  details,   App server" funcHonal  errors) DB"  Delivered?    Aborted?    Network  Hmeout?  
  88. 88. Business Impact per user
 Satisfaction Reports Business Transaction Funnel Usage Reports: •  Data Browser •  Sales by Country/State
  89. 89. Session Replay
 User SessionTracking" Session Replay" Business Service Monitoring (e.g. Order Entry, Buy Stock)!
  90. 90. Root Cause Analysis per user " KPI configuration, 
 monitoring, & alerting! Server, network, & object performance diagnostics! Error reports & analysis!
  91. 91. Active Monitoring:! Active Monitoring"   Simulate users by scripting pages and/or business transactions!   Continuous measurements from outside the firewall!   Best used for incident, problem and SLA management.! Real User Monitoring! Real User" Monitoring"   Measures traffic of ALL users, transactions and pages!   Passive solution which gives information about the user experience!   Best used for performance and capacity management!
  92. 92. But, how to become the performance leader?!
  93. 93. 5 Web Performance Best Practices "
  94. 94. 1: Create Shared Performance Objectives"
  95. 95. Business  vs.  IT  
  96. 96. Business      Invest  to  aeract  more   and  new  customers      Quick  Time  to  market      Conversion  rates      Realize  Sales  targets   IT      Quality  of  Service      Be  Cost  efficient      Follow  procedures      Standardize  
  97. 97. …..How to work together?"
  98. 98. Key Business Metrics" AJenKon   Engagement   Conversion   NEW VISITORS GROWTH CONVERSION SEARCHES RATE PAGES TIME TWEETS MENTIONS NUMBER OF VISITS PER VISIT ON SITE x   ADS SEEN ORDER LOSS VALUE BOUNCE RATE 112
  99. 99. Key Business Metrics" AJenKon   Engagement   Conversion   NEW VISITORS GROWTH CONVERSION SEARCHES RATE PAGES TIME NUMBER TWEETS MENTIONS OF VISITS PER VISIT ON SITE x   ADS SEEN ORDER LOSS VALUE BOUNCE RATE Performance  maeers!     Web  Quality  directly  impact   business  metrics   113
  100. 100. Business oriented Key Performance objectives !
  101. 101. Business oriented Key Performance objectives ! System  Management   Business  TransacHons  
  102. 102. Business oriented Key Performance objectives ! System  Management   Business  TransacHons   System  Availability   End  User  Availability  
  103. 103. Business oriented Key Performance objectives ! System  Management   Business  TransacHons   System  Availability   End  User  Availability   Build  Quality   Web  Stability  
  104. 104. Business oriented Key Performance objectives ! System  Management   Business  TransacHons   System  Availability   End  User  Availability   Build  Quality   Web  Stability   Capacity   (Peak)  Traffic  
  105. 105. Business oriented Key Performance objectives ! System  Management   Business  TransacHons   System  Availability   End  User  Availability   Build  Quality   Web  Stability   Capacity   (Peak)  Traffic   Network   Page  Speed  
  106. 106. 2: Measure from End User Perspective !
  107. 107. Monitor Key Transactions from where your End-Users are!
  108. 108. Benchmark against the competition!
  109. 109. Measure every detail of your traffic" Customers: Global! Access: Getting Mobile! Technologies: Enterprise/Virtualization/Cloud! 12 4!
  110. 110. 3. Plan for Peak Crowds"
  111. 111. 1.     Compare   Actual  Results  
  112. 112. 1.     Compare   Actual  Results   2.   With  Actual   Performance  
  113. 113. 1.     Compare   Actual  Results   2.   With  Actual   Performance   3.   To  forecast   Capacity  &   Performance  
  114. 114. Peak Traffic: Itʼs a Business KPI! " Capacity  forecast,  actuals  and  maximum   Max.  number  of  pageviews  per  hour     1.200.000 1.000.000 800.000 Christmas/EOY   Pageviews 600.000 400.000 Traffic  Realized   Traffic  Forecast   200.000 MAX  capacity   Safety  capacity   0 nov-11 Jan-11 apr110 sep-11 jan-10 jul-10 sep-10 dec-10 mrt-10 jun-10 okt-10 aug-10 jul-09 sep-09 dec-09 nov-10 jun-09 okt-09 aug-09 jul-11 Mei-09 mei-11 okt-11 nov-09 feb-10 jun-11 aug-11 apr-11 dec-11 mrt-11 feb-11 mei10
  115. 115. 4. Test and Simulate performance"
  116. 116. The performance challenge"
  117. 117. Make sure the view your customers are getting is the view you want them to see!
  118. 118. Make sure the view your customers are getting is the view you want them to see!
  119. 119. Measure performance for normal traffic…. "
  120. 120. …. AND Peak Traffic"
  121. 121. Find your ceilings "
  122. 122. Source: John Allspaw, Flickr.com what you have left The End Know what your limits are…"
  123. 123. Plan for the unexpected
 Measure performance under normal and peak conditions to find all the problems and optimization opportunities

  124. 124. 5. Tying performance to conversion:" Web Performance Analytics!
  125. 125. Is it marketing?"
  126. 126. Or the WEB Experience?"   Why  do  customers  exit?     Offer?     Performance?     FuncHonal  errors?     Technical  errors?     Design  errors?     Business  impact?     In  affected  users?     In  missed  transacKons?     In  €?  
  127. 127. Online Success requires a complete picture" IT  OperaKons:   ? Web  AnalyKcs:      System  Management      Conversion      CPU,  Memory      Pageviews      Quality  of  Service      Visitors  
  128. 128. Online Success requires a complete picture" IT  OperaKons:   Web  AnalyKcs:      System  Management      Conversion      CPU,  Memory      Pageviews      Quality  of  Service      Visitors   Web  Performance  AnalyKcs:      Assess  impact  of  performance  on  user  behaviour      Tie  customer  saHsfacHon  to  quality  of  delivery      QuanHfy  revenue  risked  
  129. 129. The Web Performance Challenge:
 Balancing Cost vs. Performance 
 Revenue" Conversion Optimization" Operations" Development"
  130. 130. The Web Performance Challenge:
 Balancing Cost vs. Performance 
 Revenue" Conversion Optimization" Operations" Development" Am  I  opHmizing  for   performance  AND  cost?  
  131. 131. The Web Performance Challenge:
 Balancing Cost vs. Performance 
 Revenue" Are  these  the  most  cost   effecHve  improvements?   Conversion Optimization" Operations" Development" Am  I  opHmizing  for   performance  AND  cost?  
  132. 132. The Web Performance Challenge:
 Balancing Cost vs. Performance 
 Is  my  performance  improvement   generaHng  conversion?   Revenue" Are  these  the  most  cost   effecHve  improvements?   Conversion Optimization" Operations" Development" Am  I  opHmizing  for   performance  AND  cost?  
  133. 133. Creating the Big Picture:
 Break down the Web Application Delivery Chain…! Management Tool! Internet! Load
 Web App DB! Client! balancer! server! server!
  134. 134. Creating the Big Picture:
 ….and relate Customer Experience….. ! Type of visitors Browser type Application Infrastructure Performance Performance Management Tool! Internet! Load
 Web App DB! Client! balancer! server! server!
  135. 135. Creating the Big Picture:
 …to revenue risked! Type of visitors Browser type Application Infrastructure Performance Performance Management Tool! Internet! Load
 Web App DB! Client! balancer! server! server! Conversion Speed/Availability Functional Technical Quality Quality & Capacity
  136. 136. Make your own business case" Current
 daily orders" (! *! Increased" conversions" +! Increased
 order value" )! ROI
 =! (days)" Cost of performance
 enhancement" 151!
  137. 137. Make your own business case" (! )! $10,000! *! 0.1607! +! 0.0551! $2,158! 23.17
 =! =! days" $50,000! $50,000! 152!
  138. 138. Actionable reporting:" Monetize your performance " Availability Speed Capacity User Experience SLA IT suppliers Shared KPI’s +/- Conversion
  139. 139. Summary"
  140. 140. 1. Create Shared Performance Objectives 2. Measure from End User perspective 3. Plan for Peak Crowds 4. Test and simulate performance 5. Create the BIG picture: Web Performance Analytics
  141. 141. Questions?!
  142. 142. Analytics Usability Performability (What did they do (How did they (Could they do on the site?)! interact with it?)! what they wanted to do?)! VOC Communilytics Competition (What were their (What were they (What are they motivations?)! saying?)! up to?)!
  143. 143. Thanks! More questions? M: jtjepkema@measureworks.nl T: @jeroentjepkema W: www.measureworks.nl

×