Performance Testing: Putting Cloud Customers Back in the Driver’s Seat

943 views
894 views

Published on

Many businesses wrongly assume they will enjoy Google.com- and Amazon.com-like performance and consistency when they enlist cloud computing services from these and other major cloud providers.

The truth is that businesses must conduct due diligence and insist on business-relevant performance guarantees in their service level agreements (SLAs). The keys for businesses success in working with cloud providers lies in understanding exactly why businesses are using the cloud and in testing performance levels from the realistic perspective of application end-users--both before and after a cloud service provider is enlisted.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
943
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
39
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • SLA aligned to goals based on end-user availability and performanceAlign SLA to type of cloud What you should expect?
  • Performance Testing: Putting Cloud Customers Back in the Driver’s Seat

    1. 1. Performance Testing:Putting Cloud Customers Back in the Driver’s Seat<br />Imad Mouline, CTO, Gomez<br />Twitter: @imadmouline<br />
    2. 2. Agenda<br />A brief overview of the performance problem in the cloud<br />Real-world data: how have cloud providers performed so far?<br />Optimization opportunities in the cloud<br />Takeaways<br />Q&A<br />
    3. 3. Typical Web Application Delivery<br />The Web Application Delivery Chain<br />3rd Party/Cloud Services<br />Browsers and devices<br />Local ISP<br />Users<br />Load Balancers<br />Web Servers<br />Mobile Components<br />App Servers<br />Internet<br />MajorISP<br />DB Servers<br />Mobile Carrier<br />Storage<br />Mainframe<br />Network<br />Content DeliveryNetworks<br />Traditional zone of control<br />
    4. 4. <ul><li>Elastic and scalable
    5. 5. Focus on my business while someone manages infrastructure</li></ul>The Web Application Delivery Chain<br />3rd Party/Cloud Services<br />Browsers and devices<br />Local ISP<br />Users<br />Load Balancers<br />Web Servers<br />Mobile Components<br />App Servers<br />Cloud<br />Internet<br />MajorISP<br />DB Servers<br />Mobile Carrier<br />Storage<br />Mainframe<br />Network<br />Content DeliveryNetworks<br />Moving Web Applications to the Cloud: Benefits<br />Traditional zone of control<br />
    6. 6. The Web Application Delivery Chain<br />Cloud is opaque<br /><ul><li>Loss of visibility and control
    7. 7. Traditional tools don’t apply</li></ul>3rd Party/Cloud Services<br />Browsers and devices<br />Local ISP<br />Users<br />Users<br />Load Balancers<br />Web Servers<br />Mobile Components<br />App Servers<br />Cloud<br />Internet<br />MajorISP<br />DB Servers<br />Mobile Carrier<br />Storage<br />Mainframe<br />Network<br />Content DeliveryNetworks<br />The Problem: The Cloud Creates Performance Concerns<br />Traditional zone of control<br />
    8. 8. The Answer: Adopt an “Outside-In” User Point of View<br /><ul><li>Full understanding of performance from user perspective</li></ul>The Web Application Delivery Chain<br />3rd Party/Cloud Services<br />Browsers and devices<br />Local ISP<br />Users<br />Users<br />Load Balancers<br />Web Servers<br />Mobile Components<br />“Outside-in” customer point of view<br />App Servers<br />Internet<br />MajorISP<br />DB Servers<br />Mobile Carrier<br />Storage<br />Mainframe<br />Network<br />Content DeliveryNetworks<br />Traditional zone of control<br />
    9. 9. The Web Application Delivery Chain<br />My users<br />Cloud is shared<br /><ul><li>Others can affect my performance</li></ul>3rd Party/Cloud Services<br />Browsers and devices<br />Local ISP<br />Load Balancers<br />Web Servers<br />Other users<br />My app<br />Mobile Components<br />App Servers<br />Other<br />app<br />Other<br />app<br />Other<br />app<br />Internet<br />Other users<br />MajorISP<br />DB Servers<br />Mobile Carrier<br />Storage<br />Mainframe<br />Other users<br />Network<br />Content DeliveryNetworks<br />The Problem: The Cloud Creates Performance Concerns<br />
    10. 10. <ul><li>Multiple contributors help diagnose issues for everyone</li></ul>The Web Application Delivery Chain<br />My users<br />Cloud is shared<br />3rd Party/Cloud Services<br />Browsers and devices<br />Local ISP<br />Load Balancers<br />Web Servers<br />Other users<br />My app<br />Mobile Components<br />“Outside-in” customer point of view<br />App Servers<br />Other<br />app<br />Other<br />app<br />Other<br />app<br />Internet<br />Other users<br />MajorISP<br />DB Servers<br />Mobile Carrier<br />Storage<br />Mainframe<br />Other users<br />Network<br />Content DeliveryNetworks<br />The Answer: Collective Intelligence<br />
    11. 11. Web Applications in the Cloud: Need to Be Able to Pinpoint Problems<br />The Web Application Delivery Chain<br />My users<br />3rd Party/Cloud Services<br />Browsers and devices<br />Local ISP<br />Load Balancers<br />Web Servers<br />Other users<br />Mobile Components<br />App Servers<br />Internet<br />Other users<br />MajorISP<br />DB Servers<br />Mobile Carrier<br />Storage<br />Mainframe<br />Other users<br />Network<br />Content DeliveryNetworks<br />
    12. 12. Introducing… CloudSleuth<br />
    13. 13. A Year In The Cloud <br />An end-user perspective on cloud performance<br />
    14. 14. What we measured<br />Cloned reference web application deployed across various IaaS and PaaS providers<br />Added various services such as CDN when available<br />Structured application to highlight performance issues<br />
    15. 15. How we measured<br />Various locations around the world<br />Backbone and Last Mile locations<br />Every 15 minutes, 24/7, for almost 1 year<br />Used a strict definition of availability.<br />
    16. 16. Not all clouds perform the same way<br />Average Response Time of Reference Application, as measured from US backbone locations<br />
    17. 17. Taking the long view – response time<br />© 2010 Gomez – All Rights Reserved<br />
    18. 18. Taking the long view - availability<br />
    19. 19. Geographic Latency – GoGrid<br />
    20. 20. Geographic Latency – S3<br />
    21. 21. Going International?<br />
    22. 22. It’s not all about network latency<br />
    23. 23. Enough scary news…<br />… let’s see something good for a change<br />
    24. 24. Geographic Latency – S3<br />
    25. 25. Added services help… a lot!<br />
    26. 26. Unique opportunities for optimization<br />Better performance, for little work, at no extra cost?<br />
    27. 27. Best Practice: Define your goals and build a plan<br />Align goals across your organization Why are we moving to the cloud? <br />Common goals include:<br />Additional Capacity – How much capacity do we need during normal and peak times?<br />Improved End-User Experience – What performance goals are we trying to deliver against?<br />Greater Elasticity – How quickly can the provider we select ramp up to meet our needs?<br />Flexible Bursting – How fast do we need to be able to access additional capacity?<br />If only there was a <br />button to push!<br />
    28. 28. Best Practice: Keep your end-users in mind<br />How was that experience for your customers?<br />Test your cloud applications the same way your customers use it:<br />What they do? <br /><ul><li>Customers care about completing tasks NOT whether the homepage is available</li></ul>Where they do it from?<br /><ul><li>Your customers don’t live in the cloud, test from their perspective</li></ul>When they do it?<br /><ul><li>Test at peak and normal traffic levels, to find all the problems</li></ul>What expectations do customers have?<br /><ul><li>Is 4 seconds fast enough or does it have to be quicker?</li></ul>Geographic disparities<br />4 sec’s<br />22 sec’s<br />Find configuration, capacity, and performance problems before end-users with pre-production testing and production monitoring<br />
    29. 29. Best Practice: Performance testing cloud capabilities<br />Evaluate vendors based on your goals…<br />Capacity<br />Test vendors to 15-20% past estimated capacity goals<br />Elasticity<br />Baseline end-user performance before & after testing<br />Test during pre-deployment and in production<br />Ramp elasticity testing to peak levels<br />Burstability<br />Isolate the cloud elements from other infrastructure to test<br />Test the “failover process”<br />
    30. 30. Best Practice: Set SLAs to match your goals<br />Set SLAs based on your goals…<br />End-user availability and response times<br />Capacity & elasticity objectives<br />Burstability goals<br />Set SLAs based on how you are using the cloud…<br />
    31. 31. Putting together a cloud management strategy<br />
    32. 32. Questions?<br />imouline@gomez.com<br />Twitter: @imadmouline<br />

    ×