Performance Testing: Putting Cloud Customers Back in the Driver’s Seat

  • 810 views
Uploaded on

Many businesses wrongly assume they will enjoy Google.com- and Amazon.com-like performance and consistency when they enlist cloud computing services from these and other major cloud providers. …

Many businesses wrongly assume they will enjoy Google.com- and Amazon.com-like performance and consistency when they enlist cloud computing services from these and other major cloud providers.

The truth is that businesses must conduct due diligence and insist on business-relevant performance guarantees in their service level agreements (SLAs). The keys for businesses success in working with cloud providers lies in understanding exactly why businesses are using the cloud and in testing performance levels from the realistic perspective of application end-users--both before and after a cloud service provider is enlisted.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
810
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
35
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • Last updated or created: Nov ‘09Moved firewall to the right; more detail in data centerKey themes:Delivering Web/mobile apps is complicated and involves many services.Talk trackWhy is it your data center monitoring tools can be reporting “green” but your users are unhappy? It’s because of something called “the Web applications delivery chain.” To deliver a Web or mobile application in today’s Internet, you must use a complex set of services and layers that are called the “Web application delivery chain.”These services must all work together to deliver the application to the user. If any one of them has an issue, your user will have a bad experience.First, your application must go through your major corporate ISP.Then, it travels thru the Internet.Increasingly, Web applications today are using third party or cloud services (we’ve seen companies where their home page has over 20 third party providers or services). It’s also very common to use a Content Delivery Network to accelerate the delivery of the Web application. Eventually, your Web or mobile application goes thru a local ISP or a mobile carrier.Ultimately it reaches your end user or customer, where it has to run on their local device – a computer or smart phone – and in their browser. A few years ago it seemed that only Internet Explorer mattered, but now there are a myriad of browsers on the market and, unfortunately for companies delivering Web applications, they all work a differently.
  • SLA aligned to goals based on end-user availability and performanceAlign SLA to type of cloud What you should expect?

Transcript

  • 1. Performance Testing:Putting Cloud Customers Back in the Driver’s Seat
    Imad Mouline, CTO, Gomez
    Twitter: @imadmouline
  • 2. Agenda
    A brief overview of the performance problem in the cloud
    Real-world data: how have cloud providers performed so far?
    Optimization opportunities in the cloud
    Takeaways
    Q&A
  • 3. Typical Web Application Delivery
    The Web Application Delivery Chain
    3rd Party/Cloud Services
    Browsers and devices
    Local ISP
    Users
    Load Balancers
    Web Servers
    Mobile Components
    App Servers
    Internet
    MajorISP
    DB Servers
    Mobile Carrier
    Storage
    Mainframe
    Network
    Content DeliveryNetworks
    Traditional zone of control
  • 4.
    • Elastic and scalable
    • 5. Focus on my business while someone manages infrastructure
    The Web Application Delivery Chain
    3rd Party/Cloud Services
    Browsers and devices
    Local ISP
    Users
    Load Balancers
    Web Servers
    Mobile Components
    App Servers
    Cloud
    Internet
    MajorISP
    DB Servers
    Mobile Carrier
    Storage
    Mainframe
    Network
    Content DeliveryNetworks
    Moving Web Applications to the Cloud: Benefits
    Traditional zone of control
  • 6. The Web Application Delivery Chain
    Cloud is opaque
    • Loss of visibility and control
    • 7. Traditional tools don’t apply
    3rd Party/Cloud Services
    Browsers and devices
    Local ISP
    Users
    Users
    Load Balancers
    Web Servers
    Mobile Components
    App Servers
    Cloud
    Internet
    MajorISP
    DB Servers
    Mobile Carrier
    Storage
    Mainframe
    Network
    Content DeliveryNetworks
    The Problem: The Cloud Creates Performance Concerns
    Traditional zone of control
  • 8. The Answer: Adopt an “Outside-In” User Point of View
    • Full understanding of performance from user perspective
    The Web Application Delivery Chain
    3rd Party/Cloud Services
    Browsers and devices
    Local ISP
    Users
    Users
    Load Balancers
    Web Servers
    Mobile Components
    “Outside-in” customer point of view
    App Servers
    Internet
    MajorISP
    DB Servers
    Mobile Carrier
    Storage
    Mainframe
    Network
    Content DeliveryNetworks
    Traditional zone of control
  • 9. The Web Application Delivery Chain
    My users
    Cloud is shared
    • Others can affect my performance
    3rd Party/Cloud Services
    Browsers and devices
    Local ISP
    Load Balancers
    Web Servers
    Other users
    My app
    Mobile Components
    App Servers
    Other
    app
    Other
    app
    Other
    app
    Internet
    Other users
    MajorISP
    DB Servers
    Mobile Carrier
    Storage
    Mainframe
    Other users
    Network
    Content DeliveryNetworks
    The Problem: The Cloud Creates Performance Concerns
  • 10.
    • Multiple contributors help diagnose issues for everyone
    The Web Application Delivery Chain
    My users
    Cloud is shared
    3rd Party/Cloud Services
    Browsers and devices
    Local ISP
    Load Balancers
    Web Servers
    Other users
    My app
    Mobile Components
    “Outside-in” customer point of view
    App Servers
    Other
    app
    Other
    app
    Other
    app
    Internet
    Other users
    MajorISP
    DB Servers
    Mobile Carrier
    Storage
    Mainframe
    Other users
    Network
    Content DeliveryNetworks
    The Answer: Collective Intelligence
  • 11. Web Applications in the Cloud: Need to Be Able to Pinpoint Problems
    The Web Application Delivery Chain
    My users
    3rd Party/Cloud Services
    Browsers and devices
    Local ISP
    Load Balancers
    Web Servers
    Other users
    Mobile Components
    App Servers
    Internet
    Other users
    MajorISP
    DB Servers
    Mobile Carrier
    Storage
    Mainframe
    Other users
    Network
    Content DeliveryNetworks
  • 12. Introducing… CloudSleuth
  • 13. A Year In The Cloud
    An end-user perspective on cloud performance
  • 14. What we measured
    Cloned reference web application deployed across various IaaS and PaaS providers
    Added various services such as CDN when available
    Structured application to highlight performance issues
  • 15. How we measured
    Various locations around the world
    Backbone and Last Mile locations
    Every 15 minutes, 24/7, for almost 1 year
    Used a strict definition of availability.
  • 16. Not all clouds perform the same way
    Average Response Time of Reference Application, as measured from US backbone locations
  • 17. Taking the long view – response time
    © 2010 Gomez – All Rights Reserved
  • 18. Taking the long view - availability
  • 19. Geographic Latency – GoGrid
  • 20. Geographic Latency – S3
  • 21. Going International?
  • 22. It’s not all about network latency
  • 23. Enough scary news…
    … let’s see something good for a change
  • 24. Geographic Latency – S3
  • 25. Added services help… a lot!
  • 26. Unique opportunities for optimization
    Better performance, for little work, at no extra cost?
  • 27. Best Practice: Define your goals and build a plan
    Align goals across your organization Why are we moving to the cloud?
    Common goals include:
    Additional Capacity – How much capacity do we need during normal and peak times?
    Improved End-User Experience – What performance goals are we trying to deliver against?
    Greater Elasticity – How quickly can the provider we select ramp up to meet our needs?
    Flexible Bursting – How fast do we need to be able to access additional capacity?
    If only there was a
    button to push!
  • 28. Best Practice: Keep your end-users in mind
    How was that experience for your customers?
    Test your cloud applications the same way your customers use it:
    What they do?
    • Customers care about completing tasks NOT whether the homepage is available
    Where they do it from?
    • Your customers don’t live in the cloud, test from their perspective
    When they do it?
    • Test at peak and normal traffic levels, to find all the problems
    What expectations do customers have?
    • Is 4 seconds fast enough or does it have to be quicker?
    Geographic disparities
    4 sec’s
    22 sec’s
    Find configuration, capacity, and performance problems before end-users with pre-production testing and production monitoring
  • 29. Best Practice: Performance testing cloud capabilities
    Evaluate vendors based on your goals…
    Capacity
    Test vendors to 15-20% past estimated capacity goals
    Elasticity
    Baseline end-user performance before & after testing
    Test during pre-deployment and in production
    Ramp elasticity testing to peak levels
    Burstability
    Isolate the cloud elements from other infrastructure to test
    Test the “failover process”
  • 30. Best Practice: Set SLAs to match your goals
    Set SLAs based on your goals…
    End-user availability and response times
    Capacity & elasticity objectives
    Burstability goals
    Set SLAs based on how you are using the cloud…
  • 31. Putting together a cloud management strategy
  • 32. Questions?
    imouline@gomez.com
    Twitter: @imadmouline