Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
CASE STUDY
Leading Retailer Chooses
Microsoft Visual Studio for
Load Testing Critical Web Services
Background
RTTS’ client...
Licensing expenditures and load generation infrastructure costs were paramount in being able to meet
budget constraints, w...
to the current Visual Studio Load Test Rig. If and when the need arises, these tests could be ported to the
Windows Azure ...
Upcoming SlideShare
Loading in …5
×

Case Study: Leading Retailer Chooses Microsoft Visual Studio for Load Testing its Critical Web Services

498 views

Published on

See how RTTS' performance/load testing team utilized Microsoft's load testing tool in Visual Studio to test a high availability web site and its web services while reducing the total cost of ownership (TCO) for the customer.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Case Study: Leading Retailer Chooses Microsoft Visual Studio for Load Testing its Critical Web Services

  1. 1. CASE STUDY Leading Retailer Chooses Microsoft Visual Studio for Load Testing Critical Web Services Background RTTS’ client operated numerous heterogeneous customer-facing (i.e. revenue producing) web applications, as well as administrative and customer support applications, that relied on a common set of services, including customer account retrieval, delivery date availability, and address validation. Using IBM’s Integration Bus solution (formerly known as WebSphere Message Broker), RTTS’ client was launching an enterprise server bus (ESB), so that these individual applications (and future applications) could more easily scale in terms of business growth. Challenges High availability and responsiveness were critical factors to the customer’s business model, since these factors directly affected order conversion for the customer-facing applications, as well as mean time to resolution regarding customer service inquiries. RTTS’s customer required a load testing solution that possessed a minimal total cost of ownership (TCO) that could scale in proportion to its business growth and the consumption of the web services provided by the ESB. Industry Retail Solution & Services Performance/Load Testing Software Test Management Test tool evaluation Tools  Microsoft Visual Studio Ultimate version
  2. 2. Licensing expenditures and load generation infrastructure costs were paramount in being able to meet budget constraints, while the ability to scale to the transaction rates and concurrency were imperative to the application and infrastructure teams. The short-term load testing goal was to be able to support the demands of at least one (1) of the customer’s primary brand’s e-retail web site using a single instance of the IBM Integration Bus. Peak usage had been observed to be approximately 7,000 online customers. Strategy RTTS put forth a proposal that would assess the available load testing solutions. Proof-of-concepts were arranged that addressed the gamut of open source load testing solutions and commercial load testing solutions. The results of the proof-of-concepts were presented to the stakeholders in conjunction with the projected costs of the solution in order to determine the best return on investment. RTTS also took ownership of the rollout of the load testing solution by providing professional services that installed the testing products and engineered the baseline performance tests. Solution Several load testing solutions were evaluated using the criteria set forth by RTTS’ customer. Open source, commercial, and cloud-based load testing solutions were evaluated via proof-of-concepts against a subset of the ESB’s suite of web services. Load test solutions that were provided by open source testing tools were deemed a risk in terms of having product support. Cloud-based testing solutions turned out not to be a fit a the time due to concerns regarding security, proprietary scripting languages, and the client’s current corporate culture. However, RTTS’ client did express interest in having the flexibility to move to a cloud- based solution, especially regarding its load generation infrastructure. In the end Microsoft Visual Studio Ultimate was selected to be the best fit based upon its capabilities, familiarity, and cost. The initial cost of Microsoft Visual Studio Ultimate was significantly less expensive when compared to some of the more established load testing solutions in the market. Microsoft’s licensing model permitted unlimited numbers of virtual users, so that testing could easily scale along with the growth of the customer’s business. Along with the ability to systematically test the ESB using a familiar Visual Studio IDE, the ability to add capacity to its load generation infrastructure was seamless. RTTS was able to incrementally add to the customer’s load generation infrastructure by simply spinning up additional virtual machines and adding them
  3. 3. to the current Visual Studio Load Test Rig. If and when the need arises, these tests could be ported to the Windows Azure cloud platform for greater flexibility for implementing its load generation infrastructure. Benefits RTTS successfully installed and configured the necessary Microsoft Visual Studio components and was able to jumpstart the load testing initiative by testing a subset of the ESB web services related to product availability, delivery dates, and shipping options for one (1) of the customer’s family of brands. Using a Visual Studio Load Test Rig of six (6) agents, load tests reaching 12,000 concurrent users were executed. Microsoft Visual Studio’s load testing capabilities were able to assist in discovering bottlenecks related to the threading model being used by the ESB’s database connection pool, as well as within the delivered HTTP configuration of the ESB’s web interface. Once the environment was optimized, 90% of all web service requests were returned in under one (1) second. The IBM Integration Bus was shown to support the processing of 55 web service requests per second with its CPU utilization averaging about 60%. At the conclusion o the project RTTS was tasked with leading the testing effort by augmenting the collection of web services being tested, incorporating additional retail brands and back-office applications, and subsequently increasing the end-user consumption of the web services. RTTS is a Microsoft Gold Partner. Since 2006, RTTS has been an education & consulting partner with Microsoft – implementing solutions and providing educational services around Microsoft's Visual Studio testing tools. RTTS has worked with the Visual Studio team to develop top-notch solutions around their excellent toolset. For more information about our solutions and services, go to www.rttsweb.com copyright Real-Time Technology Solutions, Inc. 2014

×