SlideShare a Scribd company logo
T17
Performance Testing
10/16/2014 1:30:00 PM
Build Your Custom Performance
Testing Framework
Presented by:
Prashant Suri
Rackspace
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Prashant Suri
Rackspace
Prashant Suri is a senior member of the performance engineering team at
Rackspace, the open cloud company. In his role, he helps performance test and
benchmark Rackspace's cloud products and applications, writing his own test
frameworks to help. In his early years, Prashant developed a number of products
on Java/J2EE stack but was fascinated by the vastness of performance
engineering. He enjoys improving products’ scaling and robustness. Through the
years, he has helped products, developed and used by Fortune 500, scale better.
A presenter at international testing conferences, Prashant can be reached at
prashant.suri@rackspace.com.
Build your custom Performance
Testing Framework
Prashant Suri
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Agenda
2
• Evolution of the web and growing complexity of products.
• Essentials of performance engineering.
• Popular performance test tools.
• Holistic performance test framework.
• Components of a performance test framework.
• Deep dive on an example: performance testing at RAX
• Advantages of building your own framework.
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Evolution of the Web
3
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Essentials of Performance Engineering
• Performance objectives – Latency, Throughput, Utilization,
Workload.
• Performance Modeling – Scenarios, Objectives,
Requirements, Budgets, Metrics.
• Architecture and Design Guidelines – Principles, Practices
and Patterns
• Performance and Scalability Frame – Coupling and
cohesion, Communication, Concurrency, Resource
Management, Caching, State Management, Data
Structures/Algorithms.
• Measurement, Tuning and Analysis.
4
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Popular Performance Test Tools
5
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
There’s No Silver Bullet
6
• Performance testing is more than simulating load.
• Accurate and intricate work load models need to be
implemented.
• Traffic to be simulated from world geographies.
• Testing in service oriented architectures comes with its own
challenges.
• Reporting and pattern analysis is key for successful
performance engineering.
• Performance monitors and profiling equally important.
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Holistic Performance Test Framework
7
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Components of the framework
8
Controller Agents
Database & Graphing
Engine
SimulatingLoads
•  Brain behind the complete
orchestration. Parses and
executes the data script.
•  Picks up threads from queues,
and hit application under test.
•  Repository for metrics of
response times, throughputs,
errors etc.
•  Can scale dynamically at run
time, to add load generation
capacity.
•  Maintains the test details,
workload model details and
objectives of any
performance test.
Capable of distributing load from
different data centers.
•  Distributed across geographies to
simulate identical world traffic.
•  Optimal database tuning for
handling 1000+ transactions
per second test.
Monitoringand
Debugging
Could be integrated to application
monitors and profilers.
•  Feeds all metrics data into the
Database.
•  Correlates and consolidates
all test data.
Updated information on runtime
activity from database is read and
published.
•  Logging for every granular
transaction with application under
test.
•  Historical comparisons and
pattern analysis.
Could be leveraged for analysis
and data based decision making.
•  Distributed agents through way of
logging are consolidated for
complete test.
•  Profiling and application
under test data, for analyzing
deviation and anomalies.
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Adobe CQMicrosoft® SharePointMicrosoft® ExchangeRackspace Email
Case Study – Capabilities galore!
Single F/W to simulate on all Products
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Sample Script – Design workload models!
10
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
11
Execution – Distribute across Agents
through a queue
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
12
Run time statistics of Performance Test
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Consolidated Reports
13
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Consolidated Reports
14
RACKSPACE® HOSTING | WWW.RACKSPACE.COM
Advantages of Building your own
frameworks
15
• Cost savings on commercial off-the-shelf tools.
• Flexibility to simulate complex workload at any scale
(literally!)
• Simplicity in designing complex workload models.
• No constraints and capability of leveraging as per product
needs.
• Ease of integration with CI/CD Pipeline.
• Load simulator and application resource monitors in a
single framework setup.
• Efficient and inclusive engagement for Performance
Engineering in overall product development lifecycle.
RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218
US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM
RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM
Questions…
RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218
US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM
RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM
Prashant.suri@rackspace.com

More Related Content

What's hot

Selenium Test Automation
Selenium Test AutomationSelenium Test Automation
Selenium Test Automation
BabuDevanandam
 
Test automation process
Test automation processTest automation process
Test automation process
Bharathi Krishnamurthi
 
Jbehave- Basics to Advance
Jbehave- Basics to AdvanceJbehave- Basics to Advance
Jbehave- Basics to Advance
Ravinder Singh
 
Designing a Test Automation Framework By Quontra solutions
Designing a Test Automation Framework By Quontra solutionsDesigning a Test Automation Framework By Quontra solutions
Designing a Test Automation Framework By Quontra solutions
QUONTRASOLUTIONS
 
Web and load testing with Visual Studio 2010 Ultimate
Web and load testing with Visual Studio 2010 UltimateWeb and load testing with Visual Studio 2010 Ultimate
Web and load testing with Visual Studio 2010 Ultimate
Abhimanyu Singhal
 
Silk Performer Presentation v1
Silk Performer Presentation v1Silk Performer Presentation v1
Silk Performer Presentation v1
Sun Technlogies
 
Visual Studio 2010 for testers
Visual Studio 2010 for testersVisual Studio 2010 for testers
Visual Studio 2010 for testersArpit Dubey
 
Performance Testing from Scratch + JMeter intro
Performance Testing from Scratch + JMeter introPerformance Testing from Scratch + JMeter intro
Performance Testing from Scratch + JMeter intro
Mykola Kovsh
 
Behavior-Driven Development and Automation Testing Using Cucumber Framework W...
Behavior-Driven Development and Automation Testing Using Cucumber Framework W...Behavior-Driven Development and Automation Testing Using Cucumber Framework W...
Behavior-Driven Development and Automation Testing Using Cucumber Framework W...
KMS Technology
 
Introduction to SoapUI day 1
Introduction to SoapUI day 1Introduction to SoapUI day 1
Introduction to SoapUI day 1
Qualitest
 
Using Jenkins and Jmeter to build a scalable Load Testing solution
Using Jenkins and Jmeter to build a scalable Load Testing solutionUsing Jenkins and Jmeter to build a scalable Load Testing solution
Using Jenkins and Jmeter to build a scalable Load Testing solution
Ruslan Strazhnyk
 
API Testing - The power of libraries (chai, cheerio.js ,lodash and moment.js)
API Testing - The power of libraries (chai, cheerio.js ,lodash and moment.js)API Testing - The power of libraries (chai, cheerio.js ,lodash and moment.js)
API Testing - The power of libraries (chai, cheerio.js ,lodash and moment.js)
AnuradhaRaman5
 
ATLAS Automation POC
ATLAS Automation POCATLAS Automation POC
ATLAS Automation POCaakashmc
 
Qa process
Qa processQa process
Qa process
Aila Bogasieru
 
Keyword-driven Test Automation Framework
Keyword-driven Test Automation FrameworkKeyword-driven Test Automation Framework
Keyword-driven Test Automation Framework
Mikhail Subach
 
An easy way to automate complex UI
An easy way to automate complex UIAn easy way to automate complex UI
An easy way to automate complex UI
Ivan Pashko
 
Laws of test automation framework
Laws of test automation frameworkLaws of test automation framework
Laws of test automation framework
vodqancr
 
Test Automation Architecture That Works by Bhupesh Dahal
Test Automation Architecture That Works by Bhupesh DahalTest Automation Architecture That Works by Bhupesh Dahal
Test Automation Architecture That Works by Bhupesh Dahal
QA or the Highway
 

What's hot (19)

Selenium Test Automation
Selenium Test AutomationSelenium Test Automation
Selenium Test Automation
 
Test automation process
Test automation processTest automation process
Test automation process
 
Jbehave- Basics to Advance
Jbehave- Basics to AdvanceJbehave- Basics to Advance
Jbehave- Basics to Advance
 
Designing a Test Automation Framework By Quontra solutions
Designing a Test Automation Framework By Quontra solutionsDesigning a Test Automation Framework By Quontra solutions
Designing a Test Automation Framework By Quontra solutions
 
Web and load testing with Visual Studio 2010 Ultimate
Web and load testing with Visual Studio 2010 UltimateWeb and load testing with Visual Studio 2010 Ultimate
Web and load testing with Visual Studio 2010 Ultimate
 
Silk Performer Presentation v1
Silk Performer Presentation v1Silk Performer Presentation v1
Silk Performer Presentation v1
 
Visual Studio 2010 for testers
Visual Studio 2010 for testersVisual Studio 2010 for testers
Visual Studio 2010 for testers
 
Performance Testing from Scratch + JMeter intro
Performance Testing from Scratch + JMeter introPerformance Testing from Scratch + JMeter intro
Performance Testing from Scratch + JMeter intro
 
Behavior-Driven Development and Automation Testing Using Cucumber Framework W...
Behavior-Driven Development and Automation Testing Using Cucumber Framework W...Behavior-Driven Development and Automation Testing Using Cucumber Framework W...
Behavior-Driven Development and Automation Testing Using Cucumber Framework W...
 
Introduction to SoapUI day 1
Introduction to SoapUI day 1Introduction to SoapUI day 1
Introduction to SoapUI day 1
 
Using Jenkins and Jmeter to build a scalable Load Testing solution
Using Jenkins and Jmeter to build a scalable Load Testing solutionUsing Jenkins and Jmeter to build a scalable Load Testing solution
Using Jenkins and Jmeter to build a scalable Load Testing solution
 
API Testing - The power of libraries (chai, cheerio.js ,lodash and moment.js)
API Testing - The power of libraries (chai, cheerio.js ,lodash and moment.js)API Testing - The power of libraries (chai, cheerio.js ,lodash and moment.js)
API Testing - The power of libraries (chai, cheerio.js ,lodash and moment.js)
 
ATLAS Automation POC
ATLAS Automation POCATLAS Automation POC
ATLAS Automation POC
 
Qa process
Qa processQa process
Qa process
 
Keyword-driven Test Automation Framework
Keyword-driven Test Automation FrameworkKeyword-driven Test Automation Framework
Keyword-driven Test Automation Framework
 
An easy way to automate complex UI
An easy way to automate complex UIAn easy way to automate complex UI
An easy way to automate complex UI
 
Laws of test automation framework
Laws of test automation frameworkLaws of test automation framework
Laws of test automation framework
 
Test Automation Architecture That Works by Bhupesh Dahal
Test Automation Architecture That Works by Bhupesh DahalTest Automation Architecture That Works by Bhupesh Dahal
Test Automation Architecture That Works by Bhupesh Dahal
 
test_automation_POC
test_automation_POCtest_automation_POC
test_automation_POC
 

Similar to Build Your Custom Performance Testing Framework

Raghu VM_Cloud Resume
Raghu VM_Cloud ResumeRaghu VM_Cloud Resume
Raghu VM_Cloud ResumeRaghu Ravi
 
Migrating Traditional Apps from On-Premises to the Hybrid Cloud
Migrating Traditional Apps from On-Premises to the Hybrid CloudMigrating Traditional Apps from On-Premises to the Hybrid Cloud
Migrating Traditional Apps from On-Premises to the Hybrid Cloud
Rackspace
 
DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012
DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012
DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012
Matt Tesauro
 
DevOps, CLI, APIs, Oh My! Security Gone Agile
DevOps, CLI, APIs, Oh My!  Security Gone AgileDevOps, CLI, APIs, Oh My!  Security Gone Agile
DevOps, CLI, APIs, Oh My! Security Gone Agile
Matt Tesauro
 
DevOps Toolbox: Infrastructure as code
DevOps Toolbox: Infrastructure as codeDevOps Toolbox: Infrastructure as code
DevOps Toolbox: Infrastructure as code
sriram_rajan
 
Testing at-cloud-speed sans-app-sec-austin-2013
Testing at-cloud-speed sans-app-sec-austin-2013Testing at-cloud-speed sans-app-sec-austin-2013
Testing at-cloud-speed sans-app-sec-austin-2013Matt Tesauro
 
Shayne capo 2016
Shayne capo 2016Shayne capo 2016
Shayne capo 2016
Shayne Capo
 
Geek Nights Hong Kong
Geek Nights Hong KongGeek Nights Hong Kong
Geek Nights Hong Kong
Rahul Gupta
 
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web ServicesDickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Precisely
 
CA World 2014 Release Automation Coexistence in the Enterprise
CA World 2014 Release Automation Coexistence in the EnterpriseCA World 2014 Release Automation Coexistence in the Enterprise
CA World 2014 Release Automation Coexistence in the Enterprise
Mark Sigler
 
Deploy from OpenStack Trunk into a Production Environment
Deploy from OpenStack Trunk into a Production EnvironmentDeploy from OpenStack Trunk into a Production Environment
Deploy from OpenStack Trunk into a Production EnvironmentOpenStack Foundation
 
C5 journey to_the_cloud_with_oracle_sparc
C5 journey to_the_cloud_with_oracle_sparcC5 journey to_the_cloud_with_oracle_sparc
C5 journey to_the_cloud_with_oracle_sparc
Dr. Wilfred Lin (Ph.D.)
 
Pre-Con Ed: There has to be a Better Way to Fast Test Coverage!
Pre-Con Ed: There has to be a Better Way to Fast Test Coverage!Pre-Con Ed: There has to be a Better Way to Fast Test Coverage!
Pre-Con Ed: There has to be a Better Way to Fast Test Coverage!
CA Technologies
 
Elastic-Engineering
Elastic-EngineeringElastic-Engineering
Elastic-Engineering
Araf Karsh Hamid
 
SAP virtualization
SAP virtualizationSAP virtualization
SAP virtualization
Christopher Carter
 
Transform Software Testing and Quality with the Neotys-Inflectra Platform
Transform Software Testing and Quality with the Neotys-Inflectra PlatformTransform Software Testing and Quality with the Neotys-Inflectra Platform
Transform Software Testing and Quality with the Neotys-Inflectra Platform
Inflectra
 
On Metal - The Future Of Hybrid Cloud
On Metal - The Future Of Hybrid CloudOn Metal - The Future Of Hybrid Cloud
On Metal - The Future Of Hybrid Cloud
Rackspace Asia
 
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
Tearing Down Silos and Building Your Enterprise Dev/Ops EngineTearing Down Silos and Building Your Enterprise Dev/Ops Engine
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
Rackspace
 
Cloud Done Right - PaaS is the Remedy to VM Hangover
Cloud Done Right - PaaS is the Remedy to VM HangoverCloud Done Right - PaaS is the Remedy to VM Hangover
Cloud Done Right - PaaS is the Remedy to VM Hangover
Mohamad Afshar
 
AWS Summit 2013 | India - Running High Churn Development & Test Environments,...
AWS Summit 2013 | India - Running High Churn Development & Test Environments,...AWS Summit 2013 | India - Running High Churn Development & Test Environments,...
AWS Summit 2013 | India - Running High Churn Development & Test Environments,...
Amazon Web Services
 

Similar to Build Your Custom Performance Testing Framework (20)

Raghu VM_Cloud Resume
Raghu VM_Cloud ResumeRaghu VM_Cloud Resume
Raghu VM_Cloud Resume
 
Migrating Traditional Apps from On-Premises to the Hybrid Cloud
Migrating Traditional Apps from On-Premises to the Hybrid CloudMigrating Traditional Apps from On-Premises to the Hybrid Cloud
Migrating Traditional Apps from On-Premises to the Hybrid Cloud
 
DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012
DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012
DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012
 
DevOps, CLI, APIs, Oh My! Security Gone Agile
DevOps, CLI, APIs, Oh My!  Security Gone AgileDevOps, CLI, APIs, Oh My!  Security Gone Agile
DevOps, CLI, APIs, Oh My! Security Gone Agile
 
DevOps Toolbox: Infrastructure as code
DevOps Toolbox: Infrastructure as codeDevOps Toolbox: Infrastructure as code
DevOps Toolbox: Infrastructure as code
 
Testing at-cloud-speed sans-app-sec-austin-2013
Testing at-cloud-speed sans-app-sec-austin-2013Testing at-cloud-speed sans-app-sec-austin-2013
Testing at-cloud-speed sans-app-sec-austin-2013
 
Shayne capo 2016
Shayne capo 2016Shayne capo 2016
Shayne capo 2016
 
Geek Nights Hong Kong
Geek Nights Hong KongGeek Nights Hong Kong
Geek Nights Hong Kong
 
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web ServicesDickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
 
CA World 2014 Release Automation Coexistence in the Enterprise
CA World 2014 Release Automation Coexistence in the EnterpriseCA World 2014 Release Automation Coexistence in the Enterprise
CA World 2014 Release Automation Coexistence in the Enterprise
 
Deploy from OpenStack Trunk into a Production Environment
Deploy from OpenStack Trunk into a Production EnvironmentDeploy from OpenStack Trunk into a Production Environment
Deploy from OpenStack Trunk into a Production Environment
 
C5 journey to_the_cloud_with_oracle_sparc
C5 journey to_the_cloud_with_oracle_sparcC5 journey to_the_cloud_with_oracle_sparc
C5 journey to_the_cloud_with_oracle_sparc
 
Pre-Con Ed: There has to be a Better Way to Fast Test Coverage!
Pre-Con Ed: There has to be a Better Way to Fast Test Coverage!Pre-Con Ed: There has to be a Better Way to Fast Test Coverage!
Pre-Con Ed: There has to be a Better Way to Fast Test Coverage!
 
Elastic-Engineering
Elastic-EngineeringElastic-Engineering
Elastic-Engineering
 
SAP virtualization
SAP virtualizationSAP virtualization
SAP virtualization
 
Transform Software Testing and Quality with the Neotys-Inflectra Platform
Transform Software Testing and Quality with the Neotys-Inflectra PlatformTransform Software Testing and Quality with the Neotys-Inflectra Platform
Transform Software Testing and Quality with the Neotys-Inflectra Platform
 
On Metal - The Future Of Hybrid Cloud
On Metal - The Future Of Hybrid CloudOn Metal - The Future Of Hybrid Cloud
On Metal - The Future Of Hybrid Cloud
 
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
Tearing Down Silos and Building Your Enterprise Dev/Ops EngineTearing Down Silos and Building Your Enterprise Dev/Ops Engine
Tearing Down Silos and Building Your Enterprise Dev/Ops Engine
 
Cloud Done Right - PaaS is the Remedy to VM Hangover
Cloud Done Right - PaaS is the Remedy to VM HangoverCloud Done Right - PaaS is the Remedy to VM Hangover
Cloud Done Right - PaaS is the Remedy to VM Hangover
 
AWS Summit 2013 | India - Running High Churn Development & Test Environments,...
AWS Summit 2013 | India - Running High Churn Development & Test Environments,...AWS Summit 2013 | India - Running High Churn Development & Test Environments,...
AWS Summit 2013 | India - Running High Churn Development & Test Environments,...
 

More from TechWell

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
TechWell
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
TechWell
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
TechWell
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
TechWell
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
TechWell
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
TechWell
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
TechWell
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
TechWell
 
Ma 15
Ma 15Ma 15
Ma 15
TechWell
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
TechWell
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
TechWell
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
TechWell
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
TechWell
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
TechWell
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
TechWell
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
TechWell
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
TechWell
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
TechWell
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
TechWell
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
TechWell
 

More from TechWell (20)

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
 
Ma 15
Ma 15Ma 15
Ma 15
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
 

Recently uploaded

UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Product School
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
Bhaskar Mitra
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 

Recently uploaded (20)

UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 

Build Your Custom Performance Testing Framework

  • 1. T17 Performance Testing 10/16/2014 1:30:00 PM Build Your Custom Performance Testing Framework Presented by: Prashant Suri Rackspace Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  • 2. Prashant Suri Rackspace Prashant Suri is a senior member of the performance engineering team at Rackspace, the open cloud company. In his role, he helps performance test and benchmark Rackspace's cloud products and applications, writing his own test frameworks to help. In his early years, Prashant developed a number of products on Java/J2EE stack but was fascinated by the vastness of performance engineering. He enjoys improving products’ scaling and robustness. Through the years, he has helped products, developed and used by Fortune 500, scale better. A presenter at international testing conferences, Prashant can be reached at prashant.suri@rackspace.com.
  • 3. Build your custom Performance Testing Framework Prashant Suri RACKSPACE® HOSTING | WWW.RACKSPACE.COM Agenda 2 • Evolution of the web and growing complexity of products. • Essentials of performance engineering. • Popular performance test tools. • Holistic performance test framework. • Components of a performance test framework. • Deep dive on an example: performance testing at RAX • Advantages of building your own framework.
  • 4. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Evolution of the Web 3 RACKSPACE® HOSTING | WWW.RACKSPACE.COM Essentials of Performance Engineering • Performance objectives – Latency, Throughput, Utilization, Workload. • Performance Modeling – Scenarios, Objectives, Requirements, Budgets, Metrics. • Architecture and Design Guidelines – Principles, Practices and Patterns • Performance and Scalability Frame – Coupling and cohesion, Communication, Concurrency, Resource Management, Caching, State Management, Data Structures/Algorithms. • Measurement, Tuning and Analysis. 4
  • 5. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Popular Performance Test Tools 5 RACKSPACE® HOSTING | WWW.RACKSPACE.COM There’s No Silver Bullet 6 • Performance testing is more than simulating load. • Accurate and intricate work load models need to be implemented. • Traffic to be simulated from world geographies. • Testing in service oriented architectures comes with its own challenges. • Reporting and pattern analysis is key for successful performance engineering. • Performance monitors and profiling equally important.
  • 6. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Holistic Performance Test Framework 7 RACKSPACE® HOSTING | WWW.RACKSPACE.COM Components of the framework 8 Controller Agents Database & Graphing Engine SimulatingLoads •  Brain behind the complete orchestration. Parses and executes the data script. •  Picks up threads from queues, and hit application under test. •  Repository for metrics of response times, throughputs, errors etc. •  Can scale dynamically at run time, to add load generation capacity. •  Maintains the test details, workload model details and objectives of any performance test. Capable of distributing load from different data centers. •  Distributed across geographies to simulate identical world traffic. •  Optimal database tuning for handling 1000+ transactions per second test. Monitoringand Debugging Could be integrated to application monitors and profilers. •  Feeds all metrics data into the Database. •  Correlates and consolidates all test data. Updated information on runtime activity from database is read and published. •  Logging for every granular transaction with application under test. •  Historical comparisons and pattern analysis. Could be leveraged for analysis and data based decision making. •  Distributed agents through way of logging are consolidated for complete test. •  Profiling and application under test data, for analyzing deviation and anomalies.
  • 7. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Adobe CQMicrosoft® SharePointMicrosoft® ExchangeRackspace Email Case Study – Capabilities galore! Single F/W to simulate on all Products RACKSPACE® HOSTING | WWW.RACKSPACE.COM Sample Script – Design workload models! 10
  • 8. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 11 Execution – Distribute across Agents through a queue RACKSPACE® HOSTING | WWW.RACKSPACE.COM 12 Run time statistics of Performance Test
  • 9. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Consolidated Reports 13 RACKSPACE® HOSTING | WWW.RACKSPACE.COM Consolidated Reports 14
  • 10. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Advantages of Building your own frameworks 15 • Cost savings on commercial off-the-shelf tools. • Flexibility to simulate complex workload at any scale (literally!) • Simplicity in designing complex workload models. • No constraints and capability of leveraging as per product needs. • Ease of integration with CI/CD Pipeline. • Load simulator and application resource monitors in a single framework setup. • Efficient and inclusive engagement for Performance Engineering in overall product development lifecycle. RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM Questions…
  • 11. RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM Prashant.suri@rackspace.com