SlideShare a Scribd company logo
Performance Testing 
@ 
With 100,000 concurrent users
WHO ARE WE 
• AWS for most client projects and our own 
• Enterprise/Solution Architecture + Development 
• Performance Engineering, Profiling, 
Optimization 
• "DevOps" 
• AWS for most client projects and our own
WHAT WE WANTED 
Performance test applications with 
100,000 concurrent users 
(Websocket + HTTP) 
• Easy to use 
• Traceable history of tests 
• Good fit with modern DevOps processes 
• Low maintenance and cost
99 Problems 
Quickly max out a single test node 
(or even 10 of them) 
Performance tests should be available to whole 
team 
(not just the guru) 
Shouldn't have to write another App 
(or pay A LOT for one)
REQUIREMNTS
SOLUTION 
(one of many)
STEP 1 
Build App 
(Compile, Test, 
Package)
STEP 2 
Build your environment (Puppet, Chef, etc) 
Deploy new application version
STEP 3 
Create load test slave instances
STEP 4 
Start running the test plan against new 
environment
STEP 5 
• Test slaves return results to Jenkins. 
• Then clean up 
• Shut down instances 
• Snapshot new AMI
PIPELINE 
Build app -> Create test env + Deploy -> Test new 
environment
LESSONS LEARNED 1/2 
• Performance problems found and fixed 
Garbage Collection, Serialization and encoding 
• Very helpful when sizing infrastructure 
Webservice tier network was bottleneck 
Different instance sizes picked 
• Scaling the test clients can be hard
LESSONS LEARNED 2/2 
• Performance testing via ELB 
• Ramp up slowly 
• Make sure client see DNS changes 
• ELB's not great for Websocket 
• Linux tuning 
• Ephemeral port range 
• Buffers everywhere 
• File handles, flood protection, etc 
• It does a pretty good job by default
Results & Deep Insight 
Deep Insight with dynaTrace APM beyond simple JMeter results 
Clearly understand the difference between: 
• Load generation 
• Real User Monitoring 
• Always-on deep transaction tracing
TEST AUTOMATION / CI 
• Unit Tests are fully integrated as part of CI 
• Each build also measures performance for each testcase
BONUSES 
Easily re-used for different types of build/testing 
• BYO load test tool 
• Spot instances are cheaper 
• Not just for load testing 
• Test Chef/Puppet scripts 
• OS updates 
• Browser based (Webdriver, PhantomJS, etc) 
• Auto-scale testing 
• Snapshot AMI after successful tests
Questions ? 
Thanks ! 
Twitter: msquareau 
http://m-square.com.au

More Related Content

What's hot

Neoload
Neoload Neoload
Neoload
Kumar Gupta
 
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
Postman
 
An Introduction To Automated API Testing
An Introduction To Automated API TestingAn Introduction To Automated API Testing
An Introduction To Automated API Testing
Sauce Labs
 
API Security Lifecycle
API Security LifecycleAPI Security Lifecycle
API Security Lifecycle
Apigee | Google Cloud
 
Testing microservices with rest assured
Testing microservices with rest assuredTesting microservices with rest assured
Testing microservices with rest assured
Kushan Shalindra Amarasiri - Technical QE Specialist
 
Cypress - Best Practices
Cypress - Best PracticesCypress - Best Practices
Cypress - Best Practices
Brian Mann
 
Cross browser testing using BrowserStack
Cross browser testing using BrowserStack Cross browser testing using BrowserStack
Cross browser testing using BrowserStack
RapidValue
 
Top 50 Performance Testing Interview Questions | Edureka
Top 50 Performance Testing Interview Questions | EdurekaTop 50 Performance Testing Interview Questions | Edureka
Top 50 Performance Testing Interview Questions | Edureka
Edureka!
 
API Testing: The heart of functional testing" with Bj Rollison
API Testing: The heart of functional testing" with Bj RollisonAPI Testing: The heart of functional testing" with Bj Rollison
API Testing: The heart of functional testing" with Bj Rollison
TEST Huddle
 
Introduction to performance testing
Introduction to performance testingIntroduction to performance testing
Introduction to performance testing
Tharinda Liyanage
 
Postman.ppt
Postman.pptPostman.ppt
Postman.ppt
ParrotBAD
 
How to Automate API Testing
How to Automate API TestingHow to Automate API Testing
How to Automate API Testing
Bruno Pedro
 
Postman & API Testing by Amber Race
Postman & API Testing by Amber RacePostman & API Testing by Amber Race
Postman & API Testing by Amber Race
Postman
 
Data Driven Testing
Data Driven TestingData Driven Testing
Data Driven TestingMaveryx
 
REST API testing with SpecFlow
REST API testing with SpecFlowREST API testing with SpecFlow
REST API testing with SpecFlow
Aiste Stikliute
 
Sonarqube
SonarqubeSonarqube
Performance Testing Using JMeter | Edureka
Performance Testing Using JMeter | EdurekaPerformance Testing Using JMeter | Edureka
Performance Testing Using JMeter | Edureka
Edureka!
 
Robot Framework Dos And Don'ts
Robot Framework Dos And Don'tsRobot Framework Dos And Don'ts
Robot Framework Dos And Don'ts
Pekka Klärck
 
Testing Microservices
Testing MicroservicesTesting Microservices
Testing Microservices
Nagarro
 
Getting Started With Cypress
Getting Started With CypressGetting Started With Cypress
Getting Started With Cypress
Knoldus Inc.
 

What's hot (20)

Neoload
Neoload Neoload
Neoload
 
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
 
An Introduction To Automated API Testing
An Introduction To Automated API TestingAn Introduction To Automated API Testing
An Introduction To Automated API Testing
 
API Security Lifecycle
API Security LifecycleAPI Security Lifecycle
API Security Lifecycle
 
Testing microservices with rest assured
Testing microservices with rest assuredTesting microservices with rest assured
Testing microservices with rest assured
 
Cypress - Best Practices
Cypress - Best PracticesCypress - Best Practices
Cypress - Best Practices
 
Cross browser testing using BrowserStack
Cross browser testing using BrowserStack Cross browser testing using BrowserStack
Cross browser testing using BrowserStack
 
Top 50 Performance Testing Interview Questions | Edureka
Top 50 Performance Testing Interview Questions | EdurekaTop 50 Performance Testing Interview Questions | Edureka
Top 50 Performance Testing Interview Questions | Edureka
 
API Testing: The heart of functional testing" with Bj Rollison
API Testing: The heart of functional testing" with Bj RollisonAPI Testing: The heart of functional testing" with Bj Rollison
API Testing: The heart of functional testing" with Bj Rollison
 
Introduction to performance testing
Introduction to performance testingIntroduction to performance testing
Introduction to performance testing
 
Postman.ppt
Postman.pptPostman.ppt
Postman.ppt
 
How to Automate API Testing
How to Automate API TestingHow to Automate API Testing
How to Automate API Testing
 
Postman & API Testing by Amber Race
Postman & API Testing by Amber RacePostman & API Testing by Amber Race
Postman & API Testing by Amber Race
 
Data Driven Testing
Data Driven TestingData Driven Testing
Data Driven Testing
 
REST API testing with SpecFlow
REST API testing with SpecFlowREST API testing with SpecFlow
REST API testing with SpecFlow
 
Sonarqube
SonarqubeSonarqube
Sonarqube
 
Performance Testing Using JMeter | Edureka
Performance Testing Using JMeter | EdurekaPerformance Testing Using JMeter | Edureka
Performance Testing Using JMeter | Edureka
 
Robot Framework Dos And Don'ts
Robot Framework Dos And Don'tsRobot Framework Dos And Don'ts
Robot Framework Dos And Don'ts
 
Testing Microservices
Testing MicroservicesTesting Microservices
Testing Microservices
 
Getting Started With Cypress
Getting Started With CypressGetting Started With Cypress
Getting Started With Cypress
 

Viewers also liked

Large Scale Load Testing Amazon.com’s Traffic on AWS (CPN102) | AWS re:Invent...
Large Scale Load Testing Amazon.com’s Traffic on AWS (CPN102) | AWS re:Invent...Large Scale Load Testing Amazon.com’s Traffic on AWS (CPN102) | AWS re:Invent...
Large Scale Load Testing Amazon.com’s Traffic on AWS (CPN102) | AWS re:Invent...
Amazon Web Services
 
Load Test Drupal Site Using JMeter and Amazon AWS
Load Test Drupal Site Using JMeter and Amazon AWSLoad Test Drupal Site Using JMeter and Amazon AWS
Load Test Drupal Site Using JMeter and Amazon AWS
Vladimir Ilic
 
JMeter, Docker sitting in a tree
JMeter, Docker sitting in a treeJMeter, Docker sitting in a tree
JMeter, Docker sitting in a tree
srivaths_sankaran
 
Introduction to Performance testing
Introduction to Performance testingIntroduction to Performance testing
Introduction to Performance testing
silviasiqueirahp
 
An Introduction to Performance Testing
An Introduction to Performance TestingAn Introduction to Performance Testing
An Introduction to Performance Testing
SWAAM Tech
 
Application Insights - Real time telemetry for your cross platform applications
Application Insights - Real time telemetry for your cross platform applicationsApplication Insights - Real time telemetry for your cross platform applications
Application Insights - Real time telemetry for your cross platform applications
Abhijit Jana
 
Cloud security
Cloud securityCloud security
Cloud security
François Boucher
 
Running JMeter Tests In Appvance PerformanceCloud
Running JMeter Tests In Appvance PerformanceCloudRunning JMeter Tests In Appvance PerformanceCloud
Running JMeter Tests In Appvance PerformanceCloud
Clever Moe
 
Scalable load testing using jmeter in cloud
Scalable load testing using jmeter in cloudScalable load testing using jmeter in cloud
Scalable load testing using jmeter in cloud
Vijay Rayapati
 
Application Insights - Campinas .NET - Março/2017
Application Insights - Campinas .NET - Março/2017Application Insights - Campinas .NET - Março/2017
Application Insights - Campinas .NET - Março/2017
Renato Groff
 
Load testing with Visual Studio and Azure - Andrew Siemer
Load testing with Visual Studio and Azure - Andrew SiemerLoad testing with Visual Studio and Azure - Andrew Siemer
Load testing with Visual Studio and Azure - Andrew Siemer
Andrew Siemer
 
Queues queues queues — How RabbitMQ enables reactive architectures
Queues queues queues — How RabbitMQ enables reactive architecturesQueues queues queues — How RabbitMQ enables reactive architectures
Queues queues queues — How RabbitMQ enables reactive architectures
Martin Tajur
 
Cloud-based Performance & Web Load Testing
Cloud-based Performance & Web Load TestingCloud-based Performance & Web Load Testing
Cloud-based Performance & Web Load Testing
InCycleSoftware
 
DevOps@Work 2017 - Application insights more control, more power
DevOps@Work 2017 - Application insights more control, more powerDevOps@Work 2017 - Application insights more control, more power
DevOps@Work 2017 - Application insights more control, more power
Roberto Albano
 
Java Performance Engineer's Survival Guide
Java Performance Engineer's Survival GuideJava Performance Engineer's Survival Guide
Java Performance Engineer's Survival Guide
Monica Beckwith
 
Deep-Dive to Application Insights
Deep-Dive to Application Insights Deep-Dive to Application Insights
Deep-Dive to Application Insights
Gunnar Peipman
 
How to Run a 1,000,000 VU Load Test using Apache JMeter and BlazeMeter
How to Run a 1,000,000 VU Load Test using Apache JMeter and BlazeMeterHow to Run a 1,000,000 VU Load Test using Apache JMeter and BlazeMeter
How to Run a 1,000,000 VU Load Test using Apache JMeter and BlazeMeter
Alon Girmonsky
 
Apache JMeter - A brief introduction
Apache JMeter - A brief introductionApache JMeter - A brief introduction
Apache JMeter - A brief introduction
silenceIT Inc.
 
Client-side Performance Testing
Client-side Performance TestingClient-side Performance Testing
Client-side Performance Testing
Thoughtworks
 
Performance testing jmeter
Performance testing jmeterPerformance testing jmeter
Performance testing jmeterBhojan Rajan
 

Viewers also liked (20)

Large Scale Load Testing Amazon.com’s Traffic on AWS (CPN102) | AWS re:Invent...
Large Scale Load Testing Amazon.com’s Traffic on AWS (CPN102) | AWS re:Invent...Large Scale Load Testing Amazon.com’s Traffic on AWS (CPN102) | AWS re:Invent...
Large Scale Load Testing Amazon.com’s Traffic on AWS (CPN102) | AWS re:Invent...
 
Load Test Drupal Site Using JMeter and Amazon AWS
Load Test Drupal Site Using JMeter and Amazon AWSLoad Test Drupal Site Using JMeter and Amazon AWS
Load Test Drupal Site Using JMeter and Amazon AWS
 
JMeter, Docker sitting in a tree
JMeter, Docker sitting in a treeJMeter, Docker sitting in a tree
JMeter, Docker sitting in a tree
 
Introduction to Performance testing
Introduction to Performance testingIntroduction to Performance testing
Introduction to Performance testing
 
An Introduction to Performance Testing
An Introduction to Performance TestingAn Introduction to Performance Testing
An Introduction to Performance Testing
 
Application Insights - Real time telemetry for your cross platform applications
Application Insights - Real time telemetry for your cross platform applicationsApplication Insights - Real time telemetry for your cross platform applications
Application Insights - Real time telemetry for your cross platform applications
 
Cloud security
Cloud securityCloud security
Cloud security
 
Running JMeter Tests In Appvance PerformanceCloud
Running JMeter Tests In Appvance PerformanceCloudRunning JMeter Tests In Appvance PerformanceCloud
Running JMeter Tests In Appvance PerformanceCloud
 
Scalable load testing using jmeter in cloud
Scalable load testing using jmeter in cloudScalable load testing using jmeter in cloud
Scalable load testing using jmeter in cloud
 
Application Insights - Campinas .NET - Março/2017
Application Insights - Campinas .NET - Março/2017Application Insights - Campinas .NET - Março/2017
Application Insights - Campinas .NET - Março/2017
 
Load testing with Visual Studio and Azure - Andrew Siemer
Load testing with Visual Studio and Azure - Andrew SiemerLoad testing with Visual Studio and Azure - Andrew Siemer
Load testing with Visual Studio and Azure - Andrew Siemer
 
Queues queues queues — How RabbitMQ enables reactive architectures
Queues queues queues — How RabbitMQ enables reactive architecturesQueues queues queues — How RabbitMQ enables reactive architectures
Queues queues queues — How RabbitMQ enables reactive architectures
 
Cloud-based Performance & Web Load Testing
Cloud-based Performance & Web Load TestingCloud-based Performance & Web Load Testing
Cloud-based Performance & Web Load Testing
 
DevOps@Work 2017 - Application insights more control, more power
DevOps@Work 2017 - Application insights more control, more powerDevOps@Work 2017 - Application insights more control, more power
DevOps@Work 2017 - Application insights more control, more power
 
Java Performance Engineer's Survival Guide
Java Performance Engineer's Survival GuideJava Performance Engineer's Survival Guide
Java Performance Engineer's Survival Guide
 
Deep-Dive to Application Insights
Deep-Dive to Application Insights Deep-Dive to Application Insights
Deep-Dive to Application Insights
 
How to Run a 1,000,000 VU Load Test using Apache JMeter and BlazeMeter
How to Run a 1,000,000 VU Load Test using Apache JMeter and BlazeMeterHow to Run a 1,000,000 VU Load Test using Apache JMeter and BlazeMeter
How to Run a 1,000,000 VU Load Test using Apache JMeter and BlazeMeter
 
Apache JMeter - A brief introduction
Apache JMeter - A brief introductionApache JMeter - A brief introduction
Apache JMeter - A brief introduction
 
Client-side Performance Testing
Client-side Performance TestingClient-side Performance Testing
Client-side Performance Testing
 
Performance testing jmeter
Performance testing jmeterPerformance testing jmeter
Performance testing jmeter
 

Similar to Performance testing with 100,000 concurrent users in AWS

Dev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps worldDev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps world
Davide Benvegnù
 
AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)
AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)
AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)
Amazon Web Services
 
Telerik test studio webinar deck
Telerik  test studio webinar deckTelerik  test studio webinar deck
Telerik test studio webinar deck
Dhananjay Kumar
 
Extreme Makeover OnBase Edition
Extreme Makeover OnBase EditionExtreme Makeover OnBase Edition
Extreme Makeover OnBase Edition
DataBank, A KYOCERA Group Company
 
Integration Testing as Validation and Monitoring
 Integration Testing as Validation and Monitoring Integration Testing as Validation and Monitoring
Integration Testing as Validation and Monitoring
Melissa Benua
 
MyHeritage - QA Automations in a Continuous Deployment environment
MyHeritage -  QA Automations in a Continuous Deployment environmentMyHeritage -  QA Automations in a Continuous Deployment environment
MyHeritage - QA Automations in a Continuous Deployment environment
MatanGoren
 
Building Efficient Parallel Testing Platforms with Docker
Building Efficient Parallel Testing Platforms with DockerBuilding Efficient Parallel Testing Platforms with Docker
Building Efficient Parallel Testing Platforms with Docker
Laura Frank Tacho
 
AEM OpenCloud Delivery Practices
AEM OpenCloud Delivery PracticesAEM OpenCloud Delivery Practices
AEM OpenCloud Delivery Practices
Cliffano Subagio
 
Improving Batch-Process Testing Techniques with a Domain-Specific Language
Improving Batch-Process Testing Techniques with a Domain-Specific LanguageImproving Batch-Process Testing Techniques with a Domain-Specific Language
Improving Batch-Process Testing Techniques with a Domain-Specific Language
Dr. Spock
 
Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and S...
 Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and S... Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and S...
Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and S...
Skytap Cloud
 
Dev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsDev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teams
Mahesh Arali
 
Getting Started with Serverless Architectures
Getting Started with Serverless ArchitecturesGetting Started with Serverless Architectures
Getting Started with Serverless Architectures
Amazon Web Services
 
Application Delivery Patterns for Developers - Technical 401
Application Delivery Patterns for Developers - Technical 401Application Delivery Patterns for Developers - Technical 401
Application Delivery Patterns for Developers - Technical 401
Amazon Web Services
 
Linux Collaboration Summit Keynote: Transformation: It Takes a Platform
Linux Collaboration Summit Keynote: Transformation: It Takes a PlatformLinux Collaboration Summit Keynote: Transformation: It Takes a Platform
Linux Collaboration Summit Keynote: Transformation: It Takes a Platform
cornelia davis
 
(DVO201) Scaling Your Web Applications with AWS Elastic Beanstalk
(DVO201) Scaling Your Web Applications with AWS Elastic Beanstalk(DVO201) Scaling Your Web Applications with AWS Elastic Beanstalk
(DVO201) Scaling Your Web Applications with AWS Elastic Beanstalk
Amazon Web Services
 
Continuous Integration - Mobile Practice
Continuous Integration - Mobile PracticeContinuous Integration - Mobile Practice
Continuous Integration - Mobile Practice
HARMAN Services
 
SynapseIndia drupal presentation on drupal info
SynapseIndia drupal  presentation on drupal infoSynapseIndia drupal  presentation on drupal info
SynapseIndia drupal presentation on drupal info
Synapseindiappsdevelopment
 
Configuration Management in the Cloud | AWS Public Sector Summit 2017
Configuration Management in the Cloud | AWS Public Sector Summit 2017Configuration Management in the Cloud | AWS Public Sector Summit 2017
Configuration Management in the Cloud | AWS Public Sector Summit 2017
Amazon Web Services
 
Infrastructure as Code
Infrastructure as CodeInfrastructure as Code
Infrastructure as Code
Robert Greiner
 

Similar to Performance testing with 100,000 concurrent users in AWS (20)

Dev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps worldDev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps world
 
AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)
AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)
AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)
 
Telerik test studio webinar deck
Telerik  test studio webinar deckTelerik  test studio webinar deck
Telerik test studio webinar deck
 
Extreme Makeover OnBase Edition
Extreme Makeover OnBase EditionExtreme Makeover OnBase Edition
Extreme Makeover OnBase Edition
 
Integration Testing as Validation and Monitoring
 Integration Testing as Validation and Monitoring Integration Testing as Validation and Monitoring
Integration Testing as Validation and Monitoring
 
MyHeritage - QA Automations in a Continuous Deployment environment
MyHeritage -  QA Automations in a Continuous Deployment environmentMyHeritage -  QA Automations in a Continuous Deployment environment
MyHeritage - QA Automations in a Continuous Deployment environment
 
Building Efficient Parallel Testing Platforms with Docker
Building Efficient Parallel Testing Platforms with DockerBuilding Efficient Parallel Testing Platforms with Docker
Building Efficient Parallel Testing Platforms with Docker
 
AEM OpenCloud Delivery Practices
AEM OpenCloud Delivery PracticesAEM OpenCloud Delivery Practices
AEM OpenCloud Delivery Practices
 
Improving Batch-Process Testing Techniques with a Domain-Specific Language
Improving Batch-Process Testing Techniques with a Domain-Specific LanguageImproving Batch-Process Testing Techniques with a Domain-Specific Language
Improving Batch-Process Testing Techniques with a Domain-Specific Language
 
Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and S...
 Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and S... Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and S...
Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and S...
 
Dev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsDev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teams
 
Getting Started with Serverless Architectures
Getting Started with Serverless ArchitecturesGetting Started with Serverless Architectures
Getting Started with Serverless Architectures
 
Application Delivery Patterns for Developers - Technical 401
Application Delivery Patterns for Developers - Technical 401Application Delivery Patterns for Developers - Technical 401
Application Delivery Patterns for Developers - Technical 401
 
Linux Collaboration Summit Keynote: Transformation: It Takes a Platform
Linux Collaboration Summit Keynote: Transformation: It Takes a PlatformLinux Collaboration Summit Keynote: Transformation: It Takes a Platform
Linux Collaboration Summit Keynote: Transformation: It Takes a Platform
 
QAorHighway2016
QAorHighway2016QAorHighway2016
QAorHighway2016
 
(DVO201) Scaling Your Web Applications with AWS Elastic Beanstalk
(DVO201) Scaling Your Web Applications with AWS Elastic Beanstalk(DVO201) Scaling Your Web Applications with AWS Elastic Beanstalk
(DVO201) Scaling Your Web Applications with AWS Elastic Beanstalk
 
Continuous Integration - Mobile Practice
Continuous Integration - Mobile PracticeContinuous Integration - Mobile Practice
Continuous Integration - Mobile Practice
 
SynapseIndia drupal presentation on drupal info
SynapseIndia drupal  presentation on drupal infoSynapseIndia drupal  presentation on drupal info
SynapseIndia drupal presentation on drupal info
 
Configuration Management in the Cloud | AWS Public Sector Summit 2017
Configuration Management in the Cloud | AWS Public Sector Summit 2017Configuration Management in the Cloud | AWS Public Sector Summit 2017
Configuration Management in the Cloud | AWS Public Sector Summit 2017
 
Infrastructure as Code
Infrastructure as CodeInfrastructure as Code
Infrastructure as Code
 

Recently uploaded

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 

Recently uploaded (20)

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 

Performance testing with 100,000 concurrent users in AWS

  • 1. Performance Testing @ With 100,000 concurrent users
  • 2. WHO ARE WE • AWS for most client projects and our own • Enterprise/Solution Architecture + Development • Performance Engineering, Profiling, Optimization • "DevOps" • AWS for most client projects and our own
  • 3. WHAT WE WANTED Performance test applications with 100,000 concurrent users (Websocket + HTTP) • Easy to use • Traceable history of tests • Good fit with modern DevOps processes • Low maintenance and cost
  • 4. 99 Problems Quickly max out a single test node (or even 10 of them) Performance tests should be available to whole team (not just the guru) Shouldn't have to write another App (or pay A LOT for one)
  • 7. STEP 1 Build App (Compile, Test, Package)
  • 8. STEP 2 Build your environment (Puppet, Chef, etc) Deploy new application version
  • 9. STEP 3 Create load test slave instances
  • 10. STEP 4 Start running the test plan against new environment
  • 11. STEP 5 • Test slaves return results to Jenkins. • Then clean up • Shut down instances • Snapshot new AMI
  • 12. PIPELINE Build app -> Create test env + Deploy -> Test new environment
  • 13. LESSONS LEARNED 1/2 • Performance problems found and fixed Garbage Collection, Serialization and encoding • Very helpful when sizing infrastructure Webservice tier network was bottleneck Different instance sizes picked • Scaling the test clients can be hard
  • 14. LESSONS LEARNED 2/2 • Performance testing via ELB • Ramp up slowly • Make sure client see DNS changes • ELB's not great for Websocket • Linux tuning • Ephemeral port range • Buffers everywhere • File handles, flood protection, etc • It does a pretty good job by default
  • 15. Results & Deep Insight Deep Insight with dynaTrace APM beyond simple JMeter results Clearly understand the difference between: • Load generation • Real User Monitoring • Always-on deep transaction tracing
  • 16. TEST AUTOMATION / CI • Unit Tests are fully integrated as part of CI • Each build also measures performance for each testcase
  • 17. BONUSES Easily re-used for different types of build/testing • BYO load test tool • Spot instances are cheaper • Not just for load testing • Test Chef/Puppet scripts • OS updates • Browser based (Webdriver, PhantomJS, etc) • Auto-scale testing • Snapshot AMI after successful tests
  • 18. Questions ? Thanks ! Twitter: msquareau http://m-square.com.au

Editor's Notes

  1. Kick off the Pipeline builds
  2. ELB
  3. Load Generation vers. Deep Transaction Tracing