The document discusses different approaches to load testing, including load generation techniques like record and playback, real users, programming, and mixed approaches. It also covers load testing environments such as lab vs cloud vs production environments. Finally, it provides an overview of various load testing tools and how their suitability depends on factors like the technologies being tested and testing needs. The key message is that load testing is an important part of performance risk mitigation but requires choosing the right approach and tools based on the specific testing situation.
Load testing is an important part of the performance engineering process. It remains the main way to ensure appropriate performance and reliability in production. It is important to see a bigger picture beyond stereotypical, last-moment load testing. There are multiple dimensions of load testing: environment, load generation, testing approach, life-cycle integration, feedback and analysis. This paper discusses these dimensions and how load testing tools support them.
Load testing is an important part of the performance engineering process. However the industry is changing and load testing should adjust to these changes - a stereotypical, last-moment performance check is not enough anymore. There are multiple aspects of load testing - such as environment, load generation, testing approach, life-cycle integration, feedback and analysis - and none remains static. This presentation discusses how performance testing is adapting to industry trends to remain relevant and bring value to the table.
Continuous Performance Testing: Myths and RealitiesAlexander Podelko
While development process is moving towards all things continuous, performance testing remains rather a gray area. Some continue to do it in the traditional pre-release fashion, some claim 100% automation and full integration into their continuous process. We have a full spectrum of opinions of what, when, and how should be done in regard to performance. The issue here is that context is usually not clearly specified - while context is the main factor here. Depending on context, the approach may (and probably should) be completely different. Full success in a simple (from the performance testing point of view) environment doesn't mean that you may easily replicate it in a difficult environment. The speaker will discuss the issues of making performance testing continuous in detail, illustrating them with personal experience when possible.
Load Testing Best Practices: Application complexity is increasing, yet the stringent requirements for web performance is increasing exponentially. Learn more about the three major types of load testing, determine which you need and how to conduct them.
The PAC aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing.
Since its beginning, the PAC is designed to connect performance experts during a single event. In June, during 24 hours, 20 participants convened exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.
Load testing is an important part of the performance engineering process. It remains the main way to ensure appropriate performance and reliability in production. It is important to see a bigger picture beyond stereotypical, last-moment load testing. There are multiple dimensions of load testing: environment, load generation, testing approach, life-cycle integration, feedback and analysis. This paper discusses these dimensions and how load testing tools support them.
Load testing is an important part of the performance engineering process. However the industry is changing and load testing should adjust to these changes - a stereotypical, last-moment performance check is not enough anymore. There are multiple aspects of load testing - such as environment, load generation, testing approach, life-cycle integration, feedback and analysis - and none remains static. This presentation discusses how performance testing is adapting to industry trends to remain relevant and bring value to the table.
Continuous Performance Testing: Myths and RealitiesAlexander Podelko
While development process is moving towards all things continuous, performance testing remains rather a gray area. Some continue to do it in the traditional pre-release fashion, some claim 100% automation and full integration into their continuous process. We have a full spectrum of opinions of what, when, and how should be done in regard to performance. The issue here is that context is usually not clearly specified - while context is the main factor here. Depending on context, the approach may (and probably should) be completely different. Full success in a simple (from the performance testing point of view) environment doesn't mean that you may easily replicate it in a difficult environment. The speaker will discuss the issues of making performance testing continuous in detail, illustrating them with personal experience when possible.
Load Testing Best Practices: Application complexity is increasing, yet the stringent requirements for web performance is increasing exponentially. Learn more about the three major types of load testing, determine which you need and how to conduct them.
The PAC aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing.
Since its beginning, the PAC is designed to connect performance experts during a single event. In June, during 24 hours, 20 participants convened exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.
The PAC aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing.
Since its beginning, the PAC is designed to connect performance experts during a single event. In June, during 24 hours, 20 participants convened exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.
Are you new to performance testing? This slides are for those of you who want to explore and learn where and how to start testing application performance. During this web event, our performance testing experts will reveal the key pieces and parts of performance testing, including the phases of the test and how HP LoadRunner supports each phase.
Performance issues are regularly caught too late leading to increased cost for fixing. We propose a process on how to make performance testing lightweight, execute it on early stages, reduce time and cost of fixes. Applying the same principal that we use for functional testing, performance testing could be integrated in CI/CD pipeline. Learn more about CPT in our blog: https://blog.griddynamics.com/what-is-continuous-performance-testing-and-why-it-is-needed
The most important aspect to release any product or application in the market is to deliver a satisfying user experience. And this can only be achieved when the application performs impeccably. To help understand the ways and means to ensure the same, this PPT will shed light on the essential elements under performance testing. To know more on software performance testing, performance testing, app performance testing, web performance testing, website load testing and performance tuning, go through this presentation and gear up for the upcoming ones.
it is based on the introduction of performance testing and its modules and the process which include performance testing and it consist of name of software or tools which are used to check the performance of any software.
An Introduction to Software Performance EngineeringCorrelsense
Software performance engineering is becoming increasingly important to businesses as they look to improve the non-functional performance of applications and get more out of IT investments. By leveraging performance engineering techniques, IT professionals can be indispensable in building and optimizing scalable systems. This
introductory course will teach you the essentials of software
performance engineering including :
• The performance challenges faced by Enterprise IT today
• What is software performance engineering (SPE)?
• Best practices for building scalable software systems
• The approaches to integrating SPE into IT project lifecycles
• Common frameworks for measuring application performance and service levels
• The impact of SPE on software developers, testers, capacity planes,
and other IT professionals
• Case studies from the finance, retail, and insurance industries
Instructor: Walter Kuketz, SVP and CTO, Collaborative Consulting
This training is sponsored by Correlsense, Collaborative Consulting,
and New Horizons
In many web or cloud applications, performance testing is critical part of application testing since it affects
business revenue, credibility, and customer satisfaction. Conventional software development models are known
to pushing the performance testing to the very end of project, with the expectations that, only minor tweaks
and tune up are required to meet the performance requirements from the business, however any major
performance bottlenecks found during this phase were major factors for delay in Go to Market. With more and
more companies are adapting the agile software development process which believes in performance testing
should never be an afterthought but it should tightly integrate from initial planning to production analysis of
software development lifecycle. This white paper explains how any company can integrate performance testing
into agile process, and key barriers for agile performance testing when team decides to adopt agile performance
testing.
The PAC aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing.
Since its beginning, the PAC is designed to connect performance experts during a single event. In June, during 24 hours, 20 participants convened exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.
Are you new to performance testing? This slides are for those of you who want to explore and learn where and how to start testing application performance. During this web event, our performance testing experts will reveal the key pieces and parts of performance testing, including the phases of the test and how HP LoadRunner supports each phase.
Performance issues are regularly caught too late leading to increased cost for fixing. We propose a process on how to make performance testing lightweight, execute it on early stages, reduce time and cost of fixes. Applying the same principal that we use for functional testing, performance testing could be integrated in CI/CD pipeline. Learn more about CPT in our blog: https://blog.griddynamics.com/what-is-continuous-performance-testing-and-why-it-is-needed
The most important aspect to release any product or application in the market is to deliver a satisfying user experience. And this can only be achieved when the application performs impeccably. To help understand the ways and means to ensure the same, this PPT will shed light on the essential elements under performance testing. To know more on software performance testing, performance testing, app performance testing, web performance testing, website load testing and performance tuning, go through this presentation and gear up for the upcoming ones.
it is based on the introduction of performance testing and its modules and the process which include performance testing and it consist of name of software or tools which are used to check the performance of any software.
An Introduction to Software Performance EngineeringCorrelsense
Software performance engineering is becoming increasingly important to businesses as they look to improve the non-functional performance of applications and get more out of IT investments. By leveraging performance engineering techniques, IT professionals can be indispensable in building and optimizing scalable systems. This
introductory course will teach you the essentials of software
performance engineering including :
• The performance challenges faced by Enterprise IT today
• What is software performance engineering (SPE)?
• Best practices for building scalable software systems
• The approaches to integrating SPE into IT project lifecycles
• Common frameworks for measuring application performance and service levels
• The impact of SPE on software developers, testers, capacity planes,
and other IT professionals
• Case studies from the finance, retail, and insurance industries
Instructor: Walter Kuketz, SVP and CTO, Collaborative Consulting
This training is sponsored by Correlsense, Collaborative Consulting,
and New Horizons
In many web or cloud applications, performance testing is critical part of application testing since it affects
business revenue, credibility, and customer satisfaction. Conventional software development models are known
to pushing the performance testing to the very end of project, with the expectations that, only minor tweaks
and tune up are required to meet the performance requirements from the business, however any major
performance bottlenecks found during this phase were major factors for delay in Go to Market. With more and
more companies are adapting the agile software development process which believes in performance testing
should never be an afterthought but it should tightly integrate from initial planning to production analysis of
software development lifecycle. This white paper explains how any company can integrate performance testing
into agile process, and key barriers for agile performance testing when team decides to adopt agile performance
testing.
Load testing is an important part of the performance engineering process. It remains the main way to ensure appropriate performance and reliability in production. Still it is important to see a bigger picture beyond stereotypical last-moment load testing. There are different ways to create load; a single approach may not work in all situations. Many tools allow you to use different ways of recording/playback and programming. This session discusses pros and cons of each approach, when it can be used and what tool's features we need to support it.
Performance Requirements: the Backbone of the Performance Engineering ProcessAlexander Podelko
Performance requirements should to be tracked from system's inception through its whole lifecycle including design, development, testing, operations, and maintenance. They are the backbone of the performance engineering process. However different groups of people are involved in each stage and they use their own vision, terminology, metrics, and tools that makes the subject confusing when you go into details. The presentation discusses existing issues and approaches in their relationship with the performance engineering process.
Performance is a critical factor for success of any packaged application implementations. The presentation discusses performance assurance for packaged applications on example of Oracle Enterprise Performance Management. While details are related to this particular set of applications, many approaches discussed would be applicable to most packaged applications. The presentation will discuss a holistic performance assurance approach, top-down approach to performance troubleshooting, potential performance issues and ways to address them.
Performance Testing in Context; From Simple to Rocket ScienceScott Barber
When most people think of performance testing, they think about the hard parts – the very hard parts. They think about the expensive and complicated tools that are required to simulate the activity of thousands of end-users all at the same time, while collecting tens or hundreds of thousands of measurements.
In reality, many performance issues can be detected and diagnosed with exactly the tools and knowledge you already have at your disposal using information obtained from quick, easy and cheap performance tests. In fact, much of the performance related information that stakeholders need to make good decisions and development teams need to dramatically improve system performance is easily obtainable by the performance-testing layman. The trick is knowing what performance tests to apply when, and how much time/effort is worth investing based on the business importance of performance — in other words, context!
In this hands-on tutorial (bring your laptop or risk reduced value and intermittent boredom), Scott Barber will introduce you to several techniques that the performance testing layperson can use to speed up and simplify the collection of valuable performance-related information; many of which you can use during the tutorial to test your current website if it’s accessible from the classroom. You’ll also receive an introduction to the ‘rocket science’ side of performance testing along with some things that you can do to make life easier for your resident ‘performance testing rocket scientist’.
Tools of the Trade: Load Testing - Ignite session at WebPerfDays NY 14Alexander Podelko
Tools of the Trade: Load Testing - an Ignite session at WebPerfDays NY 2014. Some consideration about load testing and selecting load testing tools - as much as could be squeezed into 5 min / 20 slides.
In this quality assurance training session, you will learn HP LoadRunner. Topics covered in this course are:
• Introduction to Performance Testing
• Why do performance testing?
• Load Testing
• Stress Testing
• Volume testing
• Performance testing tool
• Why LoadRunner
• Why LoadRunner
• LoadRunner Components
• LoadRunner Architecture
• How LoadRunner Works
• LoadRunner load testing process
• The script development process in VUGen
To know more, visit this link: https://www.mindsmapped.com/courses/quality-assurance/software-testing-quality-assurance-qa-training-with-hands-on-exercises/
In this session you will learn:
Software Testing Tool – Overview
Advantage- Automation
Disadvantage - Automation
Grouping of Automation Tool
Functional Tool
Source Code Testing Tool
Performance Tool
Test Management Tool
Security Testing Tool
For more information: https://www.mindsmapped.com/courses/quality-assurance/qa-software-testing-training-for-beginners/
In this Quality Assurance Training session, you will learn about Automation Tools Overview. Topic covered in this session are:
• SQL Basic Operators and Function
• Software Testing Tool – Overview
• Advantage- Automation
• Disadvantage - Automation
• Grouping of Automation Tool
• Functional Tool
• Source Code Testing Tool
• Performance Tool
• Test Management Tool
• Security Testing Tool
For more information, about this quality assurance training, visit this link: https://www.mindsmapped.com/courses/quality-assurance/software-testing-training-with-hands-on-project-on-e-commerce-application/
In this presentation which was delivered to testers in Manchester, I help would-be performance testers to get started in performance testing. Drawing on my experiences as a performance tester and test manager, I explain the principles of performance testing and highlight some of the pitfalls.
These slides is from a session about load testing. It starts with an introduction to non-functional testing then demonstrating the phases and criteria of load testing with K6.
Faced with two critical goals and low resources, the Kuali Student Test Team developed and implemented a test automation framework based on an open source tool ? Tsung and Amazon?s Elastic Compute Cloud (EC2). This automation framework netted success right away as it allowed the team to identify and regress critical performance issues through out development.
Kuali developers and test engineers, and technical staff from implementing institutions will learn how industry best practices for performance testing were applied to Kuali Student and will walk away with guidance on how to setup and run an open source performance testing tool to support their needs (live demonstration included).
The recent revolution in software development, including agile / iterative development, cloud computing, continuous integration, and many more, opened new opportunities for performance testing and affected its role in performance engineering. For example, early and continuous performance testing is becoming the new norm. However, performance testing in general and specific performance testing techniques should be considered in full context: including environments, products, teams, issues, goals, budgets, timeframes, risks, etc. The question is not what technique is better, the question is what technique (or what combination of techniques) to use in particular case (or, in more traditional wording, what should be performance testing strategy).
Drastic changes in the industry in recent years significantly expanded the performance testing horizon: agile development and cloud computing probably the most. Basically, instead on single way of doing performance testing (and all other were considered rather exotic), we have a full spectrum of different tests which can be done at different moments; so deciding what and when to test became a very non-trivial task heavily depending on the context. We need to create and run different tests mitigating different performance risks.
So the art of performance engineering is to find out the best strategy of combining different performance tests and other approaches to mitigate performance risks to optimize risk mitigation / costs ratio for, of course, the specific context.
This session will provide you the insights of understanding various types of performance risks, which test techniques and practices to use in a specific context to measure and evaluate system performance, and how to interpret the data derived from these tests in order to drive performance engineering excellence.
After passing all functional tests, many web applications fail under the regular load conditions. Just 100 active users may be sufficient to cause severe errors and a decline in performance. In this session, load testing of ADF applications with the Oracle Application Testing Suite and Apache JMeter will be discussed and demonstrated. With these tools, you have an analysis-instrument to do performance analysis and regression analysis after new patches/releases. A proper managed load test also gives insight in the effects configuration parameters. The speaker will share best test practices to perform load tests and offer typical solutions for creating an efficient, scalable ADF application.
Alexander Podelko - Context-Driven Performance TestingNeotys_Partner
Since its beginning, the Performance Advisory Council aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing. During this event, 12 participants convened in Chamonix (France) exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.
In this quality assurance training session, you will learn Other Testing Types. Topics covered in this course are:
• Testing the code – WBT
• Database Testing
• UI Testing
• Non Functionality Testiing
• Class Assignment
TO know more, visit this link: https://www.mindsmapped.com/courses/quality-assurance/get-practical-training-on-software-testing-quality-assurance-qa/
July webinar l How to Handle the Holiday Retail Rush with Agile Performance T...Apica
In this Q&A-style webinar, you'll learn:
1. How and why to load test at least three months prior to the holidays
2. How to integrate CI/CD into your holiday load testing
3. How to determine and evaluate load curves
Similar to Load Testing: See a Bigger Picture, ALM Forum, 2014 (20)
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
2. Load Testing:
See a Bigger Picture
Alexander Podelko
alex.podelko@oracle.com
www.alexanderpodelko.com
@apodelko
April 1, 2014
3. Speaker Bio
• Have specialized in performance for the last 17 years
• Currently performance testing and optimization of
Hyperion products at Oracle
• Board director at CMG (http://cmg.org), organization
of performance and capacity professionals
– Next conference November 3-6, 2014 in Atlanta, GA
Disclaimer: The views expressed here are my personal views only and do not necessarily represent those of my current or
previous employers. All brands and trademarks mentioned are the property of their owners.
3
4. In This Session, You’ll Learn…
• Load testing and its role in performance risk
mitigation
• Different approaches: load generation
• Different approaches: environments
• Load testing tools
4
6. The Stereotype
• Load / PerformanceTesting is:
– Last moment before deployment
– Last step in the waterfall process
– Protocol Level Record-and-Playback
– Large corporations
– Expensive tools requiring special skills
– Lab environment
– Scale-down environment
– …
6
7. Load Testing
• It is only one very specific kind of load testing
– Most popular due to easier integration in corporate SDLC
• But load testing in no way limited to this one
Technology evaluation
Infrastructure evaluation
Prototypes / POC
Component / unit
What/if
Performance troubleshooting
Performance optimization
Benchmarking
7
9. But all of them
don’t replace
load testing
Load testing
complements them in
several important ways
9
10. What Load Testing Adds
• Verification that the system handles the load
• Verification of multi-user performance
• Performance optimization
– Exactly the same load
• Debugging/verification of multi-user issues
• Testing self-regulation functionality
– Such as auto-scaling or changing the level of service
depending on load
10
11. Agenda
• Load testing and its role in performance risk
mitigation
• Different approaches: load generation
• Different approaches: environments
• Load testing tools
11
13. Load Generation
• Create test assets – run test
• A must step in load testing
• ‘Test assets’ - usually scripts or programs in
load testing
• Time constraints can make it very challenging
– Different for each product / interface
13
14. Record and Playback:
Protocol Level
• Virtual users: record communication between
two tiers and then playback an automatically
created script
• Usually after proper correlation /
parameterization
14
16. Issues
• Usually doesn't work for testing components
• Each tool support a limited number of
technologies (protocols)
• Some technologies are very time-consuming
• Workload validity in case of sophisticated logic
on the client side is not guaranteed
16
17. Recalling Challenges
• 199x – SMB (Server Message Block), DCOM,
Java RMI
• 200x – Applets and ActiveX controls
• 201x – Rich InternetApplications
It was just a short period of time whenWeb sites
were simple – it was rather an exception
17
18. Not so Simple: Example
• Back-end calculation (Financial consolidation)
– Long time, shows progress bar
– Polling back-end
– Explicit loop is needed to work properly
18
22. Manual
• Not an option for a large number of users
• Always variation in human input times
• Can be a good option to simulate quickly a
few users
• Can be used with other methods to verify
correctness
22
23. Real Users
• Full load (testing in production)
• Partial load (A/B testing, canary testing)
• You trade in the need to generate and
validate workload for a possibility of
performance issues and load variability.
23
24. Considerations
• Make sense for the following conditions
– Potential issues have minimal impact on user
satisfaction and company image
– Easy rollback of the changes
– Homogenous workload and a way to control it
– Fully parallel and scalable architecture
24
25. Record and Playback: UI Level
• Functional / regression testing tools
• Record and playback communication between
user and clientGUI
• Don't care about communication protocols /
internals
• Accurate data (real client, end-to-end)
25
26. History of the Approach
• Traditional tools, fat clients
– Require a separate machine (or a terminal session) per user
• Low-level graphical protocols
– Citrix, Remote Desktop
• Web tools, browser
– Require a separate browser instance
• Web tools, light-weight browser
– Require a separate light-weight browser instance
– For example, HtmlUnit or PhantomJS
26
27. Record and Playback: UI Level
Load Testing Tool
Virtual
Users
ServerLoad Generator
Application
Network
Browsers
27
28. Challenges
• Scalability
– Still require more resources
• Supported technologies
• Timing accuracy
• Playback accuracy
– For example, for HtmlUnit
28
29. Programming: Custom Test Harness
• Special program to generate workload
• Requires access to the API or source code
• Requires programming
• Could be cost effective solution in some
simple cases
29
30. Advantages
• Doesn’t require any special tool
• Starting version could be quickly created by
a programmer familiar with API
• Should work if API works
• You don't care what protocol is used for
communication
30
31. Disadvantages
• Efforts to update and maintain harness
increase drastically as you need to add
features
• When you have numerous products you really
need to create something like a commercial
load testing tool
31
32. Mixed Approach
• Programming using load testing tools
– Program a script instead of recording
– Run it using a load testing tool
• Implementation depends on the particular tool
– Scripting support
– Languages support
– API calls (may require external dll / client stubs)
32
33. Advantages
• Eliminates dependency on supporting
specific protocols
• Leverages all the features of the load testing
tool and allows using it as a test harness
• Sometimes simplifies work with difficult to
parameterize protocols
33
34. Considerations
• Requires access to API or source code
• Requires programming
• Minimal transaction that could be
measured is a request / API call
• Requires understanding of internals
34
35. More Considerations
• Requires a load test tool license for the
necessary number of users
• Environment may need to be set on all agents
• May require more resources on agents
• Results should be cautiously interpreted
35
36. Example 1: Essbase Query
• Multi-Dimensional Database
• C API
– Used by many applications and middleware
– Winsock scripts
• Quite difficult to parameterize and verify
• External DLL was made for major functions
36
39. Script Using External DLL
lr_load_dll("c:templr_msas2k.dll");
pCTX = Init_Context();
hr = Connect(pCTX, "ess01", "user001","password");
…
lr_start_transaction("Mdx_q1");
sprintf(report, "SELECT %s.children on columns, %s.children on
rows FROM Shipment WHERE ([Measures].[Qty Shipped], %s,
%s)", lr_eval_string("{day}"), lr_eval_string("{product}"),
lr_eval_string("{customer}"), lr_eval_string("{shipper}"));
hr = RunQuery(pCTX, report);
lr_end_transaction("Mdx_q1",LR_AUTO);
39
40. Example 2: EDS
• Essbase Deployment Services
• Middleware, no GUI interface
• Functional test scripts in Java
• Solution - creation of LoadRunner scripts
from the functional test scripts
40
42. Agenda
• Load testing and its role in performance risk
mitigation
• Different approaches: load generation
• Different approaches: environments
• Load testing tools
42
43. Environments
• Lab vs. Service (SaaS) vs. Cloud (IaaS)
– For both the SUT and load testing tool
• Test vs. Production
• No best solution, depends on your goals /
system
43
44. Scenarios
• System validation for high load
– Outside load (service or cloud), production system
– Wider scope, lower repeatability
• Performance optimization / troubleshooting
– Isolated lab environment
– Limited scope, high repeatability
• Testing in Cloud
– Lowering costs (in case of periodic tests)
– Limited scope, low repeatability
44
45. Agenda
• Load testing and its role in performance risk
mitigation
• Different approaches: load generation
• Different approaches: environments
• Load testing tools
45
46. Load Testing Tools
• Differ drastically
– Supported approaches / protocols
– Scripting / extendibility
– Supported environments
– Scalability
– Integration
– Result analysis
– Environment monitoring
– Cost/Licensing
– Available support and skills
46
47. Load Testing Tools
• There is no best tool – it depends on your
needs
• Almost every tool will work for a simple web
site
• If using more sophisticated technologies,
always check if the tool / approach supports it
47
48. Summary
• Load testing is an important way of
performance risk mitigation / part of the
performance engineering process
– Other ways don’t substitute load testing
• There are many ways to do load testing.
• There is no best approach or tool – it depends
on your needs.
48