More than just giving Apache benchmark a url to request many times, RedLine13 lets you script yourself actual users, mimicking real requests to your system. This not only lets you discover the where your bottlenecks are, but lets you to actually test different scenarios while under real load.
Jason will cover not only why you would want to load test a system, but will also cover what you want to look for when load testing, as well as show you how to write load tests to use with RedLine13. RedLine13 is an inexpensive load testing tool that turns load testing from a chore into something you'll recommend at every point. After all, who doesn't like watching things crash and burn, and getting paid to do it?
Learn how to write scripted load tests in PHP to run against your system without breaking the bank. Jason will cover not only the importance of load testing, but share stories of how load tests uncovered problems that would otherwise not have been discovered until production. Also, learn how to use load testing to learn how to deal with large traffic sites without needing to be employed by a large scale site first. We’ll be using RedLine13, an almost free load testing tool that is at the same time inexpensive, easy, and effective.
Test Driven Development emphasises quick iterations for your code. You won't code faster, but perhaps you will be more accurate.
TDD has been around for a while and I have avoided it for most of my career. I decided for the month of October all code I write I will do TDD. The result was much more positive than I thought it would be and made me rethink how I code. This talk covers the testing pyramid, the importance of unit testing and the tools I use for testing React. As part of the talk I will work through a trivial example to demonstrate how to do TDD.
Learn how to write scripted load tests in PHP to run against your system without breaking the bank. Jason will cover not only the importance of load testing, but share stories of how load tests uncovered problems that would otherwise not have been discovered until production. Also, learn how to use load testing to learn how to deal with large traffic sites without needing to be employed by a large scale site first. We’ll be using RedLine13, an almost free load testing tool that is at the same time inexpensive, easy, and effective.
Test Driven Development emphasises quick iterations for your code. You won't code faster, but perhaps you will be more accurate.
TDD has been around for a while and I have avoided it for most of my career. I decided for the month of October all code I write I will do TDD. The result was much more positive than I thought it would be and made me rethink how I code. This talk covers the testing pyramid, the importance of unit testing and the tools I use for testing React. As part of the talk I will work through a trivial example to demonstrate how to do TDD.
Large Scale Load Testing Amazon.com’s Traffic on AWS (CPN102) | AWS re:Invent...Amazon Web Services
It’s 4am and you don’t know it, but you're about to get three times the traffic you were expecting. Is your service ready to handle it? Systems are only as scalable as their weakest component. Large scale load testing in production is the best (and surest) way to ensure that services can truly scale to the unexpected. But the load generator itself can be difficult to scale, expensive to run on hundreds or thousands of hosts, challenging to keep the data secure, and time consuming to develop. The Amazon.com retail site is one of most heavily used sites in the world, and has to be ready for anything, at anytime. How do you design a load test for this in record time while keeping it cost effective? Well, you use AWS! Come learn Best Practices on how you can use Amazon SQS, Amazon S3, Amazon EC2, Amazon CloudWatch, Auto Scaling, and Amazon DynamoDB to design horizontally scalable large-scale load tests that can simulate the load that millions of users are putting onto your site. We met a tight schedule and did it under budget thanks to AWS and you can too!
Performance testing with 100,000 concurrent users in AWSMatthias Matook
M-Square build an easy scalable performance test solution on AWS, using open source tools & CI servers, to allow cost-effective testing at scale. The solution is suitable for any organisation type, from startup to enterprise.
The talk covers VPC, EC2, S3, ELB’s, AWS API scripting, automation and interesting performance issues when running massive workloads on AWS.
This presentation deals with a complex approach to application testing in back end and front end parts, tests writing and common mistakes. It also includes a short overview of libraries and frameworks for creation of tests, as well as practical examples of code.
Presentation by Pavlo Iuriichuk, Lead Software Engineer, GlobalLogic, Kyiv), delivered at an open techtalk on December 11, 2014.
More details - http://globallogic.com.ua/report-web-testing-techtalk-2014
Even for JavaScript software developers well-versed in Agile practices, using test-driven development in Node.js and Express can be challenging. In this presentation, I identify solutions to some of the most significant challenges to using TDD with Express, including mocking data in MongoDB / Mongoose, using promises to control asynchronous testing in Mocha with Chai, and separating concerns to write robust and enduring test suites.
Salesforce provides an interface for testing callouts named HttpCalloutMock used to cover remote callouts. While adequate for simple callouts, in the real world you often need something more flexible, as in the case of multiple and varying responses from the same or varying endpoints. More precise testing and coverage can be obtained by extending the standard interface. Join us as we demonstrate a solution to use to enable the flexibility required for complex integration and synchronization apps.
Enhanced Web Service Testing: A Better Mock StructureCRMScienceKirk
Salesforce provides an interface for testing callouts named HttpCalloutMock used to cover remote callouts. While adequate for simple callouts, in the real world you often need something more flexible, as in the case of multiple and varying responses from the same or varying endpoints. More precise testing and coverage can be obtained by extending the standard interface. Join us as we demonstrate a solution to use to enable the flexibility required for complex integration and synchronization apps.
How we killed our process, technology stack, and assumptions – and survivedJason Lotito
How do you introduce new ways of building products without stopping development? MeetMe.com's Jason Lotito will reveal the challenges his team encountered while changing their development process, including the perils of adding new technology to the stack. He'll also describe how the team moved from a slower development process to a faster, more efficient one — practically overnight.
Large Scale Load Testing Amazon.com’s Traffic on AWS (CPN102) | AWS re:Invent...Amazon Web Services
It’s 4am and you don’t know it, but you're about to get three times the traffic you were expecting. Is your service ready to handle it? Systems are only as scalable as their weakest component. Large scale load testing in production is the best (and surest) way to ensure that services can truly scale to the unexpected. But the load generator itself can be difficult to scale, expensive to run on hundreds or thousands of hosts, challenging to keep the data secure, and time consuming to develop. The Amazon.com retail site is one of most heavily used sites in the world, and has to be ready for anything, at anytime. How do you design a load test for this in record time while keeping it cost effective? Well, you use AWS! Come learn Best Practices on how you can use Amazon SQS, Amazon S3, Amazon EC2, Amazon CloudWatch, Auto Scaling, and Amazon DynamoDB to design horizontally scalable large-scale load tests that can simulate the load that millions of users are putting onto your site. We met a tight schedule and did it under budget thanks to AWS and you can too!
Performance testing with 100,000 concurrent users in AWSMatthias Matook
M-Square build an easy scalable performance test solution on AWS, using open source tools & CI servers, to allow cost-effective testing at scale. The solution is suitable for any organisation type, from startup to enterprise.
The talk covers VPC, EC2, S3, ELB’s, AWS API scripting, automation and interesting performance issues when running massive workloads on AWS.
This presentation deals with a complex approach to application testing in back end and front end parts, tests writing and common mistakes. It also includes a short overview of libraries and frameworks for creation of tests, as well as practical examples of code.
Presentation by Pavlo Iuriichuk, Lead Software Engineer, GlobalLogic, Kyiv), delivered at an open techtalk on December 11, 2014.
More details - http://globallogic.com.ua/report-web-testing-techtalk-2014
Even for JavaScript software developers well-versed in Agile practices, using test-driven development in Node.js and Express can be challenging. In this presentation, I identify solutions to some of the most significant challenges to using TDD with Express, including mocking data in MongoDB / Mongoose, using promises to control asynchronous testing in Mocha with Chai, and separating concerns to write robust and enduring test suites.
Salesforce provides an interface for testing callouts named HttpCalloutMock used to cover remote callouts. While adequate for simple callouts, in the real world you often need something more flexible, as in the case of multiple and varying responses from the same or varying endpoints. More precise testing and coverage can be obtained by extending the standard interface. Join us as we demonstrate a solution to use to enable the flexibility required for complex integration and synchronization apps.
Enhanced Web Service Testing: A Better Mock StructureCRMScienceKirk
Salesforce provides an interface for testing callouts named HttpCalloutMock used to cover remote callouts. While adequate for simple callouts, in the real world you often need something more flexible, as in the case of multiple and varying responses from the same or varying endpoints. More precise testing and coverage can be obtained by extending the standard interface. Join us as we demonstrate a solution to use to enable the flexibility required for complex integration and synchronization apps.
How we killed our process, technology stack, and assumptions – and survivedJason Lotito
How do you introduce new ways of building products without stopping development? MeetMe.com's Jason Lotito will reveal the challenges his team encountered while changing their development process, including the perils of adding new technology to the stack. He'll also describe how the team moved from a slower development process to a faster, more efficient one — practically overnight.
Social Media and ADHD – Turning Distractions Into DirectionsGrant Crowell
Attention Deficit Hyperactivity Disorder (ADHD/ADD.) In today’s hyper-connected internet and Social age, many of us are showing increasing symptoms of “Virtual ADD:” easily distracted, expected to multi-task more than ever before, and experiencing greater difficulty to focus long-term and prioritize. Without having the important social cues we have to work with from being in person with each other, oftentimes our online networking and relationships create big mis-communications, social fax paus, and unintentional impressions of being inconsiderate to other people’s feelings.
Compounding on the problem, both academics and behavioral psychologists that specialize in emotional intelligence, along with ADD psychologists and coaches, lack the serious expertise and personal experience needed to cover the effects of social media on people dealing from ADD: both “virtual” and genetic.
ADD is not a deficiency in a person. Honed right, it can be an incredibly special gift. But today’s understanding of how social media affects ADD, and vice versa, has huge gaps in research.
What we need today is a new type of learning: education and training from professionals with technical and communications know-how in Facebook, LinkedIn, Twitter, YouTube, OkCupid, and many more of the online communities we spend out time in building relationships, both personal and professional. They understand people in the organic sense and the virtual sense, and understand how we are evolving like technology, and how to bring us back down to earth and make us mindful of social context, and of each other, for personal happiness and professional success. These are the new “Social Stylists.”
Learn from this presentation:
• The connection between Social Media and ADD/ADHD, and vice versa.
• How professionals in social media and other Internet communications can master “Virtual ADD” and still stay focused, organized, and effective with their responsibilities, both professional and personal.
• Social Media, and other jobs and responsibilities perfect for ADD/ADHD people.
• Tools and tips for how to manage ADD/ADHD for a happy and successful, per-fessional life.
• How to use distractions to your creative advantage, and how to set realistic systems in place for shutting them off. (Including constantly checking email and one’s social media walls.)
• How to learn the hard-to-find social cues in digital media, and make more thoughtful communications that lead to less misunderstandings, and better relationships.
• Stories from successful per-fessionals who mastered their own ADHD.
Getting Things Done - Tips from someone with ADHD and OCDJason Lotito
I've had ADHD and OCD my entire life, but didn't get diagnosed with it until I was 33. Up to that point, I struggled as a developer to get things done. This talk is all about the tips, tricks, and techniques I used to get things done. From managing your time, prioritizing, and being a better leader, this talk is full of the things I still use on a daily basis to overcome having both ADHD and OCD.
"Load Testing Distributed Systems with NBomber 4.0", Anton MoldovanFwdays
The motivation of this talk is to share my experience designing and using NBomber for numerous use cases: from testing microservices to databases. Also, I want to share not only code/examples but also discuss potential issues that can be caught by load testing. At some point, we will touch on chaos testing and metastability.
- NBomber architecture
- Load testing microservices from theory to practice (HTTP, WebSockets)
- Load testing with NBomber Cluster
- Real-time reporting
- K8s integration
- Load testing databases (SQL, NoSQL)
QA Fest 2019. Антон Молдован. Load testing which you always wantedQAFest
Десь рік тому ми почали працювати над новою версією наших продуктів. Саме тоді ми почали випробовувати різні технології, архітектури, підходи, а головне це — міряти performance, бо без цього в highload проектах взагалі не вижити.
При проектуванні “любої” системи нам потрібно знати її ліміти:
- скільки паралельних запитів може обробити мікросервіс за допустиму latency?
- як багато запитів може витримати база даних, яку ми використовуємо?
- як довго потрібно чекати на Push повідомлення?
- як довго триває розподілена транзакція та між якими сервісами відбувається найбільша затримка?
І таких питань у нас було безліч. В процесі тестування ми використовували різний tooling: JMeter, ab, Gatling, але всі вони надавали дуже лімітовані можливості. Нам не вдавалося нормально покрити push flow (WebSockets/SSE), різні бази даних, було складно імітувати різний workloads (update/read).
На цій зустрічі я розповім про наш досвід застосування load testing:
- що використовуємо для тестування баз даних, мікросервісів;
- як готуємо Pull/Push тести та як адаптуємо тести під різні протоколи (HTTP/WebSockets/SSE);
- які виникають проблеми з замірами latency.
Моя доповідь дуже практична, тому після неї ви зможете з легкістю почати застосовувати load testing у себе на проекті.
Anton Moldovan "Load testing which you always wanted"Fwdays
Десь рік тому ми почали працювати над новою версією наших продуктів. Саме тоді ми почали випробовувати різні технології, архітектури, підходи, а головне це — міряти performance, бо без цього в highload проектах взагалі не вижити.
При проектуванні “любої” системи нам потрібно знати її ліміти:
скільки паралельних запитів може обробити мікросервіс за допустиму latency?
як багато запитів може витримати база даних, яку ми використовуємо?
як довго потрібно чекати на Push повідомлення?
як довго триває розподілена транзакція та між якими сервісами відбувається найбільша затримка?
І таких питань у нас було безліч. В процесі тестування ми використовували різний tooling: JMeter, ab, Gatling, але всі вони надавали дуже лімітовані можливості. Нам не вдавалося нормально покрити push flow (WebSockets/SSE), різні бази даних, було складно імітувати різний workloads (update/read).
На цій зустрічі я розповім про наш досвід застосування load testing:
що використовуємо для тестування баз даних, мікросервісів;
як готуємо Pull/Push тести та як адаптуємо тести під різні протоколи (HTTP/WebSockets/SSE);
які виникають проблеми з замірами latency.
Моя доповідь дуже практична, тому після неї ви зможете з легкістю почати застосовувати load testing у себе на проекті.
Performance benchmarks are all too often inaccurate. This talk introduces some things to look for in setting up and running benchmarks to make them effective.
Dolphin: Regression Test System for LatitudeTao Jiang
Dolphin is a regression test infrastructure built from ground up for Latitude system testing. It is a major initiative to improve test efficiency and meet test automation objective for Latitude system evaluation department.
Latitude system testing involves in large varieties of devices and device interactions, including: PG, communicator, external sensors, heart simulator, TTM, PRM, etc, as well as, different type of application server environments with different geographic configuration, which impose great engineering challenges.
Dolphin answers those challenges by standardizing test station hardware and software configuration for hardware virtualization. Each virtualized test station is then brought into Dolphin service cloud (private cloud) and becomes virtual service end point. Dolphin manages test stations farm and configures them automatically per test request. Dolphin exposes its services via RESTful JSON web service, including: Regression Testing support, Device configuration and control, Lease out a virtual test station on demand as a hardware resource.
Besides, Dolphin provides an integrated web application for all of its services, including services provided by Panda. It is a one stop shop for all Latitude system testing needs.
Methods:
1. Hardware virtualization to bring virtualized test station into cloud (cloud computing).
2. On demand web application for all exploratory testing needs.
3. Service is available through thin wrapper API (Python, Java, Javascript) for automated test development.
4. Integrated with Panda service for enrollment and data browsing.
Results:
1. Simplify hardware configuration: all test stations are standardized and no need to physically switch parts and reconnect wire.
2. On demand web application greatly simplify device setup and control.
3. Integrated with Panda for automatic enrollment. Manual test can achieve the efficiency of automated test.
4. Centralized management reduces the maintenance cost.
5. Flexibility: automated test can be developed in any language and run on any platform because virtual test station can be leased out from test farm as hardware resource.
Mal ganz ehrlich: Testen im Frontend hat noch nie viel Spaß gemacht. In meinem Talk möchte ich mit Jest eine Testbibliothek vorstellen, die genau das ändern kann. Jest lässt sich ohne viel Konfiguration direkt einsetzen und bringt alles mit, was man von einer Testbibliothek erwartet (und noch mehr).
An vielen praktischen Beispielen möchte ich meine Lieblingsfeatures wie Snapshot-Tests, Mocking oder das tolle CLI erläutern und zeigen, dass Testen im Frontend durchaus Spaß machen kann. Eine Ausrede weniger, um auf das Testen im Frontend zu verzichten!
Vorkenntnisse:
Grundlegende JavaScript-Kenntnisse.
Lernziele:
Die Teilnehmer sollen einen Überblick über die Funktionsweise und die Einsatzbereiche von Jest bekommen. Sie lernen, wie sie für Tests in Frontend-Projekten mit Jest im Werkzeugkoffer gut gerüstet sind.
I would like to share my story about how our team was building an efficient testing process, how these changes affect the development process overall, how to solve common problems of BDD-style tests with DEMO on real examples. My story begins with several failures/problems, which every team meets at the beginning of involving BDD tools in automation tests.
The next topic is including several improvements such as universal step definitions, cucumber expressions, own parameter types, text localization testing, involving REGEXP to test special symbols, etc.
After, slides cover solving irritable problems of BDD tests such as: getting, remembering and reusing unique data during test run sessions, working with API to avoid repeatable steps, file verifications in headless mode, excel files content, hash, screenshot testing, etc.
GeeCON 2017 - TestContainers. Integration testing without the hassleAnton Arhipov
TestContainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
PyCon AU 2012 - Debugging Live Python Web ApplicationsGraham Dumpleton
Monitoring tools record the result of what happened to your web application when a problem arises, but for some classes of problems, monitoring systems are only a starting point. Sometimes it is necessary to take more intrusive steps to plan for the unexpected by embedding mechanisms that will allow you to interact with a live deployed web application and extract even more detailed information.
How to test infrastructure code: automated testing for Terraform, Kubernetes,...Yevgeniy Brikman
This talk is a step-by-step, live-coding class on how to write automated tests for infrastructure code, including the code you write for use with tools such as Terraform, Kubernetes, Docker, and Packer. Topics covered include unit tests, integration tests, end-to-end tests, test parallelism, retries, error handling, static analysis, and more.
Practical Chaos Engineering will show how to start running chaos experiments in your infrastructure and will try to guide your through the principles of chaos.
(APP307) Leverage the Cloud with a Blue/Green Deployment Architecture | AWS r...Amazon Web Services
Minimizing customer impact is a key feature in successfully rolling out frequent code updates. Learn how to leverage the AWS cloud so you can minimize bug impacts, test your services in isolation with canary data, and easily roll back changes. Learn to love deployments, not fear them, with a blue/green architecture model. This talk walks you through the reasons it works for us and how we set up our AWS infrastructure, including package repositories, Elastic Load Balancing load balancers, Auto Scaling groups, internal tools, and more to help orchestrate the process. Learn to view thousands of servers as resources at your command to help improve your engineering environment, take bigger risks, and not spend weekends firefighting bad deployments.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
3. Load testing is the process of putting demand on
a software system or computing device and
measuring its response.
4. Load testing is performed to determine a
system's behavior under both normal and
anticipated peak load conditions.
5. It helps to identify the maximum operating
capacity of an application as well as any
bottlenecks and determine which element is
causing degradation.
7. Stress testing focuses on throughput.
Load testing focuses on response times.
Both can be performed at the same time.
8. Load Testing
• Putting demand on a system
• Measuring the response
• Determine behavior under
load
• Determine maximum operating
capacity
• Bottlenecks
10. Why RedLine13?
• Simple, up and running in little time
• Inexpensive, you pay for what you use
• Scriptable, so you have complete control over your
tests
• I know Rich
11. RedLine13 has a bunch of features you can read
about on the website. I’m really here to talk to
you about the cool stuff I do with RedLine13.
12. The Basics of Load Testing
The Basics of RedLine13
This is another subtitle that I’m
apparently supposed to use, in place of
the title, and the super subtitle above
the title.
20. Need
• Production-level systems to test against
• Monitoring
• CPU, memory, I/O, etc.
• Application dashboards
• Logging
• Automated systems to deploy
• Dedicated people
• A time to test
21. Learning
• Set goals, e.g.
• 50ms
• 500rps
• Test monitoring
• Scaling
• What happens to the app under load?
• What slows down first?
22. As Load Tester, you are responsible for…
• Ensuring those involved with the system under test are
aware and available
• Putting the system under test under load
• Completely destroying the system under load
• Providing a clear and concise write up of exactly what
occurred
• Take charge! It’s fun!
23. Ramp Up Time
• Start at what you expect normal traffic to be
• 500rps
• Start a second test that is just the burst traffic
• 900rps
• Ramps up over 20 mins
• Lasts 40 mins at peak
• Ramps down over 20 mins
26. Reporting Results: Short Version
• Short version: During an hour starting with 0rps to 1400 rps
in the first 10 minutes....
• ...when starting with 5 instances and scaling to 11
instances, the response time were: 50% 23ms, 75% 54ms,
95% 303ms, and 99% 1069ms.
• ...when starting with 11 instances, the response times
were: 50% 16ms, 75% 24ms, 95% 45ms, and 99% 59ms.
27. Reporting Results: Detailed Version
• Provide more information
• Results
• Story driven
• Have data supporting results prepared in case
• Account for changes such as auto-scaling
• With auto-scaling: 99% at 1069ms
• Without auto-scaling: 99% at 59ms
28. Testing elasticsearch on c3.4xlarge
Detailed Reporting
Also included in this report was a link to
the actual GitHub repository. Make sure
you are keeping your load tests in
version control!
29.
30.
31. Tag your load test in git
Associate the tag with a Load Test,
so you can replay any load test easily
33. Writing a RedLine13 Custom Load Test in Node.js
• Uploading a *.tar.gz/*.zip file
• npm install is run
• Expects a CustomTest.js file in root directory
• CustomTest.js should export a ‘class’
• Will create a new object from
• class(redLine, testNum, rand, config)
• Will call runTest() on object of class
37. var Test = require('./CustomTest');
var rl = require('./lib/RedLine');
(new Test(new rl.RedLine, 55)).runTest(function(){
console.log( "test complete");
});
app.js
38. var Test = require('./CustomTest');
var rl = require('./lib/RedLine');
(new Test(new rl.RedLine, 55)).runTest(function(){
console.log( "test complete");
});
app.js
39. var Test = require('./CustomTest');
var rl = require('./lib/RedLine');
(new Test(new rl.RedLine, 55)).runTest(function(){
console.log( "test complete");
});
app.js
40. var Test = require('./CustomTest');
var rl = require('./lib/RedLine');
(new Test(new rl.RedLine, 55)).runTest(function(){
console.log( "test complete");
});
app.js
56. var async = require( 'async' );
var sleep = require( 'sleep' );
var Timer = require( './lib/RedLine' ).Timer;
var Test = require( './lib/Test' );
// Load Testing Options
var MINUTES_TO_RUN = 60;
var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;
var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
CustomTest.js
57. var async = require( 'async' );
var sleep = require( 'sleep' );
var Timer = require( './lib/RedLine' ).Timer;
var Test = require( './lib/Test' );
// Load Testing Options
var MINUTES_TO_RUN = 60;
var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;
var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
CustomTest.js
58. var sleep = require( 'sleep' );
var Timer = require( './lib/RedLine' ).Timer;
var Test = require( './lib/Test' );
// Load Testing Options
var MINUTES_TO_RUN = 60;
var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;
var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
function rand( low, high )
{
return Math.floor( Math.random() * (high - low + 1) + low );
}
CustomTest.js
59. var sleep = require( 'sleep' );
var Timer = require( './lib/RedLine' ).Timer;
var Test = require( './lib/Test' );
// Load Testing Options
var MINUTES_TO_RUN = 60;
var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;
var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
function rand( low, high )
{
return Math.floor( Math.random() * (high - low + 1) + low );
}
CustomTest.js
60. var sleep = require( 'sleep' );
var Timer = require( './lib/RedLine' ).Timer;
var Test = require( './lib/Test' );
// Load Testing Options
var MINUTES_TO_RUN = 60;
var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;
var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
function rand( low, high )
{
return Math.floor( Math.random() * (high - low + 1) + low );
}
CustomTest.js
61. var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
function rand( low, high )
{
return Math.floor( Math.random() * (high - low + 1) + low );
}
function Reloaded( rl, testNum, rand, config )
{
this.rl = rl;
this.testNum = testNum;
this.rand = rand;
this.config = config;
CustomTest.js
70. console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );
};
Reloaded.prototype.runTest = function ( finished )
{
var timer = new Timer( 'test', this.rl );
var max = 60 * MINUTES_TO_RUN;
async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
CustomTest.js
71. console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );
};
Reloaded.prototype.runTest = function ( finished )
{
var timer = new Timer( 'test', this.rl );
var max = 60 * MINUTES_TO_RUN;
async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
CustomTest.js
72. console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );
};
Reloaded.prototype.runTest = function ( finished )
{
var timer = new Timer( 'test', this.rl );
var max = 60 * MINUTES_TO_RUN;
async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
CustomTest.js
73. async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
CustomTest.js
74. async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
CustomTest.js
75. async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
CustomTest.js
76. async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
CustomTest.js
77. async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
CustomTest.js
78. var max = 60 * MINUTES_TO_RUN;
async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
CustomTest.js
79. {
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
{
finished( true );
} );
};
CustomTest.js
80. Now, let’s hack logging
By default,RedLine doesn’t support logging
through the web UI
93. Things to Keep In Mind
• Understand expected usage
• X% of users using the app while
• Y% are chatting with one another
• Users are logging in
• Creating accounts
• Backend systems
• Determine what’s important
94. Things to Keep In Mind
• User input
• Random filters
• Weighted filters
• Cached results are expected
• Client constraints
95. Things to Keep In Mind
• User flow through service
• Try to understand how users use the app
• Script should try to mimic
96. Things to Keep In Mind
• Be care about testing a single system
• System will have logging
• System will have backend services
• You’d be surprised what can cause failure
• A Load Test helps you learn before it’s in production
97. Things to Keep In Mind
• User interaction
• MeetMe is social, so we’ve load tested chatting
• 1 test per 2 users, both chatting with one another
• Story time!
98. Things to Keep In Mind
• Have developers available
• Better, have developers with you when load testing
• Find a problem, Fix it, Reroll app, Rerun test
• FFRR
• I just made that up
• Don’t use it.
99. Things to Keep In Mind
• Start testing from your laptop
• Seriously, my MacBook Air could bring down Erlang
services
• Database indexes are a thing
• While running a load test, you can run a single client
form your laptop
100. Things to Keep In Mind
• Someone should be testing the app/service as well
• Response times are only a number
• What does 50ms vs 300ms response times feel to the
user
• What impact does 2x/3x/4x load have
• When auto-scaling, how does the client handle
101. Things to Keep In Mind
• Review how the client is using the API
• Review how the API developer expects the client to use
the API
• Model after what the client is doing
• Call out differences early
103. In Russia, Load tests You!
Good load tests will teach you things
you didn’t know about your systems.
Great load tests will help you solve those problems
before they become production problems.
104. Load Testing with RedLine13
Thanks!
@jasonlotito
jasonlotito.com
jasonlotito@gmail.com