Mutation Testing has been around for almost 20 years. Originated in academic research it has found its way into the developer’s toolbox being easy to setup, use and producing valuable results. But what is mutation testing? It’s a practice to determine the actual value of an automated test suite and automatically explore parts of the code that have yet been untested, unveiling surprises even to experienced test automation developers. Given a test suite that runs successfully, mutation testing will inject changes to the production code based on a set of rules and reruns the test to determine if the test will fail. Depending on the size of the code base the execution time increases exponentially due to the sheer amount of permutations, requiring thorough planning, focus and prioritization.
Must.Kill.Mutants. Agile Testing Days 2017Gerald Muecke
The document discusses quality assurance and mutation testing. It defines quality assurance as preventing defects in manufactured products. It then discusses mutation testing, which involves injecting faults into code based on mutation operators to verify if tests can detect the faults. The document provides examples of mutation operators and tools for different programming languages. It notes that while mutation testing has limitations, it is valuable for finding bugs in test suites and improving test quality. It recommends including mutation testing in projects and using it with common sense to iteratively improve test suites.
about mutation testing and demonstration of muJava. muJava is automated tool for mutation testing of java programs. It tests the test cases. hence good to enhance and checking effectiveness of your test suites.
Test-Driven Development is about approaching software development from a test perspective and knowing how to use the tools (e.g. JUnit, Mockito) to effectively write tests.
Source code examples @...
https://github.com/codeprimate-software/test-driven-development
This document provides an overview of mutation testing and the Pitest tool. It defines mutation testing as seeding artificial defects into source code to check if tests detect them. It describes various mutators that Pitest uses to alter code, such as changing relational operators or removing method calls. The document also outlines how to set up Pitest in a Maven project and review the test coverage report. It concludes that mutation testing can find bugs and redundant code but requires significant time to run all mutants. Not all mutants need to be killed to ensure quality.
The document provides an introduction and overview of mutation testing. It explains that mutation testing involves deliberately introducing small errors or mutations into code to test if tests can detect those errors. It discusses how mutations are made by changing operators or statements, running tests on the mutated code, and analyzing whether tests fail as they should to kill the mutations. The document uses an example of mutating a subtraction method to demonstrate how mutation testing works and what it reveals about test coverage and adequacy.
This document summarizes a survey paper on the development of mutation testing. It outlines the objectives, scope, fundamental hypotheses, related concepts, and future trends of mutation testing. It also discusses classification of research in mutation testing, practical applications using mutation testing tools, and empirical studies that have been conducted.
Testing and Mocking Object - The Art of Mocking.Deepak Singhvi
The document provides an overview of mocking objects for unit testing. It discusses the problems with testing, such as dependencies on external objects. Mocking objects allows creating test doubles that simulate real objects' behavior for testing in isolation. The document outlines best practices for mocking, such as mocking interfaces rather than concrete classes and verifying expectations. It provides examples of using EasyMock to define mock objects and expected behavior.
Unit testing tests individual units of source code to determine if they are fit for use. It is different from integration testing, which combines modules, and validation testing, which checks that a system meets specifications. The goal of unit testing is to isolate and test each part of a program to show that individual parts are correct. Unit testing provides benefits such as finding problems early, facilitating automated testing and change, and improving software quality. Guidelines for effective unit testing include knowing the objective of each test, making tests self-sufficient, ensuring tests are deterministic, following naming conventions, duplicating code for readability, testing results not implementation, and using isolation frameworks.
Must.Kill.Mutants. Agile Testing Days 2017Gerald Muecke
The document discusses quality assurance and mutation testing. It defines quality assurance as preventing defects in manufactured products. It then discusses mutation testing, which involves injecting faults into code based on mutation operators to verify if tests can detect the faults. The document provides examples of mutation operators and tools for different programming languages. It notes that while mutation testing has limitations, it is valuable for finding bugs in test suites and improving test quality. It recommends including mutation testing in projects and using it with common sense to iteratively improve test suites.
about mutation testing and demonstration of muJava. muJava is automated tool for mutation testing of java programs. It tests the test cases. hence good to enhance and checking effectiveness of your test suites.
Test-Driven Development is about approaching software development from a test perspective and knowing how to use the tools (e.g. JUnit, Mockito) to effectively write tests.
Source code examples @...
https://github.com/codeprimate-software/test-driven-development
This document provides an overview of mutation testing and the Pitest tool. It defines mutation testing as seeding artificial defects into source code to check if tests detect them. It describes various mutators that Pitest uses to alter code, such as changing relational operators or removing method calls. The document also outlines how to set up Pitest in a Maven project and review the test coverage report. It concludes that mutation testing can find bugs and redundant code but requires significant time to run all mutants. Not all mutants need to be killed to ensure quality.
The document provides an introduction and overview of mutation testing. It explains that mutation testing involves deliberately introducing small errors or mutations into code to test if tests can detect those errors. It discusses how mutations are made by changing operators or statements, running tests on the mutated code, and analyzing whether tests fail as they should to kill the mutations. The document uses an example of mutating a subtraction method to demonstrate how mutation testing works and what it reveals about test coverage and adequacy.
This document summarizes a survey paper on the development of mutation testing. It outlines the objectives, scope, fundamental hypotheses, related concepts, and future trends of mutation testing. It also discusses classification of research in mutation testing, practical applications using mutation testing tools, and empirical studies that have been conducted.
Testing and Mocking Object - The Art of Mocking.Deepak Singhvi
The document provides an overview of mocking objects for unit testing. It discusses the problems with testing, such as dependencies on external objects. Mocking objects allows creating test doubles that simulate real objects' behavior for testing in isolation. The document outlines best practices for mocking, such as mocking interfaces rather than concrete classes and verifying expectations. It provides examples of using EasyMock to define mock objects and expected behavior.
Unit testing tests individual units of source code to determine if they are fit for use. It is different from integration testing, which combines modules, and validation testing, which checks that a system meets specifications. The goal of unit testing is to isolate and test each part of a program to show that individual parts are correct. Unit testing provides benefits such as finding problems early, facilitating automated testing and change, and improving software quality. Guidelines for effective unit testing include knowing the objective of each test, making tests self-sufficient, ensuring tests are deterministic, following naming conventions, duplicating code for readability, testing results not implementation, and using isolation frameworks.
1. The document discusses various tools and frameworks for unit testing Java, Android, and C/C++ code including JUnit, EasyMock, Google Test, and Google Mock.
2. It provides information on setting up and writing tests for each framework including creating test projects, including libraries, and using mocking functionality.
3. Code examples and references are given for writing and running unit tests with each testing framework.
The document discusses Google Test, an open source unit testing framework for C++ that can also be used for testing C code, providing an overview of its features and how to implement unit tests using common patterns like test fixtures, assertions, and death tests to validate expected failures. It also covers best practices for writing effective unit tests, including keeping tests independent, focused, and fast-running.
This document provides an overview of test-driven development (TDD). It defines TDD as a technique for building software where tests are written before code to guide development. The key aspects of TDD covered are:
- Writing tests first before code, which helps improve design and ensures tests are written.
- The TDD mantra of Red-Green-Refactor, where tests initially fail (Red), code is written to pass tests (Green), then code is refactored to improve design.
- An example case study of a large Java project developed using TDD that has over 20,000 lines of unit tests providing over 90% test coverage.
The document discusses various aspects of unit testing including definitions, benefits, naming standards, and best practices. It provides definitions for terms like error, defect, failure. It outlines the benefits of unit testing like finding bugs early, enabling code refactoring. It discusses what should be tested like boundaries, error conditions. It also provides examples of good test names and guidelines for structuring and naming unit tests.
The document summarizes new features in Test::Unit 2.0, including new assertion methods, ways to omit, pend, or notify tests, and improved setup/teardown functionality like multiple setups/teardowns and setup attributes. It also discusses priority mode for reducing test time, better diff output for failures, and colorized test output.
This document discusses test driven development and unit testing. It begins with an agenda that includes the importance of unit testing, test driven development techniques, and unit testing in .NET and Eclipse. It then covers topics like the definition of a unit, myths about unit testing, how test driven development works in short cycles, and metrics for test usage. Finally, it discusses techniques like common test patterns, using mock objects, and unit testing frameworks like NUnit, MSTest, Moq and features in Visual Studio and Eclipse.
The document discusses unit testing and its benefits and limitations. It notes that while tests provide confidence that code works as intended, they cannot prevent all problems. The Boeing 737 MAX crashes are discussed as an example of issues despite passing tests due to sensor problems. Proper unit testing involves automated, repeatable, and continuous testing of isolated units with mocked dependencies. Test-driven development and design can lead to more testable and maintainable code, but tests also have costs and limitations.
The document provides an overview of Quick Test Professional (QTP), a test automation tool. It discusses key aspects of QTP including recording and running tests, using object repositories, checkpoints, parameters, actions, recovery scenarios, and programmatic descriptions.
Software testing
Developers Belief on Software Testing
Developers Responsibility for testing
Test writing methods
State based testing
Behavioural/interaction based testing
Writing a Testable Code
Flaw 1 - Constructor does Real Work
Flaw 2 - API lies about it's real dependencies
Flaw 3 - Brittle Global State & Singletons
Testing Frameworks and tools for Java...
Mockito and PowerMock...
Testing Models
Stubs Based Testing Model
Mocked Objects Based Testing Model
JUit 4.+ and TestNG
https://www.adroitlogic.com
https://developer.adroitlogic.com
Unit testing involves testing individual units or components of code to ensure they work as intended. It focuses on testing functional correctness, error handling, and input/output values. The main benefits are faster debugging, easier integration testing, and living documentation. Guidelines for effective unit testing include writing automated, independent, focused tests that cover boundaries and are easy to run and maintain.
This document provides an overview of test-driven development (TDD). TDD involves writing tests before writing code to ensure new functionality works as intended. Key principles of TDD include writing failing tests first, then code to pass the tests, and refactoring code while maintaining all tests. TDD results in higher quality, flexible, readable and maintainable code. It also helps improve both internal code quality and external functionality through a well-designed development process focused on automated testing.
Test driven development and unit testing with examples in C++Hong Le Van
Test-driven development (TDD) relies on short development cycles of writing a failing test case, producing code to pass that test, and refactoring the code. Unit testing tests individual units of code by isolating each part and showing they work correctly. Boost.Test is a popular C++ unit testing framework that allows organizing tests into suites and fixtures, and provides assertions and output of results. A minimal Boost.Test example defines a test case using BOOST_AUTO_TEST_CASE that contains an assertion like BOOST_CHECK_EQUAL to test that 2+2 equals 4.
This document discusses Test Driven Development (TDD). It defines TDD, outlines the TDD process which involves writing tests first and then code to pass the tests, and emphasizes refactoring. Benefits of TDD include improved code quality, reduced bugs, and serving as documentation. Key TDD terms and libraries like JUnit and Mockito are explained. Continuous Integration (CI) is also discussed as it automates testing and builds when code is committed.
Mutation testing is a technique that involves modifying code to generate "mutants" in order to test test quality. MuTalk is a mutation testing tool for Smalltalk that generates mutants by applying mutation "operators" that make small changes to code, such as changing logical operators. Tests are then run to determine if they can detect and "kill" the mutants. Surviving mutants indicate untested code cases. MuTalk optimizes the process by only mutating and testing covered code. Mutation testing is effective because complex faults are often coupled to simple faults detected by killing mutants.
The document discusses NUnit, an open source unit testing framework for .NET. It describes what NUnit is, how to install and use it, key concepts like test fixtures and methods, and provides an example of writing unit tests for a calculator class using NUnit's attributes. The document also covers running tests via the NUnit GUI or console, and examining the output XML file for test results.
This is a slide set with some information about programmer testing. It includes test doubles, mockit and junit.
Visit http://jpereira.eu/2011/12/03/programming-testing/ for more information. If you want to download these slides, please contact me (find my contacts on my blog)
The document provides guidelines for writing effective unit tests, including why they are important (ensuring code works as expected, protecting work from accidental breaks, serving as documentation), what should be tested (models, services, commands, utilities), when they should be written (before or after code via test-driven development), and how to write good tests (automated, repeatable, single-purpose, readable, isolated, avoiding duplication).
The document summarizes a presentation on software testing concepts and practices like test-driven development, terminology used in testing, popular tools used for testing, and resources for learning more about testing. The presentation covered topics like unit testing, integration testing, test-driven development principles and practices, popular IDEs, build tools, testing frameworks, mock object libraries, code coverage tools, load testing tools, acceptance testing tools, and continuous integration tools. It also demonstrated testing using an open-source medical record system.
The document discusses building a custom data source for Grafana using Vert.x to retrieve time series data stored in MongoDB. It describes how to connect Grafana and MongoDB through a three-tier architecture using a Vert.x microservice. The microservice would handle splitting requests, querying the database in parallel, aggregating results, and performing additional processing before returning data to Grafana. Vert.x is well-suited for this due to its asynchronous, reactive, and scalable nature. Sample code is available on GitHub to demonstrate the approach.
TAPOST Conference 2016 Riga // Better Products Faster: Let's bring the user i...icemobile
"Better products faster: let's bring the user in the userstory"
as presented at TAPOST conference in Riga 2016.
Why is it that everyone knows the importance of frequent user
testing, yet hardly anyone does it? Because user testing often is time
consuming, complex and expensive. It probably doesn’t fit in your
development process and thus feels like extra work.
To feel reassured you tell yourself to test with users once you have
something working, or at the very end of the process. This is strange,
because everybody knows that changing your product late in the
process will increase costs exponentially.
We created a way so that user testing saves time, improves the
quality and doesn’t cost a lot of money. Team driven, pragmatic and
no extra resources needed.
The talk will show how, with only 2 hours every sprint, we focused on
creating better products faster. We would love to share our learnings
and simple DIY tools that let you start user testing with your current
teams tomorrow!
1. The document discusses various tools and frameworks for unit testing Java, Android, and C/C++ code including JUnit, EasyMock, Google Test, and Google Mock.
2. It provides information on setting up and writing tests for each framework including creating test projects, including libraries, and using mocking functionality.
3. Code examples and references are given for writing and running unit tests with each testing framework.
The document discusses Google Test, an open source unit testing framework for C++ that can also be used for testing C code, providing an overview of its features and how to implement unit tests using common patterns like test fixtures, assertions, and death tests to validate expected failures. It also covers best practices for writing effective unit tests, including keeping tests independent, focused, and fast-running.
This document provides an overview of test-driven development (TDD). It defines TDD as a technique for building software where tests are written before code to guide development. The key aspects of TDD covered are:
- Writing tests first before code, which helps improve design and ensures tests are written.
- The TDD mantra of Red-Green-Refactor, where tests initially fail (Red), code is written to pass tests (Green), then code is refactored to improve design.
- An example case study of a large Java project developed using TDD that has over 20,000 lines of unit tests providing over 90% test coverage.
The document discusses various aspects of unit testing including definitions, benefits, naming standards, and best practices. It provides definitions for terms like error, defect, failure. It outlines the benefits of unit testing like finding bugs early, enabling code refactoring. It discusses what should be tested like boundaries, error conditions. It also provides examples of good test names and guidelines for structuring and naming unit tests.
The document summarizes new features in Test::Unit 2.0, including new assertion methods, ways to omit, pend, or notify tests, and improved setup/teardown functionality like multiple setups/teardowns and setup attributes. It also discusses priority mode for reducing test time, better diff output for failures, and colorized test output.
This document discusses test driven development and unit testing. It begins with an agenda that includes the importance of unit testing, test driven development techniques, and unit testing in .NET and Eclipse. It then covers topics like the definition of a unit, myths about unit testing, how test driven development works in short cycles, and metrics for test usage. Finally, it discusses techniques like common test patterns, using mock objects, and unit testing frameworks like NUnit, MSTest, Moq and features in Visual Studio and Eclipse.
The document discusses unit testing and its benefits and limitations. It notes that while tests provide confidence that code works as intended, they cannot prevent all problems. The Boeing 737 MAX crashes are discussed as an example of issues despite passing tests due to sensor problems. Proper unit testing involves automated, repeatable, and continuous testing of isolated units with mocked dependencies. Test-driven development and design can lead to more testable and maintainable code, but tests also have costs and limitations.
The document provides an overview of Quick Test Professional (QTP), a test automation tool. It discusses key aspects of QTP including recording and running tests, using object repositories, checkpoints, parameters, actions, recovery scenarios, and programmatic descriptions.
Software testing
Developers Belief on Software Testing
Developers Responsibility for testing
Test writing methods
State based testing
Behavioural/interaction based testing
Writing a Testable Code
Flaw 1 - Constructor does Real Work
Flaw 2 - API lies about it's real dependencies
Flaw 3 - Brittle Global State & Singletons
Testing Frameworks and tools for Java...
Mockito and PowerMock...
Testing Models
Stubs Based Testing Model
Mocked Objects Based Testing Model
JUit 4.+ and TestNG
https://www.adroitlogic.com
https://developer.adroitlogic.com
Unit testing involves testing individual units or components of code to ensure they work as intended. It focuses on testing functional correctness, error handling, and input/output values. The main benefits are faster debugging, easier integration testing, and living documentation. Guidelines for effective unit testing include writing automated, independent, focused tests that cover boundaries and are easy to run and maintain.
This document provides an overview of test-driven development (TDD). TDD involves writing tests before writing code to ensure new functionality works as intended. Key principles of TDD include writing failing tests first, then code to pass the tests, and refactoring code while maintaining all tests. TDD results in higher quality, flexible, readable and maintainable code. It also helps improve both internal code quality and external functionality through a well-designed development process focused on automated testing.
Test driven development and unit testing with examples in C++Hong Le Van
Test-driven development (TDD) relies on short development cycles of writing a failing test case, producing code to pass that test, and refactoring the code. Unit testing tests individual units of code by isolating each part and showing they work correctly. Boost.Test is a popular C++ unit testing framework that allows organizing tests into suites and fixtures, and provides assertions and output of results. A minimal Boost.Test example defines a test case using BOOST_AUTO_TEST_CASE that contains an assertion like BOOST_CHECK_EQUAL to test that 2+2 equals 4.
This document discusses Test Driven Development (TDD). It defines TDD, outlines the TDD process which involves writing tests first and then code to pass the tests, and emphasizes refactoring. Benefits of TDD include improved code quality, reduced bugs, and serving as documentation. Key TDD terms and libraries like JUnit and Mockito are explained. Continuous Integration (CI) is also discussed as it automates testing and builds when code is committed.
Mutation testing is a technique that involves modifying code to generate "mutants" in order to test test quality. MuTalk is a mutation testing tool for Smalltalk that generates mutants by applying mutation "operators" that make small changes to code, such as changing logical operators. Tests are then run to determine if they can detect and "kill" the mutants. Surviving mutants indicate untested code cases. MuTalk optimizes the process by only mutating and testing covered code. Mutation testing is effective because complex faults are often coupled to simple faults detected by killing mutants.
The document discusses NUnit, an open source unit testing framework for .NET. It describes what NUnit is, how to install and use it, key concepts like test fixtures and methods, and provides an example of writing unit tests for a calculator class using NUnit's attributes. The document also covers running tests via the NUnit GUI or console, and examining the output XML file for test results.
This is a slide set with some information about programmer testing. It includes test doubles, mockit and junit.
Visit http://jpereira.eu/2011/12/03/programming-testing/ for more information. If you want to download these slides, please contact me (find my contacts on my blog)
The document provides guidelines for writing effective unit tests, including why they are important (ensuring code works as expected, protecting work from accidental breaks, serving as documentation), what should be tested (models, services, commands, utilities), when they should be written (before or after code via test-driven development), and how to write good tests (automated, repeatable, single-purpose, readable, isolated, avoiding duplication).
The document summarizes a presentation on software testing concepts and practices like test-driven development, terminology used in testing, popular tools used for testing, and resources for learning more about testing. The presentation covered topics like unit testing, integration testing, test-driven development principles and practices, popular IDEs, build tools, testing frameworks, mock object libraries, code coverage tools, load testing tools, acceptance testing tools, and continuous integration tools. It also demonstrated testing using an open-source medical record system.
The document discusses building a custom data source for Grafana using Vert.x to retrieve time series data stored in MongoDB. It describes how to connect Grafana and MongoDB through a three-tier architecture using a Vert.x microservice. The microservice would handle splitting requests, querying the database in parallel, aggregating results, and performing additional processing before returning data to Grafana. Vert.x is well-suited for this due to its asynchronous, reactive, and scalable nature. Sample code is available on GitHub to demonstrate the approach.
TAPOST Conference 2016 Riga // Better Products Faster: Let's bring the user i...icemobile
"Better products faster: let's bring the user in the userstory"
as presented at TAPOST conference in Riga 2016.
Why is it that everyone knows the importance of frequent user
testing, yet hardly anyone does it? Because user testing often is time
consuming, complex and expensive. It probably doesn’t fit in your
development process and thus feels like extra work.
To feel reassured you tell yourself to test with users once you have
something working, or at the very end of the process. This is strange,
because everybody knows that changing your product late in the
process will increase costs exponentially.
We created a way so that user testing saves time, improves the
quality and doesn’t cost a lot of money. Team driven, pragmatic and
no extra resources needed.
The talk will show how, with only 2 hours every sprint, we focused on
creating better products faster. We would love to share our learnings
and simple DIY tools that let you start user testing with your current
teams tomorrow!
Esta palestra foi apresentada no dia 25/08 durante a Expotec - 2016 http://www.expotec.org.br . Ela fala sobre a necessidade de gerenciar a rede, sobre Zabbix, Grafana e o plugin que permite a integração destes dois sistemas.
As one of the most requested features in our last survey, and one of the most active open GitHub issues, alerting in Grafana is both an exciting and contentious topic. This presentation details our approach to tackling the alerting question in Grafana, and what’s coming down the pipe to allow people to manage their alerts side-by-side with their visualizations.
Consumer Driven Contracts and Your Microservice ArchitectureMarcin Grzejszczak
TDD introduced many improvements into the development process, but in our opinion the biggest impact relates to code design. Looking at the code from the usage perspective (by first writing an acceptance test) allows us to focus on usability rather than concrete implementation. Unfortunately, we usually rest on our laurels not trying to uplift this practice to the architecture level.
This presentation will show you how you can use the Spring Cloud Contract Verifier functionality in order to have a fully automated solution to stub your HTTP / Messaging collaborators. Just by adding proper configuration, you'll surround the microservices you are testing with faked stubs that are tested against their producer, making much more realistic tests.
We will write a system using the CDC approach together with Spring Boot, Spring Cloud and Spring Cloud Contract Verifier. I'll show you how easy it is to write applications that have a consumer-driven API, allowing a developer to speed up the time for writing better quality software.
No need to wonder how the best on SlideShare do it. The Masters of SlideShare provides storytelling, design, customization and promotion tips from 13 experts of the form. Learn what it takes to master this type of content marketing yourself.
Each month, join us as we highlight and discuss hot topics ranging from the future of higher education to wearable technology, best productivity hacks and secrets to hiring top talent. Upload your SlideShares, and share your expertise with the world!
Not sure what to share on SlideShare?
SlideShares that inform, inspire and educate attract the most views. Beyond that, ideas for what you can upload are limitless. We’ve selected a few popular examples to get your creative juices flowing.
SlideShare is a global platform for sharing presentations, infographics, videos and documents. It has over 18 million pieces of professional content uploaded by experts like Eric Schmidt and Guy Kawasaki. The document provides tips for setting up an account on SlideShare, uploading content, optimizing it for searchability, and sharing it on social media to build an audience and reputation as a subject matter expert.
MUTANTS KILLER - PIT: state of the art of mutation testing system Tarin Gamberini
The document discusses mutation testing and the PIT mutation testing system. It begins with an overview of mutation testing, which involves making small changes to a program to generate mutant versions and running tests against these mutants to evaluate test effectiveness. The document then covers key concepts of PIT including the various mutators it uses to generate mutants and how it leverages test coverage to efficiently run tests against mutants. An example using a sample Ticket class demonstrates how PIT reports can identify weaknesses in tests by showing mutants that were killed or lived. The document promotes using mutation testing and PIT to evaluate test quality and improve test effectiveness at finding faults.
Software testing: an introduction - 2017XavierDevroey
Software testing involves dynamically verifying that a program behaves as expected on a finite set of test cases. This is done because exhaustively testing every possible case is not feasible. Unit testing involves testing individual program units such as classes through automated tests that make assertions about the output. JUnit is a unit testing framework for Java that uses annotations to identify test methods and make assertions about the results.
The document discusses the evolution of development testing. It describes how development testing has evolved from a practice focused on individual units to one that automates broader scenarios and exercises more control over the test context. The document also covers principles for writing high-quality tests, such as using structures that make tests readable and maintainable, and practices like helper methods and object builders that reduce complexity and improve test code quality.
The document discusses various aspects of unit testing including definitions, benefits, naming standards, and best practices. It provides definitions for terms like error, defect, failure. It outlines the benefits of unit testing like finding bugs early, enabling code refactoring. It discusses what should be tested like boundaries, error conditions. It also provides examples of good test names and guidelines for structuring and naming unit tests.
The document discusses unit testing and provides guidance on effective unit testing practices. It defines key terms like error, defect, and failure. It outlines the benefits of unit testing like finding defects earlier and maintaining stable code. It discusses naming conventions and frameworks for unit tests. It provides examples of different types of unit tests and guidelines for writing good unit tests that are independent, fast, and test all functionality. The document emphasizes testing boundary conditions and errors as well as documenting test cases.
This was a workshop given on the UTN University, for the Software Engineering students. The idea is to give a brief explanation about TDD, and how to use it.
software testing is necessary to make sure the product or application is defect free, as per customer specifications. Software testing identifies fault whose removal increases the software Quality and Increases the software reliability.Testing effort is directly proportional to the complexity of the program.
This document provides an overview of software testing concepts. It discusses different types of testing like unit testing, functional testing, error path testing, boundary value testing, and equivalence partitioning. It also covers test strategies like golden path testing, black box testing, and white box testing. The purpose of a tester is explained as quantifying risk to make decisions that improve confidence and quality.
The document discusses various aspects of software testing such as test cases, test plans, test scenarios, testworthy criteria, testing types including functional, non-functional, manual and automated testing. It also covers topics like traceability matrix, test automation frameworks, fuzzing, mutation testing and references various standards and research papers related to software testing.
The document discusses various concepts related to software errors, faults, failures, and testing. It defines that an error is made during development, a fault is the manifestation of an error in the code, and a failure occurs when the fault is triggered. Testing involves exercising the software with test cases to find failures or demonstrate correct execution. There are two main approaches to identifying test cases - functional testing based on specifications and structural testing based on code. Both approaches are needed to fully test the software.
Making the Unstable Stable - An Intro To TestingCameron Presley
Does it always seem like bugs you've fixed keep coming back? Does it seem like when you fix one bug, two more crop up? What if I were to tell you there's a better way?
In this presentation, we're going to explore how to make a code base more stable by using automated testing. To start, we'll explore the business case of why you should be writing tests by looking at industry studies and personal experience. From there, we'll look at the fundamentals of testing by talking about the pros/cons of unit, integration, and UI testing. Finally, we'll look at some resources to learn how to write tests.
Intended for developers who are new to testing, by the end of this presentation, you will understand why you should write tests, and will have the concepts and tools to get started.
Prerequisites
Some knowledge with an Object-Oriented language would be beneficial, but not required.
Beyond TDD: Enabling Your Team to Continuously Deliver SoftwareChris Weldon
Many project teams have adopted unit testing as a necessary step in their development process. Many more use a test-first approach to keep their code lean. Yet, far too often these teams still suffer from many of the same impediments: recurrent integration failures with other enterprise projects, slow feedback with the customer, and sluggish release cycles. With a languishing feedback loop, the enterprise continues to put increasing pressure on development teams to deliver. How does an aspiring agile team improve to meet the demands of the enterprise?
Continuous integration is the next logical step for the team. In this talk, you’ll learn how continuous integration solves intra and inter-project integration issues without manual overhead, the value added by continuous integration, and how to leverage tools and processes to further improve the quality of your code. Finally, we discuss the gold standard of agile teams: continuous deployment. You’ll learn how continuous deployment helps close the feedback loop with your customers, increases visibility for your team, and standardizes the deployment process.
#DOAW16 - DevOps@work Roma 2016 - Testing your databasesAlessandro Alpi
In these slides we will speak about how to unit test our programmability in SQL Server and how to move from a manual process to an automated one in order to achieve the goals of DevOps
The document discusses software testing processes and techniques. It covers topics like test case design, validation testing vs defect testing, unit testing vs integration testing, interface testing, system testing, acceptance testing, regression testing, test management, deriving test cases from use cases, and test coverage. The key points are that software testing involves designing test cases, running programs with test data, comparing results to test cases, and reporting test results. Different testing techniques like unit testing, integration testing, and system testing address different levels or parts of the system. Test cases are derived from use case scenarios to validate system functionality.
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...TEST Huddle
We often claim the purpose of testing is to verify that software meets a desired level of quality. Frequently, the term “testing” is associated with checking for functional correctness. However, in large, complex software systems with an established user-base, it is also important to verify system constraints such as backward compatibility, reliability, security, accessibility, usability. Kim Herzig from Microsoft explores these issues with the latest webinar on test Huddle.
The document discusses several automated testing tools:
- PHPUnit is a unit testing framework for PHP that allows writing and running automated tests. It helps ensure code quality and finds regressions.
- Selenium is a suite of tools for automating web application testing across browsers and platforms. It includes the Selenium IDE, RC, and Grid.
- Phing is a build system based on Apache Ant for PHP projects. It uses XML build files and PHP tasks to provide an extensible and flexible build framework.
What is UFT? HP's unified functional testing.Confiz
Unified Functional Testing (UFT) is HP's main automated functional testing tool that allows users to integrate QuickTest Pro with Service Test. UFT enables automated functional testing of applications. It provides various panes like the Solution Explorer, Toolbox, Canvas, and Output panes to design and run tests. Users can create simple tests with activities like Replace String and Concatenate String, connect the test steps, and map data from multiple sources to test functionality.
Data flow analysis is a type of static code analysis that examines how values are propagated through a program. It is more effective than pattern matching or regular static analysis at finding defects related to interactions between methods and classes that may be difficult to uncover through testing alone. Static analysis tools using data flow analysis can simulate execution paths to detect potential issues without requiring the code to be compiled and run. Developers are encouraged to use static testing tools to catch defects early in development, as prevention of bugs is more efficient than finding and fixing them later.
4th lecture on Software Testing given to KTU students.Sergejus Bartos
This document discusses software testing concepts including test cases, defects, and metrics. It defines test cases as conditions to determine if an application works as intended. Testing activities include planning, analysis, design, implementation, execution, evaluation, and closure. Metrics can track test case execution status and defects over time and releases to evaluate progress. Defects are errors that cause failures to meet expectations. Attributes of defects include severity, priority, status, and resolution. Combined metrics can show test execution results alongside defect creation and resolution. Writing clear, reproducible defect reports is important to facilitate fixing issues. Testing can find bugs but not prove their absence.
Similar to Must.kill.mutants. TopConf Tallinn 2016 (20)
Takashi Kobayashi and Hironori Washizaki, "SWEBOK Guide and Future of SE Education," First International Symposium on the Future of Software Engineering (FUSE), June 3-6, 2024, Okinawa, Japan
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Łukasz Chruściel
No one wants their application to drag like a car stuck in the slow lane! Yet it’s all too common to encounter bumpy, pothole-filled solutions that slow the speed of any application. Symfony apps are not an exception.
In this talk, I will take you for a spin around the performance racetrack. We’ll explore common pitfalls - those hidden potholes on your application that can cause unexpected slowdowns. Learn how to spot these performance bumps early, and more importantly, how to navigate around them to keep your application running at top speed.
We will focus in particular on tuning your engine at the application level, making the right adjustments to ensure that your system responds like a well-oiled, high-performance race car.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j
Dr. Jesús Barrasa, Head of Solutions Architecture for EMEA, Neo4j
Découvrez les dernières innovations de Neo4j, et notamment les dernières intégrations cloud et les améliorations produits qui font de Neo4j un choix essentiel pour les développeurs qui créent des applications avec des données interconnectées et de l’IA générative.
Atelier - Innover avec l’IA Générative et les graphes de connaissancesNeo4j
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Allez au-delà du battage médiatique autour de l’IA et découvrez des techniques pratiques pour utiliser l’IA de manière responsable à travers les données de votre organisation. Explorez comment utiliser les graphes de connaissances pour augmenter la précision, la transparence et la capacité d’explication dans les systèmes d’IA générative. Vous partirez avec une expérience pratique combinant les relations entre les données et les LLM pour apporter du contexte spécifique à votre domaine et améliorer votre raisonnement.
Amenez votre ordinateur portable et nous vous guiderons sur la mise en place de votre propre pile d’IA générative, en vous fournissant des exemples pratiques et codés pour démarrer en quelques minutes.
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeAftab Hussain
Understanding variable roles in code has been found to be helpful by students
in learning programming -- could variable roles help deep neural models in
performing coding tasks? We do an exploratory study.
- These are slides of the talk given at InteNSE'23: The 1st International Workshop on Interpretability and Robustness in Neural Software Engineering, co-located with the 45th International Conference on Software Engineering, ICSE 2023, Melbourne Australia
Graspan: A Big Data System for Big Code AnalysisAftab Hussain
We built a disk-based parallel graph system, Graspan, that uses a novel edge-pair centric computation model to compute dynamic transitive closures on very large program graphs.
We implement context-sensitive pointer/alias and dataflow analyses on Graspan. An evaluation of these analyses on large codebases such as Linux shows that their Graspan implementations scale to millions of lines of code and are much simpler than their original implementations.
These analyses were used to augment the existing checkers; these augmented checkers found 132 new NULL pointer bugs and 1308 unnecessary NULL tests in Linux 4.4.0-rc5, PostgreSQL 8.3.9, and Apache httpd 2.2.18.
- Accepted in ASPLOS ‘17, Xi’an, China.
- Featured in the tutorial, Systemized Program Analyses: A Big Data Perspective on Static Analysis Scalability, ASPLOS ‘17.
- Invited for presentation at SoCal PLS ‘16.
- Invited for poster presentation at PLDI SRC ‘16.
SOCRadar's Aviation Industry Q1 Incident Report is out now!
The aviation industry has always been a prime target for cybercriminals due to its critical infrastructure and high stakes. In the first quarter of 2024, the sector faced an alarming surge in cybersecurity threats, revealing its vulnerabilities and the relentless sophistication of cyber attackers.
SOCRadar’s Aviation Industry, Quarterly Incident Report, provides an in-depth analysis of these threats, detected and examined through our extensive monitoring of hacker forums, Telegram channels, and dark web platforms.
Revolutionizing Visual Effects Mastering AI Face Swaps.pdfUndress Baby
The quest for the best AI face swap solution is marked by an amalgamation of technological prowess and artistic finesse, where cutting-edge algorithms seamlessly replace faces in images or videos with striking realism. Leveraging advanced deep learning techniques, the best AI face swap tools meticulously analyze facial features, lighting conditions, and expressions to execute flawless transformations, ensuring natural-looking results that blur the line between reality and illusion, captivating users with their ingenuity and sophistication.
Web:- https://undressbaby.com/
Why Mobile App Regression Testing is Critical for Sustained Success_ A Detail...kalichargn70th171
A dynamic process unfolds in the intricate realm of software development, dedicated to crafting and sustaining products that effortlessly address user needs. Amidst vital stages like market analysis and requirement assessments, the heart of software development lies in the meticulous creation and upkeep of source code. Code alterations are inherent, challenging code quality, particularly under stringent deadlines.
Artificia Intellicence and XPath Extension FunctionsOctavian Nadolu
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j
Dr. Jesús Barrasa, Head of Solutions Architecture for EMEA, Neo4j
Découvrez les dernières innovations de Neo4j, et notamment les dernières intégrations cloud et les améliorations produits qui font de Neo4j un choix essentiel pour les développeurs qui créent des applications avec des données interconnectées et de l’IA générative.
GraphSummit Paris - The art of the possible with Graph TechnologyNeo4j
Sudhir Hasbe, Chief Product Officer, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
3. About me
• Gerald Mücke
• Founder & CEO of DevCon5 GmbH
• Passionate Software Developer
• Focal Points
• Performance Analysis
• Test Automation
• Mutation Testing
• DevOps
• Using mutation testing for > 2 years
5. «Quality Assurance»
“is a way of preventing mistakes or defects in
manufactured products and avoiding problems when
delivering solutions or services to customers”
(Wikipedia)
6. «manufactured» products
• «The process of converting raw materials,
components, or parts into finished goods that meet a
customer's expectations or specifications.»
• Most of the critical code is written manually
• Raw Materials
• existing software or parts of it
• brain, ideas, knowledge, experience,
requirements,
• Every product is unique
(may look similar, though)
7. «Preventing» defects
• Defects are «created» in development
• Can not be prevented,
it’s human to make mistakes
• Could be detected:
the earlier, the better
• Defects manifest in production
• Or during test
• Can be prevented:
the earlier, the better
8. Sources of a
Product
• Internal Development
• QA embeddable
• QA along the pipe line
• Quality is shared effort
• More Easy to change or
influence
• External Development
• Software Vendors
• more effort required for
dedicated QA
• Less easy to change
• handoff «Waterfall» style
10. Real-Life Bugs
if( isThreadSafe() ) {
computeSingleThreaded();
} else {
computeMultiThreaded();
} Made it to Production,
Performance Impact:
500% Duration of Day
End Processing
11. Real-Life Bugs
if( ! isDevelopmentMode() ){
collectProfileDataAndSendDeveloperReport();
}
In Production,
Impact:
20% Performance loss
Compliance Violation
12. Real-Life Bugs
void function(LocalDate begin, LocalDate end, LocalDate minFrom, ...) {
//...
outerLoop:
while( it.hasNext() ) {
Object current = it.next();
Local from = funcA(current);
Local upto = funcB(current);
while(true){
if( ! isBeforeOrEqual( from , upto ) ) {
continue outerLoop;
}
if( condY(from, minFrom) ) {
from = DateUtil.addDaysToDate(upto, 1);
upto = DateUtil.getLastOfMonth(from);
from = DateUtil.min(new LocalDate[]{ end, from});
upto = DateUtil.min(new LocalDate[]{ end, upto});
void function(LocalDate begin, LocalDate end, LocalDate minFrom, ...) {
//...
outerLoop:
while( it.hasNext() ) {
Object current = it.next();
Local from = funcA(current);
Local upto = funcB(current);
while(true){
if( ! isBeforeOrEqual( from , upto ) ) {
continue outerLoop;
}
if( condY(from, minFrom) ) {
from = DateUtil.addDaysToDate(upto, 1);
upto = DateUtil.getLastOfMonth(from);
from = DateUtil.min(new LocalDate[]{ end, from});
upto = DateUtil.min(new LocalDate[]{ end, upto});
17. Good Decisions are based on
Information
Simple
Metrics
Number of Unit Tests
Line Coverage
Branch Coverage
Complex
Test Results
Code Review
Static Code Analysis
…
17
18. Code Coverage
Information about what elements of a
product have been touched by a test.
Common Coverage Metrics
Line Coverage
Condition Coverage
Branch Coverage
Semantics ?
Code
Test
Test Oracle
21. Arcance
Arts
To most of the Non-Developers
Software Development
seems increasingly like an
arcane art
Languages, Paradigms,
Frameworks
Algorithms & Datastructures
O(n), ByteCode, Lambdas...
23. Quality Gates
Decision Point
List of Checks when the Product is
ready to be released
Based on information
Based on agreement between
stakeholders
Part of Definition of Done
Evolves over time
Should not replace human judgement
25. Perspectives
Programmers
• Implement the Solution
• Provide indication the solution is
working
• claim, they did it «right»
Testers
• Show if and how the solution will fail
• have to provide information for
stakeholders to make informed
decisions
• usually don’t understand arcane arts
26. Checking vs. Testing
Things we
are aware of
but don‘t
understand
Things we
are aware of
and
understand
Things we
are neither
aware of nor
understand
Things we
understand
but are not
aware of
28
Understanding
Awareness
Unknowns Knowns
UnknownKnown
CheckingTesting
Automated
Checking
27. The Testing Pyramid of Functional
Tests
UI
Tests
Integration Tests
Unit Tests
DegreeofAutomation
36. Mutation Testing – History
Mutations testing injects faults,
based on rules, into a product
to verify if the test suite
is capable of finding it.
Fault injection technique
Concept is known since ~1970
First implementation of a mutation testing tool in 1980
Most of the time it was subject to academic research only
Recently, with increasing processing power, there is a growing interest
More academic research ongoing
Practical tooling available
37. Mutation Testing – Some Theory
Mutation testing is a special form of Fault Injection
Based on two hypotheses
1: Most of the software faults are due to small syntactic errors
2: Simple faults can cascade to more emergent faults
Assumption:
“if a mutant was introduced without the behavior of the test suite being affected, this indicated either
that the code that had been mutated was never executed (dead code) or that the test suite was unable
to locate the faults represented by the mutant” (Wikipedia)
38. Mutation Testing - Definitions
Mutant
a variation P’ of the product P created by
applying a mutant operator m
P’ = m(P)
Killed Mutant
a variation P’ in which a test has found at
least ONE error
Live Mutant
a variation P’ in which a test has found NO
errors
Mutation Operators
A function m() that creates a variation of the
Product P by applying a set of modification
rules
Inject Faults into the Product
Based on Bug Taxonomies
Mutation Score
Number of Killed Mutants / Total number of
Mutants
Also Called Mutation Coverage
39. Some more definitions
Equivalent Mutation
a variation P’ that is semantically
identical to P
Duplicate Mutation
a variation P’ that is equivalent to another
variation P’’
Weak Mutation
Fault does not lead to incorrect output
Strong Mutation
Fault propagates to incorrect output
Unstable Mutation
Any test can find the mutations generated
by it
High-Order Mutants
Mutants that are defined by a set of Low-
Level Mutants
Subsumed Mutants
One mutant subsumes another if at least
one test kills the first and every test that
kills the first also kills the second.
42. Approaches to Mutation
Testing
Byte Code Mutation
Can be done on-the-fly
Faster to apply and execute
Might be affected by compiler optimizations
Source Code Mutation
Requires recompilation after every change
Takes very long
Is not affected by compiler optimizations
Higher Level Mutations
Configuration, Architecture, Specification,
Use/Business Case, ...
No Tooling Support (yet?)
43. Mutation Testing Phases
Mutant generation
analyzing classes and generate mutations for them
Test selection
selecting the tests to run against the mutations
Mutant insertion
loading the mutations into a JVM / Runtime Environment
Mutant detection
executing tests against the loaded mutants
44. Mutation Testing 101
Modify your code
(Mutant generation)
Re-Run the Test
(Test selection + Loading)
Check if test is failing
(Detection)
class Builder {
Builder withValue(String in) {
this.value = in;
this.value = in;
return this;
}
}
@Test
public void testLeft() {
Builder b = b.withValue(„one");
assertNotNull(b);
}
If test is Green it‘s a Fail!!!
45. Related Techniques
Bebugging / Fault Seeding
randomly adding bugs, programmers are tasked to find them
Fuzzing
Injecting Faults into Test Data
For Operations:
Chaos Monkey (Simian Army, Netflix)
Randomly terminating running processes or servers to test operational procedures or fitnesse
47. Tool: PIT
Mutation Testing for Java / JVM
Operates on ByteCode modification
easy to use - works with ant, maven, gradle and
others
~ 20 Mutation Operands for altering your
code
Parallel execution
fast - can analyze in minutes what would take
earlier systems days
Active Community
actively developed & supported
Mature Tooling
Good Documentation
HTML & XML Reports
49. Interpreting Results
Live Mutants
Reflects unspecified behavior
superfluous code / unrequired semantics
Could be an actual bug that is not covered by the test suite
Could be equivalent mutation
Killed by TimeOut or MemError
Could be “real kill” (i.e. endless loop)
Could be still alive
Mutation Score
Gives an indication of the overall quality of you test suite
50. Unit Test Maturity Model
Level Description
0 No Test
1 We have a test
2 1 + We have > 0% Line Coverage
3 2 + We have > 50% Branch Coverage
4 3 + We have at least 1 effective assertion per test
5 4 + We have > 80 % Mutation Coverage
52. Timeouts
Mutating abort-conditions of loops can cause timeouts
Loop runs endlessly
Mutation is effectively killed
Mutation might not be killed
Loop runs longer (i.e. counter underrun / overrun) -> Mutation might eventually survive
Your System is just too slow / the tests takes too long
When to stop the test?
Will the test fail?
If a loop runs longer, the machine performance is important for choosing the timeout.
53. Limitations
Fault Coverage
~¼ of real faults are not coverable by mutation testing
Mutation Score
PIT does not recognize subsumed or equivalent mutations
mutation score may not be “academically” precise – context matters!
Mutation Operators
PIT has no Java concurrency mutation operands
PIT has no high-order mutation operands
PIT has no Java-language specific mutation operands
Techniques
PIT does not support sampling
54. Value has it’s cost
Mutation Testing is computationally expensive
Duration of a mutation test depends on
number of tests
test suite execution time
number of mutation operators
Processing Power
Basically:
D = xn
n = number of mutation operators
x = number of tests
55. Deviation in Mutation
Score
Impact of Mutation Operator
Selection
Size of Codebase
Computational
Effort
Mutations
Found
More Operands
Less Operands
56. Mutation Analysis of Large Code
bases
Computational
Effort
Time Cap
Size of Codebase
Break into Chunks
Mo Tu We Th Fr
57. Other
techniques
Incremental Analysis
Based on historical data
Only test code that has changed
Increases deviation
Sampling
Good for Mutation Scoring
Increases Deviation
No support for sampling in PIT
58. Challenges of Mutation Testing
Redundant Mutants
Subsuming
Duplicates
Equivalent
Equivalent Detection
Current Algorithm achieves 50% detection rate in
Research
High Order Mutations
Computational Cost
Mutation and Test Selection
Equivalent
Mutants
Subsuming/
Duplicate
Mutants
60. Some Advices
Unit Tests are usually owned by development
challenge them with Mutation Testing!
It’s NOT unit tested until mutation tested.
Don’t go on a killing spree
Set achievable goals for mutation score
Triage surviving mutants
A mutation score > 0.8 is considered good (it depends…)
Determine mutation score regularly in a sensible intervall
Every build vs. Every release
Use historical data & SCM support
Find concrete mutants as needed
Adjust mutators & scope
61. Use Cases
Finding Gaps in Test Suite
Testing Highly Exposed Code
Algorithms and Calculations
Security-related code
Transaction-related code
Assessing Test Suites / Testing Strategies / Methodologies
By comparing the mutation scores, i.e.
Developing Test Suites for Legacy Code
Finding semantic hotspots
Finding gaps in Test Suite
Forced to break Code Base into more manageable pieces
Minimizing Test Suites
Reduce number of Tests while keeping mutation score stable
! Reduces the effectiveness of the suite for detecting real faults
63. Takeaways
Don’t trust your Unit Tests unless you mutation-tested it.
Mutation Testing is the practice to find bugs in your test suite
Forget about other coverage metrics
Cheap to get, but next to no value
Include Mutation Testing in your project.
Always.
Use it with common sense
don’t go on a killing spree.
For Java
PIT is the tool to use.