As metrics go, Test Coverage is a real bad one. Not only is it easy to game on purpose, 100% does not guarantee anything. Fortunately, Mutation Testing can help!
Codemash - Mutation testing to the rescue of your testsNicolas Fränkel
Unit testing ensures your production code is relevant. But what does ensure your testing code is relevant? Come discover mutation testing and make sure your never forget another assert again. In the realm of testing, the code coverage metrics is the most often talked about. However, it doesn’t mean that the test has been useful or even that an assert has been coded. Mutation testing is a strategy to make sure that the test code is relevant. In this talk, I will explain how Code Coverage is computed and what its inherent flaw is. Afterwards, I will describe how Mutation Testing work and how it helps pointing out code that is tested but leave out corner cases. I will also demo PIT, a Java production-grade framework that enables Mutation Testing.
TestCon Europe - Mutation Testing to the Rescue of Your TestsNicolas Fränkel
Unit testing ensures your production code is relevant. But what does ensure your testing code is relevant? Come discover mutation testing and make sure your never forget another assert again.
In the realm of testing, the code coverage metrics is the most often talked about. However, it doesn’t mean that the test has been useful or even that an assert has been coded. Mutation testing is a strategy to make sure that the test code is relevant.
In this talk, Nicolas will explain how Code Coverage is computed and what its inherent flaw is. Afterwards, he will describe how Mutation Testing work and how it helps pointing out code that is tested but leave out corner cases. He will also demo PIT, a Java production-grade framework that enables Mutation Testing.
This document discusses mutation testing as a way to improve test quality beyond just code coverage. Mutation testing works by making small changes or "mutations" to the production code and seeing if tests can detect the changes. If tests do not detect a mutation, the mutant is considered surviving and indicates a problem with test quality. The document provides examples of how mutation testing can identify issues and recommends tools like PIT for performing mutation testing in Java projects. It also discusses techniques for making mutation testing more practical for continuous integration like limiting the scope of analysis.
GeeCON - Improve your tests with Mutation TestingNicolas Fränkel
This document discusses mutation testing as a way to improve test quality. Mutation testing works by modifying the source code slightly (creating mutants) and checking if tests can detect the changes. This is more effective than just measuring code coverage, which can be gamed. The document recommends using a tool like PIT to perform mutation testing, and provides tips on configuring it for performance when running as part of test suites. While mutation testing has limitations, it can help prioritize testing of critical or complex code.
ConFoo - Improve your tests with mutation testingNicolas Fränkel
The document discusses mutation testing as a way to ensure the quality of test code. Mutation testing works by modifying the source code in small ways (called mutations) and checking if tests can detect the changes. This helps identify weaknesses or gaps in test coverage that code coverage metrics alone may miss. The document recommends using a tool like PIT for Java projects, which automatically generates mutations and analyzes whether tests can kill the mutated code or if mutants survive undetected. Mutation testing is slow but can help improve test quality by finding overlooked cases.
Voxxed Days Athens - Improve your tests with Mutation TestingNicolas Fränkel
The document discusses mutation testing as a way to ensure the quality of test code. Mutation testing works by modifying the source code in small ways (called mutants) and checking if tests can detect the changes. If tests still pass despite a mutant, it means the tests are not properly validating that code section. The document recommends Java tools like PIT for mutation testing and describes common mutators. It acknowledges mutation testing is slow and imperfect but can still help improve test quality by finding overlooked cases. The document concludes that while mutation testing is useful, a variety of test types are needed to fully ensure code quality.
Craft-Conf - Improve your Tests with Mutation TestingNicolas Fränkel
You probably have a lot of tests but their goal is to ensure quality of your production code. What about the quality of your testing code? Code coverage? Nah!
Joker - Improve your tests with mutation testingNicolas Fränkel
This document discusses mutation testing as a way to evaluate test quality beyond just code coverage metrics. Mutation testing works by modifying the production code in small ways (called mutants) and then re-running the test suite to see if the tests can detect the changes. If the tests still pass despite a mutant, it indicates weaknesses in the test suite. The document provides an overview of how mutation testing works and tools like PIT that can implement it for Java projects. It also discusses strategies for making mutation testing practical for continuous integration like limiting scopes and leveraging incremental analysis between runs.
Codemash - Mutation testing to the rescue of your testsNicolas Fränkel
Unit testing ensures your production code is relevant. But what does ensure your testing code is relevant? Come discover mutation testing and make sure your never forget another assert again. In the realm of testing, the code coverage metrics is the most often talked about. However, it doesn’t mean that the test has been useful or even that an assert has been coded. Mutation testing is a strategy to make sure that the test code is relevant. In this talk, I will explain how Code Coverage is computed and what its inherent flaw is. Afterwards, I will describe how Mutation Testing work and how it helps pointing out code that is tested but leave out corner cases. I will also demo PIT, a Java production-grade framework that enables Mutation Testing.
TestCon Europe - Mutation Testing to the Rescue of Your TestsNicolas Fränkel
Unit testing ensures your production code is relevant. But what does ensure your testing code is relevant? Come discover mutation testing and make sure your never forget another assert again.
In the realm of testing, the code coverage metrics is the most often talked about. However, it doesn’t mean that the test has been useful or even that an assert has been coded. Mutation testing is a strategy to make sure that the test code is relevant.
In this talk, Nicolas will explain how Code Coverage is computed and what its inherent flaw is. Afterwards, he will describe how Mutation Testing work and how it helps pointing out code that is tested but leave out corner cases. He will also demo PIT, a Java production-grade framework that enables Mutation Testing.
This document discusses mutation testing as a way to improve test quality beyond just code coverage. Mutation testing works by making small changes or "mutations" to the production code and seeing if tests can detect the changes. If tests do not detect a mutation, the mutant is considered surviving and indicates a problem with test quality. The document provides examples of how mutation testing can identify issues and recommends tools like PIT for performing mutation testing in Java projects. It also discusses techniques for making mutation testing more practical for continuous integration like limiting the scope of analysis.
GeeCON - Improve your tests with Mutation TestingNicolas Fränkel
This document discusses mutation testing as a way to improve test quality. Mutation testing works by modifying the source code slightly (creating mutants) and checking if tests can detect the changes. This is more effective than just measuring code coverage, which can be gamed. The document recommends using a tool like PIT to perform mutation testing, and provides tips on configuring it for performance when running as part of test suites. While mutation testing has limitations, it can help prioritize testing of critical or complex code.
ConFoo - Improve your tests with mutation testingNicolas Fränkel
The document discusses mutation testing as a way to ensure the quality of test code. Mutation testing works by modifying the source code in small ways (called mutations) and checking if tests can detect the changes. This helps identify weaknesses or gaps in test coverage that code coverage metrics alone may miss. The document recommends using a tool like PIT for Java projects, which automatically generates mutations and analyzes whether tests can kill the mutated code or if mutants survive undetected. Mutation testing is slow but can help improve test quality by finding overlooked cases.
Voxxed Days Athens - Improve your tests with Mutation TestingNicolas Fränkel
The document discusses mutation testing as a way to ensure the quality of test code. Mutation testing works by modifying the source code in small ways (called mutants) and checking if tests can detect the changes. If tests still pass despite a mutant, it means the tests are not properly validating that code section. The document recommends Java tools like PIT for mutation testing and describes common mutators. It acknowledges mutation testing is slow and imperfect but can still help improve test quality by finding overlooked cases. The document concludes that while mutation testing is useful, a variety of test types are needed to fully ensure code quality.
Craft-Conf - Improve your Tests with Mutation TestingNicolas Fränkel
You probably have a lot of tests but their goal is to ensure quality of your production code. What about the quality of your testing code? Code coverage? Nah!
Joker - Improve your tests with mutation testingNicolas Fränkel
This document discusses mutation testing as a way to evaluate test quality beyond just code coverage metrics. Mutation testing works by modifying the production code in small ways (called mutants) and then re-running the test suite to see if the tests can detect the changes. If the tests still pass despite a mutant, it indicates weaknesses in the test suite. The document provides an overview of how mutation testing works and tools like PIT that can implement it for Java projects. It also discusses strategies for making mutation testing practical for continuous integration like limiting scopes and leveraging incremental analysis between runs.
DevExperience - Improve your tests with mutation testingNicolas Fränkel
Unit testing ensures your production code is relevant. But what does ensure your testing code is relevant?
In the realm of testing, the code coverage metrics is the most often talked about. However, it doesn't mean that the test has been useful or even that an assert has been coded. Mutation testing is a strategy to make sure that the test code is relevant.
I.T.A.K.E Unconference - Mutation testing to the rescue of your testsNicolas Fränkel
This document summarizes mutation testing and how it can be used to test test quality. Mutation testing works by modifying the production code in small ways (e.g. changing a relational operator) and then running the test suite to see if the tests can detect the mutations. This is more effective than just measuring code coverage at ensuring tests are rigorous. The document outlines some Java tools for mutation testing like PIT and discusses how mutation testing can identify weaknesses in a test suite. Drawbacks like performance issues are also covered, along with ways to mitigate slowness like incremental analysis.
Testing, Learning and Professionalism — 20171214David Rodenas
Two hours talk about testing, its history, its meaning and propose, a small guide of good practices, and what it is supposed to be a good professional and what is expected from us the professionals.
There are lots of talks about testing: they talk about syntax, methodologies, tools. But there is usually a missing point: Why it is important to do a test? What is important to test? What is not important to test? How to do testing?
There lots of examples in plnker just to see each step, and many surprises.
This talk is good for beginners, but also for some seasoned people that just want to light up some of the ideas that they might have been hatching.
Spoiler alert: testing will save you development time.
(automatic) Testing: from business to university and backDavid Rodenas
This talk cares about the fundamentals of testing, a little bit history of how the professional community developed what we currently know as testing, but also about why I should care about testing? why is it important to do a test? What is important to test? What is not important to test? How to do testing?
There some examples in plnker just to see each step, and many surprises.
This talk also compares what people learned in the Computer Sciences and Engineering degrees and what people does in testing. It gives some tips to catch up with current state of art and gives some points to start changing syllabus to make better engineers.
This talk is good for beginners, teachers, bosses, but also for seasoned techies that just want to light up some of the ideas that they might have been hatching.
Spoiler alert: testing will save you development time and make you a good professional.
This document provides an introduction to unit testing, including basics like why unit tests are useful, what a test case consists of, and how tests are processed and their lifecycle. It then demonstrates how to run tests from the command line and in PhpStorm. Finally, it discusses how to write real unit tests for custom code by adding test cases and removing dependencies through mocking or prophecies. The overall document serves as a tutorial for beginners on getting started with unit testing.
The document discusses mutation testing and provides tips for implementing it. It explains different types of mutations like statement, value, and decision mutations. It also discusses how to calculate mutation score and challenges like cost. Examples are provided using tools like Pitest and Stryker. Advice is given around using mocks, refactoring complex code, and targeting tests to improve mutation testing.
How to quickly add a safety net to a legacy codebaseNelis Boucké
Did you ever inherited a horrible legacy system to maintain and you were scared to introduce bugs touching it? If you did (who never did?) then this post is for you.
You can find the accompanying video and blogpost soon on
Must.Kill.Mutants. Agile Testing Days 2017Gerald Muecke
The document discusses quality assurance and mutation testing. It defines quality assurance as preventing defects in manufactured products. It then discusses mutation testing, which involves injecting faults into code based on mutation operators to verify if tests can detect the faults. The document provides examples of mutation operators and tools for different programming languages. It notes that while mutation testing has limitations, it is valuable for finding bugs in test suites and improving test quality. It recommends including mutation testing in projects and using it with common sense to iteratively improve test suites.
DOAN DuyHai – Cassandra: real world best use-cases and worst anti-patterns - ...NoSQLmatters
This document discusses Cassandra use cases and anti-patterns. It describes queue-like designs, intensive updates on the same column, and designing around a dynamic schema as anti-patterns that can lead to failures. Rate limiting, fraud prevention, and account validation are provided as examples of good use cases. Key-value modeling, clustering, compaction strategies, and time-to-live features are also overviewed.
Unit Testing like a Pro - The Circle of PurityVictor Rentea
Best practices on designing unit tests, designing testable production code, a glimpse of TDD, using mocks and isolating pure functions for easy testing. Talk distilled from http://victorrentea.ro/#unit-testing
Held at VoxxedDays Bucharest in March 2019.
The document discusses how to improperly conduct a benchmark test in order to fail. Some ways to fail include measuring the wrong metrics, using an improper testing methodology, and testing in the wrong environment. The document also provides examples of puzzles to demonstrate benchmarking issues and discusses the concept of "coordinated omission" where aspects of the system are omitted from measurement.
Break up the Monolith: Testing MicroservicesMarcus Merrell
Microservices is more than a buzzword: it’s an industry-wide tidal wave. Companies are spending millions to break up monoliths and spin up microservices, but they usually only involve QA at the very end. This talk centers around real-world experiences, posing questions you can ask your developers/product people, and offering solutions for you to help make your service more discoverable, more testable, and easier to release.
Automation for Anyone at Nutanix NEXT 2017 USChris Wahl
Are you wondering how to solve repetitive tasks with software automation, but you struggle every time you see the word “developer” or “code?” Do you know what APIs are and how they make these tasks easy to solve? In this session, we’ll explore the framework that can apply to mundane tasks (i.e. PowerShell, Pester), and we’ll discuss what open source tools are available to help solve these problems. Walk away with the advice you need to get started!
This document summarizes a presentation about the open source Verilog simulator Verilator. It discusses why open source simulation is useful, provides an introduction and overview of Verilator's features, and addresses questions about getting started with and using Verilator. The presentation also covers Verilator's performance advantages over other simulators, considerations for whether Verilator may be suitable, its future development, and how users can contribute back to the project.
White-box Unit Test Generation with Microsoft IntelliTestDávid Honfi
This document discusses white-box unit test generation using Microsoft IntelliTest. It begins with an overview of symbolic execution and how IntelliTest uses dynamic symbolic execution to generate test cases by exploring paths in the code. It then demonstrates IntelliTest's capabilities with examples. The document also discusses challenges with symbolic execution-based test generation and proposes solutions like visualizing the symbolic execution process and automatically isolating environment dependencies through source code transformations.
In this presentation, Jared Atkinson and Jonathan Johnson discuss the problem that many security professionals are facing today. How exactly do I know if my detection will actually detect the thing I want to detect? We discuss the importance of testing telemetry coverage and using abstraction to build a representative sample set of Atomic tests to validate detection coverage.
Scripts used in presentation can be found below:
Process Access: https://gist.github.com/jaredcatkinson/9c7a1af2261a752432230a4148ecfe02
Process Read: https://github.com/redcanaryco/AtomicTestHarnesses/blob/master/Windows/TestHarnesses/T1003.001_DumpLSASS/DumpLSASS.ps1
This document discusses non-idempotent-outcome (NIO) tests, which are a type of flaky test that deterministically passes when run once but fails when run twice. The document defines NIO tests and describes how detecting and fixing them can help prevent order-dependent flaky tests. It is reported that NIO tests are prevalent, with 361 detected across 127 Java test suites and 1006 Python projects. Fixes were proposed for 268 NIO tests and 192 were accepted by developers.
A conjecture, even when has been accepted during a long period of time, is not even still a proof. This presentation tries to explain how developers unconsciously create and establish paths that can be taken by their human errors in order to become system failures, and how mutation testing technique is used to effectively remove these undesired paths.
Keynote at the 5th Workshop on Validation, Analysis, and Evolution of Software Tests (VST 2022)
Website: https://rramler.github.io/vst2022/
Abstract: Nowadays, Artificial intelligence (AI) plays a critical role in automating different human-intensive tasks, including software engineering tasks. Since the late 70s, researchers have proposed automated techniques to automatically generate test data (fuzzing) or test suites (test suite generation). Proposed techniques span from simple heuristics to more advanced AI-based techniques and evolutionary intelligence in particular. While recent studies have shown that these techniques achieve high coverage and find bugs, generated tests can be hard to understand and maintain. This talk will provide an overview and reflection on state-of-the-art techniques, open challenges, and research opportunities towards more accessible tests that are easy to integrate within the DevOps cycle. To this aim, the talk will cover relevant application domains, including "traditional" software and emerging cyber-physical systems.
SnowCamp - Adding search to a legacy applicationNicolas Fränkel
Most applications evolve to a point where they need to provide search capabilities. But updating an application is always a risk. Plus, sometimes, you don’t have access to the source code. The easiest way to access the data is by getting them directly from the database.
The initial load is the easiest step. However, how do you keep the search index in sync with the database? How do you keep the latency between the search store and the source of truth, so your users don’t have to wait for the next run of the batch to access the newest changes?
In this live coding session, we will show you how you can solve this issue by connecting Elasticsearch to the database with a touch of Hazelcast.
On dit que GitHub est le CV d'un développeur. Un rapide coup d'œil à votre historique de commit et les recruteurs savent tout de vous. Cette approche comporte quelques problèmes. La plupart des entreprises ne publient même pas leur code sous une licence Open Source. Si vous travaillez pour l'une d'entre elles, et si vous n'êtes pas un développeur Open Source les soirs et les week-ends, alors vous n'avez aucune chance.
Récemment, GitHub a permis un certain degré de personnalisation de son profil. Ainsi, même si votre historique de commit a plus de blanc que de vert, vous pouvez fournir un bon point d'entrée pour les employeurs potentiels. Mais ça ne vaut que l'effort que vous y mettez et les données perdent leur valeur rapidement. Pourtant, avec un peu de travail et l'aide d'outils d'automatisation (tels que GitHub Actions), vous pouvez présenter un profil toujours à jour.
DevExperience - Improve your tests with mutation testingNicolas Fränkel
Unit testing ensures your production code is relevant. But what does ensure your testing code is relevant?
In the realm of testing, the code coverage metrics is the most often talked about. However, it doesn't mean that the test has been useful or even that an assert has been coded. Mutation testing is a strategy to make sure that the test code is relevant.
I.T.A.K.E Unconference - Mutation testing to the rescue of your testsNicolas Fränkel
This document summarizes mutation testing and how it can be used to test test quality. Mutation testing works by modifying the production code in small ways (e.g. changing a relational operator) and then running the test suite to see if the tests can detect the mutations. This is more effective than just measuring code coverage at ensuring tests are rigorous. The document outlines some Java tools for mutation testing like PIT and discusses how mutation testing can identify weaknesses in a test suite. Drawbacks like performance issues are also covered, along with ways to mitigate slowness like incremental analysis.
Testing, Learning and Professionalism — 20171214David Rodenas
Two hours talk about testing, its history, its meaning and propose, a small guide of good practices, and what it is supposed to be a good professional and what is expected from us the professionals.
There are lots of talks about testing: they talk about syntax, methodologies, tools. But there is usually a missing point: Why it is important to do a test? What is important to test? What is not important to test? How to do testing?
There lots of examples in plnker just to see each step, and many surprises.
This talk is good for beginners, but also for some seasoned people that just want to light up some of the ideas that they might have been hatching.
Spoiler alert: testing will save you development time.
(automatic) Testing: from business to university and backDavid Rodenas
This talk cares about the fundamentals of testing, a little bit history of how the professional community developed what we currently know as testing, but also about why I should care about testing? why is it important to do a test? What is important to test? What is not important to test? How to do testing?
There some examples in plnker just to see each step, and many surprises.
This talk also compares what people learned in the Computer Sciences and Engineering degrees and what people does in testing. It gives some tips to catch up with current state of art and gives some points to start changing syllabus to make better engineers.
This talk is good for beginners, teachers, bosses, but also for seasoned techies that just want to light up some of the ideas that they might have been hatching.
Spoiler alert: testing will save you development time and make you a good professional.
This document provides an introduction to unit testing, including basics like why unit tests are useful, what a test case consists of, and how tests are processed and their lifecycle. It then demonstrates how to run tests from the command line and in PhpStorm. Finally, it discusses how to write real unit tests for custom code by adding test cases and removing dependencies through mocking or prophecies. The overall document serves as a tutorial for beginners on getting started with unit testing.
The document discusses mutation testing and provides tips for implementing it. It explains different types of mutations like statement, value, and decision mutations. It also discusses how to calculate mutation score and challenges like cost. Examples are provided using tools like Pitest and Stryker. Advice is given around using mocks, refactoring complex code, and targeting tests to improve mutation testing.
How to quickly add a safety net to a legacy codebaseNelis Boucké
Did you ever inherited a horrible legacy system to maintain and you were scared to introduce bugs touching it? If you did (who never did?) then this post is for you.
You can find the accompanying video and blogpost soon on
Must.Kill.Mutants. Agile Testing Days 2017Gerald Muecke
The document discusses quality assurance and mutation testing. It defines quality assurance as preventing defects in manufactured products. It then discusses mutation testing, which involves injecting faults into code based on mutation operators to verify if tests can detect the faults. The document provides examples of mutation operators and tools for different programming languages. It notes that while mutation testing has limitations, it is valuable for finding bugs in test suites and improving test quality. It recommends including mutation testing in projects and using it with common sense to iteratively improve test suites.
DOAN DuyHai – Cassandra: real world best use-cases and worst anti-patterns - ...NoSQLmatters
This document discusses Cassandra use cases and anti-patterns. It describes queue-like designs, intensive updates on the same column, and designing around a dynamic schema as anti-patterns that can lead to failures. Rate limiting, fraud prevention, and account validation are provided as examples of good use cases. Key-value modeling, clustering, compaction strategies, and time-to-live features are also overviewed.
Unit Testing like a Pro - The Circle of PurityVictor Rentea
Best practices on designing unit tests, designing testable production code, a glimpse of TDD, using mocks and isolating pure functions for easy testing. Talk distilled from http://victorrentea.ro/#unit-testing
Held at VoxxedDays Bucharest in March 2019.
The document discusses how to improperly conduct a benchmark test in order to fail. Some ways to fail include measuring the wrong metrics, using an improper testing methodology, and testing in the wrong environment. The document also provides examples of puzzles to demonstrate benchmarking issues and discusses the concept of "coordinated omission" where aspects of the system are omitted from measurement.
Break up the Monolith: Testing MicroservicesMarcus Merrell
Microservices is more than a buzzword: it’s an industry-wide tidal wave. Companies are spending millions to break up monoliths and spin up microservices, but they usually only involve QA at the very end. This talk centers around real-world experiences, posing questions you can ask your developers/product people, and offering solutions for you to help make your service more discoverable, more testable, and easier to release.
Automation for Anyone at Nutanix NEXT 2017 USChris Wahl
Are you wondering how to solve repetitive tasks with software automation, but you struggle every time you see the word “developer” or “code?” Do you know what APIs are and how they make these tasks easy to solve? In this session, we’ll explore the framework that can apply to mundane tasks (i.e. PowerShell, Pester), and we’ll discuss what open source tools are available to help solve these problems. Walk away with the advice you need to get started!
This document summarizes a presentation about the open source Verilog simulator Verilator. It discusses why open source simulation is useful, provides an introduction and overview of Verilator's features, and addresses questions about getting started with and using Verilator. The presentation also covers Verilator's performance advantages over other simulators, considerations for whether Verilator may be suitable, its future development, and how users can contribute back to the project.
White-box Unit Test Generation with Microsoft IntelliTestDávid Honfi
This document discusses white-box unit test generation using Microsoft IntelliTest. It begins with an overview of symbolic execution and how IntelliTest uses dynamic symbolic execution to generate test cases by exploring paths in the code. It then demonstrates IntelliTest's capabilities with examples. The document also discusses challenges with symbolic execution-based test generation and proposes solutions like visualizing the symbolic execution process and automatically isolating environment dependencies through source code transformations.
In this presentation, Jared Atkinson and Jonathan Johnson discuss the problem that many security professionals are facing today. How exactly do I know if my detection will actually detect the thing I want to detect? We discuss the importance of testing telemetry coverage and using abstraction to build a representative sample set of Atomic tests to validate detection coverage.
Scripts used in presentation can be found below:
Process Access: https://gist.github.com/jaredcatkinson/9c7a1af2261a752432230a4148ecfe02
Process Read: https://github.com/redcanaryco/AtomicTestHarnesses/blob/master/Windows/TestHarnesses/T1003.001_DumpLSASS/DumpLSASS.ps1
This document discusses non-idempotent-outcome (NIO) tests, which are a type of flaky test that deterministically passes when run once but fails when run twice. The document defines NIO tests and describes how detecting and fixing them can help prevent order-dependent flaky tests. It is reported that NIO tests are prevalent, with 361 detected across 127 Java test suites and 1006 Python projects. Fixes were proposed for 268 NIO tests and 192 were accepted by developers.
A conjecture, even when has been accepted during a long period of time, is not even still a proof. This presentation tries to explain how developers unconsciously create and establish paths that can be taken by their human errors in order to become system failures, and how mutation testing technique is used to effectively remove these undesired paths.
Keynote at the 5th Workshop on Validation, Analysis, and Evolution of Software Tests (VST 2022)
Website: https://rramler.github.io/vst2022/
Abstract: Nowadays, Artificial intelligence (AI) plays a critical role in automating different human-intensive tasks, including software engineering tasks. Since the late 70s, researchers have proposed automated techniques to automatically generate test data (fuzzing) or test suites (test suite generation). Proposed techniques span from simple heuristics to more advanced AI-based techniques and evolutionary intelligence in particular. While recent studies have shown that these techniques achieve high coverage and find bugs, generated tests can be hard to understand and maintain. This talk will provide an overview and reflection on state-of-the-art techniques, open challenges, and research opportunities towards more accessible tests that are easy to integrate within the DevOps cycle. To this aim, the talk will cover relevant application domains, including "traditional" software and emerging cyber-physical systems.
Similar to Java Dominicano - Mutation testing (20)
SnowCamp - Adding search to a legacy applicationNicolas Fränkel
Most applications evolve to a point where they need to provide search capabilities. But updating an application is always a risk. Plus, sometimes, you don’t have access to the source code. The easiest way to access the data is by getting them directly from the database.
The initial load is the easiest step. However, how do you keep the search index in sync with the database? How do you keep the latency between the search store and the source of truth, so your users don’t have to wait for the next run of the batch to access the newest changes?
In this live coding session, we will show you how you can solve this issue by connecting Elasticsearch to the database with a touch of Hazelcast.
On dit que GitHub est le CV d'un développeur. Un rapide coup d'œil à votre historique de commit et les recruteurs savent tout de vous. Cette approche comporte quelques problèmes. La plupart des entreprises ne publient même pas leur code sous une licence Open Source. Si vous travaillez pour l'une d'entre elles, et si vous n'êtes pas un développeur Open Source les soirs et les week-ends, alors vous n'avez aucune chance.
Récemment, GitHub a permis un certain degré de personnalisation de son profil. Ainsi, même si votre historique de commit a plus de blanc que de vert, vous pouvez fournir un bon point d'entrée pour les employeurs potentiels. Mais ça ne vaut que l'effort que vous y mettez et les données perdent leur valeur rapidement. Pourtant, avec un peu de travail et l'aide d'outils d'automatisation (tels que GitHub Actions), vous pouvez présenter un profil toujours à jour.
Zero-downtime deployment on Kubernetes with HazelcastNicolas Fränkel
Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day because we need to handle state. Though it would be much easier to have stateless apps, and despite our best efforts in this direction, state is found in (at least) two places: sessions and databases.
You need to think keeping the state while stopping and starting application nodes. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema change with the help of Hazelcast, while keeping the service up during the entire update process.
jLove - A Change-Data-Capture use-case: designing an evergreen cacheNicolas Fränkel
When one’s app is challenged with poor performances, it’s easy to set up a cache in front of one’s SQL database. It doesn’t fix the root cause (e.g. bad schema design, bad SQL query, etc.) but it gets the job done. If the app is the only component that writes to the underlying database, it’s a no-brainer to update the cache accordingly, so the cache is always up-to-date with the data in the database.
Things start to go sour when the app is not the only component writing to the DB. Among other sources of writes, there are batches, other apps (shared databases exist unfortunately), etc. One might think about a couple of ways to keep data in sync i.e. polling the DB every now and then, DB triggers, etc. Unfortunately, they all have issues that make them unreliable and/or fragile.
You might have read about Change-Data-Capture before. It’s been described by Martin Kleppmann as turning the database inside out: it means the DB can send change events (SELECT, DELETE and UPDATE) that one can register to. Just opposite to Event Sourcing that aggregates events to produce state, CDC is about getting events out of states. Once CDC is implemented, one can subscribe to its events and update the cache accordingly. However, CDC is quite in its early stage, and implementations are quite specific.
In this talk, I’ll describe an easy-to-setup architecture that leverages CDC to have an evergreen cache.
BigData conference - Introduction to stream processingNicolas Fränkel
This document discusses stream processing and summarizes a presentation about the topic. It introduces Hazelcast Jet as a stream processing engine and covers open data standards like GTFS. It also describes a demo that uses GTFS data to enrich public transit vehicle position updates in real-time using Hazelcast Jet. The presentation discusses streaming approaches, benefits over batch processing, and provides an overview of stream processing concepts.
ADDO - Your own Kubernetes controller, not only in GoNicolas Fränkel
In Kubernetes, operators allow the API to be extended to your heart content. If one task requires too much YAML, it’s easy to create an operator to take care of the repetitive cruft, and only require a minimum amount of YAML.
On the other hand, since its beginnings, the Go language has been advertised as closer to the hardware, and is now ubiquitous in low-level programming. Kubernetes has been rewritten from Java to Go, and its whole ecosystem revolves around Go. For that reason, It’s only natural that Kubernetes provides a Go-based framework to create your own operator. While it makes sense, it requires organizations willing to go down this road to have Go developers, and/or train their teams in Go. While perfectly acceptable, this is not the only option. In fact, since Kubernetes is based on REST, why settle for Go and not use your own favorite language?
In this talk, I’ll describe what is an operator, how they work, how to design one, and finally demo a Java-based operator that is as good as a Go one.
OSCONF Jaipur - A Hitchhiker's Tour to Containerizing a Java applicationNicolas Fränkel
As “the Cloud” becomes more and more widespread, now is a good time to assess how you can containerize your Java application. I assume you’re able to write a a Dockerfile around the generated JAR. However, each time the application’s code will change, the whole image will need to be rebuilt. If you’re deploying to a local Kubernetes cluster environment, this increases that much the length of the feedback loop.
In this demo-based talk, I’ll present different ways to get your Java app in a container: Dockerfile, Jib, and Cloud Native Buildpacks. We will also have a look at what kind of Docker image they generate, how they layer the images, whether those images are compatible with skaffold, etc.
GeekcampSG 2020 - A Change-Data-Capture use-case: designing an evergreen cacheNicolas Fränkel
CDC is a brand new approach that "turns the database inside out": it allows to get events out of the database state. This can be leveraged to get a cache that is never stale.
JavaDay Istanbul - 3 improvements in your microservices architectureNicolas Fränkel
While a microservices architecture is more scalable than a monolith, it has a direct hit on performance.
To cope with that, one performance improvement is to set up a cache. It can be configured for database access, for REST calls or just to store session state across a cluster of server nodes. In this demo-based talk, I’ll show how Hazelcast In-Memory Data Grid can help you in each one of those areas and how to configure it. Hint: it’s much easier than one would expect.
At a point in the past, it was forecast that Java would die, but the JVM platform would be its legacy. And in fact, for a long time, the JVM has been tremendously successful. Wikipedia itself lists a bunch of languages that run on it, some of them close to Java e.g. Kotlin, some of them very remote e.g. Clojure.
But nowadays, the Cloud is becoming ubiquitous. Containerization is the way to go to alleviate some of the vendor lock-in issues. Kubernetes is a de facto platform. If a container needs to be killed for whatever reason (resource consumption, unhealthy, etc.), a new one needs to replace it as fast as possible. In that context, the JVM seems to be a dead-end: its startup time is huge in comparison to a native process. Likewise, it consumes a lot of memory that just increase the monthly bill.
What does that mean for us developers? Has all the time spent in learning the JVM ecosystem been invested with no hope of return over investment? Shall we need to invest even more time in new languages, frameworks, libraries, etc.? That is one possibility for sure. But we can also leverage our existing knowledge, and embrace the Cloud and containers ways with the help of some tools.
In this talk, I’ll create a simple URL shortener with a “standard” stack: Kotlin, JAX-RS and Hazelcast. Then, with the help of Quarkus and GraalVM, I’ll turn this application into a native executable with all Cloud/Container related work has been moved to the build process.
Devclub.lv - Introduction to stream processingNicolas Fränkel
While “software is eating the world”, those who are able to best manage the huge mass of data will emerge out on the top.
The batch processing model has been faithfully serving us for decades. However, it might have reached the end of its usefulness for all but some very specific use-cases. As the pace of businesses increases, most of the time, decision-makers prefer slightly wrong data sooner, than 100% accurate data later. Stream processing – or data streaming – exactly matches this usage: instead of managing the entire bulk of data, manage pieces of them as soon as they become available.
OSCONF Koshi - Zero downtime deployment with Kubernetes, Flyway and Spring BootNicolas Fränkel
Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day. You need to think about application backward compatibility, possible rollback, database schema migration, etc. I believe the later is one of the biggest pain point. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema migration with the help of Flyway, while keeping the service up during the entire update process.
JOnConf - A CDC use-case: designing an Evergreen CacheNicolas Fränkel
This document discusses using change data capture (CDC) and Hazelcast Jet to build an evergreen cache that remains in sync with a database. It covers alternatives to cache invalidation like polling and triggers, introduces CDC and the Debezium implementation, and proposes a Jet job that watches database change events, analyzes them, and updates the cache accordingly to solve the cache freshness problem.
London In-Memory Computing Meetup - A Change-Data-Capture use-case: designing...Nicolas Fränkel
When one’s app is challenged with poor performances, it’s easy to set up a cache in front of one’s SQL database. It doesn’t fix the root cause (e.g. bad schema design, bad SQL query, etc.) but it gets the job done. If the app is the only component that writes to the underlying database, it’s a no-brainer to update the cache accordingly, so the cache is always up-to-date with the data in the database.
Things start to go sour when the app is not the only component writing to the DB. Among other sources of writes, there are batches, other apps (shared databases exist, unfortunately), etc. One might think about a couple of ways to keep data in sync i.e. polling the DB every now and then, DB triggers, etc. Unfortunately, they all have issues that make them unreliable and/or fragile.
In this talk, I will describe an easy-to-setup architecture that leverages CDC to have an evergreen cache.
This talk will be about the reasons behind the new stream processing model, how it compare to the old batch model, what are their pros and cons, and a list of existing technologies implementing stream processing with their most prominent characteristics. It will contain details of one possible use-case of data streaming that is not possible with batches: display in (near) real-time all trains in Switzerland and their position on a map, beginning with an overview of all the requirements and the design. Finally, using an OpenData endpoint and the Hazelcast platform,showing a working demo implementation of it.
Java.IL - Your own Kubernetes controller, not only in Go!Nicolas Fränkel
In Kubernetes, operators allow the API to be extended to your heart content. If one task requires too much YAML, it’s easy to create an operator to take care of the repetitive cruft, and only require a minimum amount of YAML.
On the other hand, since its beginnings, the Go language has been advertised as closer to the hardware, and is now ubiquitous in low-level programming. Kubernetes has been rewritten from Java to Go, and its whole ecosystem revolves around Go. For that reason, It’s only natural that Kubernetes provides a Go-based framework to create your own operator. While it makes sense, it requires organizations willing to go down this road to have Go developers, and/or train their teams in Go. While perfectly acceptable, this is not the only option. In fact, since Kubernetes is based on REST, why settle for Go and not use your own favorite language?
In this talk, I’ll describe what is an operator, how they work, how to design one, and finally demo a Java-based operator that is as good as a Go one.
The document discusses stream processing and provides an overview of Hazelcast Jet. It begins with explaining why streaming is useful and describes different streaming approaches like event-driven programming. It then provides details on Hazelcast Jet, including its concepts of pipelines and jobs. The document also discusses open data standards like GTFS and demonstrates a sample streaming pipeline that enriches public transportation data from open APIs.
London Java Community - An Experiment in Continuous Deployment of JVM applica...Nicolas Fränkel
A couple of years ago, continuous integration in the JVM ecosystem meant Jenkins. Since that time, a lot of other tools have been made available. But new tools don’t mean new features, just new ways. Besides that, what about continuous deployment? There’s no tool that allows deploying new versions of a JVM-based application without downtime. The only way to achieve zero downtime is to have multiple nodes deployed on a platform, and let that platform achieve that e.g. Kubernetes.
And yet, achieving true continuous deployment of bytecode on one single JVM instance is possible if one changes one’s way of looking at things. What if the compilation could be seen as changes? What if those changes could be stored in a data store, and a listener on this data store could stream those changes to the running production JVM via the Attach API?
In this talk, we'll demo exactly that using Hazelcast and Hazelcast Jet - but it’s possible to re-use the principles that will be shown using other streaming technologies.
OSCONF - Your own Kubernetes controller: not only in GoNicolas Fränkel
This document discusses creating Kubernetes operators and controllers using different programming languages besides Go. It suggests that a Java-based controller is possible using the GraalVM, which allows creating native executables from Java bytecode. Key points covered include what controllers and operators are, that no specific technology stack is required, and that the JVM could be a good option for controller development with GraalVM's support for polyglot programming and creating native applications.
vKUG - Migrating Spring Boot apps from annotation-based config to FunctionalNicolas Fränkel
Migrating Spring Boot apps from annotation-based config to Functional with Kotlin - Nicolas Fränkel
In the latest years, there has been some push-back against frameworks, and more specifically annotations: some call them magic. Obviously, they make understanding the flow of the application harder.
Spring and Spring Boot latest versions go along this trend, by offering an additional way to configure beans with explicit code instead of annotations. It’s declarative in the sense it looks like configuration, though it’s based on Domain-Specific Language(s). This talk aims to demo a step-by-step process to achieve that.
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfVALiNTRY360
Salesforce Healthcare CRM, implemented by VALiNTRY360, revolutionizes patient management by enhancing patient engagement, streamlining administrative processes, and improving care coordination. Its advanced analytics, robust security, and seamless integration with telehealth services ensure that healthcare providers can deliver personalized, efficient, and secure patient care. By automating routine tasks and providing actionable insights, Salesforce Healthcare CRM enables healthcare providers to focus on delivering high-quality care, leading to better patient outcomes and higher satisfaction. VALiNTRY360's expertise ensures a tailored solution that meets the unique needs of any healthcare practice, from small clinics to large hospital systems.
For more info visit us https://valintry360.com/solutions/health-life-sciences
Microservice Teams - How the cloud changes the way we workSven Peters
A lot of technical challenges and complexity come with building a cloud-native and distributed architecture. The way we develop backend software has fundamentally changed in the last ten years. Managing a microservices architecture demands a lot of us to ensure observability and operational resiliency. But did you also change the way you run your development teams?
Sven will talk about Atlassian’s journey from a monolith to a multi-tenanted architecture and how it affected the way the engineering teams work. You will learn how we shifted to service ownership, moved to more autonomous teams (and its challenges), and established platform and enablement teams.
UI5con 2024 - Bring Your Own Design SystemPeter Muessig
How do you combine the OpenUI5/SAPUI5 programming model with a design system that makes its controls available as Web Components? Since OpenUI5/SAPUI5 1.120, the framework supports the integration of any Web Components. This makes it possible, for example, to natively embed own Web Components of your design system which are created with Stencil. The integration embeds the Web Components in a way that they can be used naturally in XMLViews, like with standard UI5 controls, and can be bound with data binding. Learn how you can also make use of the Web Components base class in OpenUI5/SAPUI5 to also integrate your Web Components and get inspired by the solution to generate a custom UI5 library providing the Web Components control wrappers for the native ones.
Transform Your Communication with Cloud-Based IVR SolutionsTheSMSPoint
Discover the power of Cloud-Based IVR Solutions to streamline communication processes. Embrace scalability and cost-efficiency while enhancing customer experiences with features like automated call routing and voice recognition. Accessible from anywhere, these solutions integrate seamlessly with existing systems, providing real-time analytics for continuous improvement. Revolutionize your communication strategy today with Cloud-Based IVR Solutions. Learn more at: https://thesmspoint.com/channel/cloud-telephony
Artificia Intellicence and XPath Extension FunctionsOctavian Nadolu
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
Malibou Pitch Deck For Its €3M Seed Roundsjcobrien
French start-up Malibou raised a €3 million Seed Round to develop its payroll and human resources
management platform for VSEs and SMEs. The financing round was led by investors Breega, Y Combinator, and FCVC.
WWDC 2024 Keynote Review: For CocoaCoders AustinPatrick Weigel
Overview of WWDC 2024 Keynote Address.
Covers: Apple Intelligence, iOS18, macOS Sequoia, iPadOS, watchOS, visionOS, and Apple TV+.
Understandable dialogue on Apple TV+
On-device app controlling AI.
Access to ChatGPT with a guest appearance by Chief Data Thief Sam Altman!
App Locking! iPhone Mirroring! And a Calculator!!
E-commerce Development Services- Hornet DynamicsHornet Dynamics
For any business hoping to succeed in the digital age, having a strong online presence is crucial. We offer Ecommerce Development Services that are customized according to your business requirements and client preferences, enabling you to create a dynamic, safe, and user-friendly online store.
Mobile app Development Services | Drona InfotechDrona Infotech
Drona Infotech is one of the Best Mobile App Development Company In Noida Maintenance and ongoing support. mobile app development Services can help you maintain and support your app after it has been launched. This includes fixing bugs, adding new features, and keeping your app up-to-date with the latest
Visit Us For :
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesQuickdice ERP
Explore the seamless transition to e-invoicing with this comprehensive guide tailored for Saudi Arabian businesses. Navigate the process effortlessly with step-by-step instructions designed to streamline implementation and enhance efficiency.
3. @nicolas_frankel
Hazelcast
HAZELCAST IMDG is an operational,
in-memory, distributed computing
platform that manages data using
in-memory storage, and performs
parallel execution for breakthrough
application speed and scale.
HAZELCAST JET is the ultra fast,
application embeddable, 3rd
generation stream processing
engine for low latency batch
and stream processing.
4. @nicolas_frankel
• Unit Testing
• Integration Testing
• End-to-end Testing
• Performance Testing
• Penetration Testing
• Exploratory Testing
• etc.
Many kinds of testing
7. @nicolas_frankel
“Code coverage is a measure used to
describe the degree to which the
source code of a program is tested”
--Wikipedia
http://en.wikipedia.org/wiki/Code_coverage
Code coverage
7
8. @nicolas_frankel
Check whether a source code line is
executed during a test
• Or Branch Coverage
Measuring Code Coverage
@nicolas_frankel #mutationtesting #devexperience18
8
9. @nicolas_frankel
Computing Code Coverage
CC =
Lexecuted
Ltotal
*100
• CC: Code
Coverage
(in percent)
• Lexecuted: Number
of executed lines
of code
• Ltotal: Number of
total lines of
code
11. @nicolas_frankel
“Is 100% code coverage realistic? Of
course it is. If you can write a line of
code, you can write another that
tests it.”
Robert Martin (Uncle Bob)
https://twitter.com/unclebobmartin/status/55966
620509667328
100% Code Coverage?
11
13. @nicolas_frankel
• Any metric can be gamed!
• Code coverage is a metric…
⇒ Code coverage can be gamed
• On purpose
• Or by accident
Code coverage as a measure of quality
13
17. @nicolas_frankel
public class Math {
public int add(int i1, int i2) {
return i1 + i2;
}
}
public class Math {
public int add(int i1, int i2) {
return i1 - i2;
}
}
21. @nicolas_frankel
• Surviving means changing the
source code did not change the
test result Bad!
• Killed means changing the source
code changed the test result
Good!
Killed or Surviving?
21
22. @nicolas_frankel
Test the code
public class Math {
public int add(int i1, int i2) {
return i1 + i2;
}
}
✔Execute Test
@Test
public void add_should_add() {
new Math().add(1, 1);
}
24. @nicolas_frankel
Test the code
✔Execute Test
public class Math {
public int add(int i1, int i2) {
return i1 + i2;
}
}
@Test
public void add_should_add() {
new Math().add(1, 1);
Assert.assertEquals(sum, 2);
}
25. @nicolas_frankel
Killed mutant
✗Execute SAME Test
public class Math {
public int add(int i1, int i2) {
return i1 + i2;
}
}
@Test
public void add_should_add() {
new Math().add(1, 1);
Assert.assertEquals(sum, 2);
}
26. @nicolas_frankel
• PIT is a tool for Mutation testing
• Available as
• Command-line tool
• Ant target
• Maven plugin
Mutation Testing in Java
26
28. @nicolas_frankel
PIT mutators sample
Name Example source Result
Conditionals Boundary > >=
Negate Conditionals == !=
Remove Conditionals foo == bar true
Math + -
Increments foo++ foo--
Invert Negatives -foo foo
Inline Constant static final FOO= 42 static final FOO = 43
Return Values return true return false
Void Method Call System.out.println("foo")
Non Void Method Call long t = System.currentTimeMillis() long t = 0
Constructor Call Date d = new Date() Date d = null;
36. @nicolas_frankel
• Analyze test code
• For each class under test
• For each mutator
• Create mutation
• For each mutation
• Run test
• Analyze result
• Aggregate results
Why so slow?
36
37. @nicolas_frankel
• Increase number of threads
• Set a limited a set of mutators
• Limit scope of target classes
• Limit number of tests
• Limit dependency distance
• Use incremental analysis
• Don’t bind to the test phase
• Use scmMutationCoverage
Workarounds
37
38. @nicolas_frankel
• Metadata stored between runs
• Mutant will not be checked again, if:
• killed, and neither class nor test have changed
• survived, and there are no new/changed tests for it
Incremental analysis
39. @nicolas_frankel
• Checks the relevance of your unit
tests
• Points out potential bugs
Is Mutation Testing the Silver Bullet?
39
40. @nicolas_frankel
• Validate the assembled application
• Integration Testing
• Check the performance
• Performance Testing
• Look out for display bugs
• End-to-end testing
• Etc.
The rest is up to you
40
41. @nicolas_frankel
• Don’t test to achieve 100%
coverage
• Test because it saves money in the
long run
• Prioritize:
• Business-critical code
• Complex code
Testing is about ROI
41