This document summarizes a collaborative project called STAMP that aims to automatically generate and amplify test assets like unit test cases and configuration files to improve software quality. The project focuses on developing open source tools for test amplification in DevOps workflows. The goal is to increase test coverage, find more bugs, and improve test quality through automatic generation and variation of test assets. The tools will integrate with development toolchains and microservices. The project involves both academic and industry partners applying the techniques to various software systems.

1. Collaborative Project
Open Research & Innovation
21 mars 2018 - Paris Expo - 16:00
Caroline LANDRY - INRIA

2. 2
STAMP Project Overview
Author: Benoit Baudry
Project Coordinator and Scientific Leader
KTH, Sweden
Speaker: Caroline Landry
Project Technical Manager
INRIA, France

3. 3
DevOps

4. 4
DevOps – automatic development
unit
perf.
fuzzing
loggingdep. inj.
UI
CI
pertur-bation
fault recov.
IDEs
libraries
container
IDS
VMs
cluster
config.

5. 5
DevOps – continuous testing
unit
perf.
fuzzing
logging
UI
pertur-bation
fault recov.
IDS
cluster
config.

6. 6
DevOps – STAMP focus
unit
logging
config.

7. 7
Pain points
● For a developer
−Writing test cases is painful
−Would be great to generate them automatically
● For a tester
−Coverage is poor
−Would be great to automatically harden test suites
● For a manager
−Regressions are bad
−Would be good to fix them before production

8. 8
Hard points
● Code coverage is insufficient
−What if we could get test cases that detect more bugs?
−What if we could get test cases that cover more code?
● Configuration testing is underrated
−What if we could automatically get variations of
configurations?
−What if configuration testing were automatically deployed?
● Log analysis is time consuming
−What if we could get test cases that reproduce the crash?

9. 9
STAMP’s concept: amplification
● Amplify (v.): to increase the size or effect of
something
https://dictionary.cambridge.org/dictionary/english/amplify

10. 10
STAMP’s concept: amplification
● Amplify (v.): to increase the size or effect of
something
● Test amplification: Increase the effect of test assets
https://dictionary.cambridge.org/dictionary/english/amplify

11. 11
STAMP’s concept: amplification
● Amplify (v.): to increase the size or effect of
something
● Test amplification: Increase the effect of test assets
−Test assets: unit test cases, configuration files, production
logs
−Measure effect: coverage, mutation score
−Automatic amplification
https://dictionary.cambridge.org/dictionary/english/amplify

12. 12
Unit test amplification

13. 13

14. 14

15. 15
STAMP in DevOps

16. 16

17. 17

18. 18
STAMP in DevOps

19. 19

20. 20

21. 21
Research questions
● What are the effective amplification operations?
● To what extent does it increase test quality?
● Are amplified assets useful for developers?
● How does amplification fit in the lifecycle?

22. 22
Focus on tool development
● Tools
−Unit testing: Dspot and Descartes
−Configuration testing: Camp
−Online testing: Evocrash
● Open source
−https://github.com/STAMP-project
● Micro-services
−Integrate in tool chains
−As a service

23. 23
Use cases
● XWiki: content management system
● CityGo: public transport recommender
● ProActive: workflow deployment
● TellUCloud: eHealth and personel safety
● Authzforce: access control
● SAT4J: constraint solving engine

24. 24
STAMP is about
● Automatic testing in DevOps
● A novel concept: test amplification
● Technology: Java / Docker tools
● Industry-academia collaboration
● Open source