The panel discusses the past and future of software testing and analysis. In the last decade, contributions included search-based software testing and using machine learning in test automation. Future directions include focusing on practical and scalable solutions, combining technologies like machine learning and symbolic execution to solve specific problems, and conducting applied research driven by industrial contexts. Impact requires collaboration between researchers and industry to understand problems.

1. Panel: Past and Future of Software
Testing and Analysis
Lionel Briand
http://www.lbriand.info
ISSTA 2021

2. Contributions in the last decade
• Search-based software testing
• E.g., scalable test data generation, e.g., at system level
• Use of Machine learning in test automation
• E.g., Regression test selection and prioritization
• Use of test execution history and change data across builds
• In continuous development and integration environments
• Testing cyber-physical systems
• Testing CPS models (falsification), simulation-driven testing
• Trace analysis and monitoring, e.g., specification languages
2

3. Future directions
• We have developed many interesting ideas and concepts
• Many of them are not practical or scalable, in most
contexts
• For example: mutation testing, metamorphic testing, etc.
• We need to focus on devising novel engineering solutions
that are scalable and practical, at least in some well-
defined contexts
• E.g., mutation analysis in embedded systems in the space
domain
3

4. Future directions
• Multi-disciplinary research: It is rarely the case that one
technology solves an actual problem
• We have many performant technologies at our disposal,
that have made big leaps forward in the last decade:
• Machine learning
• Natural Language Processing
• Meta-heuristic search
• Solvers (e.g., SMT)
• Symbolic execution
• Simulation
• How to combine them in a specific context for targeted
problems? This is hard work, e.g., Automotive DS.
4

5. Critical factors for the field
• We need fundamental research, not driven by specific
problems, and we have done that well as a community
• However, our practical impact has been limited and,
over time, this is being noticed by institutions, funding
agencies, etc.
• Other fields have had much more (visible) impact
• Testing and analysis research is a practical field that
should have substantial visible impact
5

6. Critical factors for the field
• Impact requires a thorough understanding of problems
• Present and foreseeable problems
• Problem understanding requires collaboration between
researchers and industry
• Collaboration needs to be supported and encouraged
by our community, academic institutions, and funding
agencies
6

7. Applied research, driven by
industrial contexts, focused
on scalable and applicable
solutions, is difficult and
rewarding research work
7

8. Panel: Past and Future of Software
Testing and Analysis
Lionel Briand
http://www.lbriand.info
ISSTA 2021