In software engineering, there is always a gap between the current research topics and the everyday industrial practice. However, in my experience this gap is much wider in software testing, e.g. advanced testing techniques seldom reach everyday testers. This gap can be attributed to several factors, the talk will highlight education and tools from the possible causes. In order to illustrate this gap the talk will collect and compere the topics of recent academic and industrial testing related conferences. My goal is to offer a glimpse into recent software testing research topics to practitioners, and start a discussion whether there is really a wide gap between academy and industry.

1. Hungarian Software Testing Forum (HUSTEF 2014)
The Gap Between Academic Research and
Industrial Practice in Software Testing
Budapest University of Technology and Economics
Department of Measurement and Information Systems
Zoltan Micskei
assistant professor

2. The Gap: Examples
“Automatic testing”
Academia Industry
2
—> automatic test
case generation
—> automatic
test execution

3. The Gap: Examples
“Code coverage”
Academia Industry
3
—> “We defined the
new <obscure> metric.. ”
—> “Nice, we reached 80%
<statement> coverage.”

4. Illustrating the gap

5. (Some) Recent Papers in Academia
Coverage Is Not Strongly Correlated with Test Suite Effectiveness
An improved Pareto distribution for modelling
the fault data of open source software
Automatic test case generation from Simulink
Stateflow models using model checking
Adding Contextual Guidance to the Automated Search for Probabilistic Test Profiles
Lookahead-Based Approaches for Minimizing Adaptive Distinguishing Sequences
Design and industrial evaluation of a tool supporting semi-automated website testing
Using Test Case Reduction and Prioritization to Improve Symbolic Execution
Minimum Number of Test Paths for Prime Path and Other Structural Coverage Criteria
Search-Based Testing for Embedded Telecom
Software with Complex Input Structures
Characterizing and Detecting Performance Bugs for Smartphone Applications
5
Enhancing Symbolic Execution with Veritesting
Analysis and testing of black-box component-based
systems by inferring partial models
A Hitchhiker's guide to statistical tests for assessing
randomized algorithms in software engineering
Sample from ~130 papers from 7 conferences

6. Topics in Academia
6
Test case/input
generation
Test optimization and
prioritization
Empirical evaluations

7. (Some) Recent Talks in Industry
Top Challenges in Testing Requirements
How to Stress Test Your Mobile App
Internationalization and Localization Testing
Calculating the Cost of non-quality
Breaking the Matrix - Android Testing at Scale
Wisdom of Crowds in Software Testing
Risk-Based Testing for Agile Projects
Challenges and Solutions in Global Requirements Engineering
7
Web Performance Testing with WebDriver
Key Test Design Techniques
Appium: Automation for Mobile Apps Espresso: Fresh Start to Android UI Testing
Designing for Testability: Differentiator in a Competitive Market
Sample from ~80 papers from 5 conferences

8. Topics in Academia and Industry
8
Test management
Test automation (tools,
technologies)
Test design techniques
Test case/input
generation
Test optimization and
prioritization
Empirical evaluations

9. Reducing the Gap
9

10. Factors Affecting The Gap
Education Tools

11. Education: ACM/IEEE Comp. Sci. Curricula
Software Engineering Knowledge Area
Formal Methods
Software Reliability
Software Evolution
Software Verification Validation
0 5 10 15 20 25 30 35 40 45 50
Social Issues and Prof. Practice
Systems Fundamentals
Software Engineering
Software Development Fundamentals
Programming Languages
Parallel and Distributed Computing
Platform-Based Development
Operating Systems
Networking and Communication
Intelligent Systems
Information Management
Inf. Assurance and Security
Human-Computer Interaction
Graphics and Visualization
Discrete Structures
Copmputational Science
Architecture and Organization
Software Construction
Software Design
Requirements Engineering
Tools and Environments
Joint Task Force on Computing Curricula, Association for Computing Software Project Machinery Management
(ACM) and
IEEE Computer Society. 2013. Computer Science Curricula 2013: Curriculum Guidelines for
Undergraduate Degree Programs in Computer Science. ACM, New York, NY, USA.
11
Algorithms and Compexity
Knowledge Areas in CS2013
CoreTier1 CoreTier2
0 1 2 3 4 5 6 7 8 9
Software Processes
CoreTier1 CoreTier2

12. Education: Typical Example
Creating programs vs. testing programs in BME’s curriculum
Semester Course Topics Testing
1 Basics of Programming 1 Basics, C, debugger -
2 Basics of Programming 2 OO, C++ (STL) -
3 Basics of Programming 3 Java JUnit example (2 hours)
3 Software Engineering V&V (2 hours)
12
…
5+ (Elective or spec. courses) Testing fundamentals
In other Hungarian universities similar situation, see:
K. Balla et al. “The Software Testing Curriculum in the Hungarian Education in Conjuction with
International Standards”. In: Informatika a felsőoktatásban 2011 konferencia , 2011. (In Hungarian)

13. Factors Affecting The Gap
Education Tools

14. Tools: Test Generation Tools
Model-based Testing Tools (17+13) Code-based Testing Tools (19+10)
14
http://mit.bme.hu/~micskeiz/pages/
modelbased_testing.html
http://mit.bme.hu/~micskeiz/pages/
code_based_test_generation.html
Several
mature tools
Mostly proto-types
(except
security testing)

15. Our Experiences with Code-based Tools
Evaluating Test Generators
 E.g. “Can it handle float numbers?”
15
Joint work with Lajos Cseppento
Tool 1
Tool 2
Tool 3
Tool 4

16. Our Experiences with Code-based Tools
Helping industrial adoption
 Tool for visualizing test generation problems
16
Joint work with David Honfi

17. Summary
Factors affecting the gap:
Education: Testing literacy of fresh graduates
Tools: Maturing prototype tools
Need: Do everyday testers need advanced tools and
techniques and have time for them?
17
Gap?