Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Hungarian Software Testing Forum (HUSTEF 2014) 
The Gap Between Academic Research and 
Industrial Practice in Software Tes...
The Gap: Examples 
“Automatic testing” 
Academia Industry 
2 
—> automatic test 
case generation 
—> automatic 
test execu...
The Gap: Examples 
“Code coverage” 
Academia Industry 
3 
—> “We defined the 
new <obscure> metric.. ” 
—> “Nice, we reach...
Illustrating the gap
(Some) Recent Papers in Academia 
Coverage Is Not Strongly Correlated with Test Suite Effectiveness 
An improved Pareto di...
Topics in Academia 
6 
Test case/input 
generation 
Test optimization and 
prioritization 
Empirical evaluations
(Some) Recent Talks in Industry 
Top Challenges in Testing Requirements 
How to Stress Test Your Mobile App 
International...
Topics in Academia and Industry 
8 
Test management 
Test automation (tools, 
technologies) 
Test design techniques 
Test ...
Reducing the Gap 
9
Factors Affecting The Gap 
Education Tools
Education: ACM/IEEE Comp. Sci. Curricula 
Software Engineering Knowledge Area 
Formal Methods 
Software Reliability 
Softw...
Education: Typical Example 
Creating programs vs. testing programs in BME’s curriculum 
Semester Course Topics Testing 
1 ...
Factors Affecting The Gap 
Education Tools
Tools: Test Generation Tools 
Model-based Testing Tools (17+13) Code-based Testing Tools (19+10) 
14 
http://mit.bme.hu/~m...
Our Experiences with Code-based Tools 
Evaluating Test Generators 
 E.g. “Can it handle float numbers?” 
15 
Joint work w...
Our Experiences with Code-based Tools 
Helping industrial adoption 
 Tool for visualizing test generation problems 
16 
J...
Summary 
Factors affecting the gap: 
Education: Testing literacy of fresh graduates 
Tools: Maturing prototype tools 
Need...
Upcoming SlideShare
Loading in …5
×

The Gap Between Academic Research and Industrial Practice in Software Testing

794 views

Published on

In software engineering, there is always a gap between the current research topics and the everyday industrial practice. However, in my experience this gap is much wider in software testing, e.g. advanced testing techniques seldom reach everyday testers. This gap can be attributed to several factors, the talk will highlight education and tools from the possible causes. In order to illustrate this gap the talk will collect and compere the topics of recent academic and industrial testing related conferences. My goal is to offer a glimpse into recent software testing research topics to practitioners, and start a discussion whether there is really a wide gap between academy and industry.

Published in: Software
  • Be the first to comment

  • Be the first to like this

The Gap Between Academic Research and Industrial Practice in Software Testing

  1. 1. Hungarian Software Testing Forum (HUSTEF 2014) The Gap Between Academic Research and Industrial Practice in Software Testing Budapest University of Technology and Economics Department of Measurement and Information Systems Zoltan Micskei assistant professor
  2. 2. The Gap: Examples “Automatic testing” Academia Industry 2 —> automatic test case generation —> automatic test execution
  3. 3. The Gap: Examples “Code coverage” Academia Industry 3 —> “We defined the new <obscure> metric.. ” —> “Nice, we reached 80% <statement> coverage.”
  4. 4. Illustrating the gap
  5. 5. (Some) Recent Papers in Academia Coverage Is Not Strongly Correlated with Test Suite Effectiveness An improved Pareto distribution for modelling the fault data of open source software Automatic test case generation from Simulink Stateflow models using model checking Adding Contextual Guidance to the Automated Search for Probabilistic Test Profiles Lookahead-Based Approaches for Minimizing Adaptive Distinguishing Sequences Design and industrial evaluation of a tool supporting semi-automated website testing Using Test Case Reduction and Prioritization to Improve Symbolic Execution Minimum Number of Test Paths for Prime Path and Other Structural Coverage Criteria Search-Based Testing for Embedded Telecom Software with Complex Input Structures Characterizing and Detecting Performance Bugs for Smartphone Applications 5 Enhancing Symbolic Execution with Veritesting Analysis and testing of black-box component-based systems by inferring partial models A Hitchhiker's guide to statistical tests for assessing randomized algorithms in software engineering Sample from ~130 papers from 7 conferences
  6. 6. Topics in Academia 6 Test case/input generation Test optimization and prioritization Empirical evaluations
  7. 7. (Some) Recent Talks in Industry Top Challenges in Testing Requirements How to Stress Test Your Mobile App Internationalization and Localization Testing Calculating the Cost of non-quality Breaking the Matrix - Android Testing at Scale Wisdom of Crowds in Software Testing Risk-Based Testing for Agile Projects Challenges and Solutions in Global Requirements Engineering 7 Web Performance Testing with WebDriver Key Test Design Techniques Appium: Automation for Mobile Apps Espresso: Fresh Start to Android UI Testing Designing for Testability: Differentiator in a Competitive Market Sample from ~80 papers from 5 conferences
  8. 8. Topics in Academia and Industry 8 Test management Test automation (tools, technologies) Test design techniques Test case/input generation Test optimization and prioritization Empirical evaluations
  9. 9. Reducing the Gap 9
  10. 10. Factors Affecting The Gap Education Tools
  11. 11. Education: ACM/IEEE Comp. Sci. Curricula Software Engineering Knowledge Area Formal Methods Software Reliability Software Evolution Software Verification Validation 0 5 10 15 20 25 30 35 40 45 50 Social Issues and Prof. Practice Systems Fundamentals Software Engineering Software Development Fundamentals Programming Languages Parallel and Distributed Computing Platform-Based Development Operating Systems Networking and Communication Intelligent Systems Information Management Inf. Assurance and Security Human-Computer Interaction Graphics and Visualization Discrete Structures Copmputational Science Architecture and Organization Software Construction Software Design Requirements Engineering Tools and Environments Joint Task Force on Computing Curricula, Association for Computing Software Project Machinery Management (ACM) and IEEE Computer Society. 2013. Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. ACM, New York, NY, USA. 11 Algorithms and Compexity Knowledge Areas in CS2013 CoreTier1 CoreTier2 0 1 2 3 4 5 6 7 8 9 Software Processes CoreTier1 CoreTier2
  12. 12. Education: Typical Example Creating programs vs. testing programs in BME’s curriculum Semester Course Topics Testing 1 Basics of Programming 1 Basics, C, debugger - 2 Basics of Programming 2 OO, C++ (STL) - 3 Basics of Programming 3 Java JUnit example (2 hours) 3 Software Engineering V&V (2 hours) 12 … 5+ (Elective or spec. courses) Testing fundamentals In other Hungarian universities similar situation, see: K. Balla et al. “The Software Testing Curriculum in the Hungarian Education in Conjuction with International Standards”. In: Informatika a felsőoktatásban 2011 konferencia , 2011. (In Hungarian)
  13. 13. Factors Affecting The Gap Education Tools
  14. 14. Tools: Test Generation Tools Model-based Testing Tools (17+13) Code-based Testing Tools (19+10) 14 http://mit.bme.hu/~micskeiz/pages/ modelbased_testing.html http://mit.bme.hu/~micskeiz/pages/ code_based_test_generation.html Several mature tools Mostly proto-types (except security testing)
  15. 15. Our Experiences with Code-based Tools Evaluating Test Generators  E.g. “Can it handle float numbers?” 15 Joint work with Lajos Cseppento Tool 1 Tool 2 Tool 3 Tool 4
  16. 16. Our Experiences with Code-based Tools Helping industrial adoption  Tool for visualizing test generation problems 16 Joint work with David Honfi
  17. 17. Summary Factors affecting the gap: Education: Testing literacy of fresh graduates Tools: Maturing prototype tools Need: Do everyday testers need advanced tools and techniques and have time for them? 17 Gap?

×