SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
Slides for a talk on search-based testing for Event-B models
Slides for a talk on search-based testing for Event-B models
1.
13th CREST Open Workshop<br />12th-13th of May 2011, London<br />Search-Based Software Engineering for Model-Based Testing<br />Event-B in a Nutshell<br />Test Generation <br />Approaches<br />SBT Challenges<br />Test Data Generation<br />Finite Model <br />Learning<br />Conclusions<br />Alin Stefanescu - University of Pitesti, Romania<br />*) joint work with colleagues from DEPLOY project<br />
9.
Developement supported by French and European projects:</li></ul> FP6 RODIN and FP7 DEPLOY<br />
10.
DEPLOY project (2008-2012) – funded by FP7 <br />DEPLOY :: Industrial deployment of advanced systemengineering methods for high productivity and dependability using formal methods <br /><ul><li>4 industrial partners
43.
Test generation based on Event-B<br />Model-Based Testing (MBT)<br />Model-based testing (MBT) is a newlyintroduced topic in DEPLOY<br />priority topic for industrial partners like SAP<br />challenges due to the sheer size of the state space of real-life scenarios<br />Weinvestigate search-based testing (SBT) techniques for Event-B.<br />
44.
Future MBT plugin in RODIN<br />Event-B model<br />MBT Plug-in<br />MBT <br />Users<br />{ Model-checking }<br />{ Constraint-based }<br />Extra <br />test <br />information<br />{ Search-based }<br />Test cases<br />University of Pitesti and University of Dusseldorf<br />Tool developers:<br />
45.
Test generation from Event-B<br />SBT Opportunity! <br />1. Generate a set of tests<br />(sequence of events with concrete param.)<br />2. Optimize test suite<br />(according to some criteria) – if still needed<br />Event-B model<br />ev2(..),ev5(..),...ev3()<br />ev4(..),ev2(..),...ev4(..)<br />...<br />...<br />...<br />...<br />...<br />...<br />...<br />...<br />ev3(..),ev7(..),...ev5(..)<br />...<br />...<br />...<br />...<br />...<br />ev6(..),ev5(..),...ev8<br />Global variables:<br /> var1, var2, var3, ...<br />Events<br /> ev1(p11,...), <br /> ev2(p21,...), <br /> ev3(p31,...),<br /> ... <br />ev2(..),ev5(..),...ev3()<br />ev4(..),ev2(..),...ev4(..)<br />...<br />ev3(..),ev7(..),...ev5(..)<br />
46.
What is the explicit state space<br />State Space of <br />the Event-B model<br />(3,4,{a,b},...)<br />Abstract machine<br />Event-B model<br />ev3(5)<br />Global variables:<br /> var1, var2, var3, ...<br />Events<br /> ev1(p11,...), <br /> ev2(p21,...), <br /> ev3(p31,...),<br /> ... <br />...<br />...<br />...<br />...<br />States given by the values of global variables<br />Transitions labeled by events with concrete parameters<br />
47.
Test generation from Event-B<br />State Space of <br />the Event-B model<br />(3,4,{a,b},...)<br />Approach 1:<br />Explore the state space using the <br /> ProB model checker<br />state space explosion mainly due to data<br />Try: guide the search<br />ev3(5)<br />...<br />...<br />...<br /> SBT Opportunity! <br />
48.
Test generation from Event-B – part II<br />State Space of <br />the Event-B model<br />(3,4,{a,b},...)<br />Approach 2:<br />Explore state space ignoring the data<br />(i.e. local parameters)<br />Problem 1: still large state space<br />Then: construct approximations of state space <br /> up to depth K using finite automata <br />Try 1: using machine learning and static analysis<br />Try 2: using evolutionary algorithms?<br />Problem 2: infeasible sequences<br />Try 1: constraint solving for path feasibility<br />Try 2: test data generation with metaheuristics<br /> ev3<br />...<br />...<br />...<br /> SBT Opportunities! <br />
50.
Let’s take a look at some of specific challenges for Event-B...<br />More details in:<br />A. Stefanescu, F. Ipate, R. Lefticaru, C. Tudose. <br />Towards Search-Based Testing for Event-B Models. <br />To appear in Proc. of 4th International Workshop on Search-Based Software Testing (SBST), 2011.<br />
51.
No explicit state space<br />Fact<br />Event-B has no explicit states like the EFSMs <br />no control state (as in EFSMs)<br />Problem<br />Large (possibly infinite) state space<br />testing coverage criteria must be defined<br />only recent work addressing SBT for EFSMs<br />Possible ideas:<br />coverage of all events (or a given subset of them) or coverage of all test paths of length < K<br />many other coverages possible, so industrial guidance is needed<br />consider the class of Event-B models with a special state variable (see industrial use cases from SAP, SSF, Bosch and UML-B models) <br />
52.
Non-numerical types<br />Fact<br />Event-B is based on set theory<br />set relations, powersets, functions, set comprehensions, products, records, etc.<br />Complex structured data (e.g. business domain)<br />Problem<br />fitness functions in literature mostly defined for numerical types<br />Possible solutions<br />design new fitness functions for set-based (non-numerical) types<br />efficient encoding of mixed non-numerical/numerical test data<br />
53.
Hierarchical models<br />Fact<br />Event-B supports different types of hierarchy<br />refinement from abstract to concrete levels<br />model decomposition<br />modularity <br />most industrial models use some sort of hierarchy (due to size) <br />Problem<br />no much previous work on SBT addressing hierarchical models<br />Possible ideas:<br />adapt existing work on test selection for hierarchical state machines<br />use the existing ProB model checker that can partially deal with hierarchy<br />
54.
Non-determinism<br />Fact<br />Event-B has different types of non-determinism<br />:| or :∈ operators (e.g. x :∈ {item1, ..., item20})<br />non-deterministic choice of the event to be executed when several enabled<br />non-deterministic choice of parameters (ANY construct) <br />non-deterministic initialisation of variables satisfying the set of invariants<br />Problem<br />no much previous work on SBT addressing non-deterministism<br />Possible ideas:<br />devise fitness functions that improve the chance of choosing a given path in a non-deterministic model (under certain assumptions)<br />make the non-determinism visible (model instrumentation)<br />
56.
Generating test data for a path<br />Problem<br />Given one path of events, provide the test data (event parameters) that enables the execution of the path.<br />Approach<br />genetic algorithms<br />encoding of sets into binary genes<br />mixed choromosomes (numerical and binary genes)<br />More details in:<br />I. Dinca, A. Stefanescu, F. Ipate, R. Lefticaru, C. Tudose. <br />Test Data Generation for Event-B Models using Genetic Algorithms. <br />In Proc. of 2nd International Conference on Software Engineering and Computer Systems (ICSECS'11). CCIS Series, vol. 181, pp. 76-90, Springer, 2011.<br />
63.
using statistical test like t-test and U-test</li></ul>And (of course) the winner is:<br /><ul><li>GA performs significantly better than RT on most paths </li></ul>Note: We are currently evaluating constraint-solving (mature for Event-B). It It seems to be quicker for small to medium path (with exceptions).<br />
65.
Generating finite models from Event-B<br />Problem<br />There is no explicit state space of an Event-B model<br />Approach<br />Finite automata learning (adapted L* algorithm )<br />Aproximation through cover automata<br />K-bound on the length of executions<br />Use finite automata for conformance test generation<br />More details in:<br />F. Ipate, I. Dinca, A. Stefanescu:<br />Model Learning and Test Generation for Event-B using Cover Automata. <br />Submitted to SEFM’11.<br />
66.
First experiments<br />SBT Opportunity?! <br />Preliminary approach<br />Approximation through cover automata for bound l<br />Incremental -> fits very well with model refinements<br />Minimal finite automata<br /> Sometimes difficult to find counterexamples (to the approximation)<br />Scales for medium size models:<br />Boundl<br />ev3<br />...<br />...<br />...<br />
68.
Opportunities for Search-Based Techniques<br />To wrap-up opportunities:<br /><ul><li>Test suite minimisation with multi-objective optimisation
69.
Test data generation for one path with search-based algorithms
70.
Construct finite models with evolutionary algorithms
71.
Combine ProB model-checker with meta-heuristics
72.
Combine ProB constraint-solver with meta-heuristics
73.
Experiment with different search algorithms (PSO, ACO, SA,...)</li></ul>To be answered until end of project (April 2012): <br />Which of the above work good in practice?<br />
0 likes
Be the first to like this
Views
Total views
1,193
On SlideShare
0
From Embeds
0
Number of Embeds
4
You have now unlocked unlimited access to 20M+ documents!
Unlimited Reading
Learn faster and smarter from top experts
Unlimited Downloading
Download to take your learnings offline and on the go
You also get free access to Scribd!
Instant access to millions of ebooks, audiobooks, magazines, podcasts and more.
Read and listen offline with any device.
Free access to premium services like Tuneln, Mubi and more.