Your SlideShare is downloading. ×

Model-based Testing Principles


Published on

An introductory lecture to model-based testing.

An introductory lecture to model-based testing.

Published in: Education

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Università degli Studi dell’AquilaL22. Model-based Testing (Principles) Henry Muccini DISIM, University of L’Aquila,
  • 2. Copyright NoticeThe material in these slides may be freely reproducedand distributed, partially or totally, as far as an explicitreference or acknowledge to the material author ispreserved. Henry Muccini
  • 3. AgendaModelingMBT: →Why? →Modelsfor MBT →UML-based Testing ─ Testing activities
  • 4. AGENDAModeling Model-based testing Why? Models for MBT UML-based Testing
  • 5. Modeling: there is not just programmingWhat is a model: →A model is a simplification of the reality →The reality is abstracted, since it is too complexWhy modeling →Models are built for a better understanding of the system under developmentEverything is a model ?!?
  • 6. Non-software Models …
  • 7. Specification: What
  • 8. Software Models
  • 9. Other Software Models (Web models)
  • 10. … other software models…
  • 11. … and many others… Pilot Pilot CoPilot CoPilot Multifunction Multifunction Multifunction Multifunction Display1 Display2 Display1 Display2 Display Display Display Display Processor Processor Processor Processor High speed network High speed network Mission Mission Mission Processor Processor Processor 1553 bus 1553 bus Auto-Pilot GPS Nav Radio
  • 12. Model Driven Development (MDD)Models are becoming first core assets inSoftware Engineering/Architecting: →models are primary artifacts retained as first class entities that can be analyzed and manipulated by means of automated tools →the focus moves from writing and sharing documents to writing and sharing models.
  • 13. From Document Driven to Model Driven
  • 14. Models in MDDModels in MDD can be used for: →Documenting design decisions →Guiding the coding phase →For analyzing and validating design choices ─ by complementing traditional code-level analysis techniques
  • 15. Why modeling? For Documenting and Analysis
  • 16. What is MBT?It consists in: →ExtractingTest Cases from the Model →Run the test case over the system implementationPurpose: →Tovalidate the implementation conformance to the model ─ The model itself is the oracle, i.e., it represents the expected behavior
  • 17. Model-based TestingA model based testing approachaccepts two main inputs →a model of the software under test, →a set of test generation directives which guide the test cases selectionand outputs a test specification →which includes a set of stimuli the tester should introduce in the system together with expected responses
  • 18. Modeling for testingEssentially, we need (at least) a behavioral model ofthe software system: →Labelled Transition System →StateCharts →UML State of Sequence Diagrams →IOLTS →Finite State Automata →…
  • 19. The base-line technique for designing test cases →Timely ─ Often useful in refining specifications and assessing testability before code is written →Effective ─ finds some classes of fault (e.g., missing logic) that can elude other approaches →Widely applicable ─ to any description of program behavior serving as spec ─ at any level of granularity from module to system testing. →Economical ─ typically less expensive to design and execute than structural (code-based) test cases(C) 2007 MAURO PEZZÈ & MICHAL YOUNG
  • 20. Program code is not necessary →Only a description of intended behavior is needed →Even incomplete and informal specifications can be used ─ Although precise, complete specifications lead to better test suitesEarly functional test design has side benefits →Often reveals ambiguities and inconsistency in spec →Useful for assessing testability ─ And improving test schedule and budget by improving spec →Useful explanation of specification ─ or in the extreme case (as in XP), test cases are the spec(C) 2007 MAURO PEZZÈ & MICHAL YOUNG
  • 21. Different testing strategies (functional, structural,fault-based, model-based) are most effective fordifferent classes of faultsFunctional testing is best for missing logic faults →A common problem: Some program logic was simply forgotten →Structural (code-based) testing will never focus on code that isn’t there!(C) 2007 MAURO PEZZÈ & MICHAL YOUNG
  • 22. Modeling for testingSuch a model can be produced: 1. Through a formal specification language ─ Typically referred as formal testing 2. Through a diagrammatic (visual) notation ─ Typically referred as UML-based Testing
  • 23. 1. MBT based on Formal SpecificationsSince the 80’s, many specification-based testingapproaches have been proposed, based on formallanguages such as →Z, VDM, CSP, CCS, LOTOS, SDL, and Petri Nets.More recently: →Based on dynamic behavior: ─ FSM-based (Bochmann&Petrenko, Lee&Yannakis, …) ─ LTS-based (Brinksma&Tretmans, Jard&Jeron …) →Focussing on static aspects, ─ ADT theory (Bernot&Gaudel…) ─ Z-based (Hierons…)
  • 24. Formal Specification-based TestingTorX (Côte de Resyste) → on-the-fly test generation and execution → random → LOTOS and PromelaTGV (IRISA - Rennes) →derives tests in TTCN from LOTOS or SDL → uses test purposesTVEDA (CNET - France Telecom) → derives TTCN tests from SDL specification
  • 25. 2. MBT based on the UMLUML State-machine based testing: →which might require the translation of the UML diagrams into an intermediate formal descriptionScenario-based testing: →based on the use of opportunely annotated UML Sequence DiagramState-based and Scenario-based:combine the use of interaction diagrams and state diagram,possibly annotated with further (formal) information (e.g. OCL)
  • 26. Model-based Testing: the overall idea From Leila Naslavsky Advancement report
  • 27. AsmL L TESTO OR UMLAU UTModel-based Testing Approaches UMLTe est AGEDI IS TOTEM M SOOT TF COW-SU UITE SCENTO OR SeDiTe eC UCSC-Sy ystem
  • 28. Main Testing ActivitiesTest Selection/Generation: →it consists in selecting a suitable and finite set of test cases from the possibly infinite set;Test Execution and Evaluation: →it consists in executing the code accordingly to the selected test cases and comparing real and expected results;
  • 29. Test SelectionA test case, in MBT, is typically a possible“scenario” →In scenario-based testing: ─ A scenario itself is an abstract test case →In state-based testing ─ Execution flows are extraced from the state machine and represent a test case ─ MB coverage criteria are applied on this model
  • 30. Test Case SelectionAbstract Test case = path over the LTS/FSM,corresponding to sequences of events ─ They are not “concrete” test cases, but abstract (depending on the specification) →Concrete Test cases = mapping asbtract test cases into execution statements to be launched
  • 31. Test Case Selection Model of the Test system directives inSD 1 1 C1 C2 m9 m1 Test specification outSD12 C1 C2 C3 m9 m3 m8 m1
  • 32. Test Case Selection
  • 33. Test ExecutionIt consists in running the test cases on the systemimplementation →So to compare the real execution with the expected behavior
  • 34. MBT practical concernsTraceability, i.e. relating the abstract values of the specification to the concrete values of the implementation.Execution, i.e. forcing the Implementation Under Test (IUT) to execute the specific sequence of events that has been selected.
  • 35. Testing and IndustryIn order to be suitable for industrial needs a testingapproach has to emphasize the following qualities: →Usability ─ Not ad-hoc models →Timeliness ─ Soon and even incomplete Opposed to →Tool support accuracyGoal of a suitable testing approach should be“to improve the test results accuracy without sensiblyraising the testing effort”
  • 36. MBT ToolsThere are some automated tools for MBT: →Telelogic Tau and Tau Tester →Leirios Test Designer →Conformiq Qtronic →Rhapsody Test Conductor and Automatic Test Generation →AGEDIS
  • 37. MBT in industry: challenges and issuesAs remarked by Alan Hartman in his 2006 speech at ISSTA: →“Only 1% of industry uses it”Why?
  • 38. Why?The main reasons I foresee: →Model2Code (and viceversa) traceability problems →modeling is not yet so widely recognized as part of the development process. ─ Legacy systems →because of tool/service problems →cost/effectiveness is still under exploration
  • 39. Conformiq: Two models are required inInput•SutModel.Java (in QML)•SutModel.xmi (Graphical Notation)
  • 40. QMLJust like Java, QML consists of variables andexpressions, all strictly typed and known before thecompilation.The types are divided into the same three groups thatJava has: primitives, references and values
  • 41. QMLQML extends Java with: Records Ports System block State machines
  • 42. SutModel Graphical
  • 43. SutModel Graphical
  • 44. Coverage Figure 5 – The Coverage Editor window