Successfully reported this slideshow.

Classic Formal Methods Model Checking

2,666 views

Published on

  • Be the first to comment

Classic Formal Methods Model Checking

  1. 1. Classic Model Checking in Reactive Software Systems US Naval Postgraduate School Donna A. Dulo US Dept of Army Fall 2007 SW 4920 Formal Verification & Validation of Reactive Software Systems
  2. 2. Model Checking is a cornerstone of verification engineering. - Amir Pneuli, 1996 Turing Award Recipient SW 4920 Formal Verification & Validation of Reactive Software Systems
  3. 3. Agenda Introduction to Classic Model Checking The Concept The Issues Classic Model Checkers Future Trends Conclusion SW 4920 Formal Verification & Validation of Reactive Software Systems
  4. 4. Introduction to Classic Model Checking SW 4920 Formal Verification & Validation of Reactive Software Systems
  5. 5. Classic Model Checking Classic Model Checking refers to the Set of Non-Execution Based Algorithmic Approaches for Checking a Property expressed as:  A Linear Time Logic (LTL) Formula  A Computational Tree Logic (CTL) Formula  A CTL* Formula  A Finite State Automaton Against a model, which can be expressed as:  A Finite State Machine  A Kripke Structure SW 4920 Formal Verification & Validation of Reactive Software Systems
  6. 6. Classic Model Checking • Verification of finite state systems • Verification is performed automatically • Exhaustive search of the state space of the system to determine if the specification is true or false • Terminates with a Boolean: T/F (yes/no) that demonstrates a behavior which falsifies a property • Always provides a counterexample • Fault Trace • Clues for fixing the problem SW 4920 Formal Verification & Validation of Reactive Software Systems
  7. 7. Classic Model Checking • Implemented by Algorithms with reasonable efficiency • Can be used with other methodologies such as abstraction and induction to verify some systems that are not finite space • Transform an academic discipline into a viable practical technique for designers in the field • Simulations can be run by non-mathematicians in industrial settings SW 4920 Formal Verification & Validation of Reactive Software Systems
  8. 8. Why Formal Verification Through Model Checking? • Advent of Complex, Safety Critical Systems • Exponential growth in possible state space in complex system of systems designs that need to be evaluated • What trajectories will have the fatal flaw in the system? • Formal verification conducts an exhaustive exploration of all possible behaviors • Goal is completeness in exploration • Overarching goals are Safety, Predictability and Reliability of system software SW 4920 Formal Verification & Validation of Reactive Software Systems
  9. 9. The Concept of Classical Model Checking SW 4920 Formal Verification & Validation of Reactive Software Systems
  10. 10. The Goal of Classic Model Checking Program void addSpaceParameters(Object 1) { buffer[wing1] = x; System OK buffer [wing 2] = y; resolution = x + y; } Object SW4920() { … Model } Checker ERROR Trace Requirements Requirement 1 …. Requirement 2 … Requirement 3 … …. Requirement n SW 4920 Formal Verification & Validation of Reactive Software Systems
  11. 11. The Implementation of Classic Model Checking Finite State Model System OK Model Checker ERROR Trace Temporal Logic Formula Error 1… Φ ( −> ◊ Ω) Error 2… Error 3… … Error n SW 4920 Formal Verification & Validation of Reactive Software Systems
  12. 12. Applying Model Checking to a Design In general, applying model checking to a design is comprised of three critical steps: 1. Modeling 2. Specification 3. Verification SW 4920 Formal Verification & Validation of Reactive Software Systems
  13. 13. Applying Model Checking to a Design Modeling • Conversion of design into a Formalism accepted by Model Checker • In some cases, this is a simple compilation task • For time and memory limited systems, abstraction is required • Elimination of Irrelevant details • Elimination of Unimportant/Trivial details SW 4920 Formal Verification & Validation of Reactive Software Systems
  14. 14. Applying Model Checking to a Design Specification • Statement of the properties the design must satisfy • Presented in a Logical Formalism • Use of temporal logic to assert how system behavior will evolve over Time • Critical Issue  Completeness • Impossible in complex systems to cover entire state space • Goal is as much completeness as possible in the specification SW 4920 Formal Verification & Validation of Reactive Software Systems
  15. 15. Applying Model Checking to a Design Verification • In an ideal situation, it is completely automatic, although most results need to be manually verified • Negative Result Case: • Error Trace is Provided • Used as a counterexample for the checked property • Assists designed in tracing error • Once error is traced and rectified, model checking needs to be repeated SW 4920 Formal Verification & Validation of Reactive Software Systems
  16. 16. Classic Model Checking in the Waterfall Model Classic Model Checking Requirements Design Code Test Maintenance “Modern” Model Checking SW 4920 Formal Verification & Validation of Reactive Software Systems
  17. 17. The Issues in Classical Model Checking SW 4920 Formal Verification & Validation of Reactive Software Systems
  18. 18. Benefits of Classic Model Checking • Automatic Verification • Invariants & Safety Properties • Absence of Dead-lock and Live-lock • Complex Event Sequencing Properties • In contract to testing: Gives Complete Coverage through Exhaustive Exploration of all System Paths • A solid compliment to other software quality assurance techniques • Used successfully for years in the Hardware and Protocol realms SW 4920 Formal Verification & Validation of Reactive Software Systems
  19. 19. Challenges of Classic Model Checking Model Construction SW 4920 Formal Verification & Validation of Reactive Software Systems
  20. 20. Challenges of Classic Model Checking Model Construction Program Finite State Model void addSpaceParameters(Object 1) { buffer[wing1] = x; Model buffer [wing 2] = y; resolution = x + y; GAP Checker } Object SW4920() { … } Semantic Gap: Programming Languages - Methods, Inheritance, Exceptions, etc……………. Model Description Languages - Automata SW 4920 Formal Verification & Validation of Reactive Software Systems
  21. 21. Challenges of Classic Model Checking Property Specification SW 4920 Formal Verification & Validation of Reactive Software Systems
  22. 22. Challenges of Classic Model Checking Property Specification It’s Difficult to Formalize a Requirement in Temporal Logic “Between the key being inserted and the key being removed, the ignition can be activated at most twice.” []((keyIn / <>keyRem) -> ((!activate / !keyRem) U (keyRem / ((activate / !keyRem) U (keyRem / ((!activate / !keyRem) U (keyRem / ((activate / !keyRem) U (keyRem / (!activate U keyRem)))))))))) SW 4920 Formal Verification & Validation of Reactive Software Systems
  23. 23. Challenges of Classic Model Checking State Explosion SW 4920 Formal Verification & Validation of Reactive Software Systems
  24. 24. Challenges of Classic Model Checking State Explosion • Costly in Terms of Time and Memory • A single 32 bit integer variable induces effectively 2^32 states • Explosive growth limits software scalability • Large or complex systems are impossible to verify • Completeness is not possible SW 4920 Formal Verification & Validation of Reactive Software Systems
  25. 25. Challenges of Classic Model Checking Output Interpretation SW 4920 Formal Verification & Validation of Reactive Software Systems
  26. 26. Challenges of Classic Model Checking Output Interpretation • Raw Error Traces may be 1000’s of Lines Long ERROR • Mapping of Error Traces to Code Can be Difficult Trace • Mapping made Difficult by: Error 1… • Semantic Gap Error 2… Error 3… … • Encoding of Complex Features Error n • Multiple Optimizations • Multiple Transformations SW 4920 Formal Verification & Validation of Reactive Software Systems
  27. 27. Classic Model Checking Programs SW 4920 Formal Verification & Validation of Reactive Software Systems
  28. 28. Classic Model Checkers Most popular developed in academia Classic Model Checkers available in the public domain: SPIN SMV SW 4920 Formal Verification & Validation of Reactive Software Systems
  29. 29. SMV Classic Model Checker • Symbolic Model Verifier • Developed in early 1990’s by Kenneth McMillan • Can verify specifications up to 10^1300 reachable states • Based on CTL • Input: • State Machine Model • Temporal Logic Formulas • SMV checks to see that the formulas are consistent with the model • Reports an Error Trace and in some cases a Counterexample • Open Source with many Variations/Extensions SW 4920 Formal Verification & Validation of Reactive Software Systems
  30. 30. SPIN Classic Model Checker • Automata Based Model Checking Tool • Developed in 1995 with significant popularity since • Promela (Process Meta Language) • LTL based  Formulae Converted to Buchi automata • SPIN can also act as a simulator following one possible execution path through the system and providing an execution trace • Supports modeling of asynchronous distributed algorithms and non-deterministic automata • Options: • Partial Order Reduction • State Compression • Weak Fairness Enforcement SW 4920 Formal Verification & Validation of Reactive Software Systems
  31. 31. Model Checking Example in ADA There are seven steps in model checking properties of an Ada program: 1. Conversion of the original Ada program to a safe finite-state variant 2. Translation to SEDL (SmallEiffel Dynamic Linking) compiler 3. Definition of program features to be reasoned about 4. Translation to model checker input 5. Property specification using defined features 6. Execution of the model checker 7. Interpretation of counter-examples for false results SW 4920 Formal Verification & Validation of Reactive Software Systems
  32. 32. Model Checking Example in ADA LTL State/Event Predicate Definitions SPIN Promela True Program Abstraction, or ADA Configuration, ADA to SEDL Restriction & INCA Counter- Source SEDL Specification Ada Example TRANS SMV CTL An ADA Translation Toolset SW 4920 Formal Verification & Validation of Reactive Software Systems
  33. 33. Variants of Classic Model Checking SW 4920 Formal Verification & Validation of Reactive Software Systems
  34. 34. Variants of Classic Model Checking Symbolic Trajectory Evaluation (STE) Generalized Symbolic Trajectory Evaluation (GSTE) Model Checking with Theorem Proving FORTE SW 4920 Formal Verification & Validation of Reactive Software Systems
  35. 35. Symbolic Trajectory Evaluation (STE) • Lattice Based Model Checking based on Symbolic Simulation • Less Sensitive to State Explosion • Easier to Use than Classic Model Checkers • Medium to Large Design Checking with High Automation • Technological Breakthroughs in Model Checking • Dynamic Model Pruning • Advanced Parametric Techniques • Case Splitting on Input Data Space • Limitation: Finite Model Space SW 4920 Formal Verification & Validation of Reactive Software Systems
  36. 36. Generalized Symbolic Trajectory Evaluation (GSTE) • Generalized Theory of STE • Extends STE Model Checking to include Verification of Ω-regular Properties • Maintains Time and Space Efficiency • Infinite Space • Still in Research & Development SW 4920 Formal Verification & Validation of Reactive Software Systems
  37. 37. Classic Model Checking with Theorem Proving • A Promising Approach to Verifying Large Systems with Complex Properties that do not Lend Themselves well to Either Approach Alone • Model Checking has Capacity Limits that Need to be Decomposed • Theorem Proving Composes and Decomposes Model Checking Results instead of Formalizing Arbitrary Mathematical Results • Model Checking and Theorem Proving are Tightly Integrated • Result: Model Checking Results with no Need for Translation or Reformulation • Increased Capacity Space • More Efficiency SW 4920 Formal Verification & Validation of Reactive Software Systems
  38. 38. FORTE Formal Tools Environment • Intel Custom Built Verification Environment • Classic Model Checking Engines • Binary Decision Diagrams • Theorem Proving • “FL” Functional Programming Language (Strongly Typed) • Symbolic Trajectory Evaluation • Next Generation Platform includes reFLect Programming Language • Redesign of FL Core • Enhanced Support for Reflection • A Platform Well Suited for Describing and Manipulating High Level Design Specifications SW 4920 Formal Verification & Validation of Reactive Software Systems
  39. 39. Conclusion on Classic Model Checking SW 4920 Formal Verification & Validation of Reactive Software Systems
  40. 40. Coverage Space of Classic Model Checking • Limited in Specification Coverage Specification Coverage • Specifications are Text Based • Difficult to Visualize • Difficult to Validate • Very Limited in Program Coverage • State Explosion Problem • Limited to Finite State Verification Coverage Program Coverage • Abstractions Required • High Verification Coverage The Coverage Cuboid • Automatic • 100% Coverage • No Special Drivers Required SW 4920 Formal Verification & Validation of Reactive Software Systems
  41. 41. Cost Space of Classic Model Checking • High in Specification Cost Specification Cost • Specifications are Text Based • Difficult to Visualize • Difficult to Validate • High in Program Cost • State Explosion Problem • Limited to Finite State Verification Cost • Abstractions Required • Low in Verification Cost The Cost Cuboid • Automatic Program Cost • 100% Coverage • No Special Drivers Required SW 4920 Formal Verification & Validation of Reactive Software Systems
  42. 42. Questions? SW 4920 Formal Verification & Validation of Reactive Software Systems
  43. 43. References Clark, E.M., Grumberg, O., & Peled, D.A. (1999). Model Checking. MIT Press: Cambridge. Corbett, J.C. & Pasareneau, C. (2007). Translating Ada programs for Model Checking. University of Hawaii. Drusinski, D., Michael, J.B., & Shing, M. (2007). “Three Dimensions of Formal Validation and Verification of Reactive System Behaviors. US Naval Postgraduate School, NPS-CS-07-008. Dwyer, M., Hatcliff, J. & Avrunin, G. (2004). Software Model Checking for Embedded Systems. Kansas State University. Intel Corporation. (2007). “Classic Model Checking Introduction”. www.intel.com. SW 4920 Formal Verification & Validation of Reactive Software Systems

×