State Space Exploration      for NASA’sSafety Critical Systems         Dimitra Giannakopoulou         NASA Ames Research C...
model checking  program / model   void add(Object o) {                            model checker                           ...
model checking vs. testing        testing              model checking
is it a good idea?                     Turing Award 2007               E. Clarke, A. Emerson, J. Sifakis   § Targets subt...
state-explosion problem
compositional  verification
collaborators  §  Corina Păsăreanu (CMU / NASA Ames)  §  and talented students / visitors:     Howard Barringer (Profess...
compositional verification does system made up of M1 and M2 satisfy property P?                    "   check P on entire s...
§ when we try to pick out anything by itself, we find    it hitched to everything else in the universe                   ...
assume-guarantee reasoning               introduces assumptions / reasons about triples:                       〈A〉 M 〈P〉 i...
what is a good assumption?can assumptions be generated?
examples of assumptions  §  will not invoke close on a file if open has not previously      been invoked  §  accesses to...
formalisms  §  components modeled as finite state machines (FSM)     –  FSMs assembled with parallel composition operator...
example  require in and out to alternate (property Order)                                                     Input   in  ...
parallel composition    Input                          Output            in         send           send         out       ...
property satisfaction                                            Ordererr                                                 ...
assume-guarantee reasoning     Input             in          send                                             Ordererr    ...
the weakest assumption [ASE 2002]                            P                  M              WA   §  given component M,...
learning assumptions (TACAS 2003)iterative solution +intermediate results          L* learns unknown regular language     ...
weakest assumption in AG reasoning     1.  〈A〉 M1 〈P〉     2.  〈true〉 M2 〈A〉      weakest assumption makes                 ...
L* learner                           the oraclequeries:should word w be included in L(A)?                                 ...
oracle for WA in assume-guarantee reasoning                   true / false                             1.  〈A〉 M1 〈P〉     ...
characteristics assumptions conjectured by L* are not comparable semantically  "   terminates with minimal automaton A for...
example Input                                 Ordererr                     Output                                         ...
more than 2 components…§  extension of basic rule ASYM [TACAS 2003, FMSD 2009]§  symmetric / circular rules [SAVCBS 2003...
recursive application of ASYM                                       1.  〈A2〉 M2 〈A1〉                                      ...
symmetric learning framework [SAVCBS05]  refine A1                                                                  refine...
example 1: Mars Exploration Rover §  tools: LTSA, SPIN §  model derived from JPL s Mars Exploration     Rover (MER) Reso...
autonomous rendezvous & docking §  tool: LTSA §  consists of control software, state estimator, and 4 types of sensors §...
example 3: K9 Rover Executive                                                                        K9 Rover§  tools: LT...
interfacegeneration
component interfaces§  beyond syntactic interfaces (open file before close)§  document implicit assumptions§  safe: acc...
JavaPathfinder       UML statecharts   assume-guarantee reasoning   interface generation / discharge                      ...
example: crew exploration vehicle  §  tool: JavaPathfinder  §  UML statechart model of the      Ascent and EarthOrbit fl...
results    Assumption 1:                         Assumption 2:      lasJetisson                         lsamRendezvous    ...
infinite-state components…
infinite components [CAV 2010]§  use predicate abstraction (e.g., x ≥ 0, x < 0)§  generate may and must abstraction     ...
Query(σ, C)1.    if checkSafe(σ,Cmust) != null        Lillegal(Cmay)	           Llegal(Cmust)	  2.          return no     ...
summary§  automating compositional verification was a breakthrough§  our techniques are generic§  not a panacea…      –...
http://babelfish.arc.nasa.gov/trac/jpf                   model                  checking
Upcoming SlideShare
Loading in …5
×

State Space Exploration for NASA’s Safety Critical Systems

1,111 views
1,002 views

Published on

Dr. Dimitra Giannakopoulou gave a lecture on State Space Exploration for NASA's Safety Critical Systems in the Distinguished Lecturer Series - Leon The Mathematician.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,111
On SlideShare
0
From Embeds
0
Number of Embeds
40
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

State Space Exploration for NASA’s Safety Critical Systems

  1. 1. State Space Exploration for NASA’sSafety Critical Systems Dimitra Giannakopoulou NASA Ames Research Center
  2. 2. model checking program / model void add(Object o) { model checker property buffer[head] = o; head = (head+1)%size; } Object take() { always(ϕ or … tail=(tail+1)%size; return buffer[tail]; } NO + counterexample: YES (property holds) (provides a violating execution)
  3. 3. model checking vs. testing testing model checking
  4. 4. is it a good idea? Turing Award 2007 E. Clarke, A. Emerson, J. Sifakis § Targets subtle (concurrency) errors § Input language § Safety critical applications § Properties § Successful in hardware industry § Computational complexity
  5. 5. state-explosion problem
  6. 6. compositional verification
  7. 7. collaborators §  Corina Păsăreanu (CMU / NASA Ames) §  and talented students / visitors: Howard Barringer (Professor, Univ. of Manchester) Colin Blundell (Upenn, now IBM) Jamieson Cobleigh (UMass, now MathWorks) Michael Emmi (UCLA) Mihaela Gheorgiu (Univ. of Toronto, now NASA JPL) Chang-Seo Park (UC Berkeley) Suzette Person (Univ. of Nebraska, now NASA Langley) Rishabh Singh (MIT)
  8. 8. compositional verification does system made up of M1 and M2 satisfy property P? "   check P on entire system: too many states! M1 "   use system s natural decomposition into components to break-up verification tasksatisfies P? "   check components in isolation: A M2 does M1 satisfy P?
  9. 9. § when we try to pick out anything by itself, we find it hitched to everything else in the universe John Muir
  10. 10. assume-guarantee reasoning introduces assumptions / reasons about triples: 〈A〉 M 〈P〉 is true if whenever M is part of a M1 system that satisfies A, then the system must also guarantee Psatisfies P? A simplest assume-guarantee rule (ASYM): M2 1.  〈A〉 M1 〈P〉 discharge the 2.  〈true〉 M2 〈A〉 assumption 〈true〉 M1 || M2 〈P〉
  11. 11. what is a good assumption?can assumptions be generated?
  12. 12. examples of assumptions §  will not invoke close on a file if open has not previously been invoked §  accesses to shared variable X must be protected by lock L §  (rover executive) whenever thread A reads variable V , no other thread can read V before thread A clears it first §  (spacecraft flight phases) a docking maneuver can only be invoked if the launch abort system has previously been jettisoned from the spacecraft
  13. 13. formalisms §  components modeled as finite state machines (FSM) –  FSMs assembled with parallel composition operator || •  synchronizes shared actions, interleaves remaining actions §  a safety property P is a FSM –  P describes all legal behaviors in terms of its alphabet –  Perr – complement of P •  determinize & complete P with an error state; •  bad behaviors lead to error –  component M satisfies P iff error state unreachable in (M || Perr) §  assume-guarantee reasoning –  assumptions and guarantees are FSMs –  〈A〉 M 〈P〉 holds iff error state unreachable in (A || M || Perr)
  14. 14. example require in and out to alternate (property Order) Input in send out Input Output in send ack Ordererr ack in Output send out out out in ack
  15. 15. parallel composition Input Output in send send out || ack ack
  16. 16. property satisfaction Ordererr in Input in send 0 1 0 1 2 || out out in ack crex. 1: (I0, O0) out (I0, Oerror) crex. 2: (I0, O0) in (I1, O1) send (I2, O1) out (I2, O0) out (I2, Oerror)
  17. 17. assume-guarantee reasoning Input in send Ordererr 0 1 2 in ack 0 1 || out Assumption out in send ack 0 1 out send crex 1: (I0, A0, O0) out X crex 2: (I0, A0, O0) in (I1, A0, O1) send (I2, A1, O1) out (I2, A0, O0) out X
  18. 18. the weakest assumption [ASE 2002] P M WA §  given component M, property P, and the interface ∑ of M with its environment, generate the weakest environment assumption WA such that: 〈WA〉 M 〈P〉 holds §  weakest means that for all environments E: 〈true〉 M || E 〈P〉 IFF 〈true〉 E 〈WA〉
  19. 19. learning assumptions (TACAS 2003)iterative solution +intermediate results L* learns unknown regular language U (over alphabet Σ) and produces minimal DFA A such that L(A) = U (L* originally proposed by Angluin)
  20. 20. weakest assumption in AG reasoning 1.  〈A〉 M1 〈P〉 2.  〈true〉 M2 〈A〉 weakest assumption makes rule complete 〈true〉 M1 || M2 〈P〉 for all E, 〈true〉 M || E 〈P〉 IFF 〈true〉 E 〈WA〉〈true〉 M1 || M2 〈P〉 IFF 〈true〉 M2 〈WA〉in other words:〈true〉 M2 〈WA〉 holds implies 〈true〉 M1 || M2 〈P〉 holds〈true〉 M2 〈WA〉 not holds implies 〈true〉 M1 || M2 〈P〉 not holds
  21. 21. L* learner the oraclequeries:should word w be included in L(A)? yes / noconjectures:here is an A – is L(A) = U? yes! no: word w should (not) be in L(A)
  22. 22. oracle for WA in assume-guarantee reasoning true / false 1.  〈A〉 M1 〈P〉 query: string s 2.  〈true〉 M2 〈A〉 〈s〉 M1 〈P〉 (simulate s on M1 || Perr) 〈true〉 M1 || M2 〈P〉 c ↑αA false+crex c L* conjecture: Ai 〈Ai〉 M1 〈P〉 (model check) true (model check) 〈true〉 M2 〈Ai〉 P holds in M1 || M2 false+crex c query c ↑αA P violated in M1 || M2 c ↑αA false true〈WA〉 M1 〈P〉 holds〈true〉 M2 〈WA〉 holds implies 〈true〉 M1 || M2 〈P〉 holds〈true〉 M2 〈WA〉 does not hold implies 〈true〉 M1 || M2 〈P〉 does not hold
  23. 23. characteristics assumptions conjectured by L* are not comparable semantically " terminates with minimal automaton A for U   " generates DFA candidates Ai: |A1| < | A2| < … < |A|   " produces at most n candidates, where n = |A|   " # queries: O(kn2 + n logm),   –  m is size of largest counterexample, k is size of alphabet "   for assume-guarantee reasoning, may terminate early with a smaller assumption than the weakest
  24. 24. example Input Ordererr Output in in send send out out out in ack ack Queries A 1: Oracle 1: Counterexample: ack send 〈A1〉 Input 〈Order〉 c = 〈in,send,ack,in〉 A 2: send Queries Oracle 1: Return to L*: ack 〈A2〉 Input 〈Order〉 c↑ Σ = 〈send,ack〉 True out, send Oracle 2: property Order holds 〈 true〉 Output 〈A2〉 True on Input || Output weakest assumption has 4 states
  25. 25. more than 2 components…§  extension of basic rule ASYM [TACAS 2003, FMSD 2009]§  symmetric / circular rules [SAVCBS 2003, FMSD 2009]
  26. 26. recursive application of ASYM 1.  〈A2〉 M2 〈A1〉 2.  〈true〉 M3 〈A2〉 1.  〈A1〉 M1 〈P〉 2.  〈true〉 M2 || M3 〈A1〉 3.  〈true〉 M2 || M3 〈A1〉 3.  〈true〉 M1 || (M2 || M3) 〈P〉
  27. 27. symmetric learning framework [SAVCBS05] refine A1 refine A2 L* L* refine A1 A1 A2 refine A2 〈A1〉 M1 〈P〉 〈A2〉 M2 〈P〉 false false true true L(coA1 || coA2) ⊆ L(P) true P holds in M1||M2 false counterexample P violated in M1||M2 analysis
  28. 28. example 1: Mars Exploration Rover §  tools: LTSA, SPIN §  model derived from JPL s Mars Exploration Rover (MER) Resource Arbiter –  local management of resource contention between resource consumers (e.g. science instruments, communication systems) –  consists of k user threads and one server thread (arbiter) Resource Arbiter §  checked mutual exclusion between resources (e.g. driving while capturing a U5 camera image are incompatible) U4 §  compositional verification scaled to >5 U3 request, cancel users vs. monolithic verification ran out U2 grant, deny ARB of memory [SPIN 06] U1 rescind
  29. 29. autonomous rendezvous & docking §  tool: LTSA §  consists of control software, state estimator, and 4 types of sensors §  input provided as UML state-charts, properties of type: –  you need at least two operational sensors to proceed to next mode §  3 bugs detected §  scaling achieved with compositional verification: –  monolithic verification runs out of memory after > 13M states –  compositional verification terminates successfully in secs. Largest state-space explored is less than 60K states, as opposed to > 13M. star planet tracker docking control orbital inertial sensor software state navigation GPS
  30. 30. example 3: K9 Rover Executive K9 Rover§  tools: LTSA, JavaPathfinder§  model of NASA Ames K9 Rover Executive –  executes flexible plans for autonomy –  consists of Executive thread and ExecCondChecker thread for monitoring state conditions –  checked for specific shared variable: if Executive reads its value, ExecCondChecker should not read the variable before the Executive clears it§  generated assumption of 6 states for model in LTSA [TACAS 2003]§  used generated assumption to check 8K lines of JAVA code translated from 10K lines of C++ code using the JavaPathfinder model checker [ICSE 2004]§  reduced memory used by JavaPathfinder > 3 times§  used generated assumption to perform assume-guarantee testing of C++ code using Eagle runtime monitoring framework [SAVCBS 2005, IET Software 2009]
  31. 31. interfacegeneration
  32. 32. component interfaces§  beyond syntactic interfaces (open file before close)§  document implicit assumptions§  safe: accept NO illegal sequence of calls§  permissive: accept ALL legal sequences of calls§  we use learning to generate interfaces [FASE 2009] − conjectured interfaces must be safe and permissive − queries and safety checked as in compositional scheme − permissiveness checked with heuristics
  33. 33. JavaPathfinder UML statecharts assume-guarantee reasoning interface generation / discharge jpf-cv http://babelfish.arc.nasa.gov/trac/jpf
  34. 34. example: crew exploration vehicle §  tool: JavaPathfinder §  UML statechart model of the Ascent and EarthOrbit flight phases of a spacecraft §  properties: –  “An event lsamRendezvous, which represents a docking maneuver with another spacecraft, fails if the LAS (launch abort system) is still attached to the spacecraft” –  “Event tliBurn (trans-lunar interface burn takes spacecraft out of the earth orbit and gets it into transition to the moon) can only be invoked if EDS (Earth Departure Stage) rocket is available”
  35. 35. results Assumption 1: Assumption 2: lasJetisson lsamRendezvous lasJetisson tliBurn lsamRendezvous lsamRendezvous generated interface assumptions encode Flight Rules in terms of events
  36. 36. infinite-state components…
  37. 37. infinite components [CAV 2010]§  use predicate abstraction (e.g., x ≥ 0, x < 0)§  generate may and must abstraction Lillegal(Cmay)   Llegal(Cmust)   must transition      L illegal(C)        L legal(C)   Lillegal(Cmust)   Llegal(Cmay)   may transition an interface safe w.r.t. Cmay and permissive w.r.t. Cmust is safe and permissive w.r.t. concrete component C
  38. 38. Query(σ, C)1.  if checkSafe(σ,Cmust) != null Lillegal(Cmay)   Llegal(Cmust)  2.  return no      L illegal(C)        L legal(C)  3.  cex = checkSafe(σ,Cmay)4.  if cex == null Lillegal(Cmust)   Llegal(Cmay)  5.  return yes6.  Preds = Preds U Refine(cex)7.  Query(σ, C)if concrete component is deterministic, so is the must abstraction…ARMC model checker: Java2SDK library classes, OpenSSL, NASA CEV model
  39. 39. summary§  automating compositional verification was a breakthrough§  our techniques are generic§  not a panacea… –  perform well when alphabets & assumptions are small§  design for compositional verification§  discovering good system decompositions§  timed & probabilistic systems, non functional properties§  multi core / parallelization?
  40. 40. http://babelfish.arc.nasa.gov/trac/jpf model checking

×