Your SlideShare is downloading. ×
0
Model Checking Programs with Java PathFinder Willem Visser < [email_address] > Peter Mehlitz < [email_address] > NASA Ames...
Motivation <ul><li>1997 – Deep Space 1 Remote Agent </li></ul><ul><li>1999 – Honeywell DEOS IMA O/S </li></ul><ul><li>Mode...
Program Model Checking <ul><li>10 years ago </li></ul><ul><ul><li>Almost no one did it </li></ul></ul><ul><li>5 years ago ...
Overview <ul><li>What is JPF?  </li></ul><ul><ul><li>A model checker for Java Bytecode </li></ul></ul><ul><li>Getting Star...
What is Java PathFinder (1) <ul><li>explicit state model checker for Java bytecode </li></ul><ul><li>focus is on  finding ...
What is JPF (2) <ul><li>goal is to avoid modeling effort (check the real program), or at least use a real programming lang...
Key Points <ul><li>Models can be infinite state </li></ul><ul><ul><li>Unbounded objects, threads,… </li></ul></ul><ul><ul>...
JPF Status <ul><li>developed at the Robust Software Engineering Group at NASA Ames Research Center </li></ul><ul><li>curre...
Getting and Installing JPF <ul><li>Getting JPF </li></ul><ul><ul><li>svn co https://svn.sourceforge.net/svnroot/javapathfi...
How To Run JPF <ul><li>generally speaking: like a VM (“java” replacement):   > bin/jpf <jpf-options> <test-app main class>...
JPF Configuration
Some Examples <ul><li>Remote Agent </li></ul><ul><li>K9 Rover (‘real’ model, size) </li></ul>
Remote Agent <ul><li>“ oldclassic.java ”  </li></ul><ul><ul><li>Simplified version of the deadlock encountered in the Remo...
K9 Rover <ul><li>Executes flexible plans for autonomy   </li></ul><ul><ul><li>branching on state / temporal conditions </l...
Directory Structure
Under the Hood - Toplevel Structure <ul><li>two major concepts:  Search  and  VM </li></ul><ul><li>Search is the VM driver...
Under the Hood - Search
Extending JPF - Listeners <ul><li>preferred way of extending JPF: ‘Listener’ variant of the Observer pattern - keep extens...
Extending JPF - SearchListener public interface SearchListener {   /* got the next state */     void  stateAdvanced  (Sear...
Extending JPF - VMListener public interface VMListener {    void  instructionExecuted  (JVM vm);  // VM has executed next ...
Extending JPF - Listener Example public class HeapTracker extends  GenericProperty  implements  VMListener ,  SearchListen...
Extending JPF - Listener Configuration <ul><li>listeners are usually configured, not hard coded </li></ul><ul><li>per conf...
Going Native - Model Java Interface <ul><li>JPF is a state-tracking JVM, running on top of a general JVM </li></ul><ul><li...
MJI - Why? <ul><li>one obvious reason: running native Java methods in JPF (otherwise we couldn’t run apps using standard l...
MJI - Components <ul><li>Model  class: has native method declaration, executed by JPF </li></ul><ul><li>NativePeer  class:...
MJI - How
MJI - Example <ul><li>application calls method to intercept   ..   System.out.println(“a message”); </li></ul><ul><li>mode...
Scalability - Partial Order Reduction <ul><li>concurrency is major contributor to state space explosion </li></ul><ul><li>...
POR - Scheduling Relevance
POR - Shared Objects <ul><li>to detect races, we have to identify read/write access to objects that are visible from diffe...
Choice Generator Motivation
JPF Perspective State  consists of 2 main components, the state of the JVM and the  current and next choice Generator (i.e...
Role of Choices In other words, possible existence of Choices is  what terminates the last Transition, and selection  of a...
Extensions <ul><li>JPF was built to be extended </li></ul><ul><li>Architecture is such that the core classes can be left a...
Regression Tests with JUnit
JUnit Example package gov.nasa.jpf.mc; import org.junit.Test; import org.junit.runner.JUnitCore; import gov.nasa.jpf.jvm.T...
TestJPF <ul><li>Extends junit.Assert </li></ul><ul><li>Interface methods </li></ul><ul><ul><li>runJPFDeadlock(args) </li><...
More Examples <ul><li>Stoned Hippies </li></ul><ul><ul><li>Regular version </li></ul></ul><ul><ul><li>Using MJI </li></ul>...
Stoned Hippies Germany Netherlands 5 10 2 1
Stoned Hippies Germany Netherlands 2 5 10 1 2
Stoned Hippies Germany Netherlands 3 5 10 1 2
Stoned Hippies Germany Netherlands 8 10 1 2 5
Stoned Hippies Germany Netherlands 19 1 2 5 10
Symbolic Execution <ul><li>Explicit-state model checking  cannot handle large data domains </li></ul><ul><li>Want to gener...
Concrete Execution Path (example) x = 1, y = 0 1 >? 0 x = 1 + 0 = 1 y = 1 – 0 = 1 x = 1 – 1 = 0 0 – 1 >? 0 int x, y; if (x...
Symbolic Execution Tree (example) x = X, y = Y int x, y; if (x > y) { x = x + y; y = x – y; x = x – y; if (x – y > 0) asse...
Example class Node { int elem; Node next; Node swapNode() {   if (next != null)   if (elem > next.elem) {   Node t = next;...
Challenges in Generalizing Symbolic Execution <ul><li>how to handle fields in dynamic structures? </li></ul><ul><li>how to...
Generalized Symbolic Execution <ul><li>model checker  generates and explores “symbolic” execution tree </li></ul><ul><ul><...
Algorithm  (lazy initialization) <ul><li>to symbolically execute a method </li></ul><ul><ul><li>create input objects with ...
Algorithm (aliasing) <ul><li>when method execution accesses field  f if ( f  is uninitialized) {   if ( f  is reference fi...
Algorithm (illustration) consider executing next = t.next; E0 next E1 next t null t E0 next E1 next ? next E0 next E1 t ne...
Implementation via Instrumentation program instrumentation counterexample(s)/test suite [heap+constraint+thread scheduling...
Testing with Symbolic Execution <ul><li>Focus on programs that manipulate complex data </li></ul><ul><ul><li>Java containe...
Red-Black Trees (1) The root is BLACK (2) Red nodes can only  have black children (3) All paths from a node to  its leaves...
repOk() Fragment boolean repOk(Entry e) { // root has no parent, root is black,… // RedHasOnlyBlackChildren workList = new...
Black-box TIG Symbolic Execution  <ul><li>Symbolic execution of  repOk() </li></ul><ul><ul><li>Generate new structures onl...
Symbolic Execution of repOk() Example public   static   boolean  repOk() { if  (root ==  null ) return   true ; if  (root....
White-box TIG Symbolic Execution <ul><li>Consider code coverage criterion when generating test inputs </li></ul><ul><li>Us...
repOk() x 2 abstract and concrete Symbolic Execution of Code During Lazy Initialization check  Abstract repOk() When cover...
Abstract  repOk() <ul><li>Eliminate symbolic structures that cannot be converted to a concrete structure that satisfy  rep...
White-box TIG: cover branches in  deleteEntry(Entry p) /* precondition: p. repOk()  */ private   void   deleteEntry(Entry ...
Symbolic Execution for white-box TIG if   (p.left   !=   null   &&   p.right   !=   null )   { ... Symbolic structure befo...
API Based Testing SUT ENV (m,n) m is the seq. length of API calls  & n is the number of values  used in the parameters of ...
Framework SUT with minor instrumentation ENV TestListener Abstraction Mapping + State Storage Coverage Manager JPF
Environment Skeleton M : sequence length N : parameter values A : abstraction used for (int i = 0; i < M; i++) { int x = V...
Symbolic Environment Skeleton M : sequence length A : abstraction used for (int i = 0; i < M; i++) { SymbolicInteger x = n...
Sample Output Test case number 77 for '15,L+R+P-REDroot ':  put(0);put(4);put(5);put(1);put(2);put(3);remove(4); Unique ID...
Subsumption Checking x1 x2 x3 x4 x5 + x1 > x2 &  x2 > x3 &  x2 < x4 &  x5 > x1 x1 x2 x3 x4 x5 + x1 > x2 &  x2 > x3 &  x2 <...
Existential Elimination x1 x2 x3 x4 x5 PC s1 < s2 & s4 > s3 &  s4 < s1 & s4 < s5 & s7  < s2 &  s7  > s1   s1 s4 s2 s3 s5 +...
Results from ISSTA 2006 Paper <ul><li>We compared the following techniques (1 st  3 are exhaustive; last 2 lossy) </li></u...
Symbolic Execution Demo <ul><li>Start by Running Saswat Anand’s symbolic instrumenter on the SwapValues.java File  </li></...
JPF Symbolic Execution - BEFORE Omega Interface Formula satisfiable/unsatisfiable Omega Java Version JPF
JPF Symbolic Execution - NOW Generic Decision Procedure Interface Formula satisfiable/unsatisfiable Omega Maryland JPF CVC...
Communication Methods <ul><li>Issue  </li></ul><ul><ul><li>JPF and the Interface code is in Java </li></ul></ul><ul><ul><l...
Optimization using Tables JPF State Path Condition: X > Y & Z > X & … JPF State Path Condition: pc100 <ul><li>Outside JPF ...
Optimization – Run DPs incrementally <ul><li>Some decision procedures support running in a incremental mode where you do n...
Decision Procedure Options <ul><li>+symbolic.dp= </li></ul><ul><ul><li>omega.file </li></ul></ul><ul><ul><li>omega.pipe </...
Results TCAS
Results TreeMap STP took > 1 hour
State Matching in JPF VM State Matchable + Restorable Stored State (hashed) Matchable compression Collaborator Peter Dilli...
Old Architecture StateSet int[] bool VM/Search <ul><li>Integrated collapse   compression of raw    VM state  </li></ul><ul...
New Architecture VM/Search DefaultBacktracker VM bool int[] Restorer Serializer Serialized StateSet int[] set VM
Old Scheme in the New Architecture VM/Search DefaultBacktracker FullStateSet int[] set bool int[] Collapsing (de)Serialize...
New Architecture VM/Search CollapsingRestorer FilteringSerializer DefaultBacktracker VM bool int[] objects This is the def...
FilteringSerializer  <ul><li>By default captures the fundamental VM state </li></ul><ul><ul><li>i.e. with all known unnece...
New Architecture Revisited - Abstraction VM/Search CollapsingRestorer AbstractingSerializer DefaultBacktracker VM bool int...
AbstractingSerializer <ul><li>First does FilteringSerializer </li></ul><ul><li>Also gives the ability to specify abstracti...
Examples <ul><li>K9 revisited </li></ul><ul><ul><li>Filtering visits many less states </li></ul></ul><ul><ul><li>Abstracti...
User-Interface Model Checking <ul><li>Extension to JPF to deal with UIs directly </li></ul><ul><ul><li>See extensions/ui d...
Something Completely Different <ul><li>Well, maybe not  completely … </li></ul><ul><li>We developed a new static analysis ...
Upcoming SlideShare
Loading in...5
×

Jpf model checking

537

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
537
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • To handle this case, we have to leave the ideal world of model checking (that considers all possible choices), and make use of what we know about the real world - we have to use heuristics to make the set of choices finite and manageable. However, heuristics are application and domain specific, and it would be a bad idea to hardcode them into the test drivers we give JPF to analyze. This leads to a number of requirements for the JPF choice mechanism: choice mechanisms have to be decoupled (i.e. thread choices should be indpendent of data choices, double choices from int choices etc.) choice sets and enumeration should be encapsulated in dedicated, type specific objects. The VM should only know about the most basic types, and otherwise use a generic interface to obtain choices selection of classes representing (domain specific) heuristics, and parametrization of ChoiceGenerator instances should be possible at runtime, i.e. via JPF&apos;s configuration mechanism (properties) The diagram shown above depicts this with an example that uses a &amp;quot;randomly&amp;quot; chosen velocity value of type double. As an example heuristic we use a threshold model, i.e. we want to know how the system reacts below, at, and above a certain application specific value (threshold). We reduce an infinite set of choices to only three &amp;quot;interesting&amp;quot; ones. Of course, &amp;quot;interesting&amp;quot; is quite subjective, and we probably want to play with the values (delta, threshold, or even used heuristic) efficiently, without having to rebuild the application each time we run JPF. The code example does not mention the used ChoiceGenerator class (DoubleThresholdGenerator) at all, it just specifies a symbolic name &amp;quot;velocity&amp;quot;, which JPF uses to look up an associated class name from its configuration data (initialized via property files or the command line - see Configuring JPF Runtime Options ). But it doesn&apos;t stop there. Most heuristics need further parameterization (e.g. threshold, delta), and we provide that by passing the JPF configuration data into the ChoiceGenerator constructors (e.g. the &apos;velocity.threshold&apos; property). Each ChoiceGenerator instance knows its symbolic name (e.g. &amp;quot;velocity&amp;quot;), and can use this name to look up whatever parameters it needs.
  • Transition is the sequence of instructions that leads from one state to the next. There is no context switch within a transition, it&apos;s all in the same thread. There can be multiple transitions leading out of one state (but not
  • Transcript of "Jpf model checking"

    1. 1. Model Checking Programs with Java PathFinder Willem Visser < [email_address] > Peter Mehlitz < [email_address] > NASA Ames Research Center
    2. 2. Motivation <ul><li>1997 – Deep Space 1 Remote Agent </li></ul><ul><li>1999 – Honeywell DEOS IMA O/S </li></ul><ul><li>Model extraction by hand doesn’t scale </li></ul><ul><li>Automated Program Model Checking </li></ul><ul><li>Translation-based model checking is only has good as the target model checker </li></ul><ul><ul><li>Java to PROMELA translation for SPIN </li></ul></ul>
    3. 3. Program Model Checking <ul><li>10 years ago </li></ul><ul><ul><li>Almost no one did it </li></ul></ul><ul><li>5 years ago </li></ul><ul><ul><li>Just a handful of tools existed </li></ul></ul><ul><li>Now </li></ul><ul><ul><li>BANDERA, BLAST, CBMC, dSPIN, JPF, MAGIC, SLAM, SPIN 4, Verisoft, CMC, ZING, etc. </li></ul></ul>
    4. 4. Overview <ul><li>What is JPF? </li></ul><ul><ul><li>A model checker for Java Bytecode </li></ul></ul><ul><li>Getting Started with JPF </li></ul><ul><ul><li>Downloading, Installation and Running </li></ul></ul><ul><li>Examples Part 1 </li></ul><ul><ul><li>Deadlock in the Remote Agent </li></ul></ul><ul><ul><li>K9 Rover </li></ul></ul><ul><li>Internal workings of JPF </li></ul><ul><ul><li>Model Java Interface </li></ul></ul><ul><ul><li>Listeners </li></ul></ul><ul><ul><li>Dynamic Partial-order reductions </li></ul></ul><ul><ul><li>Choice Generators </li></ul></ul><ul><ul><li>Extensions </li></ul></ul><ul><ul><li>Regression tests (JUnit Integration) </li></ul></ul><ul><li>Examples Part 2 </li></ul><ul><ul><li>Stoned Hippies in three variations </li></ul></ul><ul><ul><li>Race Detection </li></ul></ul><ul><ul><li>Numeric Exceptions </li></ul></ul><ul><ul><li>Partial Trace Analysis </li></ul></ul><ul><li>Symbolic Execution </li></ul><ul><li>How we spent the summer! </li></ul><ul><ul><li>Abstractions </li></ul></ul><ul><ul><li>Decision Procedure Extensions </li></ul></ul><ul><ul><li>User Interface Analysis </li></ul></ul><ul><ul><li>Java Static Analysis </li></ul></ul>9h00 – 10h30 11h00 – 13h00
    5. 5. What is Java PathFinder (1) <ul><li>explicit state model checker for Java bytecode </li></ul><ul><li>focus is on finding bugs in Java programs </li></ul><ul><ul><li>concurrency related: deadlocks, (races), missed signals etc. </li></ul></ul><ul><ul><li>Java runtime related: unhandled exceptions, heap usage, (cycle budgets) </li></ul></ul><ul><ul><li>but also: complex application specific assertions </li></ul></ul>
    6. 6. What is JPF (2) <ul><li>goal is to avoid modeling effort (check the real program), or at least use a real programming language for complex models </li></ul><ul><li>implies that the main challenge is scalability </li></ul><ul><li>JPF uses a variety of scalability enhancing mechanisms </li></ul><ul><ul><li>user extensible state abstraction & matching </li></ul></ul><ul><ul><li>on-the-fly partial order reduction </li></ul></ul><ul><ul><li>configurable search strategies: &quot;find the bug before you run out of memory&quot; </li></ul></ul><ul><ul><li>user definable heuristics (searches, choice generators) </li></ul></ul><ul><li>key issue is configurable extensibility : overcome scalability constraints with suitable customization (using heuristics) </li></ul>
    7. 7. Key Points <ul><li>Models can be infinite state </li></ul><ul><ul><li>Unbounded objects, threads,… </li></ul></ul><ul><ul><li>Depth-first state generation (explicit-state) </li></ul></ul><ul><ul><li>Verification requires abstraction </li></ul></ul><ul><li>Handle full Java language </li></ul><ul><ul><li>but only for closed systems </li></ul></ul><ul><ul><li>cannot directly handle native code </li></ul></ul><ul><ul><ul><li>no Input/output through GUIs, files, Networks, … </li></ul></ul></ul><ul><ul><ul><li>Must be modeled by java code instead </li></ul></ul></ul><ul><li>Allows Nondeterministic Environments </li></ul><ul><ul><li>JPF traps special nondeterministic methods </li></ul></ul><ul><li>Checks for User-defined assertions, deadlock and user-specified properties </li></ul>
    8. 8. JPF Status <ul><li>developed at the Robust Software Engineering Group at NASA Ames Research Center </li></ul><ul><li>currently in it’s fourth development cycle </li></ul><ul><ul><li>v1: Spin/Promela translator - 1999 </li></ul></ul><ul><ul><li>v2: backtrackable, state matching JVM - 2000 </li></ul></ul><ul><ul><li>v3: extension infrastructure (listeners, MJI) - 2004 </li></ul></ul><ul><ul><li>v4: symbolic execution, choice generators - 4Q 2005 </li></ul></ul><ul><li>open sourced since 04/2005 under NOSA 1.3 license: <javapathfinder.sourceforge.net> </li></ul><ul><li>it’s a first: no NASA system development hosted on public site before </li></ul><ul><li>11100 downloads since publication 04/2005 </li></ul>
    9. 9. Getting and Installing JPF <ul><li>Getting JPF </li></ul><ul><ul><li>svn co https://svn.sourceforge.net/svnroot/javapathfinder javapathfinder </li></ul></ul><ul><li>Compiling JPF </li></ul><ul><ul><li>Go to the JPF root directory </li></ul></ul><ul><ul><li>Type build-tools/bin/ant </li></ul></ul><ul><ul><ul><li>Compiles JPF </li></ul></ul></ul><ul><ul><ul><li>Adding “run-tests” will also run the regression tests </li></ul></ul></ul><ul><li>The Eclipse Route </li></ul><ul><ul><li>Get Eclipse 3.2 </li></ul></ul><ul><ul><ul><li>www.eclipse.org </li></ul></ul></ul><ul><ul><li>Install Subclipse </li></ul></ul><ul><ul><ul><li>Goto: Help – Software Updates – Find and Install – Select New Features </li></ul></ul></ul><ul><ul><ul><li>Type the following in the url field: http://subclipse.tigris.org/update_1.2.x </li></ul></ul></ul><ul><ul><ul><li>Follow the instructions </li></ul></ul></ul><ul><ul><li>Create a new SVN project by selecting “Checkout Projects from SVN” </li></ul></ul><ul><ul><li>Create a new repository for https://svn.sourceforge.net/svnroot/javapathfinder </li></ul></ul><ul><ul><li>Select trunk </li></ul></ul><ul><ul><li>It will automatically download and compile in Eclipse </li></ul></ul><ul><ul><li>Hint </li></ul></ul><ul><ul><ul><li>If you ever run JPF from outside the JPF project then put a copy of default.properties in the build/jpf/gov/nasa/jpf directory – otherwise you’ll get a vm.class error at initialization </li></ul></ul></ul>
    10. 10. How To Run JPF <ul><li>generally speaking: like a VM (“java” replacement): > bin/jpf <jpf-options> <test-app main class> </li></ul><ul><li>BUT: lots of configuration (classes) and parameterization (booleans, integers etc.) </li></ul><ul><li>JPF is an open system </li></ul><ul><li>need for flexible, extensible configuration system </li></ul><ul><li>quite powerful, but can be somewhat confusing </li></ul>
    11. 11. JPF Configuration
    12. 12. Some Examples <ul><li>Remote Agent </li></ul><ul><li>K9 Rover (‘real’ model, size) </li></ul>
    13. 13. Remote Agent <ul><li>“ oldclassic.java ” </li></ul><ul><ul><li>Simplified version of the deadlock encountered in the Remote Agent </li></ul></ul><ul><li>Fixing oldclassic! </li></ul><ul><ul><li>Or rather trying to fix … </li></ul></ul>T1 T2 signal signal notify notify if (no_action) wait(); signal();
    14. 14. K9 Rover <ul><li>Executes flexible plans for autonomy </li></ul><ul><ul><li>branching on state / temporal conditions </li></ul></ul><ul><li>Multi-threaded system </li></ul><ul><ul><li>communication through shared variables </li></ul></ul><ul><ul><li>synchronization through mutexes and condition variables </li></ul></ul><ul><li>Main functionality: 8KLOC, C++/JAVA </li></ul>commands plans Exec Executive CondChecker Database ActionExec TempChecker
    15. 15. Directory Structure
    16. 16. Under the Hood - Toplevel Structure <ul><li>two major concepts: Search and VM </li></ul><ul><li>Search is the VM driver and Property evaluator </li></ul><ul><li>VM is the state generator </li></ul>
    17. 17. Under the Hood - Search
    18. 18. Extending JPF - Listeners <ul><li>preferred way of extending JPF: ‘Listener’ variant of the Observer pattern - keep extensions out of the core classes </li></ul><ul><li>listeners can subscribe to Search and VM events </li></ul>
    19. 19. Extending JPF - SearchListener public interface SearchListener {   /* got the next state */     void stateAdvanced (Search search);   /* state was backtracked one step */   void stateBacktracked (Search search);     /* a previously generated state was restored     (can be on a completely different path) */   void stateRestored (Search search);     /* JPF encountered a property violation */   void propertyViolated (Search search);     /* we get this after we enter the search loop, but BEFORE the first forward */   void searchStarted (Search search);     /* there was some contraint hit in the search, we back out could have been turned into a property, but usually is an attribute of the search, not the application */ void searchConstraintHit (Search search);     /* we're done, either with or without a preceeding error */   void searchFinished (Search search); }
    20. 20. Extending JPF - VMListener public interface VMListener {    void instructionExecuted (JVM vm); // VM has executed next instruction   void threadStarted (JVM vm); / / new Thread entered run() method      void threadTerminated (JVM vm); // Thread exited run() method      void classLoaded (JVM vm); // new class was loaded      void objectCreated (JVM vm); // new object was created      void objectReleased (JVM vm); // object was garbage collected      void gcBegin (JVM vm); // garbage collection mark phase started      void gcEnd (JVM vm); // garbage collection sweep phase terminated      void exceptionThrown (JVM vm); // exception was thrown    void nextChoice (JVM vm); // choice generator returned new value }
    21. 21. Extending JPF - Listener Example public class HeapTracker extends GenericProperty implements VMListener , SearchListener { class PathStat { .. int heapSize = 0; .. } // helper to store additional state info PathStat stat = new PathStat(); Stack pathStats = new Stack(); public boolean check (JVM vm, Object arg) { // GenericProperty return (stat.heapSize <= maxHeapSizeLimit); } public void stateAdvanced (Search search) { // SearchListener if (search.isNewState()) {.. pathStats.push(stat); stat = (PathStat)stat.clone(); .. } public void stateBacktracked (Search search) { // SearchListener .. if (!pathStats.isEmpty()) stat = (PathStat) pathStats.pop(); } public void objectCreated (JVM vm) {.. // VMListener ElementInfo ei = vm.getLastElementInfo(); ..stat.heapSize += ei.getHeapSize(); .. } public void objectReleased (JVM vm) { // VMListener ElementInfo ei = vm.getLastElementInfo(); ..stat.heapSize -= ei.getHeapSize(); .. } ... }
    22. 22. Extending JPF - Listener Configuration <ul><li>listeners are usually configured, not hard coded </li></ul><ul><li>per configuration file: search.listener = MySearchListener vm.listener = MyVMListener jpf.listener = MyCombinedListener:MySecondListener... </li></ul><ul><li>per command line: jpf ... +jpf.listener=MyCombinedListener ... </li></ul><ul><li>hard coded: MyListener listener= new MyListener(..); .. Config config = JPF.createConfig( args); JPF jpf = new JPF( config); jpf. addSearchListener (listener); jpf. addVMListener ( listener); jpf.run(); .. </li></ul>
    23. 23. Going Native - Model Java Interface <ul><li>JPF is a state-tracking JVM, running on top of a general JVM </li></ul><ul><li>Java Native Interface (JNI) analogy: “execute one level lower” </li></ul><ul><li>Model Java Interface (MJI): execute in the host VM that runs JPF itself </li></ul>
    24. 24. MJI - Why? <ul><li>one obvious reason: running native Java methods in JPF (otherwise we couldn’t run apps using standard libraries, which have lots of native methods) </li></ul><ul><li>specific use of native methods: interface library methods to JPF runtime system (e.g. java.lang.Thread -> ThreadInfo) </li></ul><ul><li>enables usage of specialized verification API in app, interfacing to JPF functionality: int input = gov.nasa.jpf.jvm.Verify.randomInt(10); </li></ul><ul><li>but also useful for scalability reasons </li></ul><ul><ul><li>native methods can save state space </li></ul></ul><ul><ul><li>native methods are executed atomically </li></ul></ul><ul><ul><li>native methods execute much faster </li></ul></ul><ul><li>example: java.io.RandomAccessFile </li></ul>
    25. 25. MJI - Components <ul><li>Model class: has native method declaration, executed by JPF </li></ul><ul><li>NativePeer class: native method implementation, executed by JVM </li></ul><ul><li>MJIEnv : native method calling context (to get back to JPF) </li></ul>
    26. 26. MJI - How
    27. 27. MJI - Example <ul><li>application calls method to intercept .. System.out.println(“a message”); </li></ul><ul><li>model class declares the method we want to intercept (doesn’t have to be native), is executed by JPF public class PrintStream .. { .. public void println (String s) {..} // usually native method } </li></ul><ul><li>native peer has the method implementation that gets executed by host VM (not simulated by JPF) class JPF_java_io_PrintStream { .. public static void println__Ljava_lang_String_2 (MJIEnv env,int objRef, int strRef) { env.getVM().println(env.getStringObject(strRef)); } } </li></ul>0: getstatic #2 3: ldc #3 5: invokevirtual #4
    28. 28. Scalability - Partial Order Reduction <ul><li>concurrency is major contributor to state space explosion </li></ul><ul><li>reduction of thread interleavings is paramount for scalability </li></ul><ul><li>JPF employs on-the-fly Partial Order Reduction mechanism </li></ul><ul><li>leveled approach that makes use of JVM instruction set and infrastructure (memory management) </li></ul><ul><li>completely at runtime (on-the-fly) </li></ul>
    29. 29. POR - Scheduling Relevance
    30. 30. POR - Shared Objects <ul><li>to detect races, we have to identify read/write access to objects that are visible from different threads </li></ul><ul><li>expensive operation, BUT: can piggyback on garbage collection </li></ul><ul><li>two phase approach: </li></ul><ul><ul><li>mark root set with thread id (statics are shared by default) </li></ul></ul><ul><ul><li>traverse marked objects - if another thread id is reached, mark as shared </li></ul></ul><ul><li>problem: GC based on reachability, not accessibility -> need to break on certain fields (Thread.group->ThreadGroup.threads) </li></ul>
    31. 31. Choice Generator Motivation
    32. 32. JPF Perspective State consists of 2 main components, the state of the JVM and the current and next choice Generator (i.e. the objects encapsulating the choice enumeration that produces new transitions) Transition is the sequence of instructions that leads from one state. There is no context within a transition, it's all in the same thread. There can be multiple transitions leading out of one state Choice is what starts a new transition. This can be a different thread, i.e. scheduling choice, or different “random” data value.
    33. 33. Role of Choices In other words, possible existence of Choices is what terminates the last Transition, and selection of a Choice value precedes the next Transition. The first condition corresponds to creating a new ChoiceGenerator, and letting the SystemState know about it. The second condition means to query the next choice value from this ChoiceGenerator (either internally within the JVM, or in an instruction or native method).
    34. 34. Extensions <ul><li>JPF was built to be extended </li></ul><ul><li>Architecture is such that the core classes can be left alone when doing an extension </li></ul><ul><ul><li>Add code to the “extensions” directory that follows the same layout as the core classes </li></ul></ul>Numeric Extension to check for errors such as Overflow Most of this extension was provided by Aleksandar Milicevic and Sasa Misailovic from UIUC
    35. 35. Regression Tests with JUnit
    36. 36. JUnit Example package gov.nasa.jpf.mc; import org.junit.Test; import org.junit.runner.JUnitCore; import gov.nasa.jpf.jvm.TestJPF; public class TestOldClassicJPF extends TestJPF { static final String TEST_CLASS = &quot;gov.nasa.jpf.mc.oldclassic&quot;; public static void main (String[] args) { JUnitCore.main(&quot;gov.nasa.jpf.mc.TestOldClassicJPF&quot;); } @Test public void testDFSearch () { String[] args = { TEST_CLASS }; runJPFDeadlock(args); } @Test public void testBFSHeuristic () { String[] args = { &quot;+search.class=gov.nasa.jpf.search.heuristic.HeuristicSearch&quot;, &quot;+search.heuristic.class=gov.nasa.jpf.search.heuristic.BFSHeuristic&quot;, TEST_CLASS }; runJPFDeadlock(args); } }
    37. 37. TestJPF <ul><li>Extends junit.Assert </li></ul><ul><li>Interface methods </li></ul><ul><ul><li>runJPFDeadlock(args) </li></ul></ul><ul><ul><ul><li>Expects a deadlock </li></ul></ul></ul><ul><ul><li>runJPFassertionError(args) </li></ul></ul><ul><ul><ul><li>Expects an assertion violation </li></ul></ul></ul><ul><ul><li>runJPFnoAssertionError(args) </li></ul></ul><ul><ul><ul><li>Don’t want to see an assertion violation </li></ul></ul></ul><ul><ul><li>runJPFException(args) </li></ul></ul><ul><ul><ul><li>Expects an exception </li></ul></ul></ul><ul><ul><li>runJPFnoException(args) </li></ul></ul><ul><ul><ul><li>Don‘t want to see an exception </li></ul></ul></ul>
    38. 38. More Examples <ul><li>Stoned Hippies </li></ul><ul><ul><li>Regular version </li></ul></ul><ul><ul><li>Using MJI </li></ul></ul><ul><ul><li>Using a Listener </li></ul></ul><ul><li>Race Detection </li></ul><ul><ul><li>A more sophisticated use of a Listener </li></ul></ul><ul><li>Numeric Extensions </li></ul><ul><ul><li>With Junit integration </li></ul></ul><ul><li>Partial Trace Analysis </li></ul><ul><ul><li>Recording a path and then re-analyze from the end of the path </li></ul></ul>
    39. 39. Stoned Hippies Germany Netherlands 5 10 2 1
    40. 40. Stoned Hippies Germany Netherlands 2 5 10 1 2
    41. 41. Stoned Hippies Germany Netherlands 3 5 10 1 2
    42. 42. Stoned Hippies Germany Netherlands 8 10 1 2 5
    43. 43. Stoned Hippies Germany Netherlands 19 1 2 5 10
    44. 44. Symbolic Execution <ul><li>Explicit-state model checking cannot handle large data domains </li></ul><ul><li>Want to generate test cases for systems that manipulate complex data structures </li></ul>Collaborators Corina Pasareanu Sarfraz Khurshid Saswat Anand
    45. 45. Concrete Execution Path (example) x = 1, y = 0 1 >? 0 x = 1 + 0 = 1 y = 1 – 0 = 1 x = 1 – 1 = 0 0 – 1 >? 0 int x, y; if (x > y) { x = x + y; y = x – y; x = x – y; if (x – y > 0) assert(false); }
    46. 46. Symbolic Execution Tree (example) x = X, y = Y int x, y; if (x > y) { x = x + y; y = x – y; x = x – y; if (x – y > 0) assert(false); } X >? Y [ X > Y ] y = X + Y – Y = X [ X > Y ] x = X + Y – X = Y [ X > Y ] Y - X >? 0 [ X <= Y ] END [ X > Y ] x = X + Y [ X > Y, Y – X <= 0 ] END [ X > Y, Y – X > 0 ] END
    47. 47. Example class Node { int elem; Node next; Node swapNode() { if (next != null) if (elem > next.elem) { Node t = next; next = t.next; t.next = this; return t; } return this; } } ? null E0 E1 E0 E0 E1 null E0 E1 ? E0 E1 E0 E1 Input list + Constraint Output list E0 > E1 none E0 <= E1 none E0 > E1 E0 > E1 E0 > E1 E1 E0 ? E1 E0 E1 E0 E1 E0 null E0 E1 E0 ? null NullPointerException
    48. 48. Challenges in Generalizing Symbolic Execution <ul><li>how to handle fields in dynamic structures? </li></ul><ul><li>how to handle aliasing? </li></ul><ul><li>how to generate tests? </li></ul><ul><ul><li>satisfy criteria </li></ul></ul><ul><ul><li>satisfy precondition </li></ul></ul><ul><ul><li>are inequivalent </li></ul></ul>
    49. 49. Generalized Symbolic Execution <ul><li>model checker generates and explores “symbolic” execution tree </li></ul><ul><ul><li>non-determinism handles aliasing </li></ul></ul><ul><ul><ul><li>explore different heap configurations explicitly </li></ul></ul></ul><ul><ul><li>concurrency </li></ul></ul><ul><ul><li>off-the-shelf decision procedures check path conditions </li></ul></ul><ul><li>lazy initialization </li></ul><ul><ul><li>initializes program’s inputs on an “as-needed” basis </li></ul></ul><ul><ul><li>no a priori bound on input sizes </li></ul></ul><ul><li>preconditions to initialize inputs only with valid values </li></ul>
    50. 50. Algorithm (lazy initialization) <ul><li>to symbolically execute a method </li></ul><ul><ul><li>create input objects with uninitialized fields </li></ul></ul><ul><ul><li>execute! </li></ul></ul><ul><ul><ul><li>follow mainly Java semantics </li></ul></ul></ul><ul><ul><ul><li>initialize fields “as-required” </li></ul></ul></ul><ul><ul><ul><li>add constraints to path condition </li></ul></ul></ul>
    51. 51. Algorithm (aliasing) <ul><li>when method execution accesses field f if ( f is uninitialized) { if ( f is reference field of type T ) { non-deterministically initialize f to  null  a new object of class T (with uninitialized fields)  an object created during prior field initialization (alias) } if ( f is numeric/string field) initialize f to a new symbolic value } </li></ul>
    52. 52. Algorithm (illustration) consider executing next = t.next; E0 next E1 next t null t E0 next E1 next ? next E0 next E1 t next E0 next E1 next t E0 next E1 next t Precondition: acyclic list E0 E1 next t null next t E0 E1 next ? next next
    53. 53. Implementation via Instrumentation program instrumentation counterexample(s)/test suite [heap+constraint+thread scheduling] model checking decision procedure instrumented program correctness specification continue/ backtrack state: original program path condition (data) heap configuration thread scheduling
    54. 54. Testing with Symbolic Execution <ul><li>Focus on programs that manipulate complex data </li></ul><ul><ul><li>Java container classes, e.g. java.util.TreeMap </li></ul></ul><ul><li>Black box test input generation </li></ul><ul><ul><li>Using structural invariants </li></ul></ul><ul><ul><li>Using API calls </li></ul></ul><ul><ul><ul><li>With symbolic data, i.e. kind-of gray-box </li></ul></ul></ul><ul><li>White box </li></ul><ul><ul><li>Completely symbolic execution over structures </li></ul></ul>
    55. 55. Red-Black Trees (1) The root is BLACK (2) Red nodes can only have black children (3) All paths from a node to its leaves contain the same number of black nodes. Self-balancing Binary Search Trees Java TreeMap Implementation (4) Acyclic (5) Consistent Parents repOk(): conditions (1)-(5)
    56. 56. repOk() Fragment boolean repOk(Entry e) { // root has no parent, root is black,… // RedHasOnlyBlackChildren workList = new LinkedList(); workList.add(e); while (!workList.isEmpty()) { Entry current=(Entry)workList.removeFirst(); Entry cl = current.left; Entry cr = current.right; if (current.color == RED) { if(cl != null && cl.color == RED) return false; if(cr != null && cr.color == RED) return false; } if (cl != null) workList.add(cl); if (cr != null) workList.add(cr); } // equal number of black nodes on left and right sub-tree… return true; }
    57. 57. Black-box TIG Symbolic Execution <ul><li>Symbolic execution of repOk() </li></ul><ul><ul><li>Generate new structures only when repOk() returns true </li></ul></ul><ul><ul><li>Limit the size of the structures generated </li></ul></ul><ul><li>Only correct structures will be generated </li></ul><ul><ul><li>repOk() returns true after all nodes in the tree have been visited, hence they must all be concrete </li></ul></ul><ul><ul><li>symbolic (partial) structures can fail repOk() </li></ul></ul><ul><li>Similar to Korat </li></ul><ul><ul><li>Not based on symbolic execution </li></ul></ul><ul><ul><li>Cannot deal with constraints on primitive data </li></ul></ul><ul><ul><li>Korat uses custom algorithms and is much faster </li></ul></ul>
    58. 58. Symbolic Execution of repOk() Example public static boolean repOk() { if (root == null ) return true ; if (root.color == RED) return false; … Size 1
    59. 59. White-box TIG Symbolic Execution <ul><li>Consider code coverage criterion when generating test inputs </li></ul><ul><li>Use repOk() as a precondition during symbolic execution of source code </li></ul>
    60. 60. repOk() x 2 abstract and concrete Symbolic Execution of Code During Lazy Initialization check Abstract repOk() When coverage is achieved, solve the symbolic constraints to create concrete inputs Concretize inputs by symbolic execution of Concrete repOk() over symbolic structures - as with Black-box TIG - Abstract repOk() : Symbolic Structure {true,false,don’t know} Concrete repOk() : Symbolic Structure Concrete Structure
    61. 61. Abstract repOk() <ul><li>Eliminate symbolic structures that cannot be converted to a concrete structure that satisfy repOk() </li></ul><ul><ul><li>Can accept symbolic structures that could lead to illegal concrete structures, i.e. it is conservative </li></ul></ul><ul><li>Abstract RepOk() can return TRUE , FALSE or Don’t Know </li></ul><ul><ul><li>if FALSE , eliminate structure </li></ul></ul><ul><ul><li>if TRUE or Don’t Know , continue ... </li></ul></ul><ul><li>Example: (2) Red nodes have only black children. </li></ul>FALSE TRUE Don’t Know
    62. 62. White-box TIG: cover branches in deleteEntry(Entry p) /* precondition: p. repOk() */ private void deleteEntry(Entry p) { if (p.left != null && p.right != null ) { Entry s = successor(p); swapPosition(s, p); } Entry replacement = (p.left != null ? p.left : p.right); if (replacement != null ) { replacement.parent = p.parent; if (p.parent == null ) root = replacement; else if (p == p.parent.left) { p.parent.left = replacement; } else p.parent.right = replacement; p.left = p.right = p.parent = null ; if (p.color == BLACK) fixAfterDeletion(replacement); ...
    63. 63. Symbolic Execution for white-box TIG if (p.left != null && p.right != null ) { ... Symbolic structure before executing branch Concretize Concrete structure that will cover the code The symbolic structure is used as input to repOk() and lazily executed to obtain the concrete structure Symbolic structure(s) that cover the branch This structure “passes” the abstract repOk()
    64. 64. API Based Testing SUT ENV (m,n) m is the seq. length of API calls & n is the number of values used in the parameters of the calls API … put(v) del(v) Evaluate different techniques for selecting test-cases from ENV(m,n) to obtain maximum coverage
    65. 65. Framework SUT with minor instrumentation ENV TestListener Abstraction Mapping + State Storage Coverage Manager JPF
    66. 66. Environment Skeleton M : sequence length N : parameter values A : abstraction used for (int i = 0; i < M; i++) { int x = Verify.random(N - 1); switch (Verify.random(1)) { case 0: put(x); break; case 1: remove(x); break; } } Verify.ignoreIf(checkStateMatch());
    67. 67. Symbolic Environment Skeleton M : sequence length A : abstraction used for (int i = 0; i < M; i++) { SymbolicInteger x = new SymbolicInteger(“X“+i); switch (Verify.random(1)) { case 0: put(x); break; case 1: remove(x); break; } } Verify.ignoreIf(checkStateMatch());
    68. 68. Sample Output Test case number 77 for '15,L+R+P-REDroot ': put(0);put(4);put(5);put(1);put(2);put(3);remove(4); Unique ID for the test Branch Number Predicate Values Test-case to achieve above coverage Test case number 7 for '32,L-R-P+RED': X2 (0) == X1 (0) && X2 (0) < X0 (1) && X1 (0) < X0 (1) put( X0 );put( X1 );remove( X2 ); Test case number 7 for '32,L-R-P+RED': put(1);put(0);remove(0); Concrete Symbolic Path Condition with solutions Symbolic TC
    69. 69. Subsumption Checking x1 x2 x3 x4 x5 + x1 > x2 & x2 > x3 & x2 < x4 & x5 > x1 x1 x2 x3 x4 x5 + x1 > x2 & x2 > x3 & x2 < x4 & x5 > x1 If only it was this simple!
    70. 70. Existential Elimination x1 x2 x3 x4 x5 PC s1 < s2 & s4 > s3 & s4 < s1 & s4 < s5 & s7 < s2 & s7 > s1 s1 s4 s2 s3 s5 +  s1,s2,s3,s4,s5 such that x1 = s1 & x2 = s4 & x3 = s3 & x4 = s5 & x5 = s2 & PC x1 > x2 & x2 > x3 & x2 < x4 & x5 > x1
    71. 71. Results from ISSTA 2006 Paper <ul><li>We compared the following techniques (1 st 3 are exhaustive; last 2 lossy) </li></ul><ul><ul><li>Traditional Model Checking </li></ul></ul><ul><ul><li>Model Checking using Symmetry Reductions </li></ul></ul><ul><ul><li>Symbolic Execution </li></ul></ul><ul><ul><li>Model Checking using Abstract Matching on the shape of the containers </li></ul></ul><ul><ul><li>Random Testing </li></ul></ul><ul><li>Using the following container classes </li></ul><ul><ul><li>Binary Tree </li></ul></ul><ul><ul><li>Fibonacci Heap </li></ul></ul><ul><ul><li>Binomial Heap </li></ul></ul><ul><ul><li>Tree Map </li></ul></ul><ul><li>In how well they achieve </li></ul><ul><ul><li>Statement coverage </li></ul></ul><ul><ul><li>Predicate coverage </li></ul></ul><ul><li>As we increase the length of the sequence of API calls </li></ul><ul><ul><li>For the concrete cases we also increase the number of parameters to be the same as the API calls </li></ul></ul><ul><li>We found the following general result </li></ul><ul><ul><li>Traditional Model Checking failed on all but the most trivial tasks (i.e. statement coverage) </li></ul></ul><ul><ul><li>Model Checking with Symmetry reductions worked much better here (scaled to much larger sequence lengths) </li></ul></ul><ul><ul><li>Symbolic Execution worked the best of the Exhaustive techniques (got optimal coverage in a number of cases) – in one case one needed a sequence length of 14 calls to obtain branch coverage and symbolic execution could do it </li></ul></ul><ul><ul><li>Lossy techniques could obtain optimal Predicate coverage </li></ul></ul><ul><ul><li>Shape abstractions worked better than random selection but only just </li></ul></ul>See examples/issta2006 folder under the SVN repository to reproduce the experiments
    72. 72. Symbolic Execution Demo <ul><li>Start by Running Saswat Anand’s symbolic instrumenter on the SwapValues.java File </li></ul><ul><li>Then run it through JPF using symbolic execution </li></ul><ul><ul><li>We will use CVC-Lite as a decision since it is the only decision procedure that currently compiles under Windows! </li></ul></ul>
    73. 73. JPF Symbolic Execution - BEFORE Omega Interface Formula satisfiable/unsatisfiable Omega Java Version JPF
    74. 74. JPF Symbolic Execution - NOW Generic Decision Procedure Interface Formula satisfiable/unsatisfiable Omega Maryland JPF CVCLite Stanford Yices SRI STP Stanford Collaborator Saswat Anand
    75. 75. Communication Methods <ul><li>Issue </li></ul><ul><ul><li>JPF and the Interface code is in Java </li></ul></ul><ul><ul><li>Decision procedures are not in Java, mainly C/C++ code </li></ul></ul><ul><li>Various different ways of communication </li></ul><ul><li>Native </li></ul><ul><ul><li>Using JNI to call the code directly </li></ul></ul><ul><li>Pipe </li></ul><ul><ul><li>Start a process and pipe the formulas and results back and forth </li></ul></ul><ul><li>Files </li></ul><ul><ul><li>Same as Pipe but now use files as communication method </li></ul></ul>
    76. 76. Optimization using Tables JPF State Path Condition: X > Y & Z > X & … JPF State Path Condition: pc100 <ul><li>Outside JPF </li></ul><ul><li>Table of PCs – </li></ul><ul><li>… </li></ul><ul><li>X > Y </li></ul><ul><li>Z > X </li></ul><ul><li>X >Y & Z > X & … </li></ul>45 46 100 … …
    77. 77. Optimization – Run DPs incrementally <ul><li>Some decision procedures support running in a incremental mode where you do not have to send the whole formula at a time but just what was added and/or removed. </li></ul><ul><ul><li>CVCLite </li></ul></ul><ul><ul><li>Yices </li></ul></ul>
    78. 78. Decision Procedure Options <ul><li>+symbolic.dp= </li></ul><ul><ul><li>omega.file </li></ul></ul><ul><ul><li>omega.pipe </li></ul></ul><ul><ul><li>omega.native </li></ul></ul><ul><ul><li>omega.native.inc </li></ul></ul><ul><ul><ul><li>… inc - with table optimization </li></ul></ul></ul><ul><ul><li>yices.native </li></ul></ul><ul><ul><li>yices.native.inc </li></ul></ul><ul><ul><li>yices.native.incsolve </li></ul></ul><ul><ul><ul><li>… incsolve - Table optimization and incremental solving </li></ul></ul></ul><ul><ul><li>cvcl.file </li></ul></ul><ul><ul><li>cvcl.pipe </li></ul></ul><ul><ul><li>cvcl.native </li></ul></ul><ul><ul><li>cvcl.native.inc </li></ul></ul><ul><ul><li>cvcl.native.incsolve </li></ul></ul><ul><ul><li>stp.native </li></ul></ul><ul><li>If using File or Pipe one must also set </li></ul><ul><ul><li>Symbolic.<name>.exe to the executable binary for the DP </li></ul></ul><ul><li>For the rest one must set LD_LIBRARY_PATH to where the DP libraries are stored </li></ul><ul><ul><li>Extensions/symbolic/CSRC </li></ul></ul><ul><li>Currently everything works under Linux and only CVCLite under Windows </li></ul><ul><ul><li>Symbolic.cvclite.exe = cvclite.exe must be set with CVClite.exe in the Path </li></ul></ul>
    79. 79. Results TCAS
    80. 80. Results TreeMap STP took > 1 hour
    81. 81. State Matching in JPF VM State Matchable + Restorable Stored State (hashed) Matchable compression Collaborator Peter Dillinger Abstract State erase some parts of the state could be lossy Matchable
    82. 82. Old Architecture StateSet int[] bool VM/Search <ul><li>Integrated collapse compression of raw VM state </li></ul><ul><li>integrated backtracker </li></ul>
    83. 83. New Architecture VM/Search DefaultBacktracker VM bool int[] Restorer Serializer Serialized StateSet int[] set VM
    84. 84. Old Scheme in the New Architecture VM/Search DefaultBacktracker FullStateSet int[] set bool int[] Collapsing (de)Serializer int[] VM
    85. 85. New Architecture VM/Search CollapsingRestorer FilteringSerializer DefaultBacktracker VM bool int[] objects This is the default setting for JPF at the moment: vm.backtracker.class = gov.nasa.jpf.jvm.DefaultBacktracker vm.restorer.class = gov.nasa.jpf.jvm.CollapsingRestorer vm.serializer.class = gov.nasa.jpf.filter.FilteringSerializer vm.storage.class = gov.nasa.jpf.jvm.JenkinsStateSet JenkinsStateSet int[] set VM
    86. 86. FilteringSerializer <ul><li>By default captures the fundamental VM state </li></ul><ul><ul><li>i.e. with all known unnecessary information removed </li></ul></ul><ul><li>Can also ignore fields based on annotations </li></ul><ul><ul><li>@FilterField(condition=&quot;foo&quot;) static int mods = 0; </li></ul></ul><ul><ul><li>Filters the field if the condition is true, the condition statement is optional </li></ul></ul><ul><li>Does garbage collection after erasing/filtering fields </li></ul><ul><li>Does heap canonicalization to give (heap) symmetry reductions </li></ul><ul><li>By default it eliminates fields like “modCount” from the java.util classes </li></ul><ul><ul><li>Otherwise even the most trivial example will have an infinite state space </li></ul></ul><ul><ul><li>See the ModCount.java demo </li></ul></ul><ul><li>Can easily specify shape abstractions by filtering out all non-reference fields from an object </li></ul><ul><ul><li>This will allow automatically doing the ISSTA 2006 shape abstraction </li></ul></ul>
    87. 87. New Architecture Revisited - Abstraction VM/Search CollapsingRestorer AbstractingSerializer DefaultBacktracker VM bool int[] objects This is the setting for the above configuration: vm.backtracker.class = gov.nasa.jpf.jvm.DefaultBacktracker vm.restorer.class = gov.nasa.jpf.jvm.CollapsingRestorer vm.serializer.class = gov.nasa.jpf.abstraction.abstractingSerializer vm.storage.class = gov.nasa.jpf.jvm.JenkinsStateSet JenkinsStateSet int[] set VM
    88. 88. AbstractingSerializer <ul><li>First does FilteringSerializer </li></ul><ul><li>Also gives the ability to specify abstractions that are not just deleting part of the state (what FilteringSerializer does) </li></ul><ul><ul><li>The parts of the VM state is converted to an abstract state graph node </li></ul></ul><ul><ul><ul><li>Graph based abstractions can then be performed </li></ul></ul></ul><ul><ul><li>Other examples include </li></ul></ul><ul><ul><ul><li>Rounding floating point to the nearest 0.001 </li></ul></ul></ul><ul><ul><ul><li>Include only difference of two variables </li></ul></ul></ul><ul><ul><ul><li>Ignore ordering of values </li></ul></ul></ul><ul><li>Adds support for sets modulo permutations, which is currently only used for specifying the set of threads </li></ul><ul><ul><li>i.e. this now gives thread symmetry reductions </li></ul></ul>
    89. 89. Examples <ul><li>K9 revisited </li></ul><ul><ul><li>Filtering visits many less states </li></ul></ul><ul><ul><li>Abstraction doesn’t give anything here since there is no thread symmetry to exploit </li></ul></ul><ul><li>ModCount example </li></ul><ul><ul><li>Infinite state without filtering </li></ul></ul><ul><li>Dining Philosophers </li></ul><ul><ul><li>Regular setup cannot go beyond 5 philosophers </li></ul></ul><ul><ul><li>Abstraction is better than Filtering here, but takes longer </li></ul></ul><ul><ul><ul><li>There is a lot of thread symmetries here to exploit </li></ul></ul></ul><ul><li>Filtering Fields </li></ul><ul><ul><li>Show the effect of removing fields through annotations </li></ul></ul>
    90. 90. User-Interface Model Checking <ul><li>Extension to JPF to deal with UIs directly </li></ul><ul><ul><li>See extensions/ui directory </li></ul></ul><ul><ul><li>Deals with swing and awt </li></ul></ul><ul><li>New JPF front-end for querying a UI to find its structure </li></ul><ul><li>Allows the scripting of nondeterministic executions using the structure of the UI </li></ul><ul><ul><li>Very simple language right now but will be extended in the future </li></ul></ul><ul><li>This is work in progress, but it has been able to find a serious problem in a prototype launch sequencer to be used in the launch vehicle for the ARES project (going back to the Moon and on to Mars) </li></ul><ul><li>TestMe demo </li></ul><ul><ul><li>Run TestMe by itself and see if you can find a bug! </li></ul></ul><ul><ul><li>Then run UIInspector on it: </li></ul></ul><ul><ul><ul><li>JPF finds the bug </li></ul></ul></ul><ul><ul><ul><li>Replays it on the real UI for TestMe </li></ul></ul></ul>Collaborator Peter Mehlitz
    91. 91. Something Completely Different <ul><li>Well, maybe not completely … </li></ul><ul><li>We developed a new static analysis tool for Java over the summer </li></ul><ul><ul><li>Aaron Tomb (USC) did all the hard work </li></ul></ul><ul><li>Problem we were addressing </li></ul><ul><ul><li>Static analyzers for finding defects are now popular tools, even commercial successes </li></ul></ul><ul><ul><ul><li>Coverity and KlockWork to name two </li></ul></ul></ul><ul><ul><li>But once they find a possible how do you know it is a real bug </li></ul></ul><ul><li>When we find a possible bug we want an input that will lead us to the bug </li></ul><ul><li>Existing tools didn’t give us a hook into the internals to allow enough information to create such an input </li></ul><ul><li>We developed a new tool based on symbolic execution to do it </li></ul><ul><ul><li>Hooked it into SOOT </li></ul></ul><ul><ul><li>Used CVCLite as a decision procedure </li></ul></ul><ul><ul><li>Once a constraint on the inputs is found that leads to a possible bug, we solve the constraints with POOC and run the code with those inputs </li></ul></ul><ul><li>Highly customizable </li></ul><ul><ul><li>Allows intra- or inter-procedural analysis </li></ul></ul><ul><ul><li>Heuristics for finding array bounds exceptions </li></ul></ul><ul><ul><li>Etc. </li></ul></ul><ul><li>Again work in progress, but has found real NASA bugs! </li></ul>Collaborator Aaron Tomb
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×