Miniatur at FSE 2007
Upcoming SlideShare
Loading in...5
×
 

Miniatur at FSE 2007

on

  • 362 views

This is the presentation I gave about the Miniatur model checking for Java project at FSE 2007

This is the presentation I gave about the Miniatur model checking for Java project at FSE 2007

Statistics

Views

Total Views
362
Views on SlideShare
362
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Miniatur at FSE 2007 Miniatur at FSE 2007 Presentation Transcript

  • Finding Bugs Efficiently with a SAT Solver Julian Dolby, Mandana Vaziri, Frank Tip IBM Thomas J. Watson Research Center FSE 2007—Dubrovnik, Croatia—September 6, 2007
  • Outline• Background• Miniatur Contributions• Example• Miniatur Evaluation• Related Work• Conclusions and Future Work
  • Background—Finding Bugs• Finding bugs must balance coverage and precision – Tools must find bugs to be useful – Tools must minimize false positives to be useful
  • Background—Finding Bugs• Testing approaches find real bugs – Each test is an actual execution, so all bugs real – Coverage depends upon test suite, so often spotty
  • Background—Finding Bugs• Conservative static analysis approaches find all bugs – Conservative approximation of all executions – Can find false positives due to approximation
  • Background—Systematic Underapproximation• Systematic underapproximation blends testing and analysis – Explores all possible concrete execution within a finite set – Real bugs and total coverage within for set of executions
  • Background—Systematic Underapproximation• Key issue is choosing a “good” set of execution – Set must be tractable – Set must cover interesting range of all executions• Use “small scope hypothesis” to bound set of executions – Hypothesis: most data structure bugs need only few objects – E.g. Collection bugs can be seen by inserting few elements – Explore executions with small heaps (also bound loops) (small scope) (other approximation)
  • Background—Finding Bugs with SAT• Given program, specification, find concrete counter example• Use relational first-order logic (FOL) (Alloy [Jackson et al]) – Universe of atoms, and bounded relations over atoms – Relational operators, first-order formulae over relations – Tool (Kodkod[Torlak et al]) translates to CNF for SAT solver• Java in Relational FOL [Vaziri et al] [Taghdiri et al] [Dennis et al] – Heap objects as atoms, types as unary relations – Fields as binary relations, code as formulae – Encode integer operations using bit sets – Solve FOL formula program ∧ ¬(spec)
  • Miniatur Contributions• Supports much larger range of integers than previous work – Integers represented by one atom per power of 2 – Enables e.g. use of hashCode in updating a collection• Novel sparse representation of array objects – Allows subset of array indices to have values – Enables Miniatur to handle array-based collections• Novel sliced translation based on control dependence• General theoretical condition for slicing relational formulae• Demonstration on real collections and a few real programs – Arrays, integers enable analyzing all Java collections – Integers allow checking Java equality contracts
  • Example—HashMap Dataclass HashMap ... { Entry[] table; ... HashM ap ← {H1} class Entry { Array ← {A1} Object key; Entry ← {E1} Object value; Object ← {A1, E1, H1, K1, V 1} int hash; table ← { H1, A1 } Entry next;...}} key ← { E1, K1 } value ← { E1, V 1 } hash ← { E1, #65 } I ← { A1, N 1, #5 } V ← { A1, N 1, E1 }
  • Example—HashMap Dataclass HashMap ... { Entry[] table; ... HashM ap ← {H1} class Entry { Array ← {A1} Object key; Entry ← {E1} Object value; Object ← {A1, E1, H1, K1, V 1} int hash; table ← { H1, A1 } Entry next;...}} key ← { E1, K1 } value ← { E1, V 1 } hash ← { E1, #1 , E1, #64 } I ← { A1, N 1, #1 , A1, N 1, #4 } V ← { A1, N 1, E1 }
  • Example—HashMap.putpublic Object put(Object k, Object v) { int h = k.hashCode(); int i = h % table.length; for (Entry e = table[i]; e != null; e = e.next) { ... } modCount++; table[i] = new Entry(h, k, v, table[i]); return null;
  • Example—HashMap.putpublic Object put(Object k, Object v) { int h = k.hashCode(); int i = h % table.length; for (Entry e = table[i]; e != null; e = e.next) { ... } modCount++; table[i] = new Entry(h, k, v, table[i]); return null;
  • Example—HashMap.putpublic Object put(Object k, Object v) { int h = k.hashCode(); int i = h % table.length; Entry e = table[i]; if (e != null) { ... e = e.next if (e != null) { ... e = e.next if (e != null) goto end; } } modCount++; table[i] = new Entry(h, k, v, table[i]); return null;
  • Example—HashMap.putpublic Object put(Object k, Object v) { int h = k.hashCode(); int i = h % table.length; Entry e = table[i]0; if (e != null) { ... e1 = e.next; if (e1 != null) { ... e2 = e1.next; if (e2 != null) goto end; } } modCount1 = modCount0 + 1; table[i]1 = new Entry(h, k, v, table[i]0); return null;
  • Example—HashMap.putpublic Object put(Object k, Object v) { int h = k.hashCode(); int i = h % table.length; Entry e = table[i]0; if (e != null) { ... e1 = e.next; if (e1 != null) { ... e2 = e1.next; if (e2 != null) goto end; } } modCount1 = modCount0 + 1; table[i]1 = new Entry(h, k, v, table[i]0); A: assert table[i]1.next == null; // perfect hashing return null;
  • Expression for assert table[i]1.next == null ¬ (Guard(A) → E[table[i].next == null])E[table[i].next == null] ← E[table[i].next] = {N ull}E[table[i].next] ← E[table[i]].E[next] E[table[i]] ← {E[table].x.E[V 1] |sum(E[table].x.E[I1]) = E[i] } ... sum(x) ← xi ∈x int(xi ) int(j) ← int(#1) ← 1, int(#2) ← 2, int(#4) ← 4, . . . table ← {A1} , next ← { E1, N ull } T 1 ← { A1, N 1 } , i ← 5 I1 ← { A1, N 1, #1 , A1, N 1, #4 } V 1 ← { A1, N 1, E1 }
  • Expression for assert table[i]1.next == null ¬ (Guard(A) → E[table[i].next == null])E[table[i].next == null] ← E[table[i].next] = {N ull}E[table[i].next] ← E[table[i]].E[next] E[table[i]] ← {E[table].x.E[V 1] |sum(E[table].x.E[I1]) = E[i] } ... sum(x) ← xi ∈x int(xi ) int(j) ← int(#1) ← 1, int(#2) ← 2, int(#4) ← 4, . . . table ← {A1}, next ← { E1, N ull } T 1 ← { A1, N 1 } , i ← 5 I1 ← { A1, N 1, #1 , A1, N 1, #4 } V 1 ← { A1, N 1, E1 }
  • Expression for assert table[i]1.next == null ¬ (Guard(A) → E[table[i].next == null])E[table[i].next == null] ← E[table[i].next] = {N ull}E[table[i].next] ← E[table[i]].E[next] E[table[i]] ← {{A1} .x.E[V 1] |sum({A1} .x.E[I1]) = 5 } ... sum(x) ← xi ∈x int(xi ) int(j) ← int(#1) ← 1, int(#2) ← 2, int(#4) ← 4, . . . table ← {A1} , next ← { E1, N ull } T 1 ← { A1, N 1 } , i ← 5 I1 ← { A1, N 1, #1 , A1, N 1, #4 } V 1 ← { A1, N 1, E1 }
  • Expression for assert table[i]1.next == null ¬ (Guard(A) → E[table[i].next == null])E[table[i].next == null] ← E[table[i].next] = {N ull}E[table[i].next] ← E[table[i]].E[next] E[table[i]] ← {{A1} . {N 1} .E[V 1] |sum({A1} . {N 1} .E[I1]) = 5 } ... sum(x) ← xi ∈x int(xi ) int(j) ← int(#1) ← 1, int(#2) ← 2, int(#4) ← 4, . . . table ← {A1} , next ← { E1, N ull } T 1 ← { A1, N 1 } , i ← 5 I1 ← { A1, N 1, #1 , A1, N 1, #4 } V 1 ← { A1, N 1, E1 }
  • Expression for assert table[i]1.next == null ¬ (Guard(A) → E[table[i].next == null])E[table[i].next == null] ← E[table[i].next] = {N ull}E[table[i].next] ← E[table[i]].E[next] E[table[i]] ← {{A1} . {N 1} .E[V 1] |sum({#1, #4}) = 5 } ... sum(x) ← xi ∈x int(xi ) int(j) ← int(#1) ← 1, int(#2) ← 2, int(#4) ← 4, . . . table ← {A1} , next ← { E1, N ull } T 1 ← { A1, N 1 } , i ← 5 I1 ← { A1, N 1, #1 , A1, N 1, #4 } V 1 ← { A1, N 1, E1 }
  • Expression for assert table[i]1.next == null ¬ (Guard(A) → E[table[i].next == null])E[table[i].next == null] ← E[table[i].next] = {N ull}E[table[i].next] ← E[table[i]].E[next] E[table[i]] ← {{A1} . {N 1} .E[V 1] |5 = 5 } ... sum(x) ← xi ∈x int(xi ) int(j) ← int(#1) ← 1, int(#2) ← 2, int(#4) ← 4, . . . table ← {A1} , next ← { E1, N ull } T 1 ← { A1, N 1 } , i ← 5 I1 ← { A1, N 1, #1 , A1, N 1, #4 } V 1 ← { A1, N 1, E1 }
  • Expression for assert table[i]1.next == null ¬ (Guard(A) → E[table[i].next == null])E[table[i].next == null] ← E[table[i].next] = {N ull}E[table[i].next] ← E[table[i]].E[next] E[table[i]] ← {E1} ... sum(x) ← xi ∈x int(xi ) int(j) ← int(#1) ← 1, int(#2) ← 2, int(#4) ← 4, . . . table ← {A1} , next ← { E1, N ull } T 1 ← { A1, N 1 } , i ← 5 I1 ← { A1, N 1, #1 , A1, N 1, #4 } V 1 ← { A1, N 1, E1 }
  • Expression for assert table[i]1.next == null ¬ (Guard(A) → E[table[i].next == null])E[table[i].next == null] ← E[table[i].next] = {N ull}E[table[i].next] ← {N ull} E[table[i]] ← {E1} ... sum(x) ← xi ∈x int(xi ) int(j) ← int(#1) ← 1, int(#2) ← 2, int(#4) ← 4, . . . table ← {A1} , next ← { E1, N ull } T 1 ← { A1, N 1 } , i ← 5 I1 ← { A1, N 1, #1 , A1, N 1, #4 } V 1 ← { A1, N 1, E1 }
  • Guard for assert table[i]1.next == nullE[e2! = null] = F alse ∨ E[e1! = null] = F alse ∨ E[e! = null] = F alse
  • Evaluation• Miniatur tool embodies our techniques – Uses WALA for control dependence and other analyses – Uses Kodkod to generate CNF – Uses Minisat as backend SAT solver• Evaluated structural assertions in java.util collections – Check that size fields accurately reflect data structure – Use driver that inserts arbitrary objects into collection• Evaluated java equality contracts on open-source codes – (1) reflexive, (2) symmetric, (3) transitive, (4) non-null, – (5) hashCode, (6-9) compareTo properies
  • Evaluation Results: Testing java.utilclass formula Time (s) LinesLinkedList this.size = |this.header.next∗ − null| 8.2 111TreeMap this.size = 67.4 12592 |count(this.root.(lef t + right)∗ − null|TreeSet this.m.size = 80.0 12667 |this.m.root.(lef t + right)∗ − null|HashMap this.size = |this.table[].next∗ − null| 30.0 1107HashSet this.map.size = 34.8 1129 |(this.map.table[].next∗ − null|• Encode correctness of size fields• Check expressive properties against complex collections• Heap size per type, Loop unrolling
  • Evaluation Results: Testing Equals Contracts• Test each property of each equals method using harness public static void equalsTester(Object a, Object b) { if (a.equals(b)) assert b.equals(a); }• Evaluated several common, open-source benchmarks – Antlr, BCEL, Hsqldb, java cup – Found 20 concrete violations (of 2, 3, 5, 6, 7) – Most tests ran within 2 minutes• Bugs illustrated by concrete counter-examples argument a argument b ArrayType3 ArrayType2 type=54304 type=9181 basic type=UninitializedObjectType3 basic type=anonType3 ... ...
  • Related Work• Most-closely related work is checking Java with Alloy – Vaziri et al, Taghdiri et al, Dennis et al – Miniatur handles more of Java, scales better• SATURN checks C using SAT – Uses manually-tailored summaries – Checks less-rich properties – Better scaling than Miniatur• SMT solvers – Mix SAT with theories for arithmetic, arrays, etc – Can prove properties within particular theories – Current SMT solvers less expressive
  • Conclusions and Future Work• Miniatur makes SAT-based checking for Java practical – Extends prior work to better handle integers, arrays – New, sound slicing mechanism for scalability → Handles real properties on real programs → Deep structural properties on real data structures• Future work – Refinement-based approach to unrolling loops – Model concurrency