Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Comparing Metaheuristic Algorithms for Error Detection in Java Programs

576 views

Published on

Presentation in the Symposium on Search Based Software Engineering. In Szeged (Hungary), september 2011.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Comparing Metaheuristic Algorithms for Error Detection in Java Programs

  1. 1. ConclusionsIntroduction Background Algorithms Experiments & Future Work Comparing Metaheuristic Algorithms for Error Detection in Java Programs Francisco Chicano, Marco Ferreira and Enrique Alba SSBSE 2011, Szeged, Hungary, September 10-12 1 / 25
  2. 2. Conclusions Introduction Background Algorithms Experiments & Future WorkMotivationMotivation •  Concurrent software is difficult to test ... •  ... and it is in the heart of a lot of critical systems •  Techniques for proving the correctness of concurrent software are required •  Model checking → fully automatic •  Traditional techniques for this purpose have problems with large models •  We compare several metaheuristics and classical algorithms for model checking SSBSE 2011, Szeged, Hungary, September 10-12 2 / 25
  3. 3. Conclusions Introduction Background Algorithms Experiments & Future WorkExplicit State MC Properties State Explosion Heuristic MCExplicit State Model Checking •  Objective: Prove that model M satisfies the property : •  In the general case, f is a temporal logic formula (LTL, CTL, etc.) Model M LTL formula ¬ f Intersection Büchi automaton (never claim) s0 !p∨q s0 s5 s3 ∩ s0 s2 s1 s3 s1 p∧!q s2 q s2 s5 s1 s7 s4 s4 s6 s8 s9 SSBSE 2011, Szeged, Hungary, September 10-12 3 / 25
  4. 4. Conclusions Introduction Background Algorithms Experiments & Future WorkExplicit State MC Properties State Explosion Heuristic MCExplicit State Model Checking •  Objective: Prove that model M satisfies the property : •  In the general case, f is a temporal logic formula (LTL, CTL, etc.) Model M LTL formula ¬ f Intersection Büchi automaton (never claim) s0 !p∨q s0 s5 s3 ∩ s0 s2 = s1 s3 s1 p∧!q s2 q s2 s5 s1 s7 s4 s4 s6 s8 s9 SSBSE 2011, Szeged, Hungary, September 10-12 4 / 25
  5. 5. Conclusions Introduction Background Algorithms Experiments & Future WorkExplicit State MC Properties State Explosion Heuristic MCExplicit State Model Checking •  Objective: Prove that model M satisfies the property : •  In the general case, f is a temporal logic formula (LTL, CTL, etc.) Model M LTL formula ¬ f Intersection Büchi automaton (never claim) s0 !p∨q s0 s5 s3 ∩ s0 s2 = s1 s3 s1 p∧!q s2 q s2 s5 s1 s7 s4 s4 s6 s8 s9 Using Nested-DFS SSBSE 2011, Szeged, Hungary, September 10-12 5 / 25
  6. 6. Conclusions Introduction Background Algorithms Experiments & Future WorkExplicit State MC Properties State Explosion Heuristic MCSafety properties s0 s1 s3 Properties in s2 s5 JPF s7 s4 •  Exceptions s6 •  Deadlocks s8 s9 •  An error trail is an execution path ending in an error state •  The search for errors is transformed in a graph exploration problem (DFS, BFS) SSBSE 2011, Szeged, Hungary, September 10-12 6 / 25
  7. 7. Conclusions Introduction Background Algorithms Experiments & Future WorkExplicit State MC Properties State Explosion Heuristic MCState Explosion Problem •  Number of states very large even for small models s0 s1 s3 s2 Memory s5 s7 s4 s6 s8 s9 •  Example: Dining philosophers with n philosophers → 3n states •  For each state we need to store the heap and the stacks of the different threads •  Solutions: collapse compression, minimized automaton representation, bitstate hashing, partial order reduction, symmetry reduction •  Large models cannot be verified but errors can be found SSBSE 2011, Szeged, Hungary, September 10-12 7 / 25
  8. 8. Conclusions Introduction Background Algorithms Experiments & Future WorkExplicit State MC Properties State Explosion Heuristic MCHeuristic Model Checking •  The search for errors can be directed by using heuristic information 5 s0 2 2 Heuristic value s1 3 s3 s2 1 s5 0 s7 4 s4 0 s6 6 s8 7 s9 •  Different kinds of heuristic functions have been proposed in the past: •  Formula-based heuristics •  Deadlock-detection heuristics •  Structural heuristics •  State-dependent heuristics SSBSE 2011, Szeged, Hungary, September 10-12 8 / 25
  9. 9. Conclusions Introduction Background Algorithms Experiments & Future WorkAlgorithms GA GAMO PSO ACO SAClassification of Algorithms GA, GAMO, (working on) EDA PSO, SA, ACO RS RDFS Stochastic Guided BS Non-guided Determinism A* Deterministic ? Non-complete Complete DFS, BFS Completeness SSBSE 2011, Szeged, Hungary, September 10-12 9 / 25
  10. 10. Conclusions Introduction Background Algorithms Experiments & Future WorkAlgorithms GA GAMO PSO ACO SAGenetic Algorithm Solution encoding (floating point values) 0.5 0.1 0.9 0.3 0.5 0.9 Crossover Mutation 0.5 0.1 0.9 0.3 0.5 0.9 → 0.5 0.1 0.6 0.3 0.5 0.9 SSBSE 2011, Szeged, Hungary, September 10-12 10 / 25
  11. 11. Conclusions Introduction Background Algorithms Experiments & Future WorkAlgorithms GA GAMO PSO ACO SAGenetic Algorithm with Memory Operator Solution encoding (floating point values) 0.5 0.1 0.9 0.3 0.5 0.9 Index in a table of states 0 1 2 3 … … SSBSE 2011, Szeged, Hungary, September 10-12 11 / 25
  12. 12. Conclusions Introduction Background Algorithms Experiments & Future WorkAlgorithms GA GAMO PSO ACO SAParticle Swarm Optimization Particles 0.2 -1.4 -3.5 → Position (solution) 1.0 10.3 7.2 → Velocity Personal best Inertia Global best SSBSE 2011, Szeged, Hungary, September 10-12 12 / 25
  13. 13. Conclusions Introduction Background Algorithms Experiments & Future WorkAlgorithms GA GAMO PSO ACO SAAnt Colony Optimization Trail k τij•  The ant selects stochastically its next node Heuristic i ηij Ni•  The probability of selecting one node depends on the pheromone trail and the m heuristic value (optional) of the edge/node j l•  The ant stops when a complete k solution is built SSBSE 2011, Szeged, Hungary, September 10-12 13 / 25
  14. 14. Conclusions Introduction Background Algorithms Experiments & Future WorkAlgorithms GA GAMO PSO ACO SASimulated Annealing Neighbor 0.5 0.1 0.9 0.3 0.5 0.9 → 0.5 0.1 0.6 0.3 0.5 0.9 SSBSE 2011, Szeged, Hungary, September 10-12 14 / 25
  15. 15. Conclusions Introduction Background Algorithms Experiments & Future WorkParameterization Hit Rate Length of Error TrailsParameterization •  We used 3 scalable and 2 non-scalable models for the experiments Program LoC Classes Processes dinj 63 1 j+1 phij 176 3 j+1 marj 186 4 j+1 giop 746 13 7 garp 458 7 7 •  Maximum number of expanded states: 200 000 •  Fitness function: •  100 independent executions of stochastic algorithms SSBSE 2011, Szeged, Hungary, September 10-12 15 / 25
  16. 16. Conclusions Introduction Background Algorithms Experiments & Future WorkParameterization Hit Rate Length of Error TrailsParameterization •  We used 3 scalable and 2 non-scalable models for the experiments Program LoC Classes Processes j=4 to 20 dinj 63 1 j+1 phij 176 to 36 3 j=4 j+1 marj 186 4 j+1 giop 746j=2 to 1013 7 garp 458 7 7 •  Maximum number of expanded states: 200 000 •  Fitness function: •  100 independent executions of stochastic algorithms SSBSE 2011, Szeged, Hungary, September 10-12 16 / 25
  17. 17. Conclusions Introduction Background Algorithms Experiments & Future WorkParameterization Hit Rate Length of Error TrailsHit rate SSBSE 2011, Szeged, Hungary, September 10-12 17 / 25
  18. 18. Conclusions Introduction Background Algorithms Experiments & Future WorkParameterization Hit Rate Length of Error TrailsHit rate phi SSBSE 2011, Szeged, Hungary, September 10-12 18 / 25
  19. 19. Conclusions Introduction Background Algorithms Experiments & Future WorkParameterization Hit Rate Length of Error TrailsHit rate din SSBSE 2011, Szeged, Hungary, September 10-12 19 / 25
  20. 20. Conclusions Introduction Background Algorithms Experiments & Future WorkParameterization Hit Rate Length of Error TrailsHit rate SSBSE 2011, Szeged, Hungary, September 10-12 20 / 25
  21. 21. Conclusions Introduction Background Algorithms Experiments & Future WorkParameterization Hit Rate Length of Error TrailsLength of Error Trails BS: 753 SSBSE 2011, Szeged, Hungary, September 10-12 21 / 25
  22. 22. Conclusions Introduction Background Algorithms Experiments & Future WorkParameterization Hit Rate Length of Error TrailsLength of Error Trails BS: 172 DFS: 245 BS: 260 DFS: 378 SSBSE 2011, Szeged, Hungary, September 10-12 22 / 25
  23. 23. Conclusions Introduction Background Algorithms Experiments & Future WorkParameterization Hit Rate Length of Error TrailsLength of Error Trails SSBSE 2011, Szeged, Hungary, September 10-12 23 / 25
  24. 24. Conclusions Introduction Background Algorithms Experiments & Future WorkConclusions & Future WorkConclusions & Future Work Conclusions •  Metaheuristics are more effective than classical algorithms in finding errors •  Beam Search has advantages over complete search algorithms •  An even distribution of the search in depth levels tends to raise hit rate •  Stochastic algorithms obtain short error trails Future Work •  Design a stochastic complete guided algorithm to find errors and verify •  Design of hybrid algorithms to more efficiently explore the search space •  Explore the design of parallel metaheuristics for this problem SSBSE 2011, Szeged, Hungary, September 10-12 24 / 25
  25. 25. Comparing Metaheuristic Algorithms for Error Detection in Java ProgramsThanks for your attention !!! SSBSE 2011, Szeged, Hungary, September 10-12 25 / 25

×