Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2014)

1,477 views

Published on

Hyunmin's FSE presentation.

Published in: Software
  • Be the first to comment

How We Get There: A Context-Guided Search Strategy in Concolic Testing (FSE 2014)

  1. 1. How We Get There: A Context-Guided Search Strategy in Concolic Testing Hyunmin Seo and Sunghun Kim The Hong Kong University of Science and Technology Nov. 19 2014 FSE, Hong Kong
  2. 2. Concolic Testing 2 푝푐1 푝푐2 푝푐3 푝푐4 휋1 푰ퟏ b9 푃퐶 = 푝푐1⋀푝푐2⋀푝푐3⋀푝푐4⋀… 푃퐶′ = 푝푐1⋀푝푐2⋀¬푝푐3 푰ퟑ 휋2 휋1 휋3 b1 b2 b3 b4 푰ퟐ ⟹ Execution Tree 푰ퟐ b1 b2 b9 b10 휋2 휋1
  3. 3. Path Explosion Challenge • Exponentially many execution paths 3 4 conditional nodes 16 (24) execution paths
  4. 4. Path Explosion Challenge • grep – text search utility • 19K LOC • Statically - 4K branches in CFG • Dynamically - 8K branches in an execution path 4 *CFG of re_match_2_internal in grep
  5. 5. Search Strategy • Given a limited testing budget, select high-priority branches first to improve coverage fast 5 • Run 푷 with a concrete input 푰 • Execution path 흅 = 풃ퟏ풃ퟐ풃ퟑ … • Select a branch 풃풊 • Generate 푰′ for 흅′ = 풃ퟏ풃ퟐ풃ퟑ…풃 풊 • By symbolic execution and constraint solving Repeat
  6. 6. Existing Search Strategies 6 DFS BFS Random
  7. 7. Coverage-Optimized Strategies 7 • CFG – How far is any uncovered branch from this? • CarFast – How many can be covered by this? • Generational – How many are actually covered by this?
  8. 8. Context-Guided Search Considers context of a branch and selects a branch in the new context 8
  9. 9. Intuition behind CGS Explore diverse state of P by avoiding exploring the same state - “same” in the limited context 9
  10. 10. CGS Example 10 b7 b8 b7 b7 b7 b7 휋1 휋2 휋3 휋4 b5 b6
  11. 11. CGS Example b6 b6 b6 b5 11 b5 b6 b7 b8 b7 b7 b7 b7 휋1 휋2 휋3 휋4 SELECT SKIP SKIP SELECT
  12. 12. Context •k-Context of branch b • A sequence of k preceding branches of b in an execution path • Example • 3-Context of b6 : (b6, b4, b1) 12 b2 b4 b6 π1 b1 b3 b5
  13. 13. Optimal k • 1-Context • Select each branch only once • ∞-Context • Select all branches • Optimal k for the best coverage depends on • Testing budget • Characteristic of the subjects 13
  14. 14. Incremental Search 14 BFS traversal 1-context 2-context 3-context
  15. 15. Dominators if every path from the entry node to node n must go through node d 15 Node d dominates node n, Dom(b11) {b9, b7, b5, b3} 2-Context of b11 along 흅ퟏ= (b11, b9) 2-Context of b11 along 흅ퟐ= (b11, b9) 2-Context of b11 along 흅ퟏ= (b11, b1) 2-Context of b11 along 흅ퟐ= (b11, b2) b1 b2 b3 b5 b7 b9 b11 b4 b6 b8 b10 b12 π1 π2
  16. 16. Research Questions • RQ1 – Given the same testing budget, how many branches can be covered? • RQ2 – Given a target coverage goal, how many iterations are required to achieve the goal? • RQ3 – What is the effect of dominators and incremental k? • RQ4 – How different are the covered branch sets by different strategies? 16
  17. 17. Evaluation Subjects 17 Subject Testing Tool Language LOC grep CREST C 19K replace CREST C 0.5K expat CREST C 18K cdaudio CREST C 2K floppy CREST C 1.5K kbfiltr CREST C 1K tp300 CarFastTool Java 0.3K tp600 CarFastTool Java 0.6K tp1k CarFastTool Java 1.5K tp2k CarFastTool Java 2.4K tp5k CarFastTool Java 5.8K tp10k CarFastTool Java 28K
  18. 18. 18 RQ1-Coverage – C Subjects
  19. 19. 19 RQ1-Coverage – C Subjects
  20. 20. RQ1-Coverage - Java Subjects 20 350 300 250 200 150 100 tp600 0 500 1000 1500 Number of branches covered Iterations CGS CarFast
  21. 21. RQ2 – Reaching the Target (C Subjects) 21
  22. 22. RQ2 – Reaching the Target (Java Subjects) 22
  23. 23. RQ3 - Effect of Dominators 23
  24. 24. RQ3 - Effect of Incremental Search 24
  25. 25. RQ4 - Comparison of Covered Branch Sets CovCGS - A set of branches covered by CGS CovOthers - A set of branches covered by other strategies CovOthers ≤ |CovCGS| Cov CovCGS Others 25 CovCGS CovOthers
  26. 26. RQ4 - Comparison of Covered Branch Sets 26 CovCGS CovOthers replace, cdaudio, floppy, kbfiltr Tp300, tp600, tp1k, tp2k CovOthers ⊆ |CovCGS| grep CovOthers − CovCGS |CovCGS| = 0.1 ~ 3% Cov CovCGS CFG 61 1606 383
  27. 27. Threats to Validity • Precision in Symbolic Execution • Non-linear expression, Floating-Point operations, Symbolic pointer dereferencing • Input vector • Size of input, Optional arguments • External Validity • Test subjects and strategies might not be representative 27
  28. 28. Summary • Path explosion challenge in Concolic testing • Search strategies prioritizes branches according to some criteria • CGS • Selects branches in the new context • Use dominators to exclude irrelevant branches • BFS search + incrementally increase of the size of context • Evaluation on six C and six Java subjects • Achieved the highest coverage on all twelve subjects • Reached the target much faster on most subjects 28
  29. 29. Backup Slides 29
  30. 30. Coverage for C Subjects 30
  31. 31. Coverage for Java Subjects 31 Mann-Whitney U test P-value < 0.01
  32. 32. Related Work • Pruning Redundant Path • RWset [Cristian ‘08] • Interpolation [Jaffar ’13] • Function Summary • Compositional [Godefroid ‘07, ‘10] • Demand-driven compositional [Anand ‘08] • Others • Fitness-guided approach [Xie ’09], Sub-path guided [Li ‘13] • Hybrid [Majumdar ‘07] 32
  33. 33. Branch Selection in CGS 33
  34. 34. Search Strategies in KLEE 34

×