Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Efficient Statistical Debugging via 
Hierarchical Instrumentation 
1 
Zhiqiang Zuo 
Advisor: Siau-Cheng Khoo 
National Uni...
Outline 
 Introduction 
 Methodology 
 Applications 
 Related Work 
 Future Research 
2
Introduction 
 Bugs are prevalent 
 Statistical debugging 
 Bug isolation (or fault localization) 
 Bug signature disc...
Introduction 
 Assumption 
 Every program element to be potentially relevant to the 
failure 
 Consequence 
 Entire pr...
Introduction 
P 
m1 
e 
m2 
e e e e 
m3 
e e e 
mk 
e e e 
m4 
e e 
… 
… 
5
Introduction 
 Full instrumentation 
 Statistical debugging 
 Top-k suspicious 
elements 
P 
m1 
e 
m2 
e e e e 
m3 
e ...
Introduction 
 Selective 
instrumentation 
 Statistical debugging 
 Same top-k 
suspicious elements 
P 
m1 
e 
m2 
e e ...
Introduction 
 Thesis 
 We can enhance the efficiency of statistical debugging 
while upholding effectiveness by conduct...
Outline 
 Introduction 
 Methodology 
 Applications 
 Related Work 
 Future Research 
9
Methodology 
 Insight 
 Hierarchical Instrumentation (HI) 
 Coarse-grained instrumentation 
 Lightweight and full-scal...
Hierarchical Instrumentation 
 A statistical debugging approach 
 Suspiciousness measure F to quantify the correlation t...
Hierarchical Instrumentation 
 Instrumentation 
 Coarse-grained instrumentation 
 Fine-grained instrumentation 
 Coars...
Coarse-grained Pruning Measure 
 Derivation of Cp 
 Cp is an upper bound of F 
 Cp is non-decreasing 
 Cp is as close ...
Coarse-grained Ranking Measure 
 Issues of threshold θ 
 Safeness 
 θ used must be no greater than the top-kth F value ...
Outline 
 Introduction 
 Methodology 
 Applications 
 Related Work 
 Future Research 
15
Applications 
 In-house debugging 
 Run instrumented program in house to obtain executions 
 Predicated bug signature m...
Efficient Bug Signature Mining via HI 
 Perfect bug understanding 
 Bug signature 
 Comprise multiple elements providin...
Efficient Bug Signature Mining via HI 
 Full-scale instrumentation 
 High instrumentation and analysis costs 
 Selectiv...
Efficient Bug Signature Mining via HI 
 Fine-grained measure 
 Given a database D, a pattern P appears in p passing 
tra...
Efficient Bug Signature Mining via HI 
 Coarse-grained pruning measure 
 Cp = IG(0, n) 
 Given a predicate e, the funct...
Approach 
 Coarse-grained 
instrumentation 
function-entry 
P 
m1 
e 
m2 
e e e e 
m3 
e e e 
mk 
e e e 
m4 
e e 
… 
… 
2...
Approach 
 Coarse-grained 
instrumentation 
function-entry 
 Coarse-grained 
information 
Cr: ranking 
measure 
Cp: prun...
Approach 
 Rank functions in 
descending Cr 
values 
P 
C 0.8 0.4 0.55 0.7 0.0 r: 
m1 
e 
m2 
e e e e 
m3 
e e e 
mk 
e e...
Approach 
 Rank functions in 
descending Cr 
values 
 Select predicates 
within top 
functions such that 
the number of ...
Approach 
 Safe pruning 
P  θ=0.8 
m1 
C 0.9 0.75 0.6 0.85 0.2 p: 
e 
m2 
e e e e 
m3 
e e e 
mk 
e e e 
m4 
e e 
… 
… 
...
Approach 
 Safe pruning 
 θ=0.8 
 Cp(m1) ≥ θ 
Cp(m4) ≥ θ 
P 
m1 
C 0.9 0.75 0.6 0.85 0.2 p: 
e 
m2 
e e e e 
m3 
e e e ...
Efficient Bug Signature Mining via HI 
 Subjects 
 Top-k: 1 
 Sampler-cc 
29 
Subject Versions LoC Function 
s 
Tests 
...
Efficient Bug Signature Mining via HI 
 Cost comparison 
 Profile collection 
30 
 Execution time and disk storage for ...
Iterative Statistical Bug Isolation via HI 
 Field failures 
 Failures in deployed software 
 Cooperative bug isolation...
Iterative Statistical Bug Isolation via HI 
 Sampling 
 User’s perspective 
32 
 Amortize the monitoring cost to many u...
Iterative Statistical Bug Isolation via HI 
 Fine-grained measure 
 For a predicate e, pt(e) and nt(e) are the number of...
Iterative Statistical Bug Isolation via HI 
 Coarse-grained pruning measure 
 Cp = H(p, n) = 2/(1+ n/p + log2N/log2n) 
...
Approach 
 Coarse-grained 
instrumentation 
function-entry 
P 
m1 
e 
m2 
e e e e 
m3 
e e e 
mk 
e e e 
m4 
e e 
… 
… 
3...
Approach 
 Coarse-grained 
instrumentation 
function-entry 
 Coarse-grained 
information 
Cr: ranking 
measure 
Cp: prun...
Approach 
 Rank functions in 
descending Cr 
values 
 At each iteration, 
instrument 
predicates within 
one function: 
...
Approach 
 Rank functions in 
descending Cr 
values 
 At each iteration, 
instrument 
predicates within 
one function: 
...
Approach 
 Rank functions in 
descending Cr 
values 
 At each iteration, 
instrument 
predicates within 
one function: 
...
Approach 
P 
C 0.8 0.4 0.55 0.7 0.0 r: 
m1 
C 0.9 0.75 0.6 0.85 0.2 p: 
e 
m2 
e e e e 
m3 
e e e 
mk 
e e e 
m4 
e e 
… 
...
Approach 
P 
C 0.8 0.4 0.55 0.7 0.0 r: 
m1 
C 0.9 0.75 0.6 0.85 0.2 p: 
e 
m2 
e e e e 
m3 
e e e 
mk 
e e e 
m4 
e e 
… 
...
Iterative Statistical Bug Isolation via HI 
 Subjects 
 Top-k: 1 
 Sampler-cc 
44 
Subject Versions LoC Function 
s 
Te...
Iterative Statistical Bug Isolation via HI 
 Instrumentation effort 
 Percentage of predicates 
instrumented 
 Baseline...
Iterative Statistical Bug Isolation via HI 
 User’s overhead comparison 
 Our (non-sampling) iterative approach 
 Sampl...
Outline 
 Introduction 
 Methodology 
 Applications 
 Related Work 
 Future Research 
47
Related Work 
 Selective instrumentation 
 HOLMES [ICSE’09] and Adaptive bug isolation [ICSE’10] 
 Partial code instrum...
Related Work 
 Statistical bug isolation 
 Entity & Suspiciousness measure 
 Tarantula [ASE’05], Ochiai [ASE’09], CBI [...
Outline 
 Introduction 
 Methodology 
 Applications 
 Related Work 
 Future Research 
50
Future Research 
 Derive coarse-grained measures automatically 
 Pruning measure 
 Ranking measure 
 Improve debugging...
Thank you! 
questions, comments, advice? 
53
54
Coarse-grained Pruning Measure 
 Given F: N2–›R defined over the intervals ([0, X], [0, Y]), 
Cp under the same domain sh...
Necessary Condition 
 Let e be a fine-grained element, m be the composite 
construct containing e. p(e) (or p(m)) and n(e...
Coarse-grained Ranking Measure 
 Cr value of a coarse-grained element should be 
highly correlated with the F value of en...
Profile Collection 
Execution time (in seconds) during profile collection 
MPS HIMPS Ratio(%) 
Subject original coarse boo...
Profile Collection 
Disk storage space used (in KB) for profiles 
MPS HIMPS Ratio(%) 
Subject original coarse boost prune ...
Preprocessing 
Time (in seconds) and peak memory consumption (in KB) for 
MPS HIMPS Ratio(%) 
original boost prune total/o...
Mining 
Time (in seconds) and peak memory consumption (in KB) for 
MPS HIMPS Ratio(%) 
original boost prune total/original...
Preprocessing & Mining 
Time (in seconds) and peak memory consumption (in KB) for 
preprocessing and mining together 
MPS ...
Stability of Results 
Average number of successful runs and average number of 
Subject Runs Tests_total Tests_per(#) Tests...
Discussion 
 Multiple deployments 
 Deploy multiple different instrumented programs to 
different users 
 Multi-functio...
Upcoming SlideShare
Loading in …5
×

HI_slides

284 views

Published on

Slides for the research work: efficient statistical debugging via hierarchical instrumentation

  • Be the first to comment

  • Be the first to like this

HI_slides

  1. 1. Efficient Statistical Debugging via Hierarchical Instrumentation 1 Zhiqiang Zuo Advisor: Siau-Cheng Khoo National University of Singapore
  2. 2. Outline  Introduction  Methodology  Applications  Related Work  Future Research 2
  3. 3. Introduction  Bugs are prevalent  Statistical debugging  Bug isolation (or fault localization)  Bug signature discovery  Elements correlated to failure  Occur frequently in failing runs, but rarely in passing ones 3 Buggy program Failing profiles Passing profiles Statistical debugging Discriminative elements
  4. 4. Introduction  Assumption  Every program element to be potentially relevant to the failure  Consequence  Entire program instrumented  Full-scale instrumentation incurs hefty cost  Observation  Only small portions of code are relevant to a failure  98%-99% predicates monitored are irrelevant 4 • B. Liblit, M. Naik, A.X. Zheng, A. Aiken, and M.I. Jordon, “Scalable Statistical Bug Isolation”, in PLDI, 2005
  5. 5. Introduction P m1 e m2 e e e e m3 e e e mk e e e m4 e e … … 5
  6. 6. Introduction  Full instrumentation  Statistical debugging  Top-k suspicious elements P m1 e m2 e e e e m3 e e e mk e e e m4 e e … … 6
  7. 7. Introduction  Selective instrumentation  Statistical debugging  Same top-k suspicious elements P m1 e m2 e e e e m3 e e e mk e e e m4 e e … … 7
  8. 8. Introduction  Thesis  We can enhance the efficiency of statistical debugging while upholding effectiveness by conducting selective instrumentation  Selective instrumentation  Instrument only highly bug-relevant elements  Prune away unnecessary instrumentation  Challenge 8 How to perform a safe and effective selection?
  9. 9. Outline  Introduction  Methodology  Applications  Related Work  Future Research 9
  10. 10. Methodology  Insight  Hierarchical Instrumentation (HI)  Coarse-grained instrumentation  Lightweight and full-scale  To obtain information of composite constructs (e.g., functions)  Fine-grained instrumentation  Selective instrumentation; prune away unnecessary instrumentation  Guided by the coarse-grained execution information 10 Information collected and measured of composite constructs (e.g., functions) can be used to guide the selection of program elements (e.g., predicates) for subsequent instrumentation.
  11. 11. Hierarchical Instrumentation  A statistical debugging approach  Suspiciousness measure F to quantify the correlation to failures  Top-k suspicious elements returned  Hierarchical Instrumentation  Improve the efficiency by conducting selective instrumentation  Uphold the original effectiveness  i.e., produce the same top-k suspicious elements 11
  12. 12. Hierarchical Instrumentation  Instrumentation  Coarse-grained instrumentation  Fine-grained instrumentation  Coarse-grained measures  Coarse-grained measure for pruning  Coarse-grained measure for ranking 12
  13. 13. Coarse-grained Pruning Measure  Derivation of Cp  Cp is an upper bound of F  Cp is non-decreasing  Cp is as close to F as possible  Necessary Condition  Safe pruning  Given coarse-grained information Cp  Prune away unnecessary elements whose F value must be less than some threshold 13 F(e) ≥ θ  Cp(m) ≥ θ
  14. 14. Coarse-grained Ranking Measure  Issues of threshold θ  Safeness  θ used must be no greater than the top-kth F value  Effectiveness  θ should be as high as possible  Solution  Get the local top-kth F value by instrumenting a small set of fine-grained elements  Highly suspicious elements in the coarse-grained elements of high Cr values  Coarse-grained ranking measure Cr  The Cr value of a coarse-grained element is highly correlated with the F values of enclosing fine-grained elements. 14
  15. 15. Outline  Introduction  Methodology  Applications  Related Work  Future Research 15
  16. 16. Applications  In-house debugging  Run instrumented program in house to obtain executions  Predicated bug signature mining (MPS)  To reduce instrumentation and analysis cost  Cooperative debugging for field failures  Gather execution profiles from end-users running the deployed instrumented program  Cooperative bug isolation (CBI)  To reduce user’s overhead as well • C. Sun and S.C. Khoo, “Mining Succinct Predicated Bug Signatures”, in FSE, 2013 • B. Liblit, M. Naik, A.X. Zheng, A. Aiken, and M.I. Jordon, “Scalable Statistical Bug Isolation”, in PLDI, 2005 16
  17. 17. Efficient Bug Signature Mining via HI  Perfect bug understanding  Bug signature  Comprise multiple elements providing bug context information  Mining Predicated Bug Signatures (MPS)  Predicated bug signature  Suspiciousness measure  Preprocessing & bug signature mining • C. Parnin and A. Orso, “Are automated debugging techniques actually helping programmers?”, in ISSTA, 17 2011 • C. Sun and S.C. Khoo, “Mining Succinct Predicated Bug Signatures”, in FSE, 2013
  18. 18. Efficient Bug Signature Mining via HI  Full-scale instrumentation  High instrumentation and analysis costs  Selective instrumentation via HI  Prune away unnecessary instrumentation of predicates  Safe  Produce same results as original  Effective  Save costs a lot 19
  19. 19. Efficient Bug Signature Mining via HI  Fine-grained measure  Given a database D, a pattern P appears in p passing transactions, and n failing ones, and its discriminative significance is  IG is information gain, defined as 20
  20. 20. Efficient Bug Signature Mining via HI  Coarse-grained pruning measure  Cp = IG(0, n)  Given a predicate e, the function m containing e, we have:  Coarse-grained ranking measure  Cr = DS(p, n)  High correlation between Cr value of a function and average DS value of predicates within it 21 DS(p(e), n(e))≥ θ  IG(0, n(m)) ≥ θ Subject replace space grep sed gzip Overall CC 0.69 0.71 0.63 0.46 0.71 0.64
  21. 21. Approach  Coarse-grained instrumentation function-entry P m1 e m2 e e e e m3 e e e mk e e e m4 e e … … 23
  22. 22. Approach  Coarse-grained instrumentation function-entry  Coarse-grained information Cr: ranking measure Cp: pruning measure P 0.8 0.4 0.55 0.7 0.0 m1 0.9 0.75 0.6 0.85 0.2 e m2 e e e e m3 e e e mk e e e m4 e e … … 24 Cr: Cp:
  23. 23. Approach  Rank functions in descending Cr values P C 0.8 0.4 0.55 0.7 0.0 r: m1 e m2 e e e e m3 e e e mk e e e m4 e e … … 25
  24. 24. Approach  Rank functions in descending Cr values  Select predicates within top functions such that the number of predicates selected reaches percentage γ  θ=0.8 P C 0.8 0.4 0.55 0.7 0.0 r: m1 e m2 e e e e m3 e e e mk e e e m4 e e … … 0.5 0.8 0.7 26 F:
  25. 25. Approach  Safe pruning P  θ=0.8 m1 C 0.9 0.75 0.6 0.85 0.2 p: e m2 e e e e m3 e e e mk e e e m4 e e … … 27
  26. 26. Approach  Safe pruning  θ=0.8  Cp(m1) ≥ θ Cp(m4) ≥ θ P m1 C 0.9 0.75 0.6 0.85 0.2 p: e m2 e e e e m3 e e e mk e e e m4 e e … … 0.5 0.8 0.7 0.8 28 5 0.4 F:
  27. 27. Efficient Bug Signature Mining via HI  Subjects  Top-k: 1  Sampler-cc 29 Subject Versions LoC Function s Tests replace 31 564 21 5,542 space 34 6,199 131 13,585 grep 12 10,068 121 809 sed 16 14,427 163 363 gzip 9 5,680 90 213 • Software-artifact Infrastructure Repository: http://sir.unl.edu/content/sir.php • A sampled predicate-based instrumentor for C in CBI project: http://research.cs.wisc.edu/cbi/
  28. 28. Efficient Bug Signature Mining via HI  Cost comparison  Profile collection 30  Execution time and disk storage for profiles  Preprocessing and mining  Time and peak memory Profile Collection Preprocess & Mining Subject Time (%) Space (%) Time (%) Memory (%) replace 144.16 81.77 90.41 96.12 space 56.39 41.29 49.47 49.43 grep 18.90 44.31 51.40 52.05 sed 61.93 39.11 51.25 52.30 gzip 25.92 23.99 41.63 41.63 Overall 61.46 46.09 56.83 58.31
  29. 29. Iterative Statistical Bug Isolation via HI  Field failures  Failures in deployed software  Cooperative bug isolation  Execution profiles 31  Gathered from end-users running deployed instrumented programs  Predicates: branches, returns and scalar-pairs  Each profile is labeled as failing or passing  Statistical analysis  Suspiciousness measure: Importance  Low overhead  Random sampling • B. Liblit, A. Aiken, A.X. Zheng, and M.I. Jordon, “Bug Isolation via Remote Program Sampling”, in PLDI, 2003
  30. 30. Iterative Statistical Bug Isolation via HI  Sampling  User’s perspective 32  Amortize the monitoring cost to many users  Developers’ perspective  NOT reduce the total cost and size of data  Selective instrumentation via HI  At each iteration, instrument predicates within one function  Low user’s overhead  Prune away unnecessary instrumentation  Low developer’s monitoring and analysis costs  Finally return top-k predictors
  31. 31. Iterative Statistical Bug Isolation via HI  Fine-grained measure  For a predicate e, pt(e) and nt(e) are the number of passing and failing runs where e is observed to true; p(e) and n(e) are the number of passing and failing runs where 33 e is observed, no matter true or false  Importance is as follows:
  32. 32. Iterative Statistical Bug Isolation via HI  Coarse-grained pruning measure  Cp = H(p, n) = 2/(1+ n/p + log2N/log2n)  Given a predicate e, the function m containing e, the following holds under some condition:  Coarse-grained ranking measure  Cr = Cp  Importance has four variables, two of them not meaningful at coarse granularity  Ratio of threshold mass 34 Importance(e)≥ θ  H(p(m), n(m)) ≥ θ
  33. 33. Approach  Coarse-grained instrumentation function-entry P m1 e m2 e e e e m3 e e e mk e e e m4 e e … … 36
  34. 34. Approach  Coarse-grained instrumentation function-entry  Coarse-grained information Cr: ranking measure Cp: pruning measure P 0.8 0.4 0.55 0.7 0.0 m1 0.9 0.75 0.6 0.85 0.2 e m2 e e e e m3 e e e mk e e e m4 e e … … 37 Cr: Cp:
  35. 35. Approach  Rank functions in descending Cr values  At each iteration, instrument predicates within one function: Cr is highest Cp ≥ θ (0) P C 0.8 0.4 0.55 0.7 0.0 r: m1 C 0.9 0.75 0.6 0.85 0.2 p: e m2 e e e e m3 e e e mk e e e m4 e e … … 38
  36. 36. Approach  Rank functions in descending Cr values  At each iteration, instrument predicates within one function: Cr is highest Cp ≥ θ (0)  θ=0.8 P C 0.8 0.4 0.55 0.7 0.0 r: m1 C 0.9 0.75 0.6 0.85 0.2 p: e m2 e e e e m3 e e e mk e e e m4 e e … … 0.5 0.8 0.7 39 F:
  37. 37. Approach  Rank functions in descending Cr values  At each iteration, instrument predicates within one function: Cr is highest Cp ≥ θ (0.8) P C 0.8 0.4 0.55 0.7 0.0 r: m1 C 0.9 0.75 0.6 0.85 0.2 p: e m2 e e e e m3 e e e mk e e e m4 e e … … 0.5 0.8 0.7 40 F:
  38. 38. Approach P C 0.8 0.4 0.55 0.7 0.0 r: m1 C 0.9 0.75 0.6 0.85 0.2 p: e m2 e e e e m3 e e e mk e e e m4 e e … … 0.5 0.8 0.7 41 0.8 5 0.4  Rank functions in descending Cr values  At each iteration, instrument predicates within one function: Cr is highest Cp ≥ θ (0.8)  θ=0.85 F:
  39. 39. Approach P C 0.8 0.4 0.55 0.7 0.0 r: m1 C 0.9 0.75 0.6 0.85 0.2 p: e m2 e e e e m3 e e e mk e e e m4 e e … … 0.5 0.8 0.7 42 0.8 5 0.4  Rank functions in descending Cr values  At each iteration, instrument predicates within one function: Cr is highest Cp ≥ θ (0.85) F:
  40. 40. Iterative Statistical Bug Isolation via HI  Subjects  Top-k: 1  Sampler-cc 44 Subject Versions LoC Function s Tests replace 31 564 21 5,542 space 34 6,199 131 13,585 grep 12 10,068 121 809 sed 16 14,427 163 363 gzip 9 5,680 90 213 • Software-artifact Infrastructure Repository: http://sir.unl.edu/content/sir.php • A sampled predicate-based instrumentor for C in CBI project: http://research.cs.wisc.edu/cbi/
  41. 41. Iterative Statistical Bug Isolation via HI  Instrumentation effort  Percentage of predicates instrumented  Baseline approach  Non-iterative, non-sampled 45 statistical bug isolation  Top-k predictors  Our iterative approach  Same top-k predictors  Test cases  Random choose a subset of tests at each iteration 100 90 80 70 60 50 40 30 20 10 0 • B. Liblit, M. Naik, A.X. Zheng, A. Aiken, and M.I. Jordon, “Scalable Statistical Bug Isolation”, in PLDI, 2005
  42. 42. Iterative Statistical Bug Isolation via HI  User’s overhead comparison  Our (non-sampling) iterative approach  Sampling schemes with different sampling rates 46 Sampling HI Subject 1/1 1/100 1/10000 coarse fine replace 0.836 0.799 0.788 0.486 0.630 space 2.172 2.284 2.296 0.396 0.459 grep 21.960 13.042 9.248 0.211 0.389 sed 4.946 3.301 2.824 0.229 0.303 gzip 4.870 2.427 1.793 0.050 0.075 Overall 6.957 4.371 3.390 0.274 0.371 • B. Liblit, A. Aiken, A.X. Zheng, and M.I. Jordon, “Bug Isolation via Remote Program Sampling”, in PLDI, 2003
  43. 43. Outline  Introduction  Methodology  Applications  Related Work  Future Research 47
  44. 44. Related Work  Selective instrumentation  HOLMES [ICSE’09] and Adaptive bug isolation [ICSE’10]  Partial code instrumented at each iteration  Principle of locality  Hierarchical mining  Hierarchical call graphs mining for defect localization [SDM’11]  No saving in instrumentation  No pruning based on rigorous mathematical relation  No guarantee for the same results 48
  45. 45. Related Work  Statistical bug isolation  Entity & Suspiciousness measure  Tarantula [ASE’05], Ochiai [ASE’09], CBI [PLDI’05], SOBER [FSE’05]  Bug signature identification  Contextual information, bug signature  RAPID [ASE’08], Cheng et al. [ISSTA’09], Sun and Khoo [FSE’13]  Jiang and Su [ASE’07]  Other automated debugging  Program slicing, delta debugging 49
  46. 46. Outline  Introduction  Methodology  Applications  Related Work  Future Research 50
  47. 47. Future Research  Derive coarse-grained measures automatically  Pruning measure  Ranking measure  Improve debugging effectiveness  Tradeoff between efficiency and effectiveness  Abstraction of execution information  E.g., ignoring exact execution times, temporal order, dependency  Collect more detailed execution information 51
  48. 48. Thank you! questions, comments, advice? 53
  49. 49. 54
  50. 50. Coarse-grained Pruning Measure  Given F: N2–›R defined over the intervals ([0, X], [0, Y]), Cp under the same domain should satisfy: • Cp is an upper bound of F, i.e., Cp(x, y)≥F(x, y) • Cp is nondecreasing, i.e., Cp(x, y)≥Cp(x-1, y) and Cp(x, y)≥Cp(x, y-1) • Cp is as close to F as possible, i.e., Σ{Cp(x, y)-F(x, y)} is minimal 55
  51. 51. Necessary Condition  Let e be a fine-grained element, m be the composite construct containing e. p(e) (or p(m)) and n(e) (or n(m)) denote the number of passing and failing runs where e (or m) is executed. Given a θ: F(p(e), n(e)) ≥ θ  Cp(p(m), n(m)) ≥ θ 56
  52. 52. Coarse-grained Ranking Measure  Cr value of a coarse-grained element should be highly correlated with the F value of enclosing fine-grained elements.  A high θ can be acquired by instrumenting fine-grained elements in the coarse-grained element of high Cr value  Derivation  F as Cr at the coarse granularity  Cp as Cr 57
  53. 53. Profile Collection Execution time (in seconds) during profile collection MPS HIMPS Ratio(%) Subject original coarse boost prune total total/original replace 12,332 841 6,829 10,108 17,777 144.16 space 293,760 11,765 29,219 124,670 165,655 56.39 grep 148,803 934 8,638 18,557 28,129 18.90 sed 68,474 549 3,690 38,164 42,403 61.93 gzip 663,789 1,945 112,151 57,980 172,076 25.92 Overall 237,431 3,207 32,105 49,896 85,208 61.46 58
  54. 54. Profile Collection Disk storage space used (in KB) for profiles MPS HIMPS Ratio(%) Subject original coarse boost prune total total/original replace 125,883 116 13,686 89,132 102,935 81.77 space 6,242,337 1,950 401,59 1 2,174,017 2,577,558 41.29 grep 1,145,575 170 116,00 8 391,439 507,617 44.31 sed 864,367 125 49,515 288,398 338,038 39.11 gzip 821,421 29 50,932 146,060 197,020 23.99 Overall 1,839,916 478 126,34 6 617,809 744,634 46.09 59
  55. 55. Preprocessing Time (in seconds) and peak memory consumption (in KB) for MPS HIMPS Ratio(%) original boost prune total/original Subjec t Time Memory Time Memory Time Memory Time Memory replace 16.99 180,876 1.18 33,955 13.70 156,969 87.5 9 86.78 space 1,162.5 7 4,642,48 5 34.9 8 399,72 7 271.2 5 1,978,00 3 26.3 4 42.61 grep 128.63 895,591 9.33 164,82 1 41.64 466,192 39.6 2 52.05 sed 85.98 733,413 4.51 134,32 8 31.41 383,566 41.7 8 52.30 gzip 87.39 786,913 4.52 177,14 2 18.55 327,594 26.4 0 41.63 preprocessing 60
  56. 56. Mining Time (in seconds) and peak memory consumption (in KB) for MPS HIMPS Ratio(%) original boost prune total/original Subjec t Time Memory Time Memory Time Memory Time Memory replac e 28.90 240,974 0.08 16,277 26.50 231,617 91.9 6 96.12 space 813.93 2,533,41 4 1.15 62,458 670.0 8 2,294,89 4 82.4 7 90.59 grep 300.15 321,836 1.82 23,908 166.8 9 245,349 56.2 1 76.23 sed 24.39 65,952 0.05 5,539 19.38 59,425 79.6 8 90.10 gzip 56.64 70,780 0.12 6,947 34.91 49,171 61.8 5 69.47 mining 61
  57. 57. Preprocessing & Mining Time (in seconds) and peak memory consumption (in KB) for preprocessing and mining together MPS HIMPS Ratio(%) original coarse boost & prune total/original Subjec t Time Memory Time Memory Time Memory Time Memory replace 45.89 240,974 0.03 3,161 41.45 231,617 90.4 1 96.12 space 1,976.5 1 4,642,48 5 0.40 49,463 977.4 7 2,294,89 4 49.4 7 49.43 grep 428.78 895,591 0.73 47,906 219.6 9 466,192 51.4 0 52.05 sed 110.37 733,413 1.21 75,317 55.35 383,566 51.2 5 52.30 gzip 144.03 786,913 1.85 114,62 2 58.10 327,594 41.6 3 41.63 62
  58. 58. Stability of Results Average number of successful runs and average number of Subject Runs Tests_total Tests_per(#) Tests_per(%) replace 84.81 5,542 2,993 0.54 space 81.79 13,585 8,559 0.63 grep 81.83 809 542 0.67 sed 83.56 363 232 0.64 gzip 66.89 213 185 0.87 63 tests per iteration
  59. 59. Discussion  Multiple deployments  Deploy multiple different instrumented programs to different users  Multi-function instrumentation  Predicates in multiple functions are instrumented in one instrumented program  Multiple hierarchies  Package-level, class-level, function-level, block-level, statement-level, predicate-level 64

×