The document describes a technique called Falcon for fault localization in concurrent programs. Falcon uses patterns of variable accesses and thread interactions to identify suspicious patterns. It records patterns dynamically from multiple program executions. It then computes the suspiciousness of each pattern based on whether the pattern appears in passing or failing runs. The most suspicious patterns that appear frequently in failing runs but not passing runs help locate likely faults. The technique is evaluated on concurrency violations like order violations and atomicity violations.

1. Background Technique Studies Conclusion
Falcon: Fault Localization in
Concurrent Programs
Sangmin Park, Richard Vuduc,
and Mary Jean Harrold
College of Computing
Georgia Institute of Technology
1

2. Background Technique Studies Conclusion
Motivation
• Godefroid, Nagappan Developers
[MSR-TR08]
found that
Handle
concurrency
issues
• Concurrency bugs are difficult to find
• Non-deterministic behavior
• Large thread interleaving space
2

3. Background Technique Studies Conclusion
Existing Techniques
• Bug finding for concurrency bugs
• Static/dynamic data race detection
(e.g., [Ronesse99, Savage97, O’Callanhan03])
1. Techniques target only one type of
concurrency bug
• Static/dynamic atomicity violation detection
2. Techniques report benign (harmless)
(e.g., [Flanagan03, Wang06, Flanagan08])
results along with harmful results
• Dynamic pattern analysis
(e.g., [Lu06, Hammer08])
• Fault localization
• For sequential programs
(e.g., [Jones02, Liblit03, been applied to
Techniques have not Liu05, Baah08])
concurrent programs
3

4. Background Technique Studies Conclusion
Falcon Research Outline
• Background
• Technique: patterns + suspiciousness
• Empirical Studies
• Conclusion
4

5. Background Technique Studies Conclusion
Concurrency Violations
Lu et al.’s [ASPLOS08] found that
Others: 3%
Order and
Atomicity Deadlock: 30%
violations:
67%
Both occur due to undesirable
undesirable
memory-access sequences
memory-access sequences
5

6. Background Technique Studies Conclusion
Order Violation
When desired order of accesses is reversed
 Unintended program behavior
For A access
Thread T1 Thread T2
Correct order
11: void M1() 21: void M2()
W14  R24
12: { 22: {
Incorrect order
13: lock(L); 23: lock(L);
R24  W14
14: A=new Object(); 24: x=A.getX();
15: if (A) { /*…*/ } 25: unlock(L); RT2  WT1
16: unlock(L); 26: }
17: }
6

7. Background Technique Studies Conclusion
Atomicity Violation
When atomic region is interfered by another thread
 Unintended program behavior
Thread T3 For o access
31: copy(Obj o) Thread T4
Correct order
32: { 41: access()
R34  R36  W44
33: … 42: {
Incorrect order
34: int sz=o.sz(); 43: …
R34  W44  R36
35: … 44: o.clear();
45: … RT3  WT4  RT3
36: copy(sz, o,
this.buffer); 46: }
37: …
38: }
7

8. Background Technique Studies Conclusion
Patterns for Concurrency Violations
Patterns Patterns for atomicity violation
for order violation [PPoPP06, ASPLOS06]
1 RT1 * WT2 1 RT1 * WT2 * RT1
2 WT1 * RT2 2 WT1 * WT2 * RT1
3 WT1 * WT2 3 WT1 * RT2 * WT1
4 RT1 * WT2 * WT1
5 WT1 * WT2 * WT1
Note
• Concurrency violations represented as patterns
• Patterns not always concurrency violations
8

9. Background Technique Studies Conclusion
Technique For Concurrency—Falcon
Phase 1. Dynamic pattern recording
• Input: a set of executions
• Output: patterns, outcome (pass/fail)
Phase 2. Statistical analysis
• Compute suspiciousness of each pattern
Intuition
Patterns appearing in failing executions are
more suspicious than patterns appearing
in passing executions
9

10. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Initially x=0; y=0; x=0 Phase 1. Dynamic pattern
y=0 recording
Thread 1
11:if(x==0) x=1; x=1 • Monitor accesses per variable
12:if(y==0) y=1; y=1 • Monitor accesses per thread
13:if(x==2&&y==2) x=2 • Create window per variable
assert(false); y=2
• Detect patterns inside window
Thread 2
21:if(x==1) x=2; x=2 Statements: 11-21-31-12-22-32-13
22:if(y==1) y=2; y=2
x-Access : R11-W11-R21-W21-R31-R13
Thread 3 y-Access : R12-W12-R22-W22-R32-R13
31:if(x==1) x=3; x=2 x-Access : W11-W21-R31-R13
31-
32:if(y==1) y=3; y=2
y-Access : W12-W22-R32-R13
10

11. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Phase 2. Statistical analysis
Initially x=0; y=0; • Use patterns and outcome
• Compute suspiciousness
Thread 1
11:if(x==0) x=1; Run 1 (r1):
12:if(y==0) y=1; x-access:
13:if(x==2&&y==2)
assert(false); y-access:
r1 r2 r3 r4 Suspicious-
Thread 2 ness
21:if(x==1) x=2; Pattern 1:
22:if(y==1) y=2; W11-W31-R13
Pattern 2:
Thread 3 W11-W21-R13
31:if(x==1) x=3; Pattern 3:
32:if(y==1) y=3; W12-W32-R13
Pattern 4:
W12-W22-R13
11

12. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Phase 2. Statistical analysis
Initially x=0; y=0; • Use patterns and outcome
• Compute suspiciousness
Thread 1
11:if(x==0) x=1; Run 1 (r1): 11 31 21 12 32 22 13
12:if(y==0) y=1; x-access: W11 W31 R21 R13
13:if(x==2&&y==2)
y-access: W12 W32 R22 R13
assert(false);
r1 r2 r3 r4 Suspicious-
Thread 2 ness
21:if(x==1) x=2; Pattern 1: 
22:if(y==1) y=2; W11-W31-R13
Pattern 2:
Thread 3 W11-W21-R13
31:if(x==1) x=3; Pattern 3: 
32:if(y==1) y=3; W12-W32-R13
Pattern 4:
W12-W22-R13
P
12

13. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Phase 2. Statistical analysis
Initially x=0; y=0; • Use patterns and outcome
• Compute suspiciousness
Thread 1
11:if(x==0) x=1; Run 2 (r2): 11 31 21 12 22 32 13
12:if(y==0) y=1; x-access: W11 W31 R21 R13
13:if(x==2&&y==2)
y-access: W12 W22 R32 R13
assert(false);
r1 r2 r3 r4 Suspicious-
Thread 2 ness
21:if(x==1) x=2; Pattern 1:  
22:if(y==1) y=2; W11-W31-R13
Pattern 2:
Thread 3 W11-W21-R13
31:if(x==1) x=3; Pattern 3: 
32:if(y==1) y=3; W12-W32-R13
Pattern 4: 
W12-W22-R13
P P
13

14. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Phase 2. Statistical analysis
Initially x=0; y=0; • Use patterns and outcome
• Compute suspiciousness
Thread 1
11:if(x==0) x=1; Run 3 (r3): 11 21 31 12 32 22 13
12:if(y==0) y=1; x-access: W11 W21 R31 R13
13:if(x==2&&y==2)
y-access: W12 W32 R22 R13
assert(false);
r1 r2 r3 r4 Suspicious-
Thread 2 ness
21:if(x==1) x=2; Pattern 1:  
22:if(y==1) y=2; W11-W31-R13
Pattern 2: 
Thread 3 W11-W21-R13
31:if(x==1) x=3; Pattern 3:  
32:if(y==1) y=3; W12-W32-R13
Pattern 4: 
W12-W22-R13
P P P
14

15. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Phase 2. Statistical analysis
Initially x=0; y=0; • Use patterns and outcome
• Compute suspiciousness
Thread 1
11:if(x==0) x=1; Run 4 (r4): 11 21 31 12 22 32 13
12:if(y==0) y=1; x-access: W11 W21 R31 R13
13:if(x==2&&y==2)
y-access: W12 W22 R32 R13
assert(false);
r1 r2 r3 r4 Suspicious-
Thread 2 ness
21:if(x==1) x=2; Pattern 1:  
22:if(y==1) y=2; W11-W31-R13
Pattern 2:  
Thread 3 W11-W21-R13
31:if(x==1) x=3; Pattern 3:  
32:if(y==1) y=3; W12-W32-R13
Pattern 4:  
W12-W22-R13
P P P F
15

16. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Initially x=0; y=0; Jaccard formula [Abreu07]
failed (P)
Thread 1 suspiciousness(P) =
11:if(x==0) x=1; totalfailed + passed(P)
12:if(y==0) y=1;
13:if(x==2&&y==2) r1 r2 r3 r4 Suspicious-
ness
assert(false);
Pattern 1:   0.0
W11-W31-R13
Thread 2
Pattern 2:   0.5
21:if(x==1) x=2;
W11-W21-R13
22:if(y==1) y=2;
Pattern 3:   0.0
Thread 3 W12-W32-R13
31:if(x==1) x=3; Pattern 4:   0.5
32:if(y==1) y=3; W12-W22-R13
P P P F
16

17. Background Technique Studies Conclusion
Empirical Studies
• Studies
1. Determine window size needed and
its relationship to number of threads
2. Evaluate technique’s effectiveness
3. Compare technique with Cooperative Crug
Isolation (CCI) [Thakur et al.,WODA09]
4. Evaluate technique’s efficiency (see paper)
• Empirical setup
• Implemented in Java
• Evaluated on a set of subjects
17

18. Background Technique Studies Conclusion
Empirical Setup
Subjects Program LOC Failure Rate (%) Violation Type
Account 115 3 Atomicity
AirlineTickets 95 54 Atomicity
BubbleSort2 130 69 Atomicity
Contest BufWriter 255 14 Atomicity
Benchmark
Lottery 359 43 Atomicity
MergeSort 375 84 Atomicity
Shop 273 2 Atomicity
ArrayList 5866 2 Atomicity
HashSet 7086 3 Atomicity
Java
StringBuffer 1320 3 Atomicity
Collection
TreeSet 7532 3 Atomicity
Vector 709 2 Atomicity
Cache4j 3897 3 Order
RayTracer 1924 14 Atomicity
18

19. Background Technique Studies Conclusion
Empirical Study 1 – Window Size
Goals
To determine
• Window size to detect patterns for atomicity violations
• Correlation of window size and number of threads
Method
• For test case with 100 runs and subjects running with
p threads, where p={4,8,16,32,64}
• recorded patterns during execution
• computed window size needed to detect each
pattern
• Aggregated data from all subjects
19

20. Background Technique Studies Conclusion
Result – Window Size
60 • Median window size 3
Window Size to Detect Pattern
50
40
• Most patterns with
window size < 10
30
20
• Window size and
10 number of threads not
positively correlated
0
4 8 16 32 64
Number of Threads
20

21. Background Technique Studies Conclusion
Empirical Study 2 – Effectiveness
Goal
• To measure effectiveness of suspiciousness ranking
Techniques compared
• Conflict Serializability (e..g, [Wang and Stoller, PPoPP06])
• AVIO [Lu et al., ASPLOS06]
Method
• For test case with 100 runs and window size 5
• Computed suspiciousness of each pattern
• Computed fault-localization expense (number of
patterns that must be examined to find fault)
21

22. Background Technique Studies Conclusion
Result - Effectiveness
Conflict-Serializability AVIO Falcon
Subjects
Patterns (Ranking) Patterns (Ranking) Patterns (Ranking)
Account 11 (-) 7 (-) 11 (2)
AirlineTickets 4 (-) 0 (-) 4 (1)
BubbleSort2 4 (-) 1 (-) 4 (1)
BufWriter 105 (-) 25 (-) 105 (1)
BufWriter
Lottery 105
4 (-) 4 (-) 25 1
4 (1)
MergeSort 76 (-) 42 (-) 76 (1)
Shop 10 (-) 0 (-) 10 (2)
ArrayList 1 (-) 1 (-) 1 (1)
HashSet 7 (-) 3 (-) 7 (1)
StringBuffer 2 (-) 1 (-) 2 (1)
TreeSet 9 (-) 3 (-) 9 (2)
Vector 1 (-) 1 (-) 1 (1)
Cache4j 23 (1)
RayTracer 14 (-) 14 (-) 14 (2)
22
* (-) : Patterns are not ranked

23. Background Technique Studies Conclusion
Empirical Study 3 – Case Study
Goal
• To compare effectiveness of Falcon to CCI*
• CCI observes memory accesses, outcome (pass/fail)
• CCI reports suspiciousness of each memory access predicate
(i.e., memory access and previous access type pair)
Method
• Implemented the CCI technique in the Falcon toolset
• For test case with 100 runs and subjects
• Executed the tool
• Found the access and rank of actual bug
* Cooperative Crug Isolation [Thakur, Sen, Liblit, Lu, WODA09]
23

24. Background Technique Studies Conclusion
Result – Case Study ranking
CCI ranking Falcon
1. RS1CCI 1. WS1-WS3-RS2
RS2-WS3-RS1
Falcon
Subjects
…
Accesses of Bug (Rank) 2. RS6-WS2-RS6
Pattern of Bug (Rank)
Account …R W W
12. (6),S2 (4), R (6) 3. WS4-RS6-WS2
RS4R-W-R (2)
-W
AirlineTickets …R …
13. (2), W (5), R (4) 4. RS1R-W-R-WS6
-WS4 (1)
WS1-WS3-WS6
BubbleSort2
14. RS2R (7), W (2)
W (1),
15. … RS6W-R-W-RS6
-WS3 (1)
5. WS6-WS4-RS6
BufWriter CCI ranking
R (5), W (1), R (5) Falcon ranking
6. RS2R-W-R-WS6
(1)
-WS4-RS3
S3
15. WS6
R S3
Lottery 1. R (1), W (7), R (4)
… RS1
1. WS1-WS3-RS2
R-W-R (1)
7. … -WS4-WS6
RS1
MergeSort 2.W (1), R (3), W (12)
RS2 2. RS6W-R-W-RS6
-WS4 (1)
Shop 3. (15),S3 (12), R (15)
R WW R-W-R (2)
3. WS4-WS6-RS2
ArrayList
4. R (4),S4 (4), R (4)
W W 4. RS1R-W-R-RS6
-WS4 (1)
HashSet R (15), W (15), R (1) R-W-R (1)
5. RS5 5. WS6-WS4-WS6
StringBuffer R (14), W (2), R (1) R-W-R (1)
6. RS6 6. RS2-WS4-WS6
TreeSet R (10), W (5), R (10) R-W-R (2)
• CCI reports the R (2), W (2), R (2)
Vector
… 7. …
7. actual bug with lower ranks R-W-R (1)
• CCI does not provide an(2)
Cache4j W (79), R
association of buggy (1)
W-R
accesses
RayTracer R (4), W (4), W (3) R-W-W (2)
24

25. Background Technique Studies Conclusion
Future Work
• Perform more empirical studies
• With larger subjects
• With more subjects especially with order violations
• With considering multiple test cases
• Apply to other languages and systems
• Other than Java language
• Extend the technique to support
• Handling multi-variable related patterns for
atomicity violations
• Localizing deadlocks
25

26. Background Technique Studies Conclusion
Contributions
• Falcon is the first technique
• that reports and ranks patterns by
suspiciousness
• that addresses order violations and
atomicity violations with patterns
• Studies show that
• Falcon’s window size
• Small constant
• Not correlated with number of threads
• Falcon’s suspiciousness ranking technique
• Effective in identifying faulty interleavings
• More effective than ranking individual accesses
Questions? 26

27. Background Technique Studies Conclusion
Backup Slides
27

28. Background Technique Studies Conclusion
Result – Efficiency
Conflict- AVIO Atomic-Set- Falcon
Serializability (ASPLOS 06) Serializability (ICSE10)
(PPoPP 06) (ICSE 08)
Efficiency 38X 25X 13X 9.9X
(Average (Median (Used almost
slowdown) slowdown) same set of
subjects as
Falcon)
28