1. Background Technique Studies Conclusion
Falcon: Fault Localization in
Concurrent Programs
Sangmin Park, Richard Vuduc,
and Mary Jean Harrold
College of Computing
Georgia Institute of Technology
1

2. Background Technique Studies Conclusion
Motivation
• Godefroid, Nagappan Developers
[MSR-TR08]
found that
Handle
concurrency
issues
• Concurrency bugs are difficult to find
• Non-deterministic behavior
• Large thread interleaving space
2

3. Background Technique Studies Conclusion
Existing Techniques
• Bug finding for concurrency bugs
• Static/dynamic data race detection
(e.g., [Ronesse99, Savage97, O’Callanhan03])
1. Techniques target only one type of
concurrency bug
• Static/dynamic atomicity violation detection
2. Techniques report benign (harmless)
(e.g., [Flanagan03, Wang06, Flanagan08])
results along with harmful results
• Dynamic pattern analysis
(e.g., [Lu06, Hammer08])
• Fault localization
• For sequential programs
(e.g., [Jones02, Liblit03, been applied to
Techniques have not Liu05, Baah08])
concurrent programs
3

4. Background Technique Studies Conclusion
Falcon Research Outline
• Background
• Technique: patterns + suspiciousness
• Empirical Studies
• Conclusion
4

5. Background Technique Studies Conclusion
Concurrency Violations
Lu et al.’s [ASPLOS08] found that
Others: 3%
Order and
Atomicity Deadlock: 30%
violations:
67%
Both occur due to undesirable
undesirable
memory-access sequences
memory-access sequences
5

6. Background Technique Studies Conclusion
Order Violation
When desired order of accesses is reversed
 Unintended program behavior
For A access
Thread T1 Thread T2
Correct order
11: void M1() 21: void M2()
W14  R24
12: { 22: {
Incorrect order
13: lock(L); 23: lock(L);
R24  W14
14: A=new Object(); 24: x=A.getX();
15: if (A) { /*…*/ } 25: unlock(L); RT2  WT1
16: unlock(L); 26: }
17: }
6

7. Background Technique Studies Conclusion
Atomicity Violation
When atomic region is interfered by another thread
 Unintended program behavior
Thread T3 For o access
31: copy(Obj o) Thread T4
Correct order
32: { 41: access()
R34  R36  W44
33: … 42: {
Incorrect order
34: int sz=o.sz(); 43: …
R34  W44  R36
35: … 44: o.clear();
45: … RT3  WT4  RT3
36: copy(sz, o,
this.buffer); 46: }
37: …
38: }
7

8. Background Technique Studies Conclusion
Patterns for Concurrency Violations
Patterns Patterns for atomicity violation
for order violation [PPoPP06, ASPLOS06]
1 RT1 * WT2 1 RT1 * WT2 * RT1
2 WT1 * RT2 2 WT1 * WT2 * RT1
3 WT1 * WT2 3 WT1 * RT2 * WT1
4 RT1 * WT2 * WT1
5 WT1 * WT2 * WT1
Note
• Concurrency violations represented as patterns
• Patterns not always concurrency violations
8

9. Background Technique Studies Conclusion
Technique For Concurrency—Falcon
Phase 1. Dynamic pattern recording
• Input: a set of executions
• Output: patterns, outcome (pass/fail)
Phase 2. Statistical analysis
• Compute suspiciousness of each pattern
Intuition
Patterns appearing in failing executions are
more suspicious than patterns appearing
in passing executions
9

10. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Initially x=0; y=0; x=0 Phase 1. Dynamic pattern
y=0 recording
Thread 1
11:if(x==0) x=1; x=1 • Monitor accesses per variable
12:if(y==0) y=1; y=1 • Monitor accesses per thread
13:if(x==2&&y==2) x=2 • Create window per variable
assert(false); y=2
• Detect patterns inside window
Thread 2
21:if(x==1) x=2; x=2 Statements: 11-21-31-12-22-32-13
22:if(y==1) y=2; y=2
x-Access : R11-W11-R21-W21-R31-R13
Thread 3 y-Access : R12-W12-R22-W22-R32-R13
31:if(x==1) x=3; x=2 x-Access : W11-W21-R31-R13
31-
32:if(y==1) y=3; y=2
y-Access : W12-W22-R32-R13
10

11. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Phase 2. Statistical analysis
Initially x=0; y=0; • Use patterns and outcome
• Compute suspiciousness
Thread 1
11:if(x==0) x=1; Run 1 (r1):
12:if(y==0) y=1; x-access:
13:if(x==2&&y==2)
assert(false); y-access:
r1 r2 r3 r4 Suspicious-
Thread 2 ness
21:if(x==1) x=2; Pattern 1:
22:if(y==1) y=2; W11-W31-R13
Pattern 2:
Thread 3 W11-W21-R13
31:if(x==1) x=3; Pattern 3:
32:if(y==1) y=3; W12-W32-R13
Pattern 4:
W12-W22-R13
11

12. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Phase 2. Statistical analysis
Initially x=0; y=0; • Use patterns and outcome
• Compute suspiciousness
Thread 1
11:if(x==0) x=1; Run 1 (r1): 11 31 21 12 32 22 13
12:if(y==0) y=1; x-access: W11 W31 R21 R13
13:if(x==2&&y==2)
y-access: W12 W32 R22 R13
assert(false);
r1 r2 r3 r4 Suspicious-
Thread 2 ness
21:if(x==1) x=2; Pattern 1: 
22:if(y==1) y=2; W11-W31-R13
Pattern 2:
Thread 3 W11-W21-R13
31:if(x==1) x=3; Pattern 3: 
32:if(y==1) y=3; W12-W32-R13
Pattern 4:
W12-W22-R13
P
12

13. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Phase 2. Statistical analysis
Initially x=0; y=0; • Use patterns and outcome
• Compute suspiciousness
Thread 1
11:if(x==0) x=1; Run 2 (r2): 11 31 21 12 22 32 13
12:if(y==0) y=1; x-access: W11 W31 R21 R13
13:if(x==2&&y==2)
y-access: W12 W22 R32 R13
assert(false);
r1 r2 r3 r4 Suspicious-
Thread 2 ness
21:if(x==1) x=2; Pattern 1:  
22:if(y==1) y=2; W11-W31-R13
Pattern 2:
Thread 3 W11-W21-R13
31:if(x==1) x=3; Pattern 3: 
32:if(y==1) y=3; W12-W32-R13
Pattern 4: 
W12-W22-R13
P P
13

14. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Phase 2. Statistical analysis
Initially x=0; y=0; • Use patterns and outcome
• Compute suspiciousness
Thread 1
11:if(x==0) x=1; Run 3 (r3): 11 21 31 12 32 22 13
12:if(y==0) y=1; x-access: W11 W21 R31 R13
13:if(x==2&&y==2)
y-access: W12 W32 R22 R13
assert(false);
r1 r2 r3 r4 Suspicious-
Thread 2 ness
21:if(x==1) x=2; Pattern 1:  
22:if(y==1) y=2; W11-W31-R13
Pattern 2: 
Thread 3 W11-W21-R13
31:if(x==1) x=3; Pattern 3:  
32:if(y==1) y=3; W12-W32-R13
Pattern 4: 
W12-W22-R13
P P P
14

15. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Phase 2. Statistical analysis
Initially x=0; y=0; • Use patterns and outcome
• Compute suspiciousness
Thread 1
11:if(x==0) x=1; Run 4 (r4): 11 21 31 12 22 32 13
12:if(y==0) y=1; x-access: W11 W21 R31 R13
13:if(x==2&&y==2)
y-access: W12 W22 R32 R13
assert(false);
r1 r2 r3 r4 Suspicious-
Thread 2 ness
21:if(x==1) x=2; Pattern 1:  
22:if(y==1) y=2; W11-W31-R13
Pattern 2:  
Thread 3 W11-W21-R13
31:if(x==1) x=3; Pattern 3:  
32:if(y==1) y=3; W12-W32-R13
Pattern 4:  
W12-W22-R13
P P P F
15

16. Background Technique Studies Conclusion
Falcon Example on Atomicity Violation
Initially x=0; y=0; Jaccard formula [Abreu07]
failed (P)
Thread 1 suspiciousness(P) =
11:if(x==0) x=1; totalfailed + passed(P)
12:if(y==0) y=1;
13:if(x==2&&y==2) r1 r2 r3 r4 Suspicious-
ness
assert(false);
Pattern 1:   0.0
W11-W31-R13
Thread 2
Pattern 2:   0.5
21:if(x==1) x=2;
W11-W21-R13
22:if(y==1) y=2;
Pattern 3:   0.0
Thread 3 W12-W32-R13
31:if(x==1) x=3; Pattern 4:   0.5
32:if(y==1) y=3; W12-W22-R13
P P P F
16

17. Background Technique Studies Conclusion
Empirical Studies
• Studies
1. Determine window size needed and
its relationship to number of threads
2. Evaluate technique’s effectiveness
3. Compare technique with Cooperative Crug
Isolation (CCI) [Thakur et al.,WODA09]
4. Evaluate technique’s efficiency (see paper)
• Empirical setup
• Implemented in Java
• Evaluated on a set of subjects
17

18. Background Technique Studies Conclusion
Empirical Setup
Subjects Program LOC Failure Rate (%) Violation Type
Account 115 3 Atomicity
AirlineTickets 95 54 Atomicity
BubbleSort2 130 69 Atomicity
Contest BufWriter 255 14 Atomicity
Benchmark
Lottery 359 43 Atomicity
MergeSort 375 84 Atomicity
Shop 273 2 Atomicity
ArrayList 5866 2 Atomicity
HashSet 7086 3 Atomicity
Java
StringBuffer 1320 3 Atomicity
Collection
TreeSet 7532 3 Atomicity
Vector 709 2 Atomicity
Cache4j 3897 3 Order
RayTracer 1924 14 Atomicity
18

19. Background Technique Studies Conclusion
Empirical Study 1 – Window Size
Goals
To determine
• Window size to detect patterns for atomicity violations
• Correlation of window size and number of threads
Method
• For test case with 100 runs and subjects running with
p threads, where p={4,8,16,32,64}
• recorded patterns during execution
• computed window size needed to detect each
pattern
• Aggregated data from all subjects
19

20. Background Technique Studies Conclusion
Result – Window Size
60 • Median window size 3
Window Size to Detect Pattern
50
40
• Most patterns with
window size < 10
30
20
• Window size and
10 number of threads not
positively correlated
0
4 8 16 32 64
Number of Threads
20

21. Background Technique Studies Conclusion
Empirical Study 2 – Effectiveness
Goal
• To measure effectiveness of suspiciousness ranking
Techniques compared
• Conflict Serializability (e..g, [Wang and Stoller, PPoPP06])
• AVIO [Lu et al., ASPLOS06]
Method
• For test case with 100 runs and window size 5
• Computed suspiciousness of each pattern
• Computed fault-localization expense (number of
patterns that must be examined to find fault)
21

22. Background Technique Studies Conclusion
Result - Effectiveness
Conflict-Serializability AVIO Falcon
Subjects
Patterns (Ranking) Patterns (Ranking) Patterns (Ranking)
Account 11 (-) 7 (-) 11 (2)
AirlineTickets 4 (-) 0 (-) 4 (1)
BubbleSort2 4 (-) 1 (-) 4 (1)
BufWriter 105 (-) 25 (-) 105 (1)
BufWriter
Lottery 105
4 (-) 4 (-) 25 1
4 (1)
MergeSort 76 (-) 42 (-) 76 (1)
Shop 10 (-) 0 (-) 10 (2)
ArrayList 1 (-) 1 (-) 1 (1)
HashSet 7 (-) 3 (-) 7 (1)
StringBuffer 2 (-) 1 (-) 2 (1)
TreeSet 9 (-) 3 (-) 9 (2)
Vector 1 (-) 1 (-) 1 (1)
Cache4j 23 (1)
RayTracer 14 (-) 14 (-) 14 (2)
22
* (-) : Patterns are not ranked

23. Background Technique Studies Conclusion
Empirical Study 3 – Case Study
Goal
• To compare effectiveness of Falcon to CCI*
• CCI observes memory accesses, outcome (pass/fail)
• CCI reports suspiciousness of each memory access predicate
(i.e., memory access and previous access type pair)
Method
• Implemented the CCI technique in the Falcon toolset
• For test case with 100 runs and subjects
• Executed the tool
• Found the access and rank of actual bug
* Cooperative Crug Isolation [Thakur, Sen, Liblit, Lu, WODA09]
23

24. Background Technique Studies Conclusion
Result – Case Study ranking
CCI ranking Falcon
1. RS1CCI 1. WS1-WS3-RS2
RS2-WS3-RS1
Falcon
Subjects
…
Accesses of Bug (Rank) 2. RS6-WS2-RS6
Pattern of Bug (Rank)
Account …R W W
12. (6),S2 (4), R (6) 3. WS4-RS6-WS2
RS4R-W-R (2)
-W
AirlineTickets …R …
13. (2), W (5), R (4) 4. RS1R-W-R-WS6
-WS4 (1)
WS1-WS3-WS6
BubbleSort2
14. RS2R (7), W (2)
W (1),
15. … RS6W-R-W-RS6
-WS3 (1)
5. WS6-WS4-RS6
BufWriter CCI ranking
R (5), W (1), R (5) Falcon ranking
6. RS2R-W-R-WS6
(1)
-WS4-RS3
S3
15. WS6
R S3
Lottery 1. R (1), W (7), R (4)
… RS1
1. WS1-WS3-RS2
R-W-R (1)
7. … -WS4-WS6
RS1
MergeSort 2.W (1), R (3), W (12)
RS2 2. RS6W-R-W-RS6
-WS4 (1)
Shop 3. (15),S3 (12), R (15)
R WW R-W-R (2)
3. WS4-WS6-RS2
ArrayList
4. R (4),S4 (4), R (4)
W W 4. RS1R-W-R-RS6
-WS4 (1)
HashSet R (15), W (15), R (1) R-W-R (1)
5. RS5 5. WS6-WS4-WS6
StringBuffer R (14), W (2), R (1) R-W-R (1)
6. RS6 6. RS2-WS4-WS6
TreeSet R (10), W (5), R (10) R-W-R (2)
• CCI reports the R (2), W (2), R (2)
Vector
… 7. …
7. actual bug with lower ranks R-W-R (1)
• CCI does not provide an(2)
Cache4j W (79), R
association of buggy (1)
W-R
accesses
RayTracer R (4), W (4), W (3) R-W-W (2)
24

25. Background Technique Studies Conclusion
Future Work
• Perform more empirical studies
• With larger subjects
• With more subjects especially with order violations
• With considering multiple test cases
• Apply to other languages and systems
• Other than Java language
• Extend the technique to support
• Handling multi-variable related patterns for
atomicity violations
• Localizing deadlocks
25

26. Background Technique Studies Conclusion
Contributions
• Falcon is the first technique
• that reports and ranks patterns by
suspiciousness
• that addresses order violations and
atomicity violations with patterns
• Studies show that
• Falcon’s window size
• Small constant
• Not correlated with number of threads
• Falcon’s suspiciousness ranking technique
• Effective in identifying faulty interleavings
• More effective than ranking individual accesses
Questions? 26

27. Background Technique Studies Conclusion
Backup Slides
27

28. Background Technique Studies Conclusion
Result – Efficiency
Conflict- AVIO Atomic-Set- Falcon
Serializability (ASPLOS 06) Serializability (ICSE10)
(PPoPP 06) (ICSE 08)
Efficiency 38X 25X 13X 9.9X
(Average (Median (Used almost
slowdown) slowdown) same set of
subjects as
Falcon)
28