Measuring Program Comprehension: A Large-Scale Field Study with Professionals
Understanding ULS System Behavior with an Iterative Thread Pool Analysis Methodology
1. Supporting the Comprehension of System Behaviour Under Load
Mark D. Syer, Bram Adams and Ahmed E. Hassan http://sail.cs.queensu.ca/~mdsyer
How To Understand ULS Systems?How To Understand ULS Systems? Case Study: Thread PoolsCase Study: Thread Pools
Ultra-large-scale (ULS) systems dominate the fields of e-commerce and telecommunications.
!Highly concurrent and highly distributed. Behaviour is hard to understand!
!Complex architectures built on complex sub-architectures
We propose:
• an iterative methodology
• for automatically identifying and ranking deviating behaviour
• based on the level of dissimilarity between the resource usage metrics of threads.
Iterative MethodologyIterative Methodology
2. Distance Calculation2. Distance Calculation1. Metric Abstraction1. Metric Abstraction
4. Ranking4. Ranking 3. Hierarchical Clustering3. Hierarchical Clustering
Conclusions and Future WorkConclusions and Future Work
Our methodology has enabled us:
• Develop a better understanding of the subject system
• Identify and rank similar and deviating behaviour
• Tackle different problems where resource usage
_metrics are collected
Threads pools face a variety of challenges:
• Synchronization errors
• Deadlock
Thorough understanding of the system is needed to address these challenges
The subject system in our case study was a commercial ULS system that
implemented the typical thread pool architecture.
Abstracted
Metrics
Distance Matrix
Clusters
Service Request
Service Request
Handler
Thread 1
Thread 2
Thread n
…
Load Balancer
Risks? =>
New Level of
Abstraction
Threads Within A Load TestThreads Within A Load Test
letters
a b
aa ab ba bb
aaa aab aba abb baa bab bba bbb
aa ab bbba
aa ab bbba
Covariance Matrices
- - - - - - - - - - - - - - - Individual threads - - - - - - - - - - - - - - -
Threads Between Load TestsThreads Between Load Tests
• Resource thrashing
• Thread leakage
T1W1
T1W8
T1W2, T1W3,
T1W4, T1W5,
T1W6, T1W7.
Future work:
• More systems
• More design patterns and architectural styles
• Improved ranking algorithm (e.g. load testing)
.5
Ramp up
Ramp down
Iterative
aa
ab
ba bb
Height
00.51.01.52.02.5
Height
aa
ab
ba bb
00.51.01.52.02.5
