A memory leak in an application deployed on the cloud can affect the availability and reliability of the application. Therefore, to identify and ultimately resolve it quickly is highly important. However, in the production environment running on the cloud, memory leak detection is a challenge without the knowledge of the application or its internal object allocation details. This paper addresses this challenge of online detection of memory leaks in cloud-based infrastructure without having any internal application knowledge by introducing a novel machine learning-based algorithm Precog. This algorithm solely uses one metric i.e the system's memory utilization on which the application is deployed for the detection of a memory leak. The developed algorithm's accuracy was tested on 60 virtual machines manually labeled memory utilization data provided by our industry partner Huawei Munich Research Center and it was found that the proposed algorithm achieves the accuracy score of 85% with less than half a second prediction time per virtual machine.

1. 1
Online Memory Leak Detection in the Cloud-based
Infrastructures
Anshul Jindal*, Paul Staab† , Jorge Cardoso†, Michael Gerndt* and
Vladimir Podolskiy*
*Technical University of Munich (TUM), Germany
†Huawei Munich Research Center Germany
International Workshop on Artificial Intelligence for IT
Operations (AIOps 2020)
18th edition of the International Conference on Service
Oriented Computing (ICSOC 2020)
14th December 2020

2. q Introduction
q Goals
q Implementation
q Results
q Conclusion
q Future work
2
Outline
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020

3. 3
Introduction: What is memory leak?
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
q A memory leak is loss of available memory when a program fails to return memory that
it has obtained for temporary use
100%
Time
Mem.Util.
Memory Utilization hits
ceiling
Potential
Memory
Leak case

4. 4
Introduction: Memory leak on VMs in Cloud
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
q Long-running software services are susceptible to memory leaks.
q Effects:
o compromising the performance and the stability of the service.
o Catastrophic Impact on critical infrastructure.
o Requires restart of VMs.
q Challenges:
o Hard to trace during development (due to microservices architecture).
o Applications are written in different programming languages.
o Hard to monitor thousands of VMs running simultaneously for memory leak.
o For static code analysis: require source code modifications and add to performance
overhead.

5. 5
Introduction: Memory Leak Patterns
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
Linear Increasing Pattern Saw-Tooth Pattern

6. 6
Goals
Detection of memory leaks on VMs without any
insight knowledge of applications running on VM.
Minimal overhead with detection in a
reasonable time.
Scalable approach.
1
2
3
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020

7. 7
Implementation: Methodology
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
q Try to fit a line at a window from the end of the metric time series. It requires that the
line approximates the data well (based on R2).
q Fit lines for windows of different sizes and pick the best and longest one
Time
Metric
Window1: Good fit
Window2: Bad fit

8. 8
Implementation: Precog
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
… Data
Pre-Processing
Trend Lines
Fitting
Detect Trends
Minimum
R2 score
Critical
Time C
Trend
Lines
Precog
Save All
Trends
Maximum
Slope
Maximum
Duration
Training
Trends
Training
Data
New
Data
Analysis
New Data
Trends
Saved Values
Result
a VM memory utilization
timeseries data

9. 9
Implementation: Precog (Parameters)
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
Parameter Description Default
min_window Minimal size of a window in points. Only trends
present for at least this period are detected.
72 (= 6h)
max_window Maximal size of a window in points. 288 (= 24h)
min_r_squared Required for considering a line a good fit to the
data. Higher values require that the time series
must be very close to a line.
0.8
max_threshold Maximum threshold we care about. 100
critical_time If the time left until we hit the ceiling is less than this
time (measured in points), an anomaly will be
detected.
2016 (=7d)

10. 10
Evaluation Settings: Experimental Configurations
Real Cloud VMs Dataset* Synthetic Data
Memory Leaks No Memory Leaks Memory Leaks No Memory Leaks
20 40 90 90
q Evaluations were conducted on executed on a machine with 4 physical cores (3.6 GHz
Intel Core i7-4790 CPU) with hyperthreading enabled and 16 GB of RAM.
*Provided by Huawei Munich Research Center
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020

11. 11
(i) Linear
Increasing
(ii) Sawtooth
(iii) Linear
Increasing
without
trends in
training
data
(iv) Linear
Increasing with
similar trends
as training data
and correctly
not detected.
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
Results: Memory Leak Detection

12. 12
Results: Memory Leak Detection (Accuracy)
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
Dataset Sub-cases +ve Cases -ve Cases F1 Score Recall Precision
Synthetic
Linearly Increasing 30 30 0.933 0.933 0.933
Linearly Increasing
(with noise)
30 30 0.895 1.0 0.810
Sawtooth 30 30 0.83 0.73 0.956
Overall 90 90 0.9 0.9 0.91
Real Overall 20 60 0.857 0.75 1.0

13. 13Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
Prediction time within 1 second
Results: Memory Leak Detection (Training and Predict Time)

14. 14Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
Critical TimeMinimum R2 Score
Results: Memory Leak Detection (Parameter Sensitivity)

15. 15
Conclusion & Future Work
Developed Precog algorithm relevant for cloud-based infrastructures.
Precog is scalable to thousands of VMs.
Precog is able to achieve a F1-Score of 0.85 with less than half a
second prediction time
Although Precog depends on some parameters but beyond a certain
values it is insensitive.
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020

16. 16
Future Work
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
q Use of Precog for detection of memory leaks in Serverless computing.
q Extension of Precog to other metrics like Disk Consumption and Cost Budget.
q Use of other metrics like CPU, network and storage utilization for enhancing the accuracy
performance.

17. Edge-Cloud and IoT RG @ Chair of Computer Architecture and Parallel Systems
Technical University of Munich
q Research Areas:
• Self-adaptive Edge-Cloud Continuum (predictive autoscaling for VMs, offloading computation from edge to cloud).
• AI for smart Cloud operations (anomaly detection and failure predictions).
• Modelling of microservices applications and many more…
q Key Publications:
• [2020, WoSC@Middleware] Mohak Chadha, Anshul Jindal, and Michael Gerndt. 2020. "Towards Federated Learning
Using FaaS Fabric".
• [2019, ICPE ] A. Jindal, V. Podolskiy, M. Gerndt "Performance Modeling for Cloud Microservice Applications”.
• [2018, CLOUD] Vladimir Podolskiy, Anshul Jindal and Michael Gerndt. IaaS Reactive Autoscaling Performance
Challenges. The 2018 IEEE International Conference on Cloud Computing (CLOUD 2018).
17
We are…
Anshul Jindal | Online Memory Leak Detection in the Cloud-based Infrastructures | AIOPS 2020
Prof. Dr.
Michael Gerndt
Ph.D Student
Vladimir Podolskiy
Ph.D. Student
Anshul Jindal
+ Other Students
Ph.D. Student
Mohak Chadha

18. 18
Contact
anshul.jindal@tum.de
/ansjin
/Anshul_Jindal4
Thank you for your attention!
Questions ?