Step into the future of application performance monitoring as we unveil the game-changing potential of micro-metrics. In this enlightening session, we'll explore why traditional macro-metrics fall short in predicting performance problems and how to overcome this limitation. Discover the key micro-metrics that serve as powerful lead indicators, enabling you to forecast application performance with remarkable accuracy. Unleash the ability to detect and mitigate potential outages several minutes, even hours, before they impact your operations.

1. Predicting Production Outages:
Unleashing the Power of Micro-Metrics
Ram Lakshmanan
Architect yCrash

2. Predicting Memory Problems

3. 3
Healthy Application

4. 4
Acute Memory Leak

5. 5
Memory Leak

6. GC Throughput
Micrometric
Source: Garbage Collection Log

7. How does 96% GC Throughput sound?
1 day = 1440 Minutes (i.e., 24 hours x 60 minutes)
96% GC Throughput means app pausing for 57.6
minutes/day
7
What is GC Throughput?
Amount of time application spends in processing customer
transactions
vs
Amount of time application spends in processing garbage
collection activity

8. 1. GC Log
10. netstat
12. vmstat
2. Thread Dump
9. dmesg
3. Heap Dump
6. ps
8. Disk Usage
5. top 13. iostat
11. ping
14. Kernel Params
15. App Logs
16. metadata
4. Heap Substitute
7. top -H
8
Open-source script:
https://github.com/ycrash/yc-data-script
360° Data

9. Predicting Backend Slowdown

10. Application Architecture
JDBC
SOAP
MainFrame
REST
Server Thread Pool
Application Server
HTTP(S) request
10

11. Application Architecture
JDBC
SOAP
MainFrame
REST
Server Thread Pool
Application Server
HTTP(S) request
11

12. Threads with identical Stack trace
Micrometric
Source: Thread Dump

13. 13
Case Study
Backend Slowdown in a Major
Financial Institution in N.
America

14. Predicting CPU Spike

15. top –H –p <PROCESS_ID>’
Secrete Option:
15
We all might have used ‘top’

16. Case Study
Major Trading app in N.
America
https://blog.fastthread.io/2020/04/23/troubleshooting-cpu-spike-in-a-major-trading-application/
16

17. Predicting Concurrency issues

18. public void synchronized getData() {
doSomething();
}
Thread 1
Thread 2
Thread 1
BLOCKED THREADS
Concurrency Problem
18

19. BLOCKED state threads
Micrometric
Source: Thread Dump

20. Case Study
Major Leisure Travel Service
Provider
https://blog.fastthread.io/2020/04/23/troubleshooting-cpu-spike-in-a-major-trading-application/
20

21. Specific Errors/Exceptions
Micrometric
Source: Application Logs

22. My App
yCrash
agent
yCrash Server
Container/Machine
1
Every 3 minutes Micro-Metrics*
are captured
2 Metrics are transmitted
4 If problem forecasted,
360 ° data capture
is triggered
3 ML, Patterns applied on the Micro-Metrics
Cloud/On-premise
22
Micro-Metrics *
1. Garbage Collection Log
2. Thread Dump + top –H
3. Application Log
Micro-Metrics Monitoring Architecture

23. 1. GC Log
10. netstat
12. vmstat
2. Thread Dump
9. dmesg
3. Heap Dump
6. ps
8. Disk Usage
5. top 13. iostat
11. ping
14. Kernel Params
15. App Logs
16. metadata
4. Heap Substitute
7. top -H
23
Open-source script:
https://github.com/ycrash/yc-data-script
360° Data

24. Ram Lakshmanan ram@tier1app.com
@tier1app https://www.linkedin.com/company/ycrash
This deck will be published in:
https://blog.ycrash.io
Learn to troubleshoot like a pro with my online training program
24
THANK YOU
FRIENDS