Databases are fundamental to every application, and reading or writing data in a quick and reliable way is critical to ensuring happy users. Your shiny new application may have worked well in the beginning, but with more and more people using the app the response times may take a nose dive. Or, what if you push a great new feature, guaranteed to make customers happy, but then it doesn’t scale or it actually degrades the performance of existing functionality? What do you do? How do you diagnose and resolve these issues in a timely and cost efficient manner? Whilst many organizations employ a team of database administrators (DBAs) to manage database performance, it’s often a group separated from development and operational support with their own tools, scripts, and procedures. This creates inefficiency in the root cause analysis process. We want to empower customers by including deep-dive database monitoring as part of end-to-end APM, thereby providing immediate visibility of DB metrics to all groups within IT. This session covers: -Why it makes sense to include the database as part of API -What are some real world problems affecting database performance -What is a good methodology for diagnosing and understanding database performance This deck was originally presented at AppSphere 2015.

1. Is the database affecting your critical
business transactions?
Ian McGuinness Product Manager

2. A quick history of databases
• Relational databases have been around since the 70’s
– Oracle released it’s first database in 1978
• Client/Server Applications evolve
– SQL and Stored Procedure driven
• Application Complexity increases
– Object Relational Mappings help map relational to OO coded applications
• Monolithic architecture à Clustered Database e.g. Oracle RAC
• 2015
– “Structured data” still dominated by relational databases e.g. Oracle, SQL Server, MySQL etc.
– NoSQL databases building market share fast e.g. MongoDB, Hadoop
Copyright © 2015 AppDynamics. All rights reserved. 2

3. Database problems over the years
• Database problems today aren’t that different to they were
15 years a go!
Copyright © 2015 AppDynamics. All rights reserved. 3
Bad query performance
e.g. Poorly indexed queries
Conflict between users/queries
e.g. Batch activities causing lock contention for online users
Capacity Issues
e.g. Slow disk not enough IOPS for the needs of the app
Inefficient database configuration parameters
e.g. Buffer cache too small

4. Databases are not static
• Data is exploding!
• Data production will be 44x greater in 2020 versus 2009*
• App users are generating data all the time
– Companies are using that data for analytics to drive further sales
• Biggest growth in unstructured data
Copyright © 2015 AppDynamics. All rights reserved. 4
* http://www.csc.com/insights/flxwd/78931-big_data_universe_beginning_to_explode

5. Usage patterns are unregulated
• Most consumer applications have a loose usage pattern
• Spikes in concurrency may occur any time day or night
• Databases are a shared resource – contention on one part
can affect the whole
Copyright © 2015 AppDynamics. All rights reserved. 5

6. INTRODUCING APPDYNAMICS
DATABASE MONITORING

7. DB Agent
SaaS or On-Premise On-Premise
Controller
HTTP(S)
AppDynamics Integrated DB Monitoring
Key Features
§ Low overhead – Production Safe
§ Rapid Installation – Agentless
§ Detailed & comprehensive analysis
§ Current & historical granular data
§ Consolidate web based GUI
§ Support for almost all versions of DB2
LUW, MS-SQL, Oracle, Sybase ASE,
Sybase IQ, PostgreSQL, MySQL and
MongoDB
§ Server Monitoring for Windows, Linux,
AIX and Solaris

8. USE CASE EXAMPLES
#1 Long running SQL Queries

9. Slow Running Application SQL
Copyright © 2015 AppDynamics. All rights reserved. 9
• APM can track query response time, but DBMon will tell the
user why the query is bad
• Sometimes business critical queries will fall outside the top
few that are actively monitored by the DBA

10. Slow BT with high query response time
91.3% of time
spent in
Oracle call

11. Detailed Query information in DBMon

12. Coming to 4.2! – Snapshot Correlation
• Available for Java à Oracle
• Java BCI tags Oracle sessions with BT ID and GUID
• Allows drilldown from Snapshot Flowmap to specific
queries within DBMon
Copyright © 2015 AppDynamics. All rights reserved. 12

13. USE CASE EXAMPLES
#2 Poorly configured database

14. Poorly sized buffer pool (MySQL)
Number of Physical I/
O’s to populate cache

15. Before and After - Buffer Pool Re-size
• 5 minute load tests were run
• Before recommendations we can see 5 mins 34 secs spent in MySQL
• After recommendations we can see just 27 secs of MySQL time
• 92% reduction in DB Time!

16. Coming to 4.2! – Improved Health Rules
Copyright © 2015 AppDynamics. All rights reserved. 16
• Create a single health rule which applies to:
– all databases
– all databases of a certain type e.g. MySQL, SQL Server etc.
• Use Health Rules in Custom Dashboards

17. DBMon Health Rules

18. Coming to 4.2! – Custom Dashboards
• Custom dashboards now available for any DBMon metric
Copyright © 2015 AppDynamics. All rights reserved. 18

19. DBMon Custom Dashboard

20. DBMon Custom Dashboard

21. USE CASE EXAMPLES
#3 Capacity & Conflict

22. I/O Issues
• Detailed time-series metrics clearly show contention
• Time Spent in DB very high
• Wait State analysis shows PAGEIOLATCH_EX wait
• Host monitoring clearly shows increase in Read IOPS

23. Locking
34 seconds of Lock Wait Time!
88.8% of all activity time
spent in Lock Wait!

24. USE CASE EXAMPLES
#4 NoSQL Performance Issues

25. Key Concepts of the Solution
• Collection of metrics is agentless and read-only i.e. No
running code on Monitored DB
• Metrics are collected without the need to install/modify
additional components on monitored DB e.g. Modules/
Objects etc.
• Provide historical time-series information on Time
spent within DB supplemented by DB specific Metrics
• Gather information about all DB activity - not just
activity resulting from a monitored (with AppD APM)
application

26. Auto-Detection of the Cluster
• If mongoS then all ReplicaSets in the cluster will be discovered.
• If single ReplicaSet then Primary/Secondaries etc. will be monitored.
• Otherwise standalone mongod

27. Detailed Time-Series Metrics

28. Drilldown to problematic Queries

29. Conclusions
• Database performance issues will always be around!
• Proactive performance management is what’s needed
– Use a low overhead 24/7 monitoring solution to gain visibility
– Capture the right type of data to allow rapid root cause analysis and
to prioritize tuning efforts
Copyright © 2015 AppDynamics. All rights reserved. 29

30. Thank You