Performance insights twitch
•Download as PPTX, PDF•
1 like•195 views
Performance Insights is a service that provides visibility into the performance of Amazon RDS databases. It monitors database load and average active sessions to identify potential bottlenecks. The dashboard allows users to filter metrics by time frame, SQL query, user, host, and other attributes to help diagnose performance issues across different database engines like Amazon Aurora and MySQL.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Performance insights twitch
Similar to Performance insights twitch (20)
More from Kyle Hailey
More from Kyle Hailey (20)
Recently uploaded
Recently uploaded (20)
Performance insights twitch
- 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Using Performance Insights to Optimize Database Performance Kyle Hailey Amazon RDS
- 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda What is Performance Insights? Sampling Average active sessions (AAS) Bottleneck analysis Exploring Performance Insights
- 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is Performance Insights?
- 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. WhatisAmazon RDSPerformance Insights? Customers asked for • Visibility into performance of Amazon Relational Database Service (Amazon RDS) databases • Want to optimize cloud database workloads • Easy tool • Often only part-time DBA or no DBA • Single pane of glass
- 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Released 2016 • OS metrics • Process/Thread list • Up to 1 second granularity Firststep:Amazon RDS EnhancedMonitoring
- 6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Introducing: Performance Insights • Dashboard • DB load • SQL causing load • Adjustable timeframe • Filterable by attribute (SQL, User, Host, Wait) • Phased Amazon RDS delivery • Amazon Aurora, Amazon RDS for MySQL, PostgreSQL, Oracle, SQL Server, MariaDB • Guided discovery of performance problems • For both beginners & experts • Core metric “database load”
- 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. • All engines have a connections list showing • Active • Idle • We sample every second • For each active session, collect • SQL • State: CPU, I/O, lock, commit log wait, and more • Host • User • Expose as “average active sessions” (AAS) Whatis “databaseload”?
- 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Performance Insightsdashboard
- 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Sampling
- 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Sampling islikefilm
- 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Query run often Fast query run rarely Slow query User 1 User 2 User 3 Time Sampling every second
- 12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Activesessionstate idleidle idle idleQuery 1 Query 2 Query 3 Time
- 13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AASload graph User 1 User 2 User 3 User 4
- 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AASload graph User 1 User 2 User 3 User 4 Active sessions = 1 2 3 4
- 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Activesession Time User 1 1 2 3 4 5 6 7 8 9 10 Active sessions 1 2 3 4
- 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Activesession Time User 1 User 2 1 2 3 4 5 6 7 8 9 10 Active sessions 1 2 3 4
- 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Activesession Time User 1 User 2 1 2 3 4 5 6 7 8 9 10 Active sessions 1 2 3 4
- 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Average activesession Time User 1 User 2 1 2 3 4 5 6 7 8 9 10 Active sessions 1 2 3 4
- 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Average activesession Time User 1 User 2 User 3 User 4 User 5 User 6 User 7 User 8 User 9 User 10 1 2 3 4 5 6 7 8 9 10 Active sessions 1 2 3 4
- 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Activesessionstate idleidle idle idleQuery 1 Query 2 Query 3 Time
- 21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Activesessionstate Time CPU IO Wait idleidle idle idleQuery 1 Query 2 Query 3
- 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AASby sessionstate
- 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Showingper second samples
- 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AASover one minuteaverages
- 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AAScompared tomaxCPU Max vCPU
- 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Average active sessions
- 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AASrules:UsingCPU count asyardstick AAS < 1 Database is not blocked AAS ~= 0 Database basically idle Problems are in the APP not DB AAS < # of CPUs CPU available Are any single sessions 100% active? • AAS > # of CPUs Could have performance problems AAS >> # of CPUS There is a bottleneck
- 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Whenusers saythedatabaseis slow … You prove that it’s not the database AAS = 0
- 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Alsousefulfor sizing • If CPU load significantly less than #vCPU then oversized • If CPU load Is > #vCPU undersized
- 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Accessing Performance Insights
- 31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AccessingPerformance Insights
- 32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Accessto Performance Insights
- 33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customer use case: CPU bottleneck
- 34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. MaxvCPU Max vCPU
- 35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. CPUbottleneck Bottleneck
- 36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Customerusecase:CPU bottleneck Wait States
- 37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. CPUbottleneck CPU
- 38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. CPUbottleneck CPU Load SQL with high CPU
- 39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Clickand drag
- 40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Zoom in
- 41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customer use case: Wait bottleneck
- 42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Waitbottleneck
- 43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Waitbottleneck
- 44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Dashboard: Other grouping dimensions
- 45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Otherdimensions
- 46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Top hostbySQLstatement
- 47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Counter Metrics
- 48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Counter Metrics
- 49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Performance Insights across engines
- 50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Performance Insightsacross DB engines • Performance Insights supports • Amazon Aurora • MySQL • Postgres • Amazon RDS • MySQL • Postgres • Oracle • SQL Server • MariaDB • Interface is the same across different engines • Allows DBA to do performance work across different engines easily • Dashboard content same • Only difference is the wait event names, which are engine dependent
- 51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AmazonAurora MySQL—Fiveusers
- 52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AmazonAurora PostgreSQL—Five users
- 53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Oracle—Fiveusers
- 54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. PostgreSQL —Fiveusers
- 55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. MySQL—Fiveusers
- 56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What’s available
- 57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Whatis available? Available • Engines • Amazon Aurora PostgreSQL • Amazon Aurora MySQL 5.6 1.17.3 and higher • Amazon RDS for PostgreSQL 10 • Amazon RDS for MySQL 5.6.41+ and 5.7.22+ • Amazon RDS for Oracle • Amazon RDS for MariaDB 10.1 • Amazon RDS for SQL Server (all but 2008) • Functionality • DB load chart • Top N table • Wait, user, host, SQL • API/SDK • Long-term data retention • Alerts through Amazon CloudWatch • OS and DB counter metrics • Upcoming • Per SQL statistics • Execution Plans
- 58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Summary: Amazon RDSPerformance Insights DB load: Average active sessions Identifies database bottlenecks Easy Powerful Top SQL Identifies source of bottleneck Enables problem discovery Adjustable time frame Hour, day, week, and longer Questions: rdspi@amazon.com
- 59. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Kyle Hailey Amazon RDS
Editor's Notes
- I’m Kyle Hailey and I’ve been the Product Manager on Performance Insights for the past couple years. As PM I sheparded the product to market releasing it just over a year ago. Recently I’ve transitions to the position of Principal Engineer in RDS and continue to work on Performance Insights and as well as other performance monitoring features. Some background on me: I’ve been working database performance for the last 25 years and have worked worked on many of the industry’s leading data performance monitoring Products. But Performance Insights is the the most exciting project to date. Some of the products I’ve worked on in the past are Oracle Enterprise manager Where I redesigned the performance monitoring pages I worked on products at Quest such as Spotlight and Foglight At Embarcadero I led the design of the tool DB Optimizer Other things I’ve been know for are Direct memory access of database performance metrics without SQL The reason did this is was to be able to collect performance metrics with the least amount of impact on the database I was monitoring. And reading memory is one of the cheapest things we can do on a computer So if we can read performance data directly from memory we can collect performance data with almost no impact on the database being monitored. This is one of the methods we use in Performance Insights. One of the tenants in our group at Performance Insights to have the least impact on the database as possible. With all the experiences I’ve had in the industry, I’m super excited to be at RDS and apply what I now to all of the RDS databses: , MySQL, PosgreSQL , Oracle and SQL Server and Aurora Mysql and Auyrora Postgres. Being at amazon has presented some unique challenges. THe scale and rate of growth in RDS is amazing One of the most impressive aspect of this project is the scalability of PI The infrastructure we’ve built will scale up to millions of databases In the short time PI has been out we are already supporting over 100K database instances.. Typically when one uses a a tool, one installs the tool either on the database to Be monitored, or we set up a secondary machine where we install the monitoring software. With Performance Insights we have to taken care of all that. You don’t need ot worry about the installation, storage,or administration of the monitoring software. With PI we’ve eliminated all that When you create a database in RDS console there is a checked box that is on by default for Performance Insights. That’s all you neeed to do We manage it all It’s scalable and rapid The time between when we capture data and when you see it in the dashboard is typically on the order of a second And the feature come free with RDS databases.
- Whats on the agenda toay. First want to about what PI is. Then Want to talk about how we collect data which is sampling Sampling is a bit different from other tools that use time series From sampling we derive a metric “aveage active sessions” AAS AAS is our core metric Using this metric we can quicly an easily see the load on a database And because the data is what we call multi-dimentional We can not only see when load is high We can see which SQL is causing that load Which users are causing that load What hosts those SQL are originating from We can answer different questions from the same data. Look at some bottleneck examples And finaly explore some of the options in the PI interfaces
- Over the years we’ve received a lot of great feedback about our features such as automated patching, automated backups, different available ity zones, HA Infrastructure But one thing they have been asking for is visibility into the performance of Their RDS database They want an easy tool We have huge set fo customers ranging from small customers who might not Even have a DBA or a part time DBA to fto enterprise customers with teams of database experts. Customers wanted feature that was usable both by beginning users as well as expert users A tool tha was easy to use but powerful as well. So we want an interface that is both easy for non-DBAs and powerful for expert DBSa. Some commercial already had a number robust tools for performance monitoring but ON databasers like PostgreSQL or MySQL there isn’t a rich exosystems of tools
- Our first step was Enhanced monitoring which was releases about 3 years ago in 2016 We released something called It’s a bit different from Cloudwatch in that it showed the process liests Cin RDS you can’t shell into the host, so can’t see things like top With the OS process/thread list we can now see the top processes by CPU and memory usage. EM also had the option of collecing metrics down to once second granularity. CW has since announced that functionality as well But this data is OS centric and not database centeric. There are a lot of graphs to look at it and it can be challenging to know what to look at, where to start and how to correlate the dtaa. One thing I want to point out is these are time series metrics. These are time series metrics A lot of graphs to correlate and understand It’s also time series data. For example for CPU %, maybe it shows that the system is CPU saturated But how do we find where that CPU usage is coming from? We have to get other data and correlate it That data can be missing or hard to know where to get. Maybe I see that I have page ins where I’m reading memory pages from swap Well what impact does that have on my load and SQL response. I need some way to correlate that. With sampling that Peformance Insights uses, we can answer those questions
- Introducing Performance Insights Instead of many charts it has one main graph of database load Which shows quickly and clearly if there is a bottleneck Because it’s multii dimentional we can also make the correlations between When CPU demand is high and seeing which SQL are making the CPU demand high. Phased Featrues are rolled out in phased manner across of all of RDS databases We’ now support all RDS database engines and as we roll out new features we released them in a phaed approach For example over the past few months wev’e releases a fature calle dcounter metrics Which are additional time series metrics that ou can adde to the PI dahsnboard and correlate with database load. This has been released on almost all of the RDS engines, with the finaly rellout happening over the next few weeks Wanted a guided experience An interface that encourages exploration Easy But powerful
- Want to make it visually impactful See two spikes here Green spike Read spike Then see top SQL The First SQL is all green meaning it’s spending all it’s time on CPU and the main source of CPU demand As the other SQL only show relatively small CPU demand
- Want to talke a little about hwo we colecte data and how we visualize it One thing that drives engineers a little bit crazy crazy the first time I show them is That we sample data We collect once a second, then sleep and then wake up and collect again In between we don’t know what is happening and miss that activity May seem a problem but actuall collecting data describes databse actively well provides a seamliess experiences One anology is films When I go to a cinema I’m seeing 24 frames a second One example is seamless Continuity It looks continuous with mothing missing To collect everything you would have to trace That would first slow your system down and second it would be so much data you would be overwhelmef Data it woujld be hard to extract the data of interest from the massive amounts Of data colleted. With sampled data it is not only a reduced manageable amount of data But it is also multidimensiona data which allows us to correlate data and answer different questions Like for the example of when we see that CPU is staturated, the Multi dimensional data will also allow us to see what SQL and what user are causing The high CPU load
- On of the fringe benefits of sampling is That it filters out noise and allows us to easily see the data of interst