More Related Content Similar to ASHviz - Dats visualization research experiments using ASH data (20) ASHviz - Dats visualization research experiments using ASH data1. Architect, Enterprise and Cloud Manageability
Oracle America
John Beresniewicz
Data visualization research experiments using ASH data
#ASHviz
2. Our mission is to help people
see data in new ways, discover insights,
unlock endless possibilities.
3. The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any
material, code, or functionality, and should not be relied upon in making purchasing decisions. The
development, release, timing, and pricing of any features or functionality described for Oracle’s
products may change and remains at the sole discretion of Oracle Corporation.
Safe harbor statement
Copyright © 2020, Oracle and/or its affiliates3
4. Copyright © 2020, Oracle and/or its affiliates4
Two decades of Oracle performance visualization
Savant Diagnostic Center for Oracle
• “What’s going on?”
Oracle Enterprise Manager
• Performance Page
• Top Activity
• ASH Analytics
ASH is the mother lode
• System-wide in scope
• Fine grained detail
Background
5. Copyright © 2020, Oracle and/or its affiliates5
Expose ASH secrets using data visualization
Investigate event count estimation in particular
Develop facility with various data viz constructs
Purpose
6. Copyright © 2020, Oracle and/or its affiliates6
ASH dump file
• Raw production data snapshot
RStudio
• R development IDE
tidyverse
• R packages for data science
ggplot
• High quality data visualization
Data and tools
Real data and high quality free
tools enable rapid progress
7. Copyright © 2020, Oracle and/or its affiliates7
4 csv files from 4-node RAC ASH dump
• Oracle 10.2 database circa 2004
Production OLTP workload
V$EVENT_NAME data
• mapping event ID to name and waitclass name
Imported, curated, packaged for re-use
ASH data
8. Copyright © 2020, Oracle and/or its affiliates8
ashDF
• Raw csv ASH dump data from 4 RAC instances
• Curated for analysis
Events
• Split wait events out from ashDF
Core data frames
ashDF <- ashdat::ASHDUMP1
Events <- ashDF %>% filter( TIME_WAITED >0 )
9. Copyright © 2020, Oracle and/or its affiliates9
SAMPLE_TIME: 6 digits after secs
• microsecond time deltas
Almost all columns are Factors
214K sample rows from 4 instances
Added columns:
• MINIT (for minute summaries)
• STATE (cpu, IO, wait)
• STATE_CLASS (cpu or waitclass)
• EST_COUNT (event count estimate)
str(ashDF)
10. Copyright © 2020, Oracle and/or its affiliates10
Full-featured IDE
Integrated with tidyverse
Multiple project file types
• R script
• R Notebook
• R Markdown
• R Presentation
R development: Rstudio
11. Copyright © 2020, Oracle and/or its affiliates11
A Hadley Wickham creation
Collection of convenience packages for R programming
• Data science and visualization
Common philosophy and interface patterns
Designed to work together consistently
Data wrangling: tidyverse
install.packages(“tidyverse”)
dplyr
tibble
forcats
lubridate
ggplot2
12. Copyright © 2020, Oracle and/or its affiliates12
www.tidyverse.org
“The tidyverse is an opinionated collection of R packages
designed for data science. All packages share an underlying
design philosophy, grammar, and data structures.”
13. Copyright © 2020, Oracle and/or its affiliates13
R data pipeline
Samples <- ashDF %>%
# trim
select(INSTANCE_NUMBER, SAMPLE_TIME, SAMPLE_ID) %>%
# compute by instance
group_by(INSTANCE_NUMBER) %>%
# unique samples
distinct(SAMPLE_TIME, SAMPLE_ID) %>%
# get previous sample time
mutate(PREV_TIME = lag(SAMPLE_TIME, order_by = SAMPLE_ID)) %>%
# compute sample time deltas in milliseconds
mutate(DIFF_MSECS = 1000 *
as.numeric(difftime(SAMPLE_TIME, PREV_TIME, units = "secs")))
14. Copyright © 2020, Oracle and/or its affiliates14
Based on Wilkinson’s layered “Grammar of Graphics”
Plot in layers composed of:
• Data (data frame)
• Statistical transform (stat_count)
• Geometric objects (geom_histogram)
• Coordinates, faceting, position
Supports progressive iteration, refinement and composition
Many nice extension packages (gganimate, ggraph)
Visualizing data: ggplot2
15. Copyright © 2020, Oracle and/or its affiliates15
Data Frame
• Essentially a table wrapped into an object with special properties
Aesthetics
• Map data variables to graphical attributes: x, y, color, size
Geoms
• Geometric objects to render using the aesthetics
Faceting
• Subdivide plot into multiple sub-plots (e.g. by instance)
Plot components
16. Copyright © 2020, Oracle and/or its affiliates16
ggplot examples
d4 <- ashDF %>% group_by(MINIT, STATE) %>% summarize(AAS = n()/60)
#
p <- ggplot(d4, aes(x = MINIT, y = AAS, fill = STATE))
#
p + geom_area()
ggplot(data = Events,
aes(x = EVENT_ID, y = log10(TIME_WAITED))) +
geom_boxplot() +
geom_hline(yintercept = c(3,6), color = "red") +
scale_x_discrete(labels=NULL)
17. Copyright © 2020, Oracle and/or its affiliates | Confidential: Internal/Restricted/Highly Restricted17
Aesthetics and scales
• Faceting
• Filtering
• Log scales
• Point sizes
• Alpha values
• Zoom
• Colour
Geoms
• Line charts
• Histograms
• Dot plots / bubble charts
• Box-whisker plots
• Density and violin plots
• Polar coordinates
Tidy data
• Pipe
• Mutate
• Filter
• Group_by
Techniques used
18. Copyright © 2020, Oracle and/or its affiliates18
Tidyverse:
• https://www.tidyverse.org
• https://rstudio.com
Details about ASH mechanism:
• ASH Architecture and Advanced Usage
ASHviz article series on LinkedIn:
• https://www.linkedin.com/in/john-beresniewicz-986b0/detail/recent-activity/posts/
Google / Stackoverflow
Resources
19. Copyright © 2020, Oracle and/or its affiliates19
Decide on a target objective
• What is hiding in ASH to reveal?
Imagine visualizations for exposing objective
• Map to geoms
Create data frame for plotting
• Apply stats or mutations to ashDF
Plot data with geom
• Evaluate effectiveness
Tweak plot parameters, change geoms
ASHviz process
Decide
Imagine
CreatePlot
Tweak
20. Copyright © 2020, Oracle and/or its affiliates20
• Active sessions over time
• ASH sampler timing consistency
• Event latency point clouds
• Event latency distributions
• Mapping DB time to sessions and SQL
Investigations
30. Copyright © 2020, Oracle and/or its affiliates30 Copyright © 2020, Oracle and/or its affiliates68
33. Copyright © 2020, Oracle and/or its affiliates33
@JBeresniewicz
john.beresniewicz@oracle.com
Thank you