Product Overview: An Analytics Primer

Product Overview: An Analytics Primer
Paul Johnston, Zenoss
pjohnston@zenoss.com

pjohnston@zenoss.com © 2017 All Rights Reserved CONFIDENTIAL 2
An Analytics Primer
• What is Zenoss Analytics?
• Typical Use Cases
• Analytics Configuration
• Data Analysis Using TIBCO Jaspersoft™
• Demo
• Next Steps - How to get started with
Zenoss Analytics
• Q&A

What is Analytics?
• Embedded BI Tool (TIBCO Jaspersoft™)
• Open-source, “skinned” by Zenoss
• Visual Data Exploration and Analysis
using interactive “Ad Hoc” UI
• Scheduling of analysis execution to
produce reports
• Can be also used on ANY supported data
source (ODBC, JDBC, Amazon, MongoDB
etc.) e.g. external CMDB etc.
• Library of examples of using Jaspersoft
in a Zenoss Context
• Event Categorization, details etc.
• Capacity Planning for Performance
Metrics
• ETL App (Extract, Transform, Load)
• Batch processes extracts Model (ZODB),
Event (ZEP) and Performance Data
(OpenTSDB or RRD) from Zenoss
Resource Manager
• Extracted data loaded into MySQL
(MariaDB) data warehouse
• Aggregation (“roll-up”) processes
perform regular bulk analysis and store
results in the data warehouse
• Hourly and Daily Aggregates
• Business Hour Aggregations
• Nth Percentile Calculations
• Future projections

What is Analytics? - continued
• NOT an extensive library of “canned”
reports
• Shipping examples of analysis in
Jaspersoft are just that – a starting point
• Flexibility to build your own analysis
using the tool based on YOUR use cases
• ZenPack support for Analytics is evolving
• ZenPack specific capabilities ship with the
ZenPack, not with Analytics
• Provide an appropriate Performance ETL
configuration, documentation via Domain
in Jaspersoft of model and
device/component relationships
• Very much a “tool”
• Industry standard star-schema based
data warehouse
• Schema is 100% dynamic, evolves based
on what you are monitoring
• Jaspersoft is provided so that you have
SOME way to do analysis without
resorting to SQL
• ANY 3rd party BI tool (e.g. Tableau,
Business Objects) etc. can be employed
to do analysis
• Open – have at it with SQL if you are so
inclined!

Typical Uses Cases
• Where is my event
volume coming from?
• Where would a
transform help?
• Are my thresholds
appropriate or too
noisy?
• Slice and dice data by
any field - Device Class,
Event Class, Agent,
Severity, Status etc.
• What infrastructure as
an aggregate (Group,
System, Location,
Naming convention
etc.) is over/under
used?
• What does past usage
look like?
• What might future
usage look like?
• Full Details of all events
and details of every
(non-unique) event
occurrence
• Historical reporting on
performance metrics
• Full model reporting
• Monitoring
configuration reporting
Event Categorization Detailed Reporting Capacity Planning

Typical Uses Cases - continued
• Extends the multi-
tenancy in Resource
Manager to Analytics
• Identical analysis that
behaves differently
depending on the
tenant executing it (e.g.
MSP Customer, Internal
Department etc.)
• Unlimited and flexible
scheduling
• Many formats - CSV,
Excel, PDF etc.
• Many delivery -
Email, Filesystem
export, FTP
• Exceptions reporting
• Don’t deliver report
unless content is
generated
• Operational stores in
RM (OpenTSDB, ZEP)
typically record
everything for 90 days
• Analytics retention
policy configurable by
resolution
• 5 mins (30 days)
• Hourly (90 days)
• Daily and Events
(365+ days)
Multi-Tenant
Reporting
Long Term Event &
Metric Storage
Scheduled Report
Delivery

ETL Configuration
• No configuration at all is required for Model or Event information ETL
• As you extend the model (e.g. ZenPack install) or event information (e.g. add new event
fields), the ETL is dynamically adjusted and new tables/fields will get dynamically created in
the data warehouse.
• Performance Data ETL requires PER DATAPOINT template configuration – “Aliases”
• Important to build a data warehouse “metric catalog” and normalize configuration for
identical logical metrics like CPU, memory, filesystem and interface utilization
• Should this data point be ETLed?
• Which data warehouse table(s) should the metric be added to?
• Optionally, how should the monitored units be converted during ETL (e.g. memory is
monitored in free bytes, want a used percentage based on total in the data warehouse)
• Bulk monitoring template manipulation tools provided by Analytics to read/write alias config
• Vital to put ETL config under configuration management and audit after any ZP or RM
upgrade that adjusts templates

ETL Configuration

Data Warehouse Configuration
• “One-time” configuration e.g. Data retention policies, ETL scheduling etc.
• Incremental additions to aggregation configuration over time
• Shipping config does hourly/daily aggregations for all devices, all metrics, 24x7
• Unlimited numbers of additional aggregations can be configured, filtered by device group(s)
and/or metric(s)
• Business Hours – time range in day, days of week, time zone data shifting
• Nth Percentile - arbitrary nth percentile (e.g. 90, 95, 98), rolling windows (e.g. 30 days)
• Projections
• Regular, business hour or nth percentile aggregate
• Number of previous days to look at, Number of days into the future to project.
• ALL future days are (re)calculated and written to the data warehouse, daily.
• E.g. “List device name and the first date in the next 90 days that any Linux server owned by the
Marketing department is expected to have more than 70% for a trailing 30-day 95th percentile CPU
utilization between 7am to 7pm Bangalore time"

Data Analysis using Jaspersoft

Data Analysis Using TIBCO Jaspersoft™
Level of
abstraction
from physical
structure of
data warehouse
data
Physical Data
Structure
Meta Data Access
Layer
Visual Analysis
e.g. Chart, Table
Analysis – “Report
Execution”
Analysis Results –
“Report”

Data Analysis Using TIBCO Jaspersoft™
• Get Data (ETL config)
• Decide on resolution(s)
• Design Domain
• start from a Example
Domain, or a ZenPack
provided one
• Adjust to access
additional data and
expose to Analysis
(e.g. additional model
fields, perf metrics,
different resolution)
• Get sign off on Analysis
design from business
owners
• Execute ad hoc as
needed
• Schedule for regular
analysis execution and
content delivery to
stakeholders
• Choose a Presentation
Type (Bar Chart, Time
Series chart, Pivot
Table, Table)
• Decide on filters and
aggregation of data
(e.g. by device class,
group, location, time)
• Adjust analysis
iteratively and repeat
Enable Data Access
(“Domain”)
Design Analysis
(“View”)
Execute Analysis
(“Report”)

Demo - Event Categorization

Demo – Historical Performance Metrics

Demo – Historical Performance Metric Detail

Demo – Exception Reporting

Demo – Historical Raw Detail

Next Steps - How to get started with Analytics

Next Steps – How to get started with Zenoss Analytics
• Get your infrastructure monitoring working in Resource Manager
• You can’t analyze data you don’t have!!
• Take the Analytics Training Class
• 3 day class given over WebEx by Analytics SME
• This is a DEEP topic, training class attendance is deemed fundamental to your success
• Install Analytics and Configure Performance ETL
• You can’t analyze data you don’t have!!
• Get Some Analysis Requirements
• This is harder than it sounds sometimes  You can’t design analysis you don’t understand!!
• Start with a few well defined use cases (e.g. “historical reporting”) and drill in
• Try to avoid “future capacity planning” use cases initially
• Build a domain, design a view and get sign off from stakeholders on analysis execution
filters and schedules BEFORE creating or scheduling regular analysis

Product Overview: An Analytics Primer

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Product Overview: An Analytics Primer

Similar to Product Overview: An Analytics Primer (20)

More from Zenoss

More from Zenoss (18)

Recently uploaded

Recently uploaded (20)

Product Overview: An Analytics Primer