Already on a continuous or short-cycle delivery? Constantly rewiring your apps with microservice and similar architectures? Maintaining visibility and maximizing service levels once this stuff gets into production could be a regular nightmare. Coding instrumentation into your apps is time-consuming and error-prone. Instead, let machine learning do the work of adapting your monitoring to your fast-moving application environments. In this webcast learn about various types of machine learning that are optimized for operational data, and see in a demo how this could be leveraged to ensure your ops move as fast as rest of your DevOps pipeline.

1. Copyright © 2017,Oracle and/orits affiliates. All rights reserved.
Machine Learning Turbo-Charges
the Ops Portion of DevOps
DevOps.com webinar
Tania Le Voi
Director
Oracle Management Cloud
October, 2017

2. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.

3. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Program Agenda
Defining terms
Why (Dev)Ops is Perfect for Machine Learning
Making Machine Learning Smarter
Q&A
1
2
3
4
#MgmtCloud or #DevOps or #devopswebinars
@OracleMgmtCloud

4. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Program Agenda
Defining terms
Why (Dev)Ops is perfect for machine learning
Making Machine Learning Smarter
Q&A
1
2
3
4

5. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Defining Terms (source: wikipedia.com)
• Machine Learning
– Machine learning is the subfield of computer science that gives computers the ability to learn
without being explicitly programmed. Evolved from the study of pattern recognition and
computational learning theory in artificial intelligence, machine learning explores the study
and construction of algorithms that can learn from and make predictions on data.
• DevOps
– DevOps (a clipped compound of "software DEVelopment" and "information technology
OPerationS") is a term used to refer to a set of practices that emphasize the collaboration and
communication of both software developers and information technology (IT) professionals
while automating the process of software delivery and infrastructure changes.
• Systems Management or IT Operations Management
– IT Operations is responsible for the smooth functioning of the infrastructure and operational
environments that support application deployment to internal and external customers,
including the network infrastructure; server and device management; computer operations;
IT infrastructure library (ITIL) management; and help desk services for an organization.

6. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Program Agenda
Defining terms
Why (Dev)Ops is perfect for machine learning
Making Machine Learning Smarter
Q&A
1
2
3
4

7. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
• Development is creating
faster…
≥ Low-code
≥ Agile
≥ Microservices
≥ CI
• (Dev)Ops is promoting
faster…
≥ Containers
≥ IaaS & PaaS
≥ CD
≥ Packages
• (the rest of)Ops is not
moving any faster…
≥ #(*^(#^#)&^$(@^@($^
$(@)%&^$^**&^)!!!!
≥ #(*^(#^#)&^$(@^@($^
$(@)%&^$^**&^)!!!!
≥ …
We have a problem: Dev has outpaced Ops

8. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
OPTION 1:
Your Changes Don’t Hit Production Until Ops is Ready
Confidential – Oracle Internal/Restricted/Highly Restricted 8
Option 2:
You Promote Unmanaged Code Anyway
One of Two Likely Outcomes, Both Bad

9. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
It's not my
machines, it's
your code!
It's not my
code, it's your
machines!
Where’s the data?
9
What does the data mean?
The Reason: Ops Depends on Human Effort
END USER
EXPERIENCE
APPLICATION
MIDDLE TIER
DATA TIER
VIRTUALIZATION
TIER
VM CONTAINER
INFRASTRUCTURE
TIER
VM CONTAINER
Real Users
Synthetic Users
App metrics
Transactions
Server metrics
Diagnostics
Logs
Host metrics
VM metrics
Container metrics
CMDB
Tickets
Alerts
END USER
EXPERIENCE
APPLICATION
MIDDLE TIER
DATA TIER
VIRTUALIZATION
TIER
VM CONTAINER
INFRASTRUCTURE
TIER
VM CONTAINER
Real Users
Synthetic Users
App metrics
Transactions
Server metrics
Diagnostics
Logs
Host metrics
VM metrics
Container metrics
CMDB
Tickets
Alerts
END USER
EXPERIENCE
APPLICATION
MIDDLE TIER
DATA TIER
VIRTUALIZATION
TIER
VM CONTAINER
INFRASTRUCTURE
TIER
VM CONTAINER
Real Users
Synthetic Users
App metrics
Transactions
Server metrics
Diagnostics
Logs
Host metrics
VM metrics
Container metrics
CMDB
Tickets
Alerts
END USER
EXPERIENCE
APPLICATION
MIDDLE TIER
DATA TIER
VIRTUALIZATION
TIER
VM CONTAINER
INFRASTRUCTURE
TIER
VM CONTAINER
Real Users
Synthetic Users
App metrics
Transactions
Server metrics
Diagnostics
Logs
Host metrics
VM metrics
Container metrics
CMDB
Tickets
Alerts
END USER
EXPERIENCE
APPLICATION
MIDDLE TIER
DATA TIER
VIRTUALIZATION
TIER
VM CONTAINER
INFRASTRUCTURE
TIER
VM CONTAINER
Real Users
Synthetic Users
App metrics
Transactions
Server metrics
Diagnostics
Logs
Host metrics
VM metrics
Container metrics
CMDB
Tickets
Alerts
END USER
EXPERIENCE
APPLICATION
MIDDLE TIER
DATA TIER
VIRTUALIZATION
TIER
VM CONTAINER
INFRASTRUCTURE
TIER
VM CONTAINER
Real Users
Synthetic Users
App metrics
Transactions
Server metrics
Diagnostics
Logs
Host metrics
VM metrics
Container metrics
CMDB
Tickets
Alerts
END USER
EXPERIENCE
APPLICATION
MIDDLE TIER
DATA TIER
VIRTUALIZATION
TIER
VM CONTAINER
INFRASTRUCTURE
TIER
VM CONTAINER
Real Users
Synthetic Users
App metrics
Transactions
Server metrics
Diagnostics
Logs
Host metrics
VM metrics
Container metrics
CMDB
Tickets
Alerts
END USER
EXPERIENCE
APPLICATION
MIDDLE TIER
DATA TIER
VIRTUALIZATION
TIER
VM CONTAINER
INFRASTRUCTURE
TIER
VM CONTAINER
Real Users
Synthetic Users
App metrics
Transactions
Server metrics
Diagnostics
Logs
Host metrics
VM metrics
Container metrics
CMDB
Tickets
Alerts

10. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. 10
Oracle Management Cloud
INTELLIGENT,
UNIFIED PLATFORM
POWERED BY
MACHINE
LEARNING
INFORMED BY
A COMPLETE
DATA SET
HETEROGENEOUS
AND OPEN
APPLICATION
MIDDLE TIER
DATA TIER
VIRTUALIZATION
TIER
INFRASTRUCTURE
TIER
END USER
EXPERIENCE / ACTIVITY
Unified Platform
Global threat feeds
Cloud access
Identity
Real users
Synthetic users
App metrics
Transactions
Server metrics
Diagnostics logs
Host metrics
VM metrics
Container metrics
Configuration
Compliance
Tickets & Alerts
Security & Network
events

11. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. | 11
Ops Data is Perfect for Machine Learning
✓Massive volume
✓Highly patterned
✓Predictable format
✓Silos can be unified
✓Seasonal trends
✓Known sources

12. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Algorithmic Approaches to IT Ops Data
ANOMALY DETECTION
CLUSTERING
PREDICTION
CORRELATION
12

13. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Program Agenda
Defining terms
Why (Dev)Ops is perfect for machine learning
Making Machine Learning Smart for IT Ops
Q&A
1
2
3
4

14. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
ML is not smart out of the box
for every question
To make ML smarter, know the
questions you want to ask, then…
1. Enhance Algorithms
2. Increase Breadth
Maturing Machine Learning: A Three-Step Approach

15. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Increase Breadth With Data Unification & Normalization
Oracle Management Cloud Data Store
Log Analytics
IT Analytics
Infrastructure
Monitoring ComplianceOrchestration
Security
Monitoring &
Analytics
Application
Performance
Monitoring
Convert to Time Series
(Clustering & Rollup)
Base Lining &
Anomaly Detection
• Norm is repo by repo
projects: slow and
incremental.
• By centralizing data, we
are able to deliver ML
driven features more
quickly.

16. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Increase Depth With Context, Topology, & Domain Expertise
Context:
Forecasted
SLA violation
& observe
divergent
correlation.
Topology:
Tells us where
to look.
Domain
Expertise:
Allows to
identify root
cause.

17. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Applied Machine Learning for IT Operation Management
1. Metric data is automatically
baselined and logs records are
enriched
2. ITOM specific algorithms and models
are provided out of the box. User
input can be provided to further tune
the algorithms.
3. Operationalization and automation
using event processing for
notifications and remedial
orchestration actions
Oracle Confidential – Internal/Restricted/Highly Restricted17
• Is increased load anomalous or
expected at this time of the day
• Early warning for future outages or SLA
violations
• Abnormal and rare system behavior
• Capacity planning , WhatIf Analysis
• All metric and log data is continually used to
train the models
• Corrective Actions: automated scale out,
system restart, trigger diagnostic dump,
revert configuration changes
• Notification: Send alerts/notifications
through a variety of channels
• Incident: Create incident in 3rd party
ticketing system, update status, attach
evidence

18. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
APM AJAX Calls Anomalies
• Ajax call metrics baselined and anomalies identified
• Alert rules and corrective actions can be taken for anomalies

19. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Anomaly-based Alerts
Alert when CPU Utilization(%)
is anomalous
Confidential – Oracle Internal/Restricted/Highly Restricted 19
Leverage Machine Learning

20. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Alert Notifications
Confidential – Oracle Internal/Restricted/Highly Restricted 20

21. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Applied Machine Learning for Security and Compliance
1. Security log events are auto-enriched
with user, asset, and threat
intelligence context
2. Threat specific algorithms and
security models are provided out of
the box
3. Operationalization and automation
using event processing for
notifications and remedial SOC
playbook execution
Oracle Confidential – Internal/Restricted/Highly Restricted21
• Is the user privileged?
• Is the asset regulated?
• Is an accessed URL malicious?
• Users coming from anomalous IPs
• Users executing anomalous SQL queries
• Assets with anomalous configuration drift
• Identity: password reset, multi-factor
authentication, privilege change
• Asset: AV scan, endpoint data collection,
configuration change
• Incident: Create incident in 3rd party
ticketing system, update status, attach
evidence

22. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
• Common attributes in user actions are
automatically and individually baselined from
enriched security activity logs
• Anomalies from self and/or peer group activity
baselines per attribute are automatically
flagged using prebuilt machine learning models
• Behavioral threat specific remediation jobs are
available for automatic or SOC analyst guided
execution
– URL blocking, identity actions, host isolation,
endpoint data collection, incident management,
firewall rule updates etc.
22
Example: Suspicious User Activity
• Alice is executing actions from an IP
address anomalous to her source IP
baseline.
• Bob is accessing internal assets anomalous
for the sales team that he is part of.
• Neil is accessing critical assets at a time of
day that neither he nor his peer group are
not normally active.

23. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
DEMO: Matured Machine Learning in Action

24. Copyright © 2017,Oracle and/orits affiliates. All rights reserved. |
Key Takeaways
• DevOps depends on “Ops” speed
matching “Dev” speed
• The DevOps problem is well-suited to
machine learning
BUT…
• Machine Learning must be matured
• Unified data and context increases the
effectiveness of ML and analysis
Confidential – Oracle Internal/Restricted/Highly Restricted 24

25. Copyright © 2017,Oracle and/orits affiliates. All rights reserved.
Program Agenda
Defining terms
Why (Dev)Ops is perfect for machine learning
Making Machine Learning Smarter
Q&A
1
2
3
4

26. Copyright © 2017,Oracle and/orits affiliates. All rights reserved.
TRY IT FREE FOR 30 DAYS
cloud.oracle.com/tryit
oracle.com/managementcloud

27. Copyright © 2017,Oracle and/orits affiliates. All rights reserved.
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.