Independent of the source of data, the integration and analysis of event streams gets more important in the world of sensors, social media streams and Internet of Things. Events have to be accepted quickly and reliably, they have to be distributed and analysed, often with many consumers or systems interested in all or part of the events. So far this mostly a development experience, with frameworks such as Oracle Event Processing, Apache Storm or Spark Streaming. With Oracle Stream Analytics, analytics on event streams can be put in the hands of the business analyst. It simplifies the implementation of event processing solutions so that every business analyst is able to graphically and decleratively define event stream processing pipelines, without having to write a single line of code or continous query language (CQL). Event Processing is no longer “complex”! This session presents Oracle Stream Analytics directly on some selected demo use cases.

1. BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA
HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH
Oracle Stream Analytics
Simplifying Stream Processing
29.9.2016 – DOAG 2016 Big Data Days
Guido Schmutz

2. Guido Schmutz
Working for Trivadis for more than 19 years
Oracle ACE Director for Fusion Middleware and SOA
Co-Author of different books
Consultant, Trainer, Software Architect for Java, SOA & Big Data / Fast Data
Member of Trivadis Architecture Board
Technology Manager @ Trivadis
More than 25 years of software development experience
Contact: guido.schmutz@trivadis.com
Blog: http://guidoschmutz.wordpress.com
Slideshare: http://www.slideshare.net/gschmutz
Twitter: gschmutz
Oracle Stream Analytics - Simplifying Stream Processing2

3. Unser Unternehmen.
Oracle Stream Analytics - Simplifying Stream Processing3
Trivadis ist führend bei der IT-Beratung, der Systemintegration, dem Solution
Engineering und der Erbringung von IT-Services mit Fokussierung auf -
und -Technologien in der Schweiz, Deutschland, Österreich und
Dänemark. Trivadis erbringt ihre Leistungen aus den strategischen Geschäftsfeldern:
Trivadis Services übernimmt den korrespondierenden Betrieb Ihrer IT Systeme.
B E T R I E B

4. KOPENHAGEN
MÜNCHEN
LAUSANNE
BERN
ZÜRICH
BRUGG
GENF
HAMBURG
DÜSSELDORF
FRANKFURT
STUTTGART
FREIBURG
BASEL
WIEN
Mit über 600 IT- und Fachexperten bei Ihnen vor Ort.
Oracle Stream Analytics - Simplifying Stream Processing4
14 Trivadis Niederlassungen mit
über 600 Mitarbeitenden.
Über 200 Service Level Agreements.
Mehr als 4'000 Trainingsteilnehmer.
Forschungs- und Entwicklungsbudget:
CHF 5.0 Mio.
Finanziell unabhängig und
nachhaltig profitabel.
Erfahrung aus mehr als 1'900 Projekten
pro Jahr bei über 800 Kunden.

5. Agenda
1. Introduction to Streaming Analytics
2. Oracle Stream Analytics
3. Demo
Oracle Stream Analytics - Simplifying Stream Processing5

6. Introduction to Streaming Analytics
Oracle Stream Analytics - Simplifying Stream Processing6

7. Traditional Data Processing - Challenges
• Introduces too much “decision latency”
• Responses are delivered “after the fact”
• Maximum value of the identified situation is lost
• Decision are made on old and stale data
• “Data a Rest”
Oracle Stream Analytics - Simplifying Stream Processing7

8. The New Era: Streaming Data Analytics / Fast Data
• Events are analyzed and processed in
real-time as the arrive
• Decisions are timely, contextual and
based on fresh data
• Decision latency is eliminated
• “Data in motion”
Oracle Stream Analytics - Simplifying Stream Processing8

9. Event / Stream Processing Architecture
Data
Ingestion
Batch
compute
Data
Sources
Channel
Data
Consumer
Reports
Service
Analytic
Tools
Alerting
Tools
Content
Logfiles
Social
RDBMS
ERP
Sensor
Machine
(Analytical) Real-Time Data Processing
Stream/Event Processing
Result Store
Messaging
Result Store
Oracle Stream Analytics - Simplifying Stream Processing
= Data in Motion = Data at Rest
9

10. “Lambda Architecture” for Big Data
Data
Ingestion
(Analytical) Batch Data Processing
Batch
compute
Result StoreData
Sources
Channel
Data
Consumer
Reports
Service
Analytic
Tools
Alerting
Tools
Content
RDBMS
Social
ERP
Logfiles
Sensor
Machine
(Analytical) Real-Time Data Processing
Stream/Event Processing
Batch
compute
Messaging
Result Store
Query
Engine
Result Store
Computed
Information
Raw Data
(Reservoir)
Oracle Stream Analytics - Simplifying Stream Processing
= Data in Motion = Data at Rest
Pulling
Ingestion
10

11. When to Stream / When not?
Oracle Stream Analytics - Simplifying Stream Processing11
Constant low
Milliseconds & under
Low milliseconds to seconds,
delay in case of failures
10s of seconds of more,
Re-run in case of failures
Real-Time Near-Real-Time Batch

12. “No free lunch”
Oracle Stream Analytics - Simplifying Stream Processing12
Constant low
Milliseconds & under
Low milliseconds to seconds,
delay in case of failures
10s of seconds of more,
Re-run in case of failures
Real-Time Near-Real-Time Batch
“Difficult” architectures, lower latency “Easier architectures”, higher latency

13. Why Event / Stream Processing?
Oracle Stream Analytics - Simplifying Stream Processing13
Visualize Business in real-time
• Dashboards can help people to visualize, monitor and make sense of massive amount of
incoming data in real-time
Detect Urgent Situations
• Based on simple or complex analytical patterns of urgent business events
• Urgent because they happen in real-time
Automate immediate actions
• Run in the background quietly until detecting an urgent situation (risk or opportunity)
• Alerts can go to humans through email, text or push notifications or to other applications trough
message queues or service call

14. Oracle Stream Analytics - Simplifying Stream Processing15

15. Oracle Stream Analytics
Oracle Stream Analytics - Simplifying Stream Processing16

16. History of Oracle Stream Analytics
Oracle Complex Event
Processing (OCEP)
Oracle Event Processing (OEP)
Oracle Stream Explorer (SX)
Oracle Event Processing
for Java Embedded
Oracle Stream Analytics (OSA)
Oracle Edge Analytics (OAE)
BEA Weblogic Event Server
Oracle CQL
Oracle IoT Cloud Service
2016
2015
2007
2008
2012
2013
Oracle Stream Analytics - Simplifying Stream Processing17

17. OEA
• Filtering
• Correlation
• Aggregation
• Pattern
matching
Devices /
Gateways
Services
Computing Edge Enterprise
“Sea of data”
Macro-event
High-value
Actionable
In-context
EDGE
Analytics
Stream
Analytics
FOG
• High Volume
• Continuous Streaming
• Extreme Low Latency
• Disparate Sources
• Temporal Processing
• Pattern Matching
• Machine Learning
Oracle Stream Analytics: From Noise to Value
• High Volume
• Continuous Streaming
• Sub-Millisecond Latency
• Disparate Sources
• Time-Window Processing
• Pattern Matching
• High Availability / Scalability
• Coherence Integration
• Geospatial, Geofencing
• Big Data Integration
• Business Event Visualization
• Action!
Oracle Stream Analytics - Simplifying Stream Processing18

18. Oracle Stream Analytics Platform
What it does
• Compelling, friendly and visually stunning real time
streaming analytics user experience for Business users to
dynamically create and implement Instant Insight solutions
Key Features
• Analyze simulated or live data feeds to determine event
patterns, correlation, aggregation & filtering
• Pattern library for industry specific solutions
• Streams, References, Maps & Explorations
Benefits
• Accelerated delivery time
• Hides all challenges & complexities of underlying real-time
event-driven infrastructure
Oracle Stream Analytics - Simplifying Stream Processing19

19. Oracle Stream Analytics – Self-Service Stream
Processing!
Understanding of CQL Filtering, Correlation, Pattern: NOT NEEDED
Understanding of IT Deployment and Management: NOT NEEDED
Understanding of Development, Java, Best Practices: NOT NEEDED
Understanding of the Event Driven Platform: NOT NEEDED
Oracle Stream Analytics - Simplifying Stream Processing20

20. Oracle Stream Analytics – Terminology
Explorer: The Application User Interface Catalog: The repository for browsing resources
Oracle Stream Analytics - Simplifying Stream Processing21

21. Oracle Stream Analytics – Terminology
Stream: incoming flow of events that you
want to analyze (CSV, Kafka, JMS, Rest,
MQTT, …)
Exploration: application that correlates events
from streams and data sources, using filters,
groupings, summaries, ranges, and more
Oracle Stream Analytics - Simplifying Stream Processing22

22. Oracle Stream Analytics – Terminology
Shape: A blueprint of an event in a stream or
data in a data source. How the business data
is represented in the selected stream
Map: collection of geo-fences
Reference: A connection to static data that is
joined to a stream to enrich it and/or to be used in
business logic and output
Oracle Stream Analytics - Simplifying Stream Processing23

23. Oracle Stream Analytics – Terminology
Pattern: A pre-built Exploration that
addresses a particular business scenario in a
focused and simplified User Interface
Connection: collection of metadata required to
connect to an external system
Targets: defines an interface with a downstream
system
Oracle Stream Analytics - Simplifying Stream Processing24

24. Business accessibility to Geo-Streaming Analytics
Real Time Streaming Solutions face an increasing need to track "assets of interest" and
initiate actions based on encroachment of boundary proximity to fixed and moving
objects and other geographic, temporal, or event conditions.
Geo-Fence, Fence, Polygon
Geo-Streaming
Oracle Stream Analytics - Simplifying Stream Processing25

25. “ Add value to your real time streaming data discovery and analytics by applying and including
mathematical, statistical analysis to the live output stream”
“These streaming “Excel spreadsheets” really do come to life”
Expression Builder enabling calculations
Oracle Stream Analytics - Simplifying Stream Processing26

26. Concept of Connections and their reuse in Streams
Oracle Stream Analytics - Simplifying Stream Processing27

27. Decision Table for Nested IF-THEN-ELSE Rules
Oracle Stream Analytics - Simplifying Stream Processing28

28. Topology View and Navigation
Oracle Stream Analytics - Simplifying Stream Processing29

29. Relationship between Streams (Sources), References
and Explorations
Oracle Stream Analytics - Simplifying Stream Processing30

30. Demo
Oracle Stream Analytics - Simplifying Stream Processing31

31. Oracle Stream Analytics Demo Use Case: Truck
Movements
Truck
Data
Ingestion
Geo-Fencing
2016-06-02 14:39:56.605|98|27|Mark
Lochbihler|803014426|Wichita to
Little Rock Route 2|Normal|38.65|-
90.21|5187297736652502631
{"timestamp": "2016-06-02
14:39:56.991", "truckId": 99,
"driverId": 31, "driverName":
"Rommel Garcia", "routeId":
1565885487, "routeName":
"Springfield to KC Via Hanibal",
"eventType": "Normal", "latitude":
37.16, "longitude": "-94.46",
"correlationId":
5187297736652502631}
Reckless Driving
Detector
NEAR
ENTER
Truck
Driver
DashboardMovement Movement
JSON
Reckless
Driver
Oracle Stream Analytics - Simplifying Stream Processing32

32. Continuous Ingestion in Stream Processing
DB Source
Big Data
Log
Stream
Processing
IoT Sensor
Event Hub
Topic
Topic
REST
Topic
IoT GW
CDC GW
Connect
CDC
DB Source
Log CDC
Native
IoT Sensor
IoT Sensor
33
Dataflow GW
Topic
Topic
Queue
MQTT GW
Topic
Dataflow GW
Dataflow
TopicREST
33
File Source
Log
Log
Log
Social
Native
Oracle Stream Analytics - Simplifying Stream Processing33
Topic
Topic

33. Apache Kafka – High-volume messaging system
Distributed publish-subscribe messaging system
Designed for processing of high-volume, real
time activity stream data (logs, metrics
collections, social media streams, …)
Topic Semantic
does not implement JMS standard!
Initially developed at LinkedIn, now part of
Apache
Kafka Cluster
Consumer Consumer Consumer
Producer Producer Producer
Oracle Stream Analytics - Simplifying Stream Processing34

34. Demo: Oracle Stream Analytics
Oracle Stream Analytics - Simplifying Stream Processing35

35. Demo: Oracle Stream Analytics
Oracle Stream Analytics - Simplifying Stream Processing36

36. Demo: Oracle Stream Analytics
Oracle Stream Analytics - Simplifying Stream Processing37

37. Demo: Oracle Stream Analytics
Oracle Stream Analytics - Simplifying Stream Processing38

38. Summary
Oracle Stream Analytics - Simplifying Stream Processing39

39. Native Stream Processing => OEP server
Ingestion
Event
Source
Event
Source
Event
Source
Oracle Stream Analytics - Simplifying Stream Processing40
Individual Event
PPPPPPPPPPPP

40. Micro-Batch Stream Processing => Spark Streaming
Ingestion
Event
Source
Event
Source
Event
Source
Oracle Stream Analytics - Simplifying Stream Processing41
PPPPPP

41. Summary
Oracle Stream Analytics leverages the capabilities found in Oracle Event Processing
(OEP)
Empowering Business users to gain insight into real-time information and take
appropriate actions when needed => makes stream processing accessible
Makes Stream/Event Processing less technical => “Excel spread sheet” on Streams
Part of Oracle IoT Cloud Service
Support Spark Streaming as a deployment platform for Streaming ML
Interesting road map: Rule Engine, Machine Learning, Extensible Patterns
Oracle Stream Analytics - Simplifying Stream Processing42

42. Oracle Stream Analytics on Docker
Oracle Stream Analytics 12.2.1 Documentation
Oracle Stream Analytics 12.2.1 Download
Oracle Stream Analytics - Simplifying Stream Processing44

43. Guido Schmutz
Technology Manager
guido.schmutz@trivadis.com
Oracle Stream Analytics - Simplifying Stream Processing45