The document discusses the role of analytics and AI in enhancing digital businesses, emphasizing their impact on optimizing customer experience and generating new revenue streams. It highlights the importance of key performance indicators (KPIs) and control loops in data collection and processing, utilizing technologies such as streaming SQL and machine learning. The WSO2 Stream Processor is presented as a comprehensive solution for building robust digital business frameworks that leverage streaming data and analytics for real-time decision-making.

1. Architect, WSO2
Anjana Fernando
Deriving Insights for Your Digital
Business with Analytics

2. 2https://en.wikipedia.org/wiki/Greyball
Uber Greyball

3. Darkside
3

4. 4
It is impressive, why can’t we do it
Can we do that?

5. Digital Business
5

6. Digital Building Blocks (Pods)
6

7. Digital Business with Pods
7

8. 8
It is impressive, why can’t we do it
AI &
Analytics will
be the next
defensible
Moat
In business, I look for
economic castles
protected by unbreachable
'moats'.” -Warren Buffett.

9. Analytics in a Digital Business
9

10. New Digital inspired
Products and Revenue
Streams
• New way to do business (e.g.
Uber, Amazon Go)
• Product as a Service (e.g. IoT Jack
hammer, Light as a service)
• Progressive Insurance Gadget
• Sell insights ( Telcos knows where
people are, credit card companies
know what people buy and their
demographics, navigation apps
know traffic)

11. Get Close to your
Customers
• Use analytics to optimize the
experience
• Predict issues and proactively
handle them ( e.g. reschedule
automatically when flight has
missed)
• Predict churn and act
• Track the brand and manage it
• Target your marketing

12. Optimizations
• Reduce Fraud
• Logistics, day to day operations
• Analytics for hiring and
Performance appraisal
• Predictive maintenance
• Sales analytics, demand
prediction
• Security and surveillance

13. Programming Pod Control
Loops
13

14. • Understand the KPIs
• Collect data as Streams
• Analyze them using
Streaming SQL +Machine
Learning + AI
• Expose them as APIs,
Alerts, Dashboards
• Recompose them
Pod Programming Model
14

15. • A single Number that represent
performance of something
• E.g. Revenue per employee, Revenue per
sqft, conversion rate, life time value, GDP,
per capita income, HDI index
• Good KPIs are used in pairs, one that
represents quality and one that represent
quantity
– e.g. closed support issues count &
reopened support issues)
– Per capita income & HDI
KPI (Key Performance Indicators)
15

16. • Data represented as Streams
• Simple API to create Streams
and publish events to them (
via HTTP, JMS, Kafka)
• WSO2 Products already
publish many data streams
Data Collection
16

17. Control Loops are Real-time
● Control loops are
fundamentally realtime
● Batch plays a helper
function

18. 1.
Why?
● Easy for people who knows
SQL
● Expressive, short, sweet and
fast!!
● define core operations that
covers 90% of problems
● Let experts dig in when they
like by writing extensions!
● Query engine can better
optimize the executions with
Streaming SQL model.
Just like we query
data in a DB with
SQL, we query
moving data with
“Streaming SQL”

19. Streaming SQL Concepts
Filtering and Transformations just
like SQL
from Productions[amount>10] select
.. insert into ..
Windows controls how much data to
keep in memory
from Productions#window.time(1h)
group by factoryId,name
select .. insert into ..
Multiple Streams can be joined to
reason across streams
from RawMateralsH#window.time(1h)
join ProductionsH
on ..
Select .. insert into ..
Patterns let you detect complex
events across time
Select bid, ts as T From
Productions as b1
->Productions[b1.a <a]
Select .. insert into ..

20. Machine learning
• Given examples build a program
that matches those examples
• We call that program a “model”
• Major improvements in last few
years (e.g. deeplearning)
Can you “Write a program to drive a Car?”
AI & Machine Learning

21. Batch ML Algorithms
● Build a model with a ML
tool (e.g. Spark),
● Export as PMML or native
model
● Use in the Streaming SQL
Streaming ML Algorithms
● Markov Chains
● Regression
● Clustering

22. Anomaly Detection Operators
• Anomaly Detection is common use case in a
control loop
• WSO2 SP include several operators (e.g. Using
Clustering, Classification, Markov Chains)
White paper: Fraud Detection
and Prevention: A Data
Analytics Approach
Image "Reading" by Creative Stall (cc), Noun Project

23. define aggregation RawMaterialAgr
from RawMaterialStream
select name, sum(amount) ..
group by name
aggregate every min … year
Batch with Incremental Analytics
• Use of batch in most “Digital
business” use cases are
incremental
• SP Includes a incremental
operator that works in 90% of
time
• Also support Spark integration
Current Min
Current Hour
Sec
Min
Hour
0 - 1 - 5 ...
- 1
- 2 - 3 - 4 - 64 - 65 ...
- 2
- 124

24. Alerts
• Done through Streaming SQL queries
• Notifications ( sent via email, SMS,
Pager etc.)
• Goal is to give you peace of mind (
not having to check all the time)
• They should be specific and
infrequent
• They should have very low false
positives
• Let users control sensitivity

25. What is a Dashboard?
• Think a car dashboard
• It give you idea about
overall system in a glance
• It is boring when all is
good, and grab attention
when something is
wrong
• Support for drill down
and find root cause

26. • When data cross security
domains, there are
security and management
concerns
• APIs ( e.g. WSO2 APIM)
solve these problems
Often data are
accessed through the
network
– Mobile Apps
– Query interfaces
– Data integration
– As a Subscription
Expose data through API

27. •
WSO2 Stream Processor
27
27
Realtime
Batch
Machine Learning
Streaming Complex Event Processing
Streaming Incremental Time Series
Aggregations
Streaming Machine Learning
Batch Machine Learning
+ Edge Analytics

28. • Give your everything you need to build control loops
for your Pods
• Provide Streams, a powerful Streaming SQL
language, Dashboards and more
• Can provide 100K+ events per second with HA only
with two nodes ( most alternatives need 5+) and can
scale more on top of Kafka
Why WSO2 Stream Processor?
28

29. • Digital Business give you
ultimate flexibility
• Analytics & AI are key part of
Digital Business
• We envision a Digital Business
Platform composed with
Adaptable Pods
• We discussed how to build the
control loops, which provides
“intelligence”
• WSO2 SP can give you
everything you need!!
Summary
29
Image used with CC licence from
https://www.flickr.com/photos/breville/8734805199

30. Questions?
30

31. wso2.com
31