Big data analytics play a key role in the retail industry where common use cases include product recommendations, targeted marketing, dynamic offers, product selection, and A/B testing based on product placement. Most current systems use batch processing technologies like MapReduce to process transaction logs a few hours later and extract useful insights. Although such insights are useful for trends that last a long time, they often come in too late. On one hand, the store cannot act on any short term trends (e.g. very common in auction use cases) and even long-term trends would have progressed by the time action is taken. This session will focus on a sample system and explain in detail how a retail store can incorporate real-time updates into recommendation, product offers, and other analytics using WSO2 Complex Event Processor. Our system can react to short-term trends, provide dynamic offers within milliseconds, and react immediately to any new trends.

1. Catch Them in the Act:
CEP for Realtime
Ecommerce Influence
Srinath Perera
Director Research WSO2, Apache Member
(@srinath_perera)
srinath@wso2.com

2. Retail
World 7B People
- 6.5B have a phone
- 2.5B online
Global E-commerce itself was 1.5
Trillion market (215$/year for
everyone in the world).

4. Recommendations
One of the main retail
scenarios, can be extended
to targeted marketing and
deals
Search let us make sense of
unstructured Web
In retail, there are too many
choices, Recommendations
help us make a decision.
A Main driver of traffic (e.g.
70% amazon site is devoted
for recommendations, 70%
Netflix sales though

5. Finding Recommendation
Problem: based on customer activities (purchases, ratings,
views) etc., find out products that are of interest to him
- Done by finding similar items or similar users (common case)
There are often Millions of customers and even millions of
items

6. Recommendations (Collaborative Filtering)

7. What if you can freeze time!!
Most solutions are overnight
Think how you would buy something!
Research a bit and buy, often overnight is
too late.
But not all trends takes time
- People change their mind
- Trends move fast
- React to what customer is doing (do not let
him move away)
At CEP speed 400k/sec, if each event
takes a second, it takes 4 days to pass a
second in real world!!

9. Real-time Analytics
Idea is to process data as they are received
in streaming fashion
Used when we need
- Very fast output
- Lots of events (few 100k to millions)
- Processing without storing (e.g. too much
data)
Two main technologies
- Stream Processing (e.g. Strom,
http://storm-project.net/ )
- Complex Event Processing (CEP)
http://wso2.com/products/complex-event-
processor/

10. Complex Event Processing

11. Lambda Architecture with WSO2 Products

12. WSO2 Big Data Platform

13. Demo Usecases
Operations Dashboard
Automatic Offers
Recommendations based
on personal trends
Recommendations based
on overall trends

14. Demo Usecase Architecture

16. Dashboard
Dashboard to show Realtime Analytics about the site
Example query (e.g. number of purchases or views for each
tag last minute)
from transactionStream[metadata== 'purchase'
or metadata== 'select']#window.time(1 min)
select tag, count(*) as count group by tag
update tagInfo on tagInfo.tag == transactionStream.tag;

18. Automatic Offers
If user have brought books worth more than 50$ within a
day, given 25% off on the next book
from SalesStream#window.time(1d)
select sum(price) as total having total > 50
insert into ..

20. Recommendations based on
personal trends
Look at recent history of the user, and find items he has
viewed in last day
Find a user close to his recent items, and recommend
accordingly ( a listener to Recommendations Stream will do
that)
from transactionStream_user#window.time(7d)
select session_id, user_id,
custom:genLSHID(user_id,book_id,metadata) as book_id,
"user" as metadata, "0" as discount
insert into RecommendationsStream;

21. Recommendations based on
overall trends
Use items sold it last 10 minutes
Find a user close to his recent items, and recommend
accordingly ( a listener to TrendsOutputStream will
do that)
from trendTransactionStream#window.time(10 min)
select session_id, user_id,
custom:genLSHID("trend",book_id,metadata) as book_id,
"trend" as metadata
insert into TrendsOutputStream;

22. Questions?