Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
● Complex Event Processing
What ?
Why ?
● WSO2 Complex Event Processor
Scalability
Edge Computing
● Usecases
● What’s new ...
● Amount of data you analyse are big
● You don’t need data for long time
● You don’t want to store all the data
● Time cri...
• Filters & Transformations
• Windows and Aggregations
• Joins
• Sequences and Patterns
• Event Tables
Stateless
• Filters & Transformations
Stateful Inmemory
• Windows and Aggregations
• Joins
• Sequences and Patterns
Statef...
• Vertical Scaling
• Horizontal Scaling
E.g. Calculate Mean
E.g. Calculate Mean
E.g. Calculate Mean
How about scaling median ?
How about scaling median ?
If & only if we can partition !
Spark Streaming
• Supports distributed processing
• Runs micro batches
• Not supports pattern & sequence detection
Spark Streaming
• Supports distributed processing
• Runs micro batches
• Not supports pattern & sequence detection
Apache ...
Advantages
• Supports distributed processing
• Supports Partitioning
• Extendable
• Open Source
Disadvantages
• Need to wr...
Advantages
• Supports distributed processing
• Supports Partitioning
• Extendable
• Open Source
Disadvantages
• No need to...
define stream StockStream
(symbol string, volume int, price double);
@name(‘Filter Query’)
from StockStream[price > 75]
se...
define stream StockStream
(symbol string, volume int, price double);
@name(‘Filter Query’)
from StockStream[price > 75]
se...
define stream StockStream
(symbol string, volume int, price double);
@name(Filter Query’)
@dist(parallel= ‘3')
from StockS...
Siddhi CEP
Siddhi CEP
• System Monitoring (Logs, Servers)
• Finance (Financial trading)
• Banking (Credit card fraud detection)
• Marketing (Loc...
• Distributed processing with Apache Storm
• Improved High Availability
• Improved Siddhi Query Language
• Toolbox Support...
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Scalable Event Processing with WSO2CEP @  WSO2Con2015eu
Upcoming SlideShare
Loading in …5
×

Scalable Event Processing with WSO2CEP @ WSO2Con2015eu

1,428 views

Published on

Scalable Event Processing with WSO2CEP 4.0, Describing how it can scale using Apache Storm.

Published in: Data & Analytics
  • Be the first to comment

Scalable Event Processing with WSO2CEP @ WSO2Con2015eu

  1. 1. ● Complex Event Processing What ? Why ? ● WSO2 Complex Event Processor Scalability Edge Computing ● Usecases ● What’s new is WSO2 CEP 4.0 ?
  2. 2. ● Amount of data you analyse are big ● You don’t need data for long time ● You don’t want to store all the data ● Time critical analysis ○ Finding opportunities before they’re gone ○ Identify threats beforehand
  3. 3. • Filters & Transformations • Windows and Aggregations • Joins • Sequences and Patterns • Event Tables
  4. 4. Stateless • Filters & Transformations Stateful Inmemory • Windows and Aggregations • Joins • Sequences and Patterns Stateful Persisted • Event Tables
  5. 5. • Vertical Scaling • Horizontal Scaling
  6. 6. E.g. Calculate Mean
  7. 7. E.g. Calculate Mean
  8. 8. E.g. Calculate Mean
  9. 9. How about scaling median ?
  10. 10. How about scaling median ? If & only if we can partition !
  11. 11. Spark Streaming • Supports distributed processing • Runs micro batches • Not supports pattern & sequence detection
  12. 12. Spark Streaming • Supports distributed processing • Runs micro batches • Not supports pattern & sequence detection Apache Storm • Supports distributed processing • Stream processing engine
  13. 13. Advantages • Supports distributed processing • Supports Partitioning • Extendable • Open Source Disadvantages • Need to write Java code • Need to start from basic principles ( & data structures ) • Adoption for change is slow • No support to govern artifacts
  14. 14. Advantages • Supports distributed processing • Supports Partitioning • Extendable • Open Source Disadvantages • No need to write Java code (Supports SQL like query language) • No need to start from basic principles (Supports high level language) • Adoption for change is fast • Govern artifacts using Toolboxes • etc ...
  15. 15. define stream StockStream (symbol string, volume int, price double); @name(‘Filter Query’) from StockStream[price > 75] select * insert into HighPriceStockStream ; @name(‘Window Query’) from HighPriceStockStream#window.time(10 min) select symbol, sum(volume) as sumVolume insert into ResultStockStream ;
  16. 16. define stream StockStream (symbol string, volume int, price double); @name(‘Filter Query’) from StockStream[price > 75] select * insert into HighPriceStockStream ; @name(‘Window Query’) partition with (symbol of HighPriceStockStream) begin from HighPriceStockStream#window.time(10 min) select symbol, sum(volume) as sumVolume insert into ResultStockStream ; end;
  17. 17. define stream StockStream (symbol string, volume int, price double); @name(Filter Query’) @dist(parallel= ‘3') from StockStream[price > 75] select * insert into HightPriceStockStream ; @name(‘Window Query’) @dist(parallel= ‘2') partition with (symbol of HighPriceStockStream) begin from HighPriceStockStream#window.time(10 min) select symbol, sum(volume) as sumVolume insert into ResultStockStream ; end;
  18. 18. Siddhi CEP Siddhi CEP
  19. 19. • System Monitoring (Logs, Servers) • Finance (Financial trading) • Banking (Credit card fraud detection) • Marketing (Location & time based advertising) • Logistics (Taxi services, Vehicle parking) • Healthcare • E-commerce (Online retailers) • Telecommunication (Mobile offers) • System automation (Auto scaling) • IoT (Smart home, Smart buildings) • ...
  20. 20. • Distributed processing with Apache Storm • Improved High Availability • Improved Siddhi Query Language • Toolbox Support • Realtime Dashboard • Websocket + MQTT Support And lots of extensions • Geo • Timeseries • NLP • PMML & WSO2 ML Try out CEP 4.0 Alpha : https://github.com/wso2/product- cep/releases

×