Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
An Introduction to the WSO2
Analytics Platform
Sriskandarajah Suhothayan (Suho)
Technical Lead
WSO2
Why Analytics?
Why Analytics?
Reduce cost and increase revenue
Improve customer satisfaction
Improve process efficiency
Track progress ov...
WSO2 Data Analytics Platform
Collect Data
Collecting Data
* Supports custom event receivers via its pluggable architecture!
Store Data
Data Abstraction Layer enabling pluggable data
connectors
● Cassandra
● RDBMS
● HBase
● …
With a common API
Analysing
Interactive Analytics
What has happened & Why ?
● On demand analysis of data
○ Full text search support
● Selective deep a...
Batch Analytics
Summarize & Tack progress over time …
● Powered by Apache Spark
○ Up to 30x higher performance than Hadoop...
Realtime Analytics
Detect and act immediately …
● Streaming Complex Event Processing
○ Uses WSO2 Siddhi engine
○ Process m...
Predictive Analytics
Predict future & act upon it !
● Explore and build models
○ Via Spark MlLib & R (from WSO2 ML 1.1)
○ ...
Machine Learner
Communicate
Visualizing Results
Overall idea of what's going on …
● Dashboard using
○ Google Gadget
○ HTML5 + javascripts
● Support cu...
Gadget Generation Tool
Customize with few clicks …
● Support realtime and
stored data visualization
● How ?
○ Start with d...
Communicating Results
* Supports custom event publishers via its pluggable architecture!
Beyond Boundaries
● Expose analytics results
as API
○ Mobile Apps, Third Party
● Provides
○ Security, Billing,
○ Throttlin...
Developer Tools & Deployment
Developer Tools
● Event Flow
Developer Tools ...
● Event Flow
● Event Tracing
Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
● Siddhi Try It
Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
● Siddhi Try It
● Event Simulator
Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
● Siddhi Try It
● Event Simulator
● Spark Console
Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
● Siddhi Try It
● Event Simulator
● Spark Console
● Da...
Developer Tools ...
● Event Flow
● Event Tracing
● Event Statistics
● Siddhi Try It
● Event Simulator
● Spark Console
● Da...
● One Analytics Archive format for all analytics
○ Based to Carbon Archive
● Minimum HA Deployment (2 Nodes)
● Fully Distr...
Solutions
Solutions
● Fraud Detection
○ Alerts for known and
unknown frauds
○ Deep Search Analytics
○ https://goo.gl/TWV5C1
● IoT An...
Custom Solutions
UI to configure predefined
analysis with Execution Manager.
Good for Business Users
Use Cases & Case studies
Use Cases
● Smart Parking solutions
● Health & Financial Monitoring
● Smart City project, Vehicle tracking,
Building monit...
Case Studies
● Realtime Soccer Analytics (DEBS 2013)
○ https://goo.gl/c2gPrQ
● TFL Traffic Analysis
○ Using Transport for ...
How to get started ?
How to get started ?
● (WSO2 DAS) 3.0
○ Combines all types of analytics.
● (WSO2 CEP) 4.0
○ For who need to analyze realti...
WSO2 DAS vs WSO2 BAM
What's New ?
● Interactive & predictive analytics
● Data Abstraction Layer supporting
○ RDBMS, Cassan...
“The price of light is less than the cost of darkness”
~ Arthur C. Nielsen, Market Researcher & Founder of ACNielsen
Thank You
Upcoming SlideShare
Loading in …5
×

An introduction to the WSO2 Analytics Platform

641 views

Published on

Introducing WSO2 Analytics Platform 3.0, on WSO2Con 2015

Published in: Technology
  • Be the first to comment

  • Be the first to like this

An introduction to the WSO2 Analytics Platform

  1. 1. An Introduction to the WSO2 Analytics Platform Sriskandarajah Suhothayan (Suho) Technical Lead WSO2
  2. 2. Why Analytics?
  3. 3. Why Analytics? Reduce cost and increase revenue Improve customer satisfaction Improve process efficiency Track progress over time Act immediately Predict future opportunities and threats ...
  4. 4. WSO2 Data Analytics Platform
  5. 5. Collect Data
  6. 6. Collecting Data * Supports custom event receivers via its pluggable architecture!
  7. 7. Store Data Data Abstraction Layer enabling pluggable data connectors ● Cassandra ● RDBMS ● HBase ● … With a common API
  8. 8. Analysing
  9. 9. Interactive Analytics What has happened & Why ? ● On demand analysis of data ○ Full text search support ● Selective deep analysis ○ Drill down search ● Distributed data indexing. ● Near real time data indexing and retrieval message: “Error” AND date: [2015-11-01 TO 2015-11-03]
  10. 10. Batch Analytics Summarize & Tack progress over time … ● Powered by Apache Spark ○ Up to 30x higher performance than Hadoop ● Optimized in-memory processing ● Spark SQL SQL-like query language ● Schedulable Scripts ○ Simple Configs & Cron Expressions
  11. 11. Realtime Analytics Detect and act immediately … ● Streaming Complex Event Processing ○ Uses WSO2 Siddhi engine ○ Process more than 100k Events/Sec ● SiddhiQL SQL-like Query Language ● Extension Support (Geo, NLP, Math, String, Regex, … ) ● Distributed Processing with Apache Storm Siddhi
  12. 12. Predictive Analytics Predict future & act upon it ! ● Explore and build models ○ Via Spark MlLib & R (from WSO2 ML 1.1) ○ Export as PMML (from WSO2 ML 1.1) ● Run models with CEP, DAS and ESB. ● Run R Scripts, Regression and Anomaly Detection on realtime ● Deep learning & NLP (In roadmap)
  13. 13. Machine Learner
  14. 14. Communicate
  15. 15. Visualizing Results Overall idea of what's going on … ● Dashboard using ○ Google Gadget ○ HTML5 + javascripts ● Support custom gadgets and dashboards ● Gadget generation with ○ D3 and Vega
  16. 16. Gadget Generation Tool Customize with few clicks … ● Support realtime and stored data visualization ● How ? ○ Start with data in tabular format ○ Select chart type (Bar, Line, Map, ...) ○ Map columns to dimensions (X Y, color, point size, … ) ○ Also do drill downs
  17. 17. Communicating Results * Supports custom event publishers via its pluggable architecture!
  18. 18. Beyond Boundaries ● Expose analytics results as API ○ Mobile Apps, Third Party ● Provides ○ Security, Billing, ○ Throttling, Quotas & SLA ● How ? ○ Write data to database from DAS ○ Build Services via WSO2 Data Services Server or use Analytics REST API ○ Expose them as APIs via WSO2 API Manager
  19. 19. Developer Tools & Deployment
  20. 20. Developer Tools ● Event Flow
  21. 21. Developer Tools ... ● Event Flow ● Event Tracing
  22. 22. Developer Tools ... ● Event Flow ● Event Tracing ● Event Statistics
  23. 23. Developer Tools ... ● Event Flow ● Event Tracing ● Event Statistics ● Siddhi Try It
  24. 24. Developer Tools ... ● Event Flow ● Event Tracing ● Event Statistics ● Siddhi Try It ● Event Simulator
  25. 25. Developer Tools ... ● Event Flow ● Event Tracing ● Event Statistics ● Siddhi Try It ● Event Simulator ● Spark Console
  26. 26. Developer Tools ... ● Event Flow ● Event Tracing ● Event Statistics ● Siddhi Try It ● Event Simulator ● Spark Console ● Data Explorer
  27. 27. Developer Tools ... ● Event Flow ● Event Tracing ● Event Statistics ● Siddhi Try It ● Event Simulator ● Spark Console ● Data Explorer ● Analytics Tooling
  28. 28. ● One Analytics Archive format for all analytics ○ Based to Carbon Archive ● Minimum HA Deployment (2 Nodes) ● Fully Distributed Deployment (8+ Nodes) ● Realtime Distributed Deployment ○ Siddhi runs on top of Apache Storm Deployment
  29. 29. Solutions
  30. 30. Solutions ● Fraud Detection ○ Alerts for known and unknown frauds ○ Deep Search Analytics ○ https://goo.gl/TWV5C1 ● IoT Analytics (WIP) ○ With configurable Alerting & Monitoring capabilities. ● Log Analytics (WIP) ● Analytics for Products (WIP) ○ Each WSO2 Product will have analytics of its own kind.
  31. 31. Custom Solutions UI to configure predefined analysis with Execution Manager. Good for Business Users
  32. 32. Use Cases & Case studies
  33. 33. Use Cases ● Smart Parking solutions ● Health & Financial Monitoring ● Smart City project, Vehicle tracking, Building monitoring ● Railway monitoring ● Throttling and Anomaly Detection ● API Analytics ● Connected Car
  34. 34. Case Studies ● Realtime Soccer Analytics (DEBS 2013) ○ https://goo.gl/c2gPrQ ● TFL Traffic Analysis ○ Using Transport for London open data feeds. ○ http://goo.gl/04tX6k, http://goo.gl/9xNiCm ● Smart Meters (DEBS 2014) ○ Using smart home electricity data : 2000 sensors, 40 houses, 4 Billion events ○ Processes 400K events/sec, one of the four finalists at DEBS 2014
  35. 35. How to get started ?
  36. 36. How to get started ? ● (WSO2 DAS) 3.0 ○ Combines all types of analytics. ● (WSO2 CEP) 4.0 ○ For who need to analyze realtime event streaming. ● (WSO2 ML) 1.0 ○ For building Predictive Models http://wso2.com/analytics
  37. 37. WSO2 DAS vs WSO2 BAM What's New ? ● Interactive & predictive analytics ● Data Abstraction Layer supporting ○ RDBMS, Cassandra, HBase ● Improved performance with Spark & Siddhi ● Better Dashboard ● Distributed Realtime Processing ● Simple deployment model ● ...
  38. 38. “The price of light is less than the cost of darkness” ~ Arthur C. Nielsen, Market Researcher & Founder of ACNielsen
  39. 39. Thank You

×