Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Patterns for Deploying Analytics in the
Real World
Sriskandarajah Suhothayan
Associate Director / Architect,
WSO2
What’s Analytics ?
Problems to think about
• Can it handle my load ?
• How costly it is ?
• Agility ?
• Adaptability ?
• Can it analyse 3rd
p...
Where to start ?
Where to start ?
• Think Big !
Where to start ?
• Think Big !
But...
• Start simple !
• Eat Your Own Dog Food
• Analyse what you already have
Step 1 :
Find Data Inside Your Organisation...
Collect Data Internally
• Don’t worry about
– Data formats
– Data sources
– Platforms
– Protocols
Start with WSO2 DAS
it h...
Deployment for Data Collection
Step 2 :
Understand how things have been ...
Deployment for Interactive & Batch Analytics …
• Enable Searchability
– Full text search
– Drill down search
• See what ha...
Deployment for Interactive & Batch Analytics …
• Enable Searchability
– Full text search
– Drill down search
• See what ha...
Deployment for Interactive & Batch Analytics
2 Node
Deployment
Step 3 :
Keep informed ...
Deployment for Realtime Analytics
• Keep informed
– Dashboard
– Alerts
– Feedback loops
Deployment for Realtime Analytics
• Keep informed
– Dashboard
– Alerts
– Feedback loops
• High Availability
– Zero downtim...
Realtime High Availability Deployment
Minimum 2 nodes
Max throughput == 1 Node throughput
Deployment for Communicating & Alerting
Legacy & Internal
Services
Step 4 :
Think ahead ...
Deployment for Predictive Analytics
1 Node of WSO2 ML 1 Node of WSO2 ML
Minimum High Availability Deployment
All you need a
2 Node
Deployment
Step 5 :
Expanding as a Connected Business …
Deployment for Exposing Data to 3rd Parties
HTTP
Utilize API Analytics !
Analyse Business with API Analytics
• APIs involved
• Who invokes the APIs
• Extract business information from
– Payloads
...
Step 6 :
Scale with your Data ...
Scaling Analytics Deployment… The Changes !
• Realtime
– Supported by Apache Storm
• For High Memory Requirement or CPU In...
Realtime Scalable Deployment ...
Event Processing
offloaded to
Siddhi Running on
Apache Storm
Seamlessly :)
Realtime Scalable Deployment
Apache Storm Cluster + N CEP nodes
Deployment for Scalable Data Analytics
Minimum 8 Nodes
Deployment
(+ Storm if needed)
Step 7 :
Sense the world around you ...
Deployment of Analytics at the Edge
Analytics on the Edge
with WSO2 Siddhi
Push
Analytics Life Cycle
Predefined analytics
• Bundled as CApps
• Allows migration and continues integration
Dev → Test → Pre...
Deployment Management
WSO2 Servers are already puppetized !
Less configuration hazard for Devops
https://github.com/wso2/p...
Summary
• Start small and scale as you grow
• Minimum HA Deployment
– 2 Nodes
• Fully Distributed Deployment
– 8+ Nodes
– ...
Start analysing data today
for a better tomorrow !
Thank You!
#WSO2ConEU
Share your feedback this session
wso2con.com/app
Upcoming SlideShare
Loading in …5
×

Patterns for Deploying Analytics in the Real World

244 views

Published on

Abundant data is all around. The most important aspect is how you as an organization can access the data, process it, and present information to the relevant authorities on time. To gain competitive advantage the means of accessing, processing and presenting the data should be optimal, highly available and scalable.

In this talk, we will discuss different deployment patterns that can provide you with a suitable solution that lets you analyze relevant data in batch, real-time or interactively and predict future states. We will discuss how you can leverage and deploy WSO2 Data Analytics Server, WSO2 IoT Server, WSO2 Enterprise Service Bus and other WSO2 products in order to make better decisions for your organization’s success.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Patterns for Deploying Analytics in the Real World

  1. 1. Patterns for Deploying Analytics in the Real World Sriskandarajah Suhothayan Associate Director / Architect, WSO2
  2. 2. What’s Analytics ?
  3. 3. Problems to think about • Can it handle my load ? • How costly it is ? • Agility ? • Adaptability ? • Can it analyse 3rd party systems ? • etc ...
  4. 4. Where to start ?
  5. 5. Where to start ? • Think Big !
  6. 6. Where to start ? • Think Big ! But... • Start simple ! • Eat Your Own Dog Food • Analyse what you already have
  7. 7. Step 1 : Find Data Inside Your Organisation...
  8. 8. Collect Data Internally • Don’t worry about – Data formats – Data sources – Platforms – Protocols Start with WSO2 DAS it has a unified data capturing framework !
  9. 9. Deployment for Data Collection
  10. 10. Step 2 : Understand how things have been ...
  11. 11. Deployment for Interactive & Batch Analytics … • Enable Searchability – Full text search – Drill down search • See what has happened – Summarise the Data – Understand patterns and behaviors
  12. 12. Deployment for Interactive & Batch Analytics … • Enable Searchability – Full text search – Drill down search • See what has happened – Summarise the data – Understand patterns and behaviors • Simple Deployment – 2 Nodes – Use RDBMS to store the data
  13. 13. Deployment for Interactive & Batch Analytics 2 Node Deployment
  14. 14. Step 3 : Keep informed ...
  15. 15. Deployment for Realtime Analytics • Keep informed – Dashboard – Alerts – Feedback loops
  16. 16. Deployment for Realtime Analytics • Keep informed – Dashboard – Alerts – Feedback loops • High Availability – Zero downtime – Zero data loss
  17. 17. Realtime High Availability Deployment Minimum 2 nodes Max throughput == 1 Node throughput
  18. 18. Deployment for Communicating & Alerting Legacy & Internal Services
  19. 19. Step 4 : Think ahead ...
  20. 20. Deployment for Predictive Analytics 1 Node of WSO2 ML 1 Node of WSO2 ML
  21. 21. Minimum High Availability Deployment All you need a 2 Node Deployment
  22. 22. Step 5 : Expanding as a Connected Business …
  23. 23. Deployment for Exposing Data to 3rd Parties HTTP Utilize API Analytics !
  24. 24. Analyse Business with API Analytics • APIs involved • Who invokes the APIs • Extract business information from – Payloads – Resources URIs Monetize APIs !
  25. 25. Step 6 : Scale with your Data ...
  26. 26. Scaling Analytics Deployment… The Changes ! • Realtime – Supported by Apache Storm • For High Memory Requirement or CPU Intensive Processing – No query changes • Batch – Move from RDBMS to HBase/Cassandra • WSO2 DAS have a Data Abstraction Layer • Independent of underlying Data Store Seamless migration :)
  27. 27. Realtime Scalable Deployment ... Event Processing offloaded to Siddhi Running on Apache Storm Seamlessly :)
  28. 28. Realtime Scalable Deployment Apache Storm Cluster + N CEP nodes
  29. 29. Deployment for Scalable Data Analytics Minimum 8 Nodes Deployment (+ Storm if needed)
  30. 30. Step 7 : Sense the world around you ...
  31. 31. Deployment of Analytics at the Edge Analytics on the Edge with WSO2 Siddhi Push
  32. 32. Analytics Life Cycle Predefined analytics • Bundled as CApps • Allows migration and continues integration Dev → Test → Preprod → Prod Analytics on Production Environment • Interactive Analytics • Personalizing Dashboards • Customised Alerts
  33. 33. Deployment Management WSO2 Servers are already puppetized ! Less configuration hazard for Devops https://github.com/wso2/puppet-modules
  34. 34. Summary • Start small and scale as you grow • Minimum HA Deployment – 2 Nodes • Fully Distributed Deployment – 8+ Nodes – Scale based on need, horizontally and vertically • Analyser, Indexer, Receiver, Realtime (With Apache Storm), Dashboard • Use puppet to manage deployment
  35. 35. Start analysing data today for a better tomorrow !
  36. 36. Thank You! #WSO2ConEU Share your feedback this session wso2con.com/app

×