The WSO2 Business Activity Monitor - Optimizing Enterprise Data Management


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The WSO2 Business Activity Monitor - Optimizing Enterprise Data Management

  1. 1. With WSO2 Business Activity Monitor 2.4.1 Inosh Goonewardena Associate Technical Lead Gokul Balakrishnan Software Engineer Optimizing Enterprise Data Management June 2014
  2. 2. ** About the Presenters ● Inosh Goonewardena Inosh Goonewardena is an associate technical lead and is a part of the WSO2 Business Activity Monitor team. He holds a bachelor’s degree in Engineering, specializing in Computer Science & Engineering from University of Moratuwa, Sri Lanka. ● Gokul Balakrishnan Gokul is a software engineer in the Data and APIs Technology Group focusing on WSO2 Business Activity Monitor. He has a bachelor’s degree in Engineering from University of Moratuwa, Sri Lanka, and has provided consultation services on customer engagements.
  3. 3. ** ● Global enterprise, founded in 2005 by acknowledged leaders in XML, web services technologies, standards and open source ● Provides only open source platform-as-a-service for private, public and hybrid cloud deployments ● All WSO2 products are 100% open source and released under the Apache License Version 2.0. ● Is an Active Member of OASIS, Cloud Security Alliance, OSGi Alliance, AMQP Working Group, OpenID Foundation and W3C. ● Driven by Innovation ○ Launched first open source API Management solution in 2012 ○ Launched App Factory in Q2 2013 ○ Launched Enterprise Store and first open source Mobile solution in Q4 2013 About WSO2
  4. 4. ** What we deliver
  5. 5. Agenda ● Introduction to Business Activity Monitoring ● WSO2 Business Activity Monitor ● WSO2 BAM Components ● WSO2 BAM Toolboxes ● WSO2 BAM High Availability Distributed Deployment ● Product Demo
  6. 6. “The aggregation, analysis, and presentation of real-time information about activities inside organizations and involving customers and partners” - Gartner Business Activity Monitoring
  7. 7. ● Capturing data ● Data storage ● What data to capture? Aggregation
  8. 8. ● Data operations ● Building KPIs ● Operate on large amounts of historic data or new data ● Building BI Analysis
  9. 9. ● Visualizing KPIs/BI ● Custom Dashboards ● Visualization tools ● Not just dashboards! Presentation
  10. 10. ● Fully-open source solution for aggregating and analyzing data and presenting information about business activities. ● Cloud-enabled, lightweight, developer-friendly and easy-to- deploy ● High performance data capture framework ● Elastically-scalable data analysis powered by Apache Hadoop ● Pre-built Data Agents for WSO2 products WSO2 Business Activity Monitor
  11. 11. Architecture
  12. 12. Data Agents
  13. 13. ● Compatible with CEP/BAM ● Get data across to BAM − Service monitoring feature – WSO2 AS, DSS, ESB, API Manager − Mediation monitoring feature – BAM Mediator for WSO2 ESB − Custom data-agents ● Asynchronous & non-blocking ● Thrift for high performance message throughput Data Agents
  14. 14. ● Data agents are meant to be installed/plugged into the server to be monitored ● A Java SDK is provided ● Through Thrift, different languages can be supported Custom Agents
  15. 15. Data Receiver
  16. 16. ● Receives data and stores it in Cassandra ○ Scalable, big data repository ● Asynchronous & non-blocking ○ Combination of Cassandra, Thrift and the non-blocking nature results in extremely fast writes ● Shared with WSO2 CEP for real time analysis ● Supports Thrift & REST API ● Supports plugging in of different receiver types Data Receiver
  17. 17. ● Data is sent over using strongly typed Data Streams { 'name':'', 'version':'1.0.0', 'nickName': 'Phone_Retail_Shop', 'description': 'Phone Sales', 'metaData':[ {'name':'clientType','type':'STRING'} ], 'payloadData':[ {'name':'brand','type':'STRING'}, {'name':'quantity','type':'INT'}, {'name':'total','type':'INT'}, {'name':'user','type':'STRING'} ] } ● Data Streams are versioned ○ Allows for easier analysis Data Model
  18. 18. The Analyzer Engine
  19. 19. ● Powered by Apache Hadoop with querying/managing through Apache Hive ● Data transfer to and from data sources is handled through custom storage handlers ● Parallel, distributed processing through the MapReduce programming model The Analyzer Engine
  20. 20. ● Runs on local Hadoop node or delegates to Hadoop cluster o Scalable analytics o Cluster can range from a couple of nodes to 1000s ● Analysis is carried out based on analytics scripts ● Scripts are based on an easy-to-learn, SQL-like query language INSERT OVERWRITE TABLE UserTable SELECT userName, COUNT(DISTINCT orderID),SUM(quantity) FROM PhoneSalesTable WHERE version= "1.0.0" GROUP BY userName; The Analyzer Engine
  21. 21. ● Scripts can be scheduled ○ Ex: once a minute, every Wednesday at 4:15 p.m., every 30th at 12 midnight ● Polyglot data architectures are supported ○ Write summarised information to any kind of datastore ○ Custom written Hive JDBC handler used by default for output to relational DBs The Analyzer Engine
  22. 22. The Presentation Layer
  23. 23. ● Gadget Portal for the dashboard ● Gadget Generation Wizard – Tool for generating custom gadgets for the dashboard ● Activity Dashboard for correlating activities ● Message Console ● Plug in your own report server/ dashboard server The Presentation Layer
  24. 24. BAM Dashboards The Presentation Layer
  25. 25. Gadget Portal The Presentation Layer
  26. 26. Activity Dashboard The Presentation Layer
  27. 27. Activity Dashboard (cont'd)
  28. 28. Message Console (HL7) The Presentation Layer
  29. 29. Message Console (cont'd)
  30. 30. ● BAM Toolboxes are installable and hot deployable artefacts used for deploying functionalities to a BAM server ○ Stream definitions ○ Analytics scripts ○ Dashboards ● Supports plugging in of one or more of the above features ● Toolboxes for monitoring and auditing most WSO2 products are available OOTB ● Toolboxes for custom scenarios can be created easily BAM Toolboxes
  31. 31. ● WSO2 BAM can be clustered and deployed in a distributed manner to enable high-availability, fail-over scenarios ○ Partially distributed deployment ■ Cassandra (storage) and Hadoop (analyzer) nodes are clustered ○ Fully distributed deployment ■ All components of BAM are clustered (data receiver, storage, analyzer and presentation) ● Hazelcast in-memory data grids are used for clustering implementation HA Distributed BAM
  32. 32. Partially Distributed Deployment
  33. 33. Fully Distributed Deployment
  34. 34. Shared Events with CEP
  35. 35. WSO2 BAM Demonstration In this demo: ESB mediation statistics monitoring with the mediation data agent BAM Mediation stats toolbox BAM dashboard Gadget gen tool with SQL query and visualisation
  36. 36. ● UES integration for the presentation layer ● Enhanced alerting and reporting capabilities ● Enhanced toolbox support for WSO2 product stack ● Abstract data storage layer Upcoming Features
  37. 37. ?
  38. 38. ** Business Model
  39. 39. Contact us !