Second review presentation

476 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
476
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Second review presentation

  1. 1. Design and Implementation of a ServiceMonitoring Console within a Service Oriented Architecture Framework Arvind Krishnaa .J 31508104017 Guided By Dr. Chitra Babu HOD/CSE SSN College of Engineering Second Review - 30th March, 2012
  2. 2. Turmeric Runtime Platform 1 Client Runtime - Client application code written by the consumers of the service. Service Invocation Framework(SIF)
  3. 3. Turmeric Runtime Platform 1 Client Runtime - Client application code written by the consumers of the service. Service Invocation Framework(SIF) 2 Server Runtime - Deployment platform for developers to publish web services. Service Provider Framework (SPF)
  4. 4. Turmeric Runtime Platform 1 Client Runtime - Client application code written by the consumers of the service. Service Invocation Framework(SIF) 2 Server Runtime - Deployment platform for developers to publish web services. Service Provider Framework (SPF) 3 CodeGen - Generate instrumentation code for deploying services onto Turmeric.
  5. 5. Central Application Logging Framework Centralized collection of application server logs
  6. 6. Central Application Logging Framework Centralized collection of application server logs Base for collecting data for the monitoring console
  7. 7. Central Application Logging Framework Centralized collection of application server logs Base for collecting data for the monitoring console Ability to generate custom reports from this data
  8. 8. Central Application Logging Framework Centralized collection of application server logs Base for collecting data for the monitoring console Ability to generate custom reports from this data TMC is a specific instance of a report created using CAL
  9. 9. Architecture of the Monitoring Console 1 Can be abstracted to two frameworks running in parallel - metrics logging framework and metrics handler framework
  10. 10. Architecture of the Monitoring Console 1 Can be abstracted to two frameworks running in parallel - metrics logging framework and metrics handler framework 2 Both frameworks are web services deployed on an application server
  11. 11. Architecture of the Monitoring Console 1 Can be abstracted to two frameworks running in parallel - metrics logging framework and metrics handler framework 2 Both frameworks are web services deployed on an application server 3 SOA Query Metrics Service (sqms) - Logging logging handler framework
  12. 12. Architecture of the Monitoring Console 1 Can be abstracted to two frameworks running in parallel - metrics logging framework and metrics handler framework 2 Both frameworks are web services deployed on an application server 3 SOA Query Metrics Service (sqms) - Logging logging handler framework 4 Monitoring Console (console) - Metrics handler framework
  13. 13. SOA Query Metrics Service (SQMS) Figure 1: Architecture of SQMS
  14. 14. Working of SQMS 1 SPFServlet is the main servlet which handles the requests to the Turmeric runtime. 2 Initializes the CAL logger for logging metrics published by Turmeric services. 3 Metric data logged in a format called heartbeats. 4 Log message casted into a JMX MBean like object. 5 The MBean object is dispatched to the JMX servlet and published to a controller interface (Agent Service)
  15. 15. Metrics Storage Provider Java interface for defining the underlying storage mechanism
  16. 16. Metrics Storage Provider Java interface for defining the underlying storage mechanism Provides the general contract for defining the queries to store metrics into the persistence mechanism
  17. 17. Metrics Storage Provider Java interface for defining the underlying storage mechanism Provides the general contract for defining the queries to store metrics into the persistence mechanism Can be extended to provide custom implementations, for various Hibernate- compliant databases.
  18. 18. Metrics Storage Provider Java interface for defining the underlying storage mechanism Provides the general contract for defining the queries to store metrics into the persistence mechanism Can be extended to provide custom implementations, for various Hibernate- compliant databases. Currently two implementations are provided - DAO(based on a MySql storage) and Cassandra (partially implemented)
  19. 19. Metrics Logging 1 saveMetricValue(): It receives the timestamp, metric name, and a key-value pair to store the metric value, and converts it to row oriented format for MySQL storage (or name-value pair for Cassandra) 2 saveMetricSnapshot(): Encapsulates the above received metrics objects into a MBean instance and sends it to the JMX server for publication.
  20. 20. Logging helper utilities 1 Snapshot File Logger: Logs the aggregated metrics into a .log file 2 Differential Snapshot View Logger: Calculating the percentage changes between the metrics collected on two different days. 3 Monitoring System: Schedules the metrics logging process to be performed at regular intervals. Presently it is set at 60 seconds.
  21. 21. Logging Handlers 1 Metrics Storage Provider Interface Defines a framework which runs two MetricsSnapshotSchedulers in separate background threads, one for the client-side metrics and the other for the server-side metrics Calls the saveMetricsSnapshot() method to store the metric data. 2 DAO Metrics Provider : The provider implementation for MySQL persistence mechanism 3 Cassandra Storage Provider: Implementation for Cassandra storage. Any custom persistence mechanism can also be integrated.
  22. 22. Monitoring Console Service (TMC) Figure 2: Architecture of Monitoring Dashboard
  23. 23. Implementation of TMC service Responsible for constructing user interface of monitoring dashboard
  24. 24. Implementation of TMC service Responsible for constructing user interface of monitoring dashboard Retrieves the encapsulated data in the form of a MetricsData instance
  25. 25. Implementation of TMC service Responsible for constructing user interface of monitoring dashboard Retrieves the encapsulated data in the form of a MetricsData instance This object is serialized into JSON format and sent to the Google Web Toolkit’s charting API
  26. 26. Implementation of TMC service Responsible for constructing user interface of monitoring dashboard Retrieves the encapsulated data in the form of a MetricsData instance This object is serialized into JSON format and sent to the Google Web Toolkit’s charting API GWT API constructs the trends chart based on the JSON data
  27. 27. Implementation of TMC service Responsible for constructing user interface of monitoring dashboard Retrieves the encapsulated data in the form of a MetricsData instance This object is serialized into JSON format and sent to the Google Web Toolkit’s charting API GWT API constructs the trends chart based on the JSON data A REST URI is created for every JSON object, and a link is created in the dashboard.
  28. 28. Implementation of TMC service Responsible for constructing user interface of monitoring dashboard Retrieves the encapsulated data in the form of a MetricsData instance This object is serialized into JSON format and sent to the Google Web Toolkit’s charting API GWT API constructs the trends chart based on the JSON data A REST URI is created for every JSON object, and a link is created in the dashboard. The metrics detail can be exported into a CSV file as well.
  29. 29. Offline Aggregator - A Partial Implementation Objective of collecting the distributed metrics across cluster of nodes, by utilizing Cassandra storage. Only the storage into Cassandra keyspace has been implemented Hooks are provided to read data across distributed nodes, to make this feature full-fledged. Three main classes for implementing offline-aggregation 1 Readers 2 Writers 3 Erasers
  30. 30. Overview of Readers, Writers and Erasers Readers: Read the data from the offline cluster, de-serialize the MetricsData objects and display it in the dashboard. Different reader classes are available to customize metrics reading, in the future. (Classes need to be extended with custom implementation)
  31. 31. Overview of Readers, Writers and Erasers Readers: Read the data from the offline cluster, de-serialize the MetricsData objects and display it in the dashboard. Different reader classes are available to customize metrics reading, in the future. (Classes need to be extended with custom implementation) Writers: Write data into the offline cluster, by serializing the metrics into MetricsData objects. Different writer base classes available for extending with custom implementations.
  32. 32. Overview of Readers, Writers and Erasers Readers: Read the data from the offline cluster, de-serialize the MetricsData objects and display it in the dashboard. Different reader classes are available to customize metrics reading, in the future. (Classes need to be extended with custom implementation) Writers: Write data into the offline cluster, by serializing the metrics into MetricsData objects. Different writer base classes available for extending with custom implementations. Erasers: Erase the data from the offline cluster.
  33. 33. Overview of Readers, Writers and Erasers Figure 3: Overview of Reader, Writers and Erasers
  34. 34. Partial Demo 1 Nuts and bolts of Turmeric SOA services 2 Sample REST URLs for metrics 3 Walkthrough of database schema and significant implementation classes 4 Overview of data aggregator tool 5 Tour of the user interface
  35. 35. Screenshots - SMC Dashboard for an eBay service Figure 4: ShortURI Service
  36. 36. Screenshots - Turmeric Top volume graph Figure 5: Example Echo Service
  37. 37. Screenshots - Turmeric Consumer trends Figure 6: Example Echo Service
  38. 38. Data Aggregator tool Figure 7: Ad hoc querying
  39. 39. Data Aggregator tool – Contd. Figure 8: Ad hoc querying
  40. 40. Implementation Schedule Figure 9: Project Roadmap
  41. 41. References [1]Jeffrey Dean, Sanjay Ghemawat, MapReduce: Simplified Data Processing on Large Clusters, In Sixth Symposium on Operating System Design and Implementation(OSDI’04), San Francisco, CA, December, 2004 [2] Eben Hewitt, Cassandra: The Definitive Guide, O’Reilly Publications, November 2010. [3] eBay Open Source Project, Turmeric SOA platform, http: //www.ebayopensource.org/index.php/Turmeric/HomePage
  42. 42. References [4] eBay Open Source Project, Documentation of Turmeric SOA platform, https://www.ebayopensource.org/wiki/display/ TURMERICDOC110GA/Turmeric+Documentation+Overview [5] eBay Open Source Project, Turmeric Source Code, http://www.github.com/ebayopensource [6] Internal eBay documentation [7] Google Web Toolkit, http://code.google.com/webtoolkit [8] Apache Cassandra, http://cassandra.apache.org/

×