Extending the OCCI API with monitoring capabilities

536 views

Published on

A tutorial about the API for the description of a monitoring infrastructure currently discussed inside the OCCI working group.
The slides start by giving the basic concepts, proceed with a description of the entities that implement the monitoring infrastructure, and conclude with a step by step definition of a non-trivial monitoring infrastructure.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
536
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Extending the OCCI API with monitoring capabilities

  1. 1. OCCI Monitoring Augusto Ciuffoletti OCCI Monitoring Extending the OCCI API with monitoring capabilities Augusto Ciuffoletti Dept. of Computer Science - Univ. of Pisa September 13, 2013
  2. 2. OCCI Monitoring Augusto Ciuffoletti Motivation SLA is a defined target to obtain user confidence
  3. 3. OCCI Monitoring Augusto Ciuffoletti Motivation SLA is a defined target to obtain user confidence SLA is tightly related with monitoring, so we start from cloud monitoring
  4. 4. OCCI Monitoring Augusto Ciuffoletti Motivation SLA is a defined target to obtain user confidence SLA is tightly related with monitoring, so we start from cloud monitoring The user may be in its turn a service provider (inside monitoring)
  5. 5. OCCI Monitoring Augusto Ciuffoletti Motivation SLA is a defined target to obtain user confidence SLA is tightly related with monitoring, so we start from cloud monitoring The user may be in its turn a service provider (inside monitoring) The user may simply want to verify the quality of the service (outside monitoring)
  6. 6. OCCI Monitoring Augusto Ciuffoletti Motivation SLA is a defined target to obtain user confidence SLA is tightly related with monitoring, so we start from cloud monitoring The user may be in its turn a service provider (inside monitoring) The user may simply want to verify the quality of the service (outside monitoring) In both cases, the user needs the tools to define resource monitoring
  7. 7. OCCI Monitoring Augusto Ciuffoletti Motivation SLA is a defined target to obtain user confidence SLA is tightly related with monitoring, so we start from cloud monitoring The user may be in its turn a service provider (inside monitoring) The user may simply want to verify the quality of the service (outside monitoring) In both cases, the user needs the tools to define resource monitoring Keep into account the case of a composite service (many providers)
  8. 8. OCCI Monitoring Augusto Ciuffoletti Motivation SLA is a defined target to obtain user confidence SLA is tightly related with monitoring, so we start from cloud monitoring The user may be in its turn a service provider (inside monitoring) The user may simply want to verify the quality of the service (outside monitoring) In both cases, the user needs the tools to define resource monitoring Keep into account the case of a composite service (many providers) A simple API aligned with OCCI
  9. 9. OCCI Monitoring Augusto Ciuffoletti Basic functions Monitoring is made of three basic activities
  10. 10. OCCI Monitoring Augusto Ciuffoletti Basic functions Monitoring is made of three basic activities extract operational parameters from a Resource
  11. 11. OCCI Monitoring Augusto Ciuffoletti Basic functions Monitoring is made of three basic activities extract operational parameters from a Resource gather the operational parameters to obtain the measure of a metric of interest
  12. 12. OCCI Monitoring Augusto Ciuffoletti Basic functions Monitoring is made of three basic activities extract operational parameters from a Resource gather the operational parameters to obtain the measure of a metric of interest deliver the measurement to the relevant party
  13. 13. OCCI Monitoring Augusto Ciuffoletti Aggregate and deliver Distiguished activities that need the provision of computing, storage, networking resources
  14. 14. OCCI Monitoring Augusto Ciuffoletti Aggregate and deliver Distiguished activities that need the provision of computing, storage, networking resources Tightly integrated, under control of the provider
  15. 15. OCCI Monitoring Augusto Ciuffoletti Aggregate and deliver Distiguished activities that need the provision of computing, storage, networking resources Tightly integrated, under control of the provider Candidate for the introduction of a new kind of resource: the Sensor
  16. 16. OCCI Monitoring Augusto Ciuffoletti Aggregate and deliver Distiguished activities that need the provision of computing, storage, networking resources Tightly integrated, under control of the provider Candidate for the introduction of a new kind of resource: the Sensor The user that wants to exert monitoring instantiates (and pays for) a Sensor
  17. 17. OCCI Monitoring Augusto Ciuffoletti Aggregate and deliver Distiguished activities that need the provision of computing, storage, networking resources Tightly integrated, under control of the provider Candidate for the introduction of a new kind of resource: the Sensor The user that wants to exert monitoring instantiates (and pays for) a Sensor Note: a cost is associated to the Sensor, although this is not explicit in the definition of monitoring
  18. 18. OCCI Monitoring Augusto Ciuffoletti Basic functions The monitoring function is controlled by time
  19. 19. OCCI Monitoring Augusto Ciuffoletti Basic functions The monitoring function is controlled by time The native attributes of a Sensor are:
  20. 20. OCCI Monitoring Augusto Ciuffoletti Basic functions The monitoring function is controlled by time The native attributes of a Sensor are: How frequently it produces a new measurement
  21. 21. OCCI Monitoring Augusto Ciuffoletti Basic functions The monitoring function is controlled by time The native attributes of a Sensor are: How frequently it produces a new measurement During which time lapse it performs the measurements
  22. 22. OCCI Monitoring Augusto Ciuffoletti Basic functions The monitoring function is controlled by time The native attributes of a Sensor are: How frequently it produces a new measurement During which time lapse it performs the measurements Too many variants for aggregation/delivery: OCCI-mixins
  23. 23. OCCI Monitoring Augusto Ciuffoletti Basic functions The monitoring function is controlled by time The native attributes of a Sensor are: How frequently it produces a new measurement During which time lapse it performs the measurements Too many variants for aggregation/delivery: OCCI-mixins Mixins are classified using tags (sort of subtyping)
  24. 24. OCCI Monitoring Augusto Ciuffoletti Extract measurements This is an ability that a sensor has with respect to a real resource
  25. 25. OCCI Monitoring Augusto Ciuffoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation)
  26. 26. OCCI Monitoring Augusto Ciuffoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation) This is represented as a collector link from the sensor to the resource
  27. 27. OCCI Monitoring Augusto Ciuffoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation) This is represented as a collector link from the sensor to the resource The generic attributes of a collector correspond to the sampling period:
  28. 28. OCCI Monitoring Augusto Ciuffoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation) This is represented as a collector link from the sensor to the resource The generic attributes of a collector correspond to the sampling period: The sampling period
  29. 29. OCCI Monitoring Augusto Ciuffoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation) This is represented as a collector link from the sensor to the resource The generic attributes of a collector correspond to the sampling period: The sampling period The accuracy of the sampling period
  30. 30. OCCI Monitoring Augusto Ciuffoletti Extract measurements This is an ability that a sensor has with respect to a real resource Capability of extracting measurements (e.g., through resource instrumentation) This is represented as a collector link from the sensor to the resource The generic attributes of a collector correspond to the sampling period: The sampling period The accuracy of the sampling period OCCI-mixins are introduced to specify the measurement technique used
  31. 31. OCCI Monitoring Augusto Ciuffoletti Mixin sub-typing: the tags According with the OCCI core specification a mixin with no attributes (a tag) can be associated to another mixin as a sort of label
  32. 32. OCCI Monitoring Augusto Ciuffoletti Mixin sub-typing: the tags According with the OCCI core specification a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specification
  33. 33. OCCI Monitoring Augusto Ciuffoletti Mixin sub-typing: the tags According with the OCCI core specification a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specification By defining tagged mixins, the provider describes the monitoring capabilities offered to the user
  34. 34. OCCI Monitoring Augusto Ciuffoletti Mixin sub-typing: the tags According with the OCCI core specification a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specification By defining tagged mixins, the provider describes the monitoring capabilities offered to the user Next slides describe three such tags:
  35. 35. OCCI Monitoring Augusto Ciuffoletti Mixin sub-typing: the tags According with the OCCI core specification a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specification By defining tagged mixins, the provider describes the monitoring capabilities offered to the user Next slides describe three such tags: metric specific for collector links
  36. 36. OCCI Monitoring Augusto Ciuffoletti Mixin sub-typing: the tags According with the OCCI core specification a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specification By defining tagged mixins, the provider describes the monitoring capabilities offered to the user Next slides describe three such tags: metric specific for collector links aggregator specific for sensor resources
  37. 37. OCCI Monitoring Augusto Ciuffoletti Mixin sub-typing: the tags According with the OCCI core specification a mixin with no attributes (a tag) can be associated to another mixin as a sort of label Syntactic and semantic restrictions can be introduced for mixins with a certain label, thus supporting a specification By defining tagged mixins, the provider describes the monitoring capabilities offered to the user Next slides describe three such tags: metric specific for collector links aggregator specific for sensor resources publisher specific for sensor resources
  38. 38. OCCI Monitoring Augusto Ciuffoletti The metric tag The tag metric is associated with a measurement technique, and is related with a collector link;
  39. 39. OCCI Monitoring Augusto Ciuffoletti The metric tag The tag metric is associated with a measurement technique, and is related with a collector link; The mixin that has the metric tag has the following attributes:
  40. 40. OCCI Monitoring Augusto Ciuffoletti The metric tag The tag metric is associated with a measurement technique, and is related with a collector link; The mixin that has the metric tag has the following attributes: metric attribute strings that are used as an identifier to refer to the output measurement stream
  41. 41. OCCI Monitoring Augusto Ciuffoletti The metric tag The tag metric is associated with a measurement technique, and is related with a collector link; The mixin that has the metric tag has the following attributes: metric attribute strings that are used as an identifier to refer to the output measurement stream control attributes used to control the measurement process (e.g. packet length of a ping)
  42. 42. OCCI Monitoring Augusto Ciuffoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation
  43. 43. OCCI Monitoring Augusto Ciuffoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation an aggregator mixin is related with a sensor resource
  44. 44. OCCI Monitoring Augusto Ciuffoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation an aggregator mixin is related with a sensor resource the mixin with the aggregator tag are characterized by:
  45. 45. OCCI Monitoring Augusto Ciuffoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation an aggregator mixin is related with a sensor resource the mixin with the aggregator tag are characterized by: input attributes strings that are used as an identifier to refer to input measurement streams
  46. 46. OCCI Monitoring Augusto Ciuffoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation an aggregator mixin is related with a sensor resource the mixin with the aggregator tag are characterized by: input attributes strings that are used as an identifier to refer to input measurement streams control attributes that are parameters for the aggregation function (e.g. the gain in an EWMA)
  47. 47. OCCI Monitoring Augusto Ciuffoletti The aggregator tag the tag aggregator is associated with a mixin that implements an algorithm for monitoring data aggregation an aggregator mixin is related with a sensor resource the mixin with the aggregator tag are characterized by: input attributes strings that are used as an identifier to refer to input measurement streams control attributes that are parameters for the aggregation function (e.g. the gain in an EWMA) metric attributes strings that are used as an identifier to refer to the output measurement stream
  48. 48. OCCI Monitoring Augusto Ciuffoletti The publisher tag the tag publisher is associated with a mixin that implements a technique to deliver the data outside the monitoring infrastructure
  49. 49. OCCI Monitoring Augusto Ciuffoletti The publisher tag the tag publisher is associated with a mixin that implements a technique to deliver the data outside the monitoring infrastructure a publisher mixin is related with a sensor resource
  50. 50. OCCI Monitoring Augusto Ciuffoletti The publisher tag the tag publisher is associated with a mixin that implements a technique to deliver the data outside the monitoring infrastructure a publisher mixin is related with a sensor resource the mixin with a publisher tag are characterized by:
  51. 51. OCCI Monitoring Augusto Ciuffoletti The publisher tag the tag publisher is associated with a mixin that implements a technique to deliver the data outside the monitoring infrastructure a publisher mixin is related with a sensor resource the mixin with a publisher tag are characterized by: input attribute a string that is used as an identifier to refer to input measurement stream
  52. 52. OCCI Monitoring Augusto Ciuffoletti The publisher tag the tag publisher is associated with a mixin that implements a technique to deliver the data outside the monitoring infrastructure a publisher mixin is related with a sensor resource the mixin with a publisher tag are characterized by: input attribute a string that is used as an identifier to refer to input measurement stream control attributes that are parameters for the publishing function (e.g. the UDP port that is used for streaming)
  53. 53. OCCI Monitoring Augusto Ciuffoletti The role of metric and input attributes they describe the flow of monitoring data
  54. 54. OCCI Monitoring Augusto Ciuffoletti The role of metric and input attributes they describe the flow of monitoring data their values are unique identifiers in a scope corresponding to a sensor and all outgoing collectors
  55. 55. OCCI Monitoring Augusto Ciuffoletti The role of metric and input attributes they describe the flow of monitoring data their values are unique identifiers in a scope corresponding to a sensor and all outgoing collectors the provider is free to implement the transport of the measurement stream in the most convenient way
  56. 56. OCCI Monitoring Augusto Ciuffoletti Shortcuts for simple use cases here metric mixin, can be associated directly to the monitored resource and has a web service interface
  57. 57. OCCI Monitoring Augusto Ciuffoletti Shortcuts for simple use cases here metric mixin, can be associated directly to the monitored resource and has a web service interface a tagged mixin can be associated with a generic resource or link, avoiding the definition of a sensor or collector
  58. 58. OCCI Monitoring Augusto Ciuffoletti Shortcuts for simple use cases here metric mixin, can be associated directly to the monitored resource and has a web service interface a tagged mixin can be associated with a generic resource or link, avoiding the definition of a sensor or collector shortcuts effectively help very simple use cases, but are generally a source of inefficiency and complexity
  59. 59. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure The resources we want to monitor
  60. 60. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure Create one Sensor resource
  61. 61. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure Use two collectors to define the measurement activiy
  62. 62. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure Associate two metric mixins to the Collector X
  63. 63. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure And another two metric mixins to the Collector Y
  64. 64. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure Associate two aggregator mixins to the Sensor
  65. 65. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure One publisher is going to use raw data from the collector
  66. 66. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure Another is going to receive measurements from the aggregators
  67. 67. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure A frame for Collector X and its mixins
  68. 68. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure ... one for Collector Y...
  69. 69. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure ... one for the sensor
  70. 70. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure The scope of the Sensor (for metric streams ids)
  71. 71. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure Feeding the aggregators: a,b,d are measurement stream ids
  72. 72. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure Feeding publisher 2: aggregated (f,g) and raw data (e)
  73. 73. OCCI Monitoring Augusto Ciuffoletti Step by step design of a monitoring infrastructure Feeding publisher 1: measurement stream b is multicast
  74. 74. OCCI Monitoring Augusto Ciuffoletti Also in the document Conformance profiles: to accomodate the presence of providers that do not implement a monitoring interface
  75. 75. OCCI Monitoring Augusto Ciuffoletti Also in the document Conformance profiles: to accomodate the presence of providers that do not implement a monitoring interface Security issues
  76. 76. OCCI Monitoring Augusto Ciuffoletti Also in the document Conformance profiles: to accomodate the presence of providers that do not implement a monitoring interface Security issues A detailed example using the http rendering
  77. 77. OCCI Monitoring Augusto Ciuffoletti Also in the document Conformance profiles: to accomodate the presence of providers that do not implement a monitoring interface Security issues A detailed example using the http rendering ...and several bugs.
  78. 78. OCCI Monitoring Augusto Ciuffoletti Also in the document Conformance profiles: to accomodate the presence of providers that do not implement a monitoring interface Security issues A detailed example using the http rendering ...and several bugs. That’s all...

×