CeilometerThe OpenStack Measurement Project14 April 2013 @ ODS HavanaJulien Danjoujd__@Freenode // @juldanjoujulien@danjou...
What About Billing?Beginning of 2012:● Billing has been left out of OpenStack core so far as itwas not the primary problem...
Billing: 3 Step ProcessMetering Collect usage dataRating Transform usage data intobillable items and calculate costsBillin...
Ceilometer Begins● Started in May 2012
v0.1 Objectives (Folsom)Focused only on metering for billingCeilometer aims to deliver a unique point of contact for billi...
Grizzly objectivesExtended objective: cover measurement ingeneralThe project aims to become the infrastructure to collect ...
Grizzly objectivesExtended objective: cover measurement ingeneralThe project aims to become the infrastructure to collect ...
Workflow● Collect from OpenStack components● Transform meters into other meters ifnecessary● Publish meters to any destina...
CollectingNetworking Object StorageImageVolumeStorageNotification busCeilometercollector & agentAPIAPI API APIComputeAPIOs...
Ceilometercollector & agentReceiverReceiverPublisherPublisherPipelineTransformerMeterTransformerTransformerPublisher Recei...
TransformerMeterName=cpu.timeValue=5Timestamp=TTransformerMeterName=cpu.timeValue=20Timestamp=T+1MeterName=cpu.timeValue=8...
PublisherMeterCeilometerRPCPublisherUDP PublisherCeilometer collectorExternal systemAMQPsigned messageBilling, monitoring,...
Store: collectorCeilometer collectorSQLDatabaseMongoDBDatabaseMeterStorage abstraction layerMultiple backends choice,one a...
Read: APIDatabaseCeilometer APIExternal consumersystemBilling, monitoring,alarming, statistics,capacity planning…HTTP REST...
The big pictureOpenStackcomponentsCeilometer collectorCeilometer APICeilometer agentsNotification busDatabaseAPIExternal s...
RoadmapHavana I● Integrated Project ✓● Integration with Horizon● Publishing meters toother systems● Enhance SQL driver● Al...
Use cases● Primary use-case is a rating/billing pipeline● Analytics○ capacity planning tools○ adaptive scheduling algorith...
Wednesday 16:30 Introduction to ceilometer architechture17:20 Feedback from Ceilometer usersThursday 09:00 Incremental imp...
Questions?Nick Barcetnijaba@Freenode // @nijabanick@enovance.comhttp://launchpad.net/ceilometerhttp://docs.openstack.org/d...
Upcoming SlideShare
Loading in …5
×

Ceilometer presentation ods havana final - published

4,432 views

Published on

Presentation on OpenStack Ceilometer made by Nick Barcet & Julien Danjou during OpenStack Summit Portland - April 2013

Published in: Technology, Business

Ceilometer presentation ods havana final - published

  1. 1. CeilometerThe OpenStack Measurement Project14 April 2013 @ ODS HavanaJulien Danjoujd__@Freenode // @juldanjoujulien@danjou.infoNick Barcetnijaba@Freenode // @nijabanick@enovance.com
  2. 2. What About Billing?Beginning of 2012:● Billing has been left out of OpenStack core so far as itwas not the primary problem and is not a trivial one...● Yet almost every OpenStack deployment needs a wayto track usage information
  3. 3. Billing: 3 Step ProcessMetering Collect usage dataRating Transform usage data intobillable items and calculate costsBilling Create invoice, collect payment
  4. 4. Ceilometer Begins● Started in May 2012
  5. 5. v0.1 Objectives (Folsom)Focused only on metering for billingCeilometer aims to deliver a unique point of contact for billing systems toacquire all counters they need to establish customer billing, across all currentand future OpenStack components.The delivery of counters must be traceableand auditable, the counters must be easily extensible to support new projects,and agents doing data collections should be independent of the overall system.
  6. 6. Grizzly objectivesExtended objective: cover measurement ingeneralThe project aims to become the infrastructure to collect measurements withinOpenStack so that no two agents would need to be written to collect the samedata. Its primary targets are monitoring and metering, but the frameworkshould be easily expandable to collect for other needs. To that effect,Ceilometer should be able to share collected data with a variety of consumers.
  7. 7. Grizzly objectivesExtended objective: cover measurement ingeneralThe project aims to become the infrastructure to collect measurements withinOpenStack so that no two agents would need to be written to collect the samedata. Its primary targets are monitoring and metering, but the frameworkshould be easily expandable to collect for other needs. To that effect,Ceilometer should be able to share collected data with a variety of consumers.Havanas objectives should remain unchanged!
  8. 8. Workflow● Collect from OpenStack components● Transform meters into other meters ifnecessary● Publish meters to any destination (includingCeilometer itself)● Store received meters and read them via theCeilometer REST APICollect Transform Publish Store Read
  9. 9. CollectingNetworking Object StorageImageVolumeStorageNotification busCeilometercollector & agentAPIAPI API APIComputeAPIOslonotificationsPublishingpipelinePublishingPollingNew inGrizzly!
  10. 10. Ceilometercollector & agentReceiverReceiverPublisherPublisherPipelineTransformerMeterTransformerTransformerPublisher ReceiverPipeline: a set of transformers mutating meters intosomething that publishers know how to send toexternal systems.New inGrizzly!
  11. 11. TransformerMeterName=cpu.timeValue=5Timestamp=TTransformerMeterName=cpu.timeValue=20Timestamp=T+1MeterName=cpu.timeValue=80Timestamp=T+2MeterName=cpu.percentageValue=9Timestamp=T+2Transform meters into new meters!New inGrizzly!
  12. 12. PublisherMeterCeilometerRPCPublisherUDP PublisherCeilometer collectorExternal systemAMQPsigned messageBilling, monitoring,alarming, statistics,capacity planning…New inGrizzly!(not implemented yet)Yes, the frequency of the publication of each kind of meter is configurable for eachpublishing target.
  13. 13. Store: collectorCeilometer collectorSQLDatabaseMongoDBDatabaseMeterStorage abstraction layerMultiple backends choice,one at a timeHBaseDatabaseDefaultbackendNew inGrizzly!
  14. 14. Read: APIDatabaseCeilometer APIExternal consumersystemBilling, monitoring,alarming, statistics,capacity planning…HTTP RESTw/ JSON &XMLStorage abstraction layerRaw events:GET /v2/meters/cpu.timeStatistics (sum, average, min, max…):GET /v2/meters/cpu.time/statisticsFilter:GET /v2/meters/cpu.time?q.field=project_id&q.value=foobar&q.op=eqGroup statistics by period:GET /v2/meters/cpu.time/statistics?period=3600New inGrizzly!
  15. 15. The big pictureOpenStackcomponentsCeilometer collectorCeilometer APICeilometer agentsNotification busDatabaseAPIExternal system AExternal system BPublishingpipelinePublishingpipelineOslo RPCWhatever!
  16. 16. RoadmapHavana I● Integrated Project ✓● Integration with Horizon● Publishing meters toother systems● Enhance SQL driver● Alarming● Integration with Heat● Deprecating APIv1● Completing APIv2● Move publishing part toOslo and other projects● Tighter integration withNova● Nova-schedulerintegration?● Incubated Project ✓● Integrationwith Horizon● Agents for othercomponents○ Swift ✓○ Ceph?○ Nicira?● SQLAlchemy storagedriver ✓● Multi-Publisher ✓● API v2 ✓○ User accessible API ✓○ More aggregation ✓○ Multi-dimension ✓Grizzly● ?
  17. 17. Use cases● Primary use-case is a rating/billing pipeline● Analytics○ capacity planning tools○ adaptive scheduling algorithms○ derive optimal pricing models○ resource rationing with fuzzy quotas● Realistic pre-prod simulations/loadtests● Visualization○ heat-maps/graphing to reveal usage patterns● Monitoring○ integration with diverse monitoring frameworks
  18. 18. Wednesday 16:30 Introduction to ceilometer architechture17:20 Feedback from Ceilometer usersThursday 09:00 Incremental improvements grab-bag09:50 Double entry auditing of collected metrics11:00 API improvements11:50 Alarm Threshold Evaluation13:30 Time series data manipulation in nosql stores14:20 Simple messaging action for Alerting15:20 Alarm state and history management16:10 Ceilometer support for advanced billing models17:00 Supporting rich data types and aggregationCeilometer @ design summitSee you in room B116
  19. 19. Questions?Nick Barcetnijaba@Freenode // @nijabanick@enovance.comhttp://launchpad.net/ceilometerhttp://docs.openstack.org/developer/ceilometerhttp://wiki.openstack.org/ceilometerFreenode: #openstack-meteringMailing List: openstack-dev [ceilometer]Julien Danjoujd__@Freenode // @juldanjoujulien@danjou.info

×