What About Billing?Beginning of 2012:● Billing has been left out of OpenStack core so far as itwas not the primary problem and is not a trivial one...● Yet almost every OpenStack deployment needs a wayto track usage information
Billing: 3 Step ProcessMetering Collect usage dataRating Transform usage data intobillable items and calculate costsBilling Create invoice, collect payment
v0.1 Objectives (Folsom)Focused only on metering for billingCeilometer aims to deliver a unique point of contact for billing systems toacquire all counters they need to establish customer billing, across all currentand future OpenStack components.The delivery of counters must be traceableand auditable, the counters must be easily extensible to support new projects,and agents doing data collections should be independent of the overall system.
Grizzly objectivesExtended objective: cover measurement ingeneralThe project aims to become the infrastructure to collect measurements withinOpenStack so that no two agents would need to be written to collect the samedata. Its primary targets are monitoring and metering, but the frameworkshould be easily expandable to collect for other needs. To that effect,Ceilometer should be able to share collected data with a variety of consumers.
Grizzly objectivesExtended objective: cover measurement ingeneralThe project aims to become the infrastructure to collect measurements withinOpenStack so that no two agents would need to be written to collect the samedata. Its primary targets are monitoring and metering, but the frameworkshould be easily expandable to collect for other needs. To that effect,Ceilometer should be able to share collected data with a variety of consumers.Havanas objectives should remain unchanged!
Workflow● Collect from OpenStack components● Transform meters into other meters ifnecessary● Publish meters to any destination (includingCeilometer itself)● Store received meters and read them via theCeilometer REST APICollect Transform Publish Store Read
CollectingNetworking Object StorageImageVolumeStorageNotification busCeilometercollector & agentAPIAPI API APIComputeAPIOslonotificationsPublishingpipelinePublishingPollingNew inGrizzly!
Ceilometercollector & agentReceiverReceiverPublisherPublisherPipelineTransformerMeterTransformerTransformerPublisher ReceiverPipeline: a set of transformers mutating meters intosomething that publishers know how to send toexternal systems.New inGrizzly!
TransformerMeterName=cpu.timeValue=5Timestamp=TTransformerMeterName=cpu.timeValue=20Timestamp=T+1MeterName=cpu.timeValue=80Timestamp=T+2MeterName=cpu.percentageValue=9Timestamp=T+2Transform meters into new meters!New inGrizzly!
PublisherMeterCeilometerRPCPublisherUDP PublisherCeilometer collectorExternal systemAMQPsigned messageBilling, monitoring,alarming, statistics,capacity planning…New inGrizzly!(not implemented yet)Yes, the frequency of the publication of each kind of meter is configurable for eachpublishing target.
Store: collectorCeilometer collectorSQLDatabaseMongoDBDatabaseMeterStorage abstraction layerMultiple backends choice,one at a timeHBaseDatabaseDefaultbackendNew inGrizzly!
The big pictureOpenStackcomponentsCeilometer collectorCeilometer APICeilometer agentsNotification busDatabaseAPIExternal system AExternal system BPublishingpipelinePublishingpipelineOslo RPCWhatever!
RoadmapHavana I● Integrated Project ✓● Integration with Horizon● Publishing meters toother systems● Enhance SQL driver● Alarming● Integration with Heat● Deprecating APIv1● Completing APIv2● Move publishing part toOslo and other projects● Tighter integration withNova● Nova-schedulerintegration?● Incubated Project ✓● Integrationwith Horizon● Agents for othercomponents○ Swift ✓○ Ceph?○ Nicira?● SQLAlchemy storagedriver ✓● Multi-Publisher ✓● API v2 ✓○ User accessible API ✓○ More aggregation ✓○ Multi-dimension ✓Grizzly● ?
Use cases● Primary use-case is a rating/billing pipeline● Analytics○ capacity planning tools○ adaptive scheduling algorithms○ derive optimal pricing models○ resource rationing with fuzzy quotas● Realistic pre-prod simulations/loadtests● Visualization○ heat-maps/graphing to reveal usage patterns● Monitoring○ integration with diverse monitoring frameworks
Wednesday 16:30 Introduction to ceilometer architechture17:20 Feedback from Ceilometer usersThursday 09:00 Incremental improvements grab-bag09:50 Double entry auditing of collected metrics11:00 API improvements11:50 Alarm Threshold Evaluation13:30 Time series data manipulation in nosql stores14:20 Simple messaging action for Alerting15:20 Alarm state and history management16:10 Ceilometer support for advanced billing models17:00 Supporting rich data types and aggregationCeilometer @ design summitSee you in room B116