Dough:
OpenStack
Billing Project

Folsom Design Summit
2012-04-16


Luo, Zhongyue
lzyeval@gmail.com
TODO
 Intro
 Design concepts
 Architecture
 Example
 RPCs
 RDBMS model
 Billing logic
 Discussions
                   2
Intro
  What to expect from Dough
        Keep track of billing info to charge tenants
        Flexible customization of payment policies
              How much/often to charge for resource unit
              Handles prepaid or pay-as-you-go
  What Dough does not do
        Keep track of tenant’s resource usage
              Needs a monitoring/metering system
        Deduct money from account
              DIY using database as a queue
                                                           3
Design concepts
 Highly configurable
      Billing frequency, unit of measure
 Asynchronous
      All events are asynchronous
 Independent
      Pluggable clients to access other projects
 Scalability
      Handle billing of many users
 Nova conventions
      Message protocol                             4
RPC
Architecture                               Database
                                            Client

 Compute

 Network                RDBMS             Dashboard

 Storage




           Monitoring
                                Billing
           (Metering)




            NoSQL                                     5
RPC
Architecture                                         Database
                                                      Client

 Compute

  Network                   RDBMS                   Dashboard

  Storage




                Collector


 Monitoring
                                     Farmer         API daemon
 (Metering)

                                    Dispatch jobs   Subscribe or
                Collector                           unsubscribe
                                                     products /
                                                     Query info
               Check status /
              Retrieve usage /
              Create purchases                                     6
Example: creating an instance
  Compute

  Network                          RDBMS                 Dashboard

  Storage
                                                  2
                            Create subscription
                                  record


                            Collector
{'method': 'subscribe_item',
                                                               1
  'args': {
   Monitoring '62a8a396d9f44ef4a655ec1d7bd3801b',
    'user_id':                                  Farmer   API daemon
   (Metering) '864bbc5d23ea47799ae2a702927920e9',
    'tenant_id':
    'region': 'deafult',
    'item': 'instance',
    'item_type': 'm1.tiny', Collector
    'payment_type': 'hourly',
    'resource_uuid': 'uuidofinstance',
    'resource_name': 'nameofinstance',
}}
                                                                      7
Example: creating an instance
 Compute

  Network                   RDBMS                      Dashboard

  Storage

                                    Retrieve expired
                                     subscriptions
                             2
                Collector
                                         1
                    Request for
 Monitoring           charge           Farmer          API daemon
 (Metering)

                                     Dispatch jobs
                Collector


               Check status /
              Retrieve usage /
              Create purchases                                      8
Example: creating an instance
 Compute

  Network                             RDBMS                Dashboard
              Check instance
  Storage         status
                                  5
                     3          Create purchase record /
                                  Update subscription
                      Collector    status and expiring
                4                       datetime
              Retrieve instance
 Monitoring    usage quantity                  Farmer      API daemon
 (Metering)


                      Collector




                                                                        9
Example: creating an instance
 Compute

  Network                 RDBMS        2           Dashboard

  Storage
                                     RDBMS model


                                                            1
              Collector
                                                          RPCs

 Monitoring
                                  Farmer           API daemon
 (Metering)


              Collector

                              3
                                  Billing logic
                                                                10
RPCs
 subscribe_item
       Creates a subscription of a product
       Subscription status: ”creating”
 unsubscribe_item
       Updates the status of a subscription
       Subscription status: ”deleting”
 query_payment_types
 query_product_price
 query_usage_report
                                              11
RDBMS model
                                                 Created by...
 Regions          → region name
 Items            → resource name
 ItemTypes        → resource type name
                                                      Admin
 PaymentTypes → name, interval_unit,
                     interval_size, is_prepaid
 Products
     region, item, item_type, payment_type
     order_unit, order_size, price, currency
 Subscriptions → product, tenant, status              User
                     expires_at, resource_uuid
 Purchases        → subscription, quantity,           Collector
                     line_total                            12
Billing logic: Subscription states
                 Creation
                successful                            Unsubscribe
 creating                       verified

                         Item error


     Creation error
                                 error                     deleting
                                           Deletion
                                            error

                 Manually by admin


                                           Deletion successful
                             terminated
                                                                      13
Billing logic: Purchase creation
  For each non-deleted expired subscription:
      Status == ”creating”:
            Check if item finished creation(client)
            If product is prepaid:
                  Create purchase*
            Extend expires_at**
            Verify subscription
      Status == ”deleting”:
            Check if item finished deletion(client)
            If product is not prepaid:
                  Create purchase*
            Terminate subscription                    14
Billing logic: Purchase creation
  For each non-deleted expired subscription:
        Status == ”verified”:
                Check if item is valid(client)
                Create purchase*
                Extend expires_at**

  (*)Create purchase
        Insert purchase where
          line_total=quantity(from monitoring )*price/unit_size
  (**)Extend expires_at
        += dateutil.relativedelta(interval_unit=interval_size)
                                                                  15
Discussions
 Is this worth looking at?
 Any design problems?
 More requirements?




  https://github.com/lzyeval/talk-OSDS-2012
                                              16

Dough: OpenStack Billing Project

  • 1.
    Dough: OpenStack Billing Project Folsom DesignSummit 2012-04-16 Luo, Zhongyue lzyeval@gmail.com
  • 2.
    TODO Intro Designconcepts Architecture Example RPCs RDBMS model Billing logic Discussions 2
  • 3.
    Intro Whatto expect from Dough Keep track of billing info to charge tenants Flexible customization of payment policies How much/often to charge for resource unit Handles prepaid or pay-as-you-go What Dough does not do Keep track of tenant’s resource usage Needs a monitoring/metering system Deduct money from account DIY using database as a queue 3
  • 4.
    Design concepts Highlyconfigurable Billing frequency, unit of measure Asynchronous All events are asynchronous Independent Pluggable clients to access other projects Scalability Handle billing of many users Nova conventions Message protocol 4
  • 5.
    RPC Architecture Database Client Compute Network RDBMS Dashboard Storage Monitoring Billing (Metering) NoSQL 5
  • 6.
    RPC Architecture Database Client Compute Network RDBMS Dashboard Storage Collector Monitoring Farmer API daemon (Metering) Dispatch jobs Subscribe or Collector unsubscribe products / Query info Check status / Retrieve usage / Create purchases 6
  • 7.
    Example: creating aninstance Compute Network RDBMS Dashboard Storage 2 Create subscription record Collector {'method': 'subscribe_item', 1 'args': { Monitoring '62a8a396d9f44ef4a655ec1d7bd3801b', 'user_id': Farmer API daemon (Metering) '864bbc5d23ea47799ae2a702927920e9', 'tenant_id': 'region': 'deafult', 'item': 'instance', 'item_type': 'm1.tiny', Collector 'payment_type': 'hourly', 'resource_uuid': 'uuidofinstance', 'resource_name': 'nameofinstance', }} 7
  • 8.
    Example: creating aninstance Compute Network RDBMS Dashboard Storage Retrieve expired subscriptions 2 Collector 1 Request for Monitoring charge Farmer API daemon (Metering) Dispatch jobs Collector Check status / Retrieve usage / Create purchases 8
  • 9.
    Example: creating aninstance Compute Network RDBMS Dashboard Check instance Storage status 5 3 Create purchase record / Update subscription Collector status and expiring 4 datetime Retrieve instance Monitoring usage quantity Farmer API daemon (Metering) Collector 9
  • 10.
    Example: creating aninstance Compute Network RDBMS 2 Dashboard Storage RDBMS model 1 Collector RPCs Monitoring Farmer API daemon (Metering) Collector 3 Billing logic 10
  • 11.
    RPCs subscribe_item Creates a subscription of a product Subscription status: ”creating” unsubscribe_item Updates the status of a subscription Subscription status: ”deleting” query_payment_types query_product_price query_usage_report 11
  • 12.
    RDBMS model Created by... Regions → region name Items → resource name ItemTypes → resource type name Admin PaymentTypes → name, interval_unit, interval_size, is_prepaid Products region, item, item_type, payment_type order_unit, order_size, price, currency Subscriptions → product, tenant, status User expires_at, resource_uuid Purchases → subscription, quantity, Collector line_total 12
  • 13.
    Billing logic: Subscriptionstates Creation successful Unsubscribe creating verified Item error Creation error error deleting Deletion error Manually by admin Deletion successful terminated 13
  • 14.
    Billing logic: Purchasecreation For each non-deleted expired subscription: Status == ”creating”: Check if item finished creation(client) If product is prepaid: Create purchase* Extend expires_at** Verify subscription Status == ”deleting”: Check if item finished deletion(client) If product is not prepaid: Create purchase* Terminate subscription 14
  • 15.
    Billing logic: Purchasecreation For each non-deleted expired subscription: Status == ”verified”: Check if item is valid(client) Create purchase* Extend expires_at** (*)Create purchase Insert purchase where line_total=quantity(from monitoring )*price/unit_size (**)Extend expires_at += dateutil.relativedelta(interval_unit=interval_size) 15
  • 16.
    Discussions Is thisworth looking at? Any design problems? More requirements? https://github.com/lzyeval/talk-OSDS-2012 16